Method and system for large model inference path optimization based on dynamic entropy perception

Abstract

The application provides a large model inference path optimization method and system based on dynamic entropy perception, and relates to the technical field of artificial intelligence inference, wherein the method comprises: obtaining intermediate layer features, attention distribution and magnetic field original data generated by large model inference; then calculating information entropy based on the intermediate layer features and constructing a state transition graph combined with the attention data, and preprocessing the magnetic field data to obtain an environmental disturbance signal, and fusing the state transition graph and the environmental disturbance signal to generate fusion state data; then inputting the fusion state data into a maximum entropy reinforcement learning model for decision-making, outputting preliminary path branch weight and computing resource scheduling parameters; then optimizing path features based on the preliminary weight and the state transition graph to generate final path control instructions; finally, dynamically scheduling and activating the internal computing unit of the large model according to the final instructions and the scheduling parameters to form an optimized inference path. The application improves the accuracy and efficiency of large model inference under complex tasks.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence reasoning technology, and in particular to a method and system for optimizing large model reasoning paths based on dynamic entropy perception. Background Technology

[0002] The large model inference path optimization method aims to improve the computational efficiency and decision-making accuracy of artificial intelligence systems when handling complex tasks, and has broad application prospects in fields such as intelligent analysis and automated decision-making.

[0003] One existing approach assesses information uncertainty by analyzing the statistical characteristics of activation values ​​within the model and constructs a graph representation of the model's inference process using an attention mechanism. This is then leveraged within a reinforcement learning framework to dynamically adjust the computational path. This method adapts to dynamic changes within the model to some extent.

[0004] However, such methods have limitations in handling environmental interference, as environmental fluctuations can affect the stable operation of physical computing equipment, leading to deviations in path decisions based on pure software analysis during actual deployment. Furthermore, when integrating multi-source heterogeneous state information for joint decision-making, the coordination and fine-tuning mechanisms between different information sources are insufficient, affecting the robustness of the overall optimization effect. Therefore, existing technologies suffer from the technical problem of unstable model inference path optimization performance under complex physical environments. Summary of the Invention

[0005] This application provides a method and system for optimizing large model inference paths based on dynamic entropy awareness, in order to solve the problems of low accuracy and poor efficiency of large model inference under complex tasks in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for optimizing large-scale model inference paths based on dynamic entropy awareness, comprising:

[0007] Acquire intermediate layer feature data, attention distribution data, and raw magnetic field data generated by the large model during multi-round interactive inference;

[0008] Information entropy data is calculated based on the intermediate layer feature data, and a state transition graph is constructed based on the information entropy data and the attention distribution data.

[0009] The original magnetic field data is preprocessed to generate an environmental disturbance signal, and fused state data is generated based on the state transition diagram and the environmental disturbance signal.

[0010] The fused state data is input into the maximum entropy reinforcement learning model, and joint optimization decision is made through the policy network of the maximum entropy reinforcement learning model, outputting preliminary path branch weights and computational resource scheduling parameters;

[0011] Based on the initial path branch weights and the state transition diagram, a graph neural network model is used to optimize path features in order to generate the final path control instructions.

[0012] Based on the final path control command and the computing resource scheduling parameters, the computing units within the large model are dynamically scheduled and activated to form an optimized inference path.

[0013] Optionally, the step of inputting the fused state data into a maximum entropy reinforcement learning model, performing joint optimization decisions through the policy network of the maximum entropy reinforcement learning model, and outputting preliminary path branch weights and computational resource scheduling parameters includes:

[0014] The fused state data is input into the policy network of the maximum entropy reinforcement learning model. The fused state data is weighted by importance through the attention mechanism in the policy network, and weighted features are extracted. The weighted features are used to characterize the correlation strength between environmental perturbation and inference state.

[0015] The weighted features are respectively input into the path planning branch and the resource scheduling branch of the policy network. The path planning branch includes a first policy head, and the resource scheduling branch includes a second policy head.

[0016] The first strategy header output contains a first probability distribution of multiple candidate path selection actions, and the second strategy header output contains a second probability distribution of multiple resource scheduling actions. The first probability distribution is used to describe the tendency to select different inference path branches, and the second probability distribution is used to describe the tendency to allocate different proportions of computing resources.

[0017] The first probability distribution and the second probability distribution are optimized respectively using the policy gradient algorithm in the policy network;

[0018] The optimized first probability distribution and the optimized second probability distribution are sampled through the sampling layer of the policy network to generate path selection code and resource scheduling code.

[0019] The path selection encoding is input into the path decoder of the policy network to obtain the initial path branch weights;

[0020] The resource scheduling code is input into the scheduling decoder of the policy network to obtain the calculated resource scheduling parameters.

[0021] Optionally, optimizing the first probability distribution and the second probability distribution using the policy gradient algorithm in the policy network includes:

[0022] Based on the fused state data, the expected return estimates of the first probability distribution and the second probability distribution are calculated respectively. The expected return estimates are used to characterize the prediction of the long-term cumulative utility that can be obtained by choosing different actions in the current reasoning state.

[0023] Based on the expected return estimate, the first policy gradient vector corresponding to the first probability distribution and the second policy gradient vector corresponding to the second probability distribution are calculated using the generalized advantage estimation algorithm. The first policy gradient vector and the second policy gradient vector are used to indicate the direction of adjusting the corresponding policy parameters in order to improve the inference path optimization effect.

[0024] The network parameters of the first policy head are adjusted in the opposite direction of the first policy gradient vector using the policy gradient algorithm to update the first probability distribution.

[0025] The network parameters of the second policy head are adjusted in the opposite direction of the gradient vector of the second policy to update the second probability distribution.

[0026] Optionally, the preprocessing of the raw magnetic field data to generate an environmental disturbance signal, and the generation of fused state data based on the state transition diagram and the environmental disturbance signal, includes:

[0027] The original magnetic field data is processed by a convolutional neural network to extract change features, thus obtaining magnetic field change data.

[0028] The magnetic field change feature vector is input into a long short-term memory network, and sequence prediction is performed through the long short-term memory network to obtain magnetic field prediction data;

[0029] The magnetic field prediction data is normalized to generate an environmental disturbance signal.

[0030] Obtain the current node corresponding to the current inference step from the state transition diagram, and the set of neighboring nodes of the current node;

[0031] Extract the information entropy value and connection relationship of each node in the current node and the set of neighboring nodes;

[0032] The information entropy value and connection relationship of the current node and the set of neighboring nodes are combined with the environmental disturbance signal to generate fused state data.

[0033] Optionally, the step of optimizing path features using a graph neural network model based on the initial path branch weights and the state transition graph to generate final path control instructions includes:

[0034] Based on the initial path branch weights and the information entropy values ​​of each node in the state transition graph, initial node features are generated for each node. These initial node features are used to represent the initial path selection tendency and information uncertainty of the corresponding reasoning step.

[0035] The graph structure of the state transition graph and the initial node features are input into the graph neural network model. Through the message passing unit of the graph neural network model, the features of each node are passed to the adjacent nodes along the directed connection edges between the nodes.

[0036] The node aggregation unit of the graph neural network model aggregates the transmitted features received by each node from all upstream nodes, and updates the aggregated features using a nonlinear activation function to obtain the updated features of the node. The updated features are used to represent the path selection tendency corrected after the graph structure information is transmitted.

[0037] Repeat the message passing and aggregation operations until the preset number of iterations is reached;

[0038] The updated features of each node obtained after reaching the specified number of iterations are input into the output unit of the graph neural network model, and the output unit generates the final path control command.

[0039] Optionally, the step of calculating information entropy data based on the intermediate layer feature data, and constructing a state transition graph based on the information entropy data and the attention distribution data, includes:

[0040] Extract the feature vector corresponding to each inference step from the intermediate layer feature data;

[0041] For each feature vector, a probability distribution estimate is performed, and the information entropy value of each inference step is calculated based on the estimation results to form information entropy data;

[0042] Based on the information entropy data, each reasoning step is set as a corresponding node;

[0043] The attention weights between every two inference steps are extracted from the attention distribution data, and directed connections are established between the nodes of the corresponding two inference steps based on the attention weights.

[0044] Generate a state transition graph based on all the nodes and all the directed edges.

[0045] Optionally, the step of dynamically scheduling and activating computing units within the large model based on the final path control instruction and the computing resource scheduling parameters to form an optimized inference path includes:

[0046] The final path control instruction is parsed to determine the target sub-module to be executed in the large model. The target sub-module is used to perform computational tasks for specific inference steps.

[0047] According to the computing resource scheduling parameters, corresponding computing resources are allocated to the target sub-module, and the computing resources are used to provide the processing units and memory space required for the execution of the target sub-module.

[0048] The target submodule is executed using the allocated computing resources to obtain the execution result, which includes intermediate features and attention data generated in the current inference step.

[0049] The execution result is used as the input for the next inference step of the large model, and the steps of determining, allocating, and executing are repeated to form an optimized inference path.

[0050] Secondly, this application provides a large-model inference path optimization system based on dynamic entropy awareness, including:

[0051] The acquisition module is used to acquire intermediate layer feature data, attention distribution data, and raw magnetic field data generated by the large model during multi-round interactive inference.

[0052] The calculation module is used to calculate information entropy data based on the intermediate layer feature data, and to construct a state transition graph based on the information entropy data and the attention distribution data.

[0053] The generation module is used to preprocess the original magnetic field data, generate environmental disturbance signals, and generate fused state data based on the state transition diagram and the environmental disturbance signals.

[0054] The input module is used to input the fused state data into the maximum entropy reinforcement learning model, perform joint optimization decision-making through the policy network of the maximum entropy reinforcement learning model, and output preliminary path branch weights and computational resource scheduling parameters.

[0055] An optimization module is used to optimize path features based on the initial path branch weights and the state transition diagram using a graph neural network model to generate final path control instructions.

[0056] The scheduling module is used to dynamically schedule and activate the computing units within the large model according to the final path control instruction and the computing resource scheduling parameters, so as to form an optimized inference path.

[0057] Thirdly, this application provides an electronic device, comprising:

[0058] Memory, used to store computer programs;

[0059] A processor, used to implement the steps of the dynamic entropy-aware large model inference path optimization method as described in the first aspect above when executing the computer program.

[0060] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the large model inference path optimization method based on dynamic entropy awareness as described in the first aspect above.

[0061] This application provides a method for optimizing the inference path of a large model based on dynamic entropy perception. The method includes: acquiring intermediate layer feature data, attention distribution data, and raw magnetic field data generated by the large model during multi-round interactive inference; calculating information entropy data based on the intermediate layer feature data, and constructing a state transition graph based on the information entropy data and the attention distribution data; preprocessing the raw magnetic field data to generate environmental disturbance signals, and generating fused state data based on the state transition graph and the environmental disturbance signals; inputting the fused state data into a maximum entropy reinforcement learning model, performing joint optimization decisions through the policy network of the maximum entropy reinforcement learning model, and outputting preliminary path branch weights and computational resource scheduling parameters; optimizing path features using a graph neural network model based on the preliminary path branch weights and the state transition graph to generate a final path control command; and dynamically scheduling and activating the computational units within the large model according to the final path control command and the computational resource scheduling parameters to form an optimized inference path.

[0062] The technical solution provided in this application has the following beneficial effects:

[0063] This application constructs a state transition diagram to characterize the dynamic information changes and inherent dependencies of the reasoning process. Secondly, by introducing environmental perturbation signals, it enhances the method's adaptability to physical environment fluctuations, making the reasoning process more stable. Then, it uses a maximum entropy reinforcement learning model for joint decision-making, achieving a balance between exploration and utilization, and generating a preliminary scheduling scheme that balances efficiency and diversity. Subsequently, it optimizes path features based on graph neural networks, further refining path selection and improving decision accuracy. Finally, it uses a dynamic scheduling computing unit to directly translate the optimized decision into execution actions, thereby ensuring the formation of an efficient and robust reasoning path.

[0064] Furthermore, this application also strengthens the key information of the fused state data through an attention mechanism to form weighted features; then the weighted features are input into the path planning branch and the resource scheduling branch respectively, and the respective policy heads output probability distributions describing the path selection and resource allocation tendencies; then the two probability distributions are optimized using the policy gradient algorithm, and are transformed into specific preliminary path branch weights and calculated resource scheduling parameters through sampling and decoding processes.

[0065] Therefore, this application focuses on key states through an attention mechanism, which improves the effectiveness of feature representation; and achieves joint and adaptive optimization of path selection and resource scheduling through the synergy of dual-branch policy heads and policy gradient optimization, so that the preliminary decision scheme takes into account both the foresight of the reasoning direction and the rationality of computational resource scheduling.

[0066] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating a method for optimizing large model inference paths based on dynamic entropy awareness, provided in an embodiment of this application;

[0069] Figure 2 This application provides a schematic diagram illustrating a specific implementation of a large model inference path optimization method based on dynamic entropy awareness.

[0070] Figure 3 This is a schematic diagram of the structure of a large model inference path optimization system based on dynamic entropy perception, provided in an embodiment of this application. Detailed Implementation

[0071] Existing methods for optimizing inference paths in large models primarily focus on analyzing activation information and attention patterns within the model and constructing a decision framework based on these. However, such approaches face a key challenge in practical deployment: their decision-making process often relies on the software-level features of the model output, neglecting the dynamic changes in the physical environment of the computing device, such as electromagnetic fluctuations around the device. Furthermore, existing methods mostly employ phased or loosely coupled processing when handling multi-source state information. This results in the failure to fully model and utilize the synergistic relationship between internal inference states and external environmental disturbances. Consequently, in complex physical scenarios, mismatches or lags in path decision-making and resource scheduling can easily occur, affecting the overall stability and adaptability of the optimization process.

[0072] To address the aforementioned shortcomings, this application proposes a large-scale model inference path optimization method based on dynamic entropy perception. The core idea of ​​this method is as follows: First, by constructing a state transition graph that integrates internal information entropy and attention dependence, and quantifying the magnetic field disturbances of the physical environment, the internal inference state and external environmental disturbances are simultaneously collected and represented. Then, a collaborative decision-making mechanism is designed, using a maximum entropy reinforcement learning model to jointly optimize the fused state, simultaneously generating preliminary path preferences and resource scheduling schemes. Finally, a graph neural network model is used to refine the preliminary schemes, generating the final path control instructions and resource scheduling parameters, thereby dynamically guiding the inference execution process within the large model.

[0073] The above method incorporates internal state changes and external physical disturbances into a unified decision-making framework. Through multi-stage collaborative optimization, it achieves adaptive matching of inference paths and computing resources in complex dynamic environments, effectively improving the robustness and execution efficiency of the inference process. This solves the problem of unstable optimization results caused by environmental interference and insufficient multi-source information collaboration in existing technologies.

[0074] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] The core of this application is to provide a method for optimizing large-scale model inference paths based on dynamic entropy awareness. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0076] Step 101: Obtain the intermediate layer feature data, attention distribution data, and original magnetic field data generated by the large model during the multi-round interactive inference process.

[0077] In step 101, intermediate layer feature data refers to the computational results generated by the layers other than the input and output layers within the network structure of the large model during the process of processing input information. These data reflect the model's gradual understanding and abstraction of the input information.

[0078] Attention distribution data refers to the weight information generated by the attention mechanism within a large model. This information is used to measure the importance of different parts of the input information in the current computation step.

[0079] Raw magnetic field data refers to the unprocessed, raw magnetic field strength readings directly collected by an array of magnetic field sensors deployed around computing devices. These readings reflect the electromagnetic fluctuations of the physical environment in which the device is located.

[0080] In this embodiment, the large model in operation directly outputs the intermediate layer feature data and attention distribution data generated during the multi-round inference process, and simultaneously calls the pre-deployed magnetic field sensor array to synchronously collect the original magnetic field data of the physical space where the device is located, thereby providing basic information reflecting the internal computing state and the external physical environment for the subsequent path optimization process.

[0081] Step 102: Calculate information entropy data based on the intermediate layer feature data, and construct a state transition diagram based on the information entropy data and the attention distribution data.

[0082] Among them, information entropy data is a quantitative indicator for measuring information uncertainty. The larger the information entropy value, the higher the uncertainty of the calculation state of the corresponding reasoning step; the smaller the information entropy value, the lower the uncertainty.

[0083] A state transition graph is a graph data structure in which each node corresponds to a reasoning step. The information stored in the node is the information entropy value of the corresponding reasoning step. The directed edges between nodes are determined by the attention distribution data. The direction of the directed edges indicates the order of the reasoning process, and the weight value of the edges reflects the semantic dependency strength between two reasoning steps.

[0084] In this embodiment, step 102 includes the following process:

[0085] Step 1021: Extract the feature vector corresponding to each inference step from the intermediate layer feature data.

[0086] In step 1021, the feature vector is a numerical representation of the intermediate layer feature data corresponding to an inference step after dimensionality reduction and standardization.

[0087] In this embodiment, principal component analysis is applied to the intermediate layer feature data of each inference step to reduce dimensionality, and the dimensionality-reduced data is normalized to a preset numerical range, thereby obtaining the feature vector of the corresponding inference step.

[0088] Step 1022: Estimate the probability distribution of each feature vector, and calculate the information entropy value of each inference step based on the estimation results to form information entropy data.

[0089] In step 1022, probability distribution estimation is the process of estimating the parameters of a certain probability model by performing statistical analysis on the numerical distribution of the feature vector.

[0090] In this embodiment, firstly, a kernel density estimation algorithm is applied to each feature vector to estimate its multidimensional probability density distribution; secondly, based on the estimated probability density distribution, the information entropy calculation formula is substituted to calculate the information entropy value corresponding to each inference step; finally, all the calculated information entropy values ​​are arranged in the order of the inference steps to form information entropy data.

[0091] It should be noted that the specific form of the information entropy calculation formula is not limited in the embodiments of this application, and can be set accordingly according to the actual situation.

[0092] Step 1023: Based on the information entropy data, set each inference step as a corresponding node.

[0093] In this embodiment, a unique identifier is created for each reasoning step based on the order of reasoning steps recorded in the information entropy data, and the identifier is encapsulated together with the information entropy value of the step to construct the node in the state transition diagram.

[0094] Step 1024: Extract the attention weight between every two inference steps from the attention distribution data, and establish directed connection edges between the nodes of the corresponding two inference steps according to the attention weight.

[0095] In step 1024, attention weight refers to the correlation score between two inference steps read from attention distribution data, which is generated by the attention mechanism of the large model during inference.

[0096] In this embodiment, the attention distribution data is first traversed to find each pair of related inference steps and their corresponding attention weights; second, in the state transition graph, the corresponding nodes that have been created are found based on these two inference steps; finally, a directed connection edge is added between the two nodes, and the attention weight is set as the initial weight value of the directed connection edge.

[0097] Step 1025: Generate a state transition graph based on all the nodes and all the directed edges.

[0098] In this embodiment of the application, all nodes created in step 1023 are combined with all directed connecting edges created in step 1024 to form a complete graph structure containing node attribute and edge attribute data, which is the state transition graph.

[0099] Through the steps described above, this application transforms the computational uncertainties and structural dependencies implicit in the model into an intuitive graphical representation, providing a structured foundation for subsequent fusion of environmental information and collaborative optimization decision-making.

[0100] Step 103: Preprocess the original magnetic field data to generate an environmental disturbance signal, and generate fused state data based on the state transition diagram and the environmental disturbance signal.

[0101] Among them, the environmental disturbance signal is a normalized value used to quantify the intensity of electromagnetic interference in the physical environment where the equipment is located; the fused state data is a comprehensive data representation after associating and integrating the internal inference state of the model with the external environmental disturbance.

[0102] In this embodiment, step 103 includes the following process:

[0103] Step 1031: Extract change features from the original magnetic field data using a convolutional neural network to obtain magnetic field change data.

[0104] In step 1031, the magnetic field change data is a feature vector obtained by extracting local features from the original magnetic field data, which is used to characterize the short-term fluctuations in magnetic field strength.

[0105] In this embodiment, the collected raw magnetic field data is first organized into a time series; then it is input into a pre-trained two-dimensional convolutional neural network, which captures the fluctuation patterns and abrupt changes of the magnetic field signal within a short time window by performing one-dimensional convolution operations in the time dimension; finally, the network outputs the corresponding magnetic field change data.

[0106] In practical applications, for the server rooms of intelligent investment research departments of financial institutions, magnetic field sensors collect raw magnetic field strength data along the X, Y, and Z axes at a sampling frequency of 100 Hz. 100 sampling points for 1 second are used as a time series input. The convolutional neural network uses a 3×3 convolutional kernel and outputs a magnetic field change data vector containing 8 feature dimensions through layer-by-layer convolution and pooling, such as [0.56, -0.21, 0.78, 0.12, -0.45, 0.33, 0.67, -0.09].

[0107] Step 1032: Input the magnetic field change feature vector into the Long Short-Term Memory (LSTM) network, and perform sequence prediction through the LSM network to obtain magnetic field prediction data.

[0108] In step 1032, the magnetic field prediction data is a numerical sequence obtained by analyzing historical magnetic field change trends and predicting the magnetic field state in the near future.

[0109] This application does not impose specific limitations on the structural design and parameter design of convolutional neural networks and long short-term memory networks, and corresponding settings can be made according to the actual situation.

[0110] In this embodiment of the application, multiple sets of continuous magnetic field change data output in step 1031 are input into a pre-trained long short-term memory network in chronological order; the network predicts the magnetic field change characteristics of the next time step by learning the long-term dependencies in the historical sequence, and uses the prediction results as magnetic field prediction data.

[0111] In practical applications, for server rooms of intelligent investment research departments of financial institutions, when a large model is analyzing a section of a listed company's annual report, the magnetic field environment fluctuates regularly due to the periodic start-up and shutdown of surrounding air conditioning units. This application uses the magnetic field change data of the most recent five consecutive sampling times as the input sequence. Each set of data is an 8-dimensional vector representing the magnetic field fluctuations along different axes. The Long Short-Term Memory Network uses its 32 memory units to learn the short-term change patterns of the electromagnetic environment of the server room and predicts the magnetic field state that may occur at the next moment, outputting an 8-dimensional prediction vector. This prediction vector is used to represent the upcoming environmental disturbance trend and serves as the basis for subsequently adjusting the model's focus on different sections of the annual report.

[0112] Step 1033: Perform amplitude normalization processing on the magnetic field prediction data to generate an environmental disturbance signal.

[0113] In this embodiment, amplitude normalization is to map each numerical component in the magnetic field prediction data vector to a standard range between 0 and 1. This step calculates the maximum absolute value of all dimensions of the magnetic field prediction data vector, and then divides each value in the vector by the maximum value, thereby scaling the data to a preset amplitude range. The resulting scaled vector is the environmental disturbance signal.

[0114] In practical applications, when the predicted magnetic field data vector is [0.48, -0.18, 0.71, 0.10, -0.40, 0.28, 0.60, -0.05], this vector reflects the amplitude of electromagnetic disturbances in each axis that may be caused by a sudden surge in the load of surrounding network equipment in the server room. The maximum absolute value is 0.71. By dividing each value of this vector by 0.71 to normalize the amplitude, the environmental disturbance signal [0.676, -0.254, 1.000, 0.141, -0.563, 0.394, 0.845, -0.070] is obtained. This signal quantifies the relative interference intensity in different directions and is used in subsequent decision-making to assess the potential impact of the current physical environment on the complex chapter of "Discussion and Analysis by Company Management" in the model analysis.

[0115] Step 1034: Obtain the current node corresponding to the current inference step and the set of neighboring nodes of the current node from the state transition diagram.

[0116] In step 1034, the current node refers to the node in the state transition graph whose identifier is exactly the same as the inference step number that the large model is executing or is about to execute; the neighborhood node set refers to the set of all upstream and downstream nodes that are directly connected to the current node through directed edges.

[0117] In this embodiment, the current inference step number is first determined based on the execution state of the large model; then, the node whose identifier matches the number is searched in the state transition graph and identified as the current node; finally, the directed connection edges of the state transition graph are traversed, and all nodes at the other end with the current node as the starting point or ending point are collected to form a neighborhood node set.

[0118] In practical applications, when the large model's inference path reaches a paragraph with complex logical dependencies in a long document, the corresponding current node is located in the state transition graph. By querying the graph's edge structure, it is found that there are edges pointing from upstream nodes to the current node. These edges represent that the model's analysis of the preceding paragraphs provides semantic dependencies for the current paragraph's processing. At the same time, there are edges pointing from the current node to downstream nodes, indicating that the processing result of the current paragraph will directly affect the inference of subsequent paragraphs. Therefore, the relevant nodes are added to the neighborhood node set, which together constitute the local graph context of the current inference step.

[0119] Step 1035: Extract the information entropy value and connection relationship of each node in the current node and the set of neighboring nodes.

[0120] In step 1035, the connection relationship includes whether there is a directed connection edge between the two nodes, and the weight value of the directed connection edge.

[0121] In this embodiment of the application, for each node in the current node and the set of neighboring nodes, the information entropy value stored in the node attributes of the state transition graph is read; at the same time, for a node pair formed by the current node and any node in the set of neighboring nodes, the existence of a connecting edge is queried in the edge list of the state transition graph. If it exists, the weight value of the edge is recorded; if it does not exist, the connection relationship is recorded as no connection.

[0122] In practical applications, the information entropy value of the current node corresponding to the "Company Business Overview" chapter is 0.85, reflecting the degree of uncertainty of the text information in this part; the information entropy value of the upstream node corresponding to the "Industry Background" chapter, which serves as its semantic premise, is 0.72, and the edge weight of the dependency relationship between the two is 0.6; while the information entropy value of the downstream node corresponding to the "Competitive Advantage Analysis" chapter, which depends on the conclusion of the current chapter, is 0.91, and its edge weight is 0.8. These information entropy values ​​and weight data are extracted in a structured manner to quantify the strength of logical dependencies and the uncertainty of the reasoning state between different report chapters.

[0123] Step 1036: Combine the information entropy value and connection relationship of the current node and the set of neighboring nodes with the environmental disturbance signal to generate fused state data.

[0124] In this embodiment of the application, the combination operation involves arranging the information entropy values ​​of all nodes extracted in step 1035 into a sequence, and arranging the weight values ​​of all connections into another sequence; then, these two sequences are concatenated with the environmental disturbance signal vector generated in step 1033 to form a longer-dimensional comprehensive vector, which is the fused state data.

[0125] In practical applications, the information entropy value of the current node representing the uncertainty of the "Company Business Overview" chapter (0.85), the information entropy value of its premise "Industry Background" chapter (0.72), and the information entropy value of its subsequent "Competitive Advantage Analysis" chapter (0.91) are arranged into a sequence [0.85, 0.72, 0.91]. The weights of the logical dependencies between the corresponding chapters (0.6 and 0.8) are arranged into a sequence [0.6, 0.8]. Then, the environmental disturbance signal vector [0.676, -0.254, 1.000, 0.141, -0.563, 0.394, 0.845, -0.070] that quantifies the intensity of electromagnetic disturbance in the computer room is concatenated with the aforementioned two sequences to finally generate a 13-dimensional fused state data vector. This vector comprehensively represents the collaborative state of internal inference uncertainty and external physical disturbance faced by the model when processing relevant report chapters under a specific electromagnetic interference environment.

[0126] This application, through the above steps, can transform the magnetic field signal reflecting the dynamics of the physical environment into a standardized disturbance quantification index, and deeply integrate it with the graph structure information representing the uncertainty and dependency of the internal reasoning of the model, thus constructing a unified data representation that can comprehensively reflect the coordinated changes of internal and external states, laying the foundation for subsequent accurate joint decision-making.

[0127] Step 104: Input the fused state data into the maximum entropy reinforcement learning model, perform joint optimization decision-making through the policy network of the maximum entropy reinforcement learning model, and output the preliminary path branch weights and computational resource scheduling parameters.

[0128] The core structure of the maximum entropy reinforcement learning model is a policy network that takes fused state data as input. This policy network contains a shared feature extraction backbone, which is then divided into two parallel decision branches: a path planning branch and a resource scheduling branch. The feature extraction backbone typically consists of multiple fully connected layers, such as three fully connected layers, with 256, 128, and 64 neurons in each layer, respectively, and uses the ReLU activation function. The path planning branch is connected to the backbone and consists of two fully connected layers with 32 and 16 neurons, respectively. Finally, it is connected to a first policy head whose output dimension is equal to the number of candidate path actions. This policy head uses the Softmax activation function to output a first probability distribution.

[0129] The resource scheduling branch has a similar structure to the path planning branch, consisting of two fully connected layers with 32 and 16 neurons respectively. It is ultimately connected to a second policy head whose output dimension is equal to the number of candidate resource scheduling actions. This policy head also uses the Softmax activation function to output the second probability distribution. In addition, the model also includes an independent value network for estimating state values. Its structure consists of three fully connected layers with 128, 64 and 1 neurons respectively.

[0130] The training process of this model adopts a combination of offline and online methods. First, it is pre-trained on an offline dataset consisting of historical inference tasks and environmental data. The network parameters are initialized by the maximum entropy objective function, which maximizes the weighted sum of expected reward and policy entropy. Then, it is fine-tuned online in the actual deployment environment. The parameters of the policy network and value network are continuously updated by using the policy gradient algorithm, which utilizes the fused state data collected from real-time interactions, the actions performed, and the reward signals obtained. The reward signal is set comprehensively based on the speed and accuracy of the inference task.

[0131] It should be noted that the above structure is exemplary. This application does not impose specific limitations on the layer and other structural designs used in the internal structure of the maximum entropy reinforcement learning model, and can make corresponding settings according to the actual situation.

[0132] The initial path branch weight is a numerical vector, where each component represents the relative importance of the corresponding inference path branch being activated in the current decision; the computational resource scheduling parameter is a numerical vector or structure used to specify the number of computational cores and the data batch size allocated for executing the current inference step.

[0133] In this embodiment, step 104 includes the following process, such as... Figure 2 As shown:

[0134] Step 1041: Input the fused state data into the policy network of the maximum entropy reinforcement learning model, and use the attention mechanism in the policy network to weight the fused state data by importance, and extract weighted features. The weighted features are used to characterize the correlation strength between environmental perturbation and inference state.

[0135] In step 1041, the weighted feature refers to the fused state data after being processed by the attention mechanism. The importance of different dimensions in this feature vector is redistributed, so that the information dimensions that are highly related to environmental disturbances or key reasoning states receive higher weights.

[0136] In this embodiment, the fused state data is first input into the attention mechanism module in the policy network. The module generates an attention weight distribution by calculating the correlation between features of different dimensions within the fused state data. Then, the weight distribution is multiplied element-wise with the original fused state data to highlight important features and suppress minor features, thereby extracting weighted features.

[0137] In practical applications, the attention mechanism module includes a fully connected layer that maps the 13-dimensional fusion state data into a 13-dimensional attention score vector. This score vector is then normalized to a weight vector using the Softmax function. The weight vector is then multiplied element-wise with the original fusion state data to obtain a weighted feature vector that is also 13-dimensional but whose feature importance has been redistributed.

[0138] Step 1042: Input the weighted features into the path planning branch and the resource scheduling branch of the policy network respectively. The path planning branch includes a first policy head, and the resource scheduling branch includes a second policy head.

[0139] In step 1042, the path planning branch is a specific part of the policy network responsible for generating inference path selection decisions, and its end is connected to the first policy head; the resource scheduling branch is a specific part of the policy network responsible for generating computational resource allocation decisions, and its end is connected to the second policy head.

[0140] In this embodiment, the policy network has two independent branch structures. The initial part of each branch shares the same weighted features as input. The weighted features are simultaneously copied and input into the initial layers of the path planning branch and the resource scheduling branch, respectively. The subsequent neural network layers of these two branches are independent of each other and are used to process decision tasks related to path planning and resource scheduling, respectively.

[0141] Step 1043: The first strategy head output contains a first probability distribution of multiple candidate path selection actions, and the second strategy head output contains a second probability distribution of multiple resource scheduling actions. The first probability distribution is used to describe the tendency to select different inference path branches, and the second probability distribution is used to describe the tendency to allocate different proportions of computing resources.

[0142] In step 1043, the first probability distribution is a vector whose length is equal to the number of candidate path selection actions, and the value of each position in the vector represents the probability of selecting the corresponding path; the second probability distribution is also a vector whose length is equal to the number of resource scheduling actions, and each value represents the probability of selecting the corresponding resource allocation scheme.

[0143] In this embodiment, the end of the path planning branch is a first strategy head, which is a fully connected layer. This fully connected layer receives the features processed by the branch and outputs a probability vector as a first probability distribution. The end of the resource scheduling branch is a second strategy head, which has a similar structure and outputs a second probability distribution. Both probability distributions are processed by the Softmax function to ensure that the sum of all probability values ​​is 1.

[0144] In practical applications, assuming there are 3 candidate inference paths and 4 resource scheduling schemes, the first strategy head outputs a 3-dimensional probability vector, such as [0.6, 0.3, 0.1], indicating that the probability of selecting the first path is 0.6; the second strategy head outputs a 4-dimensional probability vector, such as [0.2, 0.5, 0.2, 0.1], indicating that the probability of selecting the second resource allocation scheme is 0.5.

[0145] Step 1044: Optimize the first probability distribution and the second probability distribution respectively using the policy gradient algorithm in the policy network.

[0146] Step 1044 may specifically include the following steps:

[0147] A1: Based on the fused state data, calculate the expected return estimates for the first probability distribution and the second probability distribution, respectively. The expected return estimates are used to characterize the prediction of the long-term cumulative utility that can be obtained by choosing different actions in the current inference state.

[0148] In step A1, the expected return estimate is a scalar value used to estimate the total cumulative reward that can be obtained in the future after performing an action in the current state.

[0149] In this embodiment of the application, the expected return estimate is calculated by an independently trained value network that takes the same fused state data as input and outputs an estimate of the value of the current state. For each candidate action, the expected return is calculated by combining the state value estimate and a preset reward model.

[0150] In practical applications, the value estimate of the current fusion state data output by the value network is V equal to 0.85. For the three path selection actions corresponding to the first probability distribution, according to the preset reward rules, such as the shorter the selected path and the lower the uncertainty, the higher the reward, their expected return estimates are calculated to be 0.9, 0.7, and 0.5, respectively.

[0151] A2: Based on the expected return estimate, the first policy gradient vector corresponding to the first probability distribution and the second policy gradient vector corresponding to the second probability distribution are calculated using the generalized advantage estimation algorithm. The first policy gradient vector and the second policy gradient vector are used to indicate the direction of adjustment of the corresponding policy parameters in order to improve the inference path optimization effect.

[0152] In step A2, the policy gradient vector is a direction vector calculated from the advantage estimate and used to update the policy network parameters.

[0153] In this embodiment, the expected return estimate obtained in step A1 is first used, combined with the state value V output by the value network, to calculate the advantage value A for each action using the generalized advantage estimation formula; wherein the calculation of the advantage value A can be expressed as follows: ,in Indicates timing difference error. ,in This represents the value of the current state. Represents the state value of the next state. This represents the advantage value of the action at the current moment. Indicates an immediate reward. The discount factor is 0.99. The attenuation factor is 0.95. The next moment is the advantage estimate; then, the advantage value A of each action is multiplied by the logarithmic gradient of its corresponding action probability to obtain the policy gradient component for that action. The components of all actions are summed to obtain the first policy gradient vector, and the second policy gradient vector is obtained similarly.

[0154] In practical applications, for the three actions of the first probability distribution, the calculated advantage values ​​are 0.05, -0.15, and -0.25, respectively; the gradient vector of the first policy... Through formula The calculation yielded, where It is an action The advantage value, It is an action The probability, It is an action The gradient of the probability logarithm; for example, the gradient vector calculated based on the current network parameters. [0.03, -0.05, -0.02].

[0155] A3: The policy gradient algorithm is used to adjust the network parameters of the first policy head in the opposite direction of the first policy gradient vector in order to update the first probability distribution.

[0156] A4: Adjust the network parameters of the second policy head in the opposite direction of the second policy gradient vector to update the second probability distribution.

[0157] In this embodiment, steps A3 and A4 are executed in the same way, both using the policy gradient ascent method to update parameters; specifically, the first policy gradient vector calculated in step A2 is... Multiply by a preset learning rate ,like The product result is set to 0.001, and then the network parameters of the first policy head are updated using this product result. The updated formula is as follows ,in This represents the network parameters before the first policy header was updated. This represents the network parameters updated after the first policy head; similarly, the gradient vector of the second policy is used. and learning rate Update the network parameters of the second policy head After the parameters are updated, the first probability distribution output by the first strategy head and the second probability distribution output by the second strategy head are optimized accordingly.

[0158] Step 1045: Through the sampling layer of the policy network, sample the optimized first probability distribution and the optimized second probability distribution respectively to generate path selection code and resource scheduling code.

[0159] In step 1045, the sampling layer selects a specific action by random sampling based on the optimized probability distribution; the path selection code is an integer index of the sampled path selection action, and the resource scheduling code is an integer index of the sampled resource scheduling action.

[0160] In this embodiment of the application, for example, the optimized first probability distribution is [0.65, 0.28, 0.07]. The sampling layer performs random sampling according to this distribution. Assuming the first action is selected, the generated path selection code is an integer 1. The optimized second probability distribution is [0.15, 0.55, 0.25, 0.05]. Assuming the second action is selected, the generated resource scheduling code is an integer 2.

[0161] Step 1046: Input the path selection encoding into the path decoder of the policy network to obtain the preliminary path branch weights.

[0162] Step 1047: Input the resource scheduling code into the scheduling decoder of the policy network to obtain the calculated resource scheduling parameters.

[0163] In this embodiment, both the path decoder and the scheduling decoder are fully connected neural network layers. The path decoder converts the path selection encoding into a dense vector through an embedding layer, and then maps it through a fully connected layer into a set of weight values ​​equal to the number of candidate paths. This set of weights is the initial path branch weight, for example, encoding 1 corresponds to the weight vector [0.8, 0.15, 0.05]. The scheduling decoder maps the resource scheduling encoding to specific computing resource scheduling parameters in the same way. For example, encoding 2 corresponds to the parameters of allocating 4 computing cores and a batch size of 32.

[0164] In the above process, this application utilizes the maximum entropy reinforcement learning framework to jointly evaluate and make decisions on complex situations after the fusion of environmental perturbations and inference states. By combining policy gradient optimization and random sampling, it generates preliminary path and resource scheduling schemes that consider both long-term benefits and necessary exploratory nature, providing a decision-making basis that balances efficiency and robustness for final accurate path control.

[0165] Step 105: Based on the preliminary path branch weights and the state transition diagram, use a graph neural network model to optimize path features in order to generate the final path control instructions.

[0166] The final path control instruction is a precise control signal used to instruct the large model which internal submodule or attention head should be activated in the next inference step.

[0167] In this embodiment, step 105 includes the following process:

[0168] Step 1051: Based on the preliminary path branch weights and the information entropy values ​​of each node in the state transition graph, generate initial node features for each node. The initial node features are used to represent the initial path selection tendency and information uncertainty of the corresponding reasoning step.

[0169] In step 1051, the initial node feature is a vector that integrates the path selection tendency obtained from upstream decisions with the node's inherent information uncertainty measure.

[0170] In this embodiment, the information entropy value of each node in the state transition graph is first obtained. At the same time, the path weight value of the corresponding reasoning step is found from the preliminary path branch weights according to the node's identifier. Then, the information entropy value of each node is concatenated with the path weight value corresponding to that node to form a two-dimensional feature vector, which is the initial node feature of that node.

[0171] In practical applications, taking the analysis of the node in the "Company Business Overview" chapter as an example, its information entropy value is 0.85. The path weight corresponding to this chapter is found to be 0.8 from the preliminary path branch weights. Then, these two values ​​are concatenated into a two-dimensional vector [0.85, 0.8], which serves as the initial node feature of this node.

[0172] Step 1052: Input the graph structure of the state transition graph and the initial node features into the graph neural network model, and pass the features of each node to the adjacent nodes along the directed connection edges between the nodes through the message passing unit of the graph neural network model.

[0173] In step 1052, the message passing unit is the core component of the graph neural network. This message passing unit is responsible for sending the feature information of a node to other nodes directly connected to it according to the connection relationship of the graph.

[0174] In this embodiment of the application, the graph neural network model receives a feature matrix consisting of all nodes and their initial node features, as well as an adjacency matrix describing the connection relationships between nodes; the message passing unit, according to the connection direction indicated by the adjacency matrix, for each directed connection edge in the graph, passes the initial node feature vector of the source node through a learnable linear transformation as a message to the target node.

[0175] In practical applications, for directed edges from the "Industry Background" chapter node to the "Company Business Overview" chapter node, the message passing unit will transform the initial feature vector of the "Industry Background" node and then pass it to the "Company Business Overview" node.

[0176] Step 1053: Through the node aggregation unit of the graph neural network model, the transmitted features received by each node from all upstream nodes are aggregated, and the aggregated features are updated using a nonlinear activation function to obtain the updated features of the node. The updated features are used to represent the path selection tendency corrected after the graph structure information is transmitted.

[0177] In step 1053, the node aggregation unit is responsible for merging multiple messages received by a node from different upstream nodes into a single comprehensive message, and generating a new feature representation by combining the node's own characteristics.

[0178] In this embodiment of the application, for each node in the state transition graph, the node aggregation unit collects all messages passed to that node; then, it aggregates these messages into a single aggregated feature vector by summing or averaging; next, it concatenates or adds the original initial node features of that node with the aggregated feature vector; finally, it inputs the combined features into a nonlinear activation function, such as the ReLU function, to perform a nonlinear transformation and output the updated features of that node.

[0179] In practical applications, the "Company Business Overview" node receives the transmitted features from its upstream "Industry Background" node. The aggregation unit merges these features with the initial features of the "Company Business Overview" node itself [0.85, 0.8], and calculates a new feature vector as the updated feature of the node after activation function calculation, for example [0.92, 0.75].

[0180] Step 1054: Repeat the message passing and aggregation operations until the preset number of iterations is reached.

[0181] In this embodiment of the application, steps 1052 and 1053 constitute a complete graph neural network information propagation and update process. This process is repeated multiple times. Each time it is executed, the "self-feature" used by each node in step 1053 for updating is the updated feature obtained in the previous iteration. Through multiple iterations, information can be gradually propagated from any node in the graph to other nodes, so that the final feature of each node contains global information from its multi-hop neighbors.

[0182] In practical applications, the preset number of iterations is set to 2. After the first iteration, the features of each node contain information about its first-order neighbors. After the second iteration, the features of each node further contain information about its second-order neighbors.

[0183] Step 1055: Input the updated features of each node obtained after reaching the number of iterations into the output unit of the graph neural network model, and the output unit generates the final path control command.

[0184] In this embodiment, the output unit is a fully connected neural network layer at the end of the graph neural network; the updated features of all nodes obtained after a preset number of iterations are arranged in node order and input to the output unit; the output unit calculates and outputs a selection probability score for each node, which represents the probability of activating the computing unit corresponding to the node in the next inference step; finally, the identifier of the computing unit corresponding to the node with the highest selection probability score constitutes the final path control instruction.

[0185] In practical applications, the update features of all chapter-related nodes are processed by the output unit, which outputs their respective scores. The "Company Business Overview" chapter node has the highest score, so the final path control instruction generated is "Activate the dedicated calculation submodule A for processing the Company Business Overview chapter".

[0186] In the above process, this application utilizes graph neural networks to perform deep information mining on the state transition graph that reflects the logical dependencies between chapters, refining the initial global path decision into precise node-level control instructions that consider the close semantic relationships between local parts, thereby achieving seamless connection and refined guidance from macro strategy to micro execution in complex document analysis scenarios.

[0187] Step 106: Based on the final path control instruction and the computing resource scheduling parameters, dynamically schedule and activate the computing units within the large model to form an optimized inference path.

[0188] The optimized inference path refers to the execution sequence that the large model actually traverses when processing a complete task, consisting of a series of specific computational sub-modules that are activated sequentially.

[0189] In this embodiment, step 106 includes the following process:

[0190] Step 1061: Parse the final path control instruction to determine the target sub-module to be executed in the large model. The target sub-module is used to perform computational tasks for specific inference steps.

[0191] In step 1061, the target submodule is a computational component within the large model that has a specific function and can be called independently, such as a module specifically for text understanding or a module specifically for numerical analysis.

[0192] In this embodiment, the string or encoding format of the final path control instruction is parsed to extract the computational unit identifier contained therein; then, based on the identifier, a search and match is performed in the module registry of the large model to locate the corresponding specific software function or hardware computational unit, and the function or unit is determined as the target submodule.

[0193] Step 1062: Allocate corresponding computing resources to the target submodule according to the computing resource scheduling parameters. The computing resources are used to provide the processing units and memory space required for the execution of the target submodule.

[0194] In step 1062, the computing resources include a central processing unit core, a graphics processing unit stream processor, a dedicated artificial intelligence accelerator core, and runtime memory.

[0195] In this embodiment, the number of computing cores and the batch size specified in the computing resource scheduling parameters are read. First, according to the specified number of cores, a corresponding number of idle computing cores are requested from the underlying operating system or runtime environment, and these cores are bound to the execution thread of the target submodule. Second, according to the specified batch size, a data buffer of a corresponding capacity is opened or reserved in memory to store the input and output data of the target submodule.

[0196] Step 1063: Execute the target submodule using the allocated computing resources to obtain the execution result, which includes the intermediate features and attention data generated in the current inference step.

[0197] In this embodiment, the input data to be processed is first divided according to the batch size and loaded into the data buffer allocated in step 1062; then, the execution thread bound to the computing core is started, and the computing function of the target submodule is called to process the batch of data; after the target submodule finishes running, its output intermediate layer feature data and attention distribution data are written into the specified output buffer, and these data together constitute the execution result of this step.

[0198] Step 1064: Use the execution result as the input for the next inference step of the large model, and repeat the steps of determining, allocating and executing to form an optimized inference path.

[0199] In this embodiment, the execution result obtained in step 1063, especially the intermediate layer feature data, is used as the input data for the next inference step. Then, the process returns to step 1061, and based on the newly generated final path control instruction for the next inference step, a new target sub-module is determined and new computing resources are allocated to it before execution. Through such iterative loops, the target sub-modules activated in each step are connected in the execution order to form a complete optimized inference path for the current task.

[0200] In the above process, this application transforms the abstract control instructions and resource parameters generated based on dynamic entropy perception and collaborative decision-making into specific scheduling and execution actions on physical computing devices, and constructs a complete task execution path through iterative loops, thereby ultimately implementing the intelligent optimization decisions of the upper layer into improvements in the computing efficiency and stability of the lower layer.

[0201] Figure 3 A schematic diagram of the structure of a large model inference path optimization system based on dynamic entropy awareness provided in this application embodiment is shown below. Figure 3 As shown, the system includes:

[0202] The acquisition module 31 is used to acquire intermediate layer feature data, attention distribution data, and original magnetic field data generated by the large model during multi-round interactive inference.

[0203] The calculation module 32 is used to calculate information entropy data based on the intermediate layer feature data, and to construct a state transition diagram based on the information entropy data and the attention distribution data.

[0204] The generation module 33 is used to preprocess the original magnetic field data, generate environmental disturbance signals, and generate fused state data based on the state transition diagram and the environmental disturbance signals.

[0205] The input module 34 is used to input the fused state data into the maximum entropy reinforcement learning model, perform joint optimization decision-making through the policy network of the maximum entropy reinforcement learning model, and output preliminary path branch weights and computational resource scheduling parameters.

[0206] The optimization module 35 is used to optimize path features based on the initial path branch weights and the state transition diagram using a graph neural network model to generate the final path control instructions.

[0207] The scheduling module 36 is used to dynamically schedule and activate the computing units inside the large model according to the final path control instruction and the computing resource scheduling parameters, so as to form an optimized inference path.

[0208] The large model inference path optimization system based on dynamic entropy awareness in this application is used to implement the aforementioned large model inference path optimization method based on dynamic entropy awareness. Therefore, the specific implementation of the large model inference path optimization system based on dynamic entropy awareness can be found in the embodiment section of the large model inference path optimization method based on dynamic entropy awareness mentioned above. The specific implementation can be referred to the description of the corresponding embodiment, and will not be repeated here.

[0209] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described large model inference path optimization methods based on dynamic entropy awareness.

[0210] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for optimizing large model inference paths based on dynamic entropy awareness.

[0211] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0212] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the large model inference path optimization method based on dynamic entropy awareness.

[0213] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0215] The foregoing has provided a detailed description of a method and system for optimizing large-scale model inference paths based on dynamic entropy perception, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for optimizing inference paths in large models based on dynamic entropy awareness, characterized in that, include: Acquire intermediate layer feature data, attention distribution data, and raw magnetic field data generated by the large model during multi-round interactive inference; Information entropy data is calculated based on the intermediate layer feature data, and a state transition graph is constructed based on the information entropy data and the attention distribution data. The original magnetic field data is preprocessed to generate an environmental disturbance signal, and fused state data is generated based on the state transition diagram and the environmental disturbance signal. The fused state data is input into the maximum entropy reinforcement learning model, and joint optimization decision is made through the policy network of the maximum entropy reinforcement learning model, outputting preliminary path branch weights and computational resource scheduling parameters; Based on the initial path branch weights and the state transition diagram, a graph neural network model is used to optimize path features in order to generate the final path control instructions. Based on the final path control command and the computing resource scheduling parameters, the computing units within the large model are dynamically scheduled and activated to form an optimized inference path; The step of calculating information entropy data based on the intermediate layer feature data, and constructing a state transition graph based on the information entropy data and the attention distribution data, includes: Extract the feature vector corresponding to each inference step from the intermediate layer feature data; For each feature vector, a probability distribution estimate is performed, and the information entropy value of each inference step is calculated based on the estimation results to form information entropy data; Based on the information entropy data, each reasoning step is set as a corresponding node; The attention weights between every two inference steps are extracted from the attention distribution data, and directed connections are established between the nodes of the corresponding two inference steps based on the attention weights. Generate a state transition graph based on all the nodes and all the directed edges.

2. The method according to claim 1, characterized in that, The step of inputting the fused state data into the maximum entropy reinforcement learning model, performing joint optimization decisions through the policy network of the maximum entropy reinforcement learning model, and outputting preliminary path branch weights and computational resource scheduling parameters includes: The fused state data is input into the policy network of the maximum entropy reinforcement learning model. The fused state data is weighted by importance through the attention mechanism in the policy network, and weighted features are extracted. The weighted features are used to characterize the correlation strength between environmental perturbation and inference state. The weighted features are respectively input into the path planning branch and the resource scheduling branch of the policy network. The path planning branch includes a first policy head, and the resource scheduling branch includes a second policy head. The first strategy head output contains a first probability distribution of multiple candidate path selection actions, and the second strategy head output contains a second probability distribution of multiple resource scheduling actions. The first probability distribution is used to describe the tendency to select different inference path branches, and the second probability distribution is used to describe the tendency to allocate different proportions of computing resources. The first probability distribution and the second probability distribution are optimized respectively using the policy gradient algorithm in the policy network; The optimized first probability distribution and the optimized second probability distribution are sampled through the sampling layer of the policy network to generate path selection code and resource scheduling code. The path selection encoding is input into the path decoder of the policy network to obtain the initial path branch weights; The resource scheduling code is input into the scheduling decoder of the policy network to obtain the calculated resource scheduling parameters.

3. The method according to claim 2, characterized in that, The optimization of the first probability distribution and the second probability distribution using the policy gradient algorithm in the policy network includes: Based on the fused state data, the expected return estimates of the first probability distribution and the second probability distribution are calculated respectively. The expected return estimates are used to characterize the prediction of the long-term cumulative utility that can be obtained by choosing different actions in the current reasoning state. Based on the expected return estimate, the first policy gradient vector corresponding to the first probability distribution and the second policy gradient vector corresponding to the second probability distribution are calculated using the generalized advantage estimation algorithm. The first policy gradient vector and the second policy gradient vector are used to indicate the direction of adjusting the corresponding policy parameters in order to improve the inference path optimization effect. The network parameters of the first policy head are adjusted in the opposite direction of the first policy gradient vector using the policy gradient algorithm to update the first probability distribution. The network parameters of the second policy head are adjusted in the opposite direction of the gradient vector of the second policy to update the second probability distribution.

4. The method according to claim 1, characterized in that, The process of preprocessing the raw magnetic field data to generate an environmental disturbance signal, and generating fused state data based on the state transition diagram and the environmental disturbance signal, includes: The original magnetic field data is processed by a convolutional neural network to extract change features, thus obtaining magnetic field change data. The magnetic field change feature vector is input into a long short-term memory network, and sequence prediction is performed through the long short-term memory network to obtain magnetic field prediction data; The magnetic field prediction data is normalized to generate an environmental disturbance signal. Obtain the current node corresponding to the current inference step from the state transition diagram, and the set of neighboring nodes of the current node; Extract the information entropy value and connection relationship of each node in the current node and the set of neighboring nodes; The information entropy value and connection relationship of the current node and the set of neighboring nodes are combined with the environmental disturbance signal to generate fused state data.

5. The method according to claim 1, characterized in that, The step of optimizing path features using a graph neural network model based on the initial path branch weights and the state transition graph to generate final path control instructions includes: Based on the initial path branch weights and the information entropy values ​​of each node in the state transition graph, initial node features are generated for each node. These initial node features are used to represent the initial path selection tendency and information uncertainty of the corresponding reasoning step. The graph structure of the state transition graph and the initial node features are input into the graph neural network model. Through the message passing unit of the graph neural network model, the features of each node are passed to the adjacent nodes along the directed connection edges between the nodes. The node aggregation unit of the graph neural network model aggregates the transmitted features received by each node from all upstream nodes, and updates the aggregated features using a nonlinear activation function to obtain the updated features of the node. The updated features are used to represent the path selection tendency corrected after the graph structure information is transmitted. Repeat the message passing and aggregation operations until the preset number of iterations is reached; The updated features of each node obtained after reaching the specified number of iterations are input into the output unit of the graph neural network model, and the output unit generates the final path control command.

6. The method according to claim 1, characterized in that, The step of dynamically scheduling and activating computing units within the large model based on the final path control command and the computing resource scheduling parameters to form an optimized inference path includes: The final path control instruction is parsed to determine the target sub-module to be executed in the large model. The target sub-module is used to perform computational tasks for specific inference steps. According to the computing resource scheduling parameters, corresponding computing resources are allocated to the target sub-module, and the computing resources are used to provide the processing units and memory space required for the execution of the target sub-module. The target submodule is executed using the allocated computing resources to obtain the execution result, which includes intermediate features and attention data generated in the current inference step. The execution result is used as the input for the next inference step of the large model, and the steps of determining, allocating, and executing are repeated to form an optimized inference path.

7. A large-scale model inference path optimization system based on dynamic entropy awareness, characterized in that, include: The acquisition module is used to acquire intermediate layer feature data, attention distribution data, and raw magnetic field data generated by the large model during multi-round interactive inference. The calculation module is used to calculate information entropy data based on the intermediate layer feature data, and to construct a state transition graph based on the information entropy data and the attention distribution data; The generation module is used to preprocess the original magnetic field data, generate environmental disturbance signals, and generate fused state data based on the state transition diagram and the environmental disturbance signals. The input module is used to input the fused state data into the maximum entropy reinforcement learning model, perform joint optimization decision-making through the policy network of the maximum entropy reinforcement learning model, and output preliminary path branch weights and computational resource scheduling parameters. An optimization module is used to optimize path features based on the initial path branch weights and the state transition diagram using a graph neural network model to generate final path control instructions. The scheduling module is used to dynamically schedule and activate the computing units within the large model according to the final path control instruction and the computing resource scheduling parameters, so as to form an optimized inference path; The step of calculating information entropy data based on the intermediate layer feature data, and constructing a state transition graph based on the information entropy data and the attention distribution data, includes: Extract the feature vector corresponding to each inference step from the intermediate layer feature data; For each feature vector, a probability distribution estimate is performed, and the information entropy value of each inference step is calculated based on the estimation results to form information entropy data; Based on the information entropy data, each reasoning step is set as a corresponding node; The attention weights between every two inference steps are extracted from the attention distribution data, and directed connections are established between the nodes of the corresponding two inference steps based on the attention weights. Generate a state transition graph based on all the nodes and all the directed edges.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the dynamic entropy-aware large model inference path optimization method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the large model inference path optimization method based on dynamic entropy awareness as described in any one of claims 1 to 6.

Images