Explanatory method, device and related equipment for large model reasoning process based on circuit and embedding

Abstract

The present application discloses a method, device and related equipment for interpretability of a large model reasoning process based on Circuit and Embedding. The method of the present application first runs the large model and records the intermediate states of each hidden layer during the reasoning process. An information flow graph is constructed based on the intermediate states, in which nodes represent the Token embedding representations of each hidden layer and edges represent the computational operations between hidden layers. An attribution algorithm is used to calculate the contribution of each edge in the information flow graph to the output result, and key edges are screened according to the contribution, and the Token embedding representations of key nodes are screened in the nodes associated with the key edges. Finally, prompt instructions are used to guide the large model to interpret the Token embedding representations of key nodes to generate interpretation results. By analyzing the intermediate states, constructing the information flow graph, identifying the key paths and generating natural language explanations, the reasoning process of the large model is effectively explained, and the interpretability of the large model is enhanced.

Description

Technical Field

[0001] The present application relates to the field of large models, and more specifically, to a method, device and related equipment for interpretability of large model reasoning processes based on Circuit and Embedding. Background Art

[0002] Big models have made significant progress in the field of natural language processing, but the opacity of their internal operating mechanisms, the so-called "black box" feature, has always been a major obstacle to their widespread application. It is difficult to understand how big models reach specific conclusions or generate specific texts, which directly affects the interpretability of big models, especially in scenarios involving critical decisions, such as medical diagnosis, legal consultation, etc. This lack of transparency also makes it more difficult to debug and improve big models, and it is difficult to locate the root cause of model errors or deviations, thus hindering further improvement of model performance.

[0003] Existing interpretability methods mainly focus on the overall behavior of large models, such as the relationship between input and output, or the importance of input features, ignoring the complex interactions of large models, failing to fully capture the changes and deep logic of the model when processing information, and only stay at the description of surface phenomena, unable to go deep into the "black box" of the model to explore the essence of its operation. Therefore, there is an urgent need for a method that can effectively explain the reasoning process of the model to enhance the interpretability of large models and promote the improvement and development of models. Summary of the invention

[0004] In view of the above problems, this application is proposed to provide a method, device and related equipment for interpretability of large model reasoning process based on Circuit and Embedding to improve the interpretability of large model reasoning process. The specific scheme is as follows:

[0005] First, a method for interpreting the reasoning process of a large model based on Circuit and Embedding is provided, including:

[0006] Run the large model for reasoning to obtain output results and intermediate states of each hidden layer during the reasoning process of the large model, wherein the intermediate state is an embedded representation of the token Token output by the large model in each hidden layer during the reasoning process;

[0007] An information flow graph is constructed based on the intermediate state, wherein the information flow graph includes nodes and edges, wherein the nodes represent the embedded representation of the token of each hidden layer, and the edges represent the computational operations between the hidden layers represented by the two associated nodes;

[0008] Calculate the contribution of the edges in the information flow graph to the output result by using an attribution algorithm, and select key edges in the information flow graph according to the contribution;

[0009] In the embedded representations of the tokens of the nodes in the information flow graph, the embedded representations of the tokens of the key nodes associated with the key edges are selected;

[0010] The prompt instruction Prompt is used to instruct the large model to interpret the embedded representation of the Token of the key node to obtain the interpretation result output by the large model.

[0011] Preferably, the running of the large model for reasoning to obtain the output result and the intermediate state of each hidden layer in the reasoning process of the large model includes:

[0012] Run the large model with the setting code added for reasoning to obtain the output results and the intermediate states of each hidden layer during the reasoning process of the large model. The setting code is used to record the embedded representation of the token Token output by the large model in each hidden layer during the reasoning process.

[0013] Preferably, before the large model with the setting code added is run to perform reasoning and obtain the output result and the intermediate state of each hidden layer in the large model reasoning process, it also includes:

[0014] Copy the large model source code to the current project directory;

[0015] The setting code is added to the large model source code.

[0016] Preferably, the prompt instruction Prompt is used to instruct the large model to interpret the embedded representation of the Token of the key node to obtain the interpretation result output by the large model, including:

[0017] Construct an explanation prompt word containing a placeholder, wherein the explanation prompt word is used to instruct the large model to explain the embedding representation of the token corresponding to the placeholder;

[0018] The explanation prompt word is sent to the large model, and the placeholder in the explanation prompt word is replaced with the embedded representation of the Token of the key node at a shallow position of the large model, and the explanation result finally output by the large model is obtained.

[0019] Preferably, the process of screening the key edges in the information flow graph according to the contribution degree includes:

[0020] It is determined whether the contribution is greater than a preset threshold, and if so, the edge in the information flow graph corresponding to the contribution is taken as a key edge.

[0021] Preferably, calculating the contribution of the edges in the information flow graph to the output result by using an attribution algorithm, and screening the key edges in the information flow graph according to the contribution, comprises:

[0022] For each edge in the information flow graph, calculate the similarity between the embedded representation of the Token of the node pointed to by the edge and the vector representation corresponding to the output result;

[0023] Based on the similarity, calculating the contribution of each edge to the output result;

[0024] Normalize the contribution of all edges;

[0025] The edges whose contribution is greater than a preset threshold are screened and determined as key edges in the information flow graph.

[0026] Secondly, a large model reasoning process explainability device based on Circuit and Embedding is provided, including:

[0027] A large model reasoning unit is used to run the large model for reasoning, obtain the output result and the intermediate state of each hidden layer in the large model reasoning process, and the intermediate state is the embedded representation of the token Token output by the large model in each hidden layer in the reasoning process;

[0028] A flow graph construction unit, used to construct an information flow graph based on the intermediate state, wherein the information flow graph comprises nodes and edges, wherein the nodes represent the embedded representation of the token of each hidden layer, and the edges represent the computational operations between the hidden layers represented by the two associated nodes;

[0029] A key edge screening unit, used to calculate the contribution of the edge in the information flow graph to the output result through an attribution algorithm, and screen the key edges in the information flow graph according to the contribution;

[0030] A key embedding representation screening unit, configured to screen the embedding representation of the token of the key node associated with the key edge from the embedding representation of the token of each node in the information flow graph;

[0031] The explanation result generating unit is used to instruct the large model to explain the embedded representation of the Token of the key node through a prompt instruction Prompt, so as to obtain the explanation result output by the large model.

[0032] In a third aspect, an electronic device is provided, comprising: a memory and a processor;

[0033] The memory is used to store programs;

[0034] The processor is used to execute the program to implement the various steps of the large model reasoning process explainability method based on Circuit and Embedding described in any one of the first aspects of the present application.

[0035] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the large model reasoning process explainability method based on Circuit and Embedding described in any one of the aforementioned first aspects of the present application are implemented.

[0036] In a fifth aspect, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements the various steps of the large model reasoning process explainability method based on Circuit and Embedding described in any one of the aforementioned first aspects of the present application.

[0037] By means of the above technical solution, the present application records the embedded representation of the tokens output by each hidden layer in the reasoning process of the large model, and constructs an information flow graph based on these intermediate states, so as to show the calculation process inside the model. The intermediate state of the large model contains rich information about the reasoning process of the large model, and reveals how the large model gradually processes information and finally obtains the output result. By calculating the contribution of each edge in the information flow graph to the output result through the attribution algorithm, and screening the key edges, the key calculation path that the large model relies on when generating the final output result can be effectively located, avoiding the traditional method of "black box" understanding of the internal mechanism of the large model, thereby improving the transparency of the large model. By prompting the Prompt instruction to instruct the large model to interpret the embedded representation of the Token of the key node to generate an explanation result, the abstract intermediate state of the large model is converted into a natural language description, which further improves the understandability of the interpretability, and can more intuitively understand the decision-making process of the large model, that is, how the model obtains the output result, thereby enhancing the interpretability of the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0039] Figure 1 A flowchart of a large model reasoning process explainability method based on Circuit and Embedding provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the structure of a large model reasoning process explainability device based on Circuit and Embedding provided in an embodiment of the present application;

[0041] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] First of all, the method provided in this application involves concepts related to large models and the interpretability of large models. For ease of understanding, the related concepts of large models and the interpretability of large models are introduced below.

[0044] Large models: Large models usually refer to deep learning models with huge parameters and complex architectures. These models often contain billions or even trillions of parameters and require massive amounts of data to train. They represent a major advancement in the field of artificial intelligence, demonstrating strong capabilities in a variety of complex tasks, such as natural language processing, computer vision, speech recognition, etc. Large models perform well because they are able to learn complex patterns and representations from large amounts of data and generalize them to new, unseen data. This learning ability enables them to achieve breakthrough progress in tasks such as translation, text generation, image recognition, and code generation.

[0045] The training of large models requires powerful computing resources and huge data sets. The training of large models can use specialized hardware, such as graphics processing units or tensor processing units, and distributed computing frameworks to process data in parallel. The training process may take weeks or even months and requires a lot of energy consumption. Common deep learning architectures, such as Transformer, Recurrent Neural Network (RNN), and Convolutional Neural Network (CNN), are used to build large models. The Transformer architecture is particularly popular in large language models because of its advantages in processing long sequence data.

[0046] Explainability of large models: Explainability of large models refers to the ability to understand how large models make decisions and the reasons behind their output results. Due to the complexity and huge number of parameters of large models, their internal operating mechanisms are often considered a "black box" and difficult to understand. Lack of explainability will limit people's trust in large models, especially in key fields such as medicine and finance. Therefore, improving the explainability of large models is crucial to the application and development of models.

[0047] Research on large model interpretability aims to make the decision-making process of the model transparent so that people can understand how the model uses input data to generate output results. This helps identify potential biases, errors or vulnerabilities in the model and improves the reliability and security of large models. In addition, interpretability can also help researchers better understand the learning process of large models, thereby improving the design and training methods of large models.

[0048] The embodiment of the present application provides a large model reasoning process explainability method based on Circuit and Embedding. The present application solution can be applied to a terminal with large model reasoning capability, which can be a mobile phone, a computer, a server, etc. .

[0049] Next, refer to Figure 1 The circuit and embedding-based large model reasoning process explainability method specifically includes the following steps:

[0050] Step S100, run the large model for reasoning, obtain the output result and the intermediate state of each hidden layer in the reasoning process of the large model, and the intermediate state is the embedded representation of the token Token output by the large model in each hidden layer in the reasoning process.

[0051] Specifically, the input data is sent to the pre-trained large model for reasoning, and the intermediate states in the reasoning process are recorded to prepare for the subsequent construction of the Circuit subgraph. This step not only obtains the final output result, but more importantly, captures the understanding and processing process of the large model on the input information in different hidden layers. The intermediate state refers to the embedded representation of the Token output by each hidden layer of the model. The embedded representation of the Token is Embedding. The Token is the basic unit processed by the large model. It can be a word, phrase or other meaningful data fragment. The embedded vector of the Token is usually high-dimensional and contains rich semantic information. In addition to the embedded representation of the Token, other valuable information can also be recorded, such as the attention weight in the Transformer model, which helps to understand the mechanism of the model to allocate attention between different Tokens.

[0052] Step S110: construct an information flow graph based on the intermediate state, wherein the information flow graph comprises nodes and edges, wherein the nodes represent the embedded representation of the Token of each hidden layer, and the edges represent the computational operations between the hidden layers represented by two associated nodes.

[0053] Specifically, based on the intermediate state obtained, an information flow graph is constructed to show the information transfer process within the large model. The information flow graph is the basis for constructing the Circuit subgraph. The nodes in the graph represent the token embedding representation of each hidden layer, and the edges represent the computational operations between the hidden layers represented by the associated nodes. The edges can contain weights, which can be calculated based on attention weights, gradient information, or other relevant indicators to reflect the intensity and importance of information flow, providing a basis for subsequent Circuit screening. The structure of the information flow graph can be adjusted according to the model architecture. For example, a directed acyclic graph (DAG) or a multi-layer graph can be used to more accurately represent the computational process of the large model. For some tasks, the information flow may change dynamically over time, so it is possible to consider building a dynamic graph to capture this change, so as to more fully understand the reasoning process of the model.

[0054] Step S120: Calculate the contribution of the edges in the information flow graph to the output result through an attribution algorithm, and select the key edges in the information flow graph according to the contribution.

[0055] Specifically, in order to identify the information paths that have a greater impact on the final output results, the contribution of each edge in the information flow graph to the output results can be calculated. This can be achieved through various attribution algorithms, such as integral gradient, layer-wise relevance propagation (LRP) or deep learning important features (DeepLIFT). The selection of a suitable attribution algorithm depends on the specific task and model. Considering the limitations of the attribution algorithm itself, such as the gradient saturation problem, multiple algorithms can be combined or existing algorithms can be improved to improve the accuracy of the explanation. According to the calculated contribution score, a threshold is set or the Top-N edges are selected to filter out the key edges that contribute more to the output results. These key edges and the corresponding key nodes together constitute the Circuit subgraph. The choice of threshold can be adjusted according to the specific task, or an adaptive threshold method can be used.

[0056] Step S130: Filter the embedded representation of the Token of the key node associated with the key edge from the embedded representation of the Token of each node in the information flow graph.

[0057] Specifically, after determining the key edges, identify the key nodes connected to the key edges. The embedding representation of the tokens corresponding to the key nodes plays an important role in the model reasoning process and has a significant impact on the final output results. In addition to the nodes directly connected to the key edges, indirectly related nodes can also be considered. For example, other nodes that have an impact on the key nodes can be identified through path analysis, so as to more comprehensively understand the reasoning mechanism of the model and extract the embedding representation of the tokens of these key nodes, providing a basis for the next step of explanation.

[0058] Step S140: Prompt is used to instruct the large model to interpret the embedded representation of the Token of the key node to obtain the interpretation result output by the large model.

[0059] Specifically, the pre-trained large model is used to explain the selected key nodes. The large model can be guided to generate easy-to-understand explanations by designing prompt instructions (Prompt). For example, the large model is required to explain the role of the embedded representation of the Token in the reasoning process. In order to improve the quality of the explanation, an iterative Prompt method can be used to continuously adjust the Prompt according to the initial explanation results of the large model to make it more targeted. It is also possible to consider introducing an external knowledge base or domain expert knowledge in the Prompt to enhance the accuracy and comprehensibility of the explanation. Finally, the explanation results output by the large model can be presented in the form of text descriptions, visual charts, etc. to help users understand the reasoning process of the model.

[0060] The interpretability method of large model reasoning process based on Circuit and Embedding can effectively reveal the key information flow path and decision-making basis in the large model reasoning process by constructing information flow graph, calculating edge contribution, screening key nodes and using Prompt to explain key information, thereby improving the interpretability of the large model reasoning process.

[0061] The following embodiment describes the process of running the large model for reasoning in step S100 in the above embodiment to obtain the output result and the intermediate state of each hidden layer in the large model reasoning process, which may specifically include the following steps:

[0062] Step S200, copy the large model source code to the current project directory.

[0063] Specifically, you can determine the version and source of the large model used, and obtain its complete source code, including model definition, pre-trained weights, and related dependent libraries. Then, copy these files to the specified directory of the current project and organize the file structure for subsequent modification and operation. At the same time, you can also check whether the dependencies of the code are complete, and install missing libraries or software packages if necessary. In addition, you can configure the operating environment according to the needs of the project, such as setting necessary environment variables, specifying GPU resources, etc.

[0064] Step S210: adding the setting code to the large model source code.

[0065] Specifically, the pre-designed setting code is inserted into the appropriate position of the large model source code. The core function of the setting code is to record the intermediate state of each hidden layer of the model during the reasoning process, that is, the embedded representation of the Token. This can be achieved in many ways, such as using a hook function to execute after each layer or a specific layer of the large model, extracting the output of the layer and saving it. When inserting the setting code, you can pay attention to the compatibility and maintainability of the code, and try to avoid destructive modifications to the original code. The principle of object-oriented programming can be adopted to encapsulate the setting code into an independent module, and interact with the large model through an interface. You can also choose the location of the inserted code to ensure that the required intermediate state can be correctly captured and will not affect the normal reasoning process of the model. The setting code can be used to record the embedded representation of the token Token output in each hidden layer of the large model during the reasoning process. On this basis, the setting code can also include a selection module for specifying which layers of intermediate states are recorded, and a module for extracting and storing intermediate states. In order to facilitate debugging and analysis, the setting code can also add a logging function to record the dimension, size and other information of the intermediate state.

[0066] Step S220, run the large model with the setting code added to perform reasoning, and obtain the output results and the intermediate states of each hidden layer during the reasoning process of the large model.

[0067] Specifically, after completing the code modification, run the modified large model for reasoning. During the running process, the setting code will record the embedded representation of the token output by each hidden layer of the large model, and save this information to the specified location. In order to verify the correctness of the setting code, you can first use some simple test cases to test it, and check whether the recorded intermediate states meet expectations. In the formal reasoning process, you can select appropriate input data according to the actual tasks and data, and monitor the running status of the model, such as memory usage, GPU utilization, etc. After the reasoning is completed, collect the output results of the large model and the recorded intermediate states for subsequent analysis and interpretation. To facilitate subsequent processing, the intermediate states can be saved in a format that is easy to read and process, such as NumPy arrays, JSON files, etc.

[0068] By adding setting code to the model source code and running the modified model, the intermediate states in the large model reasoning process are captured and recorded, which provides an important basis for subsequent large model interpretation and analysis, thereby gaining a deeper understanding of the large model's reasoning process and decision-making mechanism.

[0069] The following embodiment describes the process of instructing the large model to interpret the embedded representation of the Token of the key node by the prompt instruction Prompt in step S140 of the aforementioned embodiment to obtain the interpretation result output by the large model, which may specifically include the following steps:

[0070] Step S300: construct an explanation prompt word containing a placeholder, wherein the explanation prompt word is used to instruct the large model to interpret the embedded representation of the Token corresponding to the placeholder.

[0071] Specifically, the placeholder can indicate the location where the large model will be replaced, and can also indicate the expected input type. Special symbols or tags can be used to indicate the placeholder, such as <info> [X] < / info> , where [X] is a placeholder. <info>< / info> Is an identifier used to indicate a placeholder.

[0072] In order to guide large models to generate more accurate and meaningful explanations, contextual information can be provided in the prompt words. This can include the hierarchical information of the key nodes, the input data of the model, and the expected type of explanation (for example, explaining the role of the embedding representation of the token in the reasoning process, the relationship with the embedding representation of other tokens, etc.). Different prompt word templates can be designed according to specific tasks and needs, and on this basis, prompt words can also support multiple languages.

[0073] Step S310: Send the explanation prompt word to the big model, and replace the placeholder in the explanation prompt word with the embedded representation of the Token of the key node at the shallow position of the big model, and obtain the explanation result finally output by the big model.

[0074] Specifically, the specific replacement mechanism depends on the large model and deep learning framework used. The hook function provided by the framework can be used to replace at the shallow level of the large model. To improve efficiency, the explanations of multiple key nodes can be processed in batches. Multiple explanation prompts and corresponding embedding representations are combined into a batch and then sent to the large model for reasoning. If the sequence of key nodes is very long, you can consider splitting the sequence into multiple segments, explaining them separately, and then combining the explanation results.

[0075] We choose to replace placeholders at a shallow level in the model, rather than directly replacing them when inputting into the large model. This is because replacing them at the time of input will cause the embedding representation of the Token to be immediately vectorized by the large model, thereby losing its original meaning. Replacing them at a shallow level can avoid this premature vectorization and retain the semantic information in the embedding representation of the Token. In addition, replacing placeholders at a shallow level can also prevent the embedding representation of the Token from being over-abstracted at a deep level in the model, thereby retaining more original semantic information, helping to improve the accuracy and comprehensibility of the explanation, and directly using the knowledge and language capabilities of the large model itself to explain the meaning of key nodes, avoiding reliance on external interpreters or complex feature engineering, thereby simplifying the explanation process.

[0076] The following embodiment describes the process of calculating the contribution of the edge in the information flow graph to the output result by the attribution algorithm in the above step S120, and screening the key edge in the information flow graph according to the contribution, which specifically includes the following steps:

[0077] Step S400: For each edge in the information flow graph, calculate the similarity between the embedded representation of the Token of the node pointed to by the edge and the vector representation corresponding to the output result.

[0078] Specifically, the similarity between the embedding representation of the token of the node pointed to by each edge in the information flow graph and the embedding representation of the token of the target node corresponding to the last level in the large model is calculated, that is, the vector representation of the output result of the large model can be the embedding representation of the token of the target node corresponding to the last level in the large model. The calculation process of the large model can be regarded as the process of information propagation from the bottom layer to the top layer. Therefore, calculating the similarity between the embedding representation of the node pointed to by each edge and the embedding representation of the final target node can reflect the importance of the edge in the information propagation path.

[0079] Step S410: Based on the similarity, calculate the contribution of each edge to the output result.

[0080] Specifically, the contribution of each edge to the output result is calculated. The contribution calculation method can be designed according to specific needs. The similarity can be processed into understandable contribution by designing calculation rules. More complex strategies can consider factors such as the level of the edge and the type of nodes connected to the edge. For example, different weights can be assigned according to the level of the edge, and the edge closer to the output layer has a higher weight.

[0081] Step S420: normalize the contribution of all edges.

[0082] Specifically, the contribution of all edges is normalized to fall within the interval [0, 1]. Normalization can eliminate the influence of the difference in contribution values ​​between different edges, facilitating subsequent comparison and screening. Min-max normalization and z-score normalization can be used.

[0083] Step S430: Filter the edges whose contribution is greater than a preset threshold and determine them as key edges in the information flow graph.

[0084] Specifically, a preset threshold can be set to filter out edges whose contribution is greater than the threshold, and these edges are determined as key edges in the information flow graph. The choice of the threshold can be adjusted according to the specific task and data set.

[0085] Through the above steps, the key edges that contribute more to the reasoning process of the large model can be effectively identified, providing a basis for subsequent explanation generation.

[0086] The following describes the large model reasoning process explainability device based on Circuit and Embedding provided in an embodiment of the present application. The large model reasoning process explainability device based on Circuit and Embedding described below and the large model reasoning process explainability method based on Circuit and Embedding described above can refer to each other.

[0087] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a large model reasoning process explainability device based on Circuit and Embedding disclosed in an embodiment of the present application.

[0088] like Figure 2 As shown, the device may include:

[0089] A large model reasoning unit 11 is used to run the large model for reasoning, and obtain output results and intermediate states of each hidden layer in the large model reasoning process, wherein the intermediate state is an embedded representation of the token Token output by the large model in each hidden layer in the reasoning process;

[0090] A flow graph construction unit 12 is used to construct an information flow graph based on the intermediate state, wherein the information flow graph includes nodes and edges, wherein the nodes represent the embedded representation of the token of each hidden layer, and the edges represent the calculation operations between the hidden layers represented by the two associated nodes;

[0091] A key edge screening unit 13, configured to calculate the contribution of the edge in the information flow graph to the output result by using an attribution algorithm, and screen the key edges in the information flow graph according to the contribution;

[0092] A key embedding representation screening unit 14 is used to screen the embedding representation of the token of the key node associated with the key edge from the embedding representation of the token of each node in the information flow graph;

[0093] The interpretation result generating unit 15 is used to instruct the large model to interpret the embedded representation of the Token of the key node through a prompt instruction Prompt, so as to obtain the interpretation result output by the large model.

[0094] Optionally, the large model inference unit 11 runs the large model for inference to obtain the output result and the intermediate state of each hidden layer in the large model inference process, including:

[0095] Run the large model with the setting code added for reasoning to obtain the output results and the intermediate states of each hidden layer during the reasoning process of the large model. The setting code is used to record the embedded representation of the token Token output by the large model in each hidden layer during the reasoning process.

[0096] Optionally, the device of the present application may further include:

[0097] The source code copy unit 21 is used to copy the large model source code to the current project directory;

[0098] The setting code adding unit 22 is used to add the setting code to the large model source code.

[0099] Optionally, the explanation result generating unit 15 instructs the large model to explain the embedded representation of the Token of the key node through a prompt instruction Prompt, and obtains the explanation result output by the large model, including:

[0100] Construct an explanation prompt word containing a placeholder, wherein the explanation prompt word is used to instruct the large model to explain the embedding representation of the token corresponding to the placeholder;

[0101] The explanation prompt word is sent to the large model, and the placeholder in the explanation prompt word is replaced with the embedded representation of the Token of the key node at a shallow position of the large model, and the explanation result finally output by the large model is obtained.

[0102] Optionally, the process of the key edge screening unit 13 screening the key edges in the information flow graph according to the contribution degree includes:

[0103] It is determined whether the contribution is greater than a preset threshold, and if so, the edge in the information flow graph corresponding to the contribution is taken as a key edge.

[0104] Optionally, the key edge screening unit 13 calculates the contribution of the edge in the information flow graph to the output result by using an attribution algorithm, and screens the key edges in the information flow graph according to the contribution, including:

[0105] For each edge in the information flow graph, calculate the similarity between the embedded representation of the Token of the node pointed to by the edge and the vector representation corresponding to the output result;

[0106] Based on the similarity, calculating the contribution of each edge to the output result;

[0107] Normalize the contribution of all edges;

[0108] The edges whose contribution is greater than a preset threshold are screened and determined as key edges in the information flow graph.

[0109] The present application also provides an electronic device in an embodiment. Figure 3 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, teaching large screens, wearable devices, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0110] like Figure 3 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603, so as to implement the large model reasoning process interpretability method based on Circuit and Embedding in the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in RAM 603. The processing device 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0111] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0112] Also provided in an embodiment of the present application is a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any of the large model reasoning process explainability methods based on Circuit and Embedding provided in the embodiments of the present application.

[0113] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the large model reasoning process explainability methods based on Circuit and Embedding provided in the embodiments of the present application.

[0114] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0115] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0116] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0117] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0118] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A large model reasoning process explainability method based on Circuit and Embedding, characterized by: include: Run the large model for reasoning to obtain output results and intermediate states of each hidden layer during the reasoning process of the large model, wherein the intermediate state is an embedded representation of the token Token output by the large model in each hidden layer during the reasoning process; An information flow graph is constructed based on the intermediate state, wherein the information flow graph includes nodes and edges, wherein the nodes represent the embedded representation of the token of each hidden layer, and the edges represent the computational operations between the hidden layers represented by the two associated nodes; Calculating the contribution of the edges in the information flow graph to the output result by using an attribution algorithm, and screening the key edges in the information flow graph according to the contribution; In the embedded representations of the tokens of the nodes in the information flow graph, the embedded representations of the tokens of the key nodes associated with the key edges are selected; The prompt instruction Prompt is used to instruct the large model to interpret the embedded representation of the Token of the key node to obtain the interpretation result output by the large model.

2. The method according to claim 1, characterized in that The running of the large model for reasoning to obtain output results and intermediate states of each hidden layer during the large model reasoning process includes: Run the large model with the setting code added for reasoning to obtain the output results and the intermediate states of each hidden layer during the reasoning process of the large model. The setting code is used to record the embedded representation of the token Token output by the large model in each hidden layer during the reasoning process.

3. The method according to claim 2, characterized in that Before the large model with added setting code is run to perform reasoning and obtain output results and intermediate states of each hidden layer in the large model reasoning process, the method further includes: Copy the large model source code to the current project directory; The setting code is added to the large model source code.

4. The method according to claim 1, characterized in that The prompt instruction Prompt is used to instruct the large model to interpret the embedded representation of the Token of the key node to obtain the interpretation result output by the large model, including: Construct an explanation prompt word containing a placeholder, wherein the explanation prompt word is used to instruct the large model to explain the embedding representation of the token corresponding to the placeholder; The explanation prompt word is sent to the large model, and the placeholder in the explanation prompt word is replaced with the embedded representation of the Token of the key node at the shallow position of the large model, and the explanation result finally output by the large model is obtained.

5. The method according to claim 1, characterized in that The process of selecting the key edges in the information flow graph according to the contribution degree includes: It is determined whether the contribution is greater than a preset threshold, and if so, the edge in the information flow graph corresponding to the contribution is taken as a key edge.

6. The method according to claim 1, characterized in that The step of calculating the contribution of the edge in the information flow graph to the output result by using an attribution algorithm, and screening the key edge in the information flow graph according to the contribution, comprises: For each edge in the information flow graph, calculate the similarity between the embedded representation of the Token of the node pointed to by the edge and the vector representation corresponding to the output result; Based on the similarity, calculating the contribution of each edge to the output result; Normalize the contribution of all edges; The edges whose contribution is greater than a preset threshold are screened and determined as key edges in the information flow graph.

7. A large model reasoning process explainability device based on Circuit and Embedding, characterized in that: include: A large model reasoning unit is used to run the large model for reasoning, obtain the output result and the intermediate state of each hidden layer in the large model reasoning process, and the intermediate state is the embedded representation of the token Token output by the large model in each hidden layer in the reasoning process; A flow graph construction unit, used to construct an information flow graph based on the intermediate state, wherein the information flow graph comprises nodes and edges, wherein the nodes represent the embedded representation of the token of each hidden layer, and the edges represent the computational operations between the hidden layers represented by the two associated nodes; A key edge screening unit, used to calculate the contribution of the edge in the information flow graph to the output result through an attribution algorithm, and screen the key edges in the information flow graph according to the contribution; A key embedding representation screening unit, configured to screen the embedding representation of the token of the key node associated with the key edge from the embedding representation of the token of each node in the information flow graph; The explanation result generating unit is used to instruct the large model to explain the embedded representation of the Token of the key node through a prompt instruction Prompt, so as to obtain the explanation result output by the large model.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the large model reasoning process explainability method based on Circuit and Embedding as described in any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the large model reasoning process explainability method based on Circuit and Embedding as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the large model reasoning process explainability method based on Circuit and Embedding as described in any one of claims 1 to 6 is implemented.

Images