Pathological hybrid expert model-based expert behavior analysis method and system

Abstract

The invention provides an expert behavior analysis method and system based on a pathological hybrid expert model, and the method comprises the steps: calculating the sum of each row of distribution weights of a basic pathological hybrid expert model, carrying out the statistics of the distribution shape of the contribution to a slot in an input token, and carrying out the evaluation of the contribution of Tokens to the slot; calculating a relative weight average value of each expert in the MoE expert layer for different output tokens, counting the importance degree of different experts in the MoE expert layer for output, and performing expert output contribution evaluation of the MoE expert layer; counting the number of the tokens reaching a set cumulative weight, measuring the distribution difference of the token weights of different slot positions, and evaluating the number of the tokens of each slot position; carrying out inverse proportion shadow processing on the input patch by calculating linear combination weights of different slot positions, and carrying out expert attention visualization evaluation; and optimizing expert configuration in the model.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent analysis configuration control for machine learning model evaluation, and more specifically, to an expert behavior analysis method and system based on a pathological hybrid expert model. Background Art

[0002] Neural network scaling is critical to improving model quality in a variety of real-world machine learning models that have large amounts of training data and computation. For example, for computer vision, increasing model capacity can improve image classification and detection accuracy for various computer vision architectures. This general trend has prompted recent research on factors that play a key role in scaling success, including the amount of training data, model size, and the amount of computation used. While the final model quality scales exponentially with the amount of data, computation, and model size, the significant quality gains brought by larger models also bring various practical challenges, such as computational cost, ease of programming, and implementation issues on parallel devices.

[0003] In the prior art, the traditional interpretable method of pathological hybrid expert models and dense models is usually to generate attention heatmaps through the attention layer in Transformer to show the degree of attention of the model to different areas and histological morphology of pathological sections; however, this method has some deficiencies in interpretability, including: coarse explanation granularity: the attention heatmap provides a global attention map, which shows the degree of attention of the model to the entire input data, but this globality often lacks fine-grained information and cannot specify which specific features or details play a key role in the final decision; lack of specific decision path: although the attention mechanism shows which parts are paid attention to, it fails to reveal the specific decision path and logical process within the model, and does not explain in detail how each input affects the various levels and final output of the model. It is difficult for doctors and researchers to understand the specific operation steps and intermediate processes of the model; the above problems have yet to be solved; therefore, it is necessary to propose an expert behavior analysis method and system based on the pathological hybrid expert model to at least partially solve the problems existing in the prior art. Summary of the invention

[0004] A series of simplified concepts are introduced in the content of the invention, which will be further described in detail in the specific implementation method section; the content of the invention of the present invention does not mean to attempt to limit the key features and essential technical features of the technical solution claimed for protection, nor does it mean to attempt to determine the scope of protection of the technical solution claimed for protection.

[0005] To at least partially solve the above problems, the present invention provides an expert behavior analysis method based on a pathological hybrid expert model, comprising:

[0006] S100, calculate the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtain the sum of the relative weights of each input token for all slots, count the distribution shape of the input token's contribution to the slot, and evaluate the contribution of tokens to the slot;

[0007] S200, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, comparing and sorting the average values ​​of each column of the combined weights, counting the importance of different experts in the MoE expert layer for the output, and evaluating the output contribution of experts in the MoE expert layer;

[0008] S300, in the linear combination of input tokens of the basic pathology hybrid expert model, count the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots, and evaluate the number of tokens in each slot;

[0009] S400 calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the focus image area tendencies of different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model and obtains a configuration-optimized pathology hybrid expert model.

[0010] Preferably, S100 includes:

[0011] S101, creating a basic pathology mixed expert model and calculating the sum of each row of the assigned weights;

[0012] S102, according to the sum of each row of the assigned weights, the sum of the relative weights of each input token to all slots is obtained, and the total contribution of the input token to the slot is obtained;

[0013] S103, according to the total amount of input token contribution to the slot, count the distribution shape of the input token contribution to the slot, and evaluate the contribution of Tokens to the slot.

[0014] Preferably, S200 includes:

[0015] S201, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, and obtaining the average value of each column of the combined weight;

[0016] S202, comparing and sorting the average values ​​of each column of the combined weights according to the average values ​​of each column of the combined weights, and obtaining the sorting of the average values ​​of each column of the combined weights;

[0017] S203, sorting according to the average value of each column of the combined weight, counting the importance of different experts in the MoE expert layer to the output, and evaluating the output contribution of experts in the MoE expert layer.

[0018] Preferably, S300 includes:

[0019] S301, counting the number of tokens that reach a set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model;

[0020] S302, based on the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots and evaluate the number of tokens in each slot.

[0021] Preferably, S400 includes:

[0022] S401, by calculating the linear combination weights of different slots, the input patch is shadowed in inverse proportion according to the linear combination weights of the different slots;

[0023] S402, visually highlighting the attention image area tendency of different slots according to the inverse proportional shading processing; providing an intuitive image of the average input token of the corresponding slot according to the attention image area tendency, and performing visual evaluation of expert attention;

[0024] S403, based on the evaluation of the contribution of tokens to the slots, the evaluation of the contribution of the expert outputs of the MoE expert layer, the evaluation of the number of tokens in each slot and the visualization evaluation of the expert attention, optimize the expert configuration in the model, visualize the internal working mechanism of the basic pathology hybrid expert model, and obtain the configuration-optimized pathology hybrid expert model.

[0025] The present invention provides an expert behavior analysis system based on a pathological hybrid expert model, comprising:

[0026] The Tokens slot contribution evaluation module calculates the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtains the sum of the relative weights of each input token for all slots, counts the distribution shape of the input token's contribution to the slot, and evaluates the contribution of Tokens to the slot;

[0027] The MoE expert layer output contribution evaluation module calculates the relative weight average of each expert in the MoE expert layer for different output tokens, compares and sorts the average values ​​of each column of the combined weight, and counts the importance of different experts in the MoE expert layer for the output, and evaluates the output contribution of experts in the MoE expert layer;

[0028] The slot token quantity evaluation module counts the number of tokens that reach the set cumulative weight in the linear combination of the input tokens of the basic pathology hybrid expert model, measures the difference in token weight distribution of different slots, and evaluates the number of tokens in each slot;

[0029] The combined weight tendency evaluation configuration module calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the tendency of the image area of ​​interest in different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model to obtain a configuration-optimized pathology hybrid expert model.

[0030] Preferably, the Tokens slot contribution evaluation module includes:

[0031] Assign weight row calculation unit, create basic pathology mixed expert model, calculate the sum of each row of assigned weight;

[0032] The total contribution analysis unit of the token slots obtains the sum of the relative weights of each input token for all slots based on the sum of each row of the assigned weights, and obtains the total contribution of the input token to the slots;

[0033] The slot contribution distribution statistics unit counts the distribution shape of the input token's contribution to the slot based on the total amount of input token's contribution to the slot, and evaluates the contribution of tokens to the slot.

[0034] Preferably, the MoE expert layer outputs a contribution evaluation module, including:

[0035] The unit for calculating the average value of each column of the combined weight calculates the average value of the relative weights of each expert in the MoE expert layer for different output tokens, and obtains the average value of each column of the combined weight;

[0036] The combined weight mean ranking unit compares and ranks the average values ​​of each column of the combined weight according to the average values ​​of each column of the combined weight, and obtains the ranking of the average values ​​of each column of the combined weight;

[0037] The expert output importance analysis unit sorts the average values ​​of each column according to the combined weights, counts the importance of outputs of different experts in the MoE expert layer, and evaluates the output contribution of experts in the MoE expert layer.

[0038] Preferably, the slot token quantity evaluation module includes:

[0039] The unit for counting the number of tokens with cumulative weights counts the number of tokens that reach the set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model;

[0040] The token weight distribution difference measurement unit measures the token weight distribution differences of different slots based on the number of tokens that reach the set cumulative weight, and evaluates the number of tokens in each slot.

[0041] Preferably, the combined weight tendency evaluation configuration module includes:

[0042] The patch inverse proportional shadow processing unit calculates the linear combination weights of different slots and performs inverse proportional shadow processing on the input patch according to the linear combination weights of different slots;

[0043] The attention image region tendency analysis unit visually highlights the attention image region tendency of different slots based on the inverse proportional shading processing; based on the attention image region tendency, it provides an intuitive image of the average input token of the corresponding slot for visual evaluation of expert attention;

[0044] The intelligent optimization unit of the expert configuration mechanism optimizes the expert configuration in the model based on the evaluation of tokens' contribution to slots, the evaluation of MoE expert layer's expert output contribution, the evaluation of the number of tokens in each slot, and the visualization evaluation of expert attention, and visualizes the internal working mechanism of the basic pathology hybrid expert model to obtain the configuration-optimized pathology hybrid expert model.

[0045] The beneficial effects of the above technical solution include:

[0046] The present invention provides an expert behavior analysis method and system based on a pathology hybrid expert model. The method comprises the following steps: calculating the sum of each row of weights assigned by the basic pathology hybrid expert model, obtaining the sum of the relative weights of each input token for all slots, and statistically analyzing the distribution shape of the input token's contribution to the slot, and evaluating the contribution of Tokens to the slot; calculating the average relative weight of each expert in the MoE expert layer for different output tokens, comparing and sorting the average values ​​of each column of the combined weights, statistically analyzing the importance of different experts in the MoE expert layer for the output, and evaluating the output contribution of experts in the MoE expert layer; statistically analyzing the number of tokens that reach the set cumulative weight in the linear combination of the input tokens of the basic pathology hybrid expert model, measuring the difference in the distribution of token weights of different slots, and evaluating the number of tokens in each slot; calculating the linear combination weights of different slots, performing inversely proportional shading processing on the input patch, visually highlighting the tendency of the image area of ​​interest of different slots, and visually evaluating the expert's attention; optimizing the expert configuration in the model, and obtaining the configuration-optimized pathology hybrid expert model; the present invention designs a method based on a pathology hybrid expert model. The expert behavior analysis method and system are provided. By studying and evaluating the contribution and decision-making process of each expert in the model, this analysis improves the internal working mechanism of the model, optimizes the expert configuration, and improves the overall model performance. The expert behavior analysis of the hybrid expert model has significant advantages over the traditional black box neural network, especially in terms of model transparency, performance optimization, resource utilization efficiency and model interpretability. These advantages are of great significance for understanding and improving complex neural network models, especially in key areas such as medical diagnosis. The present invention solves the following technical problems, including: Interpretability: The neural network interpretation method based on the heat map greatly improves the interpretability and applicability of complex pathological image data, obtains more detailed, transparent and practical interpretations, and provides more reliable support and assistance for pathological diagnosis. Resource utilization: quantitatively analyzes the distribution of data among experts, determines whether the experts are overloaded or idle, and significantly improves the uniformity and efficiency of resource allocation. The present invention proposes an expert behavior analysis method based on a hybrid pathological expert model. The method uses the contribution and decision-making process of each expert in the model for research and evaluation, improves the internal working mechanism of the model, optimizes the expert configuration, and improves the overall model performance.

[0047] The present invention provides an expert behavior analysis method based on a pathological hybrid expert model, by studying and evaluating the contribution and decision-making process of each expert in the model; the expert selection mechanism mainly includes two aspects: a gating network and a routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for determining which experts are activated when processing specific inputs. The gating network uses a routing strategy of soft routing, and each expert receives part of the data; the gating network accepts an input token sequence, where and represent the number and dimension of the token respectively, each MoE layer has an expert, the input and output dimensions of the expert network are the same, each expert corresponds to P slots, and each expert processes the token in the corresponding slot separately ; The input token sequence forms the expert network input by assigning weights, and different expert outputs generate output token sequences by combining weights; Based on the assigned weights and combined weights of each MoE layer, this patent provides an evaluation method for expert behavior analysis of a pathology hybrid expert model; The beneficial effects of the technical solution of the present invention also include: Understanding the model decision-making mechanism: Expert behavior analysis helps pathologists and researchers understand how the model makes decisions, which experts are activated in specific pathology tasks, and their respective contributions, which helps to improve the transparency of the model's work and enhance trust in AI diagnostic results; Improving model scalability: Through expert behavior analysis, the performance of each expert can be identified and optimized to make them perform better on specific tasks. Based on the results of behavioral analysis, the expert configuration and weight distribution can be dynamically adjusted to adapt to changes in data distribution and different task requirements, thereby improving the performance and flexibility of the overall model. Improve resource utilization efficiency: In the field of pathology, processing high-resolution pathology slice images requires a lot of computing resources. Expert behavior analysis helps optimize resource allocation, ensuring that data is evenly distributed among experts, avoiding overload or idleness of some experts, and improving the computing efficiency and resource utilization of the model. Improve diagnostic accuracy: Expert behavior analysis can discover and resolve potential errors and deficiencies in the model. By adjusting and optimizing the behavior of experts, the model can maintain high accuracy on different types of pathology data, thereby improving the reliability of diagnosis.

[0048] The present invention provides an expert behavior analysis method and system based on a pathological hybrid expert model. Other advantages, objectives and features of the present invention will be reflected in part through the following description, and in part will be understood by technicians in this field through research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1This is a diagram of an embodiment of an expert behavior analysis system based on a pathological hybrid expert model of the present invention.

[0051] Figure 2 This is another embodiment diagram of an expert behavior analysis system based on a pathological hybrid expert model of the present invention.

[0052] Figure 3 This is a partial implementation diagram of an expert behavior analysis method and system based on a pathological hybrid expert model of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments so that those skilled in the art can implement the invention according to the instructions. Figure 1-Figure 3 As shown, the present invention provides an expert behavior analysis method based on a pathological hybrid expert model, comprising:

[0054] S100, calculate the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtain the sum of the relative weights of each input token for all slots, count the distribution shape of the input token's contribution to the slot, and evaluate the contribution of tokens to the slot;

[0055] S200, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, comparing and sorting the average values ​​of each column of the combined weights, counting the importance of different experts in the MoE expert layer for the output, and evaluating the output contribution of experts in the MoE expert layer;

[0056] S300, in the linear combination of input tokens of the basic pathology hybrid expert model, count the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots, and evaluate the number of tokens in each slot;

[0057] S400 calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the focus image area tendencies of different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model and obtains a configuration-optimized pathology hybrid expert model.

[0058] The principle and effect of the above technical solution are as follows: the present invention provides an expert behavior analysis method based on a pathology hybrid expert model, including: calculating the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtaining the sum of the relative weights of each input token for all slots, counting the distribution shape of the input token's contribution to the slot, and evaluating the contribution of Tokens to the slot; calculating the average relative weight of each expert in the MoE expert layer for different output tokens, comparing and sorting the average values ​​of each column of the combined weights, counting the importance of different experts in the MoE expert layer for the output, and evaluating the output contribution of experts in the MoE expert layer; in the linear combination of the input tokens of the basic pathology hybrid expert model, counting the number of tokens that reach the set cumulative weight, measuring the difference in the distribution of token weights of different slots, and evaluating the number of tokens in each slot; by calculating the linear combination weights of different slots, performing inversely proportional shading processing on the input patch, visually highlighting the tendency of the image area of ​​interest of different slots, and performing a visual evaluation of expert attention; optimizing the expert configuration in the model, and obtaining the configuration-optimized pathology hybrid expert model; the present invention designs a method based on pathology The expert behavior analysis method and system of the hybrid expert model, by studying and evaluating the contribution and decision-making process of each expert in the model, this analysis improves the internal working mechanism of the model, optimizes the expert configuration, and improves the overall model performance; the expert behavior analysis of the hybrid expert model has significant advantages over the traditional black box neural network, especially in terms of model transparency, performance optimization, resource utilization efficiency and model interpretability. These advantages are of great significance for understanding and improving complex neural network models, especially in key areas such as medical diagnosis; the present invention solves the following technical problems including: Interpretability: The neural network interpretation method based on the heat map greatly improves the interpretability and applicability of complex pathological image data, obtains more detailed, transparent and practical interpretations, and provides more reliable support and assistance for pathological diagnosis; Resource utilization: quantitatively analyzes the distribution of data among experts, determines whether the experts are overloaded or idle, and significantly improves the uniformity and efficiency of resource allocation; the present invention proposes an expert behavior analysis method based on a hybrid pathological expert model, which uses the contribution and decision-making process of each expert in the model for research and evaluation, improves the internal working mechanism of the model, optimizes the expert configuration, and improves the overall model performance;

[0059] The present invention provides an expert behavior analysis method based on a pathological hybrid expert model, by studying and evaluating the contribution and decision-making process of each expert in the model; the expert selection mechanism mainly includes two aspects: a gating network and a routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for determining which experts are activated when processing specific inputs. The gating network uses a routing strategy of soft routing, and each expert receives part of the data; the gating network accepts an input token sequence X∈A m×d , where m and d represent the number and dimension of tokens respectively. Each MoE layer has n experts, and the expert network input and output The dimensions are the same, each expert corresponds to p slots; p is 1 or more; each expert processes the token in the corresponding slot separately; the gating network parameters are expressed as Φ∈B d×(n·p) , XΦ calculates the allocation size of each token in the input tokens for different experts, and the allocation weight D performs softmax on each column of XΦ, indicating the relative score or probability of a specific token being assigned to a certain slot of a certain expert considering all tokens. The combined weight C performs softmax on each row of XΦ, indicating the weight of each expert output in the output token; the input token sequence X forms the expert network input by assigning weights Output from different experts Generate output token sequence by combining weights C Based on the distribution weight D and the combination weight C of each MoE layer; the present invention provides an evaluation method for expert behavior analysis of a pathological hybrid expert model;

[0060] The present invention provides an expert behavior analysis method based on a pathological hybrid expert model, by studying and evaluating the contribution and decision-making process of each expert in the model; the expert selection mechanism mainly includes two aspects: a gating network and a routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for deciding which experts are activated when processing specific inputs. The gating network uses a routing strategy of soft routing, and each expert receives part of the data;

[0061] The hybrid expert model architecture can expand the model capacity without significantly increasing the training or reasoning costs by introducing sparsity design. The hybrid expert model usually uses a sparse MoE layer containing several experts to replace some layers in the dense model. Each expert is an independent neural network, and the gated network or routing determines which token is sent to which expert for processing. A significant advantage of the hybrid expert model is that it can be effectively pre-trained with far fewer computing resources than the dense model, that is, under the same computing budget, the hybrid expert model can significantly expand the scale of the model or data set. Especially in the pre-training stage, compared with the dense model, the hybrid expert model can usually reach the same quality level faster. For the medical field, especially the pathology field, the hybrid expert model has many significant advantages. The amount of data in the pathology field is usually very large, including high-resolution pathology slice images. The sparsity design of the hybrid expert model can more effectively process and analyze these massive data, improve the training and reasoning efficiency of the model, and by integrating the analysis results of multiple experts, the hybrid expert model can provide more accurate and reliable diagnosis, and can dynamically adjust the configuration of experts through different pathology tasks and data, improve the adaptability and flexibility of the model, and meet the needs of different pathology diagnosis scenarios.

[0062] The beneficial effects of the technical solution of the present invention also include: understanding the model decision-making mechanism: expert behavior analysis helps pathologists and researchers understand how the model makes decisions, which experts are activated in specific pathological tasks, and their respective contributions, which helps to improve the transparency of the model's work and enhance trust in AI diagnostic results; improving model scalability: through expert behavior analysis, the performance of each expert can be identified and optimized to perform better on specific tasks. According to the results of behavioral analysis, expert configuration and weight distribution can be dynamically adjusted to adapt to changes in data distribution and different task requirements, improving the performance and flexibility of the overall model; improving resource utilization efficiency: in the field of pathology, processing high-resolution pathological slice images requires a lot of computing resources; expert behavior analysis helps optimize resource allocation, ensure that data is balanced among experts, avoid overloading or idleness of some experts, and improve the computing efficiency and resource utilization of the model; improve diagnostic accuracy: expert behavior analysis can discover and solve potential errors and deficiencies in the model; by adjusting and optimizing the behavior of experts, the model can maintain high accuracy on different types of pathological data, thereby improving the reliability of diagnosis.

[0063] In one embodiment, S100 includes:

[0064] S101, creating a basic pathology mixed expert model and calculating the sum of each row of the assigned weights;

[0065] S102, according to the sum of each row of the assigned weights, the sum of the relative weights of each input token to all slots is obtained, and the total contribution of the input token to the slot is obtained;

[0066] S103, according to the total amount of input token contribution to the slot, count the distribution shape of the input token contribution to the slot, and evaluate the contribution of Tokens to the slot.

[0067] The principle and effect of the above technical solution are as follows: create a basic pathology mixed expert model, calculate the sum of each row of the assigned weights; according to the sum of each row of the assigned weights, obtain the sum of the relative weights of each input token for all slots, and obtain the total contribution of the input token to the slot; according to the total contribution of the input token to the slot, count the distribution shape of the contribution of the input token to the slot, and evaluate the contribution of Tokens to the slot; according to the total contribution of the input token to the slot, count the distribution shape of the contribution of the input token to the slot, and evaluate the contribution of Tokens to the slot, including: assigning weights to each row to represent the relative weight of each token in the input token sequence corresponding to different slots Weights; When calculating the tokens in each slot, each token has a certain weight. If the weights of some tokens are much lower than other tokens, then their contribution to the slot is very small; According to the total contribution of the input token to the slot, the distribution shape of the input token's contribution to the slot is counted, and the contribution of tokens to the slot is evaluated; Each row of the weight distribution D represents the relative weight of each token in the input token sequence corresponding to different slots. Although each token has a certain weight when calculating the tokens in each slot, if the weights of some tokens are much lower than other tokens, then their contribution to the slot is very small. Therefore, by calculating the sum of each row of the weight distribution D, the sum of the relative weights of each input token for all slots, that is, the total contribution, can be obtained, so that the distribution shape of the input token's contribution to the slot can be counted.

[0068] In one embodiment, S200 includes:

[0069] S201, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, and obtaining the average value of each column of the combined weight;

[0070] S202, comparing and sorting the average values ​​of each column of the combined weights according to the average values ​​of each column of the combined weights, and obtaining the sorting of the average values ​​of each column of the combined weights;

[0071] S203, sorting according to the average value of each column of the combined weight, counting the importance of different experts in the MoE expert layer to the output, and evaluating the output contribution of experts in the MoE expert layer.

[0072] The principle and effect of the above technical solution are: calculate the average relative weight of each expert in the MoE expert layer for different output tokens, and obtain the average value of each column of the combined weights; compare and sort the average values ​​of each column of the combined weights according to the average value of each column of the combined weights, and obtain the sorting of the average values ​​of each column of the combined weights; sort the average values ​​of each column of the combined weights, and count the importance of different experts in the MoE expert layer for the output, and evaluate the output contribution of experts in the MoE expert layer; each column of the combined weights represents the relative weight of each expert corresponding to different output tokens. Therefore, by calculating the average value of the relative weights of each expert for different output tokens, that is, calculating the average value of each column of the combined weights, and by comparing the average values ​​of different columns, the importance of different experts in the MoE expert layer for the output is counted, and the output contribution of experts in the MoE expert layer is evaluated.

[0073] In one embodiment, S300 includes:

[0074] S301, counting the number of tokens that reach a set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model;

[0075] S302, based on the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots and evaluate the number of tokens in each slot.

[0076] The principle and effect of the above technical solution are as follows: in the linear combination of input tokens of the basic pathology hybrid expert model, the number of tokens that reach the set cumulative weight is counted; according to the number of tokens that reach the set cumulative weight, the difference in the distribution of token weights of different slots is measured, and the number of tokens in each slot is evaluated; for different slots, it is also possible to count how many tokens are needed in the linear combination of its input tokens to reach a certain cumulative weight. By counting the number of tokens required for different cumulative weights, the difference in the distribution of token weights of different slots can be measured, and the number of tokens in each slot can be evaluated.

[0077] In one embodiment, S400 includes:

[0078] S401, by calculating the linear combination weights of different slots, the input patch is shadowed in inverse proportion according to the linear combination weights of the different slots;

[0079] S402, visually highlighting the attention image area tendency of different slots according to the inverse proportional shading processing; providing an intuitive image of the average input token of the corresponding slot according to the attention image area tendency, and performing visual evaluation of expert attention;

[0080] S403, based on the evaluation of the contribution of tokens to the slots, the evaluation of the contribution of the expert outputs of the MoE expert layer, the evaluation of the number of tokens in each slot and the visualization evaluation of the expert attention, optimize the expert configuration in the model, visualize the internal working mechanism of the basic pathology hybrid expert model, and obtain the configuration-optimized pathology hybrid expert model.

[0081] The principle and effect of the above technical solution are as follows: by calculating the linear combination weights of different slots, the input patch is shaded inversely according to the linear combination weights of different slots; based on the inverse proportional shading, the focus image area tendency of different slots is visually highlighted; based on the focus image area tendency, an intuitive image of the average input token of the corresponding slot is provided to perform a visual evaluation of expert attention; based on the slot contribution evaluation of Tokens, the expert output contribution evaluation of the MoE expert layer, the number of tokens in each slot and the visual evaluation of expert attention, the expert configuration in the model is optimized, the internal working mechanism of the basic pathology hybrid expert model is visualized, and the configuration optimization of the pathology hybrid expert model is obtained; the weight allocation provides some more intuitive information about An intuitive image of how slots average input tokens. By calculating the linear combination weights of different slots and shading the input patch in inverse proportion to the weights, it can highlight how different slots tend to focus on areas of the image, and visualize expert attention. The expert selection mechanism mainly includes two aspects: the gating network and routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for deciding which experts are activated when processing specific inputs. The gating network uses a soft routing routing strategy, and each expert receives part of the data. The data allocation weights of the expert layer are calculated, and the contributions and decision-making processes of each expert in the model are studied and evaluated. The internal working mechanism of the model is enhanced, the expert configuration is optimized, and the overall model performance is improved.

[0082] The present invention provides an expert behavior analysis system based on a pathological hybrid expert model, comprising:

[0083] The Tokens slot contribution evaluation module calculates the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtains the sum of the relative weights of each input token for all slots, counts the distribution shape of the input token's contribution to the slot, and evaluates the contribution of Tokens to the slot;

[0084] The MoE expert layer output contribution evaluation module calculates the relative weight average of each expert in the MoE expert layer for different output tokens, compares and sorts the average values ​​of each column of the combined weight, and counts the importance of different experts in the MoE expert layer for the output, and evaluates the output contribution of experts in the MoE expert layer;

[0085] The slot token quantity evaluation module counts the number of tokens that reach the set cumulative weight in the linear combination of the input tokens of the basic pathology hybrid expert model, measures the difference in token weight distribution of different slots, and evaluates the number of tokens in each slot;

[0086] The combined weight tendency evaluation configuration module calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the tendency of the image area of ​​interest in different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model to obtain a configuration-optimized pathology hybrid expert model.

[0087] The principle and effect of the above technical solution are as follows: the present invention provides an expert behavior analysis system based on a pathology hybrid expert model, including: a Tokens slot contribution evaluation module, which calculates the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtains the sum of the relative weights of each input token for all slots, counts the distribution shape of the input token's contribution to the slot, and evaluates the contribution of Tokens to the slot; a MoE expert layer output contribution evaluation module, which calculates the average relative weight of each expert in the MoE expert layer for different output tokens, compares and sorts the average values ​​of each column of the combined weights, counts the importance of different experts in the MoE expert layer for the output, and evaluates the output contribution of MoE expert layer experts; a slot token quantity evaluation module, which counts the number of tokens that reach the set cumulative weight in the linear combination of the input tokens of the basic pathology hybrid expert model, measures the difference in the distribution of token weights of different slots, and evaluates the number of tokens in each slot; a combined weight tendency evaluation configuration module, which calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the tendency of the image area of ​​interest of different slots, and performs visual evaluation of expert attention; an optimization model for expert allocation The invention designs an expert behavior analysis method and system based on a hybrid pathology expert model. By studying and evaluating the contribution and decision-making process of each expert in the model, this analysis improves the internal working mechanism of the model, optimizes the expert configuration, and improves the overall model performance. The expert behavior analysis of the hybrid expert model has significant advantages over the traditional black box neural network, especially in terms of model transparency, performance optimization, resource utilization efficiency and model interpretability. These advantages are of great significance for understanding and improving complex neural network models, especially in key areas such as medical diagnosis. The invention solves the following technical problems, including: Interpretability: The neural network interpretation method based on the heat map greatly improves the interpretability and applicability of complex pathology image data, obtains more detailed, transparent and practical interpretations, and provides more reliable support and assistance for pathology diagnosis. Resource utilization: quantitatively analyzes the distribution of data among experts, determines whether the experts are overloaded or idle, and significantly improves the uniformity and efficiency of resource allocation. The invention proposes an expert behavior analysis method based on a hybrid pathology expert model. The method uses the contribution and decision-making process of each expert in the model to study and evaluate, improve the internal working mechanism of the model, optimize the expert configuration, and improve the overall model performance.

[0088] The present invention provides an expert behavior analysis system based on a pathological hybrid expert model, which studies and evaluates the contribution and decision-making process of each expert in the model; the expert selection mechanism mainly includes two aspects: a gating network and a routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for determining which experts are activated when processing specific inputs. The gating network uses a routing strategy of soft routing, and each expert receives part of the data; the gating network accepts an input token sequence X∈A m×d , where m and d represent the number and dimension of tokens respectively. Each MoE layer has n experts, and the expert network input and output The dimensions are the same, each expert corresponds to p slots; p is 1 or more; each expert processes the token in the corresponding slot separately; the gating network parameters are expressed as Φ∈B d×(n·p) , XΦ calculates the allocation size of each token in the input tokens for different experts, and the allocation weight D performs softmax on each column of XΦ, indicating the relative score or probability of a specific token being assigned to a certain slot of a certain expert considering all tokens. The combined weight C performs softmax on each row of XΦ, indicating the weight of each expert output in the output token; the input token sequence X forms the expert network input by assigning weights Output from different experts Generate output token sequence by combining weights C Based on the distribution weight D and the combination weight C of each MoE layer; the present invention provides an evaluation system for expert behavior analysis of a pathological hybrid expert model;

[0089] The present invention provides an expert behavior analysis system based on a pathological hybrid expert model, by studying and evaluating the contribution and decision-making process of each expert in the model; the expert selection mechanism mainly includes two aspects: a gating network and a routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for deciding which experts are activated when processing specific inputs. The gating network uses a routing strategy of soft routing, and each expert receives part of the data;

[0090] The hybrid expert model architecture can expand the model capacity without significantly increasing the training or reasoning costs by introducing sparsity design. The hybrid expert model usually uses a sparse MoE layer containing several experts to replace some layers in the dense model. Each expert is an independent neural network, and the gated network or routing determines which token is sent to which expert for processing. A significant advantage of the hybrid expert model is that it can be effectively pre-trained with far fewer computing resources than the dense model, that is, under the same computing budget, the hybrid expert model can significantly expand the scale of the model or data set. Especially in the pre-training stage, compared with the dense model, the hybrid expert model can usually reach the same quality level faster. For the medical field, especially the pathology field, the hybrid expert model has many significant advantages. The amount of data in the pathology field is usually very large, including high-resolution pathology slice images. The sparsity design of the hybrid expert model can more effectively process and analyze these massive data, improve the training and reasoning efficiency of the model, and by integrating the analysis results of multiple experts, the hybrid expert model can provide more accurate and reliable diagnosis, and can dynamically adjust the configuration of experts through different pathology tasks and data, improve the adaptability and flexibility of the model, and meet the needs of different pathology diagnosis scenarios.

[0091] The beneficial effects of the technical solution of the present invention also include: understanding the model decision-making mechanism: expert behavior analysis helps pathologists and researchers understand how the model makes decisions, which experts are activated in specific pathological tasks, and their respective contributions, which helps to improve the transparency of the model's work and enhance trust in AI diagnostic results; improving model scalability: through expert behavior analysis, the performance of each expert can be identified and optimized to perform better on specific tasks. According to the results of behavioral analysis, expert configuration and weight distribution can be dynamically adjusted to adapt to changes in data distribution and different task requirements, improving the performance and flexibility of the overall model; improving resource utilization efficiency: in the field of pathology, processing high-resolution pathological slice images requires a lot of computing resources; expert behavior analysis helps optimize resource allocation, ensure that data is balanced among experts, avoid overloading or idleness of some experts, and improve the computing efficiency and resource utilization of the model; improve diagnostic accuracy: expert behavior analysis can discover and solve potential errors and deficiencies in the model; by adjusting and optimizing the behavior of experts, the model can maintain high accuracy on different types of pathological data, thereby improving the reliability of diagnosis.

[0092] In one embodiment, the Tokens slot contribution evaluation module includes:

[0093] Assign weight row calculation unit, create basic pathology mixed expert model, calculate the sum of each row of assigned weight;

[0094] The total contribution analysis unit of the token slots obtains the sum of the relative weights of each input token for all slots based on the sum of each row of the assigned weights, and obtains the total contribution of the input token to the slots;

[0095] The slot contribution distribution statistics unit counts the distribution shape of the input token's contribution to the slot based on the total amount of input token's contribution to the slot, and evaluates the contribution of tokens to the slot.

[0096] The principle and effect of the above technical solution are as follows: the Tokens slot contribution evaluation module includes: the weight allocation row calculation unit, which creates a basic pathology mixed expert model and calculates the sum of each row of the weight allocation; the token slot contribution total analysis unit, which obtains the relative weight sum of each input token for all slots according to the sum of each row of the weight allocation, and obtains the total contribution of the input token to the slot; the slot contribution distribution statistics unit, which performs Tokens to slot contribution evaluation based on the total contribution of the input token to the slot, and the distribution shape of the contribution of the input token to the slot is counted; the Tokens to slot contribution evaluation includes: the weight allocation table for each row The relative weight of each token in the input token sequence corresponding to different slots is shown; when calculating the tokens in each slot, each token has a certain weight. If the weight of some tokens is much lower than that of other tokens, then their contribution to the slot is very small; according to the total contribution of the input token to the slot, the distribution shape of the input token's contribution to the slot is counted, and the contribution of tokens to the slot is evaluated; each row of the distribution weight D represents the relative weight of each token in the input token sequence corresponding to different slots. Although each token has a certain weight when calculating the tokens in each slot, if the weight of some tokens is much lower than that of other tokens, then their contribution to the slot is very small. Therefore, by calculating the sum of each row of the distribution weight D, the sum of the relative weights of each input token for all slots, that is, the total contribution, can be obtained, so that the distribution shape of the input token's contribution to the slot can be counted.

[0097] In one embodiment, the MoE expert layer outputs a contribution assessment module, including:

[0098] The unit for calculating the average value of each column of the combined weight calculates the average value of the relative weights of each expert in the MoE expert layer for different output tokens, and obtains the average value of each column of the combined weight;

[0099] The combined weight mean ranking unit compares and ranks the average values ​​of each column of the combined weight according to the average values ​​of each column of the combined weight, and obtains the ranking of the average values ​​of each column of the combined weight;

[0100] The expert output importance analysis unit sorts the average values ​​of each column according to the combined weights, counts the importance of outputs of different experts in the MoE expert layer, and evaluates the output contribution of experts in the MoE expert layer.

[0101] The principle and effect of the above technical solution are: the MoE expert layer output contribution evaluation module includes: a combined weight column mean calculation unit, which calculates the average relative weight of each expert in the MoE expert layer for different output tokens, and obtains the average of each column of the combined weight; a combined weight mean sorting unit, which compares and sorts the average of each column of the combined weight according to the average of each column of the combined weight, and obtains the sorting of the average of each column of the combined weight; an expert output importance analysis unit, which sorts the average of each column of the combined weight, and counts the importance of different experts in the MoE expert layer for the output, and evaluates the output contribution of experts in the MoE expert layer; each column of the combined weight represents the relative weight of each expert corresponding to different output tokens. Therefore, by calculating the average of the relative weights of each expert for different output tokens, that is, calculating the average of each column of the combined weight, by comparing the averages of different columns, the importance of different experts in the MoE expert layer for the output is counted, and the output contribution of experts in the MoE expert layer is evaluated.

[0102] In one embodiment, a slot token quantity evaluation module includes:

[0103] The unit for counting the number of tokens with cumulative weights counts the number of tokens that reach the set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model;

[0104] The token weight distribution difference measurement unit measures the token weight distribution differences of different slots based on the number of tokens that reach the set cumulative weight, and evaluates the number of tokens in each slot.

[0105] The principle and effect of the above technical solution are: a slot token quantity evaluation module, including: a cumulative weight token quantity statistical unit, which counts the number of tokens that reach the set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model; a token weight distribution difference measurement unit, which measures the token weight distribution difference of different slots according to the number of tokens that reach the set cumulative weight, and evaluates the number of tokens in each slot; for different slots, it is also possible to count how many tokens are needed in the linear combination of its input tokens to reach a certain cumulative weight. By counting the number of tokens required for different cumulative weights, the token weight distribution difference of different slots can be measured, and the token quantity of each slot can be evaluated.

[0106] In one embodiment, the combined weight tendency evaluation configuration module includes:

[0107] The patch inverse proportional shadow processing unit calculates the linear combination weights of different slots and performs inverse proportional shadow processing on the input patch according to the linear combination weights of different slots;

[0108] The attention image region tendency analysis unit visually highlights the attention image region tendency of different slots based on the inverse proportional shading processing; based on the attention image region tendency, it provides an intuitive image of the average input token of the corresponding slot for visual evaluation of expert attention;

[0109] The intelligent optimization unit of the expert configuration mechanism optimizes the expert configuration in the model based on the evaluation of tokens' contribution to slots, the evaluation of MoE expert layer's expert output contribution, the evaluation of the number of tokens in each slot, and the visualization evaluation of expert attention, and visualizes the internal working mechanism of the basic pathology hybrid expert model to obtain the configuration-optimized pathology hybrid expert model.

[0110] The principle and effect of the above technical solution are as follows: the combined weight tendency evaluation configuration module includes: a patch inverse proportional shadow processing unit, which calculates the linear combination weights of different slots and performs inverse proportional shadow processing on the input patch according to the linear combination weights of different slots; an image area tendency analysis unit, which visually highlights the image area tendency of different slots according to the inverse proportional shadow processing; provides an intuitive image of the average input token of the corresponding slot according to the image area tendency, and performs visual evaluation of expert attention; an expert configuration mechanism intelligent optimization unit, which optimizes the expert configuration in the model according to the slot contribution evaluation of Tokens, the expert output contribution evaluation of the MoE expert layer, the token number evaluation of each slot and the expert attention visualization evaluation, and visualizes the internal working mechanism of the basic pathology hybrid expert model. Obtain configuration optimization for the pathological hybrid expert model; assign weights to provide some more intuitive images of how slots average input tokens. By calculating the linear combination weights of different slots and shading the input patch in inverse proportion to the weights, it is possible to highlight how different slots tend to focus on areas of the image, and visualize expert attention; the expert selection mechanism mainly includes two aspects: the gating network and routing decision. The gating network is a linear layer that assigns weights to each expert based on the characteristics of the input data and is responsible for deciding which experts are activated when processing specific inputs. The gating network uses a soft routing routing strategy, and each expert receives part of the data; calculate the data allocation weights of the expert layer, and study and evaluate the contributions and decision-making processes of each expert in the model; explicitly enhance the internal working mechanism of the model, optimize the expert configuration, and improve the overall model performance.

[0111] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. An expert behavior analysis method based on pathological hybrid expert model, It is characterized in that include: S100, calculate the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtain the sum of the relative weights of each input token for all slots, count the distribution shape of the input token's contribution to the slot, and evaluate the contribution of tokens to the slot; S200, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, comparing and sorting the average values ​​of each column of the combined weights, counting the importance of different experts in the MoE expert layer for the output, and evaluating the output contribution of experts in the MoE expert layer; S300, in the linear combination of input tokens of the basic pathology hybrid expert model, count the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots, and evaluate the number of tokens in each slot; S400 calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the focus image area tendencies of different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model and obtains a configuration-optimized pathology hybrid expert model.

2. The expert behavior analysis method based on the pathological hybrid expert model according to claim 1, It is characterized in that S100 includes: S101, creating a basic pathology mixed expert model and calculating the sum of each row of the assigned weights; S102, according to the sum of each row of the assigned weights, the sum of the relative weights of each input token to all slots is obtained, and the total contribution of the input token to the slot is obtained; S103, according to the total amount of input token contribution to the slot, count the distribution shape of the input token contribution to the slot, and evaluate the contribution of Tokens to the slot.

3. The expert behavior analysis method based on the pathological hybrid expert model according to claim 1, It is characterized in that S200 includes: S201, calculating the average relative weight of each expert in the MoE expert layer for different output tokens, and obtaining the average value of each column of the combined weight; S202, comparing and sorting the average values ​​of each column of the combined weights according to the average values ​​of each column of the combined weights, and obtaining the sorting of the average values ​​of each column of the combined weights; S203, sorting according to the average value of each column of the combined weight, counting the importance of different experts in the MoE expert layer to the output, and evaluating the output contribution of experts in the MoE expert layer.

4. The expert behavior analysis method based on the pathological hybrid expert model according to claim 1, It is characterized in that S300 includes: S301, counting the number of tokens that reach a set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model; S302, based on the number of tokens that reach the set cumulative weight, measure the difference in token weight distribution of different slots and evaluate the number of tokens in each slot.

5. The expert behavior analysis method based on the pathological hybrid expert model according to claim 1, It is characterized in that S400 includes: S401, by calculating the linear combination weights of different slots, the input patch is shadowed in inverse proportion according to the linear combination weights of the different slots; S402, visually highlighting the attention image area tendency of different slots according to the inverse proportional shading processing; providing an intuitive image of the average input token of the corresponding slot according to the attention image area tendency, and performing visual evaluation of expert attention; S403, based on the evaluation of the contribution of tokens to slots, the evaluation of the contribution of MoE expert layer expert outputs, the evaluation of the number of tokens in each slot and the visualization evaluation of expert attention, optimize the expert configuration in the model, visualize the internal working mechanism of the basic pathology hybrid expert model, and obtain the configuration-optimized pathology hybrid expert model.

6. An expert behavior analysis system based on pathological hybrid expert model, It is characterized in that include: The Tokens slot contribution evaluation module calculates the sum of each row of the weights assigned by the basic pathology hybrid expert model, obtains the sum of the relative weights of each input token for all slots, counts the distribution shape of the input token's contribution to the slot, and evaluates the contribution of Tokens to the slot; The MoE expert layer output contribution evaluation module calculates the relative weight average of each expert in the MoE expert layer for different output tokens, compares and sorts the average values ​​of each column of the combined weight, and counts the importance of different experts in the MoE expert layer for the output, and evaluates the output contribution of experts in the MoE expert layer; The slot token quantity evaluation module counts the number of tokens that reach the set cumulative weight in the linear combination of the input tokens of the basic pathology hybrid expert model, measures the difference in token weight distribution of different slots, and evaluates the number of tokens in each slot; The combined weight tendency evaluation configuration module calculates the linear combination weights of different slots, performs inversely proportional shading on the input patch, visually highlights the tendency of the image area of ​​interest in different slots, and performs visual evaluation of expert attention; optimizes the expert configuration in the model to obtain a configuration-optimized pathology hybrid expert model.

7. An expert behavior analysis system based on a pathological hybrid expert model according to claim 6, It is characterized in that Tokens slot contribution evaluation module, including: Assign weight row calculation unit, create basic pathology mixed expert model, calculate the sum of each row of assigned weight; The total contribution analysis unit of the token slots obtains the sum of the relative weights of each input token for all slots based on the sum of each row of the assigned weights, and obtains the total contribution of the input token to the slots; The slot contribution distribution statistics unit counts the distribution shape of the input token's contribution to the slot based on the total amount of input token's contribution to the slot, and evaluates the contribution of tokens to the slot.

8. The expert behavior analysis system based on the pathological hybrid expert model according to claim 6, It is characterized in that The MoE expert layer outputs contribution evaluation modules, including: The unit for calculating the average value of each column of the combined weight calculates the average value of the relative weights of each expert in the MoE expert layer for different output tokens, and obtains the average value of each column of the combined weight; The combined weight mean ranking unit compares and ranks the average values ​​of each column of the combined weight according to the average values ​​of each column of the combined weight, and obtains the ranking of the average values ​​of each column of the combined weight; The expert output importance analysis unit sorts the average values ​​of each column according to the combined weights, counts the importance of outputs of different experts in the MoE expert layer, and evaluates the output contribution of experts in the MoE expert layer.

9. The expert behavior analysis system based on the pathological hybrid expert model according to claim 6, It is characterized in that Slot token quantity evaluation module, including: The unit for counting the number of tokens with cumulative weights counts the number of tokens that reach the set cumulative weight in the linear combination of input tokens of the basic pathology hybrid expert model; The token weight distribution difference measurement unit measures the token weight distribution differences of different slots based on the number of tokens that reach the set cumulative weight, and evaluates the number of tokens in each slot.

10. The expert behavior analysis system based on the pathological hybrid expert model according to claim 6, It is characterized in that The combination weight tendency assessment configuration module includes: The patch inverse proportional shadow processing unit calculates the linear combination weights of different slots and performs inverse proportional shadow processing on the input patch according to the linear combination weights of different slots; The attention image region tendency analysis unit visually highlights the attention image region tendency of different slots based on the inverse proportional shading processing; based on the attention image region tendency, it provides an intuitive image of the average input token of the corresponding slot for visual evaluation of expert attention; The intelligent optimization unit of the expert configuration mechanism optimizes the expert configuration in the model based on the evaluation of tokens' contribution to slots, the evaluation of MoE expert layer's expert output contribution, the evaluation of the number of tokens in each slot, and the visualization evaluation of expert attention, and visualizes the internal working mechanism of the basic pathology hybrid expert model to obtain the configuration-optimized pathology hybrid expert model.

Images