Neural network interpretation optimization method based on text concept representation

Abstract

The invention discloses a neural network interpretation optimization method based on text concept representation, which comprises the following steps of: aiming at a specific data set, firstly, constructing a text concept library covering data set category significant appearance characteristics on the basis of data set information, human priori knowledge and the like, and taking the text concept library as an intermediate domain between input and output of a neural network; the decision basis of the neural network is obtained through mapping in the text concept space, and interpretability of the decision process of the neural network is achieved; meanwhile, human knowledge is introduced through understanding and direct modification of a concept prediction result by human, direct intervention is carried out on a single sample decision result of the neural network, then a verification set is taken as a data basis, human modification logic is transferred to an original neural network by means of a knowledge distillation technology, and correction of self-cognition deviation is realized. According to the method, rich and easy-to-understand model reasoning logic is provided for human beings from the aspect of text concepts; and meanwhile, feedback is provided for the network in combination with rational knowledge of human beings, so that the performance of the network is optimized and improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, and specifically relates to a neural network interpretation optimization method based on text concept representation. Background Technology

[0002] Neural networks, as a powerful artificial intelligence technology, have demonstrated significant advantages in various fields such as security, finance, and medicine, and are widely used in diverse scenarios. Furthermore, with the continuous improvement of computing power, the performance of neural networks is constantly improving, bringing about tremendous changes in many industries. In the future, neural networks are expected to play an even more important role in more fields. Given the breadth of applications for neural networks, ensuring their transparency and reliable human use is crucial. However, current deep neural networks are often regarded as black-box models, providing only the decision-making result without revealing the decision-making process. This leads to a lack of trust in key areas, becoming a significant obstacle to their further development and application. On the other hand, because humans find it difficult to understand the decision-making logic of neural networks, when their decisions are unreasonable, humans cannot easily intervene to provide targeted guidance and correction. Therefore, the interpretability of neural networks and the optimization of neural network performance based on human knowledge remain two key issues in the field of machine learning.

[0003] To address the interpretability issue of neural networks, existing attribution methods can provide explanations by highlighting the regions of interest of the neural network from a pixel perspective, but the explanations are not intuitive and require further human processing. An existing method based on image visual concepts (publication number CN115797498A) can provide intuitive visual explanations from the perspective of image patch features, but the explanation information is not rich enough.

[0004] These methods can be understood and explained by humans to a certain extent, but they still cannot provide an interface for information interaction and knowledge exchange between humans and neural networks.

[0005] Therefore, it is necessary to construct more effective, easy-to-understand methods that can provide direct intervention interfaces for humans. It is essential and meaningful to explore neural network decision interpretation schemes and to explore optimization algorithms that combine human rational knowledge on this basis. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention aims to provide a neural network interpretation and optimization method based on text concept representation. This method uses text concepts as the language and carrier for knowledge exchange between humans and neural networks. By mapping the input features of the neural network to the text concept space, a one-to-one association between text concepts and decision outputs is achieved. This allows the network model's internal decision-making logic to be obtained from the perspective of text concepts, thus achieving the interpretability of the neural network. Simultaneously, it allows humans to directly modify the text concept prediction results of single samples, introducing human knowledge to provide targeted intervention and optimization for the decision output of those single samples. Furthermore, by performing single-sample correction on all erroneous samples on the validation set, and then using the validation set as data as a foundation, knowledge transfer is performed on the classification network using knowledge distillation technology, feeding back the human modification logic into the network. This method offers better network interpretability and transparency, and can be visualized from a human-understandable perspective.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A neural network interpretation optimization method based on text concept representation includes the following steps; S1, Constructing textual concept base information specific to the image dataset: Textual concepts, formally represented as human descriptions of the appearance features of a category based on category information, represent human understanding of the appearance of a specific category, encompassing features such as shape and texture. The purpose of this step is to generate a set of the most representative textual concept libraries for the given target dataset, leveraging prior human knowledge and tools such as large-scale models. This ensures accurate and comprehensive coverage of the important appearance features of all categories in the dataset, and the textual concept results for different categories should exhibit a certain degree of difference and discriminability. Following this, concept labels need to be constructed for each category based on the textual concept results. S2, Training concept probe function: Based on the text concept library and category concept labels constructed in S1, for any classification network pre-trained on the same image dataset, a corresponding concept probe function is constructed and trained. The concept probe function is used to align the activation vector of a specific convolutional layer of the pre-trained classification network with the constructed text concept library space, thereby establishing the association mapping result between the input data, concept prediction score, and final output. S3 provides the neural network decision-making explanation logic from the perspective of textual concepts: Based on the concept probe function trained in S2, for any image data input into the neural network, forward propagation yields the corresponding text concept prediction results and decision output results. Based on the text concept score, the neural network decision output is explained from the perspective of important concept features. S4, Optimization of Single-Sample Decision Results Intervention: Based on the explanations provided in S3, humans can easily understand the decision-making logic of neural networks and identify potential biases, such as over-reliance on concepts unrelated to their category while neglecting category-related concepts. Subsequently, humans directly modify the concept prediction scores, increasing the scores of concepts most relevant to the sample's category and decreasing the scores of concepts less relevant, constructing a concept score vector from a human perspective. This vector is then remapped using the concept probe function in S2 to obtain the activation vector output and decision output, thus transmitting the human modification logic to the neural network to intervene and optimize the decision results for single samples. S5, Knowledge Transfer and Performance Optimization: By utilizing the single-sample optimization method in S4, human conceptual intervention and decision optimization are applied to all samples in the validation set that made erroneous decisions, thereby improving the overall cognitive ability and classification performance of the original neural network on the validation set. Then, using the validation set as the data foundation, a student model with the same architecture and initial parameters as the original neural network is created. Knowledge distillation techniques are used to transfer the reasonable cognition and logic of the classification network on the validation set, while real class labels provide stronger constraints. In this way, without changing the original neural network architecture, the cognitive ability of the neural network is optimized by leveraging prior human knowledge and modified logic, thus truly improving the classification performance of the neural network itself.

[0008] Furthermore, in S1, the construction process of the text visual concept library should differ depending on the characteristics of the dataset; for general image datasets, there are obvious differences in overall appearance between categories, so usable text concepts can be obtained by directly extracting text concepts from the category perspective. For fine-grained image datasets, the similarity between categories is high, and there may only be differences at a certain angle in a certain part. Therefore, we can first divide the categories into parts, and then extract text concepts from the perspective of the parts to finally obtain usable text concept results.

[0009] At the same time, certain principles should be followed when extracting and constructing text concepts in order to obtain representative and distinctive text concept results, including: Distinctiveness: Distinctive language descriptions are required. For example, for an animal dataset that covers different categories, features such as "has teeth," "has claws," and "has wings" may be textual descriptions that distinguish different types of animals and can be used as concept candidates. However, for a bird dataset that only contains different bird categories, descriptions such as "has wings" and "has feathers" should not be used, as they do not have clear distinguishing properties for that dataset. Clarity: It is necessary to avoid using vague descriptions such as "a certain special shape"; Consistency: The sentence structure should adopt a relatively consistent construction method. For example, descriptions with the same meaning but different wording, such as "black wings" and "wings are black," should be avoided. Similarly, textual concepts with similar meanings but different descriptions, such as "light orange," "light yellow," and "yellow-orange," should be avoided.

[0010] Furthermore, in step S2, in order to train the concept probe function, it is necessary to prepare in advance a set of category images. Category tag set and concept tag set The training data, including the class images, is used to extract the corresponding activation vector set from the output of the last convolutional layer of the classification network. Then, using this training data, the probe function is trained through supervised learning. The input to the probe function is image data. Activation vector output in the network The output is the weight or relevance score corresponding to the concept in the concept library.

[0011] Meanwhile, the network structure of the probe function can be chosen in several ways, such as: Linear structure: The structure from the convolutional layer to the output layer at the end of the classification network is usually a simple linear structure. Therefore, a simple linear structure design can also be considered from the output of the convolutional layer to the output of the concept space. Simple nonlinear structures: To better capture the mapping relationship between concepts and decision outputs, concept probes can be designed using simple nonlinear structures.

[0012] Depending on the data characteristics and task requirements, more complex probe function structure designs can also be considered. Therefore, the loss function value for the concept probe function training process is: in, It is the concept prediction loss, specifically the mean squared error between the concept output vector and the concept label vector.

[0013] Furthermore, in S3, after the concept probe function is trained, the input data, concept prediction output, and final decision output can be associated. Then, for the input data, multiple concepts with higher scores are selected based on the text concept score results to provide rich and easy-to-understand neural network decision explanation logic from the perspective of important text concepts: Since the neural network detects these important concept features and their combinations in the input image, and in the network cognition, the existence of these concept features and their combinations is closer to the neural network's understanding of the final decision output category, the classification network identifies it as data of that category.

[0014] Furthermore, in step S4, after the human modifies and obtains the new concept score vector, the corresponding new activation vector will be obtained using the concept probe function. Specifically, based on the new concept score vector obtained through human modification... The corresponding new activation vector will be obtained by minimizing the following values. : in It is a distance function. These are hyperparameters used to control the effectiveness of the intervention. Afterwards... The new output will be obtained through forward propagation in the classification network. At this point, the human modification logic will be transmitted to the neural network, enabling human intervention in the decision-making results of a single sample.

[0015] Furthermore, in step S5, in order to improve the performance of the classification network itself, knowledge distillation technology will be used to transfer and feed back the reasonable cognitive logic of the classification network on the validation set.

[0016] Specifically, first, a network with the same structure and parameters as the original classification network is copied and used as the student model. Then, the validation set after single-sample correction is used. As new training data, samples correctly classified by the original classification network will continue to use the previous output, while samples misclassified by the network will use the output after human modification and optimization. In the knowledge distillation process, the overall objective loss function is: in, and These represent the knowledge distillation process. and The respective weight parameter values ​​of the two items, Let be the soft loss function, representing the cross-entropy between the soft labels and the probability distribution of the student network output, i.e.: This represents the total number of categories that the original neural network focuses on. Represents temperature Below, the teacher model prediction results are of the following types. The probability score, For temperature In the student model prediction results, the class The probability score is in the following form: in, The teacher model represents the first... The original logits output or the logits output after manual intervention for each category. The student model represents the first... The original prediction output for each category. Let be the hard loss function, representing the cross-entropy between the true image label and the probability distribution of the student network's output at temperature 1, i.e.: in, For class The true category label, It is the type of student network prediction output when the temperature is 1. The probability value. By fine-tuning the training on the validation set through distillation, the student network is guided to learn more reasonable category knowledge and human rational logic existing in the validation set data without losing its original category cognition, thereby effectively improving the prediction accuracy and generalization ability of the original classification network.

[0017] Another objective of this invention is to provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the above method steps.

[0018] Another objective of this invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps.

[0019] The beneficial effects of this invention are: (1) This invention constructs a text concept library to describe and summarize the appearance features of categories and data from a human perspective, and constructs a concept probe function to realize the correlation between data features, concept prediction, and decision output, thereby ultimately realizing a concept-level explanation of neural network classification decisions. It provides humans with rich and easy-to-understand internal reasoning logic of neural networks from a high-level text concept perspective, overcomes the problem of lack of interpretability of existing "black box" network models, and surpasses the simple low-level explanation method at the pixel level. This invention has better network interpretability and transparency, and can be visualized from a human-understandable perspective.

[0020] (2) This invention modifies and corrects the text concept extraction results manually, and then remaps them to obtain new prediction outputs. This realizes a neural network decision-making logic intervention mechanism that combines human knowledge, which can conveniently correct and improve the decision interpretation of single samples. Furthermore, by using knowledge distillation technology, reasonable logic existing on the validation set is fed back into the original neural network, which strengthens the neural network's subtle understanding and accurate cognition of the categories of interest, thereby improving the classification accuracy and interpretation effect of the original neural network. In this process, human reasoning ability and prior knowledge are incorporated, realizing targeted knowledge guidance of the neural network model by humans.

[0021] (3) This invention does not modify the structure or size of the original neural network, but uses knowledge distillation technology to inject and guide the neural network with overall knowledge as a tool to improve its performance. Therefore, this invention is not limited by the structural complexity of the original neural network and will not affect the deployment of the original network. Attached Figure Description

[0022] Figure 1 This is an overall framework diagram of the neural network interpretation optimization method based on text concept representation in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram illustrating the text concept library construction method based on a specific dataset in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the neural network interpretability algorithm module method based on text concepts in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram illustrating the process of human intervention and optimization of single-sample decision results in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram illustrating a method for knowledge transfer and fine-tuning optimization using knowledge distillation technology, as described in an embodiment of the present invention.

[0027] Figure 6This is an example diagram showing the simulation results of the neural network interpretability algorithm based on the text concept in an embodiment of the present invention.

[0028] Figure 7 This is an example diagram of the simulation experiment results of human intervention to optimize the decision results of a single sample, according to an embodiment of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings.

[0030] like Figure 1 As shown, the neural network interpretation optimization method based on text concept representation includes two parts: a neural network interpretability algorithm module based on text concept, used to realize the decision interpretation of the neural network; and a neural network performance optimization algorithm module based on distillation fine-tuning, used to realize the targeted performance optimization of the neural network by humans.

[0031] The text-based neural network interpretability algorithm module includes the following steps: Step 1: Based on the image and category information of a specific dataset, and leveraging dataset information, human knowledge, domain data, and large language models, construct a text concept library and category concept labels for the specific dataset. This summarizes, describes, and represents the key and unique appearance attributes of the categories in the dataset. The specific steps are as follows: Step 11, as follows Figure 2 As shown, based on the category information of the dataset, humans first need to extract textual visual concepts from the appearance characteristics of the category, such as color, shape, texture, and part. Simultaneously, due to the varying similarities between categories in the dataset, for fine-grained image classification datasets, humans first need to divide each category into parts, and then describe and form concepts from the perspective of these parts to construct textual concept results with category discrimination. For general coarse-grained image classification datasets, humans can directly describe the appearance and form concepts from the perspective of the overall features and unique attributes of the category. Furthermore, when constructing visual concepts, it is also necessary to ensure the structural consistency, semantic clarity, and category discrimination of the constructed concepts, ensuring that the textual concept results of two different categories are not completely consistent. Finally, the textual concept results obtained for each category are merged into the textual concept library of its respective dataset and deduplicated to obtain the textual concept library results for each specific dataset, forming a concept space.

[0032] Step 12: Next, concept labels need to be constructed for each category and image data based on the text concept library to prepare preliminary data for subsequent training. Specifically, the concept label is represented by a vector of a specific dimension, representing the text concept result corresponding to each image, where the vector length is the number of text concepts in the dataset. Then, the index of all text concepts contained in each category in the text concept library is located, and the element value at the corresponding position in the concept label vector is initialized to 1, while the element values ​​at other positions are initialized to 0. Following this method, concept label information can be constructed for all category data.

[0033] Step 2: To associate and map the textual concept results with the neural network decision output, a concept probe function needs to be trained. Specifically, this involves training a pre-trained base classification model and a set of images including category images. Category tag set and concept tag set The training set data, including the image data, is used to obtain the activation vectors of the last convolutional layer of the classification network through forward propagation, resulting in the output set of the classification network. Then, using this training data, the probe function is tested. In supervised training, the input to the probe function is the activation vector of the image data, and the output is the probe function. The output is the concept importance prediction score corresponding to the concepts in the text concept corpus. Therefore, the loss function value for the concept probe function training process will be designed as follows: in, It is the concept prediction loss, which represents the mean squared error between the concept label and the concept prediction output vector.

[0034] Meanwhile, the network structure of the probe function can have a variety of possible choices. For example, considering that the structure from the convolutional layer to the output layer at the end of a classification network is usually a simple linear structure, the concept probe from the output of the convolutional layer to the output of the concept space can also be designed with a simple linear structure. In addition, in order to better capture the mapping relationship between concepts and decision outputs, a non-linear structure design of an MLP with a hidden layer can also be considered for the concept probe. Furthermore, more complex probe function structure designs can be considered based on data characteristics and task requirements.

[0035] Step 3, as follows Figure 3As shown, after the concept probe function is trained, a one-to-one association mapping can be established between the input data, concept prediction output, and final decision output. Then, for any input data, several concepts with higher scores are selected based on the text concept scores, providing a rich and easily understandable neural network decision explanation logic from the perspective of text concepts: because the neural network detects these important concepts and their combinations in the input image, and because this is highly similar to the neural network's understanding of the output category, the network outputs it as that category.

[0036] Step 4: Based on the explanations provided by the neural network, humans can understand the internal reasoning logic of the neural network and discover potential biases. Then, through direct concept score modification and forward propagation, human prior knowledge and modification logic are transferred back to the original neural network, thereby influencing the neural network's decision output for a single sample. Figure 4 As shown, the specific steps are as follows: Step 41: Humans observe and directly modify the concept score results of the erroneous samples based on the category concept labels, improve the scores of concepts related to their category, and reduce the scores of concepts unrelated to their category, thus constructing a concept score vector result from the human perspective.

[0037] Step 42, modify to obtain a new concept score vector Then, the concept probe function is used to obtain the corresponding new activation vector output by minimizing the following value. : in It is a distance function. These are hyperparameters used to control the effectiveness of the intervention. Afterwards, Forward propagation will be performed in the classification network to obtain new output results. This allows the human-made modification logic to be transmitted back to the original neural network, affecting its reasoning process and decision-making results for single-sample input data.

[0038] Step 5, further, by applying the above-mentioned conceptual intervention and decision optimization process to all erroneous samples in the validation set, the overall performance and category recognition of the neural network on the validation set are first improved. Then, based on this data, knowledge distillation technology is used to transfer this rational cognition back to the original neural network, thereby optimizing and improving its own classification performance. The specific steps are as follows: Step 51: For all erroneous samples in the validation set, humans modify and optimize the concept prediction scores based on their corresponding category labels, and then obtain new decision outputs through concept probes and forward propagation. In this way, direct human knowledge participation and accurate neural network cognition on the validation set are achieved.

[0039] Step 52, as follows Figure 5 As shown, using the validation set data as new training data, a student neural network with the exact same structure and parameters as the original classification network is copied and used for knowledge distillation. Samples correctly classified by the original classification network will continue to use the previous output, while samples misclassified by the original network will use the manually modified and optimized output. The overall objective loss function in the knowledge distillation process is: in, and They represent and The respective weight parameter values ​​of the two items, Let represent the soft loss function, and be the cross-entropy between the soft labels and the probability distribution of the student network output, i.e.: in, This represents the total number of categories that the original neural network focuses on. Represents temperature Below, the teacher model prediction results are of the following types. The probability score, For temperature In the student model prediction results, the class The probability score is in the following form: in, The teacher model represents the first... The original logits output or the logits output after manual intervention for each category. The student model represents the first... The original prediction output for each category. Let be the hard loss function, representing the cross-entropy between the true image label and the probability distribution of the student network's output at temperature 1, i.e.: in, For class Category tags, It is the type of student network prediction output when the temperature is 1. The probability value.

[0040] This invention constructs a textual concept library by combining dataset information, human knowledge, domain data, and large model tools. This library can fully describe and represent human cognition of category appearance features, serving as a language and carrier for information interaction between humans and neural networks. Subsequently, a concept probe function is trained to associate input data, concept results, and decision outputs, ultimately providing humans with rich and easily understandable internal reasoning logic of neural networks from a high-level textual concept perspective.

[0041] This invention provides an interface for humans to directly modify the textual concept prediction results of single-sample data, enabling direct participation of human knowledge. This, in turn, yields new output results through concept probe mapping and forward propagation, realizing a neural network decision-making intervention and optimization mechanism that incorporates human knowledge. Furthermore, using a validation set as the data foundation and leveraging knowledge distillation technology, it achieves the transfer of human knowledge and optimization of the neural network's performance. In this process, human reasoning ability and prior knowledge are incorporated, enabling targeted knowledge guidance of the model by humans.

[0042] This invention utilizes knowledge distillation technology, thus avoiding modifications to the structure, size, or other characteristics of the original neural network. Instead, it injects and guides the neural network with overall knowledge to improve its performance. This allows the invention to be unrestricted by the structural complexity and deployment method of the original neural network.

[0043] The invention is not limited by the structural complexity and deployment method of the original neural network.

[0044] This invention presents an experimental simulation design and verification of a neural network interpretation optimization method based on text concept representation. The experiment was conducted on the CUB-200-2011 (Caltech-UCSD Birds-200-2011, CUB) dataset. CUB is a widely used dataset in fine-grained visual classification tasks, containing 200 bird categories and a total of 11,788 images, covering a rich variety of bird categories with different body sizes and species.

[0045] To ensure the smooth operation of the experiment, 20% of the data in the training set was first allocated as a validation set, resulting in 4795 images for the training set, 1198 images for the validation set, and 5794 images for the test set. Secondly, the InceptionV3 model was used as the basic architecture. In the initial stage, supervised training was performed using training images and category labels to obtain the initial classification network. Further, the experiment combined dataset information and human knowledge to construct a textual concept library for the CUB dataset, comprising 114 textual concepts covering various appearance features of eight parts of the 200 bird categories in the dataset, including shape, color, and special patterns: beak, eyes, head, wings, chest, abdomen, back, and tail. Then, a multilayer perceptron with one hidden layer was used as the concept probe structure for concept probe training. The input features were associated with the concept space to provide neural network decision explanations from the perspective of textual concepts. Examples of the explanation results are shown below. Figure 6 As shown, the input data in the image was predicted by the original classification network to be classified as "West American Gull". Among the concept prediction results, "white belly", "yellow beak", "black wings", and "spotted tail" scored the highest. Therefore, the following explanation can be given: Since the classification model detected important concepts such as "white belly", "yellow beak", "black wings", and "spotted tail" in the image, and in the model's cognition, the existence and combination of these concepts are closer to the "West American Gull" category, the input image was classified as "West American Gull". Experimental results show that the neural network interpretation optimization method based on text concept representation can provide the network's inference explanation results from the perspective of text concept, and this explanation method can be easily understood by humans.

[0046] Furthermore, once humans understand the decision-making logic of neural networks, they can identify and directly correct any irrationalities within them. Examples of such corrections include... Figure 7As shown in the figure, the classification network misclassified the input data of the "Grey-winged Gull" category as the "Silver Gull" category. Through comparative analysis of these two categories, it was found that the only significant differences in appearance are in eye and tail color. The "Grey-winged Gull" has black eyes and a grey tail, while the "Silver Gull" has yellow eyes and a black tail. Therefore, the human user increased the prediction scores for the concepts "black eyes" and "grey tail" in the concept prediction results, and reduced the prediction scores for the relative concepts "yellow eyes" and "black tail" to 0, resulting in a new concept score vector. The new concept score... Vectors intervened in the final decision-making results through concept probe function mapping and forward propagation of the classification network, improving the network's decision output and ensuring correct classification. Based on this approach, human users intervened in 312 incorrectly classified samples in the CUB dataset validation set, correcting the decisions of 278 samples. Finally, a knowledge distillation fine-tuning experiment was conducted using the validation set data. After knowledge distillation, the overall classification accuracy of the student network on the test set improved, with the most significant optimization effect observed when the teacher model temperature was set to 4, increasing the overall classification accuracy from 70.3% to 80.8%. Experimental results demonstrate that the neural network interpretation optimization method based on text concept representation can combine human interaction knowledge with the text concept interpretation results to specifically correct cognitive biases in the neural network, thereby effectively improving the performance of the classification network itself.

[0047] The aforementioned neural network interpretation optimization method based on text concept representation can be implemented as a computer program, stored on a hard disk, and loaded into a processor for execution to implement the method of the present invention.

[0048] This invention also provides a computer-readable medium storing computer program code, which, when executed by a processor, implements the neural network interpretation optimization method based on text concept representation as described above.

[0049] When a neural network interpretation optimization method based on text concept representation is implemented as a computer program, it can also be stored as an article of manufacture in a computer-readable storage medium. For example, computer-readable storage media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical discs (e.g., compact discs (CDs), digital multifunction discs (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), cards, sticks, key drives). Furthermore, the various storage media described in the embodiments of this invention can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) capable of storing, containing, and / or carrying code and / or instructions and / or data.

[0050] It should be understood that the above embodiments are merely illustrative. The embodiments described in this invention can be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processing unit can be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described in this invention, or combinations thereof.

[0051] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A neural network interpretation optimization method based on text concept representation, characterized in that, Includes the following steps; S1, Construct a text concept library information specific to the image dataset, and construct concept labels for each category based on the text concept results; S2, based on the text concept library and category concept labels, for any classification network pre-trained on the same image dataset, construct and train a corresponding concept probe function. The concept probe function is used to align the activation vector of a specific convolutional layer of the pre-trained classification network with the constructed text concept library space, thereby establishing the association mapping result between the input data, concept prediction score, and final output. S3, based on the concept probe function obtained from the training, for any image data input into the neural network, forward propagation is performed to obtain the corresponding text concept prediction result and decision output result. Based on the text concept score, the explanation basis of the neural network decision output is provided from the perspective of important concept features. S4. Based on the aforementioned explanation, the concept prediction score is directly modified to construct a concept score vector from a human perspective. Then, the activation vector output and decision output are obtained by remapping through the concept probe function. The human modification logic is then passed to the neural network in order to intervene in and optimize the decision results of a single sample. S5 utilizes the single-sample optimization method in S4 to perform human conceptual intervention and decision optimization on all samples in the validation set that made incorrect decisions. Using the validation set as the data foundation, a network with the same architecture and initial parameters as the original neural network is copied as a student model. Knowledge distillation technology is used to transfer the reasonable cognition and logic of the classification network on the validation set. At the same time, the real category labels are used to provide stronger constraints. Human prior knowledge and modified logic are used to optimize the cognitive ability of the neural network.

2. The neural network interpretation optimization method based on text concept representation according to claim 1, characterized in that, In S1, the construction process of the text visual concept library varies depending on the characteristics of the dataset. For general image datasets where there are obvious differences in overall appearance between categories, usable text concepts can be obtained by directly extracting text concepts from the category perspective. For fine-grained image datasets with high similarity, the approach is to first divide the categories into components, and then extract textual concepts from the perspective of the components to finally obtain usable textual concept results.

3. The neural network interpretation optimization method based on text concept representation according to claim 2, characterized in that, The following principles are followed when extracting and constructing text concepts to obtain representative and distinctive text concept results: Distinctiveness: Requires categorical and distinctive language descriptions; Clarity: It is necessary to avoid using descriptive methods that are vague or ambiguous in meaning; Consistency: The statement structure adopts a consistent construction method.

4. The neural network interpretation optimization method based on text concept representation according to claim 2, characterized in that, In step S2, a set of category images is prepared in advance. Category tag set and concept tag set The training data, including the class images, is used to extract the corresponding activation vector set from the output of the last convolutional layer of the classification network. Then, using this training data, the probe function is trained through supervised learning. The input to the probe function is image data. Activation vector output in the network The output is the weight or relevance score corresponding to the concept in the concept library.

5. The neural network interpretation optimization method based on text concept representation according to claim 4, characterized in that, The loss function value for the training process of the concept probe function is: in, It is the concept prediction loss, specifically the mean squared error between the concept output vector and the concept label vector.

6. The neural network interpretation optimization method based on text concept representation according to claim 5, characterized in that, In step S3, after the concept probe function is trained, the input data, concept prediction output, and final decision output are correlated. Then, for the input data, multiple concepts with high scores are selected based on the text concept score results to provide rich and easy-to-understand neural network decision explanation logic from the perspective of important text concepts.

7. The neural network interpretation optimization method based on text concept representation according to claim 6, characterized in that, In step S4, after obtaining a new concept score vector through human modification, a corresponding new activation vector is obtained using the concept probe function. Based on the new concept score vector obtained through human modification... The corresponding new activation vector will be obtained by minimizing the following values. : in It is a distance function. These are hyperparameters used to control the effectiveness of the intervention. Afterwards... The new output will be obtained through forward propagation in the classification network. At this point, the human modification logic will be transmitted to the neural network, enabling human intervention in the decision-making results of a single sample.

8. The neural network interpretation optimization method based on text concept representation according to claim 7, characterized in that, In step S5, a network with the same structure and parameters as the original classification network is first copied and used as the student model. Then, the validation set after single-sample correction is used. As new training data, samples correctly classified by the original classification network retain the previous output, while samples misclassified by the network will use the manually modified and optimized output. During knowledge distillation, the overall objective loss function is: in, and These represent the knowledge distillation process. and The respective weight parameter values ​​of the two items, Let be the soft loss function, representing the cross-entropy between the soft labels and the probability distribution of the student network output, i.e.: This represents the total number of categories that the original neural network focuses on. Represents temperature Below, the teacher model prediction results are of the following types. The probability score, For temperature In the student model prediction results, the class The probability score is in the following form: in, The teacher model represents the first... The original logits output or the logits output after manual intervention for each category. The student model represents the first... The original prediction output for each category, Let be the hard loss function, representing the cross-entropy between the true image label and the probability distribution of the student network's output at temperature 1, i.e.: in, For class The true category label, It is the type of student network prediction output when the temperature is 1. The probability value.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, A computer program is stored in a computer-readable storage medium, which, when executed by a processor, implements the steps of the method according to any one of claims 1-8.

Images