Self-supervised neural symbol fusion interpretable AI inference method and system

Abstract

The invention provides an interpretable AI inference method and system for self-supervised neural symbol fusion, and the method comprises the steps: S1, a self-supervised learning module: extracting feature information of unlabeled data, and obtaining a feature vector representing symbol knowledge; s2, a neural symbol fusion module: constructing a deep learning framework by using an AI intelligent learning strategy, and training feature vectors in combination with a reward function to obtain structured knowledge; s3, an explainable reasoning module: constructing an explainable prediction model based on neural symbol fusion, and obtaining a solved reasoning result; in the process of explaining the reasoning result, the complex reasoning process is simplified and displayed. Therefore, not only are the advantages of the neural network in processing complex modes and data association kept, but also the interpretability and logical reasoning ability of symbol reasoning are introduced, so that the performance and interpretability of the whole system are improved. Therefore, the model can optimize feature representation and logical reasoning at the same time in the training process.

Description

Technical Field

[0001] The present invention belongs to the field of AI data processing, and in particular to an explainable AI reasoning method and system of self-supervised neural symbolic fusion. Background Art

[0002] The rapid progress and development of emerging technologies such as the Internet, cloud computing, big data and artificial intelligence have become an important engine driving my country's digital economy. In particular, the interconnection of all things formed by the consumer Internet and the industrial Internet has led to the emergence of new management and decision-making models and methods in the fields of business, government affairs, security, health, finance, etc. Acquiring knowledge and wisdom from these massive data, discovering new management logic and models based on in-depth analysis and insights, and then making effective management decisions are the current fundamental research topics.

[0003] Traditional models often have limited generalization capabilities when dealing with unlabeled data or data in new fields. And traditional neural network models, especially deep learning models, often perform well in dealing with complex tasks, but their decision-making process is usually difficult to explain, which limits their application in scenarios that require high transparency and high credibility.

[0004] Therefore, there is an urgent need for an explainable AI reasoning method and system based on self-supervised neural-symbolic fusion to solve the technical problems existing in the existing technology. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide an explainable AI reasoning method and system of self-supervised neural-symbolic fusion in view of the deficiencies in the prior art.

[0006] The method of the present invention comprises the following steps:

[0007] Step S1: Use a self-supervised learning method to extract feature information of unlabeled data, pre-process and summarize the extracted feature information through a neural network model, and obtain a feature vector representing symbolic knowledge; Traditionally, feature extraction often relies on labeled data, while this method uses self-supervised learning to automatically extract feature information from unlabeled data. This reduces the reliance on a large amount of labeled data, improves data utilization and the generalization ability of the model;

[0008] Step S2: Using AI intelligent learning strategy, the feature extraction of neural network and the reasoning rules of symbolic reasoning are integrated to construct a deep learning framework. After acquiring the feature vector, the deep learning framework trains the feature vector in combination with the reward function to obtain structured knowledge. The present invention combines the feature extraction capability of neural network with the reasoning rules of symbolic reasoning to construct a deep learning framework. This integration not only takes advantage of the advantages of neural network in processing complex data, but also draws on the advantages of symbolic reasoning in logical inference and interpretability.

[0009] Step S3, construct an interpretable prediction model based on neural symbolic fusion, perform logical reasoning and decision making on structured knowledge, and obtain the solved reasoning results; in the process of interpreting the reasoning results, simplify and present the complex reasoning process. Constructing an interpretable prediction model based on neural symbolic fusion is another major innovation of this method. The model is able to perform logical reasoning and decision making, and output the solved reasoning results. In the process of interpreting the reasoning results, this method simplifies the complex reasoning process and presents it to the user, improving the transparency and interpretability of the AI ​​system.

[0010] Comprehensive effect: By combining self-supervised learning methods, neural networks and symbolic reasoning, the present invention has achieved significant improvements in reasoning ability, interpretability, generalization ability, etc.

[0011] In step S1, the self-supervised learning method is one of contrastive learning or autoencoder; the preprocessing and induction include denoising and normalizing the feature information; the neural network model is one of convolutional neural network, recurrent neural network or graph neural network, which is used to capture complex patterns and associations in the feature information and obtain feature vectors representing symbolic knowledge.

[0012] In step S2, the deep learning framework includes a feature extractor, an adversarial network generator, an adversarial network adversary and a sample classifier; the learning process of the deep learning framework includes the following steps:

[0013] S201, the feature extractor is used to obtain the feature information in step S1 and generate a sample set that conforms to the real features of the unlabeled data;

[0014] S202, the AI ​​intelligent learning strategy generates an adversarial network generator and an adversarial network antagonist based on the feature vector obtained in step S1, the feature vector uses random noise and concepts as input, generates concept corresponding samples of unlabeled data in the adversarial network generator, and the adversarial network antagonist uses a loss function to determine the difference between the sample set and the concept corresponding sample, and establishes trained label features and untrained label features;

[0015] S203, the sample classifier migrates the untrained label features to the trained label features based on the association between the symbolic knowledge, so as to predict the category of the unlabeled data.

[0016] In step S2, the reward function is used to reward the accuracy of feature extraction and the behavior of following symbolic reasoning rules. The reward function combines the category of the predicted unlabeled data with the feature vector for training to obtain structured knowledge. The calculation expression of the reward function is:

[0017] R=R1+R2

[0018]

[0019] Where R is the reward function; R1 is the rule-following reward; R2 is the feature extraction accuracy reward; N is the number of rules; α is a positive number indicating the degree of penalty for violating the rules; r k is a binary variable used to indicate whether each rule is followed, that is, 1 if the rule is followed and 0 if the rule is not followed.

[0020] In step S2, the process of obtaining structured knowledge includes the following steps:

[0021] S211, mapping the feature vector to the representation space through the embedded technology, and converting the feature vector into a low-dimensional dense vector;

[0022] S212. Describe the entities and relationships of low-dimensional dense vector data in the knowledge graph or graph neural network, and infer new entities and relationships based on rules and logic to obtain an output feature graph;

[0023] S213. In step S212, the output feature map is locally weighted summed for each output feature point by the feature extraction algorithm, and the structured knowledge is calculated after adding the bias term; the calculation expression of the feature extraction algorithm is:

[0024]

[0025] Where Q ij is the element in the i-th row and j-th column of the output feature map; σ is the activation function used to introduce nonlinearity; S( i+m,j+n ) is the element in the input feature map corresponding to the current position of the convolution kernel, that is, (i, j) is the position coordinate of an element in the output feature map, (m, n) represents the offset of a weight element in the convolution kernel relative to the center position of the convolution kernel. Since the convolution kernel slides on the input feature map for local weighted summation, (i+m, j+n) actually represents the position coordinate of an element in the area on the input feature map that overlaps with the current sliding position of the convolution kernel; L mnis the weight of the mth row and nth column in the convolution kernel; φ is the bias term; the summation symbol It means that all elements in the convolution kernel (i.e., m and n are from arrive ) traversal.

[0026] In step S2, when all rules are fully followed, the value of the rule-following reward R1 is equal to 1; when all rules are completely violated, the value of the rule-following reward R1 is equal to -α; when some rules are followed, the value of the rule-following reward R1 is between -α and 1.

[0027] In step S3, the prediction model includes a representation module and an explanation generator;

[0028] The representation module defines a relational table for structured knowledge and endows the prediction model with interpretability through an aggregation neighbor mechanism and an attention mechanism;

[0029] The explanation generator sorts the attention values ​​of the prediction model and selects the neighbor with the highest attention value as the explanation of the model prediction result.

[0030] In step S3, in the process of defining the relational table of the structured knowledge by the representation module, the independent information in the structured knowledge is encoded, and modeling is performed based on the interaction between the independent information.

[0031] The present invention also provides an explainable AI reasoning system of self-supervised neural symbolic fusion, including a preprocessing unit, a conversion unit and a reasoning explanation unit;

[0032] The preprocessing unit is used to obtain and process feature information of unlabeled data to obtain feature vector data that can be received by the conversion unit;

[0033] The conversion unit is used to perform multiple calculations and training on the feature vector to obtain structured knowledge that can be received by the reasoning and interpretation unit;

[0034] The reasoning interpretation unit is used to simplify complex structured knowledge into reasoning results.

[0035] The present invention also provides a device, comprising: a processor, a storage device and an electronic display device; there is at least one processor, the storage device stores instructions executable by the processor, and the processor is capable of executing each step of the described method; the electronic display device is used to display the reasoning results.

[0036] The present invention has the following beneficial effects:

[0037] (1) The present invention combines the powerful feature extraction capability of neural networks with the clear rule representation method of symbolic reasoning to construct a deep learning framework. This not only retains the advantages of neural networks in processing complex patterns and data associations, but also introduces the interpretability and logical reasoning ability of symbolic reasoning, thereby improving the performance and interpretability of the entire system. At the same time, a reward function is also introduced in the reasoning process, and then in the training process, not only the accuracy of feature extraction is paid attention to, but also the compliance with symbolic reasoning rules is emphasized. As a result, the model can simultaneously optimize feature representation and logical reasoning during the training process, thereby improving the generalization and reasoning ability of the model when processing unlabeled data.

[0038] (2) Through the training process of the deep learning framework, the present invention can extract structured knowledge from unlabeled data. This not only contains the characteristic information of the data, but also contains the rules and logic of symbolic reasoning, providing a solid foundation for subsequent reasoning and decision-making. And because of the integration of symbolic reasoning, the reasoning results obtained by this module have higher interpretability. This means that users or developers can more easily understand how the model makes decisions, thereby increasing the credibility and transparency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0040] Figure 1 is a flow chart of the explainable AI reasoning method of the self-supervised neural symbolic fusion of the present invention;

[0041] Figure 2 is a flow chart of the learning process of the deep learning framework of the present invention;

[0042] Figure 3 is a flow chart of the process of obtaining structured knowledge of the present invention;

[0043] Figure 4 It is a flow chart of the relationship between the neural network and symbolic reasoning of the present invention;

[0044] Figure 5 It is a flow chart of the deep learning framework of the present invention for predicting the category of unlabeled data. DETAILED DESCRIPTION

[0045] like Figure 1 As shown, an embodiment of the present invention provides an explainable AI reasoning method and system for self-supervised neural symbolic fusion, wherein the method comprises the following steps:

[0046] S1, self-supervised learning module: adopts self-supervised learning method to extract feature information of unlabeled data, pre-processes and summarizes the extracted feature information through neural network model, and obtains feature vectors representing symbolic knowledge;

[0047] In a preferred embodiment, in step S1), the self-supervised learning method is one of contrastive learning or autoencoder, and the extracted feature information is subjected to denoising and normalized data processing; the neural network model is one of convolutional neural network, recurrent neural network or graph neural network, which is used to capture complex patterns and associations in the feature information.

[0048] S2, Neural-Symbolic Fusion Module: Using AI intelligent learning strategies, the feature extraction of neural networks and the reasoning rules of symbolic reasoning are integrated, so that neural networks can use symbolic knowledge for logical reasoning when processing large-scale perception tasks. At the same time, symbolic logic can also use the representation learning ability of neural networks to optimize the reasoning process and build a deep learning framework. After obtaining feature vectors, the deep learning framework combines reward functions to train feature vectors and obtain structured knowledge.

[0049] What needs further explanation here is that AI intelligent learning strategies combine neural networks and symbolic reasoning, and use meta-learning or end-to-end learning to repeatedly train and optimize the deep learning framework composed of the fusion.

[0050] like Figure 2 As shown, further, the deep learning framework includes: a feature extractor, an adversarial network generator, an adversarial network adversary and a sample classifier; the learning process of the deep learning framework includes the following steps:

[0051] S201, a feature extractor is used to obtain the feature information in step S1) and generate a sample set that conforms to the true features of the unlabeled data;

[0052] S202, AI intelligent learning strategy is based on feature vectors, generating adversarial network generators and adversarial network adversaries, feature vectors use random noise and concepts as input, generate concept corresponding samples of unlabeled data in the adversarial network generator, and the adversarial network adversary uses a loss function to determine the difference between the sample set and the concept corresponding sample, and establish features with training labels and features without training labels;

[0053] S203, the sample classifier migrates the untrained label features to the trained label features based on the association between the symbolic knowledge, so as to predict the category of the unlabeled data.

[0054] It is necessary to further explain here that the reward function is used to reward the accuracy of feature extraction and the behavior of following symbolic reasoning rules. The reward function combines the category of the predicted unlabeled data with the feature vector for training to obtain structured knowledge; the calculation expression of the reward function is:

[0055] R=R1+R2

[0056]

[0057] Where: R is the reward function; R1 is the rule-following reward; R2 is the feature extraction accuracy reward; N is the number of rules; i is used to represent the i-th rule; α is a positive number indicating the degree of penalty for violating the rule; r k is a binary variable used to indicate whether each rule is followed, that is, 1 if the rule is followed and 0 if the rule is not followed.

[0058] like Figure 3 As shown, further, the process of obtaining structured knowledge includes the following steps:

[0059] S211, mapping the feature vector to the representation space through the embedded technology, and converting the feature vector into a low-dimensional dense vector;

[0060] S212. Describe the entities and relationships of low-dimensional dense vector data in the knowledge graph or graph neural network, and infer new entities and relationships based on rules and logic to obtain an output feature graph;

[0061] It is further explained here that rules and logical inference specifically refer to: customized rules and logic and direct use of existing technologies.

[0062] Among them, customized rules: when mapping feature vectors to representation space, you can customize mapping functions and conversion rules according to the needs of specific tasks and the characteristics of the data. For example, you can choose different embedding dimensions, embedding methods (such as Word2Vec, GloVe, etc.), and regularization methods of embedding space. Alternatively, when describing entities and relationships in knowledge graphs or graph neural networks, you can customize the types, attributes, and constraints between entities and relationships according to domain knowledge and the needs of specific tasks. Alternatively, in the feature extraction algorithm, you can customize parameters such as the size, step size, and padding method of the convolution kernel, as well as the selection of activation functions and the setting of bias items. The selection and setting of these parameters will directly affect the effect of feature extraction and the quality of structured knowledge.

[0063] Logical inference: In the process of converting feature vectors into low-dimensional dense vectors, you can use custom loss functions and optimization algorithms to guide the embedding process to ensure that the converted vectors can retain the key information of the original data. Alternatively, when inferring new entities and relationships based on rules and logic, you can customize inference rules, such as similarity calculation, path reasoning, rule matching, and other methods. These inference rules can be designed and adjusted based on domain knowledge and data characteristics. Alternatively, in the process of local weighted summation, you can customize weighting methods, such as distance-based weighting, similarity-based weighting, and so on. These weighting methods can be designed and adjusted according to the needs of specific tasks and the characteristics of the data.

[0064] Directly adopt existing technologies: When mapping feature vectors to representation space, existing embedded technologies such as Word2Vec, GloVe, FastText, etc. can be directly adopted. These technologies have been widely used in fields such as natural language processing and have proven their effectiveness. Alternatively, when describing entities and relationships in knowledge graphs or graph neural networks, existing knowledge graph construction technologies and graph neural network models can be directly adopted. For example, existing knowledge graph construction tools (such as Neo4j, Jena, etc.) can be used to build knowledge graphs; or existing graph neural network frameworks (such as PyTorchGeometric, DGL, etc.) can be used to build and train graph neural network models. Alternatively, in feature extraction algorithms, existing convolutional neural network (CNN) or graph convolutional neural network (GCN) models can be directly adopted. These models already contain components such as convolution operations, pooling operations, activation functions, etc., and can be directly used for feature extraction and acquisition of structured knowledge.

[0065] In step S212, the process of obtaining structured knowledge can either customize rules and logic to adapt to the needs of specific tasks and data characteristics, or directly adopt existing technologies to improve efficiency and reliability. The specific method to be adopted depends on factors such as the needs of specific tasks, the characteristics of data, and available resources.

[0066] S213, in step S212), the output feature map is locally weighted summed for each output feature point by the feature extraction algorithm, and the structured knowledge is calculated after adding the bias term; the calculation expression of the feature extraction algorithm is:

[0067]

[0068] Where: Q ij is the element in the i-th row and j-th column of the output feature map; σ is the activation function used to introduce nonlinearity; S (i+m,j+n)is the element in the input feature map corresponding to the current position of the convolution kernel, that is, (i, j) is the position coordinate of an element in the output feature map, (m, n) represents the offset of a weight element in the convolution kernel relative to the center position of the convolution kernel. Since the convolution kernel slides on the input feature map for local weighted summation, (i+m, j+n) actually represents the position coordinate of an element in the area on the input feature map that overlaps with the current sliding position of the convolution kernel. mn is the weight of the mth row and nth column in the convolution kernel; φ is the bias term; the summation symbol It means that all elements in the convolution kernel (i.e., m and n are from arrive ). When all rules are fully followed, the value of the R1 rule-following reward is equal to 1; when all rules are completely violated, the value of the R1 rule-following reward is equal to -α; when all rules are partially followed, the value of the rule-following reward is between -α and 1.

[0069] What needs to be further explained here is that the design of feature extraction algorithm is not limited to the above mathematical expressions, it can also include more design elements, such as the initialization method of convolution kernel, convolution step size, padding method, pooling layer design, etc. In addition, different feature extraction algorithms may need to be designed for different types of data and tasks.

[0070] When optimizing feature extraction algorithms, the accuracy and efficiency of feature extraction are usually improved by adjusting these design elements, trying different algorithm structures, using regularization techniques (such as Dropout, L1 / L2 regularization), etc. At the same time, it is also necessary to combine evaluation indicators to evaluate the performance of feature extraction algorithms for further adjustment and optimization.

[0071] S3, Explainable Reasoning Module: Build an explainable prediction model based on neural-symbolic fusion, perform logical reasoning and decision-making on structured knowledge, and obtain the solved reasoning results; in the process of explaining the reasoning results, simplify and display the complex reasoning process.

[0072] Furthermore, in step S3), the prediction model includes: a representation module and an explanation generator; the representation module defines a relational table for structured knowledge, and in the process of defining the relational table for structured knowledge by the representation module, the independent information in the structured knowledge is encoded, and the model is modeled based on the interaction between the independent information. The prediction model is given interpretability through the aggregation neighbor mechanism and the attention mechanism; the explanation generator sorts the attention values ​​of the prediction model and selects the neighbor with the highest attention value as the explanation of the model prediction result.

[0073] Based on the above-mentioned self-supervised neural-symbolic fusion explainable AI reasoning method, a self-supervised neural-symbolic fusion explainable AI reasoning system is proposed, including: a preprocessing unit, a conversion unit and a reasoning and interpretation unit; the preprocessing unit is used to acquire and process the feature information of unlabeled data to obtain feature vector data that can be received by the conversion unit; the conversion unit is used to calculate and train the feature vector multiple times to obtain structured knowledge that can be received by the reasoning and interpretation unit; the reasoning and interpretation unit is used to simplify complex structured knowledge into reasoning results.

[0074] Based on the above-mentioned method or system, a device is proposed, including: a processor, a storage device and an electronic display device; there is at least one processor, the memory stores instructions executable by the processor, and the processor can execute each step in the method or system; the electronic display device is used to display the reasoning results.

[0075] like Figure 4 As shown in the figure, on the symbolic space side, symbolic knowledge is transformed into a symbolic knowledge graph through reasoning deduction and embedded representation, and symbolic rules or axioms are extracted using inductive reasoning. This process reflects the rigor of the symbolic system in knowledge representation and reasoning.

[0076] The neural symbolic integration part focuses on integrating neural knowledge with symbolic knowledge. Through vector decoding, neural knowledge is converted into a parsable form, and then deeply interacts with symbolic knowledge through reasoning, deduction and embedded representation. In this process, neural rules or axioms, as the basis of the neural system, complement symbolic rules and jointly support the entire knowledge processing framework.

[0077] like Figure 5 As shown, this image depicts the framework of the Generative Adversarial Network (GAN) and its training and testing process.

[0078] Real samples (seen categories): These are data samples collected from the real world with clear category labels, used to train the model;

[0079] Feature extractor: extracts features from real samples, and the generated feature representation is recorded as X s ,This step is key for the model to understand the data;

[0080] Generative Adversarial Network Generator: Responsible for generating sample representations. The figure shows two generated samples. Generated samples with category labels and Zero-shot generation for unseen categories;

[0081] Generative adversarial network adversary: ​​determines whether the generated samples are real and ensures that the generator can generate high-quality samples;

[0082] Ontology encoder and ontology graph: Using ontology structure to represent knowledge, S , O U and Z represent ontology codes of different categories, which help the model understand the relationship between categories;

[0083] Training phase S (visible categories): In this phase, the model is trained using labeled data to learn how to distinguish and generate samples of the visible categories;

[0084] Testing phase U (unseen categories): The model needs to process unseen categories in the testing phase, that is, perform zero-sample learning to evaluate its generalization ability;

[0085] Classification loss & class prototype regularization: During the training process, classification loss and class prototype regularization are applied to ensure the accuracy and robustness of the classifier;

[0086] Zero-shot classifier: It is specially used to classify samples of unseen categories and is an important manifestation of the model's generalization ability;

[0087] Classification result: The classification result finally output by the model, which is used to evaluate the performance of the model.

[0088] Embodiment 1

[0089] Step 1: The preprocessing unit obtains and processes the feature information of the unlabeled data. In this step, the deep learning framework ResNet-50 is used. The framework has been pre-trained on the ImageNet dataset, so it can be used directly to extract the feature information of the image data. In this embodiment, 1,000 unlabeled cat and dog pictures were selected as input, and after processing by ResNet-50, a 512-dimensional feature vector of each picture was obtained.

[0090] What needs further explanation here is that in order to merge with symbolic knowledge, after feature extraction, the high-dimensional feature vector is mapped to a symbolically understandable low-dimensional space through knowledge distillation or embedding technology (such as TransE, DistMult, etc.).

[0091] Step 2: Input the symbolic embedding vector obtained in step 1 into a hybrid model that combines CNN and rule engine. The CNN part continues to optimize the feature representation, while the rule engine constrains and adjusts the features according to the symbolic rules in the knowledge graph. Here, an adversarial network structure can be designed, in which the generator tries to deceive the discriminator (based on the rule base) to generate feature representations that are more in line with symbolic rules.

[0092] The conversion unit calculates and trains the feature vector multiple times to obtain structured knowledge that can be received by the reasoning and interpretation unit. In this step, the neural network technology LSTM is used. The model has been pre-trained on the corpus, so it can be used directly to calculate and train the feature vector obtained in step 1.

[0093] In this embodiment, a 512-dimensional feature vector is input into the LSTM, and after multiple forward propagations and back propagations, the classification probability distribution of each image, that is, structured knowledge, is obtained.

[0094] In this step, the reward function parameters are: R1 (rule following reward): = 1 when following the rules, = -1 when violating the rules, and between -1 and 1 when partially following the rules; R2 (feature extraction accuracy reward): calculated based on the classification accuracy; penalty degree parameter: λ = 0.5.

[0095] Step 3: The reasoning explanation unit simplifies the complex structured knowledge into reasoning results. In this step, the self-supervised learning technology SimCLR is used. The model has been pre-trained on the ImageNet dataset, so it can be used directly to simplify the structured knowledge obtained in step 2.

[0096] In this embodiment, the classification probability distribution of each image is input into SimCLR, and after multiple contrastive learning and clustering, the category label of each image is obtained, which is the final inference result. In the contrastive learning process, not only the embedding vectors of different images are compared, but also the symbolic rules are used to evaluate the rationality of these vectors. For example, if two images are mistakenly classified as the same category (such as one is a cat and the other is a dog), but the symbolic rules indicate that there is a significant difference between them, this error should be corrected by adjusting the model parameters.

[0097] Step 4: During the entire process, by introducing the reward function, the model can simultaneously optimize feature representation and logical reasoning during training, thereby improving the model's generalization and reasoning capabilities when processing unlabeled data.

[0098] In this step, the reinforcement learning technology Q-learning is used. The model has been pre-trained on Atari games, so it can be used directly to optimize the model training process. In this embodiment, the classification accuracy of the model is used as a reward signal, and the parameters of the model are adjusted so that the model can simultaneously optimize feature representation and logical reasoning during the training process.

[0099] While Q-learning works well in reinforcement learning, in this scenario it may be more appropriate to use policy gradient methods (such as REINFORCE) or the Actor-Critic framework because they can more directly process classification accuracy as a reward signal.

[0100] Step 5: In the process of explaining the inference results, the complex inference process is simplified and presented. In this step, the interpretability algorithm Layer-wise Relevance Propagation (LRP) is used, which can decompose the internal state of the model while maintaining the accuracy of the model prediction, thereby obtaining the contribution of each neuron to the model output.

[0101] In this embodiment, LRP is applied to the model to obtain a classification heat map for each image, thereby intuitively showing how the model makes decisions.

[0102] In addition to generating classification heat maps, explanatory text can also be generated based on the symbolic rule base to explain why the model classifies the image into a specific category (such as "This image is classified as a cat because it has obvious whiskers and triangular ears, which is consistent with the characteristics of cats in the knowledge graph"). In this way, non-technical users can also more easily understand the decision-making process of the model.

[0103] The system can extract structured knowledge from unlabeled data, improving data utilization and the generalization ability of the model.

[0104] Through neural-symbolic fusion, the system not only retains the processing advantages of neural networks, but also introduces the interpretability and logical reasoning capabilities of symbolic reasoning.

[0105] The explainable reasoning module makes the model's decision-making process more transparent, improving the model's credibility and user acceptance.

[0106] The system provided in this embodiment includes the following modules:

[0107] Self-supervised learning module: Use self-supervised learning methods (such as contrastive learning or autoencoders) to extract feature information of unlabeled data. Preprocess and summarize the extracted feature information through a neural network model (such as convolutional neural network, recurrent neural network or graph neural network) to obtain a feature vector representing symbolic knowledge.

[0108] Neural symbolic fusion module: Using AI intelligent learning strategies, the feature extraction of neural networks and the reasoning rules of symbolic reasoning are integrated to build a deep learning framework. The deep learning framework includes feature extractors, adversarial network generators, adversarial network adversaries, and sample classifiers. The feature extractor generates a sample set that conforms to the true features of unlabeled data. The AI ​​intelligent learning strategy generates adversarial network generators and adversarial network adversaries, and uses loss functions to distinguish the differences between sample sets and concept-corresponding samples, and establishes trained labels and untrained labels. The sample classifier migrates untrained label features to trained label features based on the association between symbolic knowledge, which is used to predict the category of unlabeled data.

[0109] Explainable reasoning module: Build an explainable prediction model based on neural symbolic fusion, including a representation module and an explanation generator. The representation module defines a relational table for structured knowledge and gives the prediction model interpretability through the aggregation neighbor mechanism and attention mechanism. The explanation generator sorts the attention values ​​of the prediction model and selects the neighbor with the highest attention value as the explanation of the model prediction result.

[0110] Reasoning and interpretation: Perform logical reasoning and decision making on structured knowledge to obtain the reasoning results after solving. In the process of explaining the reasoning results, the complex reasoning process is simplified and presented to users or developers.

[0111] The present invention provides an explainable AI reasoning method and system for self-supervised neural symbolic fusion. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A self-supervised neural-symbolic fusion interpretable AI reasoning method, characterized in that: The following steps are involved: Step S1, using a self-supervised learning method to extract feature information of unlabeled data, preprocessing and summarizing the extracted feature information through a neural network model to obtain a feature vector representing symbolic knowledge; Step S2: Using AI intelligent learning strategy, the feature extraction of neural network and the reasoning rules of symbolic reasoning are integrated to construct a deep learning framework. After obtaining the feature vector, the deep learning framework trains the feature vector in combination with the reward function to obtain structured knowledge. Step S3: construct an interpretable prediction model based on neural-symbolic fusion, perform logical reasoning and decision-making on structured knowledge, and obtain the solved reasoning results; in the process of explaining the reasoning results, simplify and present the complex reasoning process.

2. The method according to claim 1, characterized in that In step S1, the self-supervised learning method is one of contrastive learning or autoencoder; the preprocessing and induction include denoising and normalizing the feature information; the neural network model is one of convolutional neural network, recurrent neural network or graph neural network, which is used to capture complex patterns and associations in the feature information and obtain feature vectors representing symbolic knowledge.

3. The method according to claim 2, characterized in that In step S2, the deep learning framework includes a feature extractor, an adversarial network generator, an adversarial network adversary and a sample classifier; the learning process of the deep learning framework includes the following steps: S201, the feature extractor is used to obtain the feature information in step S1 and generate a sample set that conforms to the real features of the unlabeled data; S202, the AI ​​intelligent learning strategy generates an adversarial network generator and an adversarial network antagonist based on the feature vector obtained in step S1, the feature vector uses random noise and concepts as input, generates concept corresponding samples of unlabeled data in the adversarial network generator, and the adversarial network antagonist uses a loss function to determine the difference between the sample set and the concept corresponding sample, and establishes trained label features and untrained label features; S203, the sample classifier migrates the untrained label features to the trained label features based on the association between the symbolic knowledge, so as to predict the category of the unlabeled data.

4. The method according to claim 3, characterized in that In step S2, the reward function is used to reward the accuracy of feature extraction and the behavior of following symbolic reasoning rules. The reward function combines the category of the predicted unlabeled data with the feature vector for training to obtain structured knowledge. The calculation expression of the reward function is: R=R1+R2 Where R is the reward function; R1 is the rule-following reward; R2 is the feature extraction accuracy reward; N is the number of rules; α is a positive number indicating the degree of penalty for violating the rules; r k is a binary variable used to indicate whether each rule is followed, that is, 1 if the rule is followed and 0 if the rule is not followed.

5. The method according to claim 4, characterized in that In step S2, the process of obtaining structured knowledge includes the following steps: S211, mapping the feature vector to the representation space through the embedded technology, and converting the feature vector into a low-dimensional dense vector; S212. Describe the entities and relationships of low-dimensional dense vector data in the knowledge graph or graph neural network, and infer new entities and relationships based on rules and logic to obtain an output feature graph; S213. In step S212, the output feature map is locally weighted summed for each output feature point by the feature extraction algorithm, and the structured knowledge is calculated after adding the bias term; the calculation expression of the feature extraction algorithm is: Where Q ij is the element in the i-th row and j-th column of the output feature map; σ is the activation function used to introduce nonlinearity; S( i+m,j+n ) is the element in the input feature map corresponding to the current position of the convolution kernel, that is, (i, j) is the position coordinate of an element in the output feature map, (m, n) represents the offset of a weight element in the convolution kernel relative to the center position of the convolution kernel. Since the convolution kernel slides on the input feature map for local weighted summation, (i+m, j+n) represents the position coordinate of an element in the area on the input feature map that overlaps with the current sliding position of the convolution kernel; L mn is the weight of the mth row and nth column in the convolution kernel; φ is the bias term; the summation symbol Represents the traversal of all elements in the convolution kernel.

6. The method according to claim 5, characterized in that In step S2, when all rules are fully followed, the value of the rule-following reward R1 is equal to 1; when all rules are completely violated, the value of the rule-following reward R1 is equal to -α; when some rules are followed, the value of the rule-following reward R1 is between -α and 1.

7. The method according to claim 6, characterized in that In step S3, the prediction model includes a representation module and an explanation generator; The representation module defines a relational table for structured knowledge and endows the prediction model with interpretability through an aggregation neighbor mechanism and an attention mechanism; The explanation generator sorts the attention values ​​of the prediction model and selects the neighbor with the highest attention value as the explanation of the model prediction result.

8. The method according to claim 7, characterized in that: In step S3, in the process of defining the relational table of the structured knowledge by the representation module, the independent information in the structured knowledge is encoded, and modeling is performed based on the interaction between the independent information.

9. An explainable AI reasoning system using self-supervised neural symbolic fusion implemented by the method according to any one of claims 1 to 8, characterized in that: It includes a preprocessing unit, a conversion unit, and a reasoning and interpretation unit; The preprocessing unit is used to obtain and process feature information of unlabeled data to obtain feature vector data that can be received by the conversion unit; The conversion unit is used to calculate and train the feature vector multiple times to obtain structured knowledge that can be received by the reasoning and interpretation unit; The reasoning interpretation unit is used to simplify complex structured knowledge into reasoning results.

10. A device, characterized in that: include: processors, storage and electronic display devices; There is at least one processor, the memory stores instructions executable by the processor, and the processor can execute each step of the method described in any one of claims 1 to 8; the electronic display device is used to display the reasoning result.

Images