High-definition image interpretable classification method and device, computer equipment and storage medium

Abstract

The invention relates to a high-definition image interpretable classification method and device, computer equipment and a storage medium. The method simulates the process of image perception, feature extraction, inductive reasoning and learning adjustment of a human brain. According to the method, on the basis of the potential space of the pre-trained StyleGAN, a conversion network is designed to simulate a brain visual cortex for feature extraction, high-dimensional potential codes are effectively converted into low-dimensional and decoupled classification features, and the problems of rule explosion and calculation collapse are relieved; a fuzzy neural network is utilized to simulate the reasoning function of cerebral apical lobe and prefrontal lobe cortex, a fuzzy neural network structure improvement method adaptive to a high-definition image interpretable classification model is designed, and interpretable classification by utilizing low-dimensional features is realized; the fuzzy rule for classification is visually displayed and analyzed, and a feature visualization method is designed based on a pre-trained StyleGAN generator, so that the interpretability is further improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment and storage medium for interpretable classification of high-definition images. Background Art

[0002] Fuzzy neural network combines fuzzy logic with neural network, and has both the self-learning ability of neural network and the interpretability of fuzzy system. Fuzzy neural network can use the learning ability of neural network to learn a set of fuzzy rules with appropriate language labels from data, so that people can easily understand the decision-making process of the model. It is an important part of explainable artificial intelligence (XAI) and has been widely used in medical, financial, education and other fields. However, fuzzy neural network is usually used to interpret structured data. For unstructured data such as images, text or speech, if the image pixels, words (characters), and a section of waveform points are directly used as the input of fuzzy neural network, on the one hand, since the activation strength of fuzzy rules is usually the product of all input membership values ​​less than 1, a large amount of input will make the activation strength of all rules almost zero, thus causing the model calculation to collapse; on the other hand, a large amount of input without semantic meaning will also reduce people's understanding and mastery of fuzzy rules. Therefore, if you want to use fuzzy rules to interpret complex unstructured information, you need to extract features in advance.

[0003] Manually defining image features is time-consuming and labor-intensive; the feature extraction method using convolutional neural network + dimensionality reduction extracts low-dimensional feature maps, and the specific semantics of the feature maps are still unknown; the automatic feature extraction method that relies solely on deep learning can hardly guarantee the effectiveness and usability of the extracted features in subsequent fuzzy reasoning. In summary, it is not easy to extract the key features required for fuzzy rule reasoning from unstructured data. Existing methods for image classification using interpretable fuzzy rules can usually only process low-dimensional and simple images, and there are few methods for high-definition images.

[0004] In existing research results, variational autoencoders are used to extract features from images, and then fuzzy neural networks are used for interpretable classification based on low-dimensional features. However, due to the insufficient reconstruction capability of VAE, the accuracy and effectiveness of feature extraction are still challenges for large-size high-definition images. In existing research results, an image-coder mapping network is constructed to extract features from images, and image reconstruction is performed using a coder-feature mapping network and a pre-trained StyleGAN generator to ensure the effectiveness of feature extraction. However, it is difficult to balance the quality of image reconstruction and the degree of decoupling of extracted features, resulting in insufficient restoration of image details, inaccurate feature extraction, and low feature decoupling. Summary of the invention

[0005] Based on this, it is necessary to provide a high-definition image interpretable classification method, device, computer equipment and storage medium to address the above technical problems.

[0006] A high-definition image interpretable classification method, the method comprising:

[0007] The acquired high-definition image is encoded using a pre-trained StyleGAN encoder to obtain the latent code and construct the latent space.

[0008] The latent code uses a transformation network to extract a low-dimensional, interpretable feature vector from complex image information; the transformation network is used to utilize the feature direction in the latent space to obtain the feature vector corresponding to the latent code.

[0009] The feature vectors are inferred and classified using an interpretable classifier to obtain a classification result with interpretable categories; the interpretable classifier is used to simulate the reasoning and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time, with the effective classification ability of the fuzzy neural network classifier, the accuracy of classification is ensured.

[0010] A pre-trained generator is used to map the latent space to the image space to obtain a reconstructed high-definition image.

[0011] In one of the embodiments, the implementation process of the conversion network includes: first determining the form of the feature direction, then finding the feature direction from the latent space, and finally designing an eigenvalue measurement method to convert the feature direction into a feature vector.

[0012] In one embodiment, the feature value of a sample w in the latent space on the hth feature is the feature direction f of the sample w in the latent space. h The cosine similarity of (w); the eigenvalue of sample w in the latent space on the hth feature is:

[0013]

[0014] Among them, s h is the eigenvalue of sample w in the latent space on the hth feature, f h (w) is the sample w and feature direction in the latent space.

[0015] The mathematical expression of the conversion network is:

[0016]

[0017] Among them, w is the potential code, T(w) is the output obtained after the potential code w is input into the conversion network, and s is the feature vector corresponding to the potential code. is the set of feature directions of w on all features, and ||·|| is the L1 norm.

[0018] In one embodiment, the method further includes a feature visualization method based on the pre-trained generator, and the specific steps of the feature visualization method include:

[0019] The latent code w is moved in the latent space so that the h-th feature of the generated image changes, and the latent code after the move is obtained as:

[0020] w new =w+δf h (w)

[0021] Among them, w is the initial latent code, w new is the potential code after the shift, f h (w) is the direction of the hth feature, and δ is the moving step size.

[0022] According to the initial potential code and the potential code after the shift, the change of the potential code is determined as:

[0023] Δw=w new -w=δf h (w)

[0024] According to the eigenvalue calculation formula of a potential coding sample on the hth feature, the eigenvalue of the potential coding on the hth feature before and after the movement is determined as:

[0025]

[0026] According to the eigenvalue of the latent code on the hth feature before and after the movement, the change of the eigenvalue is determined as:

[0027]

[0028] According to the change of potential coding and the change of feature value, determine the h The mapping relationship to Δx is:

[0029] Δx=x new -x

[0030] =G(w new )-G(w)

[0031] =G(w+Δw)-G(w)

[0032] =G(w+αf h (w))-G(w)

[0033] ≈G(w+Δs h f h (w))-G(w)

[0034] Using Δs h The mapping relationship to Δx reflects the change in feature value in the potential code, which is then reflected in the generated image using the pre-trained generator.

[0035] In one embodiment, the interpretable classifier includes a fuzzification layer, a fuzzy logic layer, and a classification layer.

[0036] In the fuzzification layer, an additional D node is added for each feature; the D nodes are initialized in a random manner and continuously adjusted during the training process.

[0037] When the antecedent of a fuzzy rule contains a D node, it means that the feature corresponding to the D node is ignored and the fuzzy rule is rewritten as a rule with a more concise antecedent.

[0038] In one embodiment, the pre-trained generator is a pre-trained StyleGAN generator.

[0039] In one of the embodiments, according to the structural design of the fuzzy neural network model of the explainable classifier, after the model is trained, the fuzzy IF THEN rules are directly extracted from the model.

[0040] A high-definition image interpretable classification device, the device comprising:

[0041] The latent coding determination module of the pre-trained StyleGAN is used to encode the acquired high-definition image using the pre-trained StyleGAN encoder to obtain the latent coding, and convert the input image space into the latent space.

[0042] The feature extraction module is used to extract low-dimensional, interpretable feature vectors from complex image information using a transformation network; the transformation network is used to use the feature direction in the latent space to obtain the feature vector corresponding to the latent code.

[0043] The interpretable classification module is used to infer and classify feature vectors using interpretable classifiers to obtain classification results with interpretable categories; the interpretable classifier is used to simulate the reasoning and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time use the effective classification ability of the fuzzy neural network classifier to ensure the accuracy of classification.

[0044] The visualization module is used to map the latent space to the image space using the pre-trained generator to obtain a reconstructed high-definition image.

[0045] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0046] A computer-readable storage medium stores a computer program, which implements the steps of any of the above methods when executed by a processor.

[0047] The above-mentioned high-definition image interpretable classification method, device, computer equipment and storage medium, the method simulates the human brain's image perception, feature extraction, inductive reasoning and adjustment learning process. This method is based on the latent space of the pre-trained StyleGAN, designs a conversion network to simulate the visual cortex of the brain for feature extraction, effectively converts high-dimensional latent codes into low-dimensional, decoupled classification features, and alleviates the problems of rule explosion and computational collapse; uses fuzzy neural networks to simulate the reasoning function of the parietal lobe and prefrontal cortex of the brain, and designs a fuzzy neural network structure improvement method adapted to the high-definition image interpretable classification model, realizing interpretable classification using low-dimensional features; visualizes and analyzes the fuzzy rules used for classification, and designs a feature visualization method based on the pre-trained StyleGAN generator, further improving interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A framework diagram of an explainable classification model for high-definition images in one embodiment;

[0049] Figure 2 is a flowchart of an explainable classification method for high-definition images in one embodiment;

[0050] Figure 3 A schematic diagram of a feature visualization process in a high-definition image interpretable classification model in one embodiment;

[0051] Figure 4 A schematic diagram of the structure of a fuzzy neural network in an interpretable classification model for high-definition images in another embodiment;

[0052] Figure 5 is the membership function of the first classification feature in another embodiment;

[0053] Figure 6 An example of a feature visualization image in another embodiment, wherein (a) is picture 1, and (b) is picture 2;

[0054] Figure 7 A schematic diagram of feature visualization of picture 1 in another embodiment;

[0055] Figure 8 This is a schematic diagram of feature visualization of picture 2 in another embodiment;

[0056] Fig. 9 A schematic diagram of fuzzy rules in another embodiment;

[0057] Fig.10 is a fuzzy rule used when a fuzzy neural network classifies a single image of category 0 in another embodiment, wherein (a) is the original image and the category label, and (b) is the fuzzy rule and its contribution;

[0058] Fig.11 is a fuzzy rule used when a fuzzy neural network classifies a single image of category 1 in another embodiment, wherein (a) is the original image and the category label, and (b) is the fuzzy rule and its contribution;

[0059] Fig.12 It is a structural block diagram of a high-definition image interpretable classification device in one embodiment;

[0060] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] The human brain is a perfect model for feature perception and reasoning decisions on complex information based on signal transmission between neurons. The human brain can not only quickly identify and classify objects, but also "know why" and give reasonable reasons and explanations. Take the human brain's classification of fish as an example. When a person sees an image of a fish, the light reflected by the image reaches the eyeball, and the image information is collected by the retina. After the retina perceives the light signal, the optic nerve converts it into an electrical signal, and continuously transmits it to the next level of nerve cells through the visual pathway in an "electro-chemical" way, and finally projects it to the visual cortex. There are nerve cells in the visual cortex that are sensitive to various stimuli, so some nerve cells will be activated by the neural current impulses from the eyes, allowing the visual cortex to extract some "features" related to the image, such as "long dorsal fin", "fish tail", "streamlined shape", "pink" and other features. Finally, the "features" extracted by the visual cortex flow into the parietal lobe and prefrontal cortex, and the neurons in these brain regions can reason and classify objects based on the extracted features. The classification process is self-explanatory. For example, the human brain will compare the object in front of it with the fish in the memory. When it believes that the key features of the object in front of it (such as the dorsal fin, tail, color) correspond to the key features of the fish in memory, it will classify the object in front of it as "fish". In daily life, people may encounter different fish, including fish of various colors, various patterns, bulging eyes and normal eyes, long tails and short tails, etc. The human brain will continue to receive this information about fish, and through continuous comparison and connection, using the images of different fish and the corresponding labels, modify the activation of the visual cortex, the extraction of features, and the activation of neurons in the parietal and prefrontal cortex, and finally complete the classification and recognition of the object "fish".

[0063] Inspired by the image perception, feature extraction, and interpretable classification process in the human brain, this application proposes a high-definition image interpretable classification method. This method designs a high-definition image interpretable classification model (abbreviated as StyleGAN-FNN) based on a pre-trained StyleGAN model and a fuzzy neural network. The model involves three components: a conversion network T, a pre-trained generator G, and an interpretable classifier C. The model framework is as follows Figure 1 As shown. The model uses the latent space of pre-trained StyleGAN Through the transformation network T, low-dimensional, semantic, and fully decoupled features s are effectively extracted from complex image information to simulate the transmission of electrical signals in the visual pathway in the human brain and the feature extraction process of the visual cortex; the low-dimensional features s are input into the interpretable classifier C for reasoning and classification, simulating the process of the parietal lobe and prefrontal cortex of the human brain classifying the features transmitted from the visual cortex.

[0064] The interpretability of the high-definition image interpretable classification method proposed in this application is reflected in two aspects: one is the semantics of each dimension in the feature vector s used for classification, and the other is the IF-THEN fuzzy rules used for classification.

[0065] In one embodiment, Figure 2 As shown, a high-definition image interpretable classification method is provided, and the method includes the following steps:

[0066] Step 100: Encode the acquired high-definition image using a pre-trained StyleGAN encoder to obtain a latent code and construct a latent space.

[0067] Specifically, considering that the latent space of the pre-trained StyleGAN contains enough information in the image, it is a feasible method to extract features directly from the latent space rather than from the input image space. Extracting classification features directly from the latent space can avoid modifying the latent space, so that the model can focus on how to obtain decoupled features, further enhancing the effectiveness of the model's feature extraction and the comprehensibility of the generated explanations.

[0068] Step 102: extracting a low-dimensional, interpretable feature vector from the complex image information using a transformation network; the transformation network is used to utilize the feature direction in the latent space to obtain a feature vector corresponding to the latent code.

[0069] Specifically, the transformation network T is used to effectively extract low-dimensional, semantic, and fully decoupled features s from complex image information to simulate the transmission of electrical signals in the visual pathway in the human brain and the feature extraction process of the visual cortex.

[0070] The conversion network T is used to realize the transformation from latent space into the feature space For the conversion network T, the input is the potential coding sample w in the latent space, and the output is the feature vector s = (s1,...,s H ), where s h It represents the feature value of w on the hth feature. The larger the value, the stronger the semantics of w on this feature; the smaller the value, the weaker the semantics of w on this feature.

[0071] Step 104: The feature vector is inferred and classified using an interpretable classifier to obtain a classification result with interpretable categories; the interpretable classifier is used to simulate the reasoning and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time, with the help of the effective classification ability of the fuzzy neural network classifier, the accuracy of classification is ensured.

[0072] Specifically, considering the characteristics of human brain reasoning, the explainable classifier is implemented using fuzzy neural networks. With the help of linguistic variables and IF-THEN rules that conform to human thinking, it simulates the reasoning and classification process of the human brain. At the same time, with the effective classification ability of the fuzzy neural network classifier, the accuracy of classification is guaranteed.

[0073] In order to enable the fuzzy neural network to flexibly learn classification rules using multiple features generated by the conversion network, the structure of the fuzzy neural network was improved. In the fuzzification layer, an additional D node was added to each feature, representing "Don't care".

[0074] Step 106: Use the pre-trained generator to map the latent space to the image space to obtain a reconstructed high-definition image.

[0075] Specifically, the high-definition image interpretable classification model (abbreviated as StyleGAN-FNN) is composed of a conversion network, an interpretable classifier, and a pre-trained generator. Its structure is as follows Figure 1 shown.

[0076] In the above-mentioned HD image interpretable classification method, the method simulates the human brain's image perception, feature extraction, inductive reasoning, and adjustment learning process. This method is based on the latent space of the pre-trained StyleGAN, designs a conversion network to simulate the visual cortex of the brain for feature extraction, effectively converts high-dimensional latent codes into low-dimensional, decoupled classification features, and alleviates the problems of rule explosion and computational collapse; uses fuzzy neural networks to simulate the reasoning function of the parietal lobe and prefrontal cortex of the brain, and designs a fuzzy neural network structure improvement method adapted to the HD image interpretable classification model, realizing interpretable classification using low-dimensional features; visualizes and analyzes the fuzzy rules used for classification, and designs a feature visualization method based on the pre-trained StyleGAN generator, further improving interpretability.

[0077] In one embodiment, the implementation process of the conversion network in step 102 includes: first determining the form of the feature direction, then finding the feature direction from the latent space, and finally designing an eigenvalue measurement method to convert the feature direction into a feature vector.

[0078] Specifically, (1) define the feature direction

[0079] For the h-th feature, define the latent space The characteristic direction of this feature is in Existing literature mainly divides feature directions into two categories: feature directions that are irrelevant to samples and feature directions that are relevant to samples. Studies have shown that the feature directions that are relevant to samples have a more complete decoupling effect and are more interpretable. Therefore, the direction e of the hth feature is defined as h=f h (w), where f h (·) is the latent coding-feature mapping function, and the set of latent coding-feature mapping functions on H features is recorded as

[0080] (2) Solving the feature direction

[0081] Since the eigenvector s=(s1,...,s H ) is the input of the downstream classification task, so it is necessary to obtain domain-related, decoupled, and controllable features. Based on the above considerations, a supervised approach is used to obtain feature directions. Specifically, the meanings of H features are first defined (such as "smile", "age", "wearing glasses", etc. in the face dataset), and then an additional multi-classification classifier C′ is pre-trained for these features to serve as supervision to train In order to obtain orthogonal feature directions, a related loss function is introduced during the training process.

[0082] (3) Solving the eigenvalue

[0083] Given the feature direction e h , measure the score s of different potential codes w on this feature h , we can get the eigenvector s corresponding to the latent code w. The existing eigenvalue measurement method is usually a qualitative manual judgment. Specifically, by inputting different latent codes into the generator G to obtain the generated image, and then manually observing the degree of prominence of the corresponding features on the image, the relative size of the eigenvalue is obtained. For example, for gender features, assuming that the feature direction points to male, if the image G(w1) is male and G(w2) is female, then the eigenvalue of w1 in gender is higher than that of w2.

[0084] However, this method of measuring eigenvalues ​​by artificial vision only gives the relative results of the eigenvalue size, and does not give the quantitative eigenvalue value. For a given latent code sample w, the eigenvalue s at the hth feature h Formula (1) can be used to measure:

[0085]

[0086] Among them, n h is the feature direction that is independent of the sample, that is, the unit normal vector of the hyperplane in the latent space that can separate the two semantics of the feature; λ>0 is the coefficient.

[0087] In one embodiment, the feature value of a sample w in the latent space on the hth feature is the feature direction f of the sample w in the latent space. hThe cosine similarity of (w); the eigenvalue of sample w in the latent space on the hth feature is:

[0088]

[0089] Among them, s h is the eigenvalue of sample w in the latent space on the hth feature, f h (w) is the sample w and feature direction in the latent space;

[0090] The mathematical expression of the conversion network is:

[0091]

[0092] Among them, w is the potential code, T(w) is the output obtained after the potential code w is input into the conversion network, and s is the feature vector corresponding to the potential code. is the set of feature directions of w on all features, and ||·|| is the L1 norm.

[0093] Specifically, inspired by formula (1), after the feature direction is obtained through training, the feature value s of sample w on the hth feature is defined as h is w and the characteristic direction f h The cosine similarity of (w) is shown in formula (2). In summary, the conversion network T can be expressed by formula (3).

[0094] The conversion network T can use the feature direction in the latent space to obtain the feature vector corresponding to the latent code. In this way, the low-dimensional, interpretable feature vector s can be input into the interpretable classifier for classification.

[0095] In one embodiment, the method further includes a feature visualization method based on the pre-trained generator, and the specific steps of the feature visualization method include:

[0096] The latent code w is moved in the latent space so that the h-th feature of the generated image changes, and the latent code after the move is obtained as:

[0097] w new =w+δf h (w) (4)

[0098] Among them, w is the initial latent code, w new is the potential code after the shift, f h (w) is the direction of the hth feature, and δ is the moving step size;

[0099] According to the initial potential code and the potential code after the shift, the change of the potential code is determined as:

[0100] Δw=w new-w=δf h (w) (5)

[0101] According to the eigenvalue calculation formula of a potential coding sample on the hth feature (i.e., formula (2)), the eigenvalue s of the potential coding on the hth feature before and after the move is determined h and for:

[0102]

[0103] Because w new is the result of changing w on the hth feature, so f h (w new )≈f h (w), so we have:

[0104]

[0105] Let's assume So we have:

[0106]

[0107] According to the eigenvalue of the latent code on the hth feature before and after the movement, the change of the eigenvalue is determined as:

[0108]

[0109] According to the change of potential coding and the change of feature value, determine the h The mapping relationship to Δx is:

[0110]

[0111] Using Δs h The mapping relationship to Δx reflects the change in feature value in the potential code, which is then reflected in the generated image using the pre-trained generator.

[0112] Formula (10) establishes the equation from Δs h Using formula (10), the change of feature value can be easily reflected in the potential code, and then reflected in the generated image using the generator G. Figure 3 This is a schematic diagram of the feature visualization process in the high-definition image interpretable classification model, which demonstrates the proposed feature visualization method. The dotted path is the feature visualization path, which shows the impact of changes in feature values ​​on the generated image for specific features. It is an intuitive display of the variables used for classification in the fuzzy neural network.

[0113] Specifically, the specific meaning of each dimension in the feature vector s directly affects the intuitiveness of the antecedent of the fuzzy rule and the degree of people's understanding of the fuzzy rule. The visualization of features is usually obtained by traversing the latent space: since the feature vector s generates a new latent code w through the encoder network, the change of s will directly cause the change of w, and then react to the generated image with the help of the generator G. Therefore, by constantly modifying s and displaying and comparing the generated images, the visualization of the features can be completed and the specific meaning of s can be found. However, in this method, since s is directly converted from the latent code w, the change of s cannot affect the generated image by affecting w; in addition, due to the complexity of the conversion network T, it is also very complicated to find the inverse function of T. For this reason, it is necessary to design a visualization method for the eigenvalues ​​for this model. It should be noted that, although according to the introduction of the feature direction solution method in the previous article, the H features are determined in advance, that is, the meaning of each feature is determined, so the eigenvalue visualization method seems to be somewhat redundant. However, when using other feature direction solution methods, especially unsupervised methods, new eigenvalue visualization methods are very important.

[0114] In one embodiment, if Figure 4 As shown, the interpretable classifier in step 104 includes a fuzzification layer, a fuzzy logic layer, and a classification layer. The fuzzification layer is used to fuzzify the input feature vector s; the fuzzy set is represented by a Gaussian membership function; the fuzzification layer has a total of 4×H nodes, in which, in addition to assigning 3 membership functions to each feature, corresponding to the terms low, medium, and high, an additional D node is added to each feature; the D node is initialized in a random manner and is continuously adjusted during the training process; when the antecedent of a fuzzy rule contains a D node, it means that the feature corresponding to the D node is ignored, and the fuzzy rule is rewritten as a rule with a simpler antecedent.

[0115] Specifically, when the antecedent of a fuzzy rule contains D nodes, it means that the corresponding feature is ignored, and the fuzzy rule can be rewritten as a rule with a more concise antecedent. For example, if a rule extracted after training is as shown in formula (11):

[0116]

[0117] It can be rewritten as formula (12):

[0118]

[0119] The “Don’t care” nodes are initialized randomly and adjusted during the training process. The addition of “Don’t care” nodes makes the feature selection and learning in the fuzzy neural network training process more flexible.

[0120] The hth feature sh The output of the zth fuzzification layer node is expressed using formula (13), where z∈{1,2,3}; Representation features h In the fuzzy set A hz The fuzzy value on h Take the value of node D, μ hz and σ hz represent the mean and standard deviation of the Gaussian membership function respectively.

[0121]

[0122] The fuzzy logic layer is used to represent the fuzzy rule base. To achieve this goal, the parameters of this layer are set to discrete 0 / 1 values. If θ (l) Represents the adjacency matrix of the lth layer and the l-1th layer in the model, then for the fuzzy logic layer, in is θ (l) The element in the i-th row and j-th column in , l = 2, ..., L-1. When the weights of the fuzzy logic layer are discrete 0 / 1 values, the model can be directly converted into a set of fuzzy rules with good interpretability.

[0123] In order to make the fuzzy logic layer of the model able to represent various forms of fuzzy rules, each fuzzy logic layer contains several fuzzy intersection nodes and fuzzy union nodes, which respectively perform fuzzy intersection operations (and operations) and fuzzy union operations (or operations). The fuzzy union nodes of layer l together constitute the fuzzy union sublayer l=2,...,L-1. Let The output of the i-th node in the layer is The output of the i-th node in the layer is According to the meaning of fuzzy intersection operation and fuzzy union operation, the calculation of the two is shown in formula (14):

[0124]

[0125] in, represents the output of the jth node in the l-1th layer, θ (l,t) express The adjacency matrix between the layer and the l-1 layer, where the elements express The connection weight between the i-th node of the layer and the j-th node of the l-1-th layer; similarly, θ (l,k) express The adjacency matrix between the layer and the l-1 layer, where the elements express The connection weight between the i-th node in the layer and the j-th node in the l-1-th layer.

[0126] In this application, the fuzzy intersection operation uses the min operator, and the fuzzy union operation uses the max operator. The fuzzy intersection layer nodes and fuzzy union layer nodes of the fuzzy logic layer essentially perform fuzzy intersection and fuzzy union operations on the upper layer nodes with a connection edge weight of 1. Considering that each fuzzy logic layer node represents a preceding connective in a fuzzy rule, the output value of the fuzzy logic layer node is s h The activation degree on the corresponding fuzzy rule. The input of the classification layer is the activation degree value calculated by each fuzzy logic layer node connected to the classification layer. Then the goal of the classification layer is to merge the activation degree values, output the scores of each category, and finally get the final classification category. In summary, the number of nodes in the classification task layer is the same as the number of classification categories, which is C. The final output of the classification layer is shown in formula (15):

[0127]

[0128] in, Represents the classification category of the final output, represents the output of the jth fuzzy logic layer node connected to the i-th classification layer node, θ (L) represents the weight matrix of the classification layer, It can be seen that in the classification layer, the weights correspond to the consequent weights of the fuzzy rules.

[0129] Fuzzy neural networks are trained using the gradient descent method. After training, the feature vector s is input into the fuzzy neural network, and the corresponding output category can be obtained through forward calculation.

[0130] In one embodiment, the pre-trained generator in step 106 is a pre-trained StyleGAN generator.

[0131] In one of the embodiments, according to the structural design of the fuzzy neural network model of the explainable classifier, after the model is trained, the fuzzy IF THEN rules are directly extracted from the model.

[0132] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0133] In a verification example, in order to test the effect of the model, the StyleGAN-FNN model was trained on the high-definition face dataset CelebA-HQ, and the features extracted by the model were analyzed using the proposed feature visualization method (see formula (10)) to show the specific meaning and decoupling effect of the features. Finally, the fuzzy rules extracted from the fuzzy neural network were combined to further verify the model's ability to interpretably classify high-definition images.

[0134] (1) Dataset

[0135] The StyleGAN model is pre-trained using the FFHQ dataset to obtain the generator G; the conversion network T is trained using the CelebA-HQ dataset with a resolution of 1024×1024. Figure 1 The training and classification of the StyleGAN-FNN model requires a latent space As input, the pre-trained StyleGAN encoder proposed in the literature (1) (E. Richardson, Alaluf Y, Patashnik O, et al. Encoding in Style: A StyleGAN encoder for image-to-image translation [C] / / Proceedings of the 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 2287-2296, doi: 10.1109 / CVPR46437.2021.00232.) is used to calculate the corresponding latent code w in the latent space for each image in the CelebA-HQ dataset.

[0136] When training the StyleGAN-FNN model, the “attractive” label provided by the CelebA-HQ dataset is used to classify the images into two categories: a classification of 0 means that the person in the image is not attractive; a classification of 1 means that the person in the image is attractive. The reason why “attractive” is chosen as the classification label is that compared with other classification tasks (such as gender classification), this classification problem is more subjective and more complex. By observing and analyzing the classification results and basis of the model on this classification task, it is expected to obtain some interesting conclusions. For the “attractive” binary classification task, 8 attributes are manually selected from the 40 attributes provided by the CelebA-HQ dataset as classification features, as shown in Table 1. Then, according to formula (2), the feature vector s = [s1,...,s8] of each image in the CelebA-HQ dataset on these 8 features is calculated, and it is combined with the attractive label to form a classification dataset to train StyleGAN-FNN. Among them, 28,000 images are used for training and 2000 images are used for testing.

[0137] Table 1 Feature information of the “Attractive” binary classification task

[0138] Feature number Feature Name 1 Baldness 2 young 3 Open mouth slightly 4 Apply lipstick 5 Bangs 6 curls 7 Willow Leaf Eyebrows 8 Pointed nose

[0139] (2) Parameter settings

[0140] In the StyleGAN-FNN model, in order to further improve the classification accuracy, a truncated Gaussian fuzzy number is used, and the mean parameter of the membership function is shown in formula (16):

[0141]

[0142] Where Q1 and Q3 represent the quarter quantile and the third quarter quantile respectively; the superscript h indicates the index of the feature, and the second value of the subscript indicates the data used to calculate the quantile: h The data is divided into two parts according to the attractive label, s h,0 and h,1 , then Q 1,- =min(Q1(s h,0 ),Q1(s h,1 ));Q 3,+ =max(Q3(s h,0 ),Q3(s h,1 )). Standard deviation of the membership function Set to 0.3, where MF∈{L,M,H}, “L”, “M”, and “H” are the abbreviations of “low”, “medium”, and “high” respectively. For the two membership functions “L” and “H”, the calculation of the membership value is shown in formula (17):

[0143]

[0144] in Represents the membership value of the h-th eigenvalue on the membership function MF. Figure 5 The membership function of the first classification feature (baldness) is shown. The truncated Gaussian fuzzy number can better avoid the influence of extreme values ​​on the membership function setting and improve the classification accuracy.

[0145] In the fuzzy neural network, the number of fuzzy logic layers is set to 1, and the number of fuzzy logic layer nodes is set to 16, including 8 fuzzy intersection nodes and 8 fuzzy union nodes.

[0146] (3) Results analysis

[0147] For the trained model, we first use correlation analysis and feature visualization methods to demonstrate the meaning of the model-generated features and the decoupling effect; then we display and analyze the classification results and classification basis of the fuzzy neural network.

[0148] 1) Decoupling effect of feature space

[0149] ① Correlation matrix

[0150] Table 2 numerically shows the orthogonality of the eight feature directions used by the model in the form of a correlation matrix. The correlation between feature directions is measured using cosine similarity. Specifically, 5,000 images are randomly selected from the training set, and the feature direction f of each image on the eight classification features is calculated. h (w), h = 1, 2, ..., 8. Then, the value of the i-th row and j-th column in the correlation matrix is ​​cos(f i (w),f j (w)), where cos(·) represents the cosine similarity calculation function. As can be seen from Table 2, each feature direction is almost orthogonal.

[0151] Table 2 Correlation matrix of eight features

[0152] Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Feature 6 Feature 7 Feature 8 Feature 1 - 0.06 -0.01 0.09 0.09 -0.05 0.01 0.06 Feature 2 - - -0.12 -0.10 -0.08 0.05 -0.01 -0.08 Feature 3 - - - -0.03 0.00 -0.00 0.03 0.01 Feature 4 - - - - -0.01 -0.04 0.00 0.01 Feature 5 - - - - - 0.02 0.04 0.02 Feature 6 - - - - - - 0.00 -0.02 Feature 7 - - - - - - - -0.02 Feature 8 - - - - - - - -

[0153] ②Feature visualization

[0154] In order to intuitively demonstrate the decoupling of features, two pictures were randomly selected from the dataset, such as Figure 6As shown, (a) is picture 1 and (b) is picture 2.

[0155] The proposed feature visualization method (see formula (10)) is used to visualize the eight classification features, where the change Δs of the hth feature is h ∈{-1,-0.5,0,0.5,1}. The feature visualization on image 1 is as follows Figure 7 As shown, each column represents the generated image corresponding to the change of the eigenvalue of a specific feature; each row represents the generated image corresponding to different features after a given eigenvalue change.

[0156] from Figure 7 It can be seen that for a single feature, changing the size of the eigenvalue will correspondingly change the visual effect of the corresponding feature in the generated image, and the trend of the feature change in the image corresponds to the meaning of the feature: the larger the eigenvalue, the stronger the semantics corresponding to the feature, and vice versa. The semantics corresponding to the feature is weaker. For example, for the first feature (baldness), the smaller the eigenvalue, the thicker the hair of the person in the generated image, and the smaller the degree of baldness; the larger the eigenvalue, the more obvious the baldness of the person in the generated image; for the eighth feature (pointed nose), the smaller the eigenvalue, the flatter the nose of the person in the generated image; the larger the eigenvalue, the pointier the nose of the person in the generated image. This shows the accuracy of the model feature extraction and the effectiveness of the feature visualization method.

[0157] from Figure 7 It can also be seen that the feature decoupling learned by the model is relatively complete: the change of a single feature in the generated image will not affect the visual effects of the remaining features. For example, when the fourth feature (lipstick) is changed, except for the color of the lipstick, the person's hairstyle, eyebrow shape, nose, etc. are all maintained; the same is true when changing other features. This shows that StyleGAN-FNN has greater advantages in feature decoupling.

[0158] In order to prevent the selected images from being peculiar, feature visualization is also performed on Image 2, such as Figure 8 As shown. It can be seen that Figure 7 Similarly, changing the eigenvalue of a single feature will affect the visual effect of the corresponding feature in the generated image without affecting the remaining features.

[0159] 2) Fuzzy rules for classification

[0160] The model was trained using the constructed "attractive" binary classification dataset. After training, the model's accuracy on the training set and test set was 74.7% and 73.1% respectively; using the pre-trained multi-classifier C' as the baseline, the classification accuracy on the training set and test set was 86.5% and 85.9% respectively. Below, the fuzzy rules used by the fuzzy neural network are displayed, and the fuzzy rules used by the fuzzy neural network to classify a single image are analyzed.

[0161] ①Fuzzy rule base

[0162] Fig. 9 The fuzzy rules extracted from the trained fuzzy neural network are shown.

[0163] Fig. 9 In the figure, c0 indicates category 0 (not attractive), c1 indicates category 1 (attractive), and the rule consequences are represented by bar graphs, which represent the weights of the rules for classifying images into various categories. It can be seen that the fuzzy neural network uses a total of 7 rules to classify the images in the CelebA-HQ dataset into two categories of "attractive or not", and the antecedent of each rule is relatively simple. Except for the antecedent of the last rule, which is an AND operation of the fuzzy sets of two features, the antecedents of the other rules only contain a fuzzy set of one feature. This is related to the design of the "Don't care" node. It can also be seen from the rule antecedents that the fuzzy neural network only classifies "attractive or not" based on the first four features (bald, young, slightly open mouth, and lipstick). This is consistent with the classification logic of "attractive or not": when people judge whether the characters in the image are attractive, they usually judge based on features such as age, whether they are bald, and whether they are wearing makeup, while eyebrow shape, curly hair, and bangs do not affect a person's attractiveness. This shows that the model can automatically select features that are useful and truly relevant to classification when providing explanations. The specific data of fuzzy rules are shown in Table 3.

[0164] Table 3 Fuzzy rule base data

[0165]

[0166]

[0167] In addition, combined with Table 3, it can be seen that when the values ​​of the first feature (bald) and the third feature (slightly open mouth) are small, the classifier tends to classify it as the first category (attractive); when the value of the second feature (young) is small, the classifier tends to classify it as the 0th category (unattractive); when the value of the third feature (slightly open mouth) is relatively average, the classifier tends to classify it as the 0th category (unattractive); when the value of the fourth feature (lipstick) is large, the classifier tends to classify it as the first category (attractive); when the value of the first feature (bald) is large and the value of the fourth feature (lipstick) is small, the classifier tends to classify it as the 0th category (unattractive). The classification basis provided by the rule base is consistent with human cognition: when people judge whether a person is attractive, they will also classify bald, old, and unlipsticked people as unattractive, and classify people with thick hair, young, and lipsticked as attractive. It should be noted here that for the third feature (mouth slightly open), the rule only reflects the classification preference when the feature value is at the average level ("M"), which is also in line with human intuition: when the mouth is not open, it means that the person may not be smiling or smiling with pursed lips; when the mouth is open to a large extent, it means that the person may be laughing, and when the mouth is opened to a moderate degree, the person will appear a little dull (see Figure 8 (4th row, 3rd column).

[0168] By demonstrating and analyzing the fuzzy rule base used by the fuzzy neural network in image classification, the interpretable classification capability of high-definition images using the StyleGAN-FNN model is verified.

[0169] ②Example explanation

[0170] In order to more intuitively demonstrate the image classification and interpretability of the model, the fuzzy rules used by the fuzzy neural network to classify a single image are analyzed.

[0171] Fig.10 is a randomly selected "unattractive" image (class label 0) and the fuzzy rules used by the fuzzy neural network to classify the image. Fig.10 (a) shows the original image and category labels; Fig.10 (b) shows the fuzzy rules used by the fuzzy neural network to classify the image and the "contribution" of these rules to the classification. Fig.10 The figure in (b) shows the activation degree, rule antecedent and rule number of the fuzzy rule. Fig.10 The right side of (b) shows the contribution of fuzzy rules to classification, where values ​​above 0 indicate positive contributions and values ​​below 0 indicate negative contributions. For two classification categories, the longer the bar above 0 and the shorter the bar below 0, the higher the probability that the classifier classifies the image into the corresponding category. The specific data of fuzzy rules are shown in Table 4.

[0172] Table 4 Fuzzy rules used when classifying a single image of category 0

[0173] Rule number Rule antecedents and activation levels Contribution to category 1 <![CDATA[s 1 spore(0.02)]]> 0.0006,0.0087 2 <![CDATA[s 1 isgood(0.95)]]> 0.8582,-0.5895 3 <![CDATA[s 2 spore(1.00)]]> 0.5194,-0.3571 4 <![CDATA[s 3 spore(0.006)]]> -0.0023,0.0029 5 <![CDATA[s 4 isgood(0.0002)]]> -0.0000,0.0001

[0174] from Fig.10 It can be seen that when classifying this image, the fuzzy neural network can easily classify it as category 0 (not attractive), because among the 5 rules used for classification, the 7th and 3rd rules, which contribute more to the classification results, both support the classification as category 0 (the contribution of the remaining three rules is so small that it is almost invisible on the bar graph). Combining the semantics of each feature, it can be seen that because the first eigenvalue is large (severe baldness) and the second eigenvalue is small (older), it supports the classification of the image as category 0 (not attractive). The reasoning process of the above fuzzy rules is consistent with human intuition.

[0175] same, Fig.11 The figure shows the fuzzy rules used by the classifier to classify an image that is "attractive" (class label is 1), where (a) is the original image and class label, and (b) is the fuzzy rules and their contributions. See Table 7 for specific data.

[0176] It can be seen that when classifying this image, the fuzzy neural network can relatively easily classify it as category 1, because the 5 rules used for classification consider it to be more likely to be classified as category 1. In addition, according to Fig.11 In (b), the classifier mainly classifies it as category 1 based on the 4th rule (the 3rd eigenvalue is small) and the 6th rule (the 4th eigenvalue is large). In connection with the semantics of each feature, it can be seen that because the 3rd eigenvalue is small (the mouth is not open) and the 4th eigenvalue is large (lipstick is applied), it supports the classification of the image as category 1 (attractive). The above reasoning process is also consistent with human reasoning.

[0177] Table 7 Fuzzy rules used when classifying a single image of category 1

[0178]

[0179]

[0180] By demonstrating the fuzzy rules used by the fuzzy neural network in classification and analyzing the rules used by the classifier when classifying a single image, the classification ability and interpretability of the model in handling image classification tasks are verified.

[0181] The StyleGAN-FNN model for classifying high-definition images proposed in this application draws on the human brain's process of feature extraction, inductive reasoning, and adjustment learning of complex information, automatically extracts decoupled and key feature elements from high-definition image information, and completes high-precision and interpretable classification through an interpretable fuzzy neural network classifier. A model architecture including a conversion network, a pre-trained StyleGAN generator, and a fuzzy neural network is designed, so that the model can directly convert high-dimensional latent space into low-dimensional feature space, greatly enhancing the decoupling effect of features; in terms of model interpretability, the structure of the fuzzy neural network is improved, the number of fuzzy rule antecedents is reduced, and the interpretability of fuzzy rules is improved. In addition, an accurate and fast feature visualization method is proposed to intuitively and clearly display the semantics of the features; in the "Is it attractive" binary classification experiment on the CelebA-HQ dataset, it is shown that the proposed model can extract fully decoupled features from the latent space, and use these features to accurately and interpretably complete the high-definition image classification task.

[0182] In one embodiment, Fig.12 As shown, a high-definition image interpretable classification device is provided, including: a latent coding determination module of a pre-trained StyleGAN, a feature extraction module, an interpretable classification module and a visualization module, wherein:

[0183] A latent code determination module of the pre-trained StyleGAN is used to encode the acquired high-definition image using the pre-trained StyleGAN encoder to obtain a latent code, thereby converting the input image space into a latent space;

[0184] The feature extraction module is used to extract low-dimensional, interpretable feature vectors from complex image information using a transformation network. The transformation network is used to use the feature direction in the latent space to obtain the feature vector corresponding to the latent code.

[0185] The interpretable classification module is used to use an interpretable classifier to infer and classify feature vectors to obtain interpretable classification results. The interpretable classifier is used to simulate the reasoning and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time, with the effective classification ability of the fuzzy neural network classifier, the accuracy of classification is guaranteed.

[0186] The visualization module is used to map the latent space to the image space using the pre-trained generator to obtain a reconstructed high-definition image.

[0187] In one of the embodiments, the implementation process of the conversion network in the feature extraction module includes: first determining the form of the feature direction, then finding the feature direction from the latent space, and finally designing an eigenvalue measurement method to convert the feature direction into a feature vector.

[0188] In one embodiment, the feature value of a sample w in the latent space on the hth feature is the feature direction f of the sample w in the latent space. h The cosine similarity of (w); the eigenvalue of sample w in the latent space at the hth feature is shown in formula (2); the mathematical expression of the conversion network is shown in formula (3).

[0189] In one embodiment, the device further includes a feature visualization module based on a pre-trained generator, the feature visualization module being used to change the h-th feature of the generated image by moving the potential code w in the potential space, thereby obtaining the potential code after the move as shown in formula (4); determining the change of the potential code according to the initial potential code and the potential code after the move; the change of the potential code is as shown in formula (5); determining the feature value of the potential code on the h-th feature before and after the move according to the feature value calculation formula of a potential code sample on the h-th feature; the feature value of the potential code on the h-th feature before and after the move is as shown in formula (6); determining the change of the feature value according to the feature value of the potential code on the h-th feature before and after the move; the change of the feature value is as shown in formula (9); determining the change from Δs according to the change of the potential code and the change of the feature value h To Δx, from Δs h The mapping relationship from Δs to Δx is shown in formula (10); h The mapping relationship to Δx reflects the change in feature value in the potential code, which is then reflected in the generated image using the pre-trained generator.

[0190] In one of the embodiments, the interpretable classifier in the interpretable classification module includes a fuzzification layer, a fuzzy logic layer and a classification layer; in the fuzzification layer, an additional D node is added for each feature; the D node is initialized in a random manner and is continuously adjusted during the training process; when the antecedent of the fuzzy rule contains a D node, it means that the feature corresponding to the D node is ignored, and the fuzzy rule is rewritten as a rule with a simpler antecedent.

[0191] In one embodiment, the pre-trained generator in the visualization module is a pre-trained StyleGAN generator.

[0192] In one of the embodiments, according to the structural design of the fuzzy neural network model of the explainable classifier, after the model is trained, the fuzzy IF THEN rules are directly extracted from the model.

[0193] For the specific definition of the high-definition image interpretable classification device, please refer to the definition of the high-definition image interpretable classification method above, which will not be repeated here. Each module in the above-mentioned high-definition image interpretable classification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0194] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a high-definition image interpretable classification method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0195] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0196] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0197] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0198] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0199] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A high-definition image interpretable classification method, characterized in that: The method comprises: The acquired high-definition image is encoded using the pre-trained StyleGAN encoder to obtain the latent code and construct the latent space; The latent code is extracted from complex image information using a transformation network to obtain a low-dimensional, interpretable feature vector; the transformation network is used to obtain a feature vector corresponding to the latent code using a feature direction in the latent space; The feature vector is inferred and classified by using an interpretable classifier to obtain a classification result with interpretable categories; the interpretable classifier is used to simulate the reasoning thinking and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time, with the help of the effective classification ability of the fuzzy neural network classifier, the accuracy of classification is ensured; The latent space is mapped to the image space using a pre-trained generator to obtain a reconstructed high-definition image.

2. The high-definition image interpretable classification method according to claim 1, characterized in that: The implementation process of the conversion network includes: first determining the form of the feature direction, then finding the feature direction from the latent space, and finally designing the eigenvalue measurement method to convert the feature direction into a feature vector.

3. The high-definition image interpretable classification method according to claim 2, characterized in that: The eigenvalue of sample w in the latent space on the hth feature is the sample w in the latent space and the feature direction f h The cosine similarity of (w); the eigenvalue of sample w in the latent space on the hth feature is: Among them, s h is the eigenvalue of sample w in the latent space on the hth feature, f h (w) is the feature direction of sample w in the latent space; The mathematical expression of the conversion network is: Among them, w is the potential code, T(w) is the output obtained after the potential code w is input into the conversion network, and s is the feature vector corresponding to the potential code. is the set of feature directions of w on all features, and ||·|| is the L1 norm.

4. The high-definition image interpretable classification method according to claim 3, characterized in that: The method further includes a feature visualization method based on a pre-trained generator, and the specific steps of the feature visualization method include: The latent code w is moved in the latent space so that the h-th feature of the generated image changes, and the latent code after the move is obtained as: w new =w+δf h (w) Among them, w is the initial latent code, w new is the potential code after the shift, f h (w) is the direction of the hth feature, and δ is the moving step size; According to the initial potential code and the potential code after the shift, the change of the potential code is determined as: Δw=w new -w=δf h (w) According to the eigenvalue calculation formula of a potential coding sample on the hth feature, the eigenvalue of the potential coding on the hth feature before and after the movement is determined as: According to the eigenvalue of the latent code on the hth feature before and after the movement, the change of the eigenvalue is determined as: According to the change of the potential code and the change of the feature value, determine the h The mapping relationship to Δx is: Δx=x new -x =G(w new )-G(w) =G(w+Δw)-G(w) =G(w+αf h (w))-G(w) ≈G(w+Δs h f h (w))-G(w) Using Δs h The mapping relationship to Δx reflects the change in feature value in the potential code, which is then reflected in the generated image using the pre-trained generator.

5. The high-definition image interpretable classification method according to claim 1, characterized in that: The interpretable classifier includes a fuzzification layer, a fuzzy logic layer, and a classification layer; In the fuzzification layer, an additional D node is added for each feature; the D nodes are initialized in a random manner and are continuously adjusted during the training process; When the antecedent of a fuzzy rule contains a D node, it means that the feature corresponding to the D node is ignored and the fuzzy rule is rewritten as a rule with a more concise antecedent.

6. The high-definition image interpretable classification method according to claim 1, characterized in that: The pretrained generator is a pre-trained StyleGAN generator.

7. The high-definition image interpretable classification method according to claim 1, characterized in that: According to the structural design of the fuzzy neural network model of the interpretable classifier, after the model is trained, the fuzzy IF THEN rules are directly extracted from the model.

8. A high-definition image interpretable classification device, characterized in that: The device comprises: A latent code determination module of the pre-trained StyleGAN is used to encode the acquired high-definition image using the pre-trained StyleGAN encoder to obtain a latent code, thereby converting the input image space into a latent space; A feature extraction module is used to extract a low-dimensional, interpretable feature vector from the complex image information using a transformation network; the transformation network is used to obtain a feature vector corresponding to the latent code using the feature direction in the latent space; An interpretable classification module is used to use an interpretable classifier to infer and classify the feature vector to obtain a classification result with an interpretable category; the interpretable classifier is used to simulate the reasoning thinking and classification process of the human brain with the help of linguistic variables and IF-THEN rules, and at the same time, with the help of the effective classification ability of the fuzzy neural network classifier, to ensure the accuracy of classification; A visualization module is used to map the latent space to an image space using a pre-trained generator to obtain a reconstructed high-definition image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the high-definition image interpretable classification method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-definition image interpretable classification method described in any one of claims 1 to 7 is implemented.

Images