Fuzzy category image classification system based on closed-loop causal generation
By introducing closed-loop causal generation technology into the image generation model, the causal characteristics of categories are specifically controlled, and the problem of difficulty in distinguishing fuzzy category samples in the existing technology is solved, and high-accuracy image classification and generation are achieved.
Patent Information
- Application Number
- CN202411987476.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-29
AI Technical Summary
The prior art is difficult to distinguish fuzzy category samples, and cannot simultaneously generate image and specifically control the causal characteristics of the hidden space.
A fuzzy class image classification system based on closed-loop causal generation is adopted to specifically control the category causal feature expression of the target in the generative model and use it in the image binary classification task, thereby improving the classification accuracy of the model for fuzzy samples.
The classification accuracy of the model for fuzzy samples is improved, the hidden space causal characteristics of the image are enhanced, and the authenticity and classification performance of the generated image are ensured.
Smart Images
Figure CN119904693A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of images, in particular to a fuzzy category image classification system based on closed-loop causal generation. Background Art
[0002] Existing fuzzy sample classification technologies include: machine learning methods based on manually extracting features and building classifiers; and using deep learning methods to adaptively obtain target features and adopt convolutional neural networks (CNN) or Transformer architectures for classification. These methods improve classification performance through various image encoding techniques or network architectures, but existing technologies have difficulty distinguishing fuzzy category samples and cannot generate images, and cannot specifically control and enhance the latent space causal features of images. Summary of the invention
[0003] In order to solve the problem that the existing technology is difficult to classify when the image features are not obvious, the present invention proposes a fuzzy category image classification system based on closed-loop causal generation. By specifically controlling the category causal feature expression of the target in the generation model and using it in the image binary classification task, the classification accuracy of the model for fuzzy samples is improved.
[0004] The present invention is achieved through the following technical solutions:
[0005] The present invention relates to a fuzzy category image classification system based on closed-loop causal generation, comprising: a data reading module, a causal generator module and a discriminator module for training, wherein: the data reading module performs image cutting unit, image normalization unit and format conversion unit processing according to input three-dimensional image information to obtain data information that can be read and recognized by subsequent modules; the causal generator module performs causal image generation processing according to the processing result of the received data reading module through a closed-loop image generation branch to obtain an enhanced version of the original image; then an accurate classification result is obtained according to the enhanced version of the original image through an image classification branch; the discriminator module performs image authenticity identification on the enhanced version of the original image according to the processing result of the causal generator to obtain an image authenticity score, thereby completing adversarial training with the generator and ensuring the authenticity of the image generated by the causal generator. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 It is a schematic diagram of the system of the present invention;
[0007] Figure 2 This is a schematic diagram of the causal perception unit;
[0008] Figure 3 It is a schematic diagram of the effect of the embodiment;
[0009] In the figure: (a) is a schematic diagram of visual effects, (b) is a schematic diagram of causal feature distribution, (c) is a schematic diagram of sample distribution, and (d) is a schematic diagram of interpretability based on machine learning features;
[0010] Figure 4 It is a schematic diagram of the effect of the embodiment;
[0011] In the figure: (a) is a schematic diagram of the visual effect, (b) is a schematic diagram of the sample distribution, and (c) is a comparison of the classification task effects. DETAILED DESCRIPTION
[0012] like Figure 1 As shown, this embodiment relates to a fuzzy category image classification system based on closed-loop causal generation, including: a data reading module, a causal generator module and a discriminator module for training.
[0013] The data reading module includes: an image cutting unit, an image normalization unit and a format conversion unit, wherein: the image cutting unit cuts the input fuzzy category 3D image to obtain a bounding box including the target area to be classified, the image normalization unit normalizes the image in the bounding box to the range of 0-1, and the format conversion unit converts the normalized image from physical pixel value parameters to a tensor type original image that can be recognized by the deep learning network.
[0014] The closed-loop image generation branch of the causal generator module includes: a first encoder, a causal perception unit and a decoder, wherein: the first encoder encodes the tensor type original image into a high-dimensional latent space to obtain latent space features; the causal perception unit performs causal screening and category counterfactual processing based on the latent space features to obtain counterfactual template features for guiding the latent space features; the decoder decodes according to the latent space features, restores to the original image resolution, obtains the generated image and re-inputs it into the first encoder to form a closed-loop path, and repeats the above steps N times until an enhanced version image of the original image with a clear category is generated.
[0015] The guidance refers to: in the process of forming a closed-loop path, the counterfactual template features are used to guide the latent space features obtained after the enhanced version image passes through the encoder, so that the feature distribution of the enhanced version image is as close as possible to the template feature distribution.
[0016] like Figure 2 As shown, the causal screening refers to: determining whether the absolute value of the weight parameter in the frozen classification head corresponding to the latent space feature is in the top 25%, and then retaining the causal feature and removing irrelevant features to obtain a causal feature set.
[0017] The category counterfactual processing refers to: judging whether the positive and negative signs of the weight parameters in the frozen classification head corresponding to the features in the causal feature set are the same as the categories of the input fuzzy category 3D image, thereby enhancing the category features and removing the non-category features to obtain the counterfactual template features.
[0018] The counterfactual template features Among them: F is the sum of the causal feature set obtained after the causal screening step, C is the enhancement strength, and max(F) represents the maximum value in the category feature.
[0019] like Figure 1 As shown, the image classification branch includes: a second encoder and a classification head, wherein: the second encoder encodes the enhanced version of the original image into high-dimensional latent space features; the classification head obtains accurate fuzzy category sample classification results based on the high-dimensional latent space features.
[0020] The discriminator module described in this embodiment adopts a multi-scale convolutional network with gradual downsampling.
[0021] The image classification method of this embodiment based on the above system includes: a training phase and a testing phase.
[0022] The training phase includes:
[0023] Step 1: Pre-train image classification branch: The image classification branch obtains the latent space feature F1 after encoding the input original image I through the encoder θ. The classification head τ classifies F1 and obtains the classification score S1, which is compared with the real category Target of the image to calculate the classification loss. And complete the pre-training of encoder θ and classification head τ;
[0024] Step 2: Jointly train the closed-loop image generation branch and the discriminator module, specifically including:
[0025] 2.1 Input the original image I into the first encoder θ pre-trained in step 1 to obtain the latent space feature F i ,i∈[1,N], where i represents the i-th iteration and N is the total number of iterations in the closed-loop generation. According to the latent space feature F i , i∈[1,N] to get the enhanced version of the original image G i ,i∈[1,N]. In the first generation process of the above closed-loop generation, the causal perception unit obtains the counterfactual template feature F through the latent space feature F1 cf And calculate the counterfactual loss F i+1 is the generated image G of the i-th round i The latent space feature obtained after passing the encoder θ, F cfis the counterfactual template feature.
[0026] 2.2 Discriminator module γ is an enhanced version of the original image G i Generate identification result R i ∈[0,1], the identification result is the authenticity probability of the image, between 0 and 1, where 0 and 1 represent the boundary probabilities of a fake image or a real image, respectively.
[0027] 2.3 Alternating adversarial training of closed-loop image generation branch and discriminator module: The total loss of training closed-loop image generation branch Among them: classification loss S i+1 To generate the image G of the i-th round i The classification score generated by the feedback to the frozen encoder and classification head τ, Target is the real image category, and the reliability loss G i is the i-th round latent space feature F i Through the decoder The generated image obtained after decoding, I is the original image, authenticity loss R i Represents G i The identification result obtained after the discriminator γ, λ i ,i∈[1,4] is the weight of different losses, w i ,i∈[1,n] is the weight of different iterations in closed-loop generation; the total loss of training the discriminator module in: Denotes the discriminator module γ given G i The authenticity probability R of the output under the condition i With G i The actual distribution distance between the ground-truth labels, Same reason.
[0028] The testing phase specifically includes:
[0029] Step 1: The data reading module reads the input fuzzy category 3D image.
[0030] Step 2: The closed-loop image generation branch generates images in a closed-loop manner based on the original image, and finally obtains an enhanced version of the original image with a clear category.
[0031] Step 3: Send the enhanced version of the image to the image classification branch to obtain accurate fuzzy category sample classification results.
[0032] After specific practical experiments, the specific environment settings are as follows: The method is verified and tested on a dataset of 278 blurred sample 3D images, of which 171 positive samples and 107 negative samples are confirmed. The dataset is randomly divided into a training set (80%) for model development and an independent test set (20%) for performance evaluation. To determine the best hyperparameters, five-fold cross validation is used in the training phase.
[0033] The specific parameters are set as follows: During image preprocessing, the original image is cropped to include: the ROI of the entire target to be classified, and then the ROI size is adjusted to 128×128×64. In order to enrich the training data and improve the robustness of the model, data augmentation techniques are used during training, including: random rotation in the range of [-15°~15°], random cropping of [0.6~1.0] ratio, and B-spline deformation. The initial learning rate and weight decay of the Adam optimizer are set to 1e-4 and 5e-4, respectively. Training is performed on a 32GB Tesla V100-SXM2 GPU for a total of 1e4 iterations. The batch size is set to 4. For the present invention, the total number of closed-loop generation n is set to 3, and the weights w1, w2, and w3 are 0.5, 1, and 1.5, respectively. In the loss function, λ1, λ2, λ3, and λ4 are set to 10, 1, 1, and 1, respectively, and the constant C in the category counterfactual formula is set to 2. The network is implemented in PyTorch.
[0034] like Figure 3 As shown, in order to intuitively analyze the images generated by the present invention, the focus is on image visualization, causal feature statistics, sample distribution and interpretability. In the positive sample, as shown in the original image, there is a lack of obvious grayscale and texture differences between the liquid and solid components in the target. The inhomogeneity of texture and grayscale is an important sign of positive samples and a key causal feature. The generated image significantly highlights the solid components in the target, resulting in a significant increase in texture and grayscale heterogeneity. Therefore, the generated image improves the classification score from 0.49 to 0.77. Given a threshold of 0.5, the prediction of this case changes from negative to positive. In contrast, for negative samples, the generated image enhances the low-grayscale liquid component within the target. It is worth noting that the present invention emphasizes the tiny partitions within the target, as shown by the yellow arrows, which are typical features of negative samples. Therefore, the classification score drops from 0.51 to 0.10, accurately correcting the classification results of this case.
[0035] From the perspective of causal feature statistics, the generated images significantly amplify the causal features. Figure 3 As shown in (b), the distributions of causal and non-causal features in the original image are similar. In the generated image, the causal category features are significantly emphasized, while the irrelevant non-causal features are effectively suppressed.
[0036] From the perspective of sample distribution, Figure 3 As shown in (c), after t-SNE dimensionality reduction, the generated samples show obvious clustering characteristics and are far away from the classification boundaries, indicating that the generated images have a higher degree of separation between different categories compared with the original samples.
[0037] From the perspective of interpretability, machine learning methods are used to quantify feature changes in the generated images, providing interpretable insights into the generation process of the present invention. Specifically, 851 machine learning features are extracted, of which: 304 have a p-value < 0.05 between the generated and original images, highlighting the huge statistical differences between the generated and original images. Figure 3 (d) Four most representative features are highlighted, demonstrating the ability of the present invention to promote categorical causal features while suppressing irrelevant features.
[0038] like Figure 3 As shown in (d), the present invention emphasizes contrast while alleviating the clustering tendency characteristics in the positive samples. This may be because the positive samples may contain multiple internal components with different textures. These different components have different densities and textures, resulting in uneven texture and density of the positive samples. Uneven texture and density usually corresponds to Figure 3 (d) shows a higher contrast and reduced pixel clustering tendency. On the other hand, negative samples are usually filled with transparent, non-viscous liquids. Therefore, negative samples are usually shown in the image with more uniform texture and grayscale, which corresponds to Figure 3 (d) Stronger long-range pixel dependence and smaller variance on the right.
[0039] like Figure 3 The comparison results of the present invention and the representative image generation method are shown in FIG. Figure 3 (a). For positive cases, the present invention can highlight the grayscale contrast between the solid and liquid components inside the target, thereby improving the classification score from 0.51 to 0.83. In the case of negative samples, the present invention performs well in alleviating uneven noise, resulting in smoother texture features, thereby reducing the classification score from 0.48 to 0.15. This shows that the present invention can eliminate the uncertainty of fuzzy samples and provide them with clear classification attributes.
[0040] like Figure 3 (b) shows the distribution of generated images. Although other methods prioritize generation diversity and quality, the images they generate still maintain similarity with the original sample distribution. In contrast, the present invention shows obvious intra-class similarity and inter-class independence, greatly improving the classification performance.
[0041] like Figure 3(c) and Table 1 show that the present invention outperforms other methods in classification performance. With the help of unique generated images, the present invention improves AUC / SPE by 13.60% / 16.37%, 13.20% / 16.00% and 16.66% / 16.67% respectively compared with using original images, 3DGAUNet and ERGAN to generate images. This highlights the excellent performance of image generation technology in image classification, especially in accurate detection of negative samples.
[0042] Table 1: Comparison between the present invention and the generation method.
[0043] As shown in Table 2, the present invention is compared with representative classification methods, where: compared with the machine learning method RMLM, the present invention extracts more target-specific features, which improves ACC and SPE by 14.55% and 10.00%, respectively. Compared with CNN-based methods (such as IAGNet), the present invention uses causal generation to assist classification, which improves ACC and SPE by 7.27% and 6.67%, respectively. Finally, compared with the Transformer-based network, the present invention can achieve an increase of 9.09% and 6.67% in ACC and SPE using only a simple classifier. The results clearly demonstrate the effectiveness of incorporating causal generation into classification tasks. It also shows that the present invention has the potential to fundamentally solve the challenge of fuzzy classification.
[0044] Table 2: Comparison between the present invention and classification methods.
[0045] Compared with the prior art, the present invention adopts a closed-loop causal image generation method to emphasize the specific category causal features in the image and suppress interfering features. The above-mentioned causal image generation method can increase the discrimination of fuzzy category samples from two levels: image and latent space features, thereby helping the classification of fuzzy category samples. The present invention integrates image generation tasks and image classification into the same framework and allows them to fully interact and guide each other, so as to complete the high-precision classification of fuzzy samples. Overall, the ability of the present invention to amplify the target causal features and make them clearly visible at the image level is undeniable. Therefore, blurred samples are classified into higher confidence and more accurate categories, improving the overall reliability of the classification process.
[0046] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A fuzzy category image classification system based on closed-loop causal generation, characterized in that: include: A data reading module, a causal generator module and a discriminator module for training, wherein: the data reading module performs image cutting unit, image normalization unit and format conversion unit processing according to the input three-dimensional image information to obtain data information read and recognized by subsequent modules; the causal generator module performs causal image generation processing according to the processing result of the received data reading module through a closed-loop image generation branch to obtain an enhanced version of the original image; then an accurate classification result is obtained according to the enhanced version of the original image through an image classification branch; the discriminator module performs image authenticity identification on the enhanced version of the original image according to the processing result of the causal generator to obtain an image authenticity score, thereby completing adversarial training with the generator and ensuring the authenticity of the image generated by the causal generator.
2. The fuzzy category image classification system based on closed-loop causal generation according to claim 1 is characterized in that: The closed-loop image generation branch of the causal generator module includes: a first encoder, a causal perception unit and a decoder, wherein: the first encoder encodes the tensor type original image into a high-dimensional latent space to obtain latent space features; the causal perception unit performs causal screening and category counterfactual processing based on the latent space features to obtain counterfactual template features for guiding the latent space features; the decoder decodes according to the latent space features, restores to the original image resolution, obtains the generated image and re-inputs it into the first encoder to form a closed-loop path, and repeats the above steps N times until an enhanced version image of the original image with a clear category is generated; The guidance refers to: in the process of forming a closed-loop path, the counterfactual template features are used to guide the latent space features obtained after the enhanced version image passes through the encoder, so that the feature distribution of the enhanced version image is as close as possible to the template feature distribution.
3. The fuzzy category image classification system based on closed-loop causal generation according to claim 1 is characterized in that: The image classification branch includes: a second encoder and a classification head, wherein: the second encoder encodes the enhanced version of the original image into high-dimensional latent space features; the classification head obtains accurate fuzzy category sample classification results based on the high-dimensional latent space features.
4. The fuzzy category image classification system based on closed-loop causal generation according to claim 1 is characterized in that: The data reading module includes: an image cutting unit, an image normalization unit and a format conversion unit, wherein: the image cutting unit cuts the input fuzzy category 3D image to obtain a bounding box including the target area to be classified, the image normalization unit normalizes the image in the bounding box to the range of 0-1, and the format conversion unit converts the normalized image from physical pixel value parameters to a tensor type original image that can be recognized by the deep learning network.
5. The fuzzy category image classification system based on closed-loop causal generation according to claim 2 is characterized in that: The causal screening refers to: determining whether the absolute value of the weight parameter in the frozen classification head corresponding to the latent space feature is in the top 25%, and then retaining the causal feature and removing irrelevant features to obtain a causal feature set.
6. The fuzzy category image classification system based on closed-loop causal generation according to claim 2 is characterized in that: The category counterfactual processing refers to: determining whether the positive and negative signs of the weight parameters in the frozen classification head corresponding to the features in the causal feature set are the same as the categories of the input fuzzy category 3D image, thereby enhancing the category features and removing the non-category features to obtain the counterfactual template features; The counterfactual template features Among them: F is the sum of the causal feature set obtained after the causal screening step, C is the enhancement strength, and max(F) represents the maximum value in the category feature.
7. An image classification method based on the system according to any one of claims 1 to 6, characterized in that: include: Training phase and testing phase; The training phase includes: Step 1: Pre-train image classification branch: The image classification branch obtains the latent space feature F1 after encoding the input original image I through the encoder θ. The classification head τ classifies F1 and obtains the classification score S1, which is compared with the real category Target of the image to calculate the classification loss. And complete the pre-training of encoder θ and classification head τ; Step 2: Jointly train the closed-loop image generation branch and the discriminator module, specifically including: 2.1 Input the original image I into the first encoder θ pre-trained in step 1 to obtain the latent space feature F i ,i∈[1,N], where i represents the i-th iteration and N is the total number of iterations in the closed-loop generation. According to the latent space feature F i , i∈[1,N] to get the enhanced version of the original image G i ,i∈[1,N]; 2.2 Discriminator module γ is an enhanced version of the original image G i Generate identification result R i ∈[0,1], the identification result is the authenticity probability of the image, between 0 and 1, where 0 and 1 represent the boundary probability of a fake image or a real image respectively; 2.3 Alternating adversarial training of closed-loop image generation branch and discriminator module.
8. The image classification method according to claim 7, characterized in that: In the first generation process of the closed-loop generation, the causal perception unit obtains the counterfactual template feature F through the latent space feature F1 cf And calculate the counterfactual loss F i+1 is the generated image G of the i-th round i The latent space feature obtained after passing the encoder θ, F cf is the counterfactual template feature.
9. The image classification method according to claim 7, characterized in that: In the alternating adversarial training, the total loss of the training closed-loop image generation branch is Among them: classification loss S i+1 To generate the image G of the i-th round i The classification score generated by the feedback to the frozen encoder and classification head τ, Target is the real image category, and the reliability loss G i is the i-th round latent space feature F i Through the decoder The generated image obtained after decoding, I is the original image, authenticity loss R i Represents G i The identification result obtained after the discriminator γ, λ i ,i∈[1,4] is the weight of different losses, w i ,i∈[1,n] is the weight of different iterations in closed-loop generation; the total loss of training the discriminator module in: Denotes the discriminator module γ given G i The authenticity probability R of the output under the condition i With G i The actual distribution distance between the ground-truth labels, Same reason.
10. The image classification method according to claim 7, characterized in that: The testing phase specifically includes: Step 1: The data reading module reads the input fuzzy category 3D image; Step 2: The closed-loop image generation branch generates images in a closed-loop manner based on the original image, and finally obtains an enhanced version of the original image with a clear category. Step 3: Send the enhanced version of the image to the image classification branch to obtain accurate fuzzy category sample classification results.