Image classification method and related device based on deep fuzzy convolutional neural network model

Abstract

The present invention provides an image classification method and related devices based on a deep fuzzy convolutional neural network model, which relate to the field of image processing. The deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to convert the deep convolution feature map into a deep convolution feature sequence, and then perform deep interaction of features in different spaces with the comprehensive fuzzy feature sequence to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image classification method and related devices based on a deep fuzzy convolutional neural network model. Background Art

[0002] Image classification is a crucial task in computer vision, widely used in fields such as autonomous driving, medical diagnosis, and remote sensing. Its goal is to assign input images to predefined categories. During the classification process, image classification models must identify discriminative information within the image, learning both local details and overall structure, and extracting key features from seemingly chaotic pixels that aid subsequent analysis. However, extracting effective discriminative features from complex image data, especially achieving accurate classification results in scenarios with uncertainty such as noise, can be challenging.

[0003] Currently, a common approach to image classification is to employ image classification models based on traditional machine learning techniques, such as support vector machines (SVM) and KNN (K-Nearest Neighbor). However, these machine learning-based image classification models rely on feature engineering, requiring tedious manual feature selection and specialized knowledge. Furthermore, their shallow model structure limits their learning and generalization capabilities when processing large datasets.

[0004] As a result, deeper networks, such as CNNs (Convolutional Neural Networks), and other deep learning models have gradually become mainstream. However, CNN-based models have strong local correlations, which can lead to a loss of spatial correlation and limitations in processing long-range dependencies in images, thus affecting their performance when processing larger and more complex patterns.

[0005] In recent years, the emergence of Transformer-based models has further advanced image classification. These models can effectively capture global information between pixels, making up for the shortcomings of CNNs. However, due to the lack of prior knowledge such as the inductive bias present in CNNs, Transformer-based models may have limitations in capturing complex details and require pre-training on large-scale datasets to achieve good classification accuracy.

[0006] Furthermore, models based on CNN and Transformer architectures struggle to learn useful discriminative features when images contain unfavorable blurring conditions, such as repetitive backgrounds, significant lighting changes, and occlusions of the target object. Furthermore, due to the complexity of their deep structures, these models are less robust and easily deceived by uncertainties such as noise, which can lead to misjudgments.

[0007] In summary, in the field of image classification, most existing image classification models only extract features at a single scale, ignoring the multi-scale features in the image. This results in incomplete feature capture. Furthermore, CNN models and models based on the Transformer architecture are less robust to uncertainties such as noise, resulting in low classification accuracy. Summary of the Invention

[0008] The purpose of the present invention is to provide an image classification method and related devices based on a deep fuzzy convolutional neural network model to improve the problems existing in the prior art.

[0009] The embodiments of the present invention can be implemented as follows:

[0010] In a first aspect, the present invention provides an image classification method based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network; the method includes:

[0011] Get the original image;

[0012] Inputting the original image into the feature extraction network, and using the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and deep convolution feature maps with increasing feature levels;

[0013] Performing feature fusion processing on the multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence;

[0014] The classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the target object in the original image belongs.

[0015] In a second aspect, the present invention provides an image classification device based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network; the image classification device includes:

[0016] An image acquisition module, used to acquire original images;

[0017] A feature extraction module is used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels;

[0018] A feature fusion module, configured to perform feature fusion processing on the plurality of fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence;

[0019] A classification module is used to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs.

[0020] In a third aspect, the present invention provides an electronic device comprising: a memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the image classification method as described in the first aspect above.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the image classification method described in the first aspect.

[0022] Compared with the prior art, the embodiment of the present invention provides an image classification method and related devices based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1A schematic diagram of the structure of a deep fuzzy convolutional neural network model provided by an embodiment of the present invention.

[0025] Figure 2 A flowchart of an image classification method provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of the structure of a feature extraction network provided by an embodiment of the present invention.

[0027] Figure 4 Schematic diagrams of the structures of the RN-1 layer and the RN-2 layer provided in an embodiment of the present invention.

[0028] Figure 5 Multiple grouping feature maps obtained by channel segmentation provided in an embodiment of the present invention.

[0029] Figure 6 A schematic diagram of the processing process of the collating unit provided in an embodiment of the present invention.

[0030] Figure 7 A structural diagram of a feature fusion network provided in an embodiment of the present invention.

[0031] Figure 8 A structural diagram of a classification network provided by an embodiment of the present invention.

[0032] Figure 9 A schematic diagram of the structure of an image classification device provided by an embodiment of the present invention.

[0033] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0035] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0036] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0037] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0038] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0039] Deep fuzzy system: From a broad perspective, a deep fuzzy system is a model that combines fuzzy theory and deep learning ideas. It can be divided into the following three types:

[0040] (1) A deep layered fuzzy system formed by stacking multiple fuzzy systems in different layering methods;

[0041] (2) Hybrid deep fuzzy systems that combine fuzzy theory or fuzzy thinking with deep learning models;

[0042] (3) Deep fuzzy neural networks that combine fuzziness with neurons and represent fuzzy processes such as fuzzification, fuzzy reasoning, and defuzzification in the form of neural networks;

[0043] They have the following characteristics: deep model structure, introduction of fuzzy theory, powerful representation ability, efficient ability to handle uncertainty, etc.

[0044] Although deep fuzzy systems have been successfully applied in energy management, financial analysis, risk detection and other fields, their potential in image recognition has not been tapped. The main reasons are as follows:

[0045] On the one hand, the high-dimensional nature of image data leads to rule explosion and excessive computational complexity for hierarchical fuzzy systems and deep fuzzy neural networks. Furthermore, the inherent complexity of image data, including texture, color, shape, and pattern, is difficult to directly map into fuzzy rules. Consequently, the designed fuzzy rules cannot fully capture the relationships between features and are unable to adapt to their dynamic and diverse nature.

[0046] On the other hand, hybrid deep blur systems, because blur systems are essentially global models, fail to fully consider the local spatial correlations between neighboring pixels in an image. Furthermore, existing models often focus on extracting features at a single scale, potentially overlooking the rich information of multi-granular features in an image. This limits their ability to fully capture the intrinsic characteristics of an image, resulting in incomplete and intricate feature representations.

[0047] Therefore, the inventors proposed a deep fuzzy convolutional neural network model for image classification, which addresses the limitations of existing CNN models, Transformer-based architecture models, and deep fuzzy systems, and improves image classification accuracy. The proposed deep fuzzy convolutional neural network model needs to solve the following detailed technical problems:

[0048] (1) How to build a multi-level / multi-granularity feature extraction framework;

[0049] (2) How to alleviate the “curse of dimensionality” problem of deep fuzzy systems in high-dimensional problems;

[0050] (3) How to effectively fuse multi-level fuzzy features;

[0051] (4) How to use features of different scales / levels / granularities for effective classification.

[0052] The following is a detailed introduction to the image classification method based on the deep fuzzy convolutional neural network model provided by an embodiment of the present invention, so as to fully explain the solution process of the above four technical problems.

[0053] First, the model structure of the deep fuzzy convolutional neural network model proposed in this paper is introduced. The deep fuzzy convolutional neural network model is a deep fuzzy system based on a multiple attention mechanism and multi-level hybrid guidance (A Multiple Attentions Based Multi-level Hybrid-guided Deep Fuzzy Convolutional Neural Network, MAMH-DFCNN), which consists of:

[0054] (1) Feature extraction network: A multi-level convolutional fuzzy feature extraction framework (MCFFEF) is used to extract deep convolutional features and multi-level fuzzy features from the original image, enriching the diversity of feature representation;

[0055] (2) Feature fusion network: Fuzzy Feature Attention Fusion Module (FFAFM) is used to fully fuse multi-level fuzzy features and reduce redundant information of multi-level / multi-granularity features;

[0056] (3) Classification network: A hybrid-driven Cross Attention Classifier (HCAC) is used to deeply interact features in different spaces to obtain the final classification results.

[0057] The feature extraction network may include an initial convolution block and K feature extraction blocks from shallow to deep, and the number K of feature extraction blocks may be 3 to 6.

[0058] For example, when K=4, see Figure 1 , Figure 1 A schematic diagram of the structure of a deep fuzzy convolutional neural network model provided by an embodiment of the present invention. Figure 1 In the example, Block 1 is the initial convolution block, and Blocks 2 to 5 are the four feature extraction blocks. This example is for illustrative purposes only and does not limit the number of feature extraction blocks.

[0059] Please refer to Figure 2 , Figure 2 This is a flowchart of an image classification method provided by an embodiment of the present invention. This method can be performed by an electronic device that deploys the aforementioned deep fuzzy convolutional neural network model. The electronic device can be, but is not limited to, a smartphone, smart tablet, personal computer, personal notebook, server, or other computing device. The method can include the following steps: S101 to S104.

[0060] S101: Acquire an original image.

[0061] In this embodiment, the original image may be a human face image, an object image (including items such as cars, mobile phones, computers, or bags), an animal image, a remote sensing image (including forests, cities, agricultural land, or water), an environmental image collected by an autonomous vehicle (including various traffic signs, pedestrians, or other vehicles on the road), or an industrial defect image (including defects on the surface of industrial products, such as scratches, stains, and cracks). The present invention does not limit the target object in the original image.

[0062] Optionally, during the training phase of the deep fuzzy convolutional neural network model, the original image can be one of a batch of images (a batch of images can be 32, 64, 128, 256, etc.) used to input the model in the training dataset. Steps S101 to S104 describe the processing of an image in the training dataset during the training process. During the application inference phase of the deep fuzzy convolutional neural network model, the original image is the image to be classified.

[0063] S102: Input the original image into a feature extraction network, and use the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels.

[0064] In this embodiment, the feature extraction network employs fuzzy inference, so the multiple fuzzy feature maps extracted are all fuzzy features. Furthermore, the feature levels of these fuzzy feature maps are arranged in ascending order, representing feature information of different scales captured in fuzzy spaces at different depths. Therefore, even in the presence of unfavorable fuzzy conditions such as repeated backgrounds, significant lighting changes, and occlusion of the target object, the multiple fuzzy feature maps extracted by the feature extraction network can still serve as effective discriminant features.

[0065] S103: Utilize a feature fusion network to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence.

[0066] In this embodiment, a comprehensive fuzzy feature sequence can be obtained by performing feature fusion processing on multiple fuzzy feature maps.

[0067] S104. Use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result.

[0068] In this embodiment, the classification result represents the category to which the target object in the original image belongs. In the classification network, the comprehensive fuzzy feature sequence and the deep convolution feature map are deeply interacted with features in different spaces to achieve the category determination of the target object in the original image.

[0069] The image classification method provided by the embodiment of the present invention is implemented based on a deep fuzzy convolutional neural network model, which includes a feature extraction network, a feature fusion network, and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness.

[0070] The following introduces the feature extraction process of the feature extraction network.

[0071] For step S102, the process of "using the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels" can include the following sub-steps S2011 to S2023.

[0072] S2011. Input the original image into the initial convolution block for feature extraction to obtain an initial feature map.

[0073] Assume that the size of the original image X is represented as: Among them, C is the number of channels, H is the image height, and W is the image width. Represents the training dataset. For the sake of convenience, a convolutional layer (Conv), a batch normalization layer (BN), and a ReLu activation function are taken as a unit, called a C_B_R unit, and its formula can be expressed as: C_B_R(X)=ReLu(BN(Conv(X))).

[0074] exist Figure 1 Based on this, please combine Figure 3 , the initial convolution block (Block 1) belongs to the pure convolution module, including a C_B_R unit and a maximum pooling unit (Max Pooling). When the original image X is input to the initial convolution block (Block 1), the output is: in, is the number of feature channels after Block 1, H1 is the height of the image after Block 1, and W1 is the width of the image after Block 1.

[0075] S2012: Input the initial feature map into the first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map.

[0076] S2013. Input the i-1th level reconstructed feature map into the i-th feature extraction block for feature extraction to obtain the i-th level fuzzy feature map and the i-th level reconstructed feature map.

[0077] In this embodiment, i=2, 3, ..., K; the final K-th level reconstructed feature map is the depth convolution feature map.

[0078] Taking K = 4 as an example, after obtaining the first-level blurred feature map and the first-level reconstructed feature map, the first-level reconstructed feature map is input into the second feature extraction block for feature extraction to obtain the second-level blurred feature map and the second-level reconstructed feature map; then the second-level reconstructed feature map is input into the third feature extraction block for feature extraction to obtain the third-level blurred feature map and the third-level reconstructed feature map; then the third-level reconstructed feature map is input into the fourth feature extraction block for feature extraction to obtain the fourth-level blurred feature map and the fourth-level reconstructed feature map. The fourth-level reconstructed feature map is the deep convolution feature map. This example is only an example and is not intended to limit the size of K.

[0079] Optionally, the kth feature extraction block includes a feature extraction unit and an arrangement unit, k=1, 2, ..., K; therefore, in the above steps S2012-S2013, the process of the kth feature extraction block performing feature extraction to obtain the kth level fuzzy feature map and the kth level reconstructed feature map may include the following sub-steps S001-S005:

[0080] S001. Use the feature extraction unit to extract features from the input feature map to obtain a convolution feature map.

[0081] In this embodiment, when k=1, the input feature map is an initial feature map; when k>1, the input feature map is a k-1th level reconstructed feature map.

[0082] For example, please continue to combine Figure 3 It can be seen that the difference between the structures of Block 2 to Block 5 is the number of RN-2 layers in the feature extraction unit. Taking Block 2 as an example: the feature extraction unit of feature extraction block 1 (Block 2) includes one RN-1 layer, two RN-2 layers, and a 1x1 convolution layer. The structure of the RN-1 layer and the structure of the RN-2 layer are as follows: Figure 4 As shown. Assume that the convolution feature map obtained after the feature extraction unit in Block 2 is H2 and W2 are the number of feature channels, image height, and image width after passing through the feature extraction unit in Block 2, respectively.

[0083] S002. Slice the convolution feature map through channels to obtain multiple grouped feature maps.

[0084] In this embodiment, each channel of each grouped feature map is part of all channels of the convolutional feature map; the number of attention subunits, the number of encoding and decoding subunits, and the number of grouped feature maps are the same.

[0085] Among them, the sorting unit in the feature extraction block adopts a grouped interval-type-two generalized fuzzy hyperbolic tangent autoencoder layer (Grouped SE-IT2GFHTAs) based on the SE attention mechanism, which includes multiple SE-IT2GFHTA modules and two splicing layers. The SE-IT2GFHTA module includes an attention subunit and a codec subunit.

[0086] When processing high-dimensional data such as images, if the feature map is directly used as the input of the fuzzy system, it often faces the "dimensionality curse" problem. Therefore, the convolution feature map is channel-sliced to obtain multiple group feature maps, and each group feature map is input into a SE-IT2GFHTA module.

[0087] After passing through the feature extraction unit in Block 2, the convolution feature map is obtained For example, Assume that the number of channel slices is num g , then right The num obtained after segmentation by the number of channels g The group feature map is as follows Figure 5 As shown, the i-th group feature map Then Figure 5 num in g The grouped feature maps are input into the num g SE-IT2GFHTA modules are used for processing. The processing diagram is shown in the following figure. Figure 6 shown.

[0088] S003. For each group feature map, use a corresponding attention sub-unit to perform squeezing and excitation operations on the group feature map in sequence based on the SE attention mechanism to obtain an attention feature map carrying multiple channel weights.

[0089] For the i-th group feature map It passes through the attention subunit and adopts the SE attention mechanism to obtain the attention feature map with different channel weights in, It is worth noting that the SE attention mechanism does not change the size of the feature map.

[0090] S004. Utilize a corresponding encoding / decoding sub-unit to extract fuzzy features based on an attention feature map carrying multiple channel weights, and obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map.

[0091] Execute the above steps S004 to S004 for each group feature map to get num g k-th level sub-fuzzy feature map and numg k-th sub-reconstructed feature map.

[0092] S005. Use two splicing layers to perform splicing operations and 1×1 convolution operations on all k-th level sub-blurred feature maps and all k-th level sub-reconstructed feature maps in sequence to obtain a k-th level fuzzy feature map and a k-th level reconstructed feature map.

[0093] In this embodiment, two splicing layers are used to perform splicing operations and 1×1 convolution operations on all k-th level sub-blurred feature maps and all k-th level sub-reconstructed feature maps in sequence, so as to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map.

[0094] The above step S004 involves fuzzy reasoning. In the process of fuzzy reasoning, when obtaining the activation strength of each rule, it is necessary to multiply the membership value of each variable (greater than 0 and less than 1). When the number of input variables of the fuzzy system is large, the problem of numerical underflow will occur, that is, the activation strength is infinitely close to 0. At this time, the rule is equivalent to not being activated, which will result in a very small final output of the fuzzy system. It is necessary to reduce the input variables of the fuzzy system based on multiple channel weights.

[0095] A codec subunit includes a dimension processing layer, an encoder, and a decoder. The encoder uses the interval-type generalized fuzzy hyperbolic tangent system (IT2GFHTS), and the decoder uses a transposed convolution operation. Figure 6 The processing process of the codec subunit in the first feature extraction block (Block 2) shown in the figure introduces sub-steps S0041 to S0048 of the above step S004, which is "using a corresponding codec subunit to extract fuzzy features based on the attention feature map carrying multiple channel weights to obtain the k-th level sub-fuzzy feature map and the k-th level sub-reconstructed feature map."

[0096] S0041. Using the dimensionality processing layer, based on multiple channel weights, the pixel intensities of each channel at each pixel position in the attention feature map are weighted averaged to obtain the mean pixel intensity at each pixel position in the attention feature map.

[0097] S0042. Use the dimension processing layer to calculate the variance of each pixel position in the attention feature map based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensity of each channel.

[0098] Attention feature maps with different channel weights For example, The pixel intensity at each pixel position in the channel is weighted and the attention feature map can be obtained. The variance at each pixel position in .

[0099] Among them, the attention feature map is calculated The process of the variance at each pixel position in is:

[0100] Hypothesized attention feature map The pixel position in a channel is expressed as The pixel intensity at this position is Among them, i and j represent the height index and width index of the pixel in the feature map, respectively, and c represents the channel index of the feature map.

[0101] Using the weights of each channel, the attention feature map The attention feature map can be obtained by weighted averaging the pixel intensities of all pixel positions in each channel. The mean value of the variance pixel intensity at each pixel position in . Among them, the mean value of the pixel intensity at the pixel position (i, j) is: The variance at this location is:

[0102] S0043. Utilize the dimension processing layer to extract the pixel intensities of each channel at a preset number of pixel positions with the largest variance from the attention feature map, and integrate them to obtain the key feature map.

[0103] In this embodiment, for the attention feature map The Z with the largest variance can be extracted from it var The pixel intensity of each channel at the pixel position is integrated to obtain the key feature map The key feature map obtained after this processing and attention feature map In comparison, the number of channels remains unchanged, but the image size is reduced from H2×W2 to Z1×Z2.

[0104] Among them, the preset number is Z var , Z var The size can be set flexibly and is not limited here.

[0105] S0044. Flatten the key feature graph to obtain a key feature sequence.

[0106] In this embodiment, the key feature map After flattening, we get the key feature sequence: {x1,x2,…,x M}, The key feature sequence includes M elements.

[0107] S0045. Use the encoder to construct fuzzy rules based on the membership function of each element in the key feature sequence.

[0108] S0046. Using an encoder, encoding the key feature sequence based on fuzzy rules to obtain a k-th level fuzzy feature sequence;

[0109] In this embodiment, for each x in the key feature sequence m (m=1,2,…,M), the membership function is designed as follows:

[0110]

[0111]

[0112] Among them, P MF is a positive membership function, N MF is the negative membership function, and are the upper and lower membership functions of the positive membership function, and is the upper and lower membership function of the negative membership function, k m (m=1,2,…,M) is the center of the membership function, and are the standard deviations of the upper and lower membership functions respectively. In order to make the membership function applicable to high-dimensional problems, and Improvements are made and and The size relationship between them is:

[0113]

[0114] Among them, D in is the input dimension, e -η The smallest positive floating-point value that can be represented by a computer.

[0115] The fuzzy rule form of the encoder (IT2GFHTS) can be expressed as:

[0116]

[0117] Among them, y n is the post-processing output, is an interval type-2 fuzzy set, c M is the consequent constant that corresponds one-to-one to the interval type-2 fuzzy set in the antecedent.

[0118] After flattening Input to the encoder (IT2GFHTS), the output is:

[0119]

[0120] Here, tanh() represents the hyperbolic tangent function.

[0121] By extending the above single-output encoder (IT2GFHTS) to multiple outputs to adapt to the structure of the autoencoder, the fuzzy rule form of the multi-output encoder (IT2GFHTS) can be obtained as follows:

[0122]

[0123] The output is:

[0124]

[0125] in, are all adjustable parameter vectors to be learned, and K is the vector composed of the centers of membership functions.

[0126] Assume that the number of output variables of the encoder (IT2GFHTS) in each SE-IT2GFHTA module is the same, both Therefore, the second-level sub-fuzzy feature sequence output by the encoder (IT2GFHTS) in the i-th SE-IT2GFHTA module is:

[0127] S0047. Reshape the k-th level sub-fuzzy feature sequence to obtain the k-th level sub-fuzzy feature graph.

[0128] In this embodiment, the second-level sub-fuzzy feature sequence Perform reshaping (i.e., reshape operation) to obtain the second-level sub-fuzzy feature map: FSFF stands for First-level Sub-fuzzy Feature.

[0129] S0048. Use the decoder to upsample the k-th level sub-blurred feature map to match the input dimension, and obtain the k-th level sub-reconstructed feature map.

[0130] In this embodiment, the second-level sub-fuzzy feature map Upsampling is achieved by transposed convolution to obtain the second-level sub-reconstructed feature map:

[0131]

[0132] Among them, FSRF is called First-level Sub-reconstructed Feature. In this way, the image height and image width of the second-level sub-reconstructed feature map obtained after the decoder processing become H2×W2, which is the same as Figure 5 Convolutional feature map before channel slicing The image height and image width are consistent.

[0133] Therefore, each codec subunit in the first feature extraction block (Block 2) extracts fuzzy features from the attention feature map carrying multiple channel weights in the manner of steps S0041 to S0048 above, and the num g Second-level sub-fuzzy feature maps, num g The second-level sub-reconstructed feature map.

[0134] Finally, for num g The second-level sub-blur feature maps are sequentially concatenated and 1×1 convolutionally operated to obtain the second-level fuzzy feature map as follows:

[0135]

[0136] For num g The second-level reconstructed feature maps are sequentially concatenated and 1×1 convolutionally performed to obtain the second-level reconstructed feature map as follows:

[0137]

[0138] Among them, for num g The second-level reconstructed feature map is obtained by sequentially performing a splicing operation and a 1×1 convolution operation on the second-level sub-reconstructed feature map. The number of channels is and Figure 5 Convolutional feature map before channel slicing The number of channels is the same.

[0139] Combine Figure 3 The above explanation of each sub-step of step S102 is based on Block 2. The feature extraction process of Block 3 to Block 5 is similar to this and will not be described in detail here.

[0140] Therefore, when K=4, Figure 3In the feature extraction network shown, the first feature extraction block (Block2) extracts the first-level fuzzy feature map (FFF) and the first-level reconstructed feature map (FRF), the second feature extraction block (Block 3) extracts the second-level fuzzy feature map (SFF) and the second-level reconstructed feature map (SRF), the third feature extraction block (Block 4) extracts the third-level fuzzy feature map (TFF) and the third-level reconstructed feature map (TRF), and the fourth feature extraction block (Block 5) extracts the fourth-level fuzzy feature map (FTFF) and the fourth-level reconstructed feature map (FTRF). The fourth-level reconstructed feature map (FTRF) is also a deep convolutional feature map (DCF).

[0141] When the original image is contaminated by noise, the recognition accuracy of traditional CNN methods is significantly reduced. This is because convolution, as a local operation, is easily misled by noise.

[0142] From the above, it can be seen that the present invention introduces SE-IT2GFHTA into the sorting unit of the feature extraction network, which has a strong uncertainty processing capability and can improve the robustness of the model. Since fuzzy systems are inherently nonlinear, SE-IT2GFHTA can perform more complex nonlinear transformations on feature maps. This process can extract more advanced semantic information, thereby enriching the granularity and hierarchy of feature patterns. In addition, the membership function within SE-IT2GFHTA is always symmetric about the origin, matching the distribution of feature maps after batch normalization (BN). This feature gives SE-IT2GFHTA a superior ability to capture features.

[0143] The following introduces the feature fusion process of the feature fusion network.

[0144] The K fuzzy feature maps extracted by the feature extraction network are: the first-level fuzzy feature map to the K-th level fuzzy feature map. These multi-level fuzzy feature maps capture feature information of different scales in fuzzy spaces of different depths, but each level of fuzzy feature map is obtained based on the reconstructed feature map of the previous level, which may lead to information duplication and redundancy.

[0145] In order to overcome the common problem of multi-level features, the present invention designs a pyramid-like feature fusion network, which includes K fuzzy transformation units, K fusion units and fusers. For example, when K=4, the results of the feature fusion network can be seen in Figure 7, the fusion unit uses N Transformer encoders, and the fuser uses one Transformer encoder. In this way, the feature fusion network can perform multi-granularity information fusion on multi-level fuzzy features to obtain a more concise and comprehensive fuzzy feature representation, while alleviating the problem of information redundancy.

[0146] Therefore, for step S103, the process of "using a feature fusion network to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence" can include the following sub-steps S1031 to S1034.

[0147] S1031. Using the kth fuzzy transformation unit, based on the learned kth linear transformation matrix, kth label vector and kth position vector, convert the kth level fuzzy feature map into a kth level fuzzy feature sequence.

[0148] Optionally, the sub-steps of step S1031 may include S10311 to S10315.

[0149] S10311. Use the k-th fuzzy transformation unit to perform a segmentation operation on the k-th level fuzzy feature map to obtain multiple k-th level fuzzy feature blocks corresponding to the k-th level fuzzy feature map, where k = 1, 2, ..., K.

[0150] Assume K = 4, combined Figure 7 , Figure 7 Each of the four fuzzy transformation units in [1] converts the input fuzzy feature map into a fuzzy feature sequence. First, each level of fuzzy feature needs to be patched to obtain fuzzy feature blocks (fuzzy patches) with a resolution of P×P as follows:

[0151]

[0152] in, Represents a first-level fuzzy feature block, Represents the number of first-level fuzzy feature blocks; represents a second-level fuzzy feature block, Represents the number of second-level fuzzy feature blocks; Represents a third-level fuzzy feature block, Represents the number of third-level fuzzy feature blocks; Represents a fourth-level fuzzy feature block, Represents the number of fourth-level fuzzy feature blocks.

[0153] S10312. Using the kth fuzzy transformation unit, adjust the number of first-level fuzzy feature blocks based on linear projection to obtain L kth-level fuzzy feature blocks.

[0154] In this embodiment, L is transformed into FFF , L SFF , L TFF and L FTFF Alignment is performed so that the number of fuzzy feature blocks at each level is L.

[0155] S10313. Utilize the k-th fuzzy transformation unit to learn the k-th linear transformation matrix, the k-th label vector, and the k-th position vector from the k-th fuzzy feature map.

[0156] Since the Transformer of the fusion unit uses latent vectors of uniform size for processing at each level, this embodiment obtains the k-th level linear transformation matrix through the learnable linear transformation g(·), and the k-th level linear transformation matrix is used to map fuzzy features of different granularities into a unified D-dimensional space. Similar to the [Class] token in ViT, this embodiment also supplements a learned k-th level label vector to represent the comprehensive information of the k-th level fuzzy feature map. In addition, in order to integrate the position information into each fuzzy feature block and retain the spatial structure between each fuzzy feature block, a k-th level position vector is also learned from the k-th level fuzzy feature map.

[0157] S10314. Using the kth fuzzy transformation unit, multiply each kth-level fuzzy feature block by the kth-level linear transformation matrix to obtain L kth-level fuzzy feature vectors;

[0158] S10315. Using the kth fuzzy transformation unit, concatenate the kth level label vector and L kth level fuzzy feature vectors and add them to the kth level position vector to obtain the kth level fuzzy feature sequence.

[0159] In this embodiment, combined with Figure 7 , the obtained first-level fuzzy feature sequence to the fourth-level fuzzy feature sequence can be expressed as follows:

[0160]

[0161] In formula (7), Represent the first to fourth level fuzzy feature sequences respectively;

[0162] In formula (7), Respectively represent the first-level linear transformation matrix to the fourth-level linear transformation matrix;

[0163] In formula (7), Represent the first level position vector to the fourth level position vector respectively;

[0164] In formula (7), Represent the first-level label vector to the fourth-level label vector respectively;

[0165] In formula (7), They represent the jth first-level fuzzy feature block, the jth second-level fuzzy feature block, the jth third-level fuzzy feature block, and the jth fourth-level fuzzy feature block respectively; and j = 1, 2, 3,…, L.

[0166] S1032. Input the k-th level fuzzy feature sequence into the k-th fusion unit to obtain the k-th level fusion sequence.

[0167] In this embodiment, combined with Figure 7 , transform the first-level fuzzy feature sequence to the fourth-level fuzzy feature sequence: By inputting them into the four fusion units respectively, we can obtain the first-level fusion sequence to the fourth-level fusion sequence, which is expressed as follows:

[0168]

[0169] in, It is the first-level fusion sequence to the fourth-level fusion sequence obtained by processing four fusion units (i.e., Transformer).

[0170] S1033 , adding the first-level fusion sequence to the K-th-level fusion sequence to obtain a superimposed fuzzy feature sequence.

[0171] In this embodiment, combined with Figure 7 , for the first to fourth level fusion sequences: Add pixel by pixel to get the superimposed fuzzy feature sequence T f , as shown below:

[0172]

[0173] S1034. Input the superimposed fuzzy feature sequence into the fusion device to obtain a comprehensive fuzzy feature sequence.

[0174] In this embodiment, combined with Figure 7 , superimpose the fuzzy feature sequence T f Input the fusion device to obtain the comprehensive fuzzy feature sequence As shown below:

[0175]

[0176] in, Represents comprehensive fuzzy feature sequence The fuzzy label sequence in Represents comprehensive fuzzy feature sequence The fuzzy feature block sequence in . Figure 7 The “C” in each sequence represents the tag sequence portion.

[0177] Combine Figure 3 and Figure 7 The above explanation of the various sub-steps of step S103 is based on the case where K=4, that is, when there are four feature extraction blocks (Block 2 to Block 5) in the feature extraction network. The explanation introduces the fusion process of the feature fusion network from the first-level fuzzy feature map output by Block 2 to Block 5 to the fourth-level fuzzy feature map.

[0178] The following is an introduction to the processing of the classification network.

[0179] See Figure 8 The classification network consists of a deep transform unit, a cross layer, two attention units, two MLP (Multi-Layer Perceptron) heads, and a classification layer. The two attention units use two hybrid-driven multi-head cross attention modules, and the classification layer uses a softmax function.

[0180] Therefore, for the above step S104, the process of "using the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain the classification result" can include the following sub-steps S1041 to S1047.

[0181] S1041. Using the depth transformation unit, based on the learned depth label vector, depth linear transformation matrix and depth position vector, the depth convolution feature map is converted into a depth convolution feature sequence.

[0182] In this embodiment, the process of converting the depth convolution feature map into the depth convolution feature sequence by the depth transformation unit includes the following four steps:

[0183] (1) After reconstructing the multi-channel depth convolution feature map into a single-channel two-dimensional depth convolution feature map, the two-dimensional depth convolution feature map is segmented to obtain multiple convolution feature blocks;

[0184] (2) Adjust the number of convolution feature blocks based on linear projection to obtain L convolution feature blocks;

[0185] (3) Learn the depth label vector, depth linear transformation matrix and depth position vector from the two-dimensional depth convolution feature map;

[0186] (4) Multiply each convolution feature block with the depth linear transformation matrix to obtain L depth vectors, and then concatenate the depth label vector with the L depth vectors and add them to the depth position vector to obtain a depth convolution feature sequence.

[0187] That is, the process in which the depth transformation unit converts the two-dimensional depth convolution feature map into the depth convolution feature sequence is similar to the process in which the fuzzy transformation unit converts the k-th level fuzzy feature map into the k-th level fuzzy feature sequence, and will not be repeated here.

[0188] The process of obtaining the deep convolution feature sequence in step (4) can be expressed as:

[0189]

[0190] Combine Figure 8 , deep convolutional feature sequence T DCF It is divided into two parts: in, Represents the deep convolution feature sequence T DCF The depth label sequence of Represents the deep convolution feature sequence T DCF The sequence of deep feature blocks in .

[0191] S1042. Using a cross layer, cross-join the deep label sequence and deep feature block sequence in the deep convolutional feature sequence with the fuzzy label sequence and fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences.

[0192] In this embodiment, combined with Figure 7 , in the cross layer, the deep convolution feature sequence T DCF The depth label sequence in and the fuzzy label sequence in the comprehensive fuzzy feature sequence After the exchange, the two original cross feature sequences T1 and T2 are obtained as follows:

[0193]

[0194] S1043 , normalize both original cross feature sequences to obtain a first cross feature sequence and a second cross feature sequence.

[0195] In this embodiment, the first cross feature sequence includes a normalized depth label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized depth feature block sequence. The first cross feature sequence T is obtained by normalizing the two original cross feature sequences T1 and T2. cross_1 and the second crossover feature sequence T cross_2 , as shown below:

[0196]

[0197] in, They are the normalized results of the normalized deep label sequence and the normalized fuzzy feature block sequence respectively; They are the normalized fuzzy label sequence and the normalized deep feature block sequence respectively.

[0198] S1044. Use two attention units to process the first cross-feature sequence and the second cross-feature sequence respectively to obtain a first output sequence and a second output sequence.

[0199] In this embodiment, T cross_1 and T cross_2 are fed into two attention units (hybrid driven multi-head cross attention module). The first attention unit is fed with the normalized depth label sequence (i.e. ) as a guide, the second attention unit uses the normalized fuzzy label sequence (i.e. ) as a guide, the query vector, key vector, and value vector of the two attention units are as follows:

[0200]

[0201]

[0202] Among them, q1, k1, and v1 are the query vector, key vector, and value vector of the first attention unit respectively; are the query transformation matrix, key transformation matrix, and value transformation matrix of the first attention unit respectively; q2, k2, and v2 are the query vector, key vector, and value vector of the second attention unit respectively; They are the query transformation matrix, key transformation matrix, and value transformation matrix of the second attention unit, respectively.

[0203] Finally, the first output sequence MSA of the two attention units is output cross_1 and the second output sequence MSA cross_2 as follows:

[0204]

[0205]

[0206] MSA cross_1 =A cross_1 ·v cross_1

[0207] MSA cross_2 =A cross_2 ·v cross_2

[0208] in, represents the scaling factor, D is the total dimension shared by all heads of the multi-head crisscross attention module, and h is the number of heads in the multi-head crisscross attention module; A cross_1 Represents the attention weight determined by the first attention unit, A cross_2 Represents the attention weight determined by the second attention unit.

[0209] S1045 , adding the first output sequence and the deep label sequence and performing normalization processing to obtain a first normalized sequence, and adding the second output sequence and the fuzzy label sequence and performing normalization processing to obtain a second normalized sequence.

[0210] In this embodiment, combined with Figure 8 , the first output sequence MSA output by the two attention units cross_1 and the second output sequence MSA cross_2 Respectively with the previous depth label sequence Fuzzy label sequence The two addition results are obtained by adding them together. This process once again realizes the deep information interaction between label sequences of different granularities. Then, the two addition results are normalized respectively to obtain the first normalized sequence and the second normalized sequence.

[0211] S1046 : Use two MLP heads to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value.

[0212] In this embodiment, combined with Figure 8 , the two MLP heads process the first normalized sequence and the second normalized sequence respectively, and the first output value y cross_1 and the second output value y cross_s It can be expressed as follows:

[0213]

[0214] S1047: Input the first output value and the second output value into the classification layer for processing to obtain a classification result.

[0215] In this embodiment, the classification layer uses the softmax function, and the classification result y 2CAC It can be expressed as:

[0216] y HCAC =softmax(y cross_1 +y cross_2 )

[0217] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0218] (1) Compared with existing models that only use convolution or only use Transformer architecture, the deep fuzzy convolutional neural network model proposed in this paper integrates convolution and multiple attention mechanisms, overcoming the local attributes and long-distance dependency problems of CNN models and the lack of bias induction problem of Transformer architecture models;

[0219] (2) In the deep fuzzy convolutional neural network model proposed in the present invention, the feature extraction network adopts a multi-level convolutional fuzzy feature extraction framework, which can extract fuzzy features at different levels. Compared with many existing single-feature models, it has richer feature representation and more powerful feature learning capabilities;

[0220] (3) In the feature extraction network, each sorting unit adopts an interval type-II generalized fuzzy hyperbolic tangent autoencoder based on the SE attention mechanism, which can make the feature extraction network have strong noise robustness based on the powerful uncertainty processing ability of the fuzzy system;

[0221] (4) In the deep fuzzy convolutional neural network model proposed in the present invention, the feature extraction network fusion network adopts a pyramid-like fuzzy feature attention fusion module, which can effectively integrate multi-level fuzzy features to obtain comprehensive fuzzy features, realize the full mining of fuzzy feature information of different granularities, and reduce the information redundancy of multi-level features;

[0222] (5) In the deep fuzzy convolutional neural network model proposed in the present invention, the classification network adopts a hybrid-driven cross-attention classifier, which can perform deep interaction of features in different spaces, thereby guiding the model to classify more comprehensively and improving the classification accuracy.

[0223] In order to execute the corresponding steps in the above method embodiment and each possible implementation method, an implementation method of an image classification device based on a deep fuzzy convolutional neural network model is given below.

[0224] See Figure 9 , Figure 9 FIG2 is a schematic diagram showing the structure of an image classification device provided by an embodiment of the present invention. The image classification device 200 includes: an image acquisition module 210 , a feature extraction module 220 , a feature fusion module 230 and a classification module 240 .

[0225] An image acquisition module 210 is used to acquire an original image;

[0226] The feature extraction module 220 is used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels;

[0227] A feature fusion module 230 is used to perform feature fusion processing on multiple fuzzy feature maps using a feature fusion network to obtain a comprehensive fuzzy feature sequence;

[0228] The classification module 240 is used to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the target object in the original image belongs.

[0229] Optionally, the feature extraction network includes an initial convolution block and K feature extraction blocks; when the feature extraction module 220 is used to perform feature extraction on the original image using the feature extraction network to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels, it is specifically used to: input the original image into the initial convolution block for feature extraction to obtain an initial feature map; input the initial feature map into the first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map; input the i-1th level reconstructed feature map into the i-th feature extraction block for feature extraction to obtain an i-th level fuzzy feature map and an i-th level reconstructed feature map; wherein, i = 2, 3,…, K; the K-th level reconstructed feature map is a deep convolution feature map.

[0230] Optionally, the kth feature extraction block includes a feature extraction unit and a sorting unit, k=1,2,…,K; the sorting unit includes multiple attention subunits, multiple encoding and decoding subunits and two splicing layers. In the process of feature extraction by the kth feature extraction block to obtain the kth level fuzzy feature map and the kth level reconstructed feature map, the feature extraction module 220 can be specifically used to: use the feature extraction unit to extract features from the input feature map to obtain a convolution feature map; wherein, when k=1, the input feature map is the initial feature map; when k>1, the input feature map is the k-1th level reconstructed feature map; channel slicing is performed on the convolution feature map to obtain multiple grouped feature maps; each channel of each grouped feature map is a part of all channels of the convolution feature map; the number of attention subunits, the number of encoding and decoding subunits and the number of grouped feature maps The amount is the same; for each group feature map, a corresponding attention sub-unit is used to perform squeezing operations and excitation operations on the group feature map based on the SE attention mechanism to obtain an attention feature map carrying multiple channel weights; a corresponding encoding and decoding sub-unit is used to extract fuzzy features based on the attention feature map carrying multiple channel weights to obtain the k-th level sub-fuzzy feature map and the k-th level sub-reconstructed feature map; two splicing layers are used to perform splicing operations and 1×1 convolution operations on all the k-th level sub-fuzzy feature maps and all the k-th level sub-reconstructed feature maps in turn to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map.

[0231] Optionally, the encoding and decoding subunit includes a dimension processing layer, an encoder and a decoder; using a corresponding encoding and decoding subunit, based on the attention feature map carrying multiple channel weights, the fuzzy feature is extracted to obtain the k-th level sub-fuzzy feature map and the k-th level sub-reconstructed feature map. In the process, the feature extraction module 220 can be specifically used to: using the dimension processing layer, based on multiple channel weights, perform weighted averaging of the pixel intensities of each channel at each pixel position in the attention feature map to obtain the mean pixel intensity at each pixel position in the attention feature map; using the dimension processing layer, based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensities of each channel, calculate the pixel intensity of each pixel in the attention feature map. variance at the position; using the dimensionality processing layer, the pixel intensities of each channel at the preset number of pixel positions with the largest variance are extracted from the attention feature map, and integrated to obtain the key feature map; the key feature map is flattened to obtain a key feature sequence; the key feature sequence includes several elements; using the encoder, a fuzzy rule is constructed based on the membership function of each element in the key feature sequence; using the encoder, the key feature sequence is encoded based on the fuzzy rule to obtain the k-th level sub-fuzzy feature sequence; the k-th level sub-fuzzy feature sequence is reshaped to obtain the k-th level sub-fuzzy feature map; using the decoder, the k-th level sub-fuzzy feature map is upsampled to match the input dimension to obtain the k-th level sub-reconstructed feature map.

[0232] Optionally, the multiple fuzzy feature maps include first-level to K-level fuzzy feature maps; the feature fusion network includes K fuzzy transformation units, K fusion units, and a fusion device. When the feature fusion module 230 is used to perform feature fusion processing on multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence, it can be specifically used to: use the k-th fuzzy transformation unit to convert the k-th fuzzy feature map into a k-th fuzzy feature sequence based on the learned k-th linear transformation matrix, k-th label vector, and k-th position vector; input the k-th fuzzy feature sequence into the k-th fusion unit to obtain a k-th fusion sequence; where k = 1, 2, ..., K; add the first-level fusion sequence to the K-th fusion sequence to obtain a superimposed fuzzy feature sequence; input the superimposed fuzzy feature sequence into the fusion device to obtain a comprehensive fuzzy feature sequence.

[0233] Optionally, the feature fusion module 230 is used to: use the kth fuzzy transformation unit to convert the kth level fuzzy feature map into a kth level fuzzy feature sequence based on the learned kth level linear transformation matrix, kth level label vector and kth level position vector, and is specifically used to: use the kth fuzzy transformation unit to perform a segmentation operation on the kth level fuzzy feature map to obtain multiple kth level fuzzy feature blocks corresponding to the kth level fuzzy feature map; use the kth fuzzy transformation unit to adjust the number of first-level fuzzy feature blocks based on linear projection to obtain L kth level fuzzy feature blocks; use the kth fuzzy transformation unit to learn the kth level linear transformation matrix, kth level label vector and kth level position vector from the kth level fuzzy feature map; use the kth fuzzy transformation unit to multiply each kth level fuzzy feature block with the kth level linear transformation matrix to obtain L kth level fuzzy feature vectors; use the kth fuzzy transformation unit to splice the kth level label vector and L kth level fuzzy feature vectors and add them to the kth level position vector to obtain the kth level fuzzy feature sequence.

[0234] Optionally, the classification network includes a deep transformation unit, a cross layer, two attention units, two MLP heads and a classification layer. In the process of using the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain the classification result, the classification module 240 can be specifically used to: use the deep transformation unit to convert the deep convolution feature map into a deep convolution feature sequence based on the learned deep label vector, deep linear transformation matrix and deep position vector; use the cross layer to cross-splice the deep label sequence and deep feature block sequence in the deep convolution feature sequence with the fuzzy label sequence and fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences; normalize both original cross feature sequences to obtain a first cross feature sequence and a second cross feature sequence; the first cross feature sequence includes the normalized The first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence; two attention units are used to process the first cross feature sequence and the second cross feature sequence respectively to obtain a first output sequence and a second output sequence; the first output sequence is added to the deep label sequence and then normalized to obtain a first normalized sequence, and the second output sequence is added to the fuzzy label sequence and then normalized to obtain a second normalized sequence; two MLP heads are used to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value; the first output value and the second output value are input into the classification layer for processing to obtain a classification result.

[0235] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the image classification device 200 described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0236] See Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 300 includes a processor 310 , a memory 320 , and a bus 330 , wherein the processor 310 is connected to the memory 320 via the bus 330 .

[0237] The memory 320 may be used to store software programs, such as the software program corresponding to the image classification apparatus 200 provided in the embodiment of the present invention. The processor 310 executes the software program stored in the memory 320 to perform various functional applications and data processing to implement the image classification method provided in the embodiment of the present invention.

[0238] Among them, the memory 320 can be but is not limited to: RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0239] Processor 310 can be an integrated circuit chip with signal processing capabilities. Processor 310 can be a general-purpose processor, including a CPU (Central Processing Unit), a Network Processor (NP), or a System on Chip (SoC). It can also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0240] I understand. Figure 10The structure shown is for illustration only. The electronic device 300 may also include Figure 10 More or fewer components than shown, or with Figure 10 Different configurations shown. Figure 10 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0241] Embodiments of the present invention further provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image classification method disclosed in the above embodiments. The computer-readable storage medium may be, but is not limited to, a USB flash drive, a mobile hard drive, ROM, RAM, PROM, EPROM, EEPROM, a FLASH disk, or an optical disk, among other media capable of storing program code.

[0242] In summary, the embodiments of the present invention provide an image classification method and related devices based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness.

[0243] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An image classification method based on a deep fuzzy convolutional neural network model, characterized in that: The deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network; the method includes: Get the original image; Inputting the original image into the feature extraction network, and using the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and deep convolution feature maps with increasing feature levels; Using the feature fusion network to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; Using the classification network, performing deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs; The feature extraction network includes an initial convolution block and K feature extraction blocks; the step of extracting features from the original image using the feature extraction network to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels includes: Inputting the original image into the initial convolution block for feature extraction to obtain an initial feature map; Inputting the initial feature map into a first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map; The i-1th level reconstructed feature map is input into the i-th feature extraction block for feature extraction to obtain the i-th level fuzzy feature map and the i-th level reconstructed feature map; wherein, ; The K-th level reconstructed feature map is the depth convolution feature map; The multiple fuzzy feature maps include first-level to K-th level fuzzy feature maps; the feature fusion network includes K fuzzy transformation units, K fusion units, and a fusion unit; the step of using the feature fusion network to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence includes: Using the k-th fuzzy transformation unit, based on the learned k-th linear transformation matrix, k-th label vector and k-th position vector, the k-th level fuzzy feature map is converted into a k-th level fuzzy feature sequence; The k-th level fuzzy feature sequence is input into the k-th fusion unit to obtain the k-th level fusion sequence; wherein, k ; Add the first-level fusion sequence to the K-th-level fusion sequence to obtain a superimposed fuzzy feature sequence; Inputting the superimposed fuzzy feature sequence into the fusion device to obtain the comprehensive fuzzy feature sequence; The classification network includes a deep transformation unit, a cross layer, two attention units, two MLP heads and a classification layer; the step of using the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result includes: Utilizing the depth transformation unit, based on the learned depth label vector, depth linear transformation matrix and depth position vector, the depth convolution feature map is converted into a depth convolution feature sequence; Using the cross layer, the deep label sequence and the deep feature block sequence in the deep convolution feature sequence are cross-joined with the fuzzy label sequence and the fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences; Normalizing the two original cross feature sequences to obtain a first cross feature sequence and a second cross feature sequence; the first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence; Using the two attention units, respectively processing the first cross feature sequence and the second cross feature sequence to obtain a first output sequence and a second output sequence; The first output sequence and the deep label sequence are added and normalized to obtain a first normalized sequence, and the second output sequence and the fuzzy label sequence are added and normalized to obtain a second normalized sequence; Using the two MLP heads to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value; The first output value and the second output value are input into a classification layer for processing to obtain the classification result.

2. The method according to claim 1, characterized in that The kth feature extraction block includes a feature extraction unit and an arrangement unit, k The arranging unit includes multiple attention subunits, multiple encoding and decoding subunits and two splicing layers; The process of performing feature extraction in the k-th feature extraction block to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map includes: The feature extraction unit is used to extract features from the input feature map to obtain a convolution feature map; wherein, when k=1, the input feature map is the initial feature map; when k>1, the input feature map is the k-1th level reconstructed feature map; Channel slicing is performed on the convolution feature map to obtain a plurality of grouped feature maps; each channel of each grouped feature map is a portion of all channels of the convolution feature map; the number of the attention subunits, the number of the encoding and decoding subunits, and the number of the grouped feature maps are the same; For each of the grouped feature maps, using a corresponding attention sub-unit, based on the SE attention mechanism, the grouped feature maps are sequentially squeezed and stimulated to obtain an attention feature map carrying multiple channel weights; Utilizing a corresponding encoding / decoding subunit, extracting fuzzy features based on the attention feature map carrying multiple channel weights to obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map; Two splicing layers are used to perform splicing operations and 1×1 convolution operations on all the k-th level sub-blurred feature maps and all the k-th level sub-reconstructed feature maps in sequence to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map.

3. The method according to claim 2, characterized in that The encoding and decoding subunit includes a dimension processing layer, an encoder and a decoder; The step of extracting fuzzy features based on the attention feature map carrying multiple channel weights using a corresponding encoding and decoding sub-unit to obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map includes: Using the dimensionality processing layer, based on the multiple channel weights, the pixel intensities of each channel at each pixel position in the attention feature map are weighted averaged to obtain the mean pixel intensity at each pixel position in the attention feature map; Utilizing the dimensionality processing layer, calculating the variance of each pixel position in the attention feature map based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensity of each channel; Utilizing the dimensionality processing layer, extracting pixel intensities of each channel at a preset number of pixel positions with the largest variance from the attention feature map, and integrating them to obtain a key feature map; Flattening the key feature graph to obtain a key feature sequence; the key feature sequence includes a plurality of elements; constructing fuzzy rules based on the membership function of each element in the key feature sequence using the encoder; Using the encoder, encoding the key feature sequence based on the fuzzy rule to obtain a k-th level sub-fuzzy feature sequence; Reshaping the k-th level sub-fuzzy feature sequence to obtain the k-th level sub-fuzzy feature graph; The decoder is used to upsample the k-th level sub-blurred feature map to match the input dimension, thereby obtaining the k-th level sub-reconstructed feature map.

4. The method according to claim 1, wherein The step of converting the k-th level fuzzy feature map into a k-th level fuzzy feature sequence using the k-th fuzzy transformation unit based on the learned k-th level linear transformation matrix, the k-th level label vector and the k-th level position vector includes: Using the k-th fuzzy transformation unit, performing a segmentation operation on the k-th level fuzzy feature map to obtain a plurality of k-th level fuzzy feature blocks corresponding to the k-th level fuzzy feature map; Using the k-th fuzzy transformation unit, adjusting the number of the first-level fuzzy feature blocks based on linear projection to obtain L k-th level fuzzy feature blocks; Using the k-th fuzzy transformation unit, learning a k-th linear transformation matrix, a k-th label vector, and a k-th position vector from the k-th fuzzy feature map; Using the kth fuzzy transformation unit, each The first fuzzy feature block and the Multiply the L-th linear transformation matrix to get Level fuzzy feature vector; Using the kth fuzzy transformation unit, the Level label vector and L first After the first fuzzy feature vector is concatenated with the Add the position vectors of the first level to get Level fuzzy feature sequence.

5. An image classification device based on a deep fuzzy convolutional neural network model, characterized in that: The deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network; The image classification device comprises: An image acquisition module, used to acquire original images; A feature extraction module is used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; A feature fusion module, configured to perform feature fusion processing on the multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence; A classification module, configured to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs; The feature extraction network includes an initial convolution block and K feature extraction blocks; the feature extraction module is specifically used to: Inputting the original image into the initial convolution block for feature extraction to obtain an initial feature map; Inputting the initial feature map into a first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map; The i-1th level reconstructed feature map is input into the i-th feature extraction block for feature extraction to obtain the i-th level fuzzy feature map and the i-th level reconstructed feature map; wherein, ; The K-th level reconstructed feature map is the depth convolution feature map; The multiple fuzzy feature maps include the first-level fuzzy feature map to the K-th-level fuzzy feature map; the feature fusion network includes K fuzzy transformation units, K fusion units and a fuser; the feature fusion module is specifically used to: Using the k-th fuzzy transformation unit, based on the learned k-th linear transformation matrix, k-th label vector and k-th position vector, the k-th level fuzzy feature map is converted into a k-th level fuzzy feature sequence; The k-th level fuzzy feature sequence is input into the k-th fusion unit to obtain the k-th level fusion sequence; wherein, k ; Add the first-level fusion sequence to the K-th-level fusion sequence to obtain a superimposed fuzzy feature sequence; Inputting the superimposed fuzzy feature sequence into the fusion device to obtain the comprehensive fuzzy feature sequence; The classification network includes a deep transformation unit, a cross layer, two attention units, two MLP heads and a classification layer; the classification module is specifically used to: Utilizing the depth transformation unit, based on the learned depth label vector, depth linear transformation matrix and depth position vector, the depth convolution feature map is converted into a depth convolution feature sequence; Using the cross layer, the deep label sequence and the deep feature block sequence in the deep convolution feature sequence are cross-joined with the fuzzy label sequence and the fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences; Normalizing the two original cross feature sequences to obtain a first cross feature sequence and a second cross feature sequence; the first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence; Using the two attention units, respectively processing the first cross feature sequence and the second cross feature sequence to obtain a first output sequence and a second output sequence; The first output sequence and the deep label sequence are added and normalized to obtain a first normalized sequence, and the second output sequence and the fuzzy label sequence are added and normalized to obtain a second normalized sequence; Using the two MLP heads to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value; The first output value and the second output value are input into a classification layer for processing to obtain the classification result.

6. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the image classification method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image classification method according to any one of claims 1 to 4 is implemented.

Images