Image classification method based on deep fuzzy convolutional neural network model and related device

Abstract

The invention provides an image classification method based on a deep fuzzy convolutional neural network model and a related device, and relates to the field of image processing. Wherein the deep fuzzy convolutional neural network model comprises a feature extraction network, a feature fusion network and a classification network. After an original image is obtained, feature extraction is performed on the original image by using a feature extraction network to obtain a plurality of fuzzy feature maps and deep convolution feature maps of which feature levels are gradually increased in sequence; performing feature fusion processing on the plurality of fuzzy feature maps by using a feature fusion network to obtain a comprehensive fuzzy feature sequence; and converting the deep convolution feature map into a deep convolution feature sequence by using a classification network, and performing deep interaction of features in different spaces with the comprehensive fuzzy feature sequence to obtain a classification result which represents the category of the original image. According to the method, multi-scale fuzzy feature extraction is realized, complex details in the image can be captured, the image classification precision is improved, and the robustness is relatively high.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image classification method and related devices based on a deep fuzzy convolutional neural network model. Background Art

[0002] Image classification technology is an important task in the field of computer vision and is widely used in fields such as autonomous driving, medical diagnosis, and remote sensing detection. Its goal is to assign input images to predefined categories. During the classification process, the image classification model needs to identify discriminative information in the image, learn local details and overall structure, and extract key features that are helpful for subsequent analysis from seemingly chaotic pixels. However, extracting effective discriminative features from complex image data, especially obtaining accurate classification results in uncertain scenarios such as noise, can still be a challenging task.

[0003] At present, a common solution in the field of image classification is to use image classification models based on traditional machine learning technologies such as support vector machines (SVM) and KNN (K-Nearest Neighbor). However, image classification models based on machine learning technology face the problem of feature engineering dependency, which requires tedious manual feature selection and professional knowledge, and due to the limitations of its shallow model structure, its learning and generalization capabilities are restricted when processing large-scale data sets.

[0004] As a result, deeper networks such as CNN (Convolutional neural network) and other deep learning models have gradually become mainstream. However, CNN-based models have strong local correlation, which may lead to the loss of spatial correlation and have certain limitations in processing long-distance dependencies in images, thus affecting its performance in processing larger and more complex patterns.

[0005] In addition, in recent years, the emergence of Transformer architecture models has further promoted the progress of image classification. Such models can effectively capture global information between pixels, making up for the shortcomings of CNN. However, due to the lack of prior knowledge such as inductive bias in CNN, models based on Transformer architecture may have limitations in capturing complex details, and require pre-training of large-scale datasets to achieve good classification accuracy.

[0006] Moreover, the two types of models based on CNN and Transformer architectures have difficulty learning useful discriminative features when there are unfavorable blur conditions in the image, such as repeated backgrounds, significant changes in lighting, and occlusion of target objects. And due to the complexity of the deep structure, the robustness of the model is poor and it is easy to be deceived by uncertain factors such as noise, which may lead to misjudgment of the model.

[0007] In summary, in the field of image classification, most existing image classification models only extract features of a single scale, ignoring the feature information of multi-scale features in the image, making the representation of feature capture incomplete. In addition, CNN models and Transformer-based models are less robust to uncertainty factors such as noise, resulting in lower classification accuracy. Summary of the invention

[0008] The purpose of the present invention is to provide an image classification method and related devices based on a deep fuzzy convolutional neural network model to improve the problems existing in the prior art.

[0009] The embodiments of the present invention can be implemented as follows:

[0010] In a first aspect, the present invention provides an image classification method based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network; the method includes:

[0011] Get the original image;

[0012] Inputting the original image into the feature extraction network, and using the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and deep convolution feature maps with increasing feature levels;

[0013] Using the feature fusion network to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence;

[0014] The classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs.

[0015] In a second aspect, the present invention provides an image classification device based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network, and a classification network; the image classification device includes:

[0016] An image acquisition module, used for acquiring original images;

[0017] A feature extraction module, used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and a deep convolution feature map with increasing feature levels;

[0018] A feature fusion module, used for performing feature fusion processing on the multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence;

[0019] The classification module is used to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs.

[0020] In a third aspect, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the image classification method as described in the first aspect above.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image classification method described in the first aspect is implemented.

[0022] Compared with the prior art, the embodiment of the present invention provides an image classification method and related devices based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0024] Figure 1A schematic diagram of the structure of a deep fuzzy convolutional neural network model provided in an embodiment of the present invention.

[0025] Figure 2 A flowchart of an image classification method provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of the structure of a feature extraction network provided in an embodiment of the present invention.

[0027] Figure 4 Schematic diagram of the structures of the RN-1 layer and the RN-2 layer respectively provided in the embodiments of the present invention.

[0028] Figure 5 Multiple grouping feature maps obtained by channel segmentation provided in an embodiment of the present invention.

[0029] Figure 6 A schematic diagram of the processing process of the sorting unit provided in an embodiment of the present invention.

[0030] Figure 7 A schematic diagram of the structure of a feature fusion network provided in an embodiment of the present invention.

[0031] Figure 8 A schematic diagram of the structure of a classification network provided in an embodiment of the present invention.

[0032] Fig. 9 A schematic diagram of the structure of an image classification device provided by an embodiment of the present invention.

[0033] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0035] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0037] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0038] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0039] Deep fuzzy system: From a broad perspective, a deep fuzzy system is a model that combines fuzzy theory and deep learning ideas, which can be divided into the following three types:

[0040] (1) A deep hierarchical fuzzy system is formed by stacking multiple fuzzy systems in different hierarchical ways;

[0041] (2) Hybrid deep fuzzy system that combines fuzzy theory or fuzzy thinking with deep learning models;

[0042] (3) Deep fuzzy neural networks that combine fuzziness with neurons and represent fuzzy processes such as fuzzification, fuzzy reasoning, and defuzzification in the form of neural networks;

[0043] They have the following characteristics: deep model structure, introduction of fuzzy theory, powerful representation ability, efficient ability to handle uncertainty, etc.

[0044] Although deep fuzzy systems have been successfully applied in energy management, financial analysis, risk detection and other fields, their potential in image recognition has not been tapped. The main reasons are as follows:

[0045] On the one hand, for hierarchical fuzzy systems and deep fuzzy neural networks, the high-dimensional properties of image data lead to the problem of rule explosion and excessive computational complexity. In addition, the inherent complexity of image data, including texture, color, shape, pattern, etc., is difficult to be directly mapped into fuzzy rules, resulting in the designed fuzzy rules being unable to fully cover the relationship between features and difficult to adapt to its dynamics and diversity;

[0046] On the other hand, for the hybrid deep blur system, since the blur system is essentially a global model, it fails to fully consider the local spatial correlation between neighboring pixels in the image. In addition, the existing models often focus on feature extraction at a single scale, which may ignore the rich information of multi-granular features in the image, which limits their ability to fully capture the intrinsic characteristics of the image, making the feature representation incomplete and intricate.

[0047] Therefore, the inventors proposed a deep fuzzy convolutional neural network model for image classification, which solves the limitations of existing CNN models, Transformer architecture models and deep fuzzy systems, and improves image classification accuracy. The proposed deep fuzzy convolutional neural network model needs to solve the following detailed technical problems:

[0048] (1) How to build a multi-level / multi-granularity feature extraction framework;

[0049] (2) How to alleviate the “curse of dimensionality” problem of deep fuzzy systems in high-dimensional problems;

[0050] (3) How to effectively fuse multi-level fuzzy features;

[0051] (4) How to use features of different scales / levels / granularities for effective classification.

[0052] The following is a detailed introduction to the image classification method based on the deep fuzzy convolutional neural network model provided in an embodiment of the present invention, so as to fully explain the solution process of the above four technical problems.

[0053] First, the model structure of the deep fuzzy convolutional neural network model proposed in the present invention is introduced. The deep fuzzy convolutional neural network model is a deep fuzzy system based on a multi-attention mechanism and multi-level hybrid guidance (A Multiple Attentions Based Multi-level Hybrid-guided Deep Fuzzy Convolutional Neural Network, MAMH-DFCNN), which consists of:

[0054] (1) Feature extraction network: A multi-level convolutional fuzzy feature extraction framework (MCFFEF) is used to extract deep convolutional features and multi-level fuzzy features from the original image, enriching the diversity of feature representation;

[0055] (2) Feature fusion network: The fuzzy feature attention fusion module (FFAFM) is used to fully fuse multi-level fuzzy features and reduce the redundant information of multi-level / multi-granularity features;

[0056] (3) Classification network: A hybrid-driven Cross Attention Classifier (HCAC) is used to deeply interact features in different spaces to obtain the final classification result.

[0057] The feature extraction network may include an initial convolution block and K feature extraction blocks from shallow to deep, and the number K of feature extraction blocks may be 3 to 6.

[0058] For example, when K = 4, see Figure 1 , Figure 1 A schematic diagram of the structure of a deep fuzzy convolutional neural network model provided in an embodiment of the present invention. Figure 1 In the example, Block 1 is the initial convolution block, and Block 2 to Block 5 are four feature extraction blocks. This example is only for illustration and is not intended to limit the number of feature extraction blocks.

[0059] Please refer to Figure 2 , Figure 2 A flowchart of an image classification method provided by an embodiment of the present invention, the execution subject of the method may be an electronic device deployed with the above-mentioned deep fuzzy convolutional neural network model, and the electronic device may be but not limited to: a computing device such as a smart phone, a smart tablet, a personal computer, a personal notebook, a server, etc. The method may include the following steps S101 to S104.

[0060] S101, obtaining an original image.

[0061] In this embodiment, the original image may be a face image, an object image (the image contains objects such as cars, mobile phones, computers or bags), an animal image, a remote sensing image (the image contains forests, cities, agricultural land or water, etc.), an environmental image collected by an autonomous vehicle (the image may include various traffic signs, pedestrians or other vehicles on the road), an industrial defect image (the image contains defects on the surface of industrial products, such as scratches, stains, cracks, etc.). The present invention does not limit the target object in the original image.

[0062] Optionally, in the training phase of the deep fuzzy convolutional neural network model, the original image may be an image of a batch of images (the number of images in a batch of images may be 32, 64, 128, 256, etc.) in the training data set used to input the model, and steps S101 to S104 are an introduction to the processing of an image in the training data set during the training process. In the application reasoning phase of the deep fuzzy convolutional neural network model, the original image is an image to be classified.

[0063] S102, inputting the original image into a feature extraction network, using the feature extraction network to extract features from the original image, and obtaining a plurality of fuzzy feature maps and a deep convolution feature map with increasing feature levels.

[0064] In this embodiment, the feature extraction network uses fuzzy reasoning, so the multiple fuzzy feature maps extracted are all fuzzy features, and the feature levels of the multiple fuzzy feature maps are successively increased, belonging to feature information of different scales captured in fuzzy spaces of different depths. Therefore, even if there are unfavorable fuzzy conditions such as repeated backgrounds, significant changes in illumination, and occlusion of target objects in the image, the multiple fuzzy feature maps extracted by the feature extraction network can still be used as effective discriminant features.

[0065] S103, using a feature fusion network to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence.

[0066] In this embodiment, a plurality of fuzzy feature maps are subjected to feature fusion processing to obtain a comprehensive fuzzy feature sequence.

[0067] S104. Use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result.

[0068] In this embodiment, the classification result represents the category to which the target object in the original image belongs. In the classification network, the comprehensive fuzzy feature sequence and the deep convolution feature map are deeply interacted with features in different spaces to achieve the category determination of the target object in the original image.

[0069] The image classification method provided by the embodiment of the present invention is implemented based on a deep fuzzy convolutional neural network model, which includes a feature extraction network, a feature fusion network and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness.

[0070] The following introduces the feature extraction process of the feature extraction network.

[0071] For step S102, the process of "using a feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels" may include the following sub-steps S2011 to S2023.

[0072] S2011. Input the original image into the initial convolution block for feature extraction to obtain an initial feature map.

[0073] Assume that the size of the original image X is represented as: Among them, C is the number of channels, H is the image height, and W is the image width. Represents the training data set. For the convenience of description, a convolutional layer (Conv), a batch normalization layer (BN) and a ReLu activation function are taken as a unit, called a C_B_R unit, and its formula can be expressed as: C_B_R(X)=ReLu(BN(Conv(X))).

[0074] exist Figure 1 Based on Figure 3 , the initial convolution block (Block 1) belongs to the pure convolution module, including a C_B_R unit and a maximum pooling unit (Max Pooling). When the original image X is input to the initial convolution block (Block1), the output is: in, is the number of feature channels passing through Block 1, H 1 is the image height after Block 1, W 1 It is the image width after Block 1.

[0075] S2012. Input the initial feature map into the first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map.

[0076] S2013. Input the i-1th level reconstructed feature map into the i-th feature extraction block for feature extraction to obtain the i-th level fuzzy feature map and the i-th level reconstructed feature map.

[0077] In this embodiment, i=2, 3, ..., K; the final K-th level reconstructed feature map is the deep convolution feature map.

[0078] Taking K=4 as an example, after obtaining the first-level fuzzy feature map and the first-level reconstructed feature map, the first-level reconstructed feature map is input into the second feature extraction block for feature extraction to obtain the second-level fuzzy feature map and the second-level reconstructed feature map; then the second-level reconstructed feature map is input into the third feature extraction block for feature extraction to obtain the third-level fuzzy feature map and the third-level reconstructed feature map; then the third-level reconstructed feature map is input into the fourth feature extraction block for feature extraction to obtain the fourth-level fuzzy feature map and the fourth-level reconstructed feature map, and the fourth-level reconstructed feature map is the deep convolution feature map. This example is only an example and is not used as a limitation on the size of K.

[0079] Optionally, the kth feature extraction block includes a feature extraction unit and a sorting unit, k=1, 2, ..., K; therefore, in the above steps S2012-S2013, the process of the kth feature extraction block performing feature extraction to obtain the kth level fuzzy feature map and the kth level reconstructed feature map may include the following sub-steps S001-S005:

[0080] S001. Use a feature extraction unit to extract features from an input feature map to obtain a convolution feature map.

[0081] In this embodiment, when k=1, the input feature map is an initial feature map; when k>1, the input feature map is a k-1th level reconstructed feature map.

[0082] For example, continue with Figure 3 It can be seen that the difference between the structures of Block 2 to Block 5 is the number of RN-2 layers in the feature extraction unit. Taking Block 2 as an example: the feature extraction unit of feature extraction block 1 (Block 2) includes one RN-1 layer, two RN-2 layers, and a 1x1 convolution layer. The structure of the RN-1 layer and the structure of the RN-2 layer are as follows: Figure 4 As shown. Assume that the convolution feature map obtained after the feature extraction unit in Block 2 is H 2 , W 2 They are the number of feature channels, image height, and image width after passing through the feature extraction unit in Block 2.

[0083] S002. Slice the convolution feature map through channels to obtain multiple grouped feature maps.

[0084] In this embodiment, each channel of each grouped feature map is a part of all channels of the convolutional feature map; the number of attention subunits, the number of encoding and decoding subunits, and the number of grouped feature maps are the same.

[0085] Among them, the sorting unit in the feature extraction block adopts a grouped interval type-2 generalized fuzzy hyperbolic tangent autoencoder layer (Grouped SE-IT2GFHTAs) based on the SE attention mechanism, which includes multiple SE-IT2GFHTA modules and two splicing layers. The SE-IT2GFHTA module includes an attention subunit and a codec subunit.

[0086] When processing high-dimensional data such as images, if the feature map is directly used as the input of the fuzzy system, it often faces the "dimensionality curse" problem. Therefore, the convolution feature map is channel sliced ​​to obtain multiple group feature maps, and each group feature map is input into a SE-IT2GFHTA module.

[0087] After passing through the feature extraction unit in Block 2, the convolution feature map is obtained For example, Assume that the number of channel slices is num g , then The num obtained after segmentation by the number of channels g The group feature map is as follows Figure 5 As shown, the i-th group feature map Then Figure 5 num g The grouped feature maps are input into the num in the sorting unit respectively g SE-IT2GFHTA modules are used for processing. The processing diagram is shown in Figure 6 shown.

[0088] S003. For each group feature map, use a corresponding attention sub-unit to perform squeezing operations and excitation operations on the group feature map in sequence based on the SE attention mechanism to obtain an attention feature map carrying multiple channel weights.

[0089] For the i-th group feature map It passes through the attention subunit and adopts the SE attention mechanism to obtain the attention feature map with different channel weights in, It is worth noting that the SE attention mechanism does not change the size of the feature map.

[0090] S004. Utilize a corresponding encoding and decoding sub-unit to extract fuzzy features based on an attention feature map carrying multiple channel weights, and obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map.

[0091] Perform the above steps S004 to S004 for each group feature map to obtain num g k-th sub-fuzzy feature map and numg The k-th sub-reconstructed feature map.

[0092] S005. Use two splicing layers to perform splicing operations and 1×1 convolution operations on all k-th level sub-fuzzy feature maps and all k-th level sub-reconstructed feature maps in sequence to obtain a k-th level fuzzy feature map and a k-th level reconstructed feature map.

[0093] In this embodiment, two splicing layers are used to perform splicing operations and 1×1 convolution operations on all k-th level sub-fuzzy feature maps and all k-th level sub-reconstructed feature maps in sequence, so as to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map.

[0094] The above step S004 involves fuzzy reasoning. In the process of fuzzy reasoning, when obtaining the activation strength of each rule, it is necessary to multiply the membership value of each variable (greater than 0 and less than 1). When the number of input variables of the fuzzy system is large, there will be a problem of numerical underflow, that is, the activation strength is infinitely close to 0. At this time, the rule is equivalent to not being activated, which will cause the final output of the fuzzy system to be very small. It is necessary to reduce the input variables of the fuzzy system based on multiple channel weights.

[0095] A codec subunit includes a dimension processing layer, an encoder, and a decoder. The encoder uses the interval type two generalized fuzzy hyperbolic tangent system (IT2GFHTS), and the decoder uses a transposed convolution operation. Figure 6 The processing process of the codec subunit in the first feature extraction block (Block 2) shown in the figure introduces the sub-steps S0041 to S0048 of the above step S004, "using a corresponding codec subunit to extract fuzzy features based on the attention feature map carrying multiple channel weights to obtain the k-th level sub-fuzzy feature map and the k-th level sub-reconstructed feature map."

[0096] S0041. Using the dimensionality processing layer, based on multiple channel weights, the pixel intensities of each channel at each pixel position in the attention feature map are weighted averaged to obtain the mean pixel intensity at each pixel position in the attention feature map.

[0097] S0042. Using the dimensionality processing layer, based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensity of each channel, calculate the variance at each pixel position in the attention feature map.

[0098] Attention feature maps with different channel weights For example, The pixel intensity at each pixel position in the channel is weighted by the variance, and the attention feature map can be obtained. The variance at each pixel position in .

[0099] Among them, the attention feature map is calculated The process of the variance at each pixel position in is:

[0100] Assuming attention feature map The pixel position in a channel is expressed as The pixel intensity at this position is Among them, i and j represent the height index and width index of the pixel in the feature map respectively, and c represents the channel index of the feature map.

[0101] Using the weights of each channel, the attention feature map The attention feature map can be obtained by weighted averaging the pixel intensities of all pixel positions in each channel. The variance pixel intensity mean at each pixel position in . Among them, the pixel intensity mean at the pixel position (i, j) is: The variance at this location is:

[0102] S0043. Utilize the dimensionality processing layer to extract the pixel intensities of each channel at a preset number of pixel positions with the largest variance from the attention feature map, and integrate them to obtain the key feature map.

[0103] In this embodiment, for the attention feature map From this we can extract the Z with the largest variance var The pixel intensity of each channel at the pixel position is integrated to obtain the key feature map The key feature map obtained after this processing And attention feature map Compared with H, the number of channels remains unchanged, but the image size increases from H 2 ×W 2 Lowered to Z 1 ×Z 2 .

[0104] Among them, the preset number is Z var , Z var The size can be set flexibly and is not limited here.

[0105] S0044. Flatten the key feature graph to obtain a key feature sequence.

[0106] In this embodiment, the key feature map After flattening, we get the key feature sequence: {x 1 ,x 2 ,…,x M}, The key feature sequence includes M elements.

[0107] S0045. Using the encoder, construct fuzzy rules based on the membership function of each element in the key feature sequence.

[0108] S0046. Using an encoder, encoding the key feature sequence based on fuzzy rules to obtain a k-th level sub-fuzzy feature sequence;

[0109] In this embodiment, for each x in the key feature sequence m (m=1,2,…,M), the designed membership function is as follows:

[0110]

[0111]

[0112] Among them, P MF is a positive membership function, N MF is the negative membership function, and are the upper and lower membership functions of the positive membership function, and is the upper and lower membership functions of the negative membership function, k m (m=1,2,…,M) is the center of the membership function, and are the standard deviations of the upper and lower membership functions, respectively. In order to make the membership function applicable to high-dimensional problems, and Improvements are made and and The size relationship between them is:

[0113]

[0114] Among them, D in is the input dimension, e -η The smallest positive floating-point value that can be represented by a computer.

[0115] The fuzzy rule form of the encoder (IT2GFHTS) can be expressed as:

[0116]

[0117] Among them, y n is the post-processing output, is an interval type-2 fuzzy set, c M is the consequent constant that corresponds one-to-one to the interval type-2 fuzzy set in the antecedent.

[0118] After flattening Input to the encoder (IT2GFHTS), the output is:

[0119]

[0120] Here, tanh() represents the hyperbolic tangent function.

[0121] By extending the above single-output encoder (IT2GFHTS) to multiple outputs to adapt to the structure of the autoencoder, the fuzzy rule form of the multi-output encoder (IT2GFHTS) can be obtained as follows:

[0122]

[0123] The output is:

[0124]

[0125] in, are all adjustable parameter vectors to be learned, and K is a vector composed of membership function centers.

[0126] Assume that the number of output variables of the encoder (IT2GFHTS) in each SE-IT2GFHTA module is the same, both Therefore, the second-level sub-fuzzy feature sequence output by the encoder (IT2GFHTS) in the i-th SE-IT2GFHTA module is:

[0127] S0047. Reshape the k-th level sub-fuzzy feature sequence to obtain the k-th level sub-fuzzy feature graph.

[0128] In this embodiment, the second-level sub-fuzzy feature sequence After reshaping (i.e. reshape operation), the second-level sub-fuzzy feature map is obtained as follows: FSFF stands for First-level Sub-fuzzy Feature.

[0129] S0048. Using the decoder, upsample the k-th level sub-blurred feature map to match the input dimension to obtain the k-th level sub-reconstructed feature map.

[0130] In this embodiment, the second-level sub-fuzzy feature map Upsampling is achieved by transposed convolution to obtain the second-level sub-reconstructed feature map:

[0131]

[0132] Among them, FSRF stands for First-level Sub-reconstructed Feature. In this way, the image height and image width of the second-level sub-reconstructed feature map obtained after the decoder processing become H 2 ×W 2 ,and Figure 5 Convolutional feature map before channel slicing The image height and image width are consistent.

[0133] Therefore, each encoding and decoding subunit in the first feature extraction block (Block 2) extracts fuzzy features from the attention feature map carrying multiple channel weights in the manner of steps S0041 to S0048 above, and num g The second-level sub-fuzzy feature maps, num g The second-level sub-reconstructed feature map.

[0134] Finally, for num g The second-level sub-blurred feature maps are concatenated and 1×1 convolved in sequence to obtain the second-level fuzzy feature map as follows:

[0135]

[0136] For num g The second-level sub-reconstructed feature maps are concatenated and 1×1 convolved in sequence to obtain the second-level reconstructed feature map as follows:

[0137]

[0138] Among them, for num g The second-level reconstructed feature map is obtained by sequentially performing concatenation and 1×1 convolution operations on the second-level sub-reconstructed feature maps. The number of channels is and Figure 5 Convolutional feature map before channel slicing The number of channels is the same.

[0139] Combination Figure 3 The above explanation of each sub-step of step S102 is based on Block 2. The feature extraction process of Block 3 to Block 5 is similar to this and will not be described in detail here.

[0140] So, when K = 4, Figure 3In the feature extraction network shown, the first feature extraction block (Block2) extracts the first-level fuzzy feature map (FFF) and the first-level reconstruction feature map (FRF), the second feature extraction block (Block 3) extracts the second-level fuzzy feature map (SFF) and the second-level reconstruction feature map (SRF), the third feature extraction block (Block 4) extracts the third-level fuzzy feature map (TFF) and the third-level reconstruction feature map (TRF), and the fourth feature extraction block (Block 5) extracts the fourth-level fuzzy feature map (FTFF) and the fourth-level reconstruction feature map (FTRF). The fourth-level reconstruction feature map (FTRF) is also a deep convolutional feature map (Deep Convolutional Feature, DCF).

[0141] When the original image is contaminated by noise, the recognition accuracy of the traditional CNN method is significantly reduced. This is because convolution, as a local operation, can be easily misled by noise.

[0142] From the above content, it can be seen that the present invention introduces SE-IT2GFHTA in the sorting unit of the feature extraction network, which has a strong uncertainty processing capability and can improve the robustness of the model. Since the fuzzy system is inherently nonlinear, SE-IT2GFHTA can perform more complex nonlinear transformations on the feature map. This process can extract more advanced semantic information, thereby enriching the granularity and hierarchy of the feature pattern. In addition, the membership function within SE-IT2GFHTA is always symmetric about the origin, which matches the distribution of the feature map after batch normalization (BN). This feature gives SE-IT2GFHTA a superior ability to capture features.

[0143] The following introduces the feature fusion process of the feature fusion network.

[0144] The K fuzzy feature maps extracted by the feature extraction network are: the first-level fuzzy feature map to the K-th level fuzzy feature map. These multi-level fuzzy feature maps capture feature information of different scales in fuzzy spaces of different depths, but each level of fuzzy feature map is obtained based on the reconstructed feature map of the previous level, which may lead to information duplication and redundancy.

[0145] In order to overcome the common problem of multi-level features, the present invention designs a pyramid-like feature fusion network, which includes K fuzzy transformation units, K fusion units and fusers. For example, when K=4, the result of the feature fusion network can be seen in Figure 7, the fusion unit uses N Transformer encoders, and the fuser uses one Transformer encoder. In this way, the feature fusion network can perform multi-granular information fusion on multi-level fuzzy features to obtain a more concise and comprehensive fuzzy feature representation, while alleviating the problem of information redundancy.

[0146] Therefore, for step S103, the process of "using a feature fusion network to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence" may include the following sub-steps S1031 to S1034.

[0147] S1031. Using the kth fuzzy transformation unit, based on the learned kth linear transformation matrix, kth label vector and kth position vector, convert the kth fuzzy feature map into a kth fuzzy feature sequence.

[0148] Optionally, sub-steps of step S1031 may include S10311 to S10315.

[0149] S10311. Using the kth fuzzy transformation unit, perform a segmentation operation on the kth level fuzzy feature map to obtain a plurality of kth level fuzzy feature blocks corresponding to the kth level fuzzy feature map, where k = 1, 2, ..., K.

[0150] Assume K = 4, combined Figure 7 , Figure 7 Each of the four fuzzy transformation units in converts the input fuzzy feature map into a fuzzy feature sequence. First, each level of fuzzy feature needs to be patch-split, and each level of fuzzy feature map is split to obtain a fuzzy feature block (fuzzy patch) with a resolution of P×P as follows:

[0151]

[0152] in, represents a first-level fuzzy feature block, Represents the number of first-level fuzzy feature blocks; represents a second-level fuzzy feature block, Represents the number of second-level fuzzy feature blocks; represents a third-level fuzzy feature block, Represents the number of third-level fuzzy feature blocks; represents a fourth-level fuzzy feature block, Represents the number of fourth-level fuzzy feature blocks.

[0153] S10312. Using the kth fuzzy transformation unit, adjust the number of first-level fuzzy feature blocks based on linear projection to obtain L kth-level fuzzy feature blocks.

[0154] In this embodiment, L is transformed into FFF , L SFF , L TFF and L FTFF Alignment is performed so that the number of fuzzy feature blocks at each level is L.

[0155] S10313. Using the k-th fuzzy transformation unit, learn the k-th linear transformation matrix, the k-th label vector and the k-th position vector from the k-th fuzzy feature map.

[0156] Since the Transformer of the fusion unit uses latent vectors of uniform size for processing at each level, this embodiment obtains the k-th linear transformation matrix through the learnable linear transformation g(·), and the k-th linear transformation matrix is ​​used to map fuzzy features of different granularities into a unified D-dimensional space. Similar to the [Class] token in ViT, this embodiment also supplements a learned k-th label vector to represent the comprehensive information of the k-th fuzzy feature map. In addition, in order to integrate the position information into each fuzzy feature block and retain the spatial structure between each fuzzy feature block, the k-th position vector is also learned from the k-th fuzzy feature map.

[0157] S10314, using the kth fuzzy transformation unit, multiplying each kth level fuzzy feature block by the kth level linear transformation matrix to obtain L kth level fuzzy feature vectors;

[0158] S10315. Using the kth fuzzy transformation unit, concatenate the kth level label vector and L kth level fuzzy feature vectors and add them to the kth level position vector to obtain the kth level fuzzy feature sequence.

[0159] In this embodiment, combined with Figure 7 , the obtained first-level fuzzy feature sequence to the fourth-level fuzzy feature sequence can be expressed as follows:

[0160]

[0161] In formula (7), Represent the first level fuzzy feature sequence to the fourth level fuzzy feature sequence respectively;

[0162] In formula (7), Respectively represent the first-level linear transformation matrix to the fourth-level linear transformation matrix;

[0163] In formula (7), Represent the first level position vector to the fourth level position vector respectively;

[0164] In formula (7), Represent the first-level label vector to the fourth-level label vector respectively;

[0165] In formula (7), They respectively represent the jth first-level fuzzy feature block, the jth second-level fuzzy feature block, the jth third-level fuzzy feature block, and the jth fourth-level fuzzy feature block; and j = 1, 2, 3, …, L.

[0166] S1032. Input the k-th level fuzzy feature sequence into the k-th fusion unit to obtain the k-th level fusion sequence.

[0167] In this embodiment, combined with Figure 7 , convert the first-level fuzzy feature sequence to the fourth-level fuzzy feature sequence: Input into four fusion units respectively, we can get the first-level fusion sequence to the fourth-level fusion sequence, which is expressed as follows:

[0168]

[0169] in, It is the first-level fusion sequence to the fourth-level fusion sequence obtained by processing four fusion units (i.e., Transformer).

[0170] S1033. Add the first-level fusion sequence to the K-th-level fusion sequence to obtain a superimposed fuzzy feature sequence.

[0171] In this embodiment, combined with Figure 7 , for the first-level fusion sequence to the fourth-level fusion sequence: Add pixel by pixel to get the superimposed fuzzy feature sequence T f , as shown below:

[0172]

[0173] S1034. Input the superimposed fuzzy feature sequence into the fuser to obtain a comprehensive fuzzy feature sequence.

[0174] In this embodiment, combined with Figure 7 , superimpose the fuzzy feature sequence T f Input the fusion device to obtain the comprehensive fuzzy feature sequence As shown below:

[0175]

[0176] in, Represents comprehensive fuzzy feature sequence The fuzzy label sequence in Represents comprehensive fuzzy feature sequence The fuzzy feature block sequence in . Figure 7 The "C" in each sequence represents the tag sequence part.

[0177] Combination Figure 3 and Figure 7 The above explanation of each sub-step of step S103 is based on the case where K=4, that is, when there are 4 feature extraction blocks (Block 2 to Block 5) in the feature extraction network. The explanation introduces the fusion process of the feature fusion network from the first-level fuzzy feature map to the fourth-level fuzzy feature map output by Block 2 to Block 5.

[0178] The following is an introduction to the processing of the classification network.

[0179] See also Figure 8 The classification network includes a deep transformation unit, a cross layer, two attention units, two MLP (Multi-Layer Perceptron) heads and a classification layer. The two attention units use two hybrid-driven multi-head cross attention modules, and the classification layer uses a softmax function.

[0180] Therefore, for the above step S104, the process of "using the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain the classification result" can include the following sub-steps S1041 to S1047.

[0181] S1041. Using a deep transformation unit, based on the learned deep label vector, deep linear transformation matrix and deep position vector, the deep convolution feature map is converted into a deep convolution feature sequence.

[0182] In this embodiment, the process of converting the deep convolution feature map into a deep convolution feature sequence by the deep transformation unit includes the following four steps:

[0183] (1) After reconstructing the multi-channel deep convolution feature map into a single-channel two-dimensional deep convolution feature map, the two-dimensional deep convolution feature map is segmented to obtain multiple convolution feature blocks;

[0184] (2) Adjust the number of convolution feature blocks based on linear projection to obtain L convolution feature blocks;

[0185] (3) Learn the depth label vector, depth linear transformation matrix and depth position vector from the two-dimensional depth convolution feature map;

[0186] (4) Multiply each convolution feature block with the depth linear transformation matrix to obtain L depth vectors, and then concatenate the depth label vector with the L depth vectors and add them to the depth position vector to obtain a deep convolution feature sequence.

[0187] That is, the process in which the depth transformation unit converts the two-dimensional depth convolution feature map into a depth convolution feature sequence is similar to the process in which the fuzzy transformation unit converts the k-th level fuzzy feature map into the k-th level fuzzy feature sequence, which will not be repeated here.

[0188] The process of obtaining the deep convolution feature sequence in step (4) can be expressed as:

[0189]

[0190] Combination Figure 8 , deep convolution feature sequence T DCF It is divided into two parts: in, Represents the deep convolution feature sequence T DCF The deep label sequence of Represents the deep convolution feature sequence T DCF The sequence of deep feature blocks in .

[0191] S1042. Using a cross layer, cross-join the deep label sequence and the deep feature block sequence in the deep convolutional feature sequence with the fuzzy label sequence and the fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross-feature sequences.

[0192] In this embodiment, combined with Figure 7 , in the cross layer, the deep convolution feature sequence T DCF The depth label sequence in Fuzzy label sequence in comprehensive fuzzy feature sequence After the exchange, the two original cross-feature sequences T1 and T2 are obtained as follows:

[0193]

[0194] S1043. Normalize both original cross-feature sequences to obtain a first cross-feature sequence and a second cross-feature sequence.

[0195] In this embodiment, the first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence. The first cross feature sequence T is obtained by normalizing the two original cross feature sequences T1 and T2. cross_1 and the second crossover feature sequence T cross_2 , as shown below:

[0196]

[0197] in, They are the normalized results of the normalized deep label sequence and the normalized fuzzy feature block sequence respectively; They are the normalized fuzzy label sequence and the normalized deep feature block sequence respectively.

[0198] S1044. Use two attention units to process the first cross-feature sequence and the second cross-feature sequence respectively to obtain a first output sequence and a second output sequence.

[0199] In this embodiment, T cross_1 and T cross_2 are fed into two attention units (hybrid-driven multi-head cross attention module). The first attention unit is fed with a normalized deep label sequence (i.e. ) as a guide, the second attention unit uses the normalized fuzzy label sequence (i.e. ) as a guide, the query vector, key vector, and value vector of the two attention units are as follows:

[0200]

[0201]

[0202] Among them, q 1 , k 1 、v 1 are the query vector, key vector, and value vector of the first attention unit, respectively; are the query transformation matrix, key transformation matrix, and value transformation matrix of the first attention unit respectively; q 2 , k 2 、v 2 are the query vector, key vector, and value vector of the second attention unit, respectively; They are the query transformation matrix, key transformation matrix, and value transformation matrix of the second attention unit, respectively.

[0203] Finally, the first output sequence MSA of the two attention units is output cross_1 and the second output sequence MSA cross_2 as follows:

[0204]

[0205]

[0206] MSA cross_1 =A cross_1 ·v cross_1

[0207] MSA cross_2 =A cross_2 ·v cross_2

[0208] in, represents the scaling factor, D is the total dimension shared by all heads of the multi-head criss-cross attention module, and h is the number of heads in the multi-head criss-cross attention module; A cross_1 Represents the attention weight determined by the first attention unit, A cross_2 Represents the attention weight determined by the second attention unit.

[0209] S1045: Add the first output sequence to the deep label sequence and perform normalization processing to obtain a first normalized sequence, and add the second output sequence to the fuzzy label sequence and perform normalization processing to obtain a second normalized sequence.

[0210] In this embodiment, combined with Figure 8 , the first output sequence MSA of the two attention units cross_1 and the second output sequence MSA cross_2 Respectively with the previous depth label sequence Fuzzy label sequence Two addition results are obtained by adding them together. This process once again realizes the deep information interaction between label sequences of different granularities. Then, the two addition results are normalized respectively to obtain the first normalized sequence and the second normalized sequence.

[0211] S1046. Use two MLP heads to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value.

[0212] In this embodiment, combined with Figure 8 , the two MLP heads process the first normalized sequence and the second normalized sequence respectively, and the first output value y cross_1 and the second output value y cross_s It can be expressed as follows:

[0213]

[0214] S1047: Input the first output value and the second output value into the classification layer for processing to obtain a classification result.

[0215] In this embodiment, the classification layer uses the softmax function, and the classification result y 2CAC It can be expressed as:

[0216] y HCAC =softmax(y cross_1 +y cross_2 )

[0217] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0218] (1) Compared with the existing models that only use convolution or only use the Transformer architecture, the deep fuzzy convolutional neural network model proposed in the present invention integrates convolution and multiple attention mechanisms, overcoming the local attributes and long-distance dependency problems of the CNN model and the lack of bias induction problem of the Transformer architecture model;

[0219] (2) In the deep fuzzy convolutional neural network model proposed in the present invention, the feature extraction network adopts a multi-level convolutional fuzzy feature extraction framework, which can extract fuzzy features at different levels. Compared with many existing single feature models, it has richer feature representation and more powerful feature learning capabilities;

[0220] (3) In the feature extraction network, each sorting unit adopts an interval type-II generalized fuzzy hyperbolic tangent autoencoder based on the SE attention mechanism, which can make the feature extraction network have strong noise robustness based on the powerful uncertainty processing ability of the fuzzy system;

[0221] (4) In the deep fuzzy convolutional neural network model proposed in the present invention, the feature extraction network fusion network adopts a pyramid-like fuzzy feature attention fusion module, which can effectively integrate multi-level fuzzy features to obtain comprehensive fuzzy features, realize the full mining of fuzzy feature information of different granularities, and reduce the information redundancy of multi-level features;

[0222] (5) In the deep fuzzy convolutional neural network model proposed in the present invention, the classification network adopts a hybrid-driven cross-attention classifier, which can perform deep interaction of features in different spaces, thereby guiding the model to classify more comprehensively and improving the classification accuracy.

[0223] In order to execute the corresponding steps in the above method embodiment and each possible implementation method, an implementation method of an image classification device based on a deep fuzzy convolutional neural network model is given below.

[0224] See also Fig. 9 , Fig. 9 The schematic diagram of the structure of the image classification device provided by the embodiment of the present invention is shown. The image classification device 200 comprises: an image acquisition module 210 , a feature extraction module 220 , a feature fusion module 230 and a classification module 240 .

[0225] An image acquisition module 210 is used to acquire an original image;

[0226] The feature extraction module 220 is used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels;

[0227] A feature fusion module 230 is used to perform feature fusion processing on multiple fuzzy feature maps using a feature fusion network to obtain a comprehensive fuzzy feature sequence;

[0228] The classification module 240 is used to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, and the classification result represents the category to which the target object in the original image belongs.

[0229] Optionally, the feature extraction network includes an initial convolution block and K feature extraction blocks; when the feature extraction module 220 is used to perform feature extraction on the original image using the feature extraction network to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels, it is specifically used to: input the original image into the initial convolution block for feature extraction to obtain an initial feature map; input the initial feature map into the first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map; input the i-1th level reconstructed feature map into the i-th feature extraction block for feature extraction to obtain an i-th level fuzzy feature map and an i-th level reconstructed feature map; wherein, i=2,3,…,K; the K-th level reconstructed feature map is a deep convolution feature map.

[0230] Optionally, the kth feature extraction block includes a feature extraction unit and a sorting unit, k=1, 2, ..., K; the sorting unit includes multiple attention sub-units, multiple encoding and decoding sub-units and two splicing layers. In the process of the kth feature extraction block performing feature extraction to obtain the kth level fuzzy feature map and the kth level reconstructed feature map, the feature extraction module 220 can be specifically used to: use the feature extraction unit to extract features from the input feature map to obtain a convolution feature map; wherein, when k=1, the input feature map is the initial feature map; when k>1, the input feature map is the k-1th level reconstructed feature map; perform channel slicing on the convolution feature map to obtain multiple grouped feature maps; each channel of each grouped feature map is a part of all channels of the convolution feature map; the number of attention sub-units, the number of encoding and decoding sub-units and the number of grouped feature maps The amount is the same; for each grouped feature map, a corresponding attention sub-unit is used to perform squeezing operations and excitation operations on the grouped feature map in sequence based on the SE attention mechanism to obtain an attention feature map carrying multiple channel weights; a corresponding encoding and decoding sub-unit is used to extract fuzzy features based on the attention feature map carrying multiple channel weights to obtain a k-th sub-fuzzy feature map and a k-th sub-reconstructed feature map; two splicing layers are used to perform splicing operations and 1×1 convolution operations on all k-th sub-fuzzy feature maps and all k-th sub-reconstructed feature maps in sequence to obtain a k-th fuzzy feature map and a k-th reconstructed feature map.

[0231] Optionally, the encoding and decoding subunit includes a dimensional processing layer, an encoder and a decoder; using a corresponding encoding and decoding subunit, extracting fuzzy features based on an attention feature map carrying multiple channel weights to obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map, the feature extraction module 220 can be specifically used to: using the dimensional processing layer, based on multiple channel weights, weighted average the pixel intensities of each channel at each pixel position in the attention feature map to obtain the mean pixel intensity at each pixel position in the attention feature map; using the dimensional processing layer, based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensities of each channel, calculate the pixel intensity of each pixel in the attention feature map. variance at the position; using the dimensionality processing layer, extract the pixel intensity of each channel at the preset number of pixel positions with the largest variance from the attention feature map, and integrate them to obtain the key feature map; flatten the key feature map to obtain a key feature sequence; the key feature sequence includes several elements; using the encoder, construct a fuzzy rule based on the membership function of each element in the key feature sequence; using the encoder, encode the key feature sequence based on the fuzzy rule to obtain the k-th level sub-fuzzy feature sequence; reshape the k-th level sub-fuzzy feature sequence to obtain the k-th level sub-fuzzy feature map; using the decoder, upsample the k-th level sub-fuzzy feature map to match the input dimension to obtain the k-th level sub-reconstructed feature map.

[0232] Optionally, the multiple fuzzy feature maps include first-level fuzzy feature maps to K-level fuzzy feature maps; the feature fusion network includes K fuzzy transformation units, K fusion units and a fuser. When the feature fusion module 230 is used to perform feature fusion processing on multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence, it can be specifically used to: use the k-th fuzzy transformation unit to convert the k-th fuzzy feature map into a k-th fuzzy feature sequence based on the learned k-th linear transformation matrix, k-th label vector and k-th position vector; input the k-th fuzzy feature sequence into the k-th fusion unit to obtain a k-th fusion sequence; where k = 1, 2, ..., K; add the first-level fusion sequence to the K-th fusion sequence to obtain a superimposed fuzzy feature sequence; input the superimposed fuzzy feature sequence into the fuser to obtain a comprehensive fuzzy feature sequence.

[0233] Optionally, the feature fusion module 230 is used to: use the kth fuzzy transformation unit to convert the kth level fuzzy feature map into a kth level fuzzy feature sequence based on the learned kth level linear transformation matrix, kth level label vector and kth level position vector, and is specifically used to: use the kth fuzzy transformation unit to perform a segmentation operation on the kth level fuzzy feature map to obtain multiple kth level fuzzy feature blocks corresponding to the kth level fuzzy feature map; use the kth fuzzy transformation unit to adjust the number of first-level fuzzy feature blocks based on linear projection to obtain L kth level fuzzy feature blocks; use the kth fuzzy transformation unit to learn the kth level linear transformation matrix, kth level label vector and kth level position vector from the kth level fuzzy feature map; use the kth fuzzy transformation unit to multiply each kth level fuzzy feature block with the kth level linear transformation matrix to obtain L kth level fuzzy feature vectors; use the kth fuzzy transformation unit to splice the kth level label vector and L kth level fuzzy feature vectors and add them to the kth level position vector to obtain the kth level fuzzy feature sequence.

[0234] Optionally, the classification network includes a deep transformation unit, a cross layer, two attention units, two MLP heads and a classification layer. In the process of using the classification network to perform deep interaction of features in different spaces between the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain the classification result, the classification module 240 can be specifically used to: use the deep transformation unit to convert the deep convolution feature map into a deep convolution feature sequence based on the learned deep label vector, deep linear transformation matrix and deep position vector; use the cross layer to cross-splice the deep label sequence and deep feature block sequence in the deep convolution feature sequence with the fuzzy label sequence and fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences; normalize both original cross feature sequences to obtain a first cross feature sequence and a second cross feature sequence; the first cross feature sequence includes the normalized The first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence; two attention units are used to process the first cross feature sequence and the second cross feature sequence respectively to obtain a first output sequence and a second output sequence; the first output sequence is added to the deep label sequence and then normalized to obtain a first normalized sequence, and the second output sequence is added to the fuzzy label sequence and then normalized to obtain a second normalized sequence; two MLP heads are used to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value; the first output value and the second output value are input into the classification layer for processing to obtain a classification result.

[0235] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the image classification device 200 described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0236] See also Fig.10 , Fig.10 The electronic device 300 includes a processor 310 , a memory 320 , and a bus 330 , wherein the processor 310 is connected to the memory 320 via the bus 330 .

[0237] The memory 320 may be used to store software programs, for example, software programs corresponding to the image classification device 200 provided in the embodiment of the present invention. The processor 310 executes various functional applications and data processing to implement the image classification method provided in the embodiment of the present invention by running the software programs stored in the memory 320.

[0238] Among them, the memory 320 can be but is not limited to: RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0239] The processor 310 may be an integrated circuit chip with signal processing capability. The processor 310 may be a general-purpose processor, including: CPU (Central Processing Unit), NP (Network Processor), SoC (System on Chip), etc.; it may also be: DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0240] Understandably, Fig.10The structure shown is for illustration only. The electronic device 300 may also include Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown. Fig.10 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0241] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the image classification method disclosed in the above embodiment is implemented. The computer-readable storage medium can be, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a FLASH disk, or an optical disk.

[0242] In summary, the embodiments of the present invention provide an image classification method and related devices based on a deep fuzzy convolutional neural network model, wherein the deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network. After acquiring the original image, the feature extraction network is used to extract features from the original image to obtain multiple fuzzy feature maps and deep convolution feature maps with increasing feature levels; the feature fusion network is used to perform feature fusion processing on multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; the classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, which represents the category to which the original image belongs. The present invention realizes multi-scale fuzzy feature extraction, can capture complex details in the image, improves image classification accuracy, and has strong robustness.

[0243] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image classification method based on a deep fuzzy convolutional neural network model, characterized in that: The deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network; the method includes: Get the original image; Inputting the original image into the feature extraction network, and using the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and deep convolution feature maps with increasing feature levels; Using the feature fusion network to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence; The classification network is used to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs.

2. The method according to claim 1, characterized in that The feature extraction network includes an initial convolution block and K feature extraction blocks; The step of extracting features from the original image using the feature extraction network to obtain a plurality of fuzzy feature maps and deep convolution feature maps with increasing feature levels comprises: Inputting the original image into the initial convolution block for feature extraction to obtain an initial feature map; Inputting the initial feature map into a first feature extraction block for feature extraction to obtain a first-level fuzzy feature map and a first-level reconstructed feature map; The i-1th level reconstructed feature map is input into the i-th feature extraction block for feature extraction to obtain the i-th level fuzzy feature map and the i-th level reconstructed feature map; wherein, i=2, 3, ..., K; the K-th level reconstructed feature map is the deep convolution feature map.

3. The method according to claim 2, characterized in that The k-th feature extraction block includes a feature extraction unit and a sorting unit, k=1, 2, ..., K; the sorting unit includes a plurality of attention sub-units, a plurality of encoding and decoding sub-units and two splicing layers; The process of extracting features by the k-th feature extraction block to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map includes: The feature extraction unit is used to extract features from the input feature map to obtain a convolution feature map; wherein, when k=1, the input feature map is the initial feature map; when k>1, the input feature map is the k-1th level reconstructed feature map; Channel slicing is performed on the convolution feature map to obtain a plurality of group feature maps; each channel of each group feature map is a part of all channels of the convolution feature map; the number of the attention subunits, the number of the encoding and decoding subunits, and the number of the group feature maps are the same; For each of the grouped feature maps, using a corresponding attention sub-unit, based on the SE attention mechanism, sequentially perform a squeezing operation and an excitation operation on the grouped feature map to obtain an attention feature map carrying multiple channel weights; Utilizing a corresponding encoding / decoding subunit, extracting fuzzy features based on the attention feature map carrying multiple channel weights to obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map; Two splicing layers are used to sequentially perform splicing operations and 1×1 convolution operations on all the k-th level sub-fuzzy feature maps and all the k-th level sub-reconstructed feature maps to obtain the k-th level fuzzy feature map and the k-th level reconstructed feature map.

4. The method according to claim 3, characterized in that The encoding and decoding subunit includes a dimension processing layer, an encoder and a decoder; The step of using a corresponding encoding and decoding subunit to extract fuzzy features based on the attention feature map carrying multiple channel weights to obtain a k-th level sub-fuzzy feature map and a k-th level sub-reconstructed feature map includes: Using the dimensionality processing layer, based on the multiple channel weights, weighted average the pixel intensities of each channel at each pixel point position in the attention feature map to obtain a mean pixel intensity at each pixel point position in the attention feature map; Utilizing the dimensionality processing layer, based on the mean pixel intensity at each pixel position in the attention feature map and the pixel intensity of each channel, the variance at each pixel position in the attention feature map is calculated; Utilizing the dimensionality processing layer, the pixel intensities of each channel at the positions of a preset number of pixels with the largest variance are extracted from the attention feature map, and integrated to obtain a key feature map; Flattening the key feature graph to obtain a key feature sequence; the key feature sequence includes a plurality of elements; Using the encoder, constructing fuzzy rules based on the membership function of each element in the key feature sequence; Using the encoder, encoding the key feature sequence based on the fuzzy rule to obtain a k-th level sub-fuzzy feature sequence; Reshaping the k-th level sub-fuzzy feature sequence to obtain the k-th level sub-fuzzy feature graph; The decoder is used to upsample the k-th level sub-blurred feature map to match the input dimension, thereby obtaining the k-th level sub-reconstructed feature map.

5. The method according to claim 1, characterized in that: The multiple fuzzy feature maps include first-level fuzzy feature maps to K-th-level fuzzy feature maps; the feature fusion network includes K fuzzy transformation units, K fusion units and a fuser; The step of using the feature fusion network to perform feature fusion processing on the multiple fuzzy feature maps to obtain a comprehensive fuzzy feature sequence includes: Using the k-th fuzzy transformation unit, based on the learned k-th linear transformation matrix, the k-th label vector and the k-th position vector, the k-th fuzzy feature map is converted into a k-th fuzzy feature sequence; Input the k-th level fuzzy feature sequence into the k-th fusion unit to obtain the k-th level fusion sequence; wherein k = 1, 2, ..., K; Add the first-level fusion sequence to the K-th-level fusion sequence to obtain a superimposed fuzzy feature sequence; The superimposed fuzzy feature sequence is input into the fuser to obtain the comprehensive fuzzy feature sequence.

6. The method according to claim 5, characterized in that The step of using the kth fuzzy transformation unit to convert the kth level fuzzy feature map into the kth level fuzzy feature sequence based on the learned kth level linear transformation matrix, the kth level label vector and the kth level position vector comprises: Using the k-th fuzzy transformation unit, a segmentation operation is performed on the k-th level fuzzy feature map to obtain a plurality of k-th level fuzzy feature blocks corresponding to the k-th level fuzzy feature map; Using the k-th fuzzy transformation unit, adjusting the number of the first-level fuzzy feature blocks based on linear projection to obtain L k-th level fuzzy feature blocks; Using the k-th fuzzy transformation unit, learning a k-th linear transformation matrix, a k-th label vector and a k-th position vector from the k-th fuzzy feature map; Using the k-th fuzzy transformation unit, multiply each k-th level fuzzy feature block by the k-th level linear transformation matrix to obtain L k-th level fuzzy feature vectors; The kth fuzzy transformation unit is used to concatenate the kth level label vector and L kth level fuzzy feature vectors and then add them to the kth level position vector to obtain a kth level fuzzy feature sequence.

7. The method according to claim 1, characterized in that The classification network includes a deep transform unit, a cross layer, two attention units, two MLP heads and a classification layer; The step of using the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result includes: Using the depth transformation unit, based on the learned depth label vector, depth linear transformation matrix and depth position vector, the depth convolution feature map is converted into a depth convolution feature sequence; Using the cross layer, the deep label sequence and the deep feature block sequence in the deep convolution feature sequence are cross-joined with the fuzzy label sequence and the fuzzy feature block sequence in the comprehensive fuzzy feature sequence to obtain two original cross feature sequences; The two original cross feature sequences are normalized to obtain a first cross feature sequence and a second cross feature sequence; the first cross feature sequence includes a normalized deep label sequence and a normalized fuzzy feature block sequence; the second cross feature sequence includes a normalized fuzzy label sequence and a normalized deep feature block sequence; Using the two attention units, respectively processing the first cross feature sequence and the second cross feature sequence to obtain a first output sequence and a second output sequence; The first output sequence is added to the deep label sequence and then normalized to obtain a first normalized sequence, and the second output sequence is added to the fuzzy label sequence and then normalized to obtain a second normalized sequence; Using the two MLP heads to process the first normalized sequence and the second normalized sequence respectively to obtain a first output value and a second output value; The first output value and the second output value are input into a classification layer for processing to obtain the classification result.

8. An image classification device based on a deep fuzzy convolutional neural network model, characterized in that: The deep fuzzy convolutional neural network model includes a feature extraction network, a feature fusion network and a classification network; The image classification device comprises: An image acquisition module, used for acquiring original images; A feature extraction module, used to input the original image into the feature extraction network, and use the feature extraction network to extract features from the original image to obtain a plurality of fuzzy feature maps and a deep convolution feature map with increasing feature levels; A feature fusion module, used for performing feature fusion processing on the multiple fuzzy feature maps using the feature fusion network to obtain a comprehensive fuzzy feature sequence; The classification module is used to use the classification network to perform deep interaction of features in different spaces on the comprehensive fuzzy feature sequence and the deep convolution feature map to obtain a classification result, wherein the classification result represents the category to which the target object in the original image belongs.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the image classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image classification method according to any one of claims 1 to 7 is implemented.

Images