Image classification method and device, electronic device, and storage medium

By improving the pooling layer of the convolutional neural network model and utilizing the secondary fuzzy importance mechanism to suppress noise propagation and optimize feature extraction, the recognition accuracy problem of traditional models under noise interference is solved, and higher image classification accuracy is achieved.

CN118587512BActive Publication Date: 2024-11-19CHANGSHU INSTITUTE OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411060859.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-19
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

Traditional convolutional neural network models are easily affected by noise during image recognition, leading to decreased recognition accuracy and an inability to effectively extract key information from regions of interest in images.

Method used

We improve the pooling layer of the convolutional neural network model by adopting a quadratic fuzzy importance mechanism. We calculate the weight values ​​of the channel and spatial importance mechanism through fuzzy logic technology, generate spatial feature maps, suppress the layer-by-layer propagation of noise in the model, and optimize the extraction of key features.

Benefits of technology

This improved the model's robustness to noise and the accuracy of image recognition, enhanced the model's ability to extract features from regions of interest, and improved the accuracy of image classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587512B_ABST
    Figure CN118587512B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the field of image classification technology, and provides an image classification method and device, electronic device, and storage medium. The method includes: obtaining an image to be classified; inputting the image to be classified into an image classification model to obtain a corresponding classification result; the image classification model is constructed based on a convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on a secondary fuzzy importance mechanism. The present disclosure can effectively utilize the characteristics of fuzzy logic technology to suppress the layer-by-layer transmission of noise in the image in the model, reduce the impact of noise on image recognition, and improve the model's robustness to noise and the accuracy of image recognition. It can also effectively utilize the importance mechanism to optimize the model's extraction of key features of the region of interest in the image, improve the generalization performance of the model and the accuracy of image classification, so that when the image classification model is used to classify the image to be classified, the accuracy of image classification is effectively improved, and the image classification result is more accurate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of image classification, and in particular to an image classification method and device, electronic equipment, and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, Convolutional Neural Network (CNN) has been widely used in fields such as communications, physics, circuits, signal processing, image processing, and computer vision. However, due to the easy presence of noise in images, traditional CNN models may be interfered by noise during the image recognition process, resulting in the model's recognition accuracy being affected. This is because, during the image recognition process, the noise in the image will be transmitted layer by layer along with the CNN model structure, making it impossible for the features extracted by the convolutional layer in the CNN model to fully represent the key information of the image's area of ​​interest, resulting in the inability of the last fully connected layer in the CNN model to accurately classify the image. Summary of the invention

[0003] The present disclosure aims to solve at least one of the problems existing in the prior art and provides an image classification method and device, an electronic device, and a storage medium.

[0004] In one aspect of the present disclosure, there is provided an image classification method, the image classification method comprising:

[0005] Get the image to be classified;

[0006] The image to be classified is input into an image classification model to obtain a corresponding classification result; wherein the image classification model is constructed based on a convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on a quadratic fuzzy importance mechanism.

[0007] Optionally, the pooling layer is used to perform a pooling operation on the feature map output by the previous convolutional layer connected to it based on the secondary fuzzy importance mechanism, generate a corresponding spatial feature map, and output the spatial feature map to the next convolutional layer connected to it.

[0008] Optionally, the pooling operation includes:

[0009] Based on fuzzy logic technology, the channel importance mechanism is used to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolutional layer connected to it;

[0010] Based on the channel weight value, determining a channel feature map;

[0011] Based on fuzzy logic technology, a spatial importance mechanism is used to calculate a shared weight map of all element positions in the channel feature map;

[0012] Based on the shared weight map, the spatial feature map is determined.

[0013] Optionally, the fuzzy logic technology is used to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolutional layer connected thereto by using the channel importance mechanism, including:

[0014] Determine the number of fuzzy partitions P , based on the fuzzy partition number P , generating the center of the Gaussian membership function and kernel width , the center of the Gaussian membership function Expressed as formula 1, where min k,z Indicates k The output of the convolutional layer z The minimum value of all elements in the feature map, max k,z Indicates k The output of the convolutional layer z The maximum value of all elements in the feature map, k represents the convolutional layer number, z Indicates the channel number and 1≤ z ≤ Z k , Z k Indicates k The number of channels of the feature map output by the convolutional layer, p Indicates the fuzzy partition sequence number and 1≤ p ≤ P :

[0015] Formula 1;

[0016] Using the Gaussian membership function, based on formula 2 k All elements in all feature maps output by the convolutional layer are blurred to obtain the corresponding P fuzzy membership values, where Indicates k The output of the convolutional layer z The element in the i-th row and j-th column of the feature map is express The corresponding p The fuzzy membership value of a Gaussian membership function is:

[0017] Formula 2;

[0018] According to formula 3, calculate the k The output of the convolutional layer zThe maximum membership and value corresponding to the feature map ,in, H k Indicates k The height of the feature map output by the convolutional layer, W k Indicates k The width of the feature map output by the convolutional layer:

[0019] Formula 3;

[0020] According to Formula 4, calculate the maximum membership and value accumulation corresponding to the kth convolutional layer :

[0021] Formula 4;

[0022] According to Formula 5, the channel weight values ​​corresponding to each feature map output by the kth convolutional layer are calculated respectively:

[0023] Formula 5;

[0024] in, represents the output of the kth convolutional layer z The channel weight value corresponding to the feature map.

[0025] Optionally, determining a channel feature map based on the channel weight value includes:

[0026] Each feature map output by the k-th convolutional layer is multiplied by its corresponding channel weight value to obtain each channel feature map corresponding to the k-th convolutional layer.

[0027] Optionally, the method of calculating the shared weight map of all element positions in the channel feature map using a spatial importance mechanism based on fuzzy logic technology includes:

[0028] Based on the Gaussian membership function, determine the corresponding element of each channel feature map corresponding to the kth convolutional layer P fuzzy membership values;

[0029] Calculate the maximum membership and value of each channel feature map corresponding to the kth convolutional layer, and obtain the corresponding spatial maximum membership value map;

[0030] Add the elements at the same position in each spatial maximum membership value map corresponding to the kth convolutional layer to obtain the corresponding channel direction element and value map;

[0031] Divide the channel direction element and each element in the value map by the number of channels Z k, get the corresponding channel direction mean map;

[0032] Each element in the channel direction mean map is divided by the sum of all elements in the channel direction mean map to obtain the shared weight map corresponding to the kth convolutional layer.

[0033] Optionally, determining the spatial feature map based on the shared weight map includes:

[0034] Each channel feature map corresponding to the k-th convolutional layer is multiplied by the corresponding shared weight map to obtain the spatial feature map corresponding to the k-th convolutional layer.

[0035] Another aspect of the present disclosure provides an image classification device, the image classification device comprising:

[0036] An acquisition module, used for acquiring images to be classified;

[0037] A classification module is used to input the image to be classified into an image classification model to obtain a corresponding classification result; wherein the image classification model is constructed based on a convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on a quadratic fuzzy importance mechanism.

[0038] Another aspect of the present disclosure provides an electronic device, including:

[0039] at least one processor; and,

[0040] a memory communicatively connected to at least one processor; wherein,

[0041] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image classification method described above.

[0042] Another aspect of the present disclosure provides a computer-readable storage medium storing a computer program, which implements the image classification method described above when executed by a processor.

[0043] Compared with the prior art, the present disclosure improves the pooling layer of the convolutional neural network model based on the secondary fuzzy importance mechanism to obtain an image classification model, which can effectively utilize the characteristics of fuzzy logic technology to suppress the layer-by-layer transmission of noise in the image in the model, reduce the impact of noise on image recognition, and improve the model's robustness to noise and the accuracy of image recognition. It can also effectively utilize the importance mechanism to optimize the model's extraction of key features of the region of interest in the image, improve the generalization performance of the model and the accuracy of image classification, so that when the image classification model is used to classify the image to be classified, the accuracy of image classification can be effectively improved, and the image classification result can be more accurate. . BRIEF DESCRIPTION OF THE DRAWINGS

[0044] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0045] Figure 1 A flowchart of an image classification method provided in one embodiment of the present disclosure;

[0046] Figure 2 A schematic diagram of a handwritten image provided by another embodiment of the present disclosure;

[0047] Figure 3 A schematic diagram of a clothing image provided for another embodiment of the present disclosure;

[0048] Figure 4 An overall architecture diagram of an image classification model provided by another embodiment of the present disclosure;

[0049] Figure 5 A flow chart of a pooling operation provided in accordance with another embodiment of the present disclosure;

[0050] Figure 6 A flow chart of a pooling operation provided in accordance with another embodiment of the present disclosure;

[0051] Figure 7 A flowchart of an image classification method provided in another embodiment of the present disclosure;

[0052] Figure 8 A schematic flow chart of a channel importance mechanism provided for another embodiment of the present disclosure;

[0053] Fig. 9 A schematic flow chart of a spatial importance mechanism provided for another embodiment of the present disclosure;

[0054] Fig.10A schematic diagram of the structure of an image classification device provided by another embodiment of the present disclosure;

[0055] Fig.11 A schematic structural diagram of an electronic device provided in another embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. However, it can be understood by those skilled in the art that in each embodiment of the present disclosure, many technical details are proposed in order to enable readers to better understand the present disclosure. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed for protection in the present disclosure can also be implemented. The division of the following embodiments is for the convenience of description and should not constitute any limitation on the specific implementation of the present disclosure. The various embodiments can be combined and referenced with each other without contradiction.

[0057] One embodiment of the present disclosure relates to an image classification method, the process of which is as follows: Figure 1 As shown, including:

[0058] Step S110, obtaining an image to be classified.

[0059] Specifically, the image to be classified can be any type of image that needs to be classified. For example, the image to be classified can be various handwritten images from the MNIST handwritten dataset, such as Figure 2 Alternatively, the image to be classified can also be various clothing images from the Fashion-MNIST clothing dataset, such as Figure 3 As shown, the images include T-shirt / top images, trousers images, pullover images, dress images, coat images, sandals images, shirt images, sneakers images, bag images, ankle boots images, etc. Of course, the images to be classified may also be other types of images, which is not limited in this embodiment.

[0060] Step S120, input the image to be classified into the image classification model to obtain the corresponding classification result; wherein the image classification model is constructed based on the convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on the quadratic fuzzy importance mechanism.

[0061] Specifically, the image classification model in this embodiment is obtained by improving the standard convolutional neural network model, especially by improving the pooling layer in the standard convolutional neural network model.

[0062] Generally speaking, a standard convolutional neural network model usually includes an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. There are multiple convolution layers and pooling layers, and each convolution layer is followed by a pooling layer. For image classification problems, a Softmax layer, i.e., a normalized exponential function layer, is usually connected after the fully connected layer to convert the output of the fully connected layer into a probability distribution. The output layer is connected after the Softmax layer to output the final classification result through the output layer.

[0063] Based on the standard convolutional neural network model, this implementation sets a pooling layer between every two adjacent convolutional layers, and all pooling layers are constructed based on the secondary fuzzy importance mechanism. Figure 4 As shown, the image classification model provided by this embodiment may include an input layer, a convolution layer, a pooling layer, a fully connected layer, a Softmax Gaussian connection layer and an output layer. Among them, the number of convolution layers and pooling layers is multiple, and a pooling layer is arranged between each two adjacent convolution layers, and all pooling layers are double fuzzy importance mechanism pooling layers. The double fuzzy importance mechanism pooling layer is used to perform pooling operations based on a secondary fuzzy importance mechanism, especially a channel importance mechanism and a spatial importance mechanism based on fuzzy logic technology.

[0064] Compared with the prior art, the image classification method provided by the embodiment of the present disclosure improves the pooling layer of the convolutional neural network model based on the secondary fuzzy importance mechanism to obtain an image classification model. The characteristics of the fuzzy logic technology can be effectively utilized to suppress the layer-by-layer transmission of noise in the image in the model, reduce the impact of noise on image recognition, and improve the model's robustness to noise and the accuracy of image recognition. The importance mechanism can also be effectively utilized to optimize the model's extraction of key features of the region of interest in the image, improve the generalization performance of the model and the accuracy of image classification, so that when the image classification model is used to classify the image to be classified, the accuracy of image classification can be effectively improved, and the image classification result can be made more accurate.

[0065] Exemplarily, the pooling layer is used to perform a pooling operation on the feature map output by the previous convolutional layer connected to it based on a quadratic fuzzy importance mechanism, generate a corresponding spatial feature map, and output the spatial feature map to the next convolutional layer connected to it.

[0066] Specifically, the convolution layer is used to extract features from the input image and output the corresponding feature map. Therefore, each pooling layer in the image classification model can generate a corresponding spatial feature map based on the feature map output by the previous convolution layer connected to it, based on the secondary fuzzy importance mechanism, and output the spatial feature map to the next convolution layer connected to it, so as to further suppress the layer-by-layer transmission of noise in the image in the model, improve the model's robustness to noise, and optimize the extraction of key features in the image, thereby improving the performance and accuracy of the model.

[0067] For example, Figure 5 As shown, the pooling operation includes:

[0068] Step S510, based on fuzzy logic technology, using the channel importance mechanism to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolutional layer connected to it.

[0069] Specifically, step S510 mainly uses fuzzy logic technology and channel importance mechanism to preliminarily suppress the layer-by-layer transmission of noise in the image in the model, and preliminarily optimizes the extraction of key features in the image.

[0070] For example, the image classification model k The output of the convolutional layer z The feature map is denoted as ,in, k represents the convolutional layer number, H k Indicates k The height of the feature map output by the convolutional layer, W k Indicates k The width of the feature map output by the convolutional layer, z Indicates the channel number and 1≤ z ≤ Z k , Z k Indicates k The number of channels of the feature map output by the convolutional layer is as follows: Step S510 includes:

[0071] Determine the number of fuzzy partitions P , where the fuzzy partition number P The number of fuzzy partitions is consistent with the number of Gaussian membership functions. P The specific value of can be selected and set according to actual needs, for example, the fuzzy partition number P The value can be 3, or the fuzzy partition number P The value can also be 5, 7, etc., which is not limited in this embodiment. P, generating two parameters of the Gaussian membership function, namely the center and kernel width Among them, the kernel width The value of can be set according to actual needs, for example, the kernel width The value can be set to 1, of course, the kernel width The value of can also be set to other values, such as 2, 3, 4, etc., which is not limited in this embodiment. The center of the Gaussian membership function It is expressed as formula 1, p Indicates the fuzzy partition sequence number and 1≤ p ≤ P , min k,z Indicates k The output of the convolutional layer z The minimum value of all elements in the feature map, max k,z Indicates k The output of the convolutional layer z The maximum value of all elements in the feature map:

[0072] Formula 1.

[0073] Using the Gaussian membership function, based on formula 2 k All elements in all feature maps output by the convolutional layer are blurred to obtain the corresponding P fuzzy membership values, where Indicates k The output of the convolutional layer z The element in the i-th row and j-th column of the feature map is express The corresponding p The fuzzy membership value of a Gaussian membership function is:

[0074] Formula 2.

[0075] In getting the k Each element in each feature map output by the convolutional layer corresponds to P After the fuzzy membership value is obtained, the first k The feature maps output by the convolutional layers correspond to P channel fuzzy membership value graph, where the k The output of the convolutional layer z The feature map corresponding to p The fuzzy membership value map of the channel is k The output of the convolutional layer z Each element in the feature map corresponds to the pThe fuzzy membership values ​​are arranged in matrix form according to the positions of the corresponding elements.

[0076] According to formula 3, calculate the k The output of the convolutional layer z The maximum membership and value corresponding to the feature map , among which, k The output of the convolutional layer z The maximum membership and value corresponding to the feature map , by calculating the k The output of the convolutional layer z The sum of all element values ​​in each fuzzy membership value map corresponding to each feature map, and the maximum value is selected from each sum value to obtain:

[0077] Formula 3.

[0078] Correspondingly, with the maximum membership and value The corresponding channel fuzzy membership value graph is k The output of the convolutional layer z The maximum membership value map of the channel corresponding to the feature map. k Each feature map output by the convolutional layer corresponds to a channel maximum membership value map, that is, k The number of channel maximum membership value maps corresponding to the convolutional layer is the same as the number of k The number of feature maps output by the convolutional layers is the same, both Z k . k The output of the convolutional layer z The channel maximum membership value map corresponding to the feature map is recorded as ,in, Indicates the height of the channel maximum membership value graph, Indicates the width of the maximum membership value graph of the channel. Since the size of the maximum membership value graph of the channel is k The output of the convolutional layer z The size of the feature maps is the same, so the size of the channel maximum membership value map is also , that is to say, The value of H k same, The value of W k same.

[0079] According to Formula 4, calculate the maximum membership and value accumulation corresponding to the kth convolutional layer :

[0080] Formula 4.

[0081] Since we have obtained the first k The output of the convolutional layer z The maximum membership and value corresponding to the feature map , the maximum membership and value is a numerical value, so k All feature maps output by the convolutional layers, i.e., the feature maps from 1 to Z k The maximum membership and value corresponding to the feature maps By adding up, we can get k The maximum membership and value accumulation corresponding to the convolutional layers , in other words, in Formula 4 It can also be expressed as .

[0082] According to formula 5, calculate the k The channel weight values ​​corresponding to each feature map output by the convolutional layer are:

[0083] Formula 5.

[0084] in, Indicates k The output of the convolutional layer z The channel weight value corresponding to the feature map. k The maximum membership and value of each feature map output by the convolutional layer are divided by k The maximum membership and value accumulation corresponding to the convolutional layers , you can get the k The channel weight values ​​corresponding to each feature map output by the convolutional layer. It can also be expressed as , so in Formula 5 It can also be expressed as .Will In the expression z The values ​​are 1 to Z k , you can get the k Each feature map output by the convolutional layer corresponds to the channel weight value.

[0085] Step S520, determining a channel feature map based on the channel weight value.

[0086] Specifically, step S520 may k The channel weight values ​​corresponding to each feature map output by the convolutional layer are respectively integrated into the corresponding feature map to obtain the first kEach feature map output by the convolutional layers corresponds to a channel feature map to further optimize the extraction of key features in the image.

[0087] Exemplarily, step S520 includes: k Each feature map output by the convolutional layer is multiplied by its corresponding channel weight value to obtain the k The channel feature maps corresponding to the convolutional layers.

[0088] For example, for k The output of the convolutional layer z For a feature map, its corresponding channel weight value Multiplying them, we can get k The convolutional layer corresponds to the z The channel feature map is the output of the kth convolutional layer. z The channel feature map corresponding to the feature map. That is, for the kth convolutional layer output z Feature Map For example, its corresponding channel feature map It can be expressed as ,in, Represents the height of the channel feature map, Indicates the width of the channel feature map. And, the channel feature map The size and k The output of the convolutional layer z Feature Map are of the same size, both ,Right now, The value of H k , The value of W k . No. k The number of channel feature maps corresponding to the convolutional layer is the same as that of the k The number of feature maps output by the convolutional layers is also the same, both Z k .Will z The values ​​are 1 to Z k , you can get the k All feature maps output by the convolutional layers correspond to channel feature maps. The channel feature map is also a preliminary enhanced feature map obtained by using the channel importance mechanism based on fuzzy logic technology.

[0089] By passing the k Each feature map output by the convolutional layer is multiplied by its corresponding channel weight value to obtain the kThe channel feature maps corresponding to the convolutional layers can well integrate the channel weight values ​​into the corresponding feature maps, thereby further optimizing the extraction of key features in the image through the channel weight values.

[0090] Step S530, based on fuzzy logic technology, using the spatial importance mechanism to calculate the shared weight map of all element positions in the channel feature map.

[0091] Specifically, step S530 mainly uses fuzzy logic technology and spatial importance mechanism to further suppress the layer-by-layer transmission of noise in the image in the model, and further optimizes the extraction of key features in the image.

[0092] For example, when k The output of the convolutional layer z Feature Map Corresponding channel feature map When, step S530 includes:

[0093] Based on the Gaussian membership function, determine the k Each element in each channel feature map corresponding to the convolution layer corresponds to P The Gaussian membership function used in this step S510 is the Gaussian membership function, and its parameters include the center and kernel width ,center It is expressed as the above formula 1. k When the fuzzy membership values ​​corresponding to the elements in the channel feature maps corresponding to the convolution layers are respectively calculated, refer to the above formula 2 and replace the fuzzy membership values ​​in formula 2 with Replace with k The convolutional layer corresponds to the z The i-th row and j-th column element in the channel feature map , you can get The corresponding p The fuzzy membership value of a Gaussian membership function ,Right now, It can be expressed as . Calculate separately z The values ​​are 1 to Z k hour p The values ​​are 1 to P of , we can get the corresponding values ​​of each element in each channel feature map corresponding to the kth convolutional layer P A fuzzy membership value.

[0094] In particular, each element in each channel feature map corresponding to the kth convolutional layer corresponds to PAfter the fuzzy membership value is obtained, the first k The channel feature maps corresponding to the convolutional layers correspond to P A fuzzy membership value graph, where k The convolutional layer corresponds to the z The channel feature map corresponds to the p A fuzzy membership value graph, composed of k The convolutional layer corresponds to the z Each element in the channel feature map corresponds to the p The fuzzy membership values ​​are arranged in matrix form according to the positions of the corresponding elements.

[0095] Calculate the k The maximum membership and value of each channel feature map corresponding to the convolution layer are obtained to obtain the corresponding spatial maximum membership value map. k Each channel feature map corresponding to the convolutional layer corresponds to P A fuzzy membership value graph, therefore, in calculating the k When the maximum membership and value of each channel feature map corresponding to the convolution layer are respectively calculated, the first k The convolutional layer corresponds to the z The sum of all element values ​​in each fuzzy membership value map corresponding to the channel feature map is then selected from each sum value and the maximum value is used as the kth convolution layer corresponding to the kth convolution layer. z The maximum membership value and value corresponding to the channel feature map, and the fuzzy membership map corresponding to the maximum membership value and value is the k The convolutional layer corresponds to the z The spatial maximum membership value map corresponding to the channel feature map.

[0096] With k The output of the convolutional layer z The maximum membership degree and value corresponding to the feature map are similar, and the kth convolution layer corresponding to the z The maximum membership and value corresponding to the channel feature map is recorded as ,but It can be expressed as . With the maximum membership and value The corresponding fuzzy membership value graph is k The convolutional layer corresponds to the z The spatial maximum membership value map corresponding to the channel feature map. k Each channel feature map corresponding to the convolutional layer has its corresponding spatial maximum membership value map, that is, k The number of spatial maximum membership value maps corresponding to the convolutional layer is the same as that of the k The number of channel feature maps corresponding to the convolutional layers is the same, bothZ k . k The convolutional layer corresponds to the z The spatial maximum membership value map corresponding to the channel feature map is recorded as ,in, Represents the height of the graph with the maximum membership value in space, Represents the width of the spatial maximum membership value map. Since the size of the spatial maximum membership value map is k The size of the channel feature map corresponding to the convolutional layer is consistent, and the size of the channel feature map is consistent with the size of its corresponding feature map. Therefore, the size of the spatial maximum membership value map is also , that is to say, The value of H k same, The value of W k same.

[0097] The first k The elements at the same position in each spatial maximum membership value map corresponding to the convolutional layer are added together to obtain the corresponding channel direction element and value map. k The convolutional layers correspond to Z k The maximum membership value graphs of each space are Therefore, the maximum membership value graphs in each space The elements are added according to their positions to obtain the corresponding sum values ​​at each position. Then the matrix composed of the sum values ​​at each position according to their positions is the corresponding channel direction element sum value map. The size of the channel direction element sum value map is also For the k For a convolutional layer, its corresponding channel direction element and value map can be recorded as .

[0098] Divide the channel direction elements and each element in the value map by the number of channels Z k , and obtain the corresponding channel direction mean map. Specifically, for the k For a convolutional layer, its corresponding channel direction element and value map Each element in is divided by the number of channels Z k , we can get the corresponding channel direction mean map , where the channel direction mean map It can be expressed as , channel direction mean map The size is The matrix of .

[0099] Divide each element in the channel direction mean map by the sum of all elements in the channel direction mean map to get the k The shared weight map corresponding to the convolutional layer. Specifically, for the k For a convolutional layer, its corresponding channel direction mean map Each element in is divided by the channel direction mean map The sum of the elements in can be obtained k The shared weight map corresponding to the convolutional layers Among them, the shared weight map The i-th row and j-th column element in is denoted by , the channel direction mean map The i-th row and j-th column element in is denoted by , the channel direction mean map The sum of all elements in is recorded as , then .

[0100] Step S540: determining a spatial feature map based on the shared weight map.

[0101] Specifically, step S540 can integrate the shared weight map into each feature map output by the convolution layer, so as to obtain a spatial feature map corresponding to each feature map, so as to further optimize the extraction of key features in the image.

[0102] Exemplarily, step S540 includes: multiplying each channel feature map corresponding to the kth convolutional layer by its corresponding shared weight map to obtain a spatial feature map corresponding to the kth convolutional layer.

[0103] Specifically, for the k The convolutional layer corresponds to the z Channel feature map For example, compare it with k The shared weight map corresponding to the convolutional layers Multiplying them, we can get k The convolutional layer corresponds to the z Spatial feature map That is, for the k For a convolutional layer, its corresponding spatial feature map It can be expressed as ,in, Represents the height of the spatial feature map, Indicates the width of the spatial feature map. Represents the multiplication of the elements at the corresponding positions of the same type matrix, that is, the channel feature map Each element in is respectively related to the shared weight graph Multiply the elements at the corresponding positions in . And, the spatial feature map The size and k The convolutional layer corresponds to the z Channel feature map are of the same size, both ,Right now, The value of H k , The value of W k .Will z The values ​​are 1 to Z k , you can get the k All spatial feature maps corresponding to the convolutional layers. The spatial feature map is also a secondary enhanced feature map obtained by using the spatial importance mechanism based on fuzzy logic technology, and is also the input of the next convolutional layer connected to the pooling layer that generates the spatial feature map.

[0104] By multiplying each channel feature map corresponding to the kth convolutional layer by its corresponding shared weight map, the spatial feature map corresponding to the kth convolutional layer is obtained. The shared weight map can be well integrated into the channel feature maps, thereby further optimizing the extraction of key features in the image through the shared weight map.

[0105] In summary, the embodiment of the present disclosure uses the pooling layer to perform a pooling operation on the convolutional feature map through the channel importance mechanism and the spatial importance mechanism. Figure 6 , for the k The feature map after convolution of the convolution layer Generally speaking, the channel importance mechanism first determines the feature map The corresponding channel maximum membership value diagram , then based on the channel maximum membership value map Determine the feature map The corresponding channel weight value , the feature map The corresponding channel weight value Multiply to get the feature map The corresponding channel feature map Spatial importance mechanism in channel feature maps Based on the above, first determine the corresponding spatial maximum membership value map , then based on the spatial maximum membership value map Determine the corresponding shared weight graph , the channel feature map And the corresponding shared weight graph Multiply them together to get the corresponding spatial feature map, which is the secondary enhanced feature map .

[0106] In order to enable those skilled in the art to better understand the above implementation, a specific example is provided below for illustration.

[0107] Combined Figure 7 , an image classification method, comprising the following steps:

[0108] Get the image to be classified, and adjust the size of the image to be classified to a specified size. The specified size here can meet the input image size requirement of the image classification model. The image to be classified can be a handwritten image or a clothing image.

[0109] Initialize the model parameters, select the size and number of convolution kernels, input the image to be classified into the convolution layer to extract features through convolution operation, and then input the feature map output by the convolution layer into the activation function, introduce nonlinear transformation, and obtain the feature map after convolution. The model parameters here refer to the parameters of the image classification model. The image classification model is obtained by improving the pooling layer in the standard convolutional neural network model. Its architecture is as follows: Figure 4 shown.

[0110] Apply the Gaussian membership function of three fuzzy partitions to all feature maps obtained after convolution to calculate the membership value of each element. P Set to 3, use the Gaussian membership function, that is, the Gaussian fuzzy membership function, to fuzzify each feature map, and obtain the fuzzy membership value corresponding to each element in each feature map, that is, the membership value, and then obtain three fuzzy membership value maps corresponding to each feature map, that is, the membership value map. Figure 8 As shown, for the k All feature maps obtained by convolution of the convolutional layers are feature maps , ,..., For example, the elements in the i-th row and j-th column are , , …, , respectively use the corresponding Gaussian fuzzy membership function based on the above formula 2 to calculate the fuzzy membership value corresponding to each element, and obtain the feature map , ,..., The three corresponding membership value graphs are the membership value graphs .

[0111] The channel importance module is used to calculate the weight value of each channel, and the channel weight value is multiplied by the corresponding feature map after convolution to obtain the channel feature map. Among them, the channel importance mechanism is used to determine the corresponding channel weight value according to the membership value map corresponding to each feature map after convolution, and determine the corresponding channel feature map based on the channel weight value. Figure 8 As shown, in the feature map , ,..., The corresponding membership value diagrams Afterwards, the feature maps are calculated based on the above formula 3 , ,..., The corresponding maximum membership and value is , ,..., . In the above formula 3, z The values ​​are 1, 2, ..., Z k , we can get 1×1 dimensional , ,..., . Then, replace the above formula 4 with z The values ​​are 1, 2, ..., Z k , calculate the maximum membership and value accumulation corresponding to the kth convolutional layer Then, replace the above formula 5 with z The values ​​are 1, 2, ..., Z k , calculate the feature map , ,..., The corresponding channel weight values , ,..., Finally, the channel weight value , ,..., Multiply by the corresponding feature map , ,..., , you can get the corresponding channel feature map , ,..., .

[0112] Apply the Gaussian membership function of three fuzzy partitions to all channel feature maps to calculate the membership value of each element. Fig. 9 As shown, for the channel feature map , ,..., For example, the elements in the i-th row and j-th column are , ,..., , replace the above formula 2 with Replace with ,Will z The values ​​are 1, 2, ..., Z k , respectively use the corresponding Gaussian membership function, that is, Gaussian fuzzy membership function to calculate the fuzzy membership value corresponding to each element, and obtain the channel feature map , ,..., The three corresponding membership value graphs are the membership value graphs .

[0113] The spatial importance module is used to calculate the weight value of each position element, and the spatial weight value is multiplied by the corresponding channel feature map to obtain the spatial feature map. Among them, the spatial importance mechanism is used to determine the corresponding spatial weight value according to the membership value map corresponding to the channel feature map, and determine the corresponding spatial feature map based on the spatial weight value. Fig. 9 As shown, in the channel feature map , ,..., The corresponding membership value diagrams Then, replace Replace with , and at the same time Replace with ,Will z The values ​​are 1, 2, ..., Z k , the channel feature maps can be calculated separately , ,..., The corresponding maximum membership and value is , ,..., , the maximum membership and value , ,..., The corresponding membership value diagrams That is the channel feature map , ,..., The corresponding spatial maximum membership value diagrams , ,..., After that, the spatial maximum membership value map , ,..., The elements at the same position are added together to obtain the corresponding channel direction elements and value maps. Then, each element in the channel direction elements and value maps is divided by the number of channels. Z k , and obtain the corresponding channel direction mean map. Then, each element in the channel direction mean map is divided by the sum of all elements in the channel direction mean map to obtain a shared weight map, i.e., the element position weight map. Finally, the element position weight map is respectively compared with the channel feature map. , ,..., Multiply to get the channel feature map , ,..., The corresponding spatial feature maps , ,..., .

[0114] The feature map adjusted by the channel and spatial attention mechanism is used as the input of the next layer, and continues to perform convolution, pooling and full connection operations. The final output layer performs the classification task of handwriting and clothing pictures. Figure 6 The steps in the dashed box belong to the pooling operation of the pooling layer in the image classification model. The spatial feature map output by the pooling layer is used as the input of the next convolutional layer connected to it, and the image is further convolved through the convolutional layer. The image after convolution of the last convolutional layer in the image classification model is used as the input of the fully connected layer. The output of the fully connected layer is connected through the softmax Gaussian connection layer, and then through the final output layer to achieve image classification.

[0115] Table 1 below shows the performance comparison of the standard CNN model and the image classification model constructed based on the secondary fuzzy importance mechanism provided by the present disclosure when used for image classification of the MNIST handwriting dataset and the Fashion-MNIST clothing dataset. As can be seen from Table 1, the performance of the image classification model is better than that of the standard CNN model.

[0116] Table 1 Performance comparison between standard CNN model and image classification model

[0117]

[0118] Another embodiment of the present disclosure relates to an image classification device, such as Fig.10 As shown, including:

[0119] An acquisition module 1010 is used to acquire an image to be classified;

[0120] The classification module 1020 is used to input the image to be classified into the image classification model to obtain the corresponding classification result; wherein the image classification model is constructed based on the convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on the quadratic fuzzy importance mechanism.

[0121] The specific implementation method of the image classification device provided in the embodiment of the present disclosure can refer to the image classification method provided in the embodiment of the present disclosure, which will not be repeated here.

[0122] Compared with the prior art, the image classification device provided in the embodiment of the present disclosure improves the pooling layer of the convolutional neural network model based on the secondary fuzzy importance mechanism to obtain an image classification model. The characteristics of the fuzzy logic technology can be effectively utilized to suppress the layer-by-layer transmission of noise in the image in the model, reduce the impact of noise on image recognition, and improve the model's robustness to noise and the accuracy of image recognition. The importance mechanism can also be effectively utilized to optimize the model's extraction of key features of the region of interest in the image, improve the generalization performance of the model and the accuracy of image classification, so that when the image classification model is used to classify the image to be classified, the accuracy of image classification can be effectively improved, and the image classification result can be made more accurate.

[0123] Another embodiment of the present disclosure relates to an electronic device, such as Fig.11 As shown, including:

[0124] at least one processor 1101; and,

[0125] A memory 1102 is communicatively connected to at least one processor 1101; wherein,

[0126] The memory 1102 stores instructions that can be executed by at least one processor 1101 . The instructions are executed by at least one processor 1101 so that the at least one processor 1101 can execute the image classification method described in the above embodiment.

[0127] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0128] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0129] Another embodiment of the present disclosure relates to a computer-readable storage medium storing a computer program, which implements the image classification method described in the above embodiment when executed by a processor.

[0130] That is, those skilled in the art can understand that all or part of the steps in the method described in the above embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0131] Those skilled in the art will appreciate that the above-mentioned embodiments are specific embodiments for implementing the present disclosure, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present disclosure.

Claims

1. An image classification method, characterized in that: The image classification method comprises: Get the image to be classified; Inputting the image to be classified into an image classification model to obtain a corresponding classification result; wherein the image classification model is constructed based on a convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on a quadratic fuzzy importance mechanism; The pooling layer is used to perform a pooling operation on the feature map output by the previous convolutional layer connected thereto based on the secondary fuzzy importance mechanism, generate a corresponding spatial feature map, and output the spatial feature map to the next convolutional layer connected thereto; The pooling operation includes: Based on fuzzy logic technology, the channel importance mechanism is used to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolutional layer connected to it; Based on the channel weight value, determining a channel feature map; Based on fuzzy logic technology, a spatial importance mechanism is used to calculate a shared weight map of all element positions in the channel feature map; Based on the shared weight map, determining the spatial feature map; The fuzzy logic technology is based on the channel importance mechanism to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolution layer connected to it, including: Determine the number of fuzzy partitions P , based on the fuzzy partition number P , generating the center of the Gaussian membership function and kernel width , the center of the Gaussian membership function Expressed as formula 1, where min k,z Indicates k The output of the convolutional layer z The minimum value of all elements in the feature map, max k,z Indicates k The output of the convolutional layer z The maximum value of all elements in the feature map, k represents the convolutional layer number, z Indicates the channel number and 1≤ z ≤ Z k , Z k Indicates k The number of channels of the feature map output by the convolutional layer, p Indicates the fuzzy partition sequence number and 1≤ p ≤ P : Formula 1; Using the Gaussian membership function, based on formula 2 k All elements in all feature maps output by the convolutional layer are blurred to obtain the corresponding P fuzzy membership values, where Indicates k The output of the convolutional layer z The element in the i-th row and j-th column of the feature map is express The corresponding p The fuzzy membership value of a Gaussian membership function is: Formula 2; According to formula 3, calculate the k The output of the convolutional layer z The maximum membership and value corresponding to the feature map ,in, H k Indicates k The height of the feature map output by the convolutional layer, W k Indicates k The width of the feature map output by the convolutional layer: Formula 3: According to Formula 4, calculate the maximum membership and value accumulation corresponding to the kth convolutional layer : Formula 4: According to Formula 5, the channel weight values ​​corresponding to each feature map output by the kth convolutional layer are calculated respectively: Formula 5: in, represents the output of the kth convolutional layer z The channel weight value corresponding to the feature map; The determining of a channel feature map based on the channel weight value comprises: Multiply each feature map output by the kth convolutional layer by its corresponding channel weight value to obtain each channel feature map corresponding to the kth convolutional layer; The method of calculating the shared weight map of all element positions in the channel feature map using a spatial importance mechanism based on fuzzy logic technology includes: Based on the Gaussian membership function, determine the corresponding element of each channel feature map corresponding to the kth convolutional layer P fuzzy membership values; Calculate the maximum membership and value of each channel feature map corresponding to the kth convolutional layer, and obtain the corresponding spatial maximum membership value map; Add the elements at the same position in each spatial maximum membership value map corresponding to the kth convolutional layer to obtain the corresponding channel direction element and value map; Divide the channel direction element and each element in the value map by the number of channels Z k , get the corresponding channel direction mean map; Dividing each element in the channel direction mean map by the sum of all elements in the channel direction mean map, to obtain the shared weight map corresponding to the kth convolutional layer; The determining the spatial feature map based on the shared weight map includes: Each channel feature map corresponding to the k-th convolutional layer is multiplied by the corresponding shared weight map to obtain the spatial feature map corresponding to the k-th convolutional layer.

2. An image classification device, characterized in that: The image classification device comprises: An acquisition module, used for acquiring images to be classified; A classification module, used for inputting the image to be classified into an image classification model to obtain a corresponding classification result; wherein the image classification model is constructed based on a convolutional neural network model, and the pooling layers in the convolutional neural network model are all constructed based on a quadratic fuzzy importance mechanism; The pooling layer is used to perform a pooling operation on the feature map output by the previous convolutional layer connected thereto based on the secondary fuzzy importance mechanism, generate a corresponding spatial feature map, and output the spatial feature map to the next convolutional layer connected thereto; The pooling operation includes: Based on fuzzy logic technology, the channel importance mechanism is used to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolutional layer connected to it; Based on the channel weight value, determining a channel feature map; Based on fuzzy logic technology, a spatial importance mechanism is used to calculate a shared weight map of all element positions in the channel feature map; Based on the shared weight map, determining the spatial feature map; The fuzzy logic technology is based on the channel importance mechanism to calculate the channel weight value corresponding to each channel in the feature map output by the previous convolution layer connected to it, including: Determine the number of fuzzy partitions P , based on the fuzzy partition number P , generating the center of the Gaussian membership function and kernel width , the center of the Gaussian membership function Expressed as formula 1, where min k,z Indicates k The output of the convolutional layer z The minimum value of all elements in the feature map, max k,z Indicates k The output of the convolutional layer z The maximum value of all elements in the feature map, k represents the convolutional layer number, z Indicates the channel number and 1≤ z ≤ Z k , Z k Indicates k The number of channels of the feature map output by the convolutional layer, p Indicates the fuzzy partition sequence number and 1≤ p ≤ P : Formula 1; Using the Gaussian membership function, based on formula 2 k All elements in all feature maps output by the convolutional layer are blurred to obtain the corresponding P fuzzy membership values, where Indicates k The output of the convolutional layer z The element in the i-th row and j-th column of the feature map is express The corresponding p The fuzzy membership value of a Gaussian membership function is: Formula 2; According to formula 3, calculate the k The output of the convolutional layer z The maximum membership and value corresponding to the feature map ,in, H k Indicates k The height of the feature map output by the convolutional layer, W k Indicates k The width of the feature map output by the convolutional layer: Formula 3: According to Formula 4, calculate the maximum membership and value accumulation corresponding to the kth convolutional layer : Formula 4: According to Formula 5, the channel weight values ​​corresponding to each feature map output by the kth convolutional layer are calculated respectively: Formula 5: in, represents the output of the kth convolutional layer z The channel weight value corresponding to the feature map; The determining of a channel feature map based on the channel weight value comprises: Multiply each feature map output by the kth convolutional layer by its corresponding channel weight value to obtain each channel feature map corresponding to the kth convolutional layer; The method of calculating the shared weight map of all element positions in the channel feature map using a spatial importance mechanism based on fuzzy logic technology includes: Based on the Gaussian membership function, determine the corresponding element of each channel feature map corresponding to the kth convolutional layer P fuzzy membership values; Calculate the maximum membership and value of each channel feature map corresponding to the kth convolutional layer, and obtain the corresponding spatial maximum membership value map; Add the elements at the same position in each spatial maximum membership value map corresponding to the kth convolutional layer to obtain the corresponding channel direction element and value map; Divide the channel direction element and each element in the value map by the number of channels Z k , get the corresponding channel direction mean map; Dividing each element in the channel direction mean map by the sum of all elements in the channel direction mean map, to obtain the shared weight map corresponding to the kth convolutional layer; The determining the spatial feature map based on the shared weight map includes: Each channel feature map corresponding to the k-th convolutional layer is multiplied by the corresponding shared weight map to obtain the spatial feature map corresponding to the k-th convolutional layer.

3. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image classification method according to claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image classification method according to claim 1 is implemented.