Image classification method, system, electronic device and storage medium

Abstract

The embodiment of the application provides a kind of image classification method, system, electronic equipment and storage medium, belong to image data processing technical field.The method comprises: obtaining the target image to be classified input into preset image classification network;In image classification network, the feature extraction of target image is carried out, and initial feature map is obtained, and frequency decomposition is carried out to obtain multiple frequency components;After feature compression, each frequency component is spliced, and first recombination feature map is obtained, and feature distribution is analyzed, to determine the target feature that needs to be removed, and the target feature is removed, to obtain first selection weight map;Based on first selection weight map, repeated calibration is carried out, to obtain second selection weight map, and second selection weight map is multiplied with initial feature map, to obtain target feature map after feature calibration;According to target feature map, classification is carried out, to obtain the classification result of target image.The application can improve the accuracy of classifying target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to an image classification method and system, an electronic device and a storage medium. BACKGROUND

[0002] In the process of processing a target image using a deep learning method, in order to extract more detailed features, an attention mechanism is usually introduced to process the image, but noise may be introduced, and when the input image is frequency decomposed, the noise may be amplified in the transformation process in the case of already existing noise, which interferes with the image classification process.

[0003] In order to avoid introducing too much noise, the image information is compressed by global average pooling, but this reduces the clarity and detail level of the image, thereby ignoring important information, resulting in an inaccurate classification result of the target image. For example, in clinical research, due to the low contrast of the medical image research target, the lesion has the characteristics of diversity and fuzzy boundary, and if a general attention mechanism method is used, the detailed features of the image cannot be extracted, thereby resulting in an inaccurate classification result of the lesion information. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose an image classification method and system, an electronic device and a storage medium, which can extract detailed information of a feature image while suppressing noise and improve the accuracy of classifying a target image.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application proposes an image classification method, which comprises: acquiring a target image to be classified, and inputting the target image into a preset image classification network; performing feature extraction on the target image in the image classification network to obtain an initial feature map, and performing frequency decomposition based on the initial feature map to obtain a plurality of frequency components; splicing a first reorganized feature map after performing feature compression on each of the frequency components, analyzing the feature distribution in the first reorganized feature map to determine a target feature to be removed, and removing the target feature to obtain a first selection weight map corresponding to the plurality of frequency components; repeatedly calibrating based on the first selection weight map to obtain a second selection weight map, and multiplying the second selection weight map by the initial feature map to obtain a target feature map after feature calibration; and classifying according to the target feature map to obtain a classification result of the target image.

[0006] According to some embodiments of the present application, the feature compression of each of the frequency components comprises: obtaining the height and width of each of the frequency components; calculating the frequency coefficient accumulation value of each of the frequency components within the height and the width; and obtaining the channel scalar corresponding to each of the frequency components according to the frequency coefficient accumulation value divided by the product of the height and the width.

[0007] According to some embodiments of the present application, the splicing of each of the frequency components after the feature compression to obtain the first reorganized feature map comprises: obtaining the channel scalar corresponding to each of the frequency components after the compression of each of the frequency components; obtaining the channel dimension of each of the channel scalars; and sequentially splicing the channel scalars of each of the frequency components according to the channel dimension to obtain the first reorganized feature map of the initial feature map.

[0008] According to some embodiments of the present application, the analysis according to the feature distribution in the first reorganized feature map to determine the target feature to be removed and the removal of the target feature to obtain the first selection weight map corresponding to the plurality of frequency components comprises: extracting a plurality of coefficient features from the first reorganized feature map; analyzing the feature distribution of each of the coefficient features to obtain a ranking score of each of the coefficient features; determining a plurality of target features from the first reorganized feature map according to the ranking score; and removing each of the target features of the first reorganized feature map to obtain the first selection weight map corresponding to the plurality of frequency components.

[0009] According to some embodiments of the present application, the analysis of the feature distribution of each of the coefficient features to obtain a ranking score of each of the coefficient features comprises: analyzing the feature distribution of each of the coefficient features to obtain a feature analysis result; and scoring each of the coefficient features according to the feature analysis result to obtain a ranking score of each of the coefficient features.

[0010] According to some embodiments of the present application, the removal of each of the target features of the first reorganized feature map to obtain the first selection weight map corresponding to the plurality of frequency components comprises: multiplying each feature of the first reorganized feature map by the corresponding selection weight after the removal of each of the target features of the first reorganized feature map, and then adding to obtain a fusion selection weight map corresponding to each of the frequency components; inputting the fusion selection weight map into a first fully connected layer to obtain first output data; and inputting the first output data into a second fully connected layer and then performing activation to generate a first selection weight map with a channel dependency relationship.

[0011] According to some embodiments of the present application, the repeating calibration based on the first selection weight map to obtain a second selection weight map comprises: inputting the first selection weight map into a feedforward network for weight conversion to output a recalibration weight map; normalizing each weight value in the recalibration weight map; and multiplying each weight value of the first selection weight map and the recalibration weight map in a corresponding channel to obtain the second selection weight map.

[0012] To achieve the above object, a second aspect of the embodiments of the present application provides an image classification system, which comprises: a target image acquisition module configured to acquire a target image to be classified and input the target image into a preset image classification network; a frequency component acquisition module configured to perform feature extraction on the target image in the image classification network to obtain an initial feature map, and perform frequency decomposition based on the initial feature map to obtain a plurality of frequency components; a first selection weight map acquisition module configured to splice a first reorganized feature map obtained after feature compression of each of the frequency components, analyze feature distribution in the first reorganized feature map to determine target features to be removed, and remove the target features to obtain a first selection weight map corresponding to the plurality of frequency components; a target feature map acquisition module configured to perform repeating calibration based on the first selection weight map to obtain a second selection weight map, and multiply the second selection weight map with the initial feature map to obtain a target feature map after feature calibration; and a classification result acquisition module configured to perform classification based on the target feature map to obtain a classification result of the target image.

[0013] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the image classification method of any one of the first aspect embodiments when executing the computer program.

[0014] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the image classification method of any one of the first aspect embodiments when executed by a processor.

[0015] The image classification method, system, electronic device, and storage medium proposed in this application can extract features from a target image through an image classification network to obtain an initial feature map, and then perform frequency decomposition on the initial feature map to obtain multiple frequency components. By compressing the features of each frequency component, the detailed and structural information of the initial feature map can be compressed and then stitched together, preserving the detailed information of the feature map. Then, based on the analysis of the features of the stitched first reconstructed feature map, inconsistent target features are screened out and removed from the first reconstructed feature map to suppress noise. Based on the noise-suppressed first selection weight map, repeated calibration is performed to obtain a second selection weight map, making meaningful image features more prominent. The second selection weight map is multiplied with the initial feature map to obtain a feature-calibrated target feature map, thus calibrating the image features of the initial feature map. Finally, classification is performed based on the target feature map. Because feature calibration has been performed, the classification result of the target image is more accurate. This application can extract detailed information from the feature image while suppressing noise, improving the accuracy of target image classification. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the image classification system provided in the embodiments of this application;

[0017] Figure 2 This is a flowchart of the image classification method provided in the embodiments of this application;

[0018] Figure 3 This is a flowchart of the attention mechanism method module provided in the embodiments of this application;

[0019] Figure 4 This is a schematic diagram illustrating feature recalibration according to an embodiment of this application;

[0020] Figure 5 This is a flowchart of feature compression of frequency components provided in an embodiment of this application;

[0021] Figure 6 This is a flowchart of the splicing of frequency components after compression, provided in an embodiment of this application;

[0022] Figure 7 This is a flowchart of obtaining the first selection weight map provided in an embodiment of this application;

[0023] Figure 8 yes Figure 7 The flowchart of step S402 in the document;

[0024] Figure 9 This is a flowchart of step S404 provided in the embodiments of this application;

[0025] Figure 10 is a step flow chart of outputting a re-calibration weight map;

[0026] Figure 11 is a functional module schematic diagram of an image classification system provided by an embodiment of the present application;

[0027] Figure 12 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0029] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0031] In the related art, the classification of target images, such as the classification of medical images, can be achieved by selecting useful sub-bands through Discrete Wavelet Transform (DWT), in which the low-frequency component of the sub-band contains the main structural information, and the high-frequency component contains the detailed information of the image. In order to extract more detailed information, an attention mechanism is usually introduced to process the image, but this will cause a data noise problem. However, with the introduction of high-frequency components, the noise in the data also increases, which destroys the basic structure of the classification target in the target image, which makes it difficult for the convolutional neural network to represent the image features through useful features in the frequency domain space, such as representing clinical features. Therefore, it is necessary to solve the data noise problem to improve the accuracy and reliability of medical image classification.

[0032] In order to avoid introducing too much noise, the image information can be compressed by global average pooling, but this will reduce the clarity and detail level of the image, thus ignoring important information, resulting in inaccurate classification results of the target image. Moreover, in the related art, it is usually assumed that all structural and detailed features have the same contribution to the latent semantics, which is unreasonable. In the frequency domain features, the information of different segments may have different effects on the ability of the image classification network to extract discriminative features. Therefore, in addition to exploring the attributes in the frequency domain, it is also necessary to implicitly learn to suppress irrelevant regions in the input image while highlighting significant features useful for a specific task, so as to improve the accuracy and effect of image classification and make the image classification network more interpretable.

[0033] Based on this, the embodiments of the present application provide an image classification method, system, electronic device and storage medium, which suppresses noise while extracting detailed information of the feature image, thereby improving the accuracy of classifying the target image.

[0034] The image classification method, system, electronic device and storage medium provided by the embodiments of the present application are specifically explained by the following embodiments. First, the image classification system in the embodiments of the present application is described.

[0035] Please refer to Figure 1 In some embodiments, the image classification system includes a server end 101, a controller 102 and a client 103.

[0036] Specifically, the server end 101 refers to a device or application program that receives requests and processes them. The server end 101 can receive image data sent by the client 103, extract and classify features of the image, and use the model to classify and return the result to the client 103. The server end 101 can extract the features of the image according to the pre-trained image classification network, calibrate the features, and compare the calibrated image features with the preset classification labels to finally obtain the classification result of the image. Exemplarily, the server end 101 can be a cloud server end, a data center or the like.

[0037] In some embodiments, the controller 102 receives an image classification request from the client 103 and sends the image classification request to the server end 101 for processing. The controller 102 is also responsible for managing and maintaining the running state of the server end 101 to ensure the normal operation of the system.

[0038] In some embodiments, the client 103 can refer to a device or an application program that initiates a request, connects to the server side 101 through a network, and sends image data to the server side 101 for classification. The client 103 is responsible for interacting with the user, receiving the user's image upload request or image query request, and sending the request to the controller 102. The client 103 is also responsible for displaying the classification results or query results to the user, providing a friendly user interface, and facilitating user operation of the image classification system. Exemplarily, the client 103 can be a personal computer, a mobile phone, a smart camera, or the like.

[0039] The image classification method in the embodiments of the present application can be described as follows.

[0040] It should be noted that in each specific embodiment of the present application, when it is necessary to process relevant data related to the identity or characteristics of the user based on user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the user's separate permission or separate consent will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the user's separate permission or separate consent, the necessary user-related data for enabling the embodiments of the present application to function normally will be obtained.

[0041] Figure 2 is an optional flowchart of the image classification method provided by the embodiments of the present application, Figure 2 The method in can include steps S101 to S105.

[0042] Step S101, obtaining a target image to be classified, and inputting the target image into a preset image classification network.

[0043] In some embodiments, the target image to be classified can be a medical image to be subjected to pathological analysis or an image to be subjected to lesion segmentation, or can be other images to be classified, such as human action images, etc. The image classification network can be a pre-trained convolutional neural network, a residual network, or the like. The image classification network is pre-trained and parameterized according to a large amount of sample data, and has good image classification capability.

[0044] Step S102, performing feature extraction on the target image in the image classification network to obtain an initial feature map, and performing frequency decomposition based on the initial feature map to obtain a plurality of frequency components.

[0045] Exemplarily, in the image classification network, feature extraction can be performed on the target image through a series of convolutional layers, pooling layers and activation functions. Specifically, the input image can be convolved by the convolutional layer to extract local features in the image. Then, the output of the convolutional layer is nonlinearly transformed by using an activation function (such as ReLU) to introduce a nonlinear factor and enhance the expression ability of the network. Further, the feature map is down-sampled by the pooling layer to reduce the dimension of the feature map and retain the key information. Finally, after stacking multiple convolutional layers and pooling layers, the obtained feature map is used as the initial feature map.

[0046] In some embodiments, the initial feature map can be frequency-decomposed using a discrete wavelet transform (DWT) to obtain a low-frequency component and three high-frequency components. The low-frequency component represents the overall trend of the image, while the high-frequency component represents the detailed information of the image. Using the discrete wavelet transform, the initial feature map can be frequency-decomposed to obtain four output components, one low-frequency component and three high-frequency components.

[0047] Specifically, the low-frequency component (denoted by LL) represents the overall information of the image, that is, the general outline and overall features of the image. The high-frequency component has a higher frequency and contains detailed and texture information of the image. The high-frequency component specifically includes: horizontal direction detailed information (denoted by LH) of the image, vertical direction detailed information (denoted by HL) of the image, and diagonal direction detailed information (denoted by HH) of the image.

[0048] It can be understood that, by frequency decomposition, different feature information of the image can be obtained from different frequency components.

[0049] In step S103, after feature compression of each frequency component, the first recombined feature map is obtained, the target features to be removed are determined according to the feature distribution in the first recombined feature map, and the target features are removed to obtain the first selection weight map corresponding to the plurality of frequency components.

[0050] It can be understood that feature compression refers to a dimension reduction or compression operation on the original features to reduce the dimension or complexity of the features while retaining important information of the original features. Exemplarily, in the feature compression operation, the first recombined feature map can be obtained by calculating the average value of each frequency component after compression, or a statistical feature value representing the features can be obtained by calculating the variance between the feature values after compression and the average value to form the first recombined feature map, or the first recombined feature map can be formed by the maximum value of the compressed features, and the like. The embodiments of the present application do not make specific limitations thereto.

[0051] In some embodiments, after frequency decomposition, a plurality of frequency components, i.e., high frequency components and low frequency components, can be obtained, and feature compression is performed on each frequency component, i.e., feature compression is performed on both high frequency components and low frequency components, so that the overall information of the initial feature image can be retained, and the details and texture information of the initial feature image can also be retained, so as to facilitate more accurate classification of the target image in the subsequent process.

[0052] It can be understood that, since feature compression is also performed on high frequency components, and due to the combined effects of signal attenuation, signal distortion and other factors, high frequency components often carry certain noise interference, and therefore, the noise needs to be removed to improve the accuracy of the classification result of the target image.

[0053] Specifically, since the noise distribution is irregular, the feature distribution of the first reorganized feature map can be analyzed, and the target feature that needs to be removed, i.e., the noise, can be determined therefrom. For example, the classifier can be used to analyze the distribution shape of the features in the first reorganized feature map, and if the distribution shape of a certain feature is obviously different from those of other features, it can be indicated that the feature has noise. Alternatively, each feature can be compared with other features to observe its distribution on the first reorganized feature map, and if the distribution of a certain feature is significantly different from those of other features, it can be indicated that the feature has noise. Alternatively, the similarity or distance between features can be calculated, and the target feature can be compared with other features, and if the similarity between the target feature and other features is low or the distance is large, it can be indicated that the feature has noise. Alternatively, a feature distribution graph can be drawn to observe the distribution of feature values, and if the distribution of a certain feature is obviously deviated from the normal condition, i.e., the distribution is inconsistent, it can be a noise feature.

[0054] In step S104, repeated calibration is performed based on the first selection weight map to obtain a second selection weight map, and the second selection weight map is multiplied with the initial feature map to obtain a target feature map after feature calibration.

[0055] In some embodiments, the weights of each feature value can be redistributed based on the first selection weight map, so that important features are more prominent, while unimportant features are suppressed, and the recognition ability of the image classification network for important features is improved, and then the number of channels is expanded one by one until the size of the obtained second selection weight map is consistent with the size of the initial feature map, so that each weight of the second selection feature map can accurately correspond to the features of the initial feature map, and the importance of each feature in the initial feature map can be intuitively expressed. Specifically, when recalibrating the features, the range of calibration of the corresponding channel can be gradually expanded, and the feature values of the corresponding channel can be multiplied, until the size of the obtained second selection weight map is consistent with the size of the initial feature map, so as to improve the accuracy of image classification.

[0056] It can be understood that the corresponding multiplication operation of the weight value can be sequentially performed on each channel, and the specific formula is as follows:

[0057]

[0058] wherein, denotes the first selection weight map, h(·) denotes the multiplication operation of the repeated channel of the first selection weight map and the recalibration weight map to the feature space; that is denotes the multiplication of the new range of channels after the expansion of each channel, until the second selection feature map obtained by the expansion reaches the channel number (C), the height (H) and the width (W) consistent with the initial feature map (P); denotes element-by-element multiplication, denotes the target feature map.

[0059] Please refer to Figure 4 , in Figure 4 , Selection denotes selection, Recalibration denotes recalibration, GAP denotes feature compression, DWT denotes frequency decomposition, denotes the first selection weight corresponding to the four frequency components, P1 denotes all coefficient features of the first reorganized feature map, p a denotes the coefficient feature corresponding to the maximum sorting score, P2 denotes the features left in the first reorganized feature map after p a is removed, denotes the first selection weight map, denotes the target feature map, Fa denotes the feature map obtained by convolution operation, Fr denotes the conversion of the two-dimensional feature map to the one-dimensional feature vector by the flattening operation on the feature activation, and Fs denotes the space composed of the feature maps generated by the convolution layer and the pooling layer and the like. In some embodiments, the initial feature map can be subjected to frequency decomposition to obtain a plurality of frequency components, and then each frequency component can be subjected to feature compression and splicing to obtain a first reorganized feature map. Then, according to the feature distribution, feature selection is performed to remove noise, so as to obtain the first selection weight corresponding to the plurality of frequency components. Further, the first selection weight map can be subjected to feature recalibration to suppress unimportant features and highlight important features, so as to obtain a second selection weight map. Finally, the second selection weight map is multiplied by the initial feature map to obtain a target feature map, and the target feature map is classified to obtain a classification result.

[0060] It can be understood that the repeated calibration can copy each weight value to the position corresponding to the original feature map, so that the second weight map can be kept consistent with the size of the initial feature map. Then, multiplying the second selected weight map with the initial feature map can multiply the weight of each pixel in the target image with the feature value at the corresponding position to obtain the weighted feature map.

[0061] It can be understood that the repeated calibration of the feature map can realize the weighting operation of the feature map, so as to highlight or suppress unimportant features. Specifically, the weighting operation can be realized by adjusting the weight, so as to change the importance of pixels at different positions in the initial feature map, extract features beneficial to the classification task, and weaken or remove features not beneficial to the classification task.

[0062] In step S105, classification is performed according to the target feature map to obtain the classification result of the target image.

[0063] In some embodiments, after obtaining the target feature map after recalibration, the target feature map is taken as input to pass through a fully connected layer or other appropriate classifier for classification. Specifically, a normalized exponential function (softmax function) can be used for multi-class classification, or other classifiers such as Support Vector Machines (SVM) can be used for classification.

[0064] Please refer to Figure 3 In some embodiments, steps S101 to S105 can be packaged into an attention mechanism method module, and the obtained attention mechanism method module is denoted as a Selection Andrecalibration Module (SRM). The attention mechanism method module is inserted into a convolution module, such as a classical residual module, to form a Residual Block Branch-Selection Andrecalibration Module (RBB-SRM), and finally form an image classification network. In this way, accurate classification of the target image can be realized. Figure 3 Part (a) in FIG. 1 is a convolutional neural network with RBB-SRM, Figure 3 Part (b) in FIG. 1 is a residual module without RBB-SRM, Figure 3(c) in the figure is a residual module inserted in the RBB-SRM, input represents input, output represents output, Conv is the abbreviation of convolution operation, BN represents Batch Normalization, ReLU is the abbreviation of Rectified Linear Unit, avepool is the abbreviation of average pooling, and FC represents the abbreviation of Fully Connected Layer.

[0065] The image classification method, system, electronic device and storage medium provided in the application can extract features of a target image through an image classification network, obtain an initial feature map, and perform frequency decomposition on the initial feature map to obtain a plurality of frequency components. By performing feature compression on each frequency component, the details and structural information of the initial feature map can be compressed and spliced, and the details of the feature map are retained. Then, the features of the first reorganized feature map obtained by splicing are analyzed to filter out target features with inconsistent distribution, and the target features are removed from the first reorganized feature map to suppress noise. The first selection weight map subjected to noise suppression is repeatedly calibrated to obtain a second selection weight map, so that meaningful image features are more prominent, and the second selection weight map is multiplied by the initial feature map to obtain a target feature map after feature calibration, so as to calibrate the image features of the initial feature map. Finally, the target feature map is classified. Since the features have been calibrated, the classification result of the target image is more accurate. The application can extract the details of the feature image while suppressing noise, and improve the accuracy of classifying the target image.

[0066] Please refer to Figure 5 In some embodiments, the frequency components include low-frequency components and high-frequency components; the feature compression on each frequency component can include steps S201 to S203:

[0067] In step S201, the height and width of each frequency component are obtained.

[0068] It can be understood that the height and width of each frequency component obtained by performing frequency decomposition on the initial feature map using the discrete wavelet transform are each half of the original initial feature map. If the frequency components after the first frequency decomposition are further subjected to frequency decomposition, the frequency components obtained by the second frequency decomposition of each frequency component are each half of the height and width of the frequency components obtained by the first frequency decomposition. The application considers the case of performing frequency decomposition on the initial feature map once, and in real application scenarios, the number of frequency decompositions can also be adjusted according to actual conditions, which is not specifically limited in the embodiments of the application.

[0069] Step S202, calculate the frequency coefficient accumulation value of each low frequency component and high frequency component in height and width.

[0070] In some embodiments, the frequency coefficient refers to the rate of change of the frequency component in time or space. By accumulating the frequency coefficients of the low frequency component and the high frequency component in height and width, the sum of the frequency coefficients corresponding to the low frequency component or the high frequency component can be obtained.

[0071] Step S203, obtain the channel scalar corresponding to each low frequency component and high frequency component according to the frequency coefficient accumulation value divided by the product of height and width.

[0072] It can be understood that the channel scalar refers to the characteristic value or feature value of the frequency component, which is used to represent the characteristics of the frequency component. In some embodiments, the channel scalar can be calculated by taking the average value of each frequency component. For example, the formula for calculating the channel scalar of the low frequency component is as follows:

[0073]

[0074] In the formula, P LL represents the channel scalar of the low frequency component, H represents the height of the initial feature image, W represents the width of the initial feature image, and LL represents the low frequency component.

[0075] It can be understood that the calculation formula of the channel scalar of the high frequency component is the same as that of the low frequency component, and the calculation formula of the channel scalar of the low frequency component can be referred to as a reference. The embodiments of the present application do not make specific limitations on this. It can be understood that the channel scalar is obtained by taking the average value of the frequency coefficient of each frequency component, which can reduce the high and low fluctuations of the frequency component, thereby reducing the interference of high frequency noise to a certain extent and improving the accuracy of classifying the target image.

[0076] Please refer to Figure 6 In some embodiments, after feature compression of each frequency component, the first recombined feature map can be obtained by splicing, which can include steps S301 to S303:

[0077] Step S301, after compression of each frequency component, the channel scalar corresponding to each frequency component is obtained.

[0078] In some embodiments, the channel scalar can be calculated by calculating the frequency coefficient average of each frequency component, so as to compress each frequency component, reduce the dimension and redundancy of data. In addition, since each frequency component is compressed instead of only low frequency components or only high frequency components, more structural information or detailed information can be retained, and the accuracy of classifying the target image can be improved. It can be understood that the channel scalar refers to a characteristic value or a feature value of the frequency component, which can be used to represent the characteristics of the frequency component.

[0079] In step S302, the channel dimension of each channel scalar is obtained.

[0080] In some embodiments, the channel dimension refers to the dimension or position corresponding to each channel scalar in the initial feature map. The channel dimension can be used for subsequent feature reorganization to ensure that the channel scalar of each frequency component can be spliced in the correct order and dimension.

[0081] In step S303, the channel scalars of each frequency component are spliced in sequence according to the channel dimension to obtain a first reorganized feature map of the initial feature map.

[0082] In some embodiments, after compressing each frequency component, the channel scalars are spliced in sequence according to the channel dimension, which can orderly combine the feature information of different frequencies, restore the feature distribution of the initial feature map, and retain important information of the compressed data.

[0083] Please refer to Figure 7 In some embodiments, according to the analysis of the feature distribution in the first reorganized feature map, the target feature to be removed is determined, and the target feature is removed to obtain a first selection weight map corresponding to the plurality of frequency components, which can include steps S401 to S404:

[0084] In step S401, a plurality of coefficient features are extracted from the first reorganized feature map.

[0085] In some embodiments, the coefficient feature refers to a feature with certain importance or contribution degree obtained after processing the first reorganized feature map. The coefficient feature can be a group of numbers, vectors or other forms of data. Specifically, the extraction method of the coefficient feature can be determined according to the specific application field and algorithm. For example, mathematical tools such as wavelet transform or Fourier transform can be used to convert the first reorganized feature map into other frequency domain or other feature space, and then analyze the feature distribution of each coefficient feature.

[0086] In step S402, the feature distribution of each coefficient feature is analyzed to obtain a ranking score of each coefficient feature.

[0087] In some embodiments, the distribution of each coefficient feature can be observed using a visualization method such as a box plot or a probability density plot. Alternatively, the differences between different coefficient features can be compared using a hypothesis test, for example, a variance analysis can be used to test the differences between different groups of features, and the coefficient features with larger differences from other groups have higher ranking scores. Alternatively, the correlation between each coefficient feature of the first reorganized feature map can be calculated, and if a certain feature has a low correlation with other features, it can be a feature with inconsistent distribution.

[0088] In some embodiments, the feature distribution of each coefficient feature can be analyzed by scoring the analysis results of the feature distribution of each coefficient feature, for example, the higher the degree of dispersion, the higher the ranking score of the coefficient feature, and so on, so that the feature distribution of each coefficient feature can be analyzed intuitively.

[0089] Step S403, determining a plurality of target features from the first reorganized feature map according to the ranking scores.

[0090] It can be understood that since the coefficient feature with the most inconsistent distribution has the highest score, the coefficient feature with the highest ranking score can be selected as the target feature, or the coefficient feature with a ranking score higher than a certain threshold can be determined as the target feature. It can be understood that a classifier can also be used to analyze the feature distribution of each coefficient feature directly and select the coefficient feature with inconsistent distribution as the target feature.

[0091] Step S404, removing each target feature of the first reorganized feature map to obtain a first selection weight map corresponding to a plurality of frequency components.

[0092] In some embodiments, since the target feature is noise, each target feature of the first reorganized feature map needs to be removed to reduce the interference of noise on the classification process and improve the accuracy of the classification result. For example, the target feature can be removed by filtering, noise reduction, etc.

[0093] For example, a linear layer and an activation layer can be used to construct a classifier to obtain a ranking score, which is used to represent all coefficient features of the first reorganized feature map, which can be denoted as P1, and the ranking score is denoted as A. The coefficient feature p a Since the larger the ranking score is, the more inconsistent the distribution of the corresponding coefficient feature is, and the greater the possibility that the coefficient feature is noise is, the coefficient feature p a is removed from P1, thereby suppressing noise. Specifically, p a After removal, the remaining features of the first reorganized feature map are P2 = ReLU(P1-p a). It can be understood that, by analyzing the distribution of each feature, the noise data can be accurately identified and removed, so that the process of repeatedly calibrating the features after ensuring the introduction of high-frequency components and feature compression will not be disturbed by noise, thereby improving the efficiency of feature calibration and the accuracy of image classification.

[0094] Please refer to Figure 8 In some embodiments, step S402 can include steps S501-S502:

[0095] Step S501: analyzing the feature distribution of each coefficient feature to obtain a feature analysis result.

[0096] In some embodiments, the feature distribution can be analyzed by methods such as visualizing the feature distribution, statistical analysis, hypothesis testing, and correlation analysis, so as to obtain the ranking score according to the distribution of each coefficient feature.

[0097] Step S502: scoring each coefficient feature according to the feature analysis result to obtain a ranking score of each coefficient feature.

[0098] In some embodiments, the distribution consistency of the coefficient feature can be scored by variance or standard deviation, and the greater the calculated variance or standard deviation, the more inconsistent the distribution of the coefficient feature, and therefore the higher the corresponding ranking score. It can be understood that the ranking score of each coefficient feature can also be obtained by other scoring methods, such as associating a classifier with a scoring function, the most inconsistent coefficient feature classified by the classifier having the highest score, etc., and the present embodiments are not limited in this regard.

[0099] Please refer to Figure 9 In some embodiments, step S404 can include steps S601-S603:

[0100] Step S601: after removing each target feature of the first reorganized feature map, multiplying each feature of the first reorganized feature map by the corresponding selection weight and adding them to obtain a fusion selection weight map corresponding to each frequency component.

[0101] In some embodiments, a filter can be set or a trained model can be used to remove each target feature of the first reorganized feature map, thereby reducing the influence of irrelevant information on the final classification result and improving the accuracy and effect of feature fusion.

[0102] It can be understood that each feature of the first recombination feature map is pre-assigned a selection weight to quantify the importance of different features, and the feature fusion is performed accordingly, so that the fusion result is more accurate and more in line with the actual situation. It can be understood that the selection weight can be adjusted according to the different contribution degrees of each feature.

[0103] In step S602, the first output data is obtained by inputting the fusion selection weight map into the first full connection layer.

[0104] In some embodiments, the fusion selection weight map can be used to adjust the contribution of each feature to the first full connection layer, so that the network can pay more attention to important features and improve the performance of the model. It can be understood that the first full connection layer can calculate and encode the relationship between each selection weight and other selection weights.

[0105] In step S603, the first selection weight map with channel dependency is generated by inputting the first output data into the second full connection layer and then activating.

[0106] It can be understood that the first output data can be input into the second full connection layer again for linear transformation, so as to nonlinearly encode each selection weight, so that the model can better capture the dependency between each selection weight.

[0107] In some embodiments, after the first output data of the first full connection layer is activated, the first output data can be input into the second full connection layer, and the second full connection layer outputs the second output data which is then activated; or the first output data is directly input into the second full connection layer and then activated, and the embodiments of the present application do not make specific limitations in this regard.

[0108] It can be understood that the activation function can nonlinearly encode the selection weight, so as to better capture the complex relationship between the selection weights. Exemplarily, the activation function can include ReLU, sigmoid, tanh, etc., and the embodiments of the present application do not make specific limitations in this regard. It can be understood that the channel dependency can help the neural network better understand the relationship between different channels (features), and further extract higher-level abstract features. For example, when processing image data, the correlation between different features of the image can be captured by establishing the channel dependency, so as to improve the accuracy of classification.

[0109] Exemplarily, the first selection weight map can be denoted as The specific calculation formula is as follows:

[0110]

[0111] wherein, RELU is an activation function, specifically, a rectified linear unit (RELU), denotes a weight matrix corresponding to the first fully connected layer, denotes a weight matrix corresponding to the second fully connected layer, and σ denotes an activation parameter.

[0112] Please refer to Figure 10 In some embodiments, obtaining the second selection weight map based on the first selection weight map through repeated calibration can include steps S701-S703:

[0113] Step S701: inputting the first selection weight map into a feedforward network for weight conversion, and outputting a recalibration weight map.

[0114] It can be understood that the feedforward network is composed of a series of neurons, and the weight conversion can be achieved by adjusting the connection weights between the neurons, thereby improving the recognition and learning ability of the model for specific features, and thus improving the performance of the model.

[0115] Illustratively, assuming in an image recognition task, the first selection weight map represents the degree of attention to different parts of the image, for example, for a face recognition task, the first selection weight map can assign higher weights to areas such as eyes, nose, and mouth, by inputting the first selection weight map into the feedforward network for weight conversion, we can adjust the network weights according to the different features of the image, so that the network pays more attention to the eyes, nose, and mouth of the face; illustratively, for example, in pathological recognition, a higher weight is assigned to the diseased area, then after inputting the first selection weight map into the image classification network, the image classification network can pay more attention to the diseased area, thereby improving the efficiency and accuracy of image recognition.

[0116] Step S702: normalizing each weight value in the recalibration weight map.

[0117] It can be understood that in order to facilitate subsequent processing and calculation, each weight value of the recalibration weight map can be normalized to ensure that the weight value of each feature is within a suitable range. It can be understood that normalization is achieved by mapping the weight value to a certain specific range, for example, between [0, 1] or [-1, 1], the normalized weight value is easier to compare and process, avoiding the problem caused by too large or too small weight value.

[0118] Step S703: multiplying each weight value of the first selection weight map and the recalibration weight map in the corresponding channel to obtain the second selection weight map.

[0119] In some embodiments, each weight value of the first selection weight map and the re-calibration weight map can be multiplied by the corresponding channel to obtain a second selection weight map, so as to further adjust the weight to more accurately express the attention of the image classification network to different features and improve the performance of the image classification network.

[0120] It can be understood that, in order to enable the second selection weight map to correspond to the initial feature map, so that each weight of the second selection feature map can accurately correspond to the feature of the initial feature map and intuitively express the importance of each feature in the initial feature map, the size of the second selection weight map should be consistent with the size of the initial feature map. Specifically, the accuracy of image classification can be improved by gradually expanding the range of channels of the first selection weight map and the re-calibration weight map and multiplying the feature values of the corresponding channels, until the size of the obtained second selection weight map is consistent with the size of the initial feature map.

[0121] It can be understood that the corresponding multiplication operation of weight values can be performed on each channel in sequence, and the specific formula is as follows:

[0122]

[0123] wherein, represents the first selection weight map, and h(·) represents a repeated channel multiplication operation of aligning the first selection weight map and the re-calibration weight map in the feature space; that is, represents that the channel multiplication in the new range is performed after the expansion of the channel each time, until the second selection feature map obtained by expansion reaches the number of channels (C), the height (H) and the width (W) consistent with the initial feature map (P); represents element-by-element multiplication, represents the target feature map.

[0124] In some embodiments, the probability distribution of the target feature map for each class label can be calculated by mapping the target feature map to the class label, and specifically, the value of the feature map can be mapped to the probability distribution by using a normalized exponential function (softmax function). The class label can be stored in a database.

[0125] It can be understood that, based on the probability distribution, different strategies can be used to determine the classification result of the target image. For example, the class with the highest probability can be selected as the final classification result. In addition, a threshold can also be set, and only when the probability of a certain class exceeds the threshold, the class is taken as the classification result, so as to improve the accuracy and reliability of the classification.

[0126] In some embodiments, one target image can include multiple image features, and thus, multiple classification results can be obtained according to the image features and multiple category labels. Specifically, whether a target feature map includes a corresponding category can be determined by the probability of each category label in the target feature map. For example, in identifying lesions, multiple types of lesions can be identified, such as different types of tumors or cerebrovascular lesions in brain magnetic resonance images.

[0127] Referring to Figure 11 The embodiments of the present application also provide an image classification system, which can implement the above image classification method. The image classification system comprises:

[0128] The target image acquisition module 1101 is configured to acquire a target image to be classified and input the target image into a preset image classification network.

[0129] The frequency component acquisition module 1102 is configured to perform feature extraction on the target image in the image classification network to obtain an initial feature map, and perform frequency decomposition based on the initial feature map to obtain multiple frequency components.

[0130] The first selection weight map acquisition module 1103 is configured to splice the feature compression of each frequency component to obtain a first recombined feature map, analyze the feature distribution in the first recombined feature map to determine a target feature to be removed, and remove the target feature to obtain a first selection weight map corresponding to the multiple frequency components.

[0131] The target feature map acquisition module 1104 is configured to perform repeated calibration based on the first selection weight map to obtain a second selection weight map, and multiply the second selection weight map with the initial feature map to obtain a target feature map after feature calibration.

[0132] The classification result acquisition module 1105 is configured to perform classification according to the target feature map to obtain a classification result of the target image.

[0133] In some embodiments, the target image acquisition module 1101 is configured to acquire a target image to be classified and input the target image into a preset image classification network.

[0134] In some embodiments, the target image to be classified can be a medical image to be subjected to pathological analysis or an image to be subjected to lesion segmentation, or can be other images to be classified, such as human action images, etc. The image classification network can be a pre-trained convolutional neural network, a residual network, etc. The image classification network is pre-trained and parameterized according to a large amount of sample data, and has good image classification capability.

[0135] In some embodiments, the frequency component obtaining module 1102 is configured to perform feature extraction on the target image in the image classification network to obtain an initial feature map, and perform frequency decomposition based on the initial feature map to obtain a plurality of frequency components.

[0136] For example, in the image classification network, the target image can be subjected to feature extraction through a series of convolution layers, pooling layers and activation functions. Specifically, the input image can be subjected to convolution operation through the convolution layer to extract local features in the image. Then, the output of the convolution layer is subjected to nonlinear transformation through the use of the activation function (such as ReLU) to introduce nonlinear factors and enhance the expression ability of the network. Further, the feature map is subjected to down-sampling through the pooling layer to reduce the dimension of the feature map and retain key information. Finally, through the stacking of a plurality of convolution layers and pooling layers, the obtained feature map is taken as the initial feature map.

[0137] In some embodiments, the initial feature map can be subjected to frequency decomposition using the discrete wavelet transform (DWT) to obtain one low-frequency component and three high-frequency components. The low-frequency component represents the overall trend of the image, while the high-frequency component represents the detailed information of the image. Using the discrete wavelet transform, the initial feature map can be subjected to frequency decomposition to obtain four output components, namely one low-frequency component and three high-frequency components.

[0138] Specifically, the low-frequency component (denoted as LL) represents the overall information of the image, i.e. the general outline and overall features of the image. The high-frequency component has a higher frequency and contains the detailed and texture information of the image. The high-frequency component specifically includes the horizontal direction detailed information of the image (denoted as LH), the vertical direction detailed information of the image (denoted as HL), and the diagonal direction detailed information of the image (denoted as HH).

[0139] It can be understood that through the frequency decomposition, different feature information of the image can be obtained from different frequency components.

[0140] In some embodiments, the first selection weight map obtaining module 1103 is configured to splice the feature compression of each frequency component to obtain a first recombined feature map, analyze the feature distribution in the first recombined feature map to determine the target feature to be removed, and remove the target feature to obtain a first selection weight map corresponding to the plurality of frequency components.

[0141] It can be understood that feature compression refers to the operation of reducing the dimension or compressing the original feature to reduce the dimension or complexity of the feature while retaining the important information of the original feature. For example, in the feature compression operation, the first reorganized feature map can be obtained by calculating the average value of each frequency component after compression, or a statistical feature value representing the feature can be obtained by the variance between the feature value after compression and the average value to form the first reorganized feature map, or the first reorganized feature map can be formed by the maximum value of the compressed feature, and the like. The embodiments of the present application do not make specific limitations thereto.

[0142] In some embodiments, a plurality of frequency components, i.e., high frequency components and low frequency components, can be obtained after frequency decomposition, and feature compression is performed on each frequency component, i.e., feature compression is performed on both high frequency components and low frequency components, so that both the overall information of the initial feature image and the detail and texture information of the initial feature image can be retained to facilitate more accurate classification of the target image in the subsequent process.

[0143] It can be understood that since the high frequency component is also subjected to feature compression, and due to the combined effects of signal attenuation, signal distortion and other factors, the high frequency component often carries certain noise interference, and therefore, the noise needs to be removed to improve the accuracy of the classification result of the target image.

[0144] Specifically, since the noise distribution is irregular, the feature distribution of the first reorganized feature map can be analyzed to determine the target feature, i.e., the noise, that needs to be removed. For example, the classifier can be used to analyze the distribution shape of the feature in the first reorganized feature map, and if the distribution shape of a certain feature is significantly different from those of other features, it may indicate that the feature has noise. Alternatively, each feature can be compared with other features to observe its distribution on the first reorganized feature map, and if the distribution of a certain feature is significantly different from those of other features, it may indicate that the feature has noise. Alternatively, the similarity or distance between features can be calculated, and the target feature can be compared with other features, and if the similarity between the target feature and other features is low or the distance is large, it may indicate that the feature has noise. Alternatively, a feature distribution graph can be drawn to observe the distribution of the feature value, and if the distribution of a certain feature is significantly deviated from the normal condition, i.e., the distribution is inconsistent, it may be a noise feature.

[0145] In some embodiments, the target feature map acquisition module 1104 is configured to perform repeated calibration based on the first selection weight map to obtain a second selection weight map, and multiply the second selection weight map with the initial feature map to obtain a target feature map after feature calibration.

[0146] In some embodiments, the weight of each feature value can be re-allocated based on the first selection weight map, so that important features are more prominent, while unimportant features are suppressed, and the recognition ability of the image classification network for important features is improved, and then the number of channels is expanded one by one until the size of the second selection weight map obtained is consistent with the size of the initial feature map, so that each weight of the second selection feature map can accurately correspond to the features of the initial feature map, and the importance of each feature in the initial feature map is intuitively expressed. Specifically, the range of corresponding channel calibration can be gradually expanded during feature recalibration, and the feature values of the corresponding channels are multiplied, until the size of the second selection weight map obtained is consistent with the size of the initial feature map, so as to improve the accuracy of image classification.

[0147] It can be understood that the corresponding multiplication operation of the weight value can be performed on each channel in turn, and the specific formula is as follows:

[0148]

[0149] wherein, represents the first selection weight map, and h(·) represents a repeated channel multiplication operation of aligning the first selection weight map and the recalibration weight map in the feature space; that is, represents that the channel multiplication of the new range is performed after the expansion of each channel, until the second selection feature map obtained by expansion reaches the number of channels (C), height (H) and width (W) consistent with the initial feature map (P); represents element-by-element multiplication, represents the target feature map.

[0150] Please refer to Figure 4 , in Figure 4 , Selection represents selection, Recalibration represents recalibration, GAP represents feature compression, DWT represents frequency decomposition, represents the first selection weight corresponding to the four frequency components, P1 represents all coefficient features of the first recombined feature map, p a represents the coefficient feature corresponding to the maximum sorting score, P2 represents the first recombined feature map after p a is removed, represents the first selection weight map, The target feature map is represented, Fa represents a feature map obtained by convolution operation, Fr refers to a one-dimensional feature vector converted from a two-dimensional feature map by a flattening operation on feature activation, and Fs refers to a space composed of feature maps generated by convolution layers and pooling layers and the like. In some embodiments, the initial feature map can be subjected to frequency decomposition to obtain a plurality of frequency components, and then each frequency component is subjected to feature compression and splicing to obtain a first reorganized feature map. Then, feature selection is performed according to the feature distribution, and noise is removed, so as to obtain a first selection weight map corresponding to the plurality of frequency components. Further, the first selection weight map can be subjected to feature recalibration to suppress unimportant features and highlight important features, so as to obtain a second selection weight map. Finally, the second selection weight map is multiplied with the initial feature map to obtain a target feature map, and the target feature map is classified to obtain a classification result.

[0151] It can be understood that the repeated calibration can copy each weight value to the position corresponding to the original feature map, so that the size of the second weight map is consistent with that of the initial feature map. Then, multiplying the second selection weight map with the initial feature map can multiply the weight of each pixel in the target image with the feature value at the corresponding position to obtain a weighted feature map.

[0152] It can be understood that the repeated calibration of the feature map can realize the weighting operation of the feature map, so as to highlight or suppress unimportant features. Specifically, the weighting operation can be realized by adjusting the weight, so as to change the importance of pixels at different positions in the initial feature map, extract features beneficial to the classification task, and weaken or remove features not beneficial to the classification task.

[0153] In some embodiments, the classification result acquisition module 1105 is configured to classify according to the target feature map to obtain a classification result of the target image.

[0154] In some embodiments, after obtaining the target feature map after recalibration, the target feature map is taken as an input to be classified by a fully connected layer or other appropriate classifier. Specifically, a normalized exponential function (softmax function) can be used for multi-class classification, or other classifiers such as Support Vector Machines (SVM) can be used for classification.

[0155] Please refer to Figure 3In some embodiments, steps S101 to S105 can be packaged into an attention mechanism method module, and the obtained attention mechanism method module is represented as a selection-recalibration module (SRM). The SRM is inserted into a convolution module, such as a classical residual module, to form a residual block branch-selection and recalibration module (RBB-SRM), and finally form an image classification network. The target image can be accurately classified. Figure 3 (a) in FIG. 1 is a convolutional neural network with the RBB-SRM, Figure 3 (b) in FIG. 1 is a residual module without the RBB-SRM, Figure 3 (c) in FIG. 1 is a residual module with the RBB-SRM, input represents input, output represents output, Conv is the abbreviation of convolution operation, BN represents batch normalization (Batch Normalization), ReLU is the abbreviation of rectified linear unit (Rectified Linear Unit), avepool is the abbreviation of average pooling, and FC represents the abbreviation of fully connected layer (Fully Connected Layer).

[0156] The image classification method, system, electronic device, and storage medium provided in the present application can extract features of a target image through an image classification network, obtain an initial feature map, and perform frequency decomposition on the initial feature map to obtain a plurality of frequency components. By performing feature compression on each frequency component, the details and structural information of the initial feature map can be compressed and spliced, and the details of the feature map are retained. Then, the features of the first reorganized feature map obtained by splicing are analyzed to filter out target features with inconsistent distributions, and the target features are removed from the first reorganized feature map to suppress noise. Based on the first selection weight map after noise suppression, repeated calibration is performed to obtain a second selection weight map, so that meaningful image features are more prominent. The second selection weight map is multiplied by the initial feature map to obtain a target feature map after feature calibration, so as to calibrate the image features of the initial feature map. Finally, the target feature map is classified, and since the features have been calibrated, the classification result of the target image is more accurate. The present application can extract the details of the feature image while suppressing noise, thereby improving the accuracy of classifying the target image.

[0157] The specific implementation of the image classification system is basically the same as the specific implementation of the image classification method described above, and will not be repeated here. The image classification system can also be provided with other functional modules to implement the image classification method in the above embodiments, provided that the requirements of the embodiments of the present application are met.

[0158] The embodiments of the present application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the image classification method described above when executing the computer program. The electronic device can be any intelligent terminal, including a tablet computer, a vehicle-mounted computer, etc.

[0159] Please refer to Figure 12 , Figure 12 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0160] The processor 1201 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0161] The memory 1202 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1202 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1202 and called and executed by the processor 1201 to implement the image classification method of the embodiments of the present application.

[0162] The input / output interface 1203 is used to realize information input and output.

[0163] The communication interface 1204 is used to realize the communication interaction between the device and other devices. The communication can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0164] The bus 1205 transmits information between various components (such as the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204) of the device.

[0165] The processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204 are connected to each other through a bus 1205 to realize communication connection between devices inside.

[0166] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the image classification method.

[0167] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0168] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0169] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0170] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purposes of the embodiments of the present application.

[0171] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0172] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a recited step or its integral sub-steps or additional steps whether or not readily ascertainable from the description or the like. Further, the words "a" or "an", as used herein in the disclosure and elsewhere, are used indiscriminately and are to be interpreted in the same way, i.e. as meaning "one or more".

[0173] It should be understood that, in the application, "at least one" and "several" refer to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0174] In several embodiments provided in the application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0175] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0176] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0177] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, a network device, or the like) to perform all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and various program storage media.

[0178] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. An image classification method, characterized by, The method comprises: acquiring a target image to be classified, and inputting the target image into a preset image classification network; in the image classification network, performing feature extraction on the target image to obtain an initial feature map, and performing frequency decomposition based on the initial feature map to obtain a plurality of frequency components; after feature compression of each of the frequency components, obtaining a channel scalar corresponding to each of the frequency components; analyzing the feature distribution of each of the coefficient features to obtain a ranking score of each of the coefficient features; and determining a plurality of target features from the first reorganized feature map according to the ranking score; after removing each of the target features of the first reorganized feature map, multiplying each feature of the first reorganized feature map by a corresponding selection weight and then adding them to obtain a fusion selection weight map corresponding to each of the frequency components; inputting the fusion selection weight map into a first fully connected layer to obtain first output data; and inputting the first output data into a second fully connected layer and then performing activation to generate a first selection weight map with channel dependency; based on the first selection weight map, repeatedly calibrating to obtain a second selection weight map, and multiplying the second selection weight map by the initial feature map to obtain a target feature map after feature calibration; classifying according to the target feature map to obtain a classification result of the target image.

2. The image classification method of claim 1, wherein, The feature compression of each of the frequency components comprises: acquiring the height and width of each of the frequency components; calculating the frequency coefficient cumulative value of each of the frequency components within the height and the width; dividing the frequency coefficient cumulative value by the product of the height and the width to obtain a channel scalar corresponding to each of the frequency components.

3. The image classification method of claim 2, wherein, After feature compression of each of the frequency components, the first reorganized feature map is obtained by: obtaining a channel scalar corresponding to each of the frequency components after compression of each of the frequency components; acquiring the channel dimension of each of the channel scalars; sequentially splicing the channel scalars of each of the frequency components according to the channel dimension to obtain the first reorganized feature map of the initial feature map.

4. The image classification method of claim 1, wherein, The analysis of the feature distribution of each of the coefficient features to obtain the ranking score of each of the coefficient features comprises: analyzing the feature distribution of each of the coefficient features to obtain a feature analysis result; scoring each of the coefficient features according to the feature analysis result to obtain the ranking score of each of the coefficient features.

5. The image classification method of claim 1, wherein, The repeated calibration based on the first selection weight map to obtain the second selection weight map comprises: inputting the first selection weight map into a feedforward network for weight conversion to output a recalibration weight map; normalizing each weight value in the recalibration weight map; multiplying each weight value of the first selection weight map and the recalibration weight map in the corresponding channel to obtain the second selection weight map.

6. An image classification system characterized by, The system comprises: a target image acquisition module, configured to acquire a target image to be classified, and input the target image into a preset image classification network; The frequency component acquisition module is configured to perform feature extraction on the target image in the image classification network to obtain an initial feature map, and perform frequency decomposition based on the initial feature map to obtain a plurality of frequency components. The first selection weight map acquisition module is configured to splice the frequency components after feature compression to obtain a first reorganized feature map, extract a plurality of coefficient features from the first reorganized feature map, analyze the feature distribution of each coefficient feature to obtain a ranking score of each coefficient feature, determine a plurality of target features from the first reorganized feature map according to the ranking score, multiply each feature of the first reorganized feature map by a corresponding selection weight after removing each target feature of the first reorganized feature map, and add the features to obtain a fusion selection weight map corresponding to each frequency component, input the fusion selection weight map into a first fully connected layer to obtain first output data, and input the first output data into a second fully connected layer and activate the second fully connected layer to generate a first selection weight map with a channel dependency relationship. The target feature map acquisition module is configured to perform repeated calibration based on the first selection weight map to obtain a second selection weight map, multiply the second selection weight map by the initial feature map, and obtain a target feature map after feature calibration. The classification result acquisition module is configured to perform classification based on the target feature map to obtain a classification result of the target image.

7. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the image classification method of any one of claims 1 to 5 when executing the computer program.

8. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the image classification method of any one of claims 1 to 5.

Images