Image classification method

Abstract

The invention relates to the technical field of image processing, and provides an image classification method, comprising: acquiring an image to be classified and a classification network, the classification network comprising a first module, a second module, a third module and a fourth module; the to-be-classified image after preprocessing is input to the first module to output a first feature map, the first unit is used for extracting features, and the second unit is used for enhancing feature representation; inputting the first feature map into a second module to output a second feature map; inputting the second feature map into a third module to output a third feature map; and inputting the third feature map into a fourth module to output a classification result. According to the method, by increasing the complexity of feature extraction and enhancement, the risk of overfitting can be reduced while the complexity of a classification network is kept, and more global information can be captured by increasing deeper feature extraction, so that the limitation of local features is reduced, and the accuracy and efficiency of a classification task can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image classification method. Background Art

[0002] In medical image classification tasks, convolutional neural networks (CNNs) are used due to their feature extraction capabilities. By combining different types of CNN architectures, the accuracy and generalization ability of the model can be further improved.

[0003] Different types of CNN architectures can be used to perform different tasks, for example, combining Faster Net, CSP (Channel-Spatial Pyramid), and PCBS (Patch-based Convolutional Boosting System). Faster Net provides efficient feature extraction capabilities, CSP enhances multi-scale feature extraction, and PCBS improves the classification performance of local features.

[0004] However, in the process of combining, there are problems such as overfitting, high computational complexity, feature redundancy, local feature limitations and high memory usage, which lead to low accuracy and efficiency of classification tasks. Summary of the invention

[0005] The present application provides an image classification method to solve the problems of low accuracy and efficiency in classification tasks.

[0006] The present application provides an image classification method, comprising: Acquire an image to be classified and a classification network, wherein the classification network includes a first module, a second module, a third module and a fourth module; Inputting the preprocessed image to be classified into the first module to output a first feature map, wherein the first module includes a first unit and at least two second units, wherein the first unit is used to extract features, and the second unit is used to enhance feature representation; Inputting the first feature map into the second module to output a second feature map, the second module comprising a first unit and at least three second units; Inputting the second feature map into the third module to output a third feature map, the third module comprising a first unit and at least four second units; The third feature map is input to the fourth module to output a classification result, and the fourth module includes a first unit and at least three second units.

[0007] In some feasible embodiments, the step of inputting the preprocessed image to be classified into the first module to output a first feature map includes: Inputting the preprocessed image to be classified into the first unit to output a first intermediate feature map, wherein the number of channels of the first intermediate feature map is a multiple of the first number of the image to be classified; Inputting the first intermediate feature map into the second unit to output a second intermediate feature map; The second intermediate feature map is input to the second unit to output a first feature map.

[0008] In some feasible embodiments, the first unit includes a first-size convolution and a second-size convolution; The step of inputting the preprocessed image to be classified into the first unit to output a first intermediate feature map comprises: Inputting the preprocessed image to be classified into two convolutions of the first size respectively to output a first convolution map and a second convolution map; Inputting the first convolution map into the second-size convolution to output a third convolution map; Inputting the third convolution map into the first-size convolution to output a fourth convolution map; The second convolution map and the fourth convolution map are passed through an activation function to output a first intermediate feature map.

[0009] In some feasible embodiments, the second unit includes a segmentation layer, a first feature fusion block, and a first convolution enhancement block; The step of inputting the first intermediate feature map to the second unit to output a second intermediate feature map comprises: Inputting the first intermediate feature map to the segmentation layer to output a first segmentation map and a second segmentation map; Inputting the second segmentation map into the first feature fusion block to output a first fusion map; Inputting the first fusion map to the first convolution enhancement block to output a first enhanced map; A splicing operation is performed on the second segmentation map and the first enhanced map to output a second intermediate feature map.

[0010] In some feasible embodiments, the second unit further includes a second feature fusion block and a second convolution enhancement block; The step of inputting the first intermediate feature map to the second unit to output a second intermediate feature map comprises: Inputting the first enhanced image into the second feature fusion block to output a second fusion image; Inputting the second fused image into the second convolution enhancement block to output a second enhanced image; A splicing operation is performed on the second enhanced image, the second segmentation image, and the first enhanced image to output a second intermediate feature map.

[0011] In some feasible embodiments, the first feature fusion block includes a first convolutional layer, a fast block, and a second convolutional layer; The step of inputting the second segmentation map into the first feature fusion block to output a first fusion map includes: Inputting the second segmentation map into two of the first convolutional layers respectively to output a fifth convolutional map and a sixth convolutional map; Inputting the sixth convolutional map into the fast block to output a first local feature map; Performing a splicing operation on the fifth convolutional map and the first local feature map to output a first spliced ​​map; The first spliced ​​image is input into the second convolutional layer to output a first fused image.

[0012] In some feasible embodiments, the first convolution layer includes a two-dimensional convolution layer and a first batch of normalization layers; The step of inputting the second segmentation map to two of the first convolutional layers respectively to output a fifth convolutional map and a sixth convolutional map comprises: Inputting the second segmentation map into the two-dimensional convolutional layer to output a seventh convolutional map; Inputting the seventh convolutional map into the first batch of normalization layers to output a first transformed feature map; The first transformed feature map passes through an activation function to output a fifth convolution map or a sixth convolution map.

[0013] In some feasible embodiments, the first convolution enhancement block includes a third convolution layer and a second batch normalization layer; The step of inputting the first fusion map to the first convolution enhancement block to output a first enhanced map comprises: Inputting the first fusion map to the third convolutional layer to output an eighth convolutional map; Inputting the eighth convolutional map into the second batch normalization layer to output a second transformed feature map; The second transformed feature map is passed through an activation function to output a first enhanced map.

[0014] In some feasible embodiments, the third convolutional layer includes a two-dimensional convolutional layer; The step of inputting the first fusion graph into the third convolutional layer to output an eighth convolutional graph comprises: Performing a segmentation operation on the first fusion image to generate a first segmentation image and a second segmentation image, wherein the number of channels of the first segmentation image is one quarter of that of the first fusion image, and the number of channels of the second segmentation image is three quarters of that of the first fusion image; Inputting the first segmentation map into the two-dimensional convolutional layer to output a ninth convolutional map; A splicing operation is performed on the ninth convolution map and the second segmentation map to output an eighth convolution map.

[0015] In some feasible embodiments, the classification network further includes: a fourth convolutional layer and a maximum pooling layer; The step of inputting the preprocessed image to be classified into the first module to output the first feature map comprises: Inputting the image to be classified into the fourth convolutional layer to output a fifth convolutional map; The fifth convolutional map is input into the maximum pooling layer to output the preprocessed image to be classified.

[0016] It can be seen from the above technical solution that the present application provides an image classification method, including: obtaining an image to be classified and a classification network, wherein the classification network includes a first module, a second module, a third module and a fourth module; then inputting the preprocessed image to be classified into the first module to output a first feature map, the first module includes a first unit and at least two second units, the first unit is used to extract features, and the second unit is used to enhance feature representation; inputting the first feature map into the second module to output a second feature map, the second module includes a first unit and at least three second units; inputting the second feature map into the third module to output a third feature map, the third module includes a first unit and at least four second units; inputting the third feature map into the fourth module to output a classification result, the fourth module includes a first unit and at least three second units. The method can reduce the risk of overfitting while maintaining the complexity of the classification network by gradually increasing the complexity of feature extraction and enhancement, and can capture more global information by adding deeper feature extraction units, thereby reducing the limitations of local features. It can effectively process images and improve the accuracy and efficiency of classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of the image classification method provided in the embodiment of the present application; Figure 2 A schematic diagram of a classified network structure provided for an embodiment of the present application; Figure 3 A schematic diagram of the image processing flow to be classified provided in an embodiment of the present application; Figure 4 A schematic diagram of the first unit structure provided in an embodiment of the present application; Figure 5 A schematic diagram of the second unit processing flow provided in an embodiment of the present application; Figure 6 A schematic diagram of the first feature fusion block structure provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of the first convolutional layer provided in an embodiment of the present application; Figure 8 A schematic diagram of a first convolution enhancement block provided in an embodiment of the present application; Fig. 9 A schematic diagram of the third convolutional layer structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0020] Medical images refer to images of the internal structure of the human body obtained through imaging technologies such as X-rays, CT, MRI, and ultrasound. Medical image classification tasks are mainly used for disease diagnosis, lesion localization, condition assessment, auxiliary surgical planning, pathological analysis, etc.

[0021] In medical imaging data, diseases or lesions appear as local patterns or structures in the image, and meaningful feature representations can be automatically learned from the original pixel values. Through a series of convolutional layers and pooling layers, CNN can capture local and global features in the image. Different CNN architectures, such as Res Net, Inception, and VGG, have different characteristics, and models can be built by combining these architectures. For example, Res Net solves the gradient vanishing problem of deep networks by introducing residual connections, thereby improving the training stability of the model.

[0022] By fusing different CNN architectures, we can take full advantage of their strengths and improve the overall performance of the model. For example, we can combine the residual connection of Res Net with the multi-scale characteristics of the Inception module. Another example is combining FasterNet, CSP, and PCBS.

[0023] Faster Net is a convolutional neural network that optimizes the depth and parameter configuration of the convolution layer. Unlike other deep networks, Faster Net reduces computational complexity by reducing network depth and improving convolution operations. This enables Faster Net to achieve fast and accurate feature extraction at a lower computational cost. Through a streamlined network architecture and efficient computing methods, the speed and performance of image processing are optimized.

[0024] The CSP network enhances feature extraction capabilities through a channel space pyramid structure. This structure divides image feature extraction into different scales, extracts features at multiple spatial levels, and combines channel features and spatial features to enhance the model's ability to capture information at different scales. The CSP pyramid structure can effectively capture details and local features in the image. And by processing features at multiple levels, it enhances the understanding of image structure and details.

[0025] PCBS uses a convolution enhancement system based on image block division, which divides the input image into several small blocks and performs independent convolution operations on each block. It can capture local features more carefully and effectively enhance the analysis ability of local details when processing high-resolution images. PCBS improves the overall classification performance by enhancing local features, making the model perform better in the recognition and classification of fine-grained features. By conducting in-depth analysis of local blocks of the image, the model's ability to process complex images is improved.

[0026] In some embodiments, the input medical image is preprocessed, for example, normalized, cropped, etc., to ensure that the image is suitable for model input. The preprocessed image is input into Faster Net to extract preliminary features, and the feature map output by Faster Net is input into the CSP module. The CSP module fuses features through different channels and spatial pyramid structures to capture information at different scales. The feature map output by CSP is input into the PCBS module. The PCBS module enhances local feature representation and reduces feature redundancy by independently processing small blocks of the image. The feature map enhanced by PCBS is input into the fully connected layer or other classifier to output the final classification result.

[0027] However, due to the relatively small size of medical imaging datasets and the combination of different modules, the model is prone to overfitting, the use of multi-layer convolutional networks and complex feature fusion techniques will increase the computational burden, duplicate features may appear when using multi-scale feature extraction, only focusing on local features may cause the model to fail to capture global information, and the use of large convolutional kernels and deep networks will lead to increased memory consumption. As a result, the accuracy and efficiency of classification tasks are low.

[0028] like Figure 1 To solve the above problems, the present invention provides an image classification method, including: S100: Obtain an image to be classified and a classification network.

[0029] In the medical image classification task, the image to be classified can be medical image data, and the image data can come from different imaging technologies, including but not limited to X-ray images, computed tomography (CT) images, magnetic resonance imaging (MRI) images, ultrasound images, digital pathology images, optical coherence tomography (OCT) images, and fluorescein angiography (FA) images.

[0030] Among them, X-ray images are used to show bone structure, lung conditions, etc. Computerized tomography images provide cross-sectional images. Magnetic resonance imaging images provide detailed images of soft tissues. Ultrasound images are used to show fetal development, heart conditions, abdominal organs, etc. Digital pathology images are used for microscopic image analysis of tissue samples. Optical coherence tomography images are used to display ophthalmic imaging. Fluorescein fundus angiography images are also used to display eye images, such as diabetic retinopathy.

[0031] Before the image to be classified is input into the classification network, in some embodiments, preprocessing is performed on the image to be classified, and the preprocessing includes one or more of image normalization, resizing, cropping and padding, data augmentation, and channel conversion. The preprocessing can enable the classification network to be effectively trained and classified.

[0032] Image normalization is to scale the pixel values ​​of an image to a specific range to improve the convergence speed of the classification network. Resizing is to adjust the image to a uniform size to meet the input requirements of the classification network. Cropping and padding are to remove irrelevant areas or make the image reach the required size. Data augmentation increases the diversity of the data set through operations such as rotation, flipping, and scaling to improve the generalization ability of the classification network. For grayscale images, they can be converted to single-channel images; for color images, they are kept in their original RGB format.

[0033] In some embodiments, the image to be classified is a color image with a height and a width of 224 pixels, that is, the image to be classified is a color image with a resolution of 224×224 pixels.

[0034] After preprocessing, the image to be classified is processed into a color image with a resolution of 224×224 pixels and input into the classification network. Before input, a pre-trained classification network is obtained. The training of the classification network at least includes the steps of obtaining a data set, defining a model, training, adjusting parameters, and verification testing. The specific implementation method is known to those skilled in the art and will not be elaborated here.

[0035] like Figure 2 , Figure 3 As shown, the pre-trained classification network is used for medical image classification, and the classification network includes a first module, a second module, a third module, and a fourth module. The first module includes a first unit and at least two second units, the second module includes a first unit and at least three second units, the third module includes a first unit and at least four second units, and the fourth module includes a first unit and at least three second units, wherein the second module has the same structure as the fourth module.

[0036] To facilitate the extraction and classification of images to be classified, in some embodiments, the classification network also includes: a fourth convolutional layer and a maximum pooling layer; the image to be classified is input into the fourth convolutional layer to output a fifth convolutional map; the fifth convolutional map is input into the maximum pooling layer to output the preprocessed image to be classified.

[0037] After preprocessing, the size of the image to be classified is changed from (3,244,244) to (64,112,112) through the convolution layer, and then changed to (64,56,56) through the maximum pooling layer. That is, the size of the image to be classified after preprocessing is (64,56,56).

[0038] S200: Inputting the preprocessed image to be classified into a first module to output a first feature map.

[0039] Since the first module includes a first unit and two second units, in some embodiments, the preprocessed image to be classified is first input into the first unit, and a first intermediate feature map is output. Figure 4 As shown, the first unit includes multiple convolutional layers (CONV), namely, first-size convolution and second-size convolution.

[0040] Taking the first size convolution as 1×1 convolution and the second size convolution as 3×3 convolution as an example, the preprocessed image to be classified is input into two 1×1 CONVs respectively to obtain the first convolution map and the second convolution map, and then the first convolution map is input into 3×3 CONV to obtain the third convolution map, and then the third convolution map is input into 1×1 CONV to obtain the fourth convolution map, and the second convolution map and the fourth convolution map are processed by the RELU function to obtain the first intermediate feature map. After being processed by the first unit, the size is changed from (64,56,56) to (256,56,56), that is, the size of the first intermediate feature map is (256,56,56).

[0041] Then the first intermediate feature map is input into the second unit, and the second intermediate feature map is output, such as Figure 5 As shown, the second unit includes a segmentation layer, a first feature fusion block and a first convolution enhancement block. In some embodiments, the first intermediate feature map is input into the segmentation layer to output a first segmentation map and a second segmentation map; the second segmentation map is input into the first feature fusion block to output a first fusion map; the first fusion map is input into the first convolution enhancement block to output a first enhancement map; and a concatenation operation is performed on the second segmentation map and the first enhancement map to output a second intermediate feature map.

[0042] The segmentation layer divides the input image into a first segmentation map and a second segmentation map. The segmentation layer evenly divides the first intermediate feature map into two blocks along the second dimension. These two blocks are the first segmentation map and the second segmentation map. The segmentation layer is used to process features in parallel to capture information from different aspects. The second segmentation map is input to the first feature fusion block, and the output first fusion map contains features from the second segmentation map; the first fusion map is input to the first convolution enhancement block, and the output first enhancement map enhances the features in the first fusion map. The second segmentation map and the first enhancement map are spliced ​​together to form a second intermediate feature map. The splicing operation merges two different feature representations to provide richer information.

[0043] Among them, Figure 6 As shown in the figure, the first feature fusion block includes the first convolutional layer, the fast block and the second convolutional layer; the first feature fusion block is a Faster CSP block constructed by combining Faster Block and CSP, that is, a fast block. Faster Block can accelerate the training and reasoning process of the classification network, while the CSP network can optimize the gradient flow and parameter usage efficiency. The FasterCSP block can enhance the feature extraction capability of the network while reducing the computational burden and improving the efficiency and performance of the classification network. Through the combination of parallel convolution and fast feature extraction, the first feature fusion block can generate a diverse feature representation. This diversity helps the network to generalize better in subsequent classification tasks.

[0044] In some embodiments, the second segmentation map is first input into two first convolutional layers respectively to output a fifth convolutional map and a sixth convolutional map, the sixth convolutional map is input into the fast block to output a first local feature map, a splicing operation is performed on the fifth convolutional map and the first local feature map to output a first spliced ​​map, and the first spliced ​​map is input into the second convolutional layer to output a first fusion map.

[0045] The Faster CSP block divides the input layer into two parts, one of which is subjected to CBS convolution, i.e. the first convolution layer, and the other part is subjected to CBS convolution first and then passes through the Faster Block. The data is then concatenated and then subjected to CBS convolution to obtain the output of the block structure, and the input and output scales are consistent. The first local feature map output contains features from the sixth convolution map, the first spliced ​​map output contains a combination of two different features, and the first fusion map output contains features from the first spliced ​​map.

[0046] Among them, the first convolutional layer and the second convolutional layer have the same structure, namely CBS convolution, such as Figure 7 As shown, the CBS convolution includes a two-dimensional convolution layer and a batch normalization (BN) layer. For the first convolution layer, in some embodiments, the second segmentation map is input into the two-dimensional convolution layer, the seventh convolution map is output, and the seventh convolution map is input into the first batch normalization layer, the first transformation feature map is output, and the first transformation feature map is passed through an activation function to output a fifth convolution map or a sixth convolution map.

[0047] For the second convolution layer, in some embodiments, the first spliced ​​image is input into a two-dimensional convolution layer, and the output of the two-dimensional convolution layer is input into the first batch of normalization layers, and the output of the first batch of normalization layers is passed through an activation function to output a first fusion image.

[0048] The first fusion image is input to the first convolution enhancement block, and the first enhanced image is output, where Figure 8 As shown, the first convolution enhancement block includes a third convolution layer and a second batch normalization layer. In some embodiments, the first fusion map is input to the third convolution layer to output the eighth convolution map, the eighth convolution map is input to the second batch normalization layer, the second transformation feature map is output, and the second transformation feature map is passed through the activation function to output the first enhancement map. The normalization layer helps stabilize the training process, prevents the gradient from disappearing or exploding, and helps accelerate convergence. The activation function introduces nonlinear mapping, enabling the classification network to fit more complex functional relationships.

[0049] Among them, Fig. 9As shown, the third convolution layer includes a two-dimensional convolution layer; in some embodiments, a split operation is performed on the first fusion map to generate a first segmentation map and a second segmentation map, the number of channels of the first segmentation map is one quarter of the first fusion map, and the number of channels of the second segmentation map is three quarters of the first fusion map; the first segmentation map is input into the two-dimensional convolution layer, and a ninth convolution map is output; a splicing operation is performed on the ninth convolution map and the second segmentation map, and an eighth convolution map is output.

[0050] By segmenting and processing the first fusion map separately, the classification network can better capture different feature levels. The small part of the features after segmentation, that is, the first segmentation map, is further enhanced through convolution, while the larger part of the features, that is, the second segmentation map, remains unchanged or slightly processed. By splicing the ninth convolution map and the second segmentation map, the classification network can integrate the information of the two parts to obtain a more comprehensive feature representation.

[0051] While maintaining efficient computing and powerful feature extraction capabilities, the model's ability to handle incomplete inputs is further enhanced. By combining the efficient computing of Faster Block with the partial convolution characteristics of the PCBS block, the first convolution enhancement block, and introducing the cross-stage partial connection of CSP Net and the ELAN (Enhanced Local Aggregation Network) structure, the PFaster CSPELAN block can optimize feature fusion and gradient flow while ensuring computing efficiency, thereby improving the robustness and adaptability of the model.

[0052] Continue to see Figure 5 In some embodiments, the second unit also includes a second feature fusion block and a second convolution enhancement block, wherein the second feature fusion block has the same structure as the first feature fusion block, and the second convolution enhancement block has the same structure as the first convolution enhancement block. Repeatedly processing features using the same structure helps alleviate the gradient vanishing problem in deep networks.

[0053] After outputting the first enhanced image, the first enhanced image is input into the second feature fusion block to output the second fusion image; the second fusion image is input into the second convolution enhancement block to output the second enhanced image; and a splicing operation is performed on the second enhanced image, the second segmentation image and the first enhanced image to output the second intermediate feature image.

[0054] By sending the first enhanced map to the feature fusion block and the convolution enhancement block again, the features can be further refined and enhanced, and the expressive power of the classification network can be increased. The splicing operation combines the feature maps of different stages, namely the first enhanced map, the second enhanced map, and the second segmentation map, which can fuse multi-scale information and improve the training stability of the classification network through skip connections and other methods. Although the segmentation operation is introduced, only part of the feature map is convolved, which helps to maintain computational efficiency to a certain extent. At the same time, the splicing operation also avoids the redundancy that may be caused by completely independent feature processing paths.

[0055] In summary, the first unit is used to extract and enhance key features from the original input and output a preliminary enhanced feature map. The second unit further processes these features on this basis and outputs a more refined and comprehensive feature representation through repeated feature fusion and enhancement operations.

[0056] The first module performs preliminary feature extraction and enhancement on the input data, and the two second units can further fuse and enhance these features to provide more detailed information.

[0057] S300: Input the first feature map to the second module to output a second feature map.

[0058] For the second module, the output of the first module, i.e., the first feature map, is further processed. Since the second module includes a first unit and three second units, wherein the first unit and the second unit of the second module have the same structure as the first module, the second module continues to perform feature refinement and enhancement based on the first module.

[0059] The processing process of the first unit and the second unit is the same as that of the first unit and the second unit in the first module. With more second units, deeper feature information can be captured.

[0060] For example, Figure 3 As shown, the first feature map is the output of the first module. The first feature map is input into the first unit of the second module. The first unit adjusts the size of the first feature map from (256, 56, 56) to (512, 28, 28). The output of the first unit is input into the first second unit, the output of the first second unit is input into the second second unit, the output of the second second unit is input into the third second unit, and the output of the third second unit is the second feature map.

[0061] S400: Input the second feature map to the third module to output a third feature map.

[0062] Similar to the second module, the input of the third module is the output of the second module. The third module further deepens the learning level of features through more second units and improves the expression ability of the classification network. Based on the first and second modules, the third module can provide a more comprehensive and detailed feature representation.

[0063] For example, Figure 3 As shown, the second feature map is the output of the second module. The second feature map is input into the first unit of the third module. The first unit adjusts the size of the second feature map (512, 28, 28) to (1024, 14, 14). The output of the first unit is input into the first second unit, the output of the first second unit is input into the second second unit, the output of the second second unit is input into the third second unit, the output of the third second unit is input into the fourth second unit, and the output of the fourth second unit is the third feature map.

[0064] S500: Input the third feature map to the fourth module to output a classification result.

[0065] Similar to the second and third modules, the input of the fourth module is the output of the third module. The fourth module further optimizes the feature representation based on the output of the third module. The classification network can capture more layers of features, from low-level textures to high-level semantic information. Prepare the final output for classification, regression or other task results.

[0066] For example, Figure 3 As shown, the third feature map is the output of the third module. The third feature map is input into the first unit of the fourth module. The first unit adjusts the size of the third feature map from (1024, 14, 14) to (2048, 7, 7). The output of the first unit is input into the first second unit, the output of the first second unit is input into the second second unit, the output of the second second unit is input into the third second unit, and the output of the third second unit is the classification result.

[0067] According to the first module, the second module, the third module and the fourth module, as the module level goes deeper, the size of the feature map gradually decreases, while the number of channels gradually increases. In other words, the classification network becomes more detailed when processing features and can capture higher-level feature information.

[0068] The number of channels increases from 256 to 512, 1024, and 2048. As the classification network goes deeper, it can capture more feature types and details. The spatial resolution of the feature map decreases from (56, 56) to (28, 28), then to (14, 14), and finally to (7, 7), indicating that as the classification network goes deeper, the focus of the feature map shifts from local details to more abstract concepts. The feature map size is finally reduced to (7, 7), which helps the classification network to integrate and optimize features and prepare for classification tasks.

[0069] In some embodiments, the classification network further includes a global pooling layer and a fully connected layer, the global pooling layer is connected to the fourth module, and the fully connected layer is connected to the global pooling layer to output a classification result, and the classification result represents the probability estimation of the classification network that the input image belongs to a certain category. For example, a vector in which each element corresponds to a category and represents the probability corresponding to the category.

[0070] For example, if the image to be classified is a breast tissue pathology section image containing a tumor, the classification network may output a probability vector, such as [0.1, 0.9], indicating that the probability that the image is "benign" is 10% and the probability that it is "malignant" is 90%.

[0071] In some embodiments, medical image classification tasks can also be performed by Resnet50 with an accuracy of 0.9145, and the accuracy of the classification network provided in this application is 0.9382.

[0072] It can be seen from the above technical solution that the present application provides an image classification method, including: obtaining an image to be classified and a classification network, wherein the classification network includes a first module, a second module, a third module and a fourth module; then inputting the preprocessed image to be classified into the first module to output a first feature map, the first module includes a first unit and at least two second units, the first unit is used to extract features, and the second unit is used to enhance feature representation; inputting the first feature map into the second module to output a second feature map, the second module includes a first unit and at least three second units; inputting the second feature map into the third module to output a third feature map, the third module includes a first unit and at least four second units; inputting the third feature map into the fourth module to output a classification result, the fourth module includes a first unit and at least three second units. The method can reduce the risk of overfitting while maintaining the complexity of the classification network by gradually increasing the complexity of feature extraction and enhancement, and can capture more global information by adding deeper feature extraction units, thereby reducing the limitations of local features. It can effectively process images and improve the accuracy and efficiency of classification tasks.

[0073] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. An image classification method, characterized in that: include: Acquire an image to be classified and a classification network, wherein the classification network includes a first module, a second module, a third module and a fourth module; Inputting the preprocessed image to be classified into the first module to output a first feature map, wherein the first module includes a first unit and at least two second units, wherein the first unit is used to extract features, and the second unit is used to enhance feature representation; Inputting the first feature map into the second module to output a second feature map, the second module comprising a first unit and at least three second units; Inputting the second feature map into the third module to output a third feature map, the third module comprising a first unit and at least four second units; The third feature map is input to the fourth module to output a classification result, and the fourth module includes a first unit and at least three second units.

2. The image classification method according to claim 1, characterized in that: The step of inputting the preprocessed image to be classified into the first module to output a first feature map comprises: Inputting the preprocessed image to be classified into the first unit to output a first intermediate feature map, wherein the number of channels of the first intermediate feature map is a multiple of the first number of the image to be classified; Inputting the first intermediate feature map into the second unit to output a second intermediate feature map; The second intermediate feature map is input to the second unit to output a first feature map.

3. The image classification method according to claim 2, characterized in that: The first unit includes a first-size convolution and a second-size convolution; The step of inputting the preprocessed image to be classified into the first unit to output a first intermediate feature map comprises: Inputting the preprocessed image to be classified into two convolutions of the first size respectively to output a first convolution map and a second convolution map; Inputting the first convolution map into the second-size convolution to output a third convolution map; Inputting the third convolution map into the first-size convolution to output a fourth convolution map; The second convolution map and the fourth convolution map are passed through an activation function to output a first intermediate feature map.

4. The image classification method according to claim 2, characterized in that: The second unit includes a segmentation layer, a first feature fusion block and a first convolution enhancement block; The step of inputting the first intermediate feature map to the second unit to output a second intermediate feature map comprises: Inputting the first intermediate feature map to the segmentation layer to output a first segmentation map and a second segmentation map; Inputting the second segmentation map into the first feature fusion block to output a first fusion map; Inputting the first fusion map to the first convolution enhancement block to output a first enhanced map; A splicing operation is performed on the second segmentation map and the first enhanced map to output a second intermediate feature map.

5. The image classification method according to claim 4, characterized in that: The second unit also includes a second feature fusion block and a second convolution enhancement block; The step of inputting the first intermediate feature map to the second unit to output a second intermediate feature map comprises: Inputting the first enhanced image into the second feature fusion block to output a second fusion image; Inputting the second fused image into the second convolution enhancement block to output a second enhanced image; A concatenation operation is performed on the second enhanced image, the second segmentation image, and the first enhanced image to output a second intermediate feature map.

6. The image classification method according to claim 4, characterized in that: The first feature fusion block includes a first convolutional layer, a fast block and a second convolutional layer; The step of inputting the second segmentation map into the first feature fusion block to output a first fusion map includes: Inputting the second segmentation map into two of the first convolutional layers respectively to output a fifth convolutional map and a sixth convolutional map; Inputting the sixth convolutional map into the fast block to output a first local feature map; Performing a splicing operation on the fifth convolutional map and the first local feature map to output a first spliced ​​map; The first spliced ​​image is input into the second convolutional layer to output a first fused image.

7. The image classification method according to claim 6, characterized in that: The first convolutional layer includes a two-dimensional convolutional layer and a first batch of normalization layers; The step of inputting the second segmentation map to two of the first convolutional layers respectively to output a fifth convolutional map and a sixth convolutional map comprises: Inputting the second segmentation map into the two-dimensional convolutional layer to output a seventh convolutional map; Inputting the seventh convolutional map into the first batch of normalization layers to output a first transformed feature map; The first transformed feature map passes through an activation function to output a fifth convolution map or a sixth convolution map.

8. The image classification method according to claim 4, characterized in that: The first convolution enhancement block includes a third convolution layer and a second batch of normalization layers; The step of inputting the first fusion map to the first convolution enhancement block to output a first enhanced map comprises: Inputting the first fusion map to the third convolutional layer to output an eighth convolutional map; Inputting the eighth convolutional map into the second batch normalization layer to output a second transformed feature map; The second transformed feature map is passed through an activation function to output a first enhanced map.

9. The image classification method according to claim 8, characterized in that: The third convolutional layer includes a two-dimensional convolutional layer; The step of inputting the first fusion graph into the third convolutional layer to output an eighth convolutional graph comprises: Performing a segmentation operation on the first fusion image to generate a first segmentation image and a second segmentation image, wherein the number of channels of the first segmentation image is one quarter of that of the first fusion image, and the number of channels of the second segmentation image is three quarters of that of the first fusion image; Inputting the first segmentation map into the two-dimensional convolutional layer to output a ninth convolutional map; A splicing operation is performed on the ninth convolution map and the second segmentation map to output an eighth convolution map.

10. The image classification method according to claim 1, characterized in that: The classification network also includes: a fourth convolutional layer and a maximum pooling layer; The step of inputting the preprocessed image to be classified into the first module to output the first feature map comprises: Inputting the image to be classified into the fourth convolutional layer to output a fifth convolutional map; The fifth convolutional map is input into the maximum pooling layer to output the preprocessed image to be classified.

Images