Image classification systems, methods and computer devices

Abstract

This invention relates to an image classification system, method, and computer device. The classification result is obtained by extracting features from an input medical image through a first classification network to obtain a first feature vector of an abnormal region in the medical image. Then, a first feature region detection network extracts the location information of the first feature vector to obtain the first location information of the abnormal region in the medical image. A first feature region acquisition unit obtains a first feature region image corresponding to the first location information in the medical image. Finally, an output network classifies the first feature region image and outputs the classification result. In this application, the first feature region image is automatically obtained through a neural network, which improves the accuracy of the obtained first feature region image and thus improves the accuracy of the classification result obtained based on the first feature region image.

Description

[0001] This application is a divisional application of the application filed on May 24, 2019, with application number 2019104379344 and invention title "Image Classification System, Method, Apparatus and Storage Medium". Technical Field

[0002] This invention relates to the field of deep learning technology, and in particular to an image classification system, method, and computer device. Background Technology

[0003] With the continuous development of artificial intelligence technology, neural network models are often used to process medical images. For example, Alzheimer's disease (AD), a common neurodegenerative disease, can be diagnosed by processing brain images using neural network models to determine whether there are abnormal regions in the brain images corresponding to AD, thereby identifying AD and related lesions.

[0004] In traditional methods, taking Alzheimer's disease (AD) as an example, brain images can be manually annotated to obtain AD-related feature regions, such as the hippocampus, temporal lobe, amygdala, and cingulate gyrus. Template image registration is then used to select multiple regions of interest from these feature regions, and feature information is extracted from each region of interest and input into a preset neural network model to obtain the brain image processing result. This processing result can characterize whether there are abnormal regions in the brain image corresponding to AD and related lesions. Summary of the Invention

[0005] Based on this, an image classification system, method, and computer device are provided.

[0006] In a first aspect, an image classification system includes:

[0007] The first classification network is used to extract features from the input medical image to obtain the first feature vector of the abnormal region in the medical image;

[0008] The first feature region detection network is used to extract the location information of the first feature vector to obtain the first location information of the abnormal region in the medical image;

[0009] The first feature region acquisition unit is used to obtain a first feature region image in a medical image corresponding to the first location information based on the first location information.

[0010] The output network is used to classify the image of the first feature region and output the classification result.

[0011] In one embodiment, the output network includes:

[0012] The second classification network is used to extract features from the first feature region image to obtain the second feature vector of the abnormal region in the first feature region image;

[0013] The second feature region detection network is used to extract the location information of the second feature vector to obtain the second location information of the abnormal region in the first feature region image;

[0014] The second feature region acquisition unit is used to obtain a second feature region image in the first feature region image that corresponds to the second location information based on the second location information.

[0015] The third classification network is used to classify the image of the second feature region to obtain the classification result.

[0016] In one embodiment, the system further includes:

[0017] An image processing unit is used to perform feature removal processing on the region corresponding to the first feature region image or the second feature region image in a medical image to obtain an updated medical image; the updated medical image is used to obtain an updated classification result through the image classification system, and the updated classification result includes the classification result obtained from feature regions other than the first feature region image or the second feature region image.

[0018] In one embodiment, the first classification network described above includes a fully connected layer;

[0019] The first feature vector of the abnormal region in the medical image is obtained from the fully connected layer.

[0020] In one embodiment, the training of the first classification network, the second classification network, the third classification network, the first feature region detection network, and the second feature region detection network is performed jointly.

[0021] Secondly, an image classification method, the method comprising:

[0022] The medical image is input into the first classification network to extract abnormal region features, and the first feature vector of the abnormal region in the medical image is obtained.

[0023] The first feature vector is input into the first feature region detection network to extract location information, thereby obtaining the first location information of the abnormal region in the medical image.

[0024] Obtain the image of the first feature region in the medical image that corresponds to the first location information;

[0025] The image of the first feature region is classified to obtain the classification result.

[0026] In one embodiment, the above-described classification process for the first feature region image to obtain a classification result includes:

[0027] The first feature region image is input into the second classification network to extract abnormal region features, thereby obtaining the second feature vector of the abnormal region in the first feature region image.

[0028] The second feature vector is input into the second feature region detection network to extract location information, thereby obtaining the second location information of the abnormal region in the first feature region image.

[0029] Obtain the second feature region image in the first feature region image that corresponds to the second location information;

[0030] The image of the second feature region is classified to obtain the classification result.

[0031] In one embodiment, the method further includes:

[0032] The region corresponding to the second feature region in the medical image is subjected to feature removal processing to obtain an updated medical image; the updated medical image is used to obtain an updated classification result through any of the above image classification methods, and the updated classification result includes the classification result obtained from the feature region image or the feature region other than the second feature region.

[0033] Thirdly, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the image classification method described above.

[0034] Fourthly, a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image classification method described above.

[0035] The aforementioned image classification system, method, and computer device obtain classification results by extracting features from the input medical image through a first classification network to obtain a first feature vector of an abnormal region in the medical image. Then, a first feature region detection network extracts the location information of the first feature vector to obtain the first location information of the abnormal region in the medical image. A first feature region acquisition unit obtains a first feature region image corresponding to the first location information in the medical image. Finally, an output network classifies the first feature region image and outputs the classification result. In this application, the first location information of the abnormal region in the medical image is obtained through a first classification network and a first feature detection network, and the first feature region image is obtained based on this first location information. This ensures that the first feature region image is automatically obtained through a neural network, avoiding the need for manual annotation to obtain the first location information and then template image registration to obtain the first feature region image. This improves the accuracy of the obtained first feature region image and, consequently, the accuracy of the classification result obtained from the first feature region image. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of an image classification system in one embodiment;

[0037] Figure 2 This is a schematic diagram of the structure of an image classification system in another embodiment;

[0038] Figure 3 This is a schematic diagram of the structure of an image classification system in another embodiment;

[0039] Figure 4 This is a schematic diagram of the structure of an image classification system in another embodiment;

[0040] Figure 5 This is a flowchart illustrating an image classification method in one embodiment;

[0041] Figure 6 This is a flowchart illustrating the image classification method in another embodiment;

[0042] Figure 7 This is an internal structural diagram of a computer device in one embodiment.

[0043] Explanation of reference numerals in the attached figures

[0044] 100. First classification network; 200. First feature region detection network;

[0045] 300, First feature region acquisition unit; 400, Output network;

[0046] 101. Fully connected layer; 401. Second classification network;

[0047] 402. Second feature region detection network; 403. Second feature region acquisition unit;

[0048] 404, Third Classification Network; 500, Image Processing Unit. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0050] Figure 1 This is a schematic diagram of an image classification system in one embodiment. The image classification system includes: a first classification network 100, used to extract features from an input medical image to obtain a first feature vector of an abnormal region in the medical image; and a first feature region detection network 200, used to extract location information from the first feature vector to obtain first location information of the abnormal region in the medical image. First feature region acquisition unit 300 The network is used to obtain a first feature region image in a medical image corresponding to the first location information based on the first location information; the output network 400 is used to classify the first feature region image and output the classification result.

[0051] Specifically, the first classification network 100 is used to classify the input medical images. During the classification process, feature extraction is performed to obtain the first feature vector of the abnormal region in the medical image. When the medical image is input into the first classification network 100, the medical image can be preprocessed. Taking the medical image as an MRI image, and the abnormal region in the medical image corresponding to the lesion region of AD as an example, a brain structure MRI image is acquired through a magnetic resonance scanning device. This MRI image can be a T1-weighted image that facilitates observation of anatomical structures, or a T2-weighted image that better displays tissue lesions. This embodiment of the application does not limit this. The above MRI image is labeled, such as AD, mild cognitive impairment (MCI), and normal sample (NC). Then, a series of operations are performed on the MRI image, including rotation, resampling, resizing, skull removal, image non-uniformity correction, histogram matching, and grayscale normalization, so that the image size is 256×256×256mm. 3 This ensures that the MRI image is oriented in a standard Cartesian LPI coordinate system, with a grayscale range of (-1, 1). Optionally, the medical image size can also be 48×48×48mm. 364×64×64mm 3 128×128×128mm 3 The first classification network 100 can be any of a CNN network, a ResNet residual network, or a DenseNet three-dimensional densely connected network incorporating a feature weighting module SEBlock. Taking the first classification network 100 as a CNN network as an example, this CNN network can be designed based on the DenseNet network. In a traditional convolutional neural network, if the number of network layers is L, there will be L connections. However, in DenseNet, when the number of network layers is L, there will be L(L+1) / 2 network connections, meaning that the input of each layer consists of the outputs of all previous layers. The number of feature maps output by the convolutional layers in each module of the entire network is very small (less than 100), unlike the large width that appears in traditional convolutional neural networks. The main part of DenseNet consists of several densely weighted modules (Denseblocks). Each Dense Block's 3×3×3 convolution is preceded by a 1×1×1 convolution operation, which can compress the number of input feature maps. Its output is used as the input of the 3×3×3 convolution. Based on the DenseNet network structure design described above, the output of each layer is connected to the outputs of all preceding layers via channels to serve as the input for the next layer. Therefore, the number of output channels for each DenseBlock is enormous. To reduce memory usage and simultaneously fuse the features of each output channel, a 1×1×1 convolutional operation, called a Transition layer, is placed between every two DenseBlocks. A Dilation Block composed of dilated convolutional modules is added after each DenseBlock to expand the receptive field of the convolutional kernel. A compression-activation module is added after the 3×3×3 convolution of the DenseBlock to obtain the weights of different channels of the feature map. At the same time, a bypass composed of residual attention modules is added to each DenseBlock to obtain the weights of different voxels of the feature map.

[0052] Based on this, during the pre-classification of medical images using the first classification network 100, a first feature vector is extracted. This first feature vector includes the first location information of abnormal regions in the medical image. At this point, the first feature region detection network 200 can extract the location information from the first feature vector to obtain the first location information of abnormal regions in the medical image. Specifically, the first feature region detection network 200... Classification process using a weighted classifier It outputs the first location information of the abnormal region. The first feature region detection network 200 can be composed of a multi-layer fully connected neural network. For example, First The feature region detection network 200 can be composed of a four-layer fully connected neural network. The number of input channels in the first layer of this network is the same as that in the second layer. The first fully connected layer of a classifier network 100 has the same number of input channels, and the number of input channels in each subsequent layer is compressed to the number of channels in the previous layer. 1 / 2, the fourth layer, i.e., the output layer of the first feature region detection network 200, can consist of 6 output channels, namely anomaly... The coordinates of the region's center point (xc, yc, zc) and the radii of the anomaly region along the three axes (Rx, Ry, Rz) are used to obtain the first position. Setting information.

[0053] Based on the obtained first location information, the first feature region acquisition unit 300 can obtain the first feature region image corresponding to the first location information in the medical image. The first feature region acquisition unit 300 can obtain the first feature region image corresponding to the first location information by using a mask function that masks non-abnormal regions in the medical image. The mask function ensures that the abnormal region acquisition operation is differentiable, thus allowing the first feature region detection network 200 to be optimized using the gradient backpropagation method of a neural network. Optionally, the following function can be used as the mask function:

[0054] M(x,y,z,x c ,y c ,z c ,R x ,R y ,R z =[h(xl x(RAS) )]·[h(yl y(RAS) )]·[h(zl z(RAS) )],in:

[0055] l x(LPI) =x c -R x , l y(LPI) =y c -R y , l z(LPI) =z c -R z , l x(RAS) =x c +R x , l y(RAS) =y c +R y , l z(RAS) =z c +R z , x, y, z are the coordinates of each voxel in the masked image.

[0056] When the first feature region image is obtained through the first feature region acquisition unit 300, it can be classified through the output network 400 to output a classification result. For example, the output network 400 can be a CNN network, which classifies the first feature region image and outputs a classification result. The output network 400 can also include a set of networks: a classification network, a feature region detection network, a feature region acquisition unit, and a classification network. Through this set of networks, feature information is further extracted to obtain a more convergent feature region image as the first feature region image. This first feature region image is then input into the classification network for classification processing to obtain a classification result. This embodiment of the application does not limit this. The classification result is used to indicate abnormal regions in medical images.

[0057] The image classification system described above obtains a classification result by extracting features from the input medical image using a first classification network to obtain a first feature vector of an abnormal region in the medical image. Then, a first feature region detection network extracts the location information of the first feature vector to obtain the first location information of the abnormal region in the medical image. A first feature region acquisition unit then obtains a first feature region image corresponding to the first location information in the medical image. Finally, an output network classifies the first feature region image and outputs the classification result. In this embodiment, the first location information of the abnormal region in the medical image is obtained through the first classification network and the first feature detection network, and the first feature region image is obtained based on this first location information. This ensures that the first feature region image is automatically obtained through a neural network, avoiding the need for manual annotation to obtain the first location information and then template image registration to obtain the first feature region image. This improves the accuracy of the obtained first feature region image and, consequently, the accuracy of the classification result obtained from the first feature region image.

[0058] Figure 2 This is a schematic diagram of the image classification system in another embodiment. The output network 400 includes: a second classification network 401, used to extract features from the first feature region image to obtain a second feature vector of an abnormal region in the first feature region image; a second feature region detection network 402, used to extract position information from the second feature vector to obtain second position information of the abnormal region in the first feature region image; a second feature region acquisition unit 403, used to obtain a second feature region image in the first feature region image corresponding to the second position information based on the second position information; and a third classification network 404, used to classify the second feature region image to obtain a classification result.

[0059] Specifically, the output network 400 includes a second classification network 401, which can be any of a CNN network, a ResNet residual network, or a DenseNet three-dimensional densely connected network combined with a feature weighting module SEBlock, used to extract features from the first feature region image to obtain a second feature vector of the abnormal region in the first feature region image. Then, the second feature region detection network 402 extracts the location information of the second feature vector to obtain the second location information of the abnormal region in the first feature region image. Since the second location information is the location information extracted from the second feature vector, and the second feature vector is obtained by extracting features from the abnormal region in the first feature region image, while the first feature region image is obtained by extracting features from the abnormal region in the medical image, which is obtained by removing some non-abnormal regions from the medical image, the second location information is the location information of the abnormal region further extracted after removing some non-abnormal regions from the medical image. Therefore, the second location information is more accurate than the first location information. Based on this, the second feature region acquisition unit 403 can obtain the second feature region image corresponding to the second location information in the first feature region image according to the second location information. For example, the second feature region acquisition unit 403 can obtain the second feature region map corresponding to the second position information through a mask function. The masking function includes a variable parameter k, which is used to control the masking function. form Based on the above embodiments, when k is a large positive number, for example, when k is 10, The mask function can be approximated Treating it as a step function can yield better masking results. .Can The mask function and the input image of the first classification network 100 are compared. Multiply to obtain the masked image, which retains the voxel values ​​of abnormal regions while setting the voxel values ​​of other regions to 0, i.e., the above. Second feature region image The second feature region image is then input into the third classification network 404 to perform classification processing on the second feature region image and obtain the classification result. The third classification network 404 can be any of a CNN network, a ResNet residual network, or a DenseNet three-dimensional densely connected network incorporating a feature weighting module SEBlock; this embodiment of the application does not impose any limitations on this.

[0060] The output network of the above image classification system includes: a second classification network for extracting features from the first feature region image to obtain a second feature vector of an abnormal region in the first feature region image; a second feature region detection network for extracting positional information from the second feature vector to obtain second positional information of the abnormal region in the first feature region image; a second feature region acquisition unit for obtaining a second feature region image in the first feature region image corresponding to the second positional information based on the second positional information; and a third classification network for classifying the second feature region image to obtain a classification result. This ensures that the classification result obtained through the output network is a further feature extraction of the abnormal region in the first feature region image to obtain more accurate second positional information of the abnormal region, and a classification result obtained by classifying the second feature region image corresponding to the more accurate second positional information, thereby further improving the accuracy of the classification result.

[0061] Figure 3 This is a schematic diagram of the structure of an image classification system in another embodiment. The image classification system further includes: an image processing unit 500, used to perform feature removal processing on the regions corresponding to a first feature region image or a second feature region image in a medical image to obtain an updated medical image; the updated medical image is used to obtain an updated classification result through the image classification system, the updated classification result including classification results obtained from feature regions other than the first feature region image or the second feature region image.

[0062] Specifically, based on the above embodiments, the image classification system further includes: an image processing unit 500, used to perform feature removal processing on the region corresponding to the first feature region image or the second feature region image in the medical image to obtain an updated medical image, and to obtain a revised classification result through the image classification system using the updated medical image, wherein the updated classification result includes classification results obtained from feature regions other than the first feature region image or the second feature region image. Taking feature removal processing on the second feature region image as an example, the image processing unit 500 can perform Gaussian blur processing on the region corresponding to the second location information in the medical image according to the above-mentioned second location information to eliminate the discriminative ability of the region and obtain an updated medical image. The updated medical image is then input into the first classification network 100 to extract features from the updated medical image, obtaining the first feature vector of the next abnormal region in the medical image. The first feature region detection network 200 then extracts the location information of the first feature vector of the next abnormal region, obtaining the first location information of the next abnormal region. The first feature region acquisition unit 300 then obtains the next first feature region image corresponding to the first location information of the next abnormal region based on the first location information of the next abnormal region. The output network 400 then performs classification processing on the next first feature region image, outputting the next classification result. This classification result is obtained from feature regions other than the second feature region. Furthermore, the image processing unit 500 can perform multiple iterations to sequentially obtain multiple updated medical images. Each updated medical image is processed by the first classification network 100, the first feature region detection network 200, the first feature region acquisition unit 300, and the output network 400 to obtain the corresponding classification result. The classification results of each updated medical image can then be fused to obtain the overall classification result of the medical image.

[0063] The aforementioned image classification system further includes an image processing unit for performing feature removal processing on the region corresponding to the second feature region image in the medical image to obtain an updated medical image. The updated medical image is used to obtain an updated classification result through the image classification system. The updated classification result includes the classification result obtained from the feature region outside the first feature region image or the second feature region. This allows the image classification system to obtain the classification result of the next abnormal region after obtaining the classification result of an abnormal region by performing feature removal processing on the abnormal region. This enables the image classification system to obtain classification results for multiple abnormal regions, thereby making the classification results obtained by the image classification system more comprehensive.

[0064] Figure 4This is a schematic diagram of the structure of an image classification system in another embodiment. Optionally, the first classification network 100 mentioned above includes a fully connected layer 101; the first feature vector of the abnormal region in the medical image is obtained from the fully connected layer 101.

[0065] Specifically, the first classification network 100 may include a fully connected layer 101, which is used to extract features of abnormal regions in medical images. The fully connected layer 101 can extract features of abnormal regions in medical images during the process of the first classification network classifying medical images, thereby obtaining the first feature vector corresponding to the abnormal regions in medical images.

[0066] Optionally, the training of the first classification network 100, the second classification network 401, the third classification network 402, the first feature region detection network 200, and the second feature region detection network 402 is performed jointly.

[0067] Specifically, the first classification network 100, the second classification network 401, the third classification network 402, the first feature region detection network 200, and the second feature region detection network 402 can be jointly trained using a loss function. Based on the above embodiments, the network structure of the image classification system of this application can be seen to be divided into three levels, and its loss function can be described by the following formula:

[0068]

[0069] in:

[0070] L rank (p t (s) ,p t (s+1) )=max{0,p t (s) -p t (s+1) +margin}

[0071] In the formula, X represents the input training image; s represents each layer of the automatic diagnostic network; L class Y represents the classification loss function in the automatic disease diagnosis task; (s) Y represents the category prediction probability vector; * L represents the true class vector; rank p represents the sorting loss function; t (s) This represents the predicted probability of the diagnostic network for the true class t of the image X; margin is a pre-defined non-negative constant.

[0072] The loss function L(x) can be divided into two parts: the classification loss function Lclass and the ranking loss function L rank .

[0073] Classification loss function L class This is used to improve classification accuracy, specifically to enhance the classification performance of the first classification network 100, the second classification network 401, and the third classification network 404, maximizing their probability of predicting the correct label. A common objective function used in conventional classification tasks (such as cross-entropy) can be used as the loss function L. class .

[0074] Sorting loss function L rank The sorting loss function L is used to improve the accuracy of anomaly region location. rank When the p of the later layers of the network t p is greater than that of the adjacent preceding layers. t At that time, L rank The margin parameter is relatively small; that is, the training objective of the first feature region detection network 200 and the second feature region detection network 402 is to make the second classification network 401 (after the first feature region detection network 200) and the third classification network 402 (after the second feature region detection network 402) more accurate. The margin parameter is a settable positive number, ranging from 0 to 1. The larger the value, the more the system will optimize towards a trend where the diagnostic performance of later layers is significantly better than that of adjacent earlier layers.

[0075] In the specific training process, an alternating training strategy can be adopted. For example, first, the training parameters of the first feature region detection network 200 and the second feature region detection network 402 are fixed, and then the classification loss function L is used. class Train the first classification network 100, the second classification network 401, and the third classification network 404 until L rank Once the function converges, the training parameters of the first classification network 100, the second classification network 401, and the third classification network 404 will be fixed, and the ranking loss function L will be used. rank Train the first feature region detection network 200 and the second feature region detection network 402 until L class The function converges; repeat the above training process until the total loss function L(x) converges.

[0076] Before training begins, the first classification network 100, the second classification network 401, the third classification network 402, the first feature region detection network 200, and the second feature region detection network 402 can be pre-trained respectively. The pre-trained parameters can be used as initialization parameters to jointly train the first classification network 100, the second classification network 401, the third classification network 402, the first feature region detection network 200, and the second feature region detection network 402.

[0077] Figure 5 This is a flowchart illustrating an image classification method in one embodiment. This embodiment describes the specific process of automatically obtaining classification results using a neural network. Figure 5 As shown, the method includes the following steps:

[0078] S101. Input the medical image into the first classification network to extract abnormal region features and obtain the first feature vector of the abnormal region in the medical image.

[0079] S102. Input the first feature vector into the first feature region detection network to extract the location information and obtain the first location information of the abnormal region in the medical image.

[0080] S103. Obtain the image of the first feature region in the medical image that corresponds to the first location information.

[0081] S104. Classify the image of the first feature region to obtain the classification result.

[0082] The image classification method provided in this embodiment is similar in principle and technical effect to the system embodiment described above, and will not be repeated here.

[0083] Figure 6 This is a flowchart illustrating an image classification method in one embodiment. This embodiment describes the specific process of obtaining a classification result from a first feature region image. Figure 6 As shown, one possible implementation method of S104 "classifying the image of the first feature region to obtain the classification result" includes the following steps:

[0084] S201. Input the first feature region image into the second classification network to extract abnormal region features and obtain the second feature vector of the abnormal region in the first feature region image.

[0085] S202. Input the second feature vector into the second feature region detection network to extract location information and obtain the second location information of the abnormal region in the first feature region image.

[0086] S203. Obtain the second feature region image in the first feature region image that corresponds to the second location information.

[0087] S204. Classify the image of the second feature region to obtain the classification result.

[0088] The image classification method provided in this embodiment is similar in principle and technical effect to the system embodiment described above, and will not be repeated here.

[0089] Optionally, the regions corresponding to the first or second feature region images in the medical image are subjected to feature removal processing to obtain an updated medical image. The updated medical image is used for, for example,... Figure 5 or Figure 6 The image classification method shown obtains updated classification results, which include classification results obtained from feature regions other than the first feature region image or the second feature region.

[0090] The image classification method provided in this embodiment is similar in principle and technical effect to the system embodiment described above, and will not be repeated here.

[0091] It should be understood that, although Figure 5 or Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Furthermore, Figure 5 or Figure 6 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0092] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an image classification method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0093] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0094] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0095] The medical image is input into the first classification network to extract abnormal region features, and the first feature vector of the abnormal region in the medical image is obtained.

[0096] The first feature vector is input into the first feature region detection network to extract location information, thereby obtaining the first location information of the abnormal region in the medical image.

[0097] Obtain the image of the first feature region in the medical image that corresponds to the first location information;

[0098] The image of the first feature region is classified to obtain the classification result.

[0099] In one embodiment, when the processor executes the computer program, it further performs the following steps: inputting the first feature region image into a second classification network to extract abnormal region features, thereby obtaining a second feature vector of the abnormal region in the first feature region image; inputting the second feature vector into a second feature region detection network to extract location information, thereby obtaining second location information of the abnormal region in the first feature region image; obtaining a second feature region image in the first feature region image corresponding to the second location information; and performing classification processing on the second feature region image to obtain the classification result.

[0100] In one embodiment, when the processor executes the computer program, it further performs the following steps: performing feature removal processing on the region corresponding to the first feature region or the second feature region image in the medical image to obtain an updated medical image; the updated medical image is used for... Figure 5 or Figure 6 The image classification method shown obtains updated classification results, which include classification results obtained from feature regions other than the first feature region image or the second feature region.

[0101] The terminal device provided in this embodiment is similar in principle and technical effect to the method embodiment described above, and will not be repeated here.

[0102] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0103] The medical image is input into the first classification network to extract abnormal region features, and the first feature vector of the abnormal region in the medical image is obtained.

[0104] The first feature vector is input into the first feature region detection network to extract location information, thereby obtaining the first location information of the abnormal region in the medical image.

[0105] Obtain the image of the first feature region in the medical image that corresponds to the first location information;

[0106] The image of the first feature region is classified to obtain the classification result.

[0107] In one embodiment, when a computer program is executed by a processor, it performs the following steps: inputting the first feature region image into a second classification network to extract abnormal region features, thereby obtaining a second feature vector of the abnormal region in the first feature region image; inputting the second feature vector into a second feature region detection network to extract location information, thereby obtaining second location information of the abnormal region in the first feature region image; obtaining a second feature region image in the first feature region image corresponding to the second location information; and performing classification processing on the second feature region image to obtain the classification result.

[0108] In one embodiment, when a computer program is executed by a processor, it performs the following steps: performing feature removal processing on the region corresponding to the first feature region or the second feature region image in the medical image to obtain an updated medical image; the updated medical image is used for... Figure 5 or Figure 6 The image classification method shown obtains updated classification results, which include classification results obtained from feature regions other than the first feature region image or the second feature region.

[0109] The computer-readable storage medium provided in this embodiment is similar in principle and technical effect to the method embodiment described above, and will not be repeated here.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An image classification system, characterized in that, The system includes: The first classification network is used to classify the input medical image and extract features during the classification process to obtain the first feature vector of the abnormal region in the medical image. A first feature region detection network is used to extract positional information from the first feature vector to obtain the coordinates of the center point of the abnormal region in the medical image along three axes, and the radius of the abnormal region along the three axes. The first feature region detection network includes a multi-layer fully connected neural network. The number of input channels in the first layer of the first feature region detection network is the same as the number of channels in the first fully connected layer of the first classification network. The number of input channels in each subsequent layer is compressed to half of the previous layer. The output layer of the neural network of the first feature region detection network includes 6 output channels, which are the coordinates of the center point of the abnormal region along the three axes and the radius of the abnormal region along the three axes, respectively. The first feature region acquisition unit is used to obtain the first feature region image in the medical image based on the coordinates of the center point of the abnormal region in three axes and the radius of the abnormal region in three axes. The output network, which includes a classification network, a feature region detection network, a feature region acquisition unit, and a set of other networks, is used to obtain a feature region image with more convergent features as the first feature region image. This first feature region image is then input into the classification network for classification processing, and the classification result is output. An image processing unit is used to perform Gaussian blur processing on the region corresponding to the first feature region in the medical image to eliminate the discriminative ability of the region and obtain an updated medical image; the updated medical image is used to obtain the classification result of the next abnormal region in the medical image through the image classification system.

2. The system according to claim 1, characterized in that, The output network includes: The second classification network is used to extract features from the first feature region image to obtain the second feature vector of the abnormal region in the first feature region image. The second feature region detection network is used to extract the location information of the second feature vector to obtain the second location information of the abnormal region in the first feature region image; The second feature region acquisition unit is used to obtain a second feature region image in the first feature region image that corresponds to the second location information based on the second location information. A third classification network is used to classify the image of the second feature region to obtain the classification result.

3. The system according to claim 2, characterized in that, The system also includes: An image processing unit is used to perform feature removal processing on the region corresponding to the second feature region image to obtain an updated medical image; the updated medical image is used to obtain an updated classification result through the image classification system, and the updated classification result includes the classification result obtained from feature regions other than the second feature region.

4. The system according to claim 2, characterized in that, The training of the first classification network, the second classification network, the third classification network, the first feature region detection network, and the second feature region detection network is carried out jointly.

5. The system according to claim 2, characterized in that, The training process for the first classification network, the second classification network, the third classification network, the first feature region detection network, and the second feature region detection network includes: With the training parameters of the first and second feature region detection networks fixed, the first, second, and third classification networks are trained using the classification loss function until the ranking loss function converges; with the training parameters of the first, second, and third classification networks fixed, the first and second feature region detection networks are trained using the ranking loss function until the classification loss function converges; the above training process is repeated until the total loss function determined based on the classification loss function and the ranking loss function converges.

6. The system according to claim 1, characterized in that, The first feature region acquisition unit is used to obtain the coordinates of the center point of the abnormal region in three axes and the first feature region image corresponding to the radius of the abnormal region in the three axes by using a mask function that masks the non-abnormal region in the medical image; the mask function includes variable parameters, which are used to control the shape of the mask function.

7. The system according to claim 6, characterized in that, The variable parameter takes the value of a positive number, and the first feature region acquisition unit is further configured to multiply the mask function and the medical image to obtain the masked image, and determine the first feature region image based on the voxel values ​​in the masked image; wherein, the voxel values ​​of the remaining regions in the masked image other than the abnormal region are set to 0.

8. An image classification method, characterized in that, The method includes: The medical image is input into the first classification network for classification. During the classification process, abnormal region features are extracted to obtain the first feature vector of the abnormal region in the medical image. The first feature vector is input into a first feature region detection network for location information extraction to obtain the coordinates of the center point of the abnormal region in the medical image along three axes, and the radius of the abnormal region along three axes. The first feature region detection network includes a multi-layer fully connected neural network. The number of input channels in the first layer of the first feature region detection network is the same as the number of channels in the first fully connected layer of the first classification network. The number of input channels in each subsequent layer is compressed to half of the previous layer. The output layer of the neural network of the first feature region detection network includes 6 output channels, which are the coordinates of the center point of the abnormal region along three axes and the radius of the abnormal region along three axes, respectively. Obtain the coordinates of the center point of the abnormal region in the medical image along three axes, and the first feature region image corresponding to the radius of the abnormal region along the three axes; The first feature region image is classified by the output network to obtain a classification result. The output network includes a classification network, a feature region detection network, a feature region acquisition unit, and a set of networks of the classification network. It is used to obtain a feature region image with more convergent features as the first feature region image. This first feature region image is input into the classification network for classification processing and the classification result is output. The region corresponding to the first feature region in the medical image is subjected to Gaussian blurring to eliminate the discriminative power of the region, resulting in an updated medical image; the updated medical image is used to obtain the classification result of the next abnormal region in the medical image through the image classification system.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 8.

Images