Brain CT image classification method and device

Abstract

The invention provides a brain CT image classification method and device, and relates to the technical field of medical image processing. The method comprises the steps that brain CT images are collected to construct a data set, the data set is preprocessed, and superpixel images of various scales are obtained; semantic information of each superpixel area in the superpixel image is extracted, wherein the semantic information comprises representation information and geometric information; respectively calculating attention weights of the representation information and the geometric information, fusing the attention weights of the two kinds of semantic information, and calculating a mixed attention feature; performing mixed attention feature calculation step by step from coarse granularity to fine granularity by using a hierarchical fusion structure to realize feature fusion from coarse granularity to fine granularity; and inputting a brain CT image for classification. Compared with a traditional deep learning algorithm, the method can more accurately classify diseases existing in the brain CT image, and the effectiveness of the method is verified in experiments.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a brain CT image classification method and device. Background Art

[0002] Brain diseases are common diseases that threaten human life and health. Early detection and timely treatment of brain diseases are crucial. Brain CT is an essential examination in actual clinical diagnosis, but with the massive growth of patients and brain CT images, reading medical images has become a huge burden for radiologists. Therefore, using multi-label classification of brain CT images has become an urgent task for computer-assisted doctors to diagnose diseases.

[0003] With the development of deep learning and image processing, convolutional neural networks (CNN) and their derivatives such as ResNet and Inception have been widely used in image classification tasks. In the field of natural image processing, these CNN-based methods can show good performance in extracting image features and predicting labels. However, compared with natural images, brain CT image processing has the following challenges: 1) Brain CT images contain more noise information, which will cause great interference to the model's recognition of lesion information; 2) The lesion area in the brain CT image usually does not have a stable shape and position, which makes it difficult for the model to accurately diagnose the disease. In traditional methods, since the features extracted by CNN map the corresponding grids of the same shape in the image, and the grids cannot completely segment the lesion area, this increases the difficulty of accurately identifying lesion information for the CNN-based brain CT image classification model.

[0004] Studies have shown that superpixels used for image segmentation tasks are composed of a series of pixels with similar features. It aggregates pixels with similar features into connected pixel blocks to reduce noise and reduce dimensions. Superpixels can retain local details of the original image and highlight local features, which is conducive to the extraction and expression of high-level features of the image. The number of superpixels is much smaller than the number of pixels in the original image. Using superpixels instead of pixels as the primitives of image processing can greatly reduce the computational complexity of subsequent image processing and improve the efficiency of image processing algorithms. Superpixels of different division scales have different degrees of noise reduction capabilities and feature richness: coarse-grained superpixel maps have fewer feature values, which have stronger noise reduction capabilities and can effectively divide more obvious lesion areas in the image; fine-grained superpixel maps have more feature values, so they have richer image information, but insufficient noise reduction capabilities. Therefore, how to achieve superpixel feature fusion from coarse-grained to fine-grained to better express image information is of great significance for brain CT image classification. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a brain CT image classification method and device, which utilizes superpixel features of multiple division scales to guide the model to focus on the lesion area from coarse granularity to fine granularity, so as to extract more discriminative lesion information and improve the accuracy of lesion information identification.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In one aspect, a method for classifying brain CT images is provided, the method comprising the following steps:

[0008] S1. Acquire brain CT images to construct a data set, preprocess the data set, and obtain superpixel images of multiple scales;

[0009] S2, extracting semantic information of each superpixel region in the superpixel map, wherein the semantic information includes representation information and geometric information;

[0010] S3, respectively calculating the attention weights of the representation information and the geometric information, fusing the attention weights of the two semantic information, and calculating the mixed attention feature;

[0011] S4, using the hierarchical fusion structure to gradually calculate the hybrid attention features from coarse-grained to fine-grained, and realize the feature fusion from coarse-grained to fine-grained;

[0012] S5. Input brain CT images for classification.

[0013] Optionally, the step S1 specifically includes:

[0014] S11, constructing a data set: collecting brain CT images to construct a data set, each piece of data includes a brain CT image and a classification label vector;

[0015] S12, dividing the data set: randomly dividing the data in the data set into a training set, a validation set and a test set; wherein the training set is used to learn the parameters of the neural network; the validation set is used to determine the hyperparameters of the neural network; and the test set is used to verify the classification effect of the neural network;

[0016] S13, obtaining superpixel images of various scales: Based on the super-hierarchical segmentation algorithm, for a given brain CT image I and a set superpixel segmentation scale t, t∈{scale 1 ,scale 2 ,...,scale S}Where S represents the number of superpixels set to generate a superpixel map at segmentation scale t , the calculation process is as follows:

[0017]

[0018] in It represents the super-pixel map obtained when the segmentation scale of image I is t, and it represents the super-pixel division method provided by the super-level segmentation algorithm.

[0019] Optionally, in step S2, extracting the representation information of the superpixel region specifically includes:

[0020] Use the ResNet model to extract the superpixel map obtained in step S1 Visual features ;

[0021] , the specific formula is as follows:

[0022]

[0023] The visual features of the superpixel image Mapping to superpixel graph At the position corresponding to each superpixel area in , the specific formula is as follows:

[0024]

[0025] in, represents the i-th superpixel region in the superpixel map with segmentation scale t of image I, Represents the visual features of the superpixel map Mapped to the i-th superpixel region The mapping result of

[0026] Aggregate the visual features of the same superpixel area and take the average:

[0027]

[0028] Among them, j represents the jth superpixel feature point in the superpixel area, N i represents the number of feature points in the i-th superpixel region, stack(·) represents the aggregated features of t regions superimposed to obtain the representation information of image I when the segmentation scale is t. .

[0029] Optionally, in step S2, extracting geometric information of a superpixel region specifically includes:

[0030] Divide different superpixel areas and calculate the centroid and radius of each superpixel area. The specific formula is as follows:

[0031]

[0032]

[0033] in, represents the centroid of the ith superpixel region of the superpixel map of image I at segmentation scale t, x j and j are the horizontal and vertical coordinates of the j-th pixel area, represents the radius of the corresponding superpixel area, count(·) represents the number of calculated feature points;

[0034] The extracted geometric information is expressed as:

[0035]

[0036] in, are the abscissa and ordinate of the centroid of the ith superpixel region in the superpixel map of image I at segmentation scale t.

[0037] Optionally, the step S3 specifically includes:

[0038] S31. Calculate the attention weight of the representation information: Take the input feature X as the key K and value V, and extract the representation information As the query Q, the attention weight representing the information is calculated using the following formula:

[0039]

[0040] Among them, ω A is the attention weight representing the information;

[0041] S32. Calculate the attention weight of geometric information: For the extracted geometric information , and use it as input to calculate the geometric association vector λ(m,n) between every two superpixel regions in each superpixel map. The specific calculation formula is as follows:

[0042]

[0043] Where m and m represent the mth and nth superpixel regions respectively, and x m and x n Represents the horizontal coordinates of the centroids of the two superpixel regions, y m and n Represents the ordinates of the centroids of the two superpixel regions, r m and r n Respectively represent the radius of the two superpixel regions;

[0044] The attention weight of geometric information is calculated using the following formula:

[0045]

[0046] Among them, ω G is the attention weight of geometric information, W G is the parameter to be learned, and ReLU(·) is the activation function;

[0047] S33, fusion of the attention weights of two semantic information: The attention weights of the representation information and geometric information are fused by the following formula:

[0048]

[0049] Among them, ω AG is the attention weight that integrates representation information and geometric information;

[0050] S34. Calculate the mixed attention feature: take the input feature X as the value V and ω AG Calculate the weighted sum to get the hybrid attention feature Attn that integrates the two semantic information. The specific formula is as follows:

[0051] .

[0052] Optionally, the step S4 specifically includes:

[0053] Design a hierarchical fusion structure. For each layer of the fusion structure, the specific calculation formula is as follows:

[0054]

[0055] Among them, X i and denote the input and output of the i-th layer respectively, MMA(·) denotes the hybrid multi-head attention module that implements the algorithm of step S3, SIE(·) denotes the semantic information extraction algorithm of step S2, Represents the superpixel feature map of image I at segmentation scale t.

[0056] Optionally, the step S5 specifically includes:

[0057] After completing steps S1-S4, the brain CT image dataset is input into the model, and the brain CT images are classified according to the prediction results output by the model.

[0058] In another aspect, a brain CT image classification device is provided, for implementing any of the above methods, the device comprising:

[0059] A data acquisition module, used to collect brain CT images to construct a data set, pre-process the data set, and obtain super-pixel images of various scales;

[0060] An information extraction module, used to extract semantic information of each superpixel region in the superpixel map, wherein the semantic information includes representation information and geometric information;

[0061] A calculation module, used to calculate the attention weights of the representation information and the geometric information respectively, fuse the attention weights of the two semantic information, and calculate the mixed attention feature;

[0062] A fusion module is used to gradually calculate the hybrid attention features from coarse-grained to fine-grained using a hierarchical fusion structure to achieve feature fusion from coarse-grained to fine-grained;

[0063] The classification module is used to input brain CT images for classification.

[0064] In another aspect, an electronic device is provided, the electronic device comprising:

[0065] processor;

[0066] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned brain CT image classification method are implemented.

[0067] On the other hand, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the steps of the above-mentioned brain CT image classification method.

[0068] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0069] The present invention can effectively reduce the noise of brain CT images and divide the lesion area; geometric information can effectively help the model extract the geometric association between superpixel areas; the layered feature fusion model can integrate the advantages of superpixels at different segmentation scales, and guide the model to focus on the detailed information of the lesion area from coarse granularity to fine granularity. Therefore, the method of the present invention is reasonable and reliable, and can provide powerful help for the classification of brain CT images. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 is a flow chart of a brain CT image classification method provided by an embodiment of the present invention;

[0072] Figure 2It is a visualization display diagram of the original image and super-pixel images of different division scales provided by an embodiment of the present invention;

[0073] Figure 3 is a framework diagram of a brain CT image classification method provided by an embodiment of the present invention;

[0074] Figure 4 is a schematic diagram of semantic information extraction provided by an embodiment of the present invention;

[0075] Figure 5 This is a comparative schematic diagram of the division of CT images of cerebral hemorrhage using traditional grids and superpixels;

[0076] Figure 6 is a structural schematic diagram of a brain CT image classification device provided by an embodiment of the present invention;

[0077] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the word "exemplary" is used to present concepts in a specific way.

[0080] The embodiment of the present invention provides a brain CT image classification method, which can be implemented by an electronic device, and the electronic device can be a terminal or a server. Figure 1 As shown, the processing flow of the method may include the following steps:

[0081] S1. Collect brain CT images to construct a data set, pre-process the data set, and obtain super-pixel images of various scales. This step specifically includes:

[0082] S11. Constructing a data set: Collect brain CT images to construct a data set. Each piece of data contains a brain CT image and a classification label vector. , where T represents the number of disease categories collected.

[0083] In the embodiment of the present invention, based on the CQ500 data set, brain CT images are collected, and 451 cases of brain CT data are actually obtained, with a total of 22,773 brain CT images, and then invalid images are removed to construct a new data set. The real label information of each brain CT image contains 14 diagnostic categories of cerebral hemorrhage-related diseases: intracranial hemorrhage, intracerebral hemorrhage, ventricular hemorrhage, subdural hemorrhage, epidural hemorrhage, subarachnoid hemorrhage, left cerebral hemorrhage, right cerebral hemorrhage, chronic hemorrhage, fracture, skull fracture, other fractures, midline deviation, and mass effect.

[0084] First, the labels of the brain CT image dataset are processed. The original dataset is annotated by multiple radiology experts. Multiple experts may have different annotations for the same disease. Therefore, in the case of inconsistent annotations, the choices of the majority of experts that meet the threshold conditions are selected as the true labels.

[0085] Then, according to the determined label information, the data set was divided into 204 confirmed cases and 247 unconfirmed cases. 742 corresponding lesion slice images were selected from the confirmed cases according to the category labels, and 1045 slice images of the same physiological parts as the confirmed cases were selected from the unconfirmed cases. A total of 1787 brain CT images were used as the data set. .

[0086] Each data contains the RGB matrix I generated by its brain CT image, , and 14 brain disease diagnosis classification label vectors , T = 14. The elements of the classification label vector Indicates that the brain CT image has a brain disease corresponding to the i-th label, It is normal.

[0087] S12, dividing the data set: randomly dividing the data in the data set into a training set, a validation set and a test set in a ratio of 8:1:1. The training set is used to learn the parameters of the neural network; the validation set is used to determine the hyperparameters of the neural network; and the test set is used to verify the classification effect of the neural network.

[0088] S13, obtaining superpixel images of various scales: Based on the super hierarchy segmentation algorithm (SH), for a given brain CT image I and a set superpixel segmentation scale t, t∈{scale 1 ,scale 2 ,...,scale S}, where S represents the number of superpixels set to generate a superpixel map at segmentation scale t , the calculation process is as follows:

[0089]

[0090] in It represents the super-pixel map obtained when the segmentation scale of image I is t, and it represents the super-pixel division method provided by the super-level segmentation algorithm.

[0091] In the embodiment of the present invention, the superpixel images with segmentation scales of 5, 10, 15 and 20 are selected, that is, the segmentation scale t∈{scale 5 ,scale 10 ,scale 15 ,scale 20}, and its visualization results are as follows Figure 2 shown.

[0092] S2, extracting semantic information of each superpixel region in the superpixel map. Figure 3 As shown in (a), the semantic information extracted by the semantic information extraction (Semantic Information Extraction, SIE) algorithm proposed in the present invention includes representation information and geometric information of the superpixel area.

[0093] Specifically, Figure 4 As shown, for superpixel images of all scales, the present invention proposes an Appearance Information Extraction (AIE) algorithm to extract the representation information of the superpixel region, specifically including:

[0094] First, the ResNet model is used to extract the superpixel map obtained in step S1 Visual features ;

[0095] , the specific formula is as follows:

[0096]

[0097] In the embodiment of the present invention, .

[0098] Secondly, the visual features of the superpixel image Mapping to superpixel graph At the position corresponding to each superpixel area in , the specific formula is as follows:

[0099]

[0100] in, represents the i-th superpixel region in the superpixel map with segmentation scale t of image I, Represents the visual features of the superpixel map Mapped to the i-th superpixel region The mapping result.

[0101] Finally, the visual features of the same superpixel area are aggregated and averaged:

[0102]

[0103] Among them, j represents the jth superpixel feature point in the superpixel area, N i represents the number of feature points in the i-th superpixel region, stack(·) represents the aggregated features of t regions superimposed to obtain the representation information of image I when the segmentation scale is t. , .

[0104] The representation information extracts the visual information of the superpixel region itself, but ignores the spatial correlation between superpixel regions. Therefore, the present invention proposes a geometry information extraction (GIE) algorithm, which will extract information such as the relative position and size of the superpixel region, and further explore the geometric relationship between superpixel regions. The geometric information includes the centroid coordinates and radius of the superpixel region, which are used to represent the relative position and size of the superpixel region, respectively.

[0105] Extracting the geometric information of the superpixel area specifically includes:

[0106] First, different superpixel regions are divided from the complete superpixel feature map.

[0107] Then, the centroid and radius of each superpixel region are calculated. The specific formula is as follows:

[0108]

[0109]

[0110] in, represents the centroid of the ith superpixel region of the superpixel map of image I at segmentation scale t, x j and j are the horizontal and vertical coordinates of the j-th pixel area, Indicates the radius of the corresponding superpixel area, and count(·) indicates the number of calculated feature points. Since the sizes of different superpixel areas in the superpixel feature map of the same brain CT image vary greatly, in the embodiment of the present invention, the square root of the number of feature points is taken as the radius to indicate the size of the superpixel area.

[0111] Finally, the extracted geometric information is expressed as:

[0112]

[0113] in, are the abscissa and ordinate of the centroid of the ith superpixel region in the superpixel map of image I at segmentation scale t.

[0114] S3. Calculate the attention weights of the representation information and the geometric information respectively, fuse the attention weights of the two semantic information, and calculate the mixed attention feature.

[0115] In the embodiment of the present invention, a mixed multi-head attention (MMA) module is proposed to realize the fusion of super pixel region representation information and geometric information. Figure 3 As shown in (b). First, the attention weights of the geometric information and representation information of the superpixel region are calculated respectively; then, the two are fused and the attention features are calculated with the input features to fuse the two semantic information into the image features. The specific steps are as follows:

[0116] S31. Calculate the attention weight of the representation information: Take the input feature X as the key K and value V, and extract the representation information As the query Q, the attention weight representing the information is calculated using the following formula:

[0117]

[0118] Among them, ω A is the attention weight representing the information, , which will be used to fuse with the attention weight of geometric information.

[0119] S32. Calculate the attention weight of geometric information: For the extracted geometric information , and use it as input to calculate the geometric association vector λ(m,n) between every two superpixel regions in each superpixel map. The specific calculation formula is as follows:

[0120]

[0121] Where m and m represent the mth and nth superpixel regions respectively, and x m and x n Represents the horizontal coordinates of the centroids of the two superpixel regions, y m and n Represents the ordinates of the centroids of the two superpixel regions, r m and r n Represent the radii of the two superpixel regions respectively.

[0122] After calculating the geometric association vectors between all superpixel regions using the above formula, the attention weight of the geometric information is calculated using the following formula:

[0123]

[0124] Among them, ω G is the attention weight of geometric information, , W G is the parameter to be learned, ReLU(·) is the activation function, and λ is the geometric association vector λ(m,n).

[0125] S33, fusion of the attention weights of two semantic information: The attention weights of the representation information and geometric information are fused by the following formula:

[0126]

[0127] Among them, ω AG is the attention weight that fuses representation information and geometric information.

[0128] S34. Calculate the mixed attention feature: get the attention weight ω that combines the two semantic information AG After that, the input feature X is used as the value V, and ω AG Calculate the weighted sum to get the hybrid attention feature Attn that integrates the two semantic information. The specific formula is as follows:

[0129] .

[0130] S4. Use the hierarchical fusion structure to gradually calculate the mixed attention features from coarse-grained to fine-grained, and realize feature fusion from coarse-grained to fine-grained.

[0131] Multi-scale superpixel images have more low-level image features, which enables them to reduce noise and divide effective lesion areas. Superpixel images of different division scales have different degrees of noise reduction capabilities and feature richness: coarse-grained superpixel images have fewer eigenvalues, which have stronger noise reduction capabilities and can effectively divide more obvious lesion areas in the image; fine-grained superpixel images have more eigenvalues, so they have richer image information, but insufficient noise reduction capabilities. Therefore, the present invention designs a hierarchical fusion structure (HFS), such as Figure 3 As shown in (c), the mixed attention feature calculation is performed step by step from coarse-grained to fine-grained, so that the model can effectively reduce noise, focus on the lesion area and extract more disease information inside the lesion area.

[0132] For each layer of fusion structure, the specific calculation formula is as follows:

[0133]

[0134] Among them, X i and denote the input and output of the i-th layer respectively, MMA(·) denotes the hybrid multi-head attention module that implements the algorithm of step S3, SIE(·) denotes the semantic information extraction algorithm of step S2, Represents the superpixel feature map of image I at segmentation scale t.

[0135] S5. Input brain CT images for classification.

[0136] After completing the above steps S1-S4, the brain CT image dataset is input into the model, and the brain CT images are classified according to the prediction results output by the model.

[0137] Figure 5 (a) and (b) are comparative schematic diagrams of the division of CT images of cerebral hemorrhage disease using traditional grids and superpixels, respectively. It can be seen that by adopting the brain CT image classification method based on multi-scale superpixel hierarchical attention described in the present invention, the accuracy of lesion information recognition can be significantly improved.

[0138] In order to further verify the effectiveness of the method described in the present invention, the proposed method was experimented on the brain CT medical classification dataset and compared with the traditional method. In the specific implementation process, the present invention is compared with the mainstream methods ResNet-34, ResNet-101, Inception-v4 and Inception-ResNet-v2 methods applied to natural image classification tasks. The experiment adopts the currently widely used evaluation indicators: Accuracy (ACC), Sensitivity (SEN), Specificity (SPE), Positive Prediction Value (PPV), Negative Predictive Value (NPV), and comprehensive evaluation index (F1-score). The results are shown in Table 1:

[0139] Table 1 Comparative experiment between the method proposed by the present invention and the traditional method

[0140]

[0141] It can be seen from Table 1 that the performance of the method proposed in the present invention is higher than that of the traditional method in almost all evaluation indicators, so the effectiveness of the method proposed in the present invention in extracting disease information from brain CT images can be proved. It can be noted that the performance of the method proposed in the present invention is slightly lower than that of ResNet-34 in the SPE indicator. At the same time, more advanced models such as ResNet-101, Inception-v4 and Inception-ResNet-v2 relative to ResNet-34 also perform poorly under the SPE indicator. After analysis, the reasons for the slightly lower SPE evaluation index are as follows: 1) The labels in the data set used in the present invention are unevenly distributed, including the imbalance in the number of positive and negative labels and the imbalance in the number of different disease labels; 2) Due to the data bias in 1), the disease labels are more often predicted as negative during the model training process, because this will bring higher scores to the model. Therefore, it can be considered that the method proposed in the present invention has a stronger feature expression ability for brain CT images in the case of comprehensive consideration of the experimental results in Table 1 and the above two reasons.

[0142] At the same time, the present invention conducts ablation experiments on the proposed different modules. Table 2 shows the performance of the model when the present invention uses different segmentation scales, and Table 3 shows the influence of the geometric information of the superpixel region proposed and used by the present invention on the model performance.

[0143] Table 2 Ablation experiments of the proposed method at different segmentation scales

[0144]

[0145] Table 3 Ablation experiment of geometric features of the method proposed in this invention

[0146]

[0147] In Table 2, (a), (b), (c) and (d) show the performance comparison of the models at a single segmentation scale, (e) and (f) show the performance comparison of the models at multiple segmentation scales of 5, 10 and 5, 10, 15, and Ours shows the comparison of the models at multiple segmentation scales of 5, 10, 15, and 20. It can be seen that the method proposed in the present invention has the best performance when the segmentation scale is 5, 10, 15, and 20. In Table 3, (a) shows the performance of the model when the model does not use geometric information at multiple segmentation scales, and Ours shows the performance of the model when the geometric information is used. It can be seen that the geometric information proposed in the present invention can significantly improve the classification performance of the model, which means that the proposed method can effectively extract the geometric association between superpixel regions and help the model better extract lesion information in brain CT images.

[0148] In summary, the present invention proposes to use superpixel images of multiple segmentation scales to make the model focus on the lesion information of brain CT images from coarse granularity to fine granularity. Experimental results show that the superpixels used in the present invention can effectively achieve denoising of brain CT images and division of lesion areas; geometric information can effectively help the model extract geometric associations between superpixel areas; the layered feature fusion model can integrate the advantages of superpixels at different segmentation scales, and guide the model to focus on the detailed information of the lesion area from coarse granularity to fine granularity. Therefore, the method described in the present invention is reasonable and reliable, and can provide powerful assistance for the classification of brain CT images.

[0149] Accordingly, an embodiment of the present invention further provides a brain CT image classification device, such as Figure 6 As shown, the device comprises:

[0150] The data acquisition module 201 is used to collect brain CT images to construct a data set, pre-process the data set, and obtain super-pixel images of various scales;

[0151] An information extraction module 202 is used to extract semantic information of each superpixel region in the superpixel map, wherein the semantic information includes representation information and geometric information;

[0152] A calculation module 203, used to calculate the attention weights of the representation information and the geometric information respectively, fuse the attention weights of the two semantic information, and calculate the mixed attention feature;

[0153] A fusion module 204 is used to gradually calculate the hybrid attention features from coarse granularity to fine granularity using a hierarchical fusion structure to achieve feature fusion from coarse granularity to fine granularity;

[0154] The classification module 205 is used to input the brain CT image for classification.

[0155] For ease of explanation, Figure 6 Only the main components of the device are shown. The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0156] In an exemplary embodiment, the present invention further provides an electronic device, the electronic device comprising:

[0157] processor;

[0158] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned brain CT image classification method are implemented.

[0159] Figure 7is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the electronic device 300 may include a processor 3001 and a memory 3002. Optionally, the electronic device 300 may further include a transceiver 3003. The processor 3001, the memory 3002 and the transceiver 3003 may be connected, for example, via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the processor 3001, implement the steps of the above-mentioned brain CT image classification method.

[0160] In a specific implementation, as an embodiment, the processor 3001 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 are shown in FIG.

[0161] In a specific implementation, as an embodiment, the electronic device 300 may also include multiple processors, such as Figure 7 3001 and processor 3004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0162] The memory 3002 is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor 3001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0163] The transceiver 3003 is used to communicate with a network device or a terminal device.

[0164] Optionally, the transceiver 3003 may include a receiver and a transmitter, wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0165] Optionally, the transceiver 3003 may be integrated with the processor 3001 , or may exist independently and be coupled to the processor 3001 via an interface circuit of the electronic device 300 , which is not specifically limited in the embodiment of the present invention.

[0166] It should be noted that Figure 7 The structure of the electronic device 300 shown in the figure does not constitute a limitation on the electronic device. The actual electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects of the electronic device 300 can refer to the technical effects of the above-mentioned method embodiment, which will not be repeated here.

[0167] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is loaded and executed by a processor to implement the steps of the brain CT image classification method as described above. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0169] References in the specification to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments", etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the relevant art to implement such feature, structure, or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0170] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0171] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0172] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0173] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0176] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0177] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A brain CT image classification method, characterized in that: The following steps are involved: S1. Acquire brain CT images to construct a data set, preprocess the data set, and obtain superpixel images of multiple scales; S2, extracting semantic information of each superpixel region in the superpixel map, wherein the semantic information includes representation information and geometric information; S3, respectively calculating the attention weights of the representation information and the geometric information, fusing the attention weights of the two semantic information, and calculating the mixed attention feature; S4, using the hierarchical fusion structure to gradually calculate the hybrid attention features from coarse-grained to fine-grained, and realize the feature fusion from coarse-grained to fine-grained; S5. Input brain CT images for classification.

2. The brain CT image classification method according to claim 1, characterized in that: The step S1 specifically includes: S11, constructing a data set: collecting brain CT images to construct a data set, each piece of data includes a brain CT image and a classification label vector; S12, dividing the data set: randomly dividing the data in the data set into a training set, a validation set and a test set; wherein the training set is used to learn the parameters of the neural network; the validation set is used to determine the hyperparameters of the neural network; and the test set is used to verify the classification effect of the neural network; S13, obtaining superpixel images of various scales: Based on the super-hierarchical segmentation algorithm, for a given brain CT image I and a set superpixel segmentation scale t, t∈{scale 1 ,scale 2 ,...,scale S }Where S represents the number of superpixels set, generating a superpixel map at segmentation scale t , the calculation process is as follows: ； in It represents the super-pixel map obtained when the segmentation scale of image I is t, and it represents the super-pixel division method provided by the super-level segmentation algorithm.

3. The brain CT image classification method according to claim 1, characterized in that: In step S2, extracting the representation information of the superpixel region specifically includes: Use the ResNet model to extract the superpixel map obtained in step S1 Visual features ; , the specific formula is as follows: ； The visual features of the superpixel image Mapping to superpixel graph At the position corresponding to each superpixel area in , the specific formula is as follows: ； in, represents the i-th superpixel region in the superpixel map with segmentation scale t of image I, Represents the visual features of the superpixel map Mapped to the i-th superpixel region The mapping result of Aggregate the visual features of the same superpixel area and take the average: ； Among them, j represents the jth superpixel feature point in the superpixel area, N i represents the number of feature points in the i-th superpixel region, stack(·) represents the aggregated features of t regions superimposed to obtain the representation information of image I when the segmentation scale is t. .

4. The brain CT image classification method according to claim 1, characterized in that: In step S2, extracting geometric information of the superpixel region specifically includes: Divide different superpixel areas and calculate the centroid and radius of each superpixel area. The specific formula is as follows: ； ； in, represents the centroid of the ith superpixel region of the superpixel map of image I at segmentation scale t, x j and j are the horizontal and vertical coordinates of the j-th pixel area, represents the radius of the corresponding superpixel area, count(·) represents the number of calculated feature points; The extracted geometric information is expressed as: ； in, are the abscissa and ordinate of the centroid of the ith superpixel region in the superpixel map of image I at segmentation scale t.

5. The brain CT image classification method according to claim 1, characterized in that: The step S3 specifically includes: S31. Calculate the attention weight of the representation information: Take the input feature X as the key K and value V, and extract the representation information As the query Q, the attention weight representing the information is calculated using the following formula: ； Among them, ω A is the attention weight representing the information; S32. Calculate the attention weight of geometric information: For the extracted geometric information , and use it as input to calculate the geometric association vector λ(m,n) between every two superpixel regions in each superpixel map. The specific calculation formula is as follows: ； Where m and m represent the mth and nth superpixel regions respectively, and x m and x n Represents the horizontal coordinates of the centroids of the two superpixel regions, y m and n Represents the ordinates of the centroids of the two superpixel regions, r m and r n Respectively represent the radius of the two superpixel regions; The attention weight of geometric information is calculated using the following formula: ； Among them, ω G is the attention weight of geometric information, W G is the parameter to be learned, and ReLU(·) is the activation function; S33, fusion of the attention weights of two semantic information: The attention weights of the representation information and geometric information are fused by the following formula: ； Among them, ω AG is the attention weight that integrates representation information and geometric information; S34. Calculate the mixed attention feature: take the input feature X as the value V and ω AG Calculate the weighted sum to get the hybrid attention feature Attn that integrates the two semantic information. The specific formula is as follows: 。 6. The brain CT image classification method according to claim 1, characterized in that: The step S4 specifically includes: Design a hierarchical fusion structure. For each layer of the fusion structure, the specific calculation formula is as follows: ； Among them, X i and denote the input and output of the i-th layer respectively, MMA(·) denotes the hybrid multi-head attention module that implements the algorithm of step S3, SIE(·) denotes the semantic information extraction algorithm of step S2, Represents the superpixel feature map of image I at segmentation scale t.

7. The brain CT image classification method according to claim 1, characterized in that: The step S5 specifically includes: After completing steps S1-S4, the brain CT image dataset is input into the model, and the brain CT images are classified according to the prediction results output by the model.

8. A brain CT image classification device, the device being used to implement the method according to any one of claims 1 to 7, characterized in that: The device comprises: A data acquisition module, used to collect brain CT images to construct a data set, pre-process the data set, and obtain super-pixel images of various scales; An information extraction module, used to extract semantic information of each superpixel region in the superpixel map, wherein the semantic information includes representation information and geometric information; A calculation module, used to calculate the attention weights of the representation information and the geometric information respectively, fuse the attention weights of the two semantic information, and calculate the mixed attention feature; A fusion module is used to gradually calculate the hybrid attention features from coarse-grained to fine-grained using a hierarchical fusion structure to achieve feature fusion from coarse-grained to fine-grained; The classification module is used to input brain CT images for classification.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images