A method, device and equipment for processing three-dimensional image data

Abstract

The embodiment of the present invention provides a method, device and equipment for processing three-dimensional image data, the method comprising: obtaining three-dimensional image data of a target symptom image; preprocessing the three-dimensional image data to obtain image data to be classified; inputting the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; generating a target focus point distribution map based on the classification result and outputting it. The embodiment of the present invention realizes accurate and effective classification of three-dimensional image data, while having low resource consumption and fast operation speed.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method, device and equipment for processing three-dimensional image data. Background Art

[0002] At present, the domestic artificial intelligence diagnostic model for MCI (Mild Cognitive Impairment) is mainly based on convolutional neural networks, which require a lot of computing resources, take a long time, and cannot grasp the attention relationship between image blocks.

[0003] At present, the artificial diagnosis model for classification diagnosis and recognition of three-dimensional images in China is mainly based on convolutional neural networks, which can classify and diagnose a variety of nervous system diseases and facilitate clinical work, such as:

[0004] Identification of AD and MCI based on structural imaging;

[0005] Identification of demyelinating disease on diffusion-weighted imaging;

[0006] Identification of moyamoya disease using cerebral blood flow imaging (arterial spin labeling (ASL)).

[0007] The computing resources required for the global attention mechanism neural network based on Transformer (feature transformer model) are very time-consuming.

[0008] Although the local attention mechanism such as NesT (nested model) optimizes learning efficiency, the attention analysis scheme for local areas often fails to accurately capture the corresponding features due to differences in the design of fixed local blocks and the lack of effective communication of block information, resulting in reduced learning efficiency. Summary of the invention

[0009] The present invention provides a method, device and apparatus for processing three-dimensional image data. The solution of the present invention realizes differentiated block design and attention communication for different three-dimensional image data characteristics, realizes accurate and effective classification, and has low resource consumption and fast operation speed.

[0010] To solve the above technical problems, the embodiments of the present invention provide the following solutions:

[0011] A method for processing three-dimensional image data, the method comprising:

[0012] acquiring three-dimensional image data of a target symptom image;

[0013] Preprocessing the three-dimensional image data to obtain image data to be classified;

[0014] Inputting the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data;

[0015] According to the classification results, a target focus point distribution map is generated and output.

[0016] Optionally, preprocessing the three-dimensional image data to obtain image data to be classified includes:

[0017] Converting the format of the three-dimensional image data to obtain three-dimensional image data in a target format;

[0018] Performing non-uniform field correction processing on the three-dimensional image data in the target format to obtain corrected three-dimensional image data;

[0019] The corrected three-dimensional image data is registered to a target standard space, and numerical normalization processing is performed on the corrected three-dimensional image data in the target standard space to obtain image data to be classified.

[0020] Optionally, the target symptom classification model is trained by the following process:

[0021] Acquire a training data set of target symptom images; the training data set of target symptom images includes at least one image data to be trained of a target symptom image;

[0022] Performing block processing on the image data to be trained in the training data set of the target symptom image to obtain a fixed block group;

[0023] Performing iterative attention calculation on the fixed block group until an output block group is obtained;

[0024] According to the output block group, classification calculation is performed through a multi-layer perceptron to obtain a classification result.

[0025] Optionally, performing block processing on the image data to be trained in the training data set of the target symptom image to obtain a fixed block group includes:

[0026] Dividing each of the image data to be trained into blocks to obtain a plurality of blocks corresponding to the image data to be trained;

[0027] Performing linear dimension embedding processing on the multiple blocks to obtain multiple one-dimensional vector blocks;

[0028] Acquiring symptom characteristics and evolution characteristics of a target symptom image;

[0029] Determine the block integration plan based on the symptom characteristics and evolution characteristics;

[0030] The plurality of one-dimensional vector blocks are integrated through the block integration scheme to obtain a fixed block group.

[0031] Optionally, performing iterative attention calculation on the fixed block group until an output block group is obtained includes:

[0032] Performing multiple local multi-head attention calculations on the fixed block group, and performing block group change processing on the fixed block group according to a preset change mechanism until a target block group is obtained;

[0033] Aggregation processing is performed on the target block group to obtain an output block group.

[0034] Optionally, the block group change process includes at least one of the following:

[0035] Move a preset distance toward a specific vector;

[0036] Block group position exchange;

[0037] Block group rotation;

[0038] Block group deformation.

[0039] Optionally, the method for processing three-dimensional image data further includes:

[0040] The attention value is extracted through back propagation, and an attention focus distribution map is generated according to the classification result.

[0041] The present invention also provides a three-dimensional image data processing device, the device comprising:

[0042] An acquisition module, used for acquiring three-dimensional image data of a target symptom image;

[0043] A processing module is used to preprocess the three-dimensional image data to obtain image data to be classified; input the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; based on the classification result, a target focus point distribution map is generated and output.

[0044] The present invention also provides a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above method.

[0046] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method.

[0047] The above solution of the present invention includes at least the following beneficial effects:

[0048] The above scheme of the present invention obtains three-dimensional image data of the target symptom image; pre-processes the three-dimensional image data to obtain image data to be classified; inputs the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; generates a target focus point distribution map based on the classification result and outputs it. The scheme of the present invention realizes accurate and effective classification of three-dimensional image data, while having low resource consumption and fast operation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a process of a method for processing three-dimensional image data according to an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of three-dimensional image data of Alzheimer's disease and normal cognition in a specific embodiment provided by the present invention;

[0051] Figure 3 A schematic diagram of three-dimensional image data of demyelination and normal cognition in a specific embodiment provided by the present invention;

[0052] Figure 4 A schematic diagram of three-dimensional image data of moyamoya disease and normal cognition in a specific embodiment provided by the present invention;

[0053] Figure 5 A schematic diagram of a dilated convolution kernel when the dilation rate is 1 in a specific embodiment provided by the present invention;

[0054] Figure 6 A schematic diagram of a dilated convolution kernel when the dilation rate is 2 in a specific embodiment provided by the present invention;

[0055] Figure 7 Schematic diagram of a processing flow in a transform encoder according to an embodiment of the present invention;

[0056] Figure 8 A schematic diagram of a process for processing three-dimensional image data of target symptoms and health status in a specific embodiment provided by the present invention;

[0057] Fig. 9A schematic diagram of a flow chart of preprocessing three-dimensional image data of a target symptom in a specific embodiment provided by the present invention;

[0058] Fig.10 A schematic diagram of dividing the gray matter cortex from the three-dimensional image data block of the target symptom in a specific embodiment provided by the present invention;

[0059] Fig.11 A schematic diagram of dividing white matter from a three-dimensional image data block of a target symptom in a specific embodiment provided by the present invention;

[0060] Fig.12 A schematic diagram of a flow chart of iterative processing of multiple blocks in a specific embodiment provided by the present invention;

[0061] Fig.13 Schematic diagram of the structure of a three-dimensional image data processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0063] like Figure 1 As shown, the present invention provides a method for processing three-dimensional image data, the method comprising:

[0064] Step 11, obtaining three-dimensional image data of the target symptom image;

[0065] Step 12, preprocessing the three-dimensional image data to obtain image data to be classified;

[0066] Step 13, inputting the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data;

[0067] Step 14: Generate a target focus point distribution map based on the classification results and output it.

[0068] In this embodiment, the three-dimensional image data is obtained according to the target symptom image, and the three-dimensional image data is preprocessed to obtain the image data to be classified. The preprocessing here is used to eliminate the adverse effects caused by the singular three-dimensional image data to improve the accuracy and quality of the three-dimensional image data;

[0069] Here, the purpose of preprocessing the three-dimensional image data is explained:

[0070] By preprocessing the three-dimensional image data, the differences caused by different MRI devices, device status, scanning parameters and other equipment conditions can be reduced; the three-dimensional images of individual images can be standardized to improve the internal consistency of the data; data and content can be preprocessed, such as scalp stripping, reducing time layer disturbances, checking image head motion parameters, performing field inhomogeneity correction, etc., while performing image quality checks, maximizing image information density and signal-to-noise ratio, and improving data credibility.

[0071] The image data to be classified is input into the trained target symptom classification model to obtain the classification result, and finally a target attention point distribution map is generated according to the classification result and output, wherein the target attention point distribution map is a visual display of the classification result; thus, accurate and effective classification of three-dimensional image data is achieved, while resource consumption is low and running speed is fast.

[0072] The three-dimensional image data here is suitable for classification diagnosis based on non-time series medical image data, and can be T1-weighted imaging, T2-weighted imaging, T2-weighted Flair (fluid attenuated inversion recovery) imaging based on structural magnetic resonance imaging sMRI (structural magnetic resonance imaging), or cerebral blood flow imaging based on arterial spin labeling (ASL), or susceptibility-weighted imaging, and the present application is not limited thereto; wherein, the non-time series medical image data is preferably images other than BOLD (blood oxygenation level dependent);

[0073] The target symptom image is a certain type of neurological disease with imaging indications and manifestations. For example, in Alzheimer's disease (AD), a certain degree of gray matter volume atrophy, mainly in the hippocampus and temporal lobe, often occurs. The mining and identification of T1-weighted imaging data features can be used to achieve classification diagnosis with healthy people; in cerebrovascular diseases such as moyamoya disease, the abnormal regional cerebral blood flow perfusion level caused by cerebrovascular abnormalities can be used to mine and identify arterial spin labeling (ASL) data features to achieve classification diagnosis with healthy people; in white matter diseases such as myelinosis, abnormal non-water signals appear on T2-weighted images, and the mining and identification of T2-weighted Flair imaging data features can be used to achieve classification diagnosis with healthy people; this application is not limited to this;

[0074] The target symptom image information includes the symptom characteristics and evolution characteristics of the target symptom.

[0075] In an optional embodiment of the present invention, step 12 includes:

[0076] Step 121, converting the format of the three-dimensional image data to obtain three-dimensional image data in a target format;

[0077] Step 122, performing non-uniform field correction processing on the three-dimensional image data in the target format to obtain corrected three-dimensional image data;

[0078] Step 123 , registering the corrected three-dimensional image data to the target standard space, and performing numerical normalization processing on the corrected three-dimensional image data in the target standard space to obtain image data to be classified.

[0079] In this embodiment, the format of the three-dimensional image data is converted. Since the three-dimensional image data is obtained according to the target symptom image, the target symptom image is preferably a medical image. Therefore, the format of the three-dimensional image data before the format conversion is preferably a dicom (Digital Imaging and Communications in Medicine) file format. The format conversion is performed on the three-dimensional image data to obtain the three-dimensional image data in the target format. The target format is preferably a Nifty (Neuroimaging Informatics Technology Initiative) format.

[0080] The three-dimensional image data in the target format is subjected to non-uniform field correction processing. The non-uniform field correction processing here is to improve the imaging quality of the three-dimensional image data in the target format, so that the corrected three-dimensional image data has a better visual effect than the three-dimensional image data in the target format without correction, so as to obtain a better classification result when subsequently input into the trained target symptom classification model;

[0081] The corrected three-dimensional image data is registered to a target standard space, which is preferably an MNI (Montreal Neurological Institute) standard space. The MNI space is a brain imaging coordinate system established based on a series of magnetic resonance images of a normal human brain, thereby ensuring the isotropy of the corrected three-dimensional image data;

[0082] Here, the isotropy of three-dimensional image data is explained:

[0083] Since the pixel spacing of the image in the three scanning directions of x, y and z is inconsistent during the process of acquiring the three-dimensional image data, the three-dimensional image data is defined according to the DICOM format, the x direction corresponds to the left and right direction of the scanned object; the y direction corresponds to the front and back directions of the scanned object; the z direction corresponds to the up and down directions of the scanned object; the pixel spacing in the x and y directions is small, and the resolution is high; while the pixel spacing in the z direction is large; this leads to the anisotropy of the three-dimensional image data in the x, y and z directions; in order to facilitate subsequent processing, in the preprocessing process, it is preferred to adjust the pixel spacing to isotropic through image interpolation, that is, the pixel spacing in the three directions of xyz is consistent.

[0084] Furthermore, the corrected three-dimensional image data in the target standard space is numerically normalized. The normalization processing here is used to limit the corrected three-dimensional image data in the target standard space to a certain range, thereby eliminating the adverse effects caused by singular data, and finally obtaining the image data to be classified; after the above-mentioned preprocessing process, after inputting into the trained target symptom classification model, a more accurate and effective classification result can be obtained.

[0085] In an optional embodiment of the present invention, the target symptom classification model is trained by the following process:

[0086] Step a, obtaining a training data set of target symptom images; the training data set of target symptom images includes at least one image data to be trained of a target symptom image;

[0087] Step b, performing block processing on the image data to be trained in the training data set of the target symptom image to obtain a fixed block group;

[0088] Step c, performing iterative attention calculation on the fixed block group until an output block group is obtained;

[0089] Step d: performing classification calculations through a multi-layer perceptron according to the output block group to obtain a classification result.

[0090] In this embodiment, a training data set is obtained, and block processing is performed on the image data to be trained in the training data set to obtain a fixed block group. The block division processing here includes at least one of block division, embedding processing, and integration processing. Further, attention calculation is performed on the fixed block group, and the attention calculation is repeated according to a preset change mechanism, and the information features are fused until an output block group is obtained. The output block group here is preferably Where E is the embedding dimension, x L1_out is the output block group, (H L1 ,WL1 ,D L1 ) is the image size of the output block group;

[0091] Then, according to the output block group, a classification calculation is performed through a multi-layer perceptron (MLP) to obtain a classification result; a multi-layer perceptron is a forward-structured artificial neural network that maps a set of input vectors to a set of output vectors; after the target block is input into the multi-layer perceptron, at least two classification results can be obtained;

[0092] In addition, it should be noted that the target symptom classification model is trained using a large amount of image data to be trained in the training data set, and the classification results of the target symptom classification model are compared with the correct classification results of the image data to be trained. When the accuracy rate is greater than the preset probability, the target symptom classification model is considered to be trained.

[0093] In an optional embodiment of the present invention, step b comprises:

[0094] Step b1, dividing each of the image data to be trained into blocks to obtain a plurality of blocks corresponding to the image data to be trained;

[0095] Step b2, performing linear dimension embedding processing on the multiple blocks to obtain multiple one-dimensional vector blocks;

[0096] Step b3, obtaining target symptom image information, wherein the target symptom image information includes symptom characteristics and evolution characteristics;

[0097] Step b4, determining a block integration scheme according to the target symptom image information;

[0098] Step b5: integrating the multiple one-dimensional vector blocks using the block integration scheme to obtain a fixed block group.

[0099] In this embodiment, the pre-processed image data to be trained is divided into blocks, divided into a plurality of blocks of preset sizes, and a linear dimension embedding process is performed on the plurality of blocks. Here, the linear dimension embedding process embeds each block into a vector x pe ∈R E , where x pe is the one-dimensional vector block after each block is embedded, and E is the embedding dimension;

[0100] Obtain target symptom image information, where the target symptom image information includes symptom characteristics and evolution characteristics, and determine a block integration scheme according to the target symptom image information. Here, determining the block integration scheme according to the target symptom image information may be a customized block integration scheme according to the symptom characteristics and evolution characteristics, or a block integration scheme calculated according to the symptom characteristics and evolution characteristics, and the present application is not limited thereto; the block integration scheme determined according to the symptom characteristics and evolution characteristics is more sensitive to data details; wherein the symptom characteristics include at least one of the research object, parameter characteristics of the symptom, image characteristics of the symptom, and volume element (Volume Pixel) of the symptom; a voxel is the smallest unit of image data in space;

[0101] Integrate the multiple one-dimensional vector blocks according to the block integration scheme. The integration processing here can be a combination of spatially adjacent block groups (e.g., a combination of all block groups in a 3×3×3 space), or a combination of spatially discrete block groups (e.g., a combination of 27 mutually spaced blocks in a 5×5×5 space), or other customized block group combinations, which are not limited to this application; after the integration processing, the multiple one-dimensional vector blocks are integrated into a fixed block group;

[0102] like Figure 2 As shown, in a specific embodiment, due to the difference in the research objects, according to the symptom characteristics of brain tumors, taking Alzheimer's disease (AD) as an example, Alzheimer's disease patients often suffer from the death of brain neurons due to the neurotoxic factors of the disease course, which is manifested in the image data as a decrease in the thickness of the gray matter of the T1-weighted structural image. In addition, since the gray matter distribution in the neurological image is mainly in the cerebral cortex, a 3×3×1 rectangular block or a 3×3×3 cube block is used, mainly for the gray matter cortical area of ​​the brain, that is, the sulci, gyri and other structures on the surface of the brain, to move the blocks voxel by voxel, which can more accurately capture the structural image change characteristics of Alzheimer's disease (AD) patients, thereby performing classification, identification and diagnosis.

[0103] like Figure 3 As shown, in another specific embodiment, taking demyelinating disease as an example, patients with demyelinating disease often suffer from ischemic necrosis of cerebral nerve fibers under the influence of regional inflammation and vascular factors, which manifests as abnormal signals in the white matter of the brain in the T2-weighted Flair structural image. In addition, since the gray matter on the neuroimaging is mainly distributed in the brain parenchyma, between the cortical gray matter and the subcutaneous gray matter, by using a 3×1×1 rectangular block, a cube with an expansion coefficient of 2, or a radial block, the block is mainly moved voxel by voxel in the white matter area of ​​the brain, which can more accurately capture the structural imaging change characteristics of patients with demyelinating disease, thereby performing classification, identification and diagnosis.

[0104] like Figure 4 As shown, in another specific embodiment, taking moyamoya disease as an example, moyamoya disease is a cerebrovascular disease characterized by chronic progressive stenosis or occlusion of the bilateral internal carotid artery terminals and the anterior cerebral artery and the beginning of the middle cerebral artery, and secondary formation of abnormal vascular network at the skull base, and often manifests ischemia along the gray matter and white matter. In the arterial spin labeling (ASL) data, it is manifested as abnormal perfusion along the vascular supply area. By using ellipsoidal, octahedral, or radial blocks, mainly moving the blocks voxel by voxel along the cerebral artery network, the characteristics of cerebral blood flow perfusion changes in patients with moyamoya disease can be captured more accurately, so as to perform classification, identification and diagnosis.

[0105] It can be seen that according to the differences in the above-mentioned target symptoms, the target symptom image information including symptom characteristics and evolution characteristics are different. Determining the block integration plan can make it easier to capture the characteristics of the symptoms, fully realize the communication between information, and greatly save the settlement amount.

[0106] like Figure 2 and Figure 3 As shown, in a specific embodiment, according to the image characteristics of the symptom, the manifestation of symptom C in the brain is the generation of a tumor in the infratentorial area of ​​the brain. At this time, according to the image characteristics of symptom C, the block integration scheme is determined to locate the peripheral expansion points by expanding the convolution kernel; it should be noted that the expansion convolution kernel here is to expand the convolution kernel to the scale of the expansion scale constraint, and fill the area not occupied by the original convolution kernel with zeros, Figure 2 The expansion rate is d=1, the convolution kernel in the 3×3 space is expanded to the 5×5 space, and the area not occupied by the original convolution kernel is filled with zeros; when the expansion rate is d=2, the convolution kernel in the 3×3 space is expanded to the 7×7 space, and the area not occupied by the original convolution kernel is filled with zeros.

[0107] In an optional embodiment of the present invention, step c comprises:

[0108] Step c1, performing multiple local multi-head attention calculations on the fixed block group, and performing block group change processing on the fixed block group according to a preset change mechanism until a target block group is obtained;

[0109] Step c2, performing aggregation processing on the target block group to obtain an output block group.

[0110] In this embodiment, attention calculation is performed on at least two fixed block groups. The attention calculation is preferably performed by transform encoder to perform local multi-head attention calculation. Optionally, the formula Calculate, where Attention(Q,K,V) is the attention value between any two fixed block groups, Q is the query vector, K is the key value vector, V is the value vector, and d kis the number of blocks;

[0111] Furthermore, the fixed block group is subjected to block group change processing according to the preset change mechanism. In an optional embodiment of the present invention, the block group change processing includes at least one of the following: moving a preset distance to a specific vector; exchanging the position of the block group; rotating the block group; deforming the block group. The exchange of information during the movement is achieved through the change of the block group. The shape of the block group can be a cube, a cuboid, or an irregular shape, and the present application is not limited thereto.

[0112] After each block group change, the attention value of the local multi-head attention calculation of the fixed block group is calculated until the target block group is obtained, and the target block group is aggregated to obtain the output block group;

[0113] Here, the aggregation processing of the target block group includes averaging processing or maximum value processing.

[0114] like Figure 7 As shown, it should be noted that the transform encoder processes the input data as follows:

[0115] Performing primary normalization processing on the input data to obtain a first normalized value;

[0116] Perform a multi-head attention mechanism on the first normalized value to obtain an attention value;

[0117] Perform a first-level XOR operation on the input data and the attention value to obtain a first XOR operation result;

[0118] Performing secondary normalization processing on the result of the first XOR operation as an input value to obtain a second normalized value;

[0119] The second normalized value is input into the multilayer perceptron as an input value, and a secondary XOR operation is performed on the output of the multilayer perceptron and the first XOR operation result to obtain a second XOR operation result.

[0120] In an optional embodiment of the present invention, the method for processing three-dimensional image data further includes:

[0121] Step 15, extracting the attention value through back propagation, and generating an attention point distribution map according to the classification result.

[0122] In this embodiment, in step 14, a target focus distribution map is generated according to the classification result, preferably by using a Grad-CAM (Gradient-weighted Class Activation Mapping) algorithm to generate a classification contribution distribution map of a visualized three-dimensional image;

[0123] Furthermore, based on the attention points, the attention value of each layer is extracted through back propagation, and an attention point distribution map is generated for the trained target symptom classification model for the input data. The attention point distribution map is used to visualize the attention points in the classification results of the three-dimensional image data.

[0124] like Figure 8 As shown, in a specific embodiment, the three-dimensional image data of the target symptoms (NC / MCI (mild cognitive impairment) / AD) of the brain and the three-dimensional image data of the healthy state are used as input, and the three-dimensional image data are preprocessed to obtain multiple block sizes (P H ,P W ,P D ) blocks;

[0125] The preprocessing process is as follows: Fig. 9 As shown, for the standard image in standard space, isotropic, scalp-removed, and space-corrected, according to the characteristics of the onset of the target symptom, the classified image data (three-dimensional image data of the target symptom or health state) x∈R C×H×W×D Perform block division (Split Patch) to obtain multiple blocks of size (P H ,P W ,P D ) blocks (3×3×1 rectangular blocks, mainly moving blocks of gray matter areas along the surface of the cerebral cortex to capture relevant information), each block is Perform linear dimension embedding processing on multiple blocks to obtain multiple one-dimensional vector blocks x pe ∈R E , where C is the number of input channels and E is the embedding dimension;

[0126] Among them, Fig.10 and Fig.11 As shown in Figure 2, when the white matter and gray matter cortex of the brain are divided into blocks, Fig.10 The white dots in the figure are the gray matter cortex. Fig.11 The white dots in the figure are white matter. It can be seen that the three-dimensional image data of the brain can be well divided by block division.

[0127] According to the image information, determine the block integration plan and integrate multiple one-dimensional vector blocks into fixed block groups. Then fix the block group x L1b Input layer Among them, (H L1 ,W L1 ,D L1) is the block size obtained after embedding processing, Each position has E values, B L1 is the number of groups in the first layer, S is the number of blocks contained in the block group, and H L1 ×W L1 ×D L1 =B L1 ×S;

[0128] The fixed block group of the first layer is transformed by the encoder Perform local multi-head attention calculation for a fixed block group x of the input layer L1bi Embed into vectors Q, K, V of equal length and calculate through the attention mechanism After each layer is calculated by the transform encoder for the attention mechanism, the block group is changed according to the preset block group change mechanism to achieve the function of information exchange between adjacent modules. The local multi-head attention calculation is repeated according to the preset block group change mechanism to fully integrate the information features between the fixed block groups until the target block group is obtained. Aggregate the target block group by averaging or taking the maximum value to obtain the output block group Until only one group is left, the group is input into MLP for classification calculation, and the first classification result is NC (Normal cognitive), and the second classification result is the target symptom;

[0129] Furthermore, the classification contribution is visualized by the Grad-CAM algorithm and presented by the target focus distribution map, and the attention focus is extracted by back-propagation to obtain the attention value of each layer, which is presented by the attention focus distribution map;

[0130] The above process is performed by repeating the block iterative processing n times on multiple blocks;

[0131] The specific process of block iteration here is as follows Fig.12 As shown: the input of the embedding process and the output of the previous embedding process are positionally encoded at the same time. Positional encoding is used to encode the relative position of the block, retain the position information of the block, and obtain the input encoding result and the output encoding result respectively;

[0132] Further, the input encoding result is processed by a multi-head attention mechanism to obtain a first attention result, and the first attention result and the input encoding result are respectively used as the input result and the output result for addition and normalization to obtain a first normalized result;

[0133] Processing the first normalized result through a feedforward network to obtain a first feedforward result;

[0134] The first normalization result and the first feedforward result are respectively used as the input result and the output result for addition and normalization processing to obtain an input processing result;

[0135] The output encoding result is processed by a multi-head attention mechanism to obtain a second attention result, and the second attention result and the input encoding result are respectively added and normalized as the input result and the output result to obtain the output processing result;

[0136] Further, the input processing result and the output processing result are processed by a multi-head attention mechanism to obtain a third attention result, and the third attention result and the output processing result are respectively used as the input result and the output result for addition and normalization processing to obtain a second normalized result;

[0137] Processing the second normalized result through a feedforward network to obtain a second feedforward result;

[0138] The second normalized result and the second feedforward result are respectively used as the input result and the output result for addition and normalization processing to obtain a third normalized result;

[0139] The third normalization result is processed linearly and processed by the two-class Softmax function in turn to obtain the output result (finally reaching the three-classification of NC / MCI / AD); MR (magnetic resonance) is used to quickly screen patients with Alzheimer's disease and early mild Alzheimer's disease to assist doctors in clinical diagnosis.

[0140] The embodiment of the present invention obtains three-dimensional image data of a target symptom image; pre-processes the three-dimensional image data to obtain image data to be classified; inputs the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on target symptom image information of the three-dimensional image data; generates a target focus point distribution map based on the classification result and outputs it; and achieves accurate and effective classification of the three-dimensional image data, while having low resource consumption and fast running speed.

[0141] like Fig.13 As shown, the present invention further provides a three-dimensional image data processing device 130, the device 130 comprising:

[0142] An acquisition module 131 is used to acquire three-dimensional image data of a target symptom image;

[0143] The processing module 132 is used to pre-process the three-dimensional image data to obtain image data to be classified; input the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; based on the classification result, generate a target focus point distribution map and output it.

[0144] Optionally, preprocessing the three-dimensional image data to obtain image data to be classified includes:

[0145] Converting the format of the three-dimensional image data to obtain three-dimensional image data in a target format;

[0146] Performing non-uniform field correction processing on the three-dimensional image data in the target format to obtain corrected three-dimensional image data;

[0147] The corrected three-dimensional image data is registered to a target standard space, and numerical normalization processing is performed on the corrected three-dimensional image data in the target standard space to obtain image data to be classified.

[0148] Optionally, the target symptom classification model is trained by the following process:

[0149] Acquire a training data set of target symptom images; the training data set of target symptom images includes at least one image data to be trained of a target symptom image;

[0150] Performing block processing on the image data to be trained in the training data set of the target symptom image to obtain a fixed block group;

[0151] Performing iterative attention calculation on the fixed block group until an output block group is obtained;

[0152] According to the output block group, classification calculation is performed through a multi-layer perceptron to obtain a classification result.

[0153] Optionally, performing block processing on the image data to be trained in the training data set of the target symptom image to obtain a fixed block group includes:

[0154] Dividing each of the image data to be trained into blocks to obtain a plurality of blocks corresponding to the image data to be trained;

[0155] Performing linear dimension embedding processing on the multiple blocks to obtain multiple one-dimensional vector blocks;

[0156] Acquiring symptom characteristics and evolution characteristics of a target symptom image;

[0157] Determine the block integration plan based on the symptom characteristics and evolution characteristics;

[0158] The plurality of one-dimensional vector blocks are integrated through the block integration scheme to obtain a fixed block group.

[0159] Optionally, performing iterative attention calculation on the fixed block group until an output block group is obtained includes:

[0160] Performing multiple local multi-head attention calculations on the fixed block group, and performing block group change processing on the fixed block group according to a preset change mechanism until a target block group is obtained;

[0161] Aggregation processing is performed on the target block group to obtain an output block group.

[0162] Optionally, the block group change process includes at least one of the following:

[0163] Move a preset distance toward a specific vector;

[0164] Block group position exchange;

[0165] Block group rotation;

[0166] Block group deformation.

[0167] Optionally, the processing module 132 further includes:

[0168] The attention value is extracted through back propagation, and an attention focus distribution map is generated according to the classification result.

[0169] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0170] An embodiment of the present invention further provides a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0171] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above method.

[0172] It should be noted that the computing device is a computing device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the computing device and can achieve the same technical effects.

[0173] An embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0174] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0176] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0179] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0180] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0181] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0182] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing three-dimensional image data, characterized in that: The method comprises: Acquire three-dimensional image data of a target symptom image; the target symptom image is three-dimensional image data of mild cognitive impairment or Alzheimer's disease and three-dimensional image data of a healthy state; Preprocessing the three-dimensional image data to obtain image data to be classified; Inputting the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; Extract the attention value through back propagation, generate a target attention point distribution map according to the classification result, and output it; The target symptom classification model is trained through the following process: Acquire a training data set of target symptom images; the training data set of target symptom images includes at least one image data to be trained of a target symptom image; Dividing each image data to be trained in the training data set of the target symptom image into blocks to obtain a plurality of blocks corresponding to the image data to be trained; Performing linear dimension embedding processing on the multiple blocks to obtain multiple one-dimensional vector blocks; Acquiring target symptom image information, wherein the target symptom image information includes symptom characteristics and evolution characteristics; The block integration scheme is calculated based on the symptom characteristics and evolution characteristics; Wherein, the symptom characteristics include at least one of parameter characteristics, image characteristics, and volume elements of the symptoms of the research subject; According to the image characteristics of the symptom, the manifestation of symptom C in the brain is a tumor in the subtentorial area of ​​the brain. At this time, according to the image characteristics of symptom C, the block integration scheme is determined to locate the peripheral expansion points by expanding the convolution kernel; wherein the expansion convolution kernel is to expand the convolution kernel to the scale of the expansion scale constraint, and fill the area not occupied by the original convolution kernel with zeros; the multiple one-dimensional vector blocks are integrated and processed by the block integration scheme to obtain a fixed block group; Performing multiple local multi-head attention calculations on the fixed block group, and performing block group change processing on the fixed block group according to a preset change mechanism until a target block group is obtained; The block group change process includes at least one of the following: Move a preset distance to a specific vector; Block group position exchange; Block group rotation; Block group deformation; Performing aggregation processing on the target block group to obtain an output block group; According to the output block group, classification calculation is performed through a multi-layer perceptron to obtain a classification result.

2. The method for processing three-dimensional image data according to claim 1, characterized in that: Preprocessing the three-dimensional image data to obtain image data to be classified includes: Converting the format of the three-dimensional image data to obtain three-dimensional image data in a target format; Performing non-uniform field correction processing on the three-dimensional image data in the target format to obtain corrected three-dimensional image data; The corrected three-dimensional image data is registered to a target standard space, and numerical normalization processing is performed on the corrected three-dimensional image data in the target standard space to obtain image data to be classified.

3. A three-dimensional image data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring three-dimensional image data of a target symptom image; the target symptom image is three-dimensional image data of mild cognitive impairment or Alzheimer's disease and three-dimensional image data of a healthy state; A processing module is used to pre-process the three-dimensional image data to obtain image data to be classified; input the image data to be classified into a trained target symptom classification model to obtain a classification result; the trained target symptom classification model is trained based on the target symptom image information of the three-dimensional image data; extract the attention value through back propagation, generate a target attention point distribution map based on the classification result, and output it; The target symptom classification model is trained through the following process: Acquire a training data set of target symptom images; the training data set of target symptom images includes at least one image data to be trained of a target symptom image; Dividing each image data to be trained in the training data set of the target symptom image into blocks to obtain a plurality of blocks corresponding to the image data to be trained; Performing linear dimension embedding processing on the multiple blocks to obtain multiple one-dimensional vector blocks; Acquiring target symptom image information, wherein the target symptom image information includes symptom characteristics and evolution characteristics; The block integration scheme is calculated based on the symptom characteristics and evolution characteristics; Wherein, the symptom characteristics include at least one of parameter characteristics, image characteristics, and volume elements of the symptoms of the research subject; According to the image characteristics of the symptom, the manifestation of symptom C in the brain is a tumor in the subtentorial area of ​​the brain. At this time, according to the image characteristics of symptom C, the block integration scheme is determined to locate the peripheral expansion points by expanding the convolution kernel; wherein the expansion convolution kernel is to expand the convolution kernel to the scale of the expansion scale constraint, and fill the area not occupied by the original convolution kernel with zeros; the multiple one-dimensional vector blocks are integrated and processed by the block integration scheme to obtain a fixed block group; Performing multiple local multi-head attention calculations on the fixed block group, and performing block group change processing on the fixed block group according to a preset change mechanism until a target block group is obtained; The block group change process includes at least one of the following: Move a preset distance to a specific vector; Block group position exchange; Block group rotation; Block group deformation; Performing aggregation processing on the target block group to obtain an output block group; According to the output block group, classification calculation is performed through a multi-layer perceptron to obtain a classification result.

4. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 2.

Images