Image processing method, device and computer readable storage medium

Abstract

The present invention discloses an image processing method, device and computer-readable storage medium, wherein the method comprises: obtaining image features corresponding to a plurality of brain images, then determining fusion features corresponding to the plurality of brain images based on the image features and a deep learning model, wherein the fusion features include local features and global features corresponding to the image features, then flattening and linear mapping the fusion features to determine a plurality of vectors of preset lengths, and finally determining target brain age data based on the vectors. The present invention can obtain fusion features including local features and global features corresponding to the image features based on the image features corresponding to the plurality of brain images and a deep learning model, and predicting brain age through fusion features can more accurately predict target brain age data, thereby improving the accuracy of brain age prediction.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device and computer-readable storage medium. Background Art

[0002] The human brain is the most complex physiological system in the human body. As people age, the structure and morphology of the brain will undergo a series of changes, and people will be accompanied by forgetfulness, emotionality and other manifestations, but at the same time, their understanding, judgment and comprehensive analysis abilities will also be enhanced to a certain extent. This structural change can be observed and measured through neuroimaging.

[0003] At present, the commonly used brain age algorithms are mainly divided into two categories. One is the machine learning method, which is based on artificially extracted and constructed features. It obtains effective features through feature extraction, feature clarification, and feature selection, and constructs a regression model to obtain the prediction results. This type of method has strong interpretability, but the accuracy is not high. The extracted features lead to incomplete original data images and partial loss. The other is the deep learning method. Unlike the machine learning method, deep learning does not rely on features, has less image loss, and can extract more age-related features. Among them, the commonly used method is the convolution model, but convolution has certain limitations. Convolution extracts relatively fixed features and does not take into account the importance of different features, resulting in inaccurate brain age prediction.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an image processing method, device and computer-readable storage medium, aiming to solve the technical problem of inaccurate brain age prediction.

[0006] To achieve the above object, the present invention provides an image processing method, which comprises the following steps:

[0007] Obtaining image features corresponding to each brain image;

[0008] Based on the image features and the deep learning model, determining fusion features corresponding to each brain image, wherein the fusion features include local features and global features corresponding to the image features;

[0009] Flattening and linear mapping the fused features to determine a plurality of vectors of preset lengths;

[0010] Based on the vector, target brain age data is determined.

[0011] Furthermore, the step of determining fusion features corresponding to each brain image based on the image features and the deep learning model, wherein the fusion features include local features and global features corresponding to the image features, comprises:

[0012] Inputting the image features into a local feature learning model in a deep learning model for model training to obtain the local features;

[0013] Inputting the image features into a global feature learning model in a deep learning model for model training to obtain the global features;

[0014] Based on the local features and the global features, a fusion feature is determined.

[0015] Furthermore, the step of inputting the image features into a local feature learning model in a deep learning model for model training to obtain the local features includes:

[0016] Determine a sub-local feature corresponding to the image feature based on the image feature and a convolution group in a local feature learning model;

[0017] The local feature is obtained based on the sub-local feature and the maximum pooling layer in the local feature learning model.

[0018] Furthermore, the step of inputting the image features into a global feature learning model in a deep learning model for model training to obtain the global features includes:

[0019] Determine a first sagittal image feature based on the image feature and a displacer module in a first sub-global feature learning model in the global feature learning model;

[0020] Determine a second sagittal image feature based on the first sagittal image feature and a block clipping module in the first sub-global feature learning model;

[0021] Determine a third sagittal image feature based on the second sagittal image feature and the Transformer module in the first sub-global feature learning model;

[0022] Determine a fourth sagittal image feature based on the third sagittal image feature and the block merging module in the first sub-global feature learning model;

[0023] The global feature is obtained based on the fourth sagittal plane image feature.

[0024] Furthermore, the step of obtaining the global feature based on the fourth sagittal plane image feature comprises:

[0025] Determine a first cross-sectional image feature based on the fourth sagittal image feature and a displacer module in a second sub-global feature learning model in the global feature learning model;

[0026] Determine a second cross-sectional image feature based on the first cross-sectional image feature and a block clipping module in the second sub-global feature learning model;

[0027] Determine a third cross-sectional image feature based on the second cross-sectional image feature and the Transformer module in the second sub-global feature learning model;

[0028] Determine a fourth cross-sectional image feature based on the third cross-sectional image feature and the block merging module in the second sub-global feature learning model;

[0029] The global feature is obtained based on the fourth cross-sectional image feature.

[0030] Furthermore, the step of obtaining the global feature based on the cross-sectional image feature comprises:

[0031] Determine a first coronal image feature based on the fourth cross-sectional image feature and a displacer module in a third sub-global feature learning model in the global feature learning model;

[0032] Determine a second coronal plane image feature based on the first coronal plane image feature and a block clipping module in the third sub-global feature learning model;

[0033] Determine a third coronal plane image feature based on the second coronal plane image feature and the Transformer module in the third sub-global feature learning model;

[0034] The global feature is obtained based on the third coronal plane image feature and the block merging module in the third sub-global feature learning model.

[0035] Furthermore, the step of determining target brain age data based on the vector includes:

[0036] Obtaining preset Gaussian distributions corresponding to different ages, and determining expected values ​​of each of the vectors based on the preset Gaussian distributions;

[0037] Based on the expected value, target brain age data is determined.

[0038] Furthermore, the step of obtaining preset Gaussian distributions corresponding to different ages and determining expected values ​​of each of the vectors based on the preset Gaussian distributions includes:

[0039] Based on the preset parameters, determine the preset Gaussian distribution corresponding to different ages;

[0040] Fitting the preset Gaussian distribution based on each of the vectors to determine a target Gaussian distribution corresponding to each of the vectors;

[0041] Based on the target Gaussian distribution, an expected value of each of the vectors is determined.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides an image processing device, which includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, and the image processing program implements the steps of the aforementioned image processing method when executed by the processor.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which an image processing program is stored, and when the image processing program is executed by a processor, the steps of the aforementioned image processing method are implemented.

[0044] The present invention obtains image features corresponding to multiple brain images, and then determines fusion features corresponding to the multiple brain images based on the image features and a deep learning model, wherein the fusion features include local features and global features corresponding to the image features, and then flattens and linearly maps the fusion features to determine multiple vectors of preset lengths, and finally determines target brain age data based on the vectors. According to the image features corresponding to the multiple brain images and a deep learning model, fusion features including local features and global features corresponding to the image features can be obtained. Brain age prediction using fusion features can more accurately predict target brain age data, thereby improving the accuracy of brain age prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a schematic diagram of the structure of an image processing device in a hardware operating environment involved in an embodiment of the present invention;

[0046] Figure 2 FIG. 4 is a flow chart of a first embodiment of an image processing method according to the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0049] like Figure 1 As shown, Figure 1 It is a schematic diagram of the structure of an image processing device in a hardware operating environment involved in an embodiment of the present invention.

[0050] The image processing device of the embodiment of the present invention can be a PC, or it can be a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a portable computer, or other portable image processing device.

[0051] like Figure 1 As shown, the image processing device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Optionally, the image processing device may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the image processing device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of mobile image processing devices, vibration recognition related functions, etc.; of course, the image processing device may also be configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0053] Those skilled in the art will understand that Figure 1 The structure of the image processing device shown in the figure does not constitute a limitation on the image processing device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image processing program.

[0055] exist Figure 1 In the image processing device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the image processing program stored in the memory 1005.

[0056] In this embodiment, the image processing device includes: a memory 1005, a processor 1001, and an image processing program stored in the memory 1005 and executable on the processor 1001, wherein the processor 1001 calls the image processing program stored in the memory 1005 and executes the steps of the image processing method in each of the following embodiments.

[0057] The present invention also provides an image processing method, referring to Figure 2 , Figure 2 FIG. 4 is a flow chart of a first embodiment of an image processing method according to the present invention.

[0058] In this embodiment, the image processing method includes the following steps:

[0059] Step S101, obtaining image features corresponding to each brain image;

[0060] Step S102, determining fusion features corresponding to each brain image based on the image features and the deep learning model, wherein the fusion features include local features and global features corresponding to the image features;

[0061] Step S103, flattening and linear mapping the fused features to determine a plurality of vectors of preset lengths;

[0062] Step S104: determining target brain age data based on the vector.

[0063] In this embodiment, a nuclear magnetic resonance image of the target individual's brain is first acquired, so that a preset number of brain images corresponding to the target individual's brain can be obtained based on the nuclear magnetic resonance image, and a plurality of brain images are obtained. Specifically, after acquiring the nuclear magnetic resonance image of the target individual's brain, data preprocessing is performed on the nuclear magnetic resonance image of the target individual's brain to obtain a target image, and then the target image is subjected to random 3D rotation and random 3D translation, that is, a corresponding 3D image is generated based on the target image, and randomly rotated or translated on the 3D level, and the rotated or translated image is added to the brain image to obtain a preset number of brain images.

[0064] For example, the preset number is 5, and 5 brain images can be determined, wherein 1 of the 5 brain images is the target image itself, and the other 4 are images generated after the target image is subjected to random 3D rotation and random 3D translation. In addition, the steps of data preprocessing include: centerline alignment, that is, aligning the centerline of the brain in the MRI image with the centerline of the image to obtain an aligned MRI image; neck removal, that is, removing the neck part in the aligned MRI image to obtain a MRI image without the neck; scalp removal, that is, removing the scalp part in the MRI image without the neck to obtain a MRI image without the head. The scalp-free MRI image; MNI space standardization, that is, using the MNI standard template, converting the scalp-free MRI image to the MNI space, which is a coordinate system established based on a series of normal human brain MRI images, to obtain the MRI image in the MNI space; bias field correction, that is, correcting the brightness difference on the MRI image in the MNI space caused by the scanner itself and the deviation in the scanning process, to obtain the corrected MRI image; 2mm space standardization, that is, converting the corrected MRI image to the 2mm space to obtain the target image. The MRI images are unified through the process of data preprocessing for subsequent processing. In some other embodiments, the steps of data preprocessing may only include some steps of centerline alignment, neck removal, scalp removal, MNI space standardization, and bias field correction, and may not be limited to the above steps. There are other data preprocessing operations suitable for actual scenarios.

[0065] In this embodiment, when each brain image is acquired, image features corresponding to each brain image can be obtained according to each acquired brain image, wherein each brain image is a medical image, and the dimension of each brain image is 3D.

[0066] The image features are input into the deep learning model, and then the deep learning model performs model training on the image features, wherein the deep learning model includes a local feature learning model and a global feature learning model. Specifically, the image features are input into the deep learning model, and the local feature learning model and the global feature learning model in the deep learning model perform model training on the image features respectively. The local feature learning model obtains local features after training the image feature model, and the global feature learning model obtains global features after training the image feature model. After obtaining the local features and the global features, the local features and the global features are fused to obtain fused features. Specifically, the local features and the global features can be connected by the basic operator concat to obtain the fused features.

[0067] Then, the fused features are flattened and linearly mapped to determine multiple vectors of preset lengths, wherein the number of vectors can be the same as the number of brain images. Specifically, after the fused features are processed by the operator view, the processed results are input into the linear classifier for linear mapping to obtain vectors of preset length, wherein the view can arrange the original data into a row, and then select new data from a row in sequence according to the parameters in the view. After obtaining the fused features, the obtained fused features can be input into the deep learning model multiple times to obtain the final fused features, and the specific number of times the fused features are repeatedly input into the deep learning model can be manually set. Among them, the preset length can be reasonably set, for example, the preset length is 18.

[0068] Finally, the preset Gaussian distribution corresponding to different ages is obtained, and the expected value of each of the vectors is determined according to the preset Gaussian distribution, and then the accurate target brain age data is obtained according to the expected value. However, the present design is not limited to this. In other embodiments, the target brain age data can also be determined directly according to each vector, for example, the target brain age data can be determined according to the mean or median of each vector, that is, the mean of each vector is first calculated, and then the average value corresponding to the mean of each vector is used as the target brain age data, or the median of each vector is first determined, and the average value of each median is used as the target brain age data.

[0069] The image processing method proposed in this embodiment obtains image features corresponding to multiple brain images, and then determines fusion features corresponding to the multiple brain images based on the image features and a deep learning model, wherein the fusion features include local features and global features corresponding to the image features, and then the fusion features are flattened and linearly mapped to determine multiple vectors of preset lengths. Finally, based on the vectors, target brain age data is determined. According to the image features corresponding to the multiple brain images and a deep learning model, fusion features including local features and global features corresponding to the image features can be obtained. Brain age prediction through fusion features can more accurately predict target brain age data, thereby improving the accuracy of brain age prediction.

[0070] Based on the first embodiment, a second embodiment of the image processing method of the present invention is proposed. In this embodiment, step S102 includes:

[0071] Step S201, inputting the image features into a local feature learning model in a deep learning model for model training to obtain the local features;

[0072] Step S202, inputting the image features into a global feature learning model in a deep learning model for model training to obtain the global features;

[0073] Step S203: determining fusion features based on the local features and the global features.

[0074] In this embodiment, the acquired image features are input into the local feature learning model in the deep learning model for model training, wherein the local feature learning adopts the commonly used convolution method, and the translation invariance and locality of convolution can better help extract local features.

[0075] The acquired image features are input into the global feature learning model in the deep learning model for model training to obtain the global features, wherein the global feature learning model contains 3 sub-parts, each of which is composed of a displacer, a block cropper, a transformer and a block merge, and each part realizes data splitting and feature extraction in different axes, namely sagittal plane, transverse section and coronal plane. The displacer realizes the conversion between different axes.

[0076] Finally, based on the obtained local features and global features, the fused features are obtained through the basic operator concat connection.

[0077] It should be noted that before predicting brain age, an initial deep learning model including a local feature learning model and a global feature learning model can be preset, and then training samples can be obtained. The training samples include brain images of multiple target individuals. Each target individual can correspond to an image, and the label of each sample corresponds to the age of the subject. The model is trained by inputting each training sample into the initial deep learning model to obtain the trained deep learning model and the sample fusion features corresponding to each sample, and finally the target age is predicted through distribution learning and expected regression.

[0078] The image processing method proposed in this embodiment obtains the local features by inputting the image features into a local feature learning model in a deep learning model for model training, and then obtains the global features by inputting the image features into a global feature learning model in the deep learning model for model training, and then predicting the target brain age data more accurately by fusing the features based on the local features and the global features, thereby improving the accuracy of brain age prediction.

[0079] Based on the second embodiment, a third embodiment of the image processing method of the present invention is proposed. In this embodiment, step S201 includes:

[0080] Step S301, determining a sub-local feature corresponding to the image feature based on the image feature and a convolution group in a local feature learning model;

[0081] Step S302, obtaining the local feature based on the sub-local feature and the maximum pooling layer in the local feature learning model.

[0082] In this embodiment, the image features are input into the local feature learning model for model training. The local feature learning model includes two convolution groups and two maximum pooling layers. The image features are first processed by the convolution group and then by the maximum pooling layer. The convolution group includes a 3D convolution layer, a batch normalization layer, and a ReLU activation layer, and the convolution kernel size is 3.

[0083] Specifically, the image features are input into the local feature learning model for model training. After a convolution group processing and then a maximum pooling layer processing, the input can be downsampled to half of the original size. After another convolution group processing and a maximum pooling layer processing, the input can be downsampled to a quarter of the original size. For example, the input data is encoded as an image feature of size (24×28×24×4N) and then downsampled to (12×14×12×4N) after the first convolution group and maximum pooling layer processing, and then the image feature downsampled to (12×14×12×4N) is processed by the second convolution group and maximum pooling layer and then downsampled to (6×7×6×4N), where the image feature N of size (24×28×24×4N) is the number of channels of the input image, 4 is the batch size, 24 is the X-axis data, 28 is the Y-axis data, and 24 is the Z-axis data.

[0084] The image processing method proposed in this embodiment determines the sub-local features corresponding to the image features based on the image features and the convolution group in the local feature learning model, and then obtains the local features based on the sub-local features and the maximum pooling layer in the local feature learning model. The local features can be better extracted through the convolution group, thereby improving the accuracy of predicting the target brain age data.

[0085] Based on the second embodiment, a fourth embodiment of the image processing method of the present invention is proposed. In this embodiment, step S202 includes:

[0086] Step S401, determining a first sagittal plane image feature based on the image feature and a displacer module in a first sub-global feature learning model in the global feature learning model;

[0087] Step S402, determining a second sagittal image feature based on the first sagittal image feature and a block clipping module in the first sub-global feature learning model;

[0088] Step S403, determining a third sagittal plane image feature based on the second sagittal plane image feature and the Transformer module in the first sub-global feature learning model;

[0089] Step S404, determining a fourth sagittal plane image feature based on the third sagittal plane image feature and a block merging module in the first sub-global feature learning model;

[0090] Step S405: obtaining the global feature based on the fourth sagittal plane image feature.

[0091] It should be noted that the global feature learning model includes a first sub-global feature learning model, a second sub-global feature learning model, and a third sub-global feature learning model. Each model is composed of a permutator module, a block cropping module, a Transformer module, and a block merging module. When the image features are input into the global feature learning model, they are first trained by the first sub-global feature learning model, then the second sub-global feature learning model is trained, and finally the global features are obtained through model training of the third sub-global feature learning model.

[0092] In this embodiment, when the image feature is input to the first sub-global feature learning model, the permutator module first processes the sagittal plane corresponding to the image feature to obtain the first sagittal image feature, then the block cropping module processes the sagittal image feature to divide the first sagittal image feature into different small blocks, and flattens the features of each small block to obtain multiple second sagittal image features, then the multiple second sagittal image features are input to the standard Transformer module to obtain multiple third sagittal image features, wherein the Transformer module is composed of layer normalization (LN), multi-head self-attention mechanism (MSA), and multi-layer perceptron (MLP), and its input is serialized features, and finally the block merging module recombines and downsamples the multiple third sagittal image features to obtain the fourth sagittal image feature, and the second sub-global feature learning model and the third sub-global feature learning model are trained according to the fourth sagittal image feature to obtain the global feature. Specifically, when the image feature is (N, C, H, W, D), the permutator module processes the image feature to obtain the first sagittal image feature of (N*H, C, W, D), and then the block cropping module processes the first sagittal image feature (N*H, C, W, D) to obtain 64 second sagittal image features (N*H*64, C, W / 32, D / 32), and then the Transformer module processes the 64 second sagittal image features (N*H*64, C, W / 32, D / 32) to obtain 64 third sagittal image features (N*H*64, C, W / 32, D / 32), and finally the block merging module processes the 64 third sagittal image features (N*H*64, C, W / 32, D / 32) to obtain the fourth sagittal image features (N, C, H, W / 4, D / 4).

[0093] The image processing method proposed in this embodiment determines the first sagittal image feature based on the image feature and the displacer module in the first sub-global feature learning model in the global feature learning model, then determines the second sagittal image feature based on the first sagittal image feature and the block cropping module in the first sub-global feature learning model, and then determines the third sagittal image feature based on the second sagittal image feature and the Transformer module in the first sub-global feature learning model, then determines the fourth sagittal image feature based on the third sagittal image feature and the block merging module in the first sub-global feature learning model, and finally obtains the global feature based on the fourth sagittal image feature. The global feature can be better extracted through the fourth sagittal image feature, thereby improving the accuracy of predicting the target brain age data.

[0094] Based on the fourth embodiment, a fifth embodiment of the image processing method of the present invention is proposed. In this embodiment, step S405 includes:

[0095] Step S501, determining a first cross-sectional image feature based on the fourth sagittal image feature and a displacer module in a second sub-global feature learning model in the global feature learning model;

[0096] Step S502, determining a second cross-sectional image feature based on the first cross-sectional image feature and a block clipping module in the second sub-global feature learning model;

[0097] Step S503, determining a third cross-sectional image feature based on the second cross-sectional image feature and the Transformer module in the second sub-global feature learning model;

[0098] Step S504, determining a fourth cross-sectional image feature based on the third cross-sectional image feature and the block merging module in the second sub-global feature learning model;

[0099] Step S505: obtaining the global feature based on the fourth cross-sectional image feature.

[0100] In this embodiment, the fourth sagittal image feature is input into the second sub-global feature learning model. First, the displacer module processes the cross-section corresponding to the fourth sagittal image feature to obtain the first cross-sectional image feature. Then, the block cropping module processes the first cross-sectional image feature to divide the first cross-sectional image feature into different small blocks, and flattens the features of each small block to obtain multiple second cross-sectional image features. Then, the multiple second cross-sectional image features are input into the standard Transformer module to obtain multiple third cross-sectional image features. Finally, the block merging module recombines and downsamples the multiple third cross-sectional image features to obtain the fourth cross-sectional image feature. The third sub-global feature learning model is trained according to the fourth cross-sectional image feature to obtain the global feature.

[0101] Specifically, when the fourth sagittal image feature is (N, C, H, W / 4, D / 4), the permutator module processes the fourth sagittal image feature to obtain the first cross-sectional image feature of (N*W / 4, C, H, D / 4), and the rear block clipping module processes the first cross-sectional image feature (N*W / 4, C, H, D / 4) to obtain 64 second cross-sectional image features (N*W / 4*64, C, H / 8, D / 32), and then Transfo The rmer module processes the 64 second cross-sectional image features (N*W / 4*64, C, H / 8, D / 32) to obtain 64 third cross-sectional image features (N*W / 4*64, C, H / 32, D / 128), and finally the block merging module processes the 64 third cross-sectional image features (N*W / 4*64, C, H / 32, D / 128) to obtain the fourth cross-sectional image features (N, C, H / 4, W / 4, D / 16).

[0102] The image processing method proposed in this embodiment determines the first cross-sectional image feature based on the fourth sagittal image feature and the displacer module in the second sub-global feature learning model in the global feature learning model, then determines the second cross-sectional image feature based on the first cross-sectional image feature and the block cropping module in the second sub-global feature learning model, and then determines the third cross-sectional image feature based on the second cross-sectional image feature and the Transformer module in the second sub-global feature learning model, then determines the fourth cross-sectional image feature based on the third cross-sectional image feature and the block merging module in the second sub-global feature learning model, and finally obtains the global feature based on the fourth cross-sectional image feature. The global feature can be better extracted through the fourth cross-sectional image feature, thereby improving the accuracy of predicting the target brain age data.

[0103] Based on the fifth embodiment, a sixth embodiment of the image processing method of the present invention is proposed. In this embodiment, step S503 includes:

[0104] Step S601, determining a first coronal plane image feature based on the fourth cross-sectional image feature and a displacer module in a third sub-global feature learning model in the global feature learning model;

[0105] Step S602, determining a second coronal plane image feature based on the first coronal plane image feature and a block clipping module in the third sub-global feature learning model;

[0106] Step S603, determining a third coronal plane image feature based on the second coronal plane image feature and the Transformer module in the third sub-global feature learning model;

[0107] Step S604: obtaining the global feature based on the third coronal plane image feature and the block merging module in the third sub-global feature learning model.

[0108] In this embodiment, the fourth cross-sectional image feature is input into the third sub-global feature learning model. First, the permutator module processes the coronal plane corresponding to the fourth cross-sectional image feature to obtain the first coronal plane image feature. Then, the block cropping module processes the first coronal plane image feature to divide the first coronal plane image feature into different small blocks, and flattens the features of each small block to obtain multiple second coronal plane image features. Then, the multiple second coronal plane image features are input into the standard Transformer module to obtain multiple third coronal plane image features. Finally, the block merging module recombines and downsamples the multiple third coronal plane image features to obtain the global feature.

[0109] Specifically, when the fourth cross-sectional image feature is (N, C, H / 4, W / 4, D / 16), the permutator module processes the fourth cross-sectional image feature to obtain a first coronal image feature of (N*D / 16, C, H / 4, W / 4), and the rear block clipping module processes the first coronal image feature (N*D / 16, C, H / 4, W / 4) to obtain 64 second coronal image features (N*D / 16*64, C, H / 32, W / 32), and then Tra The nsformer module processes the 64 second coronal image features (N*D / 16*64, C, H / 32, W / 32) to obtain 64 third coronal image features (N*D / 16*64, C, H / 128, W / 128). Finally, the block merging module processes the 64 third coronal image features (N*D / 16*64, C, H / 128, W / 128) to obtain the global features (N, C, H / 16, W / 16, D / 16).

[0110] The image processing method proposed in this embodiment determines the first coronal plane image feature based on the fourth cross-sectional image feature and the displacer module in the third sub-global feature learning model in the global feature learning model; determines the second coronal plane image feature based on the first coronal plane image feature and the block clipping module in the third sub-global feature learning model; determines the third coronal plane image feature based on the second coronal plane image feature and the Transformer module in the third sub-global feature learning model; obtains the global feature based on the third coronal plane image feature and the block merging module in the third sub-global feature learning model, and the global feature can be better extracted through the third coronal plane image feature, thereby improving the accuracy of predicting the target brain age data.

[0111] Based on the first embodiment, a seventh embodiment of the image processing method of the present invention is proposed. In this embodiment, step S103 includes:

[0112] Step S701, obtaining preset Gaussian distributions corresponding to different ages, and determining expected values ​​of each of the vectors based on the preset Gaussian distributions;

[0113] In this embodiment, different ages are taken as expected values, and preset Gaussian distributions corresponding to different ages are obtained according to preset standard deviations. Each vector is fitted with the preset Gaussian distribution to determine the expected value of each vector.

[0114] Specifically, in the age range of the actual target population, obtain the Gaussian distribution corresponding to different ages. For example, if the age range of the target population covers 12 to 84 years old, set the standard deviation as a hyperparameter, the value of the standard deviation is 2, and each age from 12 to 84 years old is taken as the expected value to generate the Gaussian distribution of each age from 12 to 84 years old as the preset Gaussian distribution. The formula is:

[0115]

[0116] Among them, l k is the expected value, θ is the standard deviation, and the obtained vector of length M is used to fit the preset Gaussian distribution to determine the target Gaussian distribution that the vector obeys, thereby determining the expected value of each vector.

[0117] It should be noted that Gaussian distribution is normal distribution. If the random variable X obeys a mathematical expectation of μ and variance of σ 2 The normal distribution is denoted by N(μ, σ 2 ). Its expected value μ determines its position, and its standard deviation σ determines the amplitude of the distribution. When μ=0, σ=1, the normal distribution is the standard normal distribution.

[0118] Step S702: determining target brain age data based on the expected value.

[0119] In this embodiment, each brain image corresponds to a vector. After the expected value of each vector is obtained through distribution learning, the confidence interval and predicted value corresponding to the age of the target individual are determined according to the expected value of each vector. For example, the expected values ​​of each vector are arranged from small to large, and the range from the minimum value to the maximum value of each expected value is used as the confidence interval. The average value of the expected values ​​excluding the minimum value and the maximum value is obtained, and the average value is the predicted value. For example, for a target individual whose actual age is 41 years old, the expected values ​​of the five vectors corresponding to the five images to be processed corresponding to his brain are obtained, which are 4 2.53 years old, 42.02 years old, 41.67 years old, 40.53 years old, and 40.83 years old. Arrange the 5 expected values ​​from small to large. The range from the minimum value of the 5 expected values ​​to the maximum value of the 5 expected values ​​is the confidence interval. The confidence interval corresponding to the age of the target individual is [40.53 years old, 42.53 years old]. This confidence interval is the range interval of the predicted age of the target individual. Remove the minimum and maximum values, and the average of the remaining 3 expected values ​​is calculated to be 40.9 years old. The predicted value corresponding to the age of the target individual is 40.9 years old.

[0120] The image processing method proposed in this embodiment obtains preset Gaussian distributions corresponding to different ages, determines the expected values ​​of each of the vectors based on the preset Gaussian distributions, and then determines the target brain age data based on the expected values. The expected values ​​are obtained through the preset Gaussian distribution, thereby improving the accuracy of predicting the target brain age data.

[0121] Based on the seventh embodiment, an eighth embodiment of the image processing method of the present invention is proposed. In this embodiment, step S702 includes:

[0122] Step S801, determining preset Gaussian distributions corresponding to different ages based on preset parameters;

[0123] Step S802, fitting the preset Gaussian distribution based on each of the vectors to determine a target Gaussian distribution corresponding to each of the vectors;

[0124] Step S803: determining the expected value of each of the vectors based on the target Gaussian distribution.

[0125] In this embodiment, different ages are taken as expected values, and preset Gaussian distributions corresponding to different ages are generated according to preset standard deviations. The vectors corresponding to each brain image are fitted with the preset Gaussian distributions to determine the expected values ​​of each vector.

[0126] Specifically, in the age range of the actual target population, a Gaussian distribution corresponding to different ages is generated. For example, if the age range of the target population covers 12 to 84 years old, each age from 12 to 84 years old is taken as the expected value, and a Gaussian distribution of each age from 12 to 84 years old is generated. The standard deviation is taken as a hyperparameter, and the value of the standard deviation is set to 2.

[0127] It should be noted that in the process of fitting the Gaussian distribution, the loss function used as the optimization target is the KL divergence, and considering that the fitted target Gaussian distribution may have multiple Gaussians, the normal KL divergence is not sufficient to meet the optimization of the similarity between multiple Gaussians and single Gaussian targets, so a symmetric KL divergence is used.

[0128] In some other embodiments, in order to reduce the consumption of computational complexity, vectors corresponding to different ages are sampled according to the length M of the vector, and the sampling interval △l is defined as 4. The sampling formula is L=(12+Δ1*k|k=0,1,...,M). When M=18, L=(12,16,20,...,84), which represents sampling of ages ranging from 12 to 84 years old. Finally, a vector with a length of 18 is obtained. A preset Gaussian distribution is generated based on the vector, and the vector is fit with the preset Gaussian distribution to determine the target Gaussian distribution that the vector obeys, thereby determining the expected value of the vector.

[0129] The data processing method proposed in this embodiment determines the preset Gaussian distribution corresponding to different ages based on preset parameters, then fits the preset Gaussian distribution based on each of the vectors to determine the target Gaussian distribution corresponding to each of the vectors, and then determines the expected value of each of the vectors based on the target Gaussian distribution. The expected value of the vector is obtained by fitting the preset Gaussian distribution, so that the target brain age data can be accurately predicted by the expected value.

[0130] In addition, an embodiment of the present invention further provides an image processing device, the image processing device comprising: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the steps of the image processing method described above when executed by the processor.

[0131] In addition, an embodiment of the present invention further provides a computer-readable storage medium having an image processing program stored thereon, and when the image processing program is executed by a processor, the steps of the image processing method described above are implemented.

[0132] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0133] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling an image processing device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0135] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, It is characterized in that The image processing method comprises the following steps: Obtaining image features corresponding to each brain image; Based on the image features and the deep learning model, determining fusion features corresponding to each brain image, wherein the fusion features include local features and global features corresponding to the image features; Flattening and linear mapping the fused features to determine a plurality of vectors of preset lengths; Based on the vector, determining target brain age data; The step of determining the fusion features corresponding to each brain image based on the image features and the deep learning model comprises: inputting the image features into a local feature learning model in the deep learning model for model training to obtain the local features; inputting the image features into a global feature learning model in the deep learning model for model training to obtain the global features; and determining the fusion features based on the local features and the global features; The step of inputting the image features into a global feature learning model in a deep learning model for model training to obtain the global features comprises: Determine a first sagittal image feature based on the image feature and the displacer module in the first sub-global feature learning model in the global feature learning model; determine a second sagittal image feature based on the first sagittal image feature and the block clipping module in the first sub-global feature learning model; determine a third sagittal image feature based on the second sagittal image feature and the Transformer module in the first sub-global feature learning model; determine a fourth sagittal image feature based on the third sagittal image feature and the block merging module in the first sub-global feature learning model; and obtain the global feature based on the fourth sagittal image feature; The step of determining target brain age data based on the vector comprises: Obtain preset Gaussian distributions corresponding to different ages, and determine expected values ​​of each of the vectors based on the preset Gaussian distributions; and determine target brain age data based on the expected values.

2. The image processing method according to claim 1, It is characterized in that The step of inputting the image features into a local feature learning model in a deep learning model for model training to obtain the local features comprises: Determine a sub-local feature corresponding to the image feature based on the image feature and a convolution group in a local feature learning model; The local feature is obtained based on the sub-local feature and the maximum pooling layer in the local feature learning model.

3. The image processing method according to claim 1, It is characterized in that The step of obtaining the global feature based on the fourth sagittal plane image feature comprises: Determine a first cross-sectional image feature based on the fourth sagittal image feature and a displacer module in a second sub-global feature learning model in the global feature learning model; Determine a second cross-sectional image feature based on the first cross-sectional image feature and a block clipping module in the second sub-global feature learning model; Determine a third cross-sectional image feature based on the second cross-sectional image feature and the Transformer module in the second sub-global feature learning model; Determine a fourth cross-sectional image feature based on the third cross-sectional image feature and a block merging module in the second sub-global feature learning model; The global feature is obtained based on the fourth cross-sectional image feature.

4. The image processing method according to claim 3, It is characterized in that The step of obtaining the global feature based on the fourth cross-sectional image feature comprises: Determine a first coronal plane image feature based on the fourth cross-sectional image feature and a displacer module in a third sub-global feature learning model in the global feature learning model; Determine a second coronal plane image feature based on the first coronal plane image feature and a block clipping module in the third sub-global feature learning model; Determine a third coronal plane image feature based on the second coronal plane image feature and the Transformer module in the third sub-global feature learning model; The global feature is obtained based on the third coronal plane image feature and the block merging module in the third sub-global feature learning model.

5. The image processing method according to claim 1, It is characterized in that The step of obtaining preset Gaussian distributions corresponding to different ages and determining expected values ​​of each of the vectors based on the preset Gaussian distributions comprises: Based on the preset parameters, determine the preset Gaussian distribution corresponding to different ages; Fitting the preset Gaussian distribution based on each of the vectors to determine a target Gaussian distribution corresponding to each of the vectors; Based on the target Gaussian distribution, an expected value of each of the vectors is determined.

6. An image processing device, It is characterized in that The image processing device comprises: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the steps of the image processing method according to any one of claims 1 to 5 when executed by the processor.

7. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

Images