Tumor lesion edema area segmentation model training method, segmentation method and device

Abstract

The application relates to a tumor lesion edema area segmentation model training method, a segmentation method and a device, and relates to the technical field of medical image processing. The tumor lesion edema area segmentation model training method comprises the following steps: acquiring FLAIR sequence magnetic resonance imaging sample data, wherein the magnetic resonance imaging sample data is labeled with a brain tumor lesion edema area; inputting the magnetic resonance imaging sample data into an encoder of a to-be-trained model, performing feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution, and obtaining a first result; mapping the first result to a frequency domain based on discrete cosine transformation to obtain a second result; inputting the second result into a decoder of the to-be-trained model, and performing supervised training on the to-be-trained model based on a preset overall loss function to obtain a target segmentation model after training. The application has the effect of improving the tumor lesion edema area segmentation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a tumor lesion edema region segmentation model training method, a segmentation method and a device. BACKGROUND

[0002] The tumor lesion edema region refers to the tissue edema region around or inside the tumor caused by tumor growth, inflammatory response or vascular abnormalities. This edema usually appears as a low-density or low-signal region on imaging. The segmentation of the tumor lesion edema region is crucial for assessing the invasion range of the tumor and developing a radiotherapy plan. Precise segmentation of the edema region not only helps to determine the radiotherapy target area, but also provides important information for prognosis evaluation. The segmentation of the edema region refers to the process of separating the edema region around the tumor lesion from the background or other tissues in the medical image.

[0003] Currently, the segmentation of the tumor lesion edema region is usually performed by a threshold-based segmentation method, which involves image preprocessing of magnetic resonance imaging, setting a threshold based on the gray level histogram of the preprocessed image, and marking the region with a gray value below the threshold as the edema region to finally achieve the segmentation of the edema region. In this method, once the edema region of the tumor lesion is complex, the image of the magnetic resonance imaging may have a problem of uneven gray scale, and the boundary between the edema region and normal tissue is difficult to accurately identify, resulting in poor accuracy of the segmentation of the tumor lesion edema region. SUMMARY

[0004] To improve the accuracy of the segmentation of the tumor lesion edema region, the present application provides a tumor lesion edema region segmentation model training method, a segmentation method and a device.

[0005] In a first aspect of the present application, a tumor lesion edema region segmentation model training method is provided, which specifically comprises:

[0006] Obtaining magnetic resonance imaging sample data of FLAIR sequence, which is labeled with the edema region of a brain tumor lesion;

[0007] Inputting the magnetic resonance imaging sample data into an encoder of a to-be-trained model, and performing feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result;

[0008] Mapping the first result to a frequency domain based on discrete cosine transform to obtain a second result;

[0009] Inputting the second result into a decoder of the to-be-trained model, and performing supervised training on the to-be-trained model based on a preset overall loss function to obtain a trained target segmentation model.

[0010] By adopting the technical scheme, after the magnetic resonance imaging sample data is acquired, the magnetic resonance imaging sample data is input into an encoder of a to-be-trained model as training data to train the model. After the magnetic resonance imaging sample data is convoluted by the encoder and features are preliminarily extracted, a first result is obtained based on frequency domain convolution processing, global feature representation of the magnetic resonance imaging sample data is realized, the deficiency of traditional convolution in which only local features can be acquired is made up, the expression capability of the features is enhanced to some extent, and the model can be better understood. Then, the first result is mapped to a frequency domain based on discrete cosine transformation, long-distance correlation of frequency and channel is enhanced, the feature extraction process is further optimized, the distinguishability of the edema region is improved, finally, a second result is input into the encoder, and the to-be-trained model is supervised and trained through an overall loss function, so that the model can acquire better performance based on high-quality extracted features, and the target segmentation model has better capability of segmenting the edema region. Subsequently, the edema region can be segmented more accurately through the target segmentation model.

[0011] In an implementation manner, the feature extraction on the magnetic resonance imaging sample data based on the frequency domain convolution obtains a first result, and specifically includes:

[0012] The magnetic resonance imaging sample data is convoluted by an encoder in the to-be-trained model to obtain initial extracted features;

[0013] The initial extracted features are subjected to global average pooling processing to obtain a first processing result, and the first processing result is subjected to dimension reduction processing and processing through a batch normalization layer and an activation function to obtain a second processing result;

[0014] The second processing result is processed by a Fourier unit to obtain a first branch result, and the second processing result is processed by a local Fourier unit to obtain a second branch result;

[0015] The second processing result is processed by a skip connection to obtain a third branch result, and the first branch result, the second branch result and the third branch result are superimposed to obtain a superimposed result;

[0016] The superimposed result is subjected to dimension recovery processing by a convolution layer to obtain the first result.

[0017] In an implementation manner, the second processing result is processed by the Fourier unit to obtain the first branch result, and specifically includes:

[0018] The second processing result is converted into a first real part quantity and a first imaginary part quantity in a frequency domain by the Fourier unit, and the first real part quantity and the first imaginary part quantity are spliced along a channel direction to obtain a first splicing result;

[0019] The first splicing result is processed through a preset convolutional layer, a batch normalization layer and an activation function to obtain a frequency domain convolutional feature;

[0020] The frequency domain convolutional feature is split into a second real part and a second imaginary part, and inverse fast Fourier transform is performed on the second real part and the second imaginary part to obtain a first branch result.

[0021] In an embodiment, the second processing result is processed through a local Fourier unit to obtain a second branch result, specifically including:

[0022] The second processing result is divided into a preset number of partial features through a local Fourier unit, and each partial feature is spliced along the channel direction to obtain a second splicing result;

[0023] The second splicing result is processed through a Fourier unit to obtain corresponding target global information;

[0024] The target global information is copied and spatially offset to obtain a second branch result, which is the same in resolution size and channel number as the second processing result.

[0025] In an embodiment, the first result is mapped into a frequency domain based on discrete cosine transform to obtain a second result, specifically including:

[0026] The first result is flattened to obtain a first flattened result, and the discrete cosine transform is flattened to obtain a second flattened result;

[0027] The first flattened result and the second flattened result are processed through a preset matrix multiplication to obtain first frequency feature values on at least one channel;

[0028] Each first frequency feature value corresponding to a single channel is input into a preset self-attention mechanism to obtain second frequency feature values of the corresponding channel, and based on each second frequency feature value corresponding to a single channel, a feature vector of the corresponding channel is constructed;

[0029] Each feature vector is input into a preset self-attention mechanism to obtain a corresponding processed feature vector, and all frequency feature values in a single processed feature vector are fused to obtain a corresponding second result.

[0030] In an embodiment, the to-be-trained model is supervised trained based on a preset overall loss function to obtain a trained target segmentation model, specifically including:

[0031] Based on the preset overall loss function, the output of each network layer in the decoder is weakly supervised by a preset Sobel operator to obtain a corresponding edema area edge feature output result;

[0032] Based on the edema area edge feature output result corresponding to each network layer, a final output result of the to-be-trained model is obtained, and the final output result is strongly supervised by the Sobel operator to obtain a trained target segmentation model; wherein the overall loss function is:

[0033] ;

[0034] ;

[0035] In the formula, indicates the overall loss function, indicates the loss function of the Sobel operator supervision on the to-be-trained model, indicates the loss function of the edema area segmentation in the training process, and both indicate weight coefficients, indicates the loss function of the weak supervision on the output of the i-th network layer in the training process, indicates the corresponding weight coefficient, indicates the loss function of the edema area edge predicted by the to-be-trained model in the training process, and k indicates the total number of network layers.

[0036] In a second aspect of the present application, a tumor lesion edema area segmentation method is provided, specifically comprising:

[0037] Obtaining to-be-segmented magnetic resonance imaging data and a target segmentation model obtained by the tumor lesion edema area segmentation model training method of the first aspect;

[0038] Inputting the to-be-segmented magnetic resonance imaging data into the target segmentation model to obtain a segmented edema area.

[0039] By using the above technical solution, the to-be-segmented magnetic resonance imaging data is input into the target segmentation model, the edge of the tumor lesion edema area in the to-be-segmented magnetic resonance imaging data is accurately identified by the target segmentation model, and the tumor lesion edema area is combined to realize accurate segmentation of the edema area in the to-be-segmented magnetic resonance imaging data.

[0040] In a third aspect of the present application, a tumor lesion edema area segmentation model training device is provided, specifically comprising:

[0041] The data acquisition module is used to acquire magnetic resonance imaging sample data of FLAIR sequences, wherein the magnetic resonance imaging sample data is marked with the edema area of ​​the brain tumor lesion;

[0042] The feature extraction module is used to input the magnetic resonance imaging sample data into the encoder of the model to be trained, and to extract features from the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result;

[0043] The feature optimization module is used to map the first result to the frequency domain based on discrete cosine transform to obtain the second result;

[0044] The model training module is used to input the second result into the decoder of the model to be trained, and to perform supervised training on the model to be trained based on a preset overall loss function, so as to obtain a trained target segmentation model.

[0045] By adopting the above technical solution, after the data acquisition module acquires the magnetic resonance imaging sample data, the feature extraction module performs feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain the first result. Then, the feature optimization module maps the first result to the frequency domain based on discrete cosine transform to obtain the second result. Finally, the model training module performs supervised training on the model to be trained based on the preset overall loss function to obtain the trained target segmentation model.

[0046] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when loaded and executed by a processor, performs the steps of the method described in any one of the first aspects.

[0047] A fifth aspect of this application provides an electronic device, specifically comprising:

[0048] A processor, a memory, and a computer program stored in the memory and capable of running on the processor, the processor being configured to load and execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0049] In summary, this application includes at least one of the following beneficial technical effects:

[0050] After the encoder performs convolutional processing on the MRI sample data to initially extract features, it performs frequency domain convolution to obtain the first result. This achieves effective global feature representation of the MRI sample data, overcoming the limitation of traditional convolution which can only extract local features. This enhances the expressive power of the features to a certain extent, allowing the model to better understand them. Next, the first result is mapped to the frequency domain based on discrete cosine transform to enhance the long-distance correlation between frequency and channels, further optimizing the feature extraction process and improving the discriminability of edema areas. Finally, the second result is input into the encoder, and the model is trained under supervision using the overall loss function. This allows the model to achieve better performance based on high-quality extracted features, thus enabling the target segmentation model to better segment edema areas. Subsequent segmentation using the target segmentation model can then more accurately segment edema areas. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating a method for training a segmentation model of a tumor lesion edema area according to an embodiment of this application.

[0052] Figure 2 This is a schematic flowchart illustrating the process of determining a second result provided in an embodiment of this application;

[0053] Figure 3 This is a flowchart illustrating a method for segmenting the edematous area of ​​a tumor lesion according to an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the structure of a segmentation model training device for edema areas of tumor lesions provided in an embodiment of this application;

[0055] Figure 5 This is a schematic diagram of another segmentation device for edema areas of tumor lesions provided in an embodiment of this application.

[0056] Figure labeling: 11. Data acquisition module; 12. Feature extraction module; 13. Feature optimization module; 14. Model training module; 21. Information acquisition module; 22. Edema segmentation module. Detailed Implementation

[0057] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0058] In the description of the embodiments of the present application, the words "exemplary", "for example", or "e.g." are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as "exemplary", "for example", or "e.g." in the embodiments of the present application should not be construed as being more preferred or advantageous than other embodiments or design schemes. In fact, the words "exemplary", "for example", or "e.g." are used to present the relevant concept in a specific manner.

[0059] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, B alone, and A and B together. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0060] Referring to Figure 1 The embodiments of the present application disclose a flowchart of a tumor lesion edema area segmentation model training method, which can be implemented by relying on a computer program and can also run on a tumor lesion edema area segmentation model training device based on the Von Neumann system. The computer program can be integrated in an application or run as an independent tool class application, and specifically includes:

[0061] S101: Obtain magnetic resonance imaging sample data of a FLAIR sequence.

[0062] Specifically, the magnetic resonance imaging (MRI) sample data is labeled with an edema area of a brain tumor lesion, and is manually labeled (the edema area existing in the magnetic resonance imaging sample data is outlined) by a neuroradiologist in advance. The fluid attenuated inversion recovery (FLAIR) sequence is a kind of MRI sequence based on inversion recovery technology, which highlights the signal of bound water or pathological tissue by selectively suppressing the signal of free water (such as cerebrospinal fluid, cyst fluid), so as to more clearly display the lesions or abnormalities in the brain tissue. The MRI sequence is a combination of scanning parameters and radio frequency pulses in magnetic resonance imaging (MRI) for generating images. Different sequences can highlight different tissue characteristics or pathological changes. Among them, the MRI data of the FLAIR sequence can highlight the edema area around the tumor, which is an important basis for accurately segmenting the edema area of the tumor lesion. In addition, the magnetic resonance imaging sample data is the MRI data of the FLAIR sequence of a brain tumor patient. The brain tumor patient can be a glioblastoma (GBM) patient, and in other embodiments, it can also be a patient with pituitary adenoma or meningioma and other brain tumors.

[0063] Further, the execution subject of the tumor lesion edema area segmentation model training method disclosed in the embodiments of the present application is a server, the server is wirelessly connected with a terminal, the server is an independent physical server, and can also be a cluster composed of multiple physical servers. The terminal is a personal computer or a tablet computer. An implementation scenario is that when a target segmentation model needs to be trained, the target segmentation model is a model capable of automatically segmenting the edema area of a tumor lesion in the FLAIR sequence of a brain tumor patient, a person sends at least one set of FLAIR sequence magnetic resonance imaging data of a historical patient diagnosed as glioblastoma (GBM) to the server through the terminal, and the server obtains the magnetic resonance imaging data sent by the terminal as magnetic resonance imaging sample data, trains the model, and finally obtains the target segmentation model.

[0064] S102: input the magnetic resonance imaging sample data into the encoder of the model to be trained, and perform feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result.

[0065] Specifically, after the magnetic resonance imaging sample data is acquired, the magnetic resonance imaging sample data is input to a to-be-trained model to enter a training stage, wherein the to-be-trained model is a convolutional neural network (CNN) model, and the convolutional neural network model includes an encoder and a decoder. It should be noted that the convolutional neural network model is widely used in the segmentation of medical images or images, especially the segmentation of tumor lesion edema areas. For the segmentation of tumor lesion edema areas, the convolutional neural network model usually extracts texture features related to the edema area to realize the differentiation or segmentation of the edema area.

[0066] The magnetic resonance imaging sample data is input to the encoder of the to-be-trained model, and feature extraction needs to be performed on the magnetic resonance imaging sample data based on frequency domain convolution. One implementable embodiment is that the encoder in the to-be-trained model performs down-convolution operation on the magnetic resonance imaging sample data, specifically performs convolution operation on the magnetic resonance imaging sample data through a preset convolution kernel, to obtain features extracted from the magnetic resonance imaging sample data, that is, initial extracted features. Then, the initial extracted features are subjected to global average pooling processing to obtain a first processing result, thereby reducing the noise and calculation amount of the initial extracted features. The frequency domain convolution is an important concept in signal processing and image processing, which performs convolution operation after converting the signal or image to the frequency domain (such as Fourier transform domain), and then converts back to the time domain or spatial domain. Feature extraction through frequency domain convolution can enhance the expression ability of the features to some extent, so that the model can better understand. Traditional convolution usually uses a small size convolution kernel, which limits the receptive field, while frequency domain convolution can well expand the receptive field.

[0067] Further, the first processing result is subjected to dimension reduction processing through a convolution layer with a convolution kernel size of 1*1, that is, Conv operation, and is processed through a batch normalization (BN) layer and an activation function to obtain a second processing result. The activation function can be a rectified linear unit (ReLU) with nonlinear characteristics, which can effectively enhance the expression ability of the neural network. In other embodiments, the activation function can also be Sigmoid or Tanh. The batch normalization layer solves the problem of unstable parameter distribution, and the activation function solves the limitation of the linear model, and the two work together to make the model more efficient and stable.

[0068] Further, the second processing result is processed by a Fourier unit (FU) to obtain a first branch result, where the Fourier unit is a module or layer based on Fourier transform in deep learning, has the ability to obtain a global receptive field, and enhances the modeling ability of the model for periodic signals by converting input data to the frequency domain and extracting frequency-related features. The specific processing process is: the second processing result is converted into a first real part and a first imaginary part in the frequency domain by the Fourier unit, and the conversion process is expressed as A R , A I = FFT(F R ), A R represents the first real part, A I represents the first imaginary part, F R represents the second processing result, and FFT represents fast Fourier transform. Then, the first real part and the first imaginary part are spliced along the channel direction, that is, the two components are combined in the channel dimension to obtain a first spliced result. The first spliced result is processed by a preset convolution layer, a batch normalization layer and an activation function to obtain a frequency domain convolution feature. Considering that any operation in the frequency domain has a global range of influence on the spatial domain, that is, any operation has a global receptive field, the convolution kernel size of the convolution layer in the present application embodiment is 1*1. Further, the frequency domain convolution feature is split into a second real part and a second imaginary part, and the split function is used to split the frequency domain convolution feature, where the split function is a function for splitting a tensor along a specified dimension into multiple sub-tensors. It is usually used to split a larger tensor into multiple smaller parts for separate processing or passing to different network branches. Finally, the second real part and the second imaginary part are subjected to inverse fast Fourier transform (IFFT) to obtain the first branch result, that is, the frequency domain feature is converted back to the spatial domain feature, which is convenient for practical application, and the spatial domain feature reflects the original characteristics of the data in the original domain.

[0069] Further, the second processing result is processed by a local Fourier unit (LFU) to obtain a second branch result, wherein the local Fourier unit is a neural network module combining the ideas of Fourier transform and deep learning, and is mainly used for extracting local frequency domain features of signals or images. A feasible way to process the second processing result is to divide the second processing result into a preset number of partial features by the local Fourier unit, specifically, the second processing result is divided or segmented into a preset number of partial features by a split function, and the preset number is 4, and in other embodiments, the preset number can also be other reasonable numbers. Then, each partial feature is spliced along the same channel direction to obtain a second splicing result. Then, the second splicing result is processed by a Fourier unit to obtain global frequency features corresponding to the second splicing result, i.e., target global information. The target global information is copied and spatially shifted, specifically, the target global information is copied by a shift function, and the copying number is 3 times, so that the total number (4) of the copied result and the target global information is the same as the above-mentioned preset number, and further, each copied result and the target global information are spatially shifted, i.e., each copied result and the target global information are translated in the spatial domain to obtain the second branch result, wherein the second branch result is the same as the second processing result in resolution size and channel number.

[0070] Further, the second processing result is processed by a local Fourier unit (LFU) to obtain a second branch result, wherein the local Fourier unit is a neural network module combining the ideas of Fourier transform and deep learning, and is mainly used for extracting local frequency domain features of signals or images. A feasible way to process the second processing result is to divide the second processing result into a preset number of partial features by the local Fourier unit, specifically, the second processing result is divided or segmented into a preset number of partial features by a split function, and the preset number is 4, and in other embodiments, the preset number can also be other reasonable numbers. Then, each partial feature is spliced along the same channel direction to obtain a second splicing result. Then, the second splicing result is processed by a Fourier unit to obtain global frequency features corresponding to the second splicing result, i.e., target global information. The target global information is copied and spatially shifted, specifically, the target global information is copied by a shift function, and the copying number is 3 times, so that the total number (4) of the copied result and the target global information is the same as the above-mentioned preset number, and further, each copied result and the target global information are spatially shifted, i.e., each copied result and the target global information are translated in the spatial domain to obtain the second branch result, wherein the second branch result is the same as the second processing result in resolution size and channel number.

[0071] S103: Map the first result to the frequency domain based on the discrete cosine transform to obtain a second result.

[0072] Specifically, the discrete cosine transform (DCT) is a mathematical transform that converts a signal or data from the time domain (or spatial domain) to the frequency domain, which is a kind of basis function that can more efficiently represent and process data. After the first result is determined, the first result needs to be mapped to the frequency domain of the discrete cosine transform. One implementable implementation is to perform flattening processing on the first result by using a preset flatten function to obtain a first flattened result. Flattening processing is to convert multi-dimensional data into a one-dimensional vector while retaining all original feature information. Similarly, the flatten function is used to perform flattening processing on the discrete cosine transform to obtain a second flattened result. Then, the first flattened result and the second flattened result are processed by using a preset matrix multiplication to obtain first frequency feature values on at least one channel. The information on a single channel is described by a plurality of first frequency feature values. The frequency feature value refers to the representation of the signal or data in the frequency domain, which reflects the intensity or energy distribution of the signal at different frequencies.

[0073] Further, in order to fully associate the information between the respective frequency feature values corresponding to a single channel, a long-distance correlation of the full frequency is constructed, and the respective first frequency feature values corresponding to a single channel are input into a self-attention mechanism to obtain second frequency feature values of the corresponding channel. Then, the dimensions of all second frequency feature values corresponding to each channel are transformed as a whole to obtain a feature vector corresponding to the channel, and each feature vector contains all second frequency feature values on the corresponding channel. Then, all feature vectors are input into the self-attention mechanism to realize the interaction between the frequency feature values of different channels to obtain a processed feature vector corresponding to each feature vector, thereby constructing the global relationship between all channels under the full frequency. Finally, all frequency feature values in a single processed feature vector are fused by frequency aggregation composed of global average pooling to obtain a corresponding second result, thereby further optimizing the feature extraction process for the magnetic resonance imaging sample data and improving the distinguishability of the edema region to provide more accurate feature representation for subsequent edema region segmentation. For example, if the number of channels or depth C of the discrete cosine transform is 64, the specific process of obtaining the second result can be referred to as Figure 2 where H represents the height of the feature map, and W represents the width of the feature map.

[0074] S104: input the second result into the decoder of the to-be-trained model, and supervise the training of the to-be-trained model based on a preset overall loss function to obtain a trained target segmentation model.

[0075] Specifically, the up-sampling operation is performed by each network layer in the decoder, and finally the low-resolution feature map is restored to a high-resolution output. During the down-sampling process, in order to enable the model to more accurately obtain the edge features of the tumor lesion edema area, a Sobel operator is used to weakly supervise the output of each network layer in the trained model, wherein a convolution kernel with two preset weights is used to implement the Sobel operator. The Sobel operator is an algorithm for edge detection, which realizes edge detection by calculating the gradient of each pixel point in the image, which is a prior art and will not be described here. In the embodiments of the present application, when training the trained model, the Sobel operator is used to extract the edge features of the tumor lesion edema area from the input of each network layer, and the edge features are used as supervision information to constrain the learning of the fine edge features by the single network layer of the trained model, so that the edge features in the output of each network layer are more accurate.

[0076] Further, after the second result is determined, the second result is input into the decoder of the trained model, and the outputs of each network layer in the decoder are weakly supervised by the Sobel operator based on the preset overall loss function, to obtain the corresponding edema area edge feature output result, i.e. the output feature. The model is guided to pay attention to the edge features of the edema area in each network layer. Then, in order to obtain multi-level edge features, the output features of each network layer are fused, which is specifically described as follows: , wherein, represents the output feature of the i-th network layer, represents the output feature of the i+1-th network layer, represents the fused multi-level edge feature, and so on, to obtain the final output result of the trained model, i.e. the prediction result of the trained model, which is the segmentation result of the edema area. During the training process, the final output result is strongly supervised by the Sobel operator, the edema area edge in the prediction result of the model is constrained, then the overall loss function is minimized, and the parameters of the model are adjusted by the backpropagation algorithm until the model converges, and finally the trained target segmentation model is obtained. Wherein, the overall loss function is:

[0077] ;

[0078] ;

[0079] In the formula, represents the overall loss function, represents the loss function when the Sobel operator supervises the trained model, represents the loss function when the edema area is segmented during the training process, and both represent weight coefficients, denotes a loss function in the training process when the output of the i-th network layer is weakly supervised, denotes corresponding weight coefficients, denotes a loss function in the training process when the model to be trained predicts the edge of the edema region, k denotes the total number of network layers. In the embodiments of the present application, is 1e-6, is 1. It should be noted that, It can be understood as a loss function when the model to be trained predicts the edema region based on the extracted texture features of the edema region itself. The loss function can use a cross-entropy loss function or a focal loss function. In addition, in the training process of the target segmentation model in the embodiments of the present application, the overall loss function for supervision constraint combines the dimension of the edema region itself and the dimension of the edge or contour of the edema region, so that the target segmentation model is more accurate for the segmentation of the edema region in the input magnetic resonance imaging.

[0080] The implementation principle of the tumor lesion edema region segmentation model training method in the embodiments of the present application is as follows: after the encoder convolves the magnetic resonance imaging sample data and preliminarily extracts the features, a first result is obtained based on frequency domain convolution processing, realizing effective representation of global features of the magnetic resonance imaging sample data, making up for the deficiency of traditional convolution that can only obtain local features, and to some extent, enhancing the expression ability of the features, so that the model can better understand. Then, the first result is mapped to the frequency domain based on discrete cosine transform, enhancing the long-distance correlation of frequency and channel, further optimizing the feature extraction process, improving the distinguishability of the edema region, and finally inputting the second result into the encoder and supervising the training of the model to be trained through the overall loss function, so that the model can obtain better performance based on high-quality extracted features, and thus the target segmentation model has better ability to segment the edema region. Through the target segmentation model, the edema region can be segmented more accurately.

[0081] Referring to Figure 3 The embodiments of the present application disclose a flowchart of a tumor lesion edema region segmentation method, which can be implemented by relying on a computer program, and can also run on a tumor lesion edema region segmentation device based on the Von Neumann system. The computer program can be integrated in an application, or can run as an independent tool application. Specifically, it comprises:

[0082] S201: obtaining magnetic resonance imaging data to be segmented and a target segmentation model obtained by using the tumor lesion edema region segmentation model training method in any of the above embodiments.

[0083] S202: inputting the magnetic resonance imaging data to be segmented into the target segmentation model to obtain the segmented edema region.

[0084] Specifically, after the target segmentation model is determined, the receiving terminal sends the to-be-segmented magnetic resonance imaging data, inputs the to-be-segmented magnetic resonance imaging data into the target segmentation model, accurately identifies the edge of the tumor lesion edema area in the to-be-segmented magnetic resonance imaging data through the target segmentation model, and combines the division of the tumor lesion edema area to achieve accurate segmentation of the edema area in the to-be-segmented magnetic resonance imaging data. The to-be-segmented magnetic resonance imaging data is the FLAIR sequence magnetic resonance imaging data actually needing edema area segmentation. The magnetic resonance imaging data is a magnetic resonance imaging image.

[0085] The following is an embodiment of the device of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0086] Please refer to Figure 4 The structure diagram of the tumor lesion edema area segmentation model training device provided in the embodiment of the present application. The tumor lesion edema area segmentation model training device can be realized by software, hardware or a combination of the two to become all or part of the device. The device comprises a data acquisition module 11, a feature extraction module 12, a feature optimization module 13 and a model training module 14.

[0087] The data acquisition module 11 is used to acquire FLAIR sequence magnetic resonance imaging sample data, and the magnetic resonance imaging sample data is labeled with the edema area of the brain tumor lesion;

[0088] The feature extraction module 12 is used to input the magnetic resonance imaging sample data into the encoder of the to-be-trained model, and perform feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result;

[0089] The feature optimization module 13 is used to map the first result to the frequency domain based on discrete cosine transform to obtain a second result;

[0090] The model training module 14 is used to input the second result into the decoder of the to-be-trained model, and supervise the training of the to-be-trained model based on a preset overall loss function to obtain a trained target segmentation model.

[0091] Optionally, the feature extraction module 12 is specifically used for:

[0092] The encoder in the to-be-trained model is used to perform convolution operation on the magnetic resonance imaging sample data to obtain initial extracted features;

[0093] The initial extracted features are subjected to global average pooling processing to obtain a first processing result, and the first processing result is subjected to dimension reduction processing and processing through a batch normalization layer and an activation function to obtain a second processing result;

[0094] The second processing result is processed through a Fourier unit to obtain a first branch result, and the second processing result is processed through a local Fourier unit to obtain a second branch result;

[0095] The second processing result is processed through a skip connection to obtain a third branch result, and the first branch result, the second branch result and the third branch result are superimposed to obtain a superimposed result;

[0096] The superimposed result is processed through a convolution layer for dimension recovery to obtain the first result.

[0097] Optionally, the feature extraction module 12 is specifically configured to:

[0098] The second processing result is converted into a first real part and a first imaginary part in a frequency domain through a Fourier unit, and the first real part and the first imaginary part are spliced along a channel direction to obtain a first spliced result;

[0099] The first spliced result is processed through a preset convolution layer, a batch normalization layer and an activation function to obtain a frequency domain convolution feature;

[0100] The frequency domain convolution feature is split into a second real part and a second imaginary part, and the second real part and the second imaginary part are inverse fast Fourier transformed to obtain the first branch result.

[0101] Optionally, the feature extraction module 12 is specifically configured to:

[0102] The second processing result is divided into a preset number of part features through a local Fourier unit, and each part feature is spliced along a channel direction to obtain a second spliced result;

[0103] The second spliced result is processed through a Fourier unit to obtain corresponding target global information;

[0104] The target global information is copied and spatially offset to obtain the second branch result, and the second branch result is the same as the second processing result in resolution size and channel number.

[0105] Optionally, the feature optimization module 13 is specifically configured to:

[0106] The first result is flattened to obtain a first flattened result, and the discrete cosine transform is flattened to obtain a second flattened result;

[0107] The first flattened result and the second flattened result are processed through a preset matrix multiplication to obtain a first frequency feature value on at least one channel;

[0108] The first frequency characteristic value corresponding to each single channel is input into a preset self-attention mechanism to obtain a second frequency characteristic value of the corresponding channel, and a feature vector of the corresponding channel is constructed based on each second frequency characteristic value corresponding to each single channel.

[0109] Each feature vector is input into a preset self-attention mechanism to obtain a corresponding processed feature vector, and all frequency characteristic values in each processed feature vector are fused to obtain a corresponding second result.

[0110] Optionally, the model training module 14 is specifically configured to:

[0111] Based on the preset overall loss function, the output of each network layer in the decoder is weakly supervised by a preset Sobel operator to obtain a corresponding edema area edge feature output result;

[0112] Based on the edema area edge feature output result corresponding to each network layer, a final output result of the to-be-trained model is obtained, and the final output result is strongly supervised by the Sobel operator to obtain a trained target segmentation model; wherein the overall loss function is:

[0113] ;

[0114] ;

[0115] In the formula, indicates the overall loss function, indicates the loss function of the Sobel operator supervision on the to-be-trained model, indicates the loss function of the edema area segmentation in the training process, and both indicate a weight coefficient, indicates the loss function of the weak supervision on the output of the i-th network layer in the training process, indicates a corresponding weight coefficient, indicates the loss function of the edema area edge predicted by the to-be-trained model in the training process, and k indicates the total number of network layers.

[0116] It should be noted that the tumor lesion edema area segmentation model training device provided in the above embodiment is only exemplified by the division of the above functional modules when performing the tumor lesion edema area segmentation model training method. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the tumor lesion edema area segmentation model training device and the tumor lesion edema area segmentation method embodiment provided in the above embodiment belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

[0117] See Figure 5 The structure diagram of the tumor lesion edema area segmentation device provided in the embodiment of the present application is shown. The tumor lesion edema area segmentation device can be realized by software, hardware or a combination of the two to become all or part of the device. The device includes an information acquisition module 21 and an edema segmentation module 22.

[0118] The information acquisition module 21 is configured to acquire the magnetic resonance imaging data to be segmented and the target segmentation model obtained by using the tumor lesion edema area segmentation model training method in any of the above embodiments. The magnetic resonance imaging data to be segmented is the FLAIR sequence magnetic resonance imaging data actually needed for edema area segmentation;

[0119] The edema segmentation module 22 is configured to input the magnetic resonance imaging data to be segmented into the target segmentation model to obtain the segmented edema area.

[0120] It should be noted that the tumor lesion edema area segmentation device provided in the above embodiment is only exemplified by the division of the above functional modules when performing the tumor lesion edema area segmentation method. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the tumor lesion edema area segmentation device and the tumor lesion edema area segmentation method embodiment provided in the above embodiment belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

[0121] The embodiment of the present application also discloses a computer readable storage medium, and the computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to realize the tumor lesion edema area segmentation model training method and the segmentation method provided in the above embodiment.

[0122] The computer program can be stored in a computer readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc., the computer readable medium includes any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. which can carry computer program code, it should be noted that the computer readable medium includes but is not limited to the above components.

[0123] The tumor lesion edema area segmentation model training method and the segmentation method of the above embodiment are stored in the computer readable storage medium by the computer readable storage medium, and are loaded and executed on the processor, so as to facilitate the storage and application of the above method.

[0124] The computer readable storage medium stores a computer program, and the computer program is loaded and executed by the processor, so as to realize the tumor lesion edema area segmentation model training method and the segmentation method.

[0125] The electronic device can be a desktop computer, a notebook computer or a cloud server, and the electronic device includes but is not limited to a processor and a memory, for example, the electronic device can also include an input / output device, a network access device and a bus, etc.

[0126] The processor can be a central processing unit (CPU), of course, according to the actual use, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), ready-to-program gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. can also be used, the general-purpose processor can be a microprocessor or any conventional processor, etc., and the present application does not limit this.

[0127] The memory can be an internal storage unit of the electronic device, for example, a hard disk or a memory of the electronic device, or an external storage device of the electronic device, for example, a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the electronic device, etc., and the memory can also be a combination of the internal storage unit and the external storage device of the electronic device, the memory is used to store computer programs and other programs and data required by the electronic device, and the memory can also be used to temporarily store data that has been output or will be output, and the present application does not limit this.

[0128] The tumor lesion edema area segmentation model training method and the segmentation method of the above-mentioned embodiments are stored in the memory of the electronic device by the electronic device, and are loaded and executed on the processor of the electronic device, which is convenient to use.

[0129] The above is only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. The present application is intended to cover any variations, uses, or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for training a segmentation model of edema areas in tumor lesions, characterized in that, The method comprises: acquiring magnetic resonance imaging sample data of a FLAIR sequence, the magnetic resonance imaging sample data being labeled with an edema area of a brain tumor lesion; inputting the magnetic resonance imaging sample data into an encoder of a to-be-trained model, and performing feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result, comprising: performing convolution operation on the magnetic resonance imaging sample data through the encoder in the to-be-trained model to obtain initial extracted features; performing global average pooling processing on the initial extracted features to obtain a first processing result, and performing dimension reduction processing on the first processing result and processing through a batch normalization layer and an activation function to obtain a second processing result; processing the second processing result through a Fourier unit to obtain a first branch result, and processing the second processing result through a local Fourier unit to obtain a second branch result; processing the second processing result through a skip connection to obtain a third branch result, and superimposing the first branch result, the second branch result and the third branch result to obtain a superimposed result; performing dimension restoration processing on the superimposed result through a convolution layer to obtain the first result; mapping the first result into a frequency domain based on discrete cosine transform to obtain a second result; inputting the second result into a decoder of the to-be-trained model, and performing supervised training on the to-be-trained model based on a preset overall loss function to obtain a target segmentation model after training.

2. The method of claim 1, wherein the method is performed by a computer system. The processing of the second processing result through the Fourier unit to obtain the first branch result specifically comprises: the second processing result is converted into a first real part quantity and a first imaginary part quantity in the frequency domain through the Fourier unit, and the first real part quantity and the first imaginary part quantity are spliced along the channel direction to obtain a first spliced result; the first spliced result is processed through a preset convolution layer, a batch normalization layer and an activation function to obtain a frequency domain convolution feature; the frequency domain convolution feature is split into a second real part quantity and a second imaginary part quantity, and the second real part quantity and the second imaginary part quantity are inverse fast Fourier transformed to obtain the first branch result.

3. The method of claim 1, wherein the method is performed by a computer system. The processing of the second processing result through the local Fourier unit to obtain the second branch result specifically comprises: the second processing result is divided into a preset number of part features through the local Fourier unit, and each part feature is spliced along the channel direction to obtain a second spliced result; the second spliced result is processed through the Fourier unit to obtain corresponding target global information; the target global information is copied and spatially offset to obtain the second branch result, and the second branch result is the same as the second processing result in resolution size and channel number.

4. The method of claim 1, wherein the method is performed by a computer system. The mapping of the first result into the frequency domain based on the discrete cosine transform to obtain the second result specifically comprises: the first result is flattened to obtain a first flattened result, and the discrete cosine transform is flattened to obtain a second flattened processing; The first frequency characteristic value corresponding to each of the single channels is input into a preset self-attention mechanism to obtain a second frequency characteristic value of the corresponding channel, and a feature vector of the corresponding channel is constructed based on each of the second frequency characteristic values corresponding to the single channels; The feature vectors are input into a preset self-attention mechanism to obtain a corresponding processed feature vector, and all frequency characteristic values in a single processed feature vector are fused to obtain a corresponding second result. The training completed target segmentation model is obtained by supervising training of the to-be-trained model based on the preset overall loss function, and specifically includes:

5. The method of claim 1, wherein the method is performed by a computer system. The output of each network layer in the decoder is weakly supervised by a preset Sobel operator based on the preset overall loss function to obtain a corresponding edema region edge feature output result; The final output result of the to-be-trained model is obtained based on the edema region edge feature output results corresponding to each network layer, and the final output result is strongly supervised by the Sobel operator to obtain the training completed target segmentation model; wherein the overall loss function is: The method comprises: ； ； In the formula, represents the overall loss function, represents the loss function of the Sobel operator when supervising the to-be-trained model, represents the loss function when segmenting the edema area in the training process, and both represent the weight coefficient, represents the loss function when weakly supervising the output of the i-th network layer in the training process, represents the corresponding weight coefficient, represents the loss function when the to-be-trained model predicts the edge of the edema area in the training process.

6. A method for segmenting the edematous area of ​​a tumor lesion, characterized in that, Obtaining to-be-segmented magnetic resonance imaging data and a target segmentation model obtained by the tumor lesion edema region segmentation model training method of any one of claims 1 to 5, wherein the to-be-segmented magnetic resonance imaging data is FLAIR sequence magnetic resonance imaging data actually requiring edema region segmentation; The to-be-segmented magnetic resonance imaging data is input into the target segmentation model to obtain a segmented edema region. It comprises: 7.A device for training a model for segmenting an edema region of a tumor lesion, for implementing the method for training a model for segmenting an edema region of a tumor lesion according to any one of claims 1 to 5, characterized in that, A data acquisition module (11) is configured to acquire FLAIR sequence magnetic resonance imaging sample data, wherein the magnetic resonance imaging sample data is labeled with an edema region of a brain tumor lesion; A feature extraction module (12) is configured to input the magnetic resonance imaging sample data into an encoder of a to-be-trained model, and perform feature extraction on the magnetic resonance imaging sample data based on frequency domain convolution to obtain a first result; A feature optimization module (13) is configured to map the first result to a frequency domain based on discrete cosine transform to obtain a second result; A model training module (14) is configured to input the second result into a decoder of the to-be-trained model, and supervise training of the to-be-trained model based on a preset overall loss function to obtain a training completed target segmentation model. The computer program is loaded and executed by the processor to realize the method of any one of claims 1-6.

8. A computer-readable storage medium having stored therein a computer program, characterized in that, The processor loads and executes the computer program to realize the method of any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, ​

Images