Brain tissue image segmentation model method based on Self-Attention mechanism

By introducing the Self-Attention mechanism and an approximate theory to fit the offset in the brain medical image segmentation model, a mathematical model that conforms to the grayscale unevenness characteristics was established, and the noise information and offset field information in the existing model were solved. The problem of imperfect model achieves higher segmentation accuracy and speed.

CN116402832BActive Publication Date: 2024-12-06WUHAN VOCATIONAL COLLEGE OF SOFTWARE & ENG (WUHAN OPEN UNIV)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310372887.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-10
Publication Date
2024-12-06
Estimated Expiration
2043-04-10

AI Technical Summary

Technical Problem

Due to the imperfection of noise information and offset field information models, the existing brain medical image segmentation model leads to low segmentation accuracy and insufficient processing capability for grayscale uneven image.

Method used

Using the brain tissue image segmentation model based on the Self-Attention mechanism, the offset field information model is established through the approximate theory, and combined with the statistical model and the level set method, the mathematical image characteristics of the brain is constructed that conforms to the grayscale uneven brain medical image characteristics. Model, using local energy segmentation model and variant U-Net segmentation model for image segmentation.

Benefits of technology

It improves the accuracy and speed of brain image segmentation, effectively deals with the problem of grayscale unevenness caused by artifacts and strong noise, and improves the stability and accuracy of the segmentation model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116402832B_ABST
    Figure CN116402832B_ABST
Patent Text Reader

Abstract

A brain tissue image segmentation model method based on the Self-Attention mechanism adopts the following steps: first, according to the grayscale inhomogeneity characteristics of artifacts and noise in multimodal brain images, a mathematical model of brain images based on grayscale inhomogeneity is established; then, two implementation methods are designed to solve the problem of brain image segmentation: implementation method (1) uses the local approximate estimation method to establish an offset field information model, and on this basis, an energy segmentation model based on local regions is established and the model calculation is accelerated and optimized; implementation method (2) constructs a variant U-Net segmentation model based on Self-Attention on the basis of image preprocessing, and uses LSM to correct the model results; finally, the results of the image to be segmented are obtained by using the two implementation methods. Through the implementation of the energy model based on local regions in method (1), the segmentation accuracy problem caused by grayscale inhomogeneity characteristics such as artifacts and strong noise in brain images is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a brain tissue image segmentation model method, in particular to a brain tissue image segmentation model method based on a Self-Attention mechanism. Background Art

[0002] There are many research objects based on brain medical images. Such images are easily affected by external factors such as illumination, shooting angle and frequency inhomogeneity of the receiving coil, and thus tend to show grayscale inhomogeneity. This characteristic is mainly reflected in the systematic changes in the local statistical characteristics of the image. The existing energy-based image segmentation models are all constructed based on the mathematical model shown in formula (1).

[0003] I(x)=b(x)J(x)+n(x),x∈Ω (1)

[0004] In formula (1), I(x) represents the original image, b(x) represents the offset field information model of the grayscale uneven area in the image, J(x) represents the real image, and N(x) represents the noise information model of the image.

[0005] The problems existing in the above mathematical models are mainly manifested in: the imperfection of the mathematical models of the noise information and offset field information of the image to be segmented and the fact that most of the existing models ignore the noise information model as a premise to construct the segmentation model. The establishment of such models leads to the segmentation model designed on the basis of this assumption being less robust to noise information and having low accuracy in the segmentation of medical images with uneven grayscale. Summary of the invention

[0006] The purpose of the present invention is to provide a brain tissue image segmentation model method based on the Self-Attention mechanism. This method uses the approximate theoretical fitting offset method to establish an offset field information model of the brain image, and combines the theoretical knowledge of the statistical model and the level set method to establish a mathematical model that conforms to the grayscale uneven brain medical image characteristics to solve the above problems.

[0007] A brain tissue image segmentation model method based on the Self-Attention mechanism adopts the following steps:

[0008] Firstly, according to the grayscale inhomogeneity characteristics of artifacts and noise in multimodal brain images, a mathematical model of brain images based on grayscale inhomogeneity was established.

[0009] Then, two implementation methods are designed to solve the problem of brain image segmentation: Implementation method (1) uses the local approximate estimation method to establish the offset field information model, and on this basis, establishes an energy segmentation model based on the local area and performs accelerated optimization of the model calculation; Implementation method (2) constructs a variant U-Net segmentation model based on Self-Attention on the basis of image preprocessing, and uses LSM to correct the model results;

[0010] Finally, the results of the image to be segmented are obtained using two implementation methods.

[0011] Furthermore, the offset field information model of brain images is established using the approximate theoretical fitting offset method. The theoretical knowledge of statistical models and the level set method are combined to establish a mathematical model that conforms to the characteristics of brain medical images with uneven grayscale. The quality of the model will directly affect the accuracy of the segmentation model in processing images.

[0012] Furthermore, the implementation method (1) uses the energy-based LSM model as the segmentation framework and constructs a segmentation model based on the mathematical model of the brain image with uneven grayscale. The specific method is as follows:

[0013] Firstly, the image to be segmented is input, and the offset field information model is constructed by approximately estimating the offset field of the neighboring area of ​​the image pixel point using the local block model as a unit to obtain the corrected image;

[0014] Then, a segmentation model based on local energy is established by using rectangular blocks and circular initialization contours: a local energy information model is constructed using the similarity between pixels, and a regularized information model based on variable parameters is used to stabilize the process of curve iteration;

[0015] Then, the model is embedded in the LSM framework, and the intelligent optimization algorithm is used to optimize the solution of the model and improve the speed of curve evolution;

[0016] Finally, the segmentation result is obtained.

[0017] The offset field information model is based on the statistical approximation theory. The offset field information model based on local block information is constructed using the approximate estimation method, which can not only effectively obtain the essential characteristics of the image, but also achieve the effect of noise reduction.

[0018] Furthermore, implementation method (2) is implemented in the form of a U-Net network model based on the Self-Attention mechanism. The Self-Attention mechanism can effectively extract the features of associated pixels. The purpose of this method is to increase the speed of the deep network and to add global associated features while considering local image features;

[0019] The specific method of implementing method (2) is as follows:

[0020] First, input the image to be segmented and build a variant deep convolutional network model: introduce Self-Attention into the receptive field of the deep network to replace part of the receptive field and further extract the features of the associated pixels; especially at the blurred edges of the brain image, the self-attention mechanism is used to effectively extract the features of the associated pixels at the edge, which can better segment the edge details. At the same time, a deep forward feedback operation is performed, which can refine the number of receptive fields of the traditional network and improve the speed of the network; then use the advantages of the underlying characteristics of the LSM image to correct the segmented image; finally, output the segmentation result;

[0021] The self-attention mechanism characteristics are migrated to the deep network structure, and the advantage of Self-Attention in efficiently extracting related pixel features is utilized. The mutated U-Net network fully utilizes the correlation characteristics of the image, ultimately improving the accuracy and speed of brain image segmentation.

[0022] The beneficial effects of the present invention are as follows: by implementing the energy model based on the local region in method (1), the segmentation accuracy problem caused by grayscale inhomogeneity such as artifacts and strong noise in brain images is improved, and the existing similar methods are solved due to the construction problem in the image mathematical model and the stability of the solution in the energy segmentation model and the low segmentation accuracy. More importantly, LSM, as a typical energy information model, can better solve the optimization problem of the model in method (2). BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a model framework for brain tissue image segmentation;

[0024] Figure 2 This is a schematic diagram of the brain image information model;

[0025] Figure 3 Model diagram for implementation method (1);

[0026] Figure 4 It is a variant U-Net segmentation model based on Self-Attention;

[0027] Figure 5 The figure is a comparison chart of the segmentation results of the model of the present invention constructed using implementation scheme (1) and the LIF model and LSACM model.

[0028] Figure 6 This is a segmentation comparison diagram of the model of the present invention constructed using implementation scheme (2) and the LGD model and LSACM model. DETAILED DESCRIPTION

[0029] A brain tissue image segmentation model method based on the Self-Attention mechanism adopts the following steps:

[0030] Firstly, according to the grayscale inhomogeneity characteristics of artifacts and noise in multimodal brain images, a mathematical model of brain images based on grayscale inhomogeneity was established.

[0031] Then, two implementation methods are designed to solve the problem of brain image segmentation: Figure 1 The implementation method (1) shown in the right part uses the local approximate estimation method to establish the offset field information model. On this basis, an energy segmentation model based on the local area is established and the model calculation is accelerated and optimized; Figure 1 The implementation method (2) shown in the left part builds a variant U-Net segmentation model based on Self-Attention based on image preprocessing, and uses LSM to correct the model results;

[0032] Finally, the results of the image to be segmented are obtained using two implementation methods.

[0033] By implementing the energy model based on the local area in the implementation method (1), the segmentation accuracy problem caused by grayscale inhomogeneity such as artifacts and strong noise in brain images is improved, and the existing similar methods have solved the problems of the construction of image mathematical models and the stability of the solution in the energy segmentation model and the low segmentation accuracy. More importantly, LSM, as a typical energy information model, can better solve the optimization problem of the model in the implementation method (2).

[0034] like Figure 2 As shown, the left side is the original image, and the right side represents the offset field information model and the noise information model respectively. The present invention takes brain medical images as the research object, uses the method of approximate theoretical fitting offset to establish the offset field information model of brain images, and combines the theoretical knowledge of statistical models and the level set method to establish a mathematical model that conforms to the grayscale uneven brain medical image characteristics. The quality of the model will directly affect the accuracy of the segmentation model in processing images.

[0035] Segmentation model implementation method (1)

[0036] like Figure 1 The implementation method (1) shown specifically uses the energy-based LSM model as the segmentation framework and constructs a segmentation model based on the mathematical model of the brain image with uneven grayscale.

[0037] Specific plans adopted include Figure 3 As shown:

[0038] Firstly, the image to be segmented is input, and the offset field information model is constructed by approximately estimating the offset field of the neighboring area of ​​the image pixel point using the local block model as a unit to obtain the corrected image;

[0039] Then, a segmentation model based on local energy is established by using rectangular blocks and circular initialization contours: a local energy information model is constructed using the similarity between pixels, and a regularized information model based on variable parameters is used to stabilize the process of curve iteration;

[0040] Then, the model is embedded in the LSM framework, and intelligent optimization algorithms (such as QPSO) are used to optimize the solution of the model and improve the speed of curve evolution;

[0041] Finally, the segmentation result is obtained.

[0042] The offset field information model is based on the statistical approximation theory. The offset field information model based on local block information is constructed by using the approximate estimation method, which can not only effectively obtain the essential characteristics of the image, but also achieve the effect of noise reduction. In the existing segmentation model, only the offset information is considered from a global perspective, and the offset information in the local area of ​​the image is not considered. For brain images, especially the information at the blurred edges, the local characteristics are crucial, which directly affects the accuracy of the segmentation edge.

[0043] Segmentation model implementation method (2)

[0044] like Figure 1 The implementation scheme (2) shown is specifically implemented based on statistical theory and the U-Net network model based on the self-attention mechanism as a research method. The idea of ​​the self-attention mechanism comes from the Encoder structure in the Transformer model in natural language processing. The self-attention mechanism in the structure can effectively extract the features of related pixels. Different from the prototype, this project uses full convolution instead of full connection in the later operation. The purpose of this method is to improve the speed of the deep network and add global correlation features while considering local image features.

[0045] Figure 4 The specific details of implementing method (2) are shown:

[0046] First, input the image to be segmented and build a variant deep convolutional network model: introduce Self-Attention into the receptive field of the deep network to replace part of the receptive field and further extract the features of associated pixels; especially at the blurred edges of the brain image, the self-attention mechanism is used to effectively extract the features of associated pixels at the edge, which can better segment the edge details. At the same time, a deep forward feedback operation is performed, which can refine the number of receptive fields of the traditional network and increase the speed of the network; then use the advantages of the underlying characteristics of the LSM image to correct the segmented image; finally, output the segmentation result.

[0047] The self-attention mechanism characteristics are migrated to the deep network structure, and the advantage of Self-Attention in efficiently extracting related pixel features is utilized. The mutated U-Net network fully utilizes the correlation characteristics of the image, ultimately improving the accuracy and speed of brain image segmentation.

[0048] Test results explanation:

[0049] LIF model and LSACM are two segmentation models based on local image information proposed by Zhang Kaihua. These two models have good effects on brain tissue image segmentation. Therefore, the segmentation model of the present invention in Implementation Example 1 is compared with these two typical models to verify the effectiveness of the present invention. Figure 5 As shown in Figure 1, the comparison results of the three segmentation models show that the model of the present invention performs better in segmentation accuracy. The main reason is that the model of the present invention can effectively handle the segmentation problem of complex contours and weak boundary images, and is more suitable for the segmentation of brain tissue images with complex structures and blurred edges.

[0050] In addition, the LGD model is a local Gaussian distribution fitting energy-driven segmentation model proposed by Wang Li. The model has a good effect in segmenting brain images with uneven grayscale. The model of the present invention is used in implementation scheme 2 together with LGD and LSACM to conduct experimental research on the segmentation of brain tissue with uneven grayscale and complex contours. Figure 6 As shown in the figure, compared with the typical LGD and LSACM models, the new model performs better in processing brain images with blurred edges and high noise. The reason is that the segmentation model in the present invention introduces a partial Self-Attention mechanism to replace part of the receptive field in the deep network receptive field, which is more conducive to obtaining blurred edge details.

[0051] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the elements of the claims be included in the present invention. Any reference numeral in a claim should not be regarded as limiting the claim to which it relates.

Claims

1. A brain tissue image segmentation model method based on the Self-Attention mechanism, characterized by: Use the following steps: According to the grayscale inhomogeneity characteristics of artifacts and noise in multimodal brain images, a mathematical model of brain images based on grayscale inhomogeneity is established; Implementation method (1) Using the local approximate estimation method to establish the offset field information model, on this basis, establish the energy segmentation model based on the local area and perform accelerated optimization of the model calculation; The implementation method (1) uses the energy-based LSM model as the segmentation framework and constructs a segmentation model based on the mathematical model of the brain image with uneven grayscale. The specific method is as follows: Firstly, the image to be segmented is input, and the offset field information model is constructed by approximately estimating the offset field of the neighboring area of ​​the image pixel point using the local block model as a unit to obtain the corrected image; Then, a segmentation model based on local energy is established by using rectangular blocks and circular initialization contours: a local energy information model is constructed using the similarity between pixels, and a regularized information model based on variable parameters is used to stabilize the process of curve iteration; Then, the model is embedded in the LSM framework, and the intelligent optimization algorithm is used to optimize the solution of the model and improve the speed of curve evolution; Finally, the segmentation result is obtained; The offset field information model is based on the statistical approximation theory. The offset field information model based on local block information is constructed using the approximate estimation method, which can not only effectively obtain the essential characteristics of the image, but also achieve the effect of noise reduction.

2. According to claim 1, a brain tissue image segmentation model method based on the Self-Attention mechanism is characterized by: The offset field information model of brain images is established by using the approximate theoretical fitting offset method. The theoretical knowledge of statistical models and the level set method are combined to establish a mathematical model that conforms to the characteristics of grayscale uneven brain medical images. The quality of the model will directly affect the accuracy of the segmentation model in processing images.

Citation Information

Patent Citations

  • Improved U-Net brain tumor segmentation method based on attention mechanism and multi-scale feature fusion

    CN115424103A

  • A System and Computer-Implemented Method for Segmenting an Image

    US20200167930A1