Brain texture analysis method based on lbp extraction and tcnn neural network

Abstract

The application discloses a brain texture analysis method based on LBP extraction and a TCNN neural network, MRI images are pretreated by using MIPAV software, the pretreated images are registered to a standard brain template space MNI, and Min-Max is used for standardization processing; a circular LBP method is used for texture feature extraction of the MRI images, and a bilinear interpolation method is used for calculating the gray value; the brain texture subjected to the feature extraction is input into a neural network TCNN composed of a Transformer and a CNN, and texture classification is carried out; a loss function is calculated according to the original MRI images and the images subjected to the texture feature extraction, and a back propagation algorithm is used for updating parameters until a set training number is met, and the method designed in the application adopts a new neural network architecture TCNN for texture classification, improves the accuracy of brain texture feature extraction and classification, and has practical significance and good prospects.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a brain texture analysis method based on LBP extraction and TCNN neural network. Background Technology

[0002] Texture is an inherent property of an object's surface, and regions in an image often exhibit texture properties. In recent years, with the continuous development of neuroimaging technology and in-depth research in brain science, brain texture analysis has become a highly popular and closely watched field. Brain texture refers to the texture features within brain tissue, including features such as cortical surface morphology and the gray-white matter interface. Research on brain texture analysis aims to explore the differences in texture features between different brain structures, thereby providing fundamental support for biological and neuroscience research.

[0003] However, traditional brain texture analysis methods suffer from several problems, such as low processing efficiency, high computational complexity, and incomplete feature extraction. In recent years, deep learning-based methods have gradually become a research hotspot. Convolutional Neural Networks (CNNs), as an important method in deep learning, have achieved significant success in image recognition and classification, and are increasingly being applied to brain texture analysis. However, the pooling layers of CNNs lose a large amount of valuable information and ignore the correlation between local and global features, thus reducing detection accuracy. While the multi-head attention feature of Transformers offers excellent globality and strong multimodal fusion capabilities, this requires large datasets, greater computational resources, and more time. Using Transformers alone cannot achieve the desired results; therefore, a new model architecture is needed to address these issues. Furthermore, Local Binary Pattern (LBP) features, as a commonly used texture feature description method, are widely applied in image processing. LBP features are simple to compute and robust to image noise and brightness variations, and have certain application prospects in the field of brain texture analysis. However, the window size of the LBP method is fixed (3×3) and is independent of the image content, which leads to errors in the extraction of texture primitive features and makes it difficult to adapt to the requirements of textures with different roughness and scale. Summary of the Invention

[0004] The purpose of this invention is to explore a brain texture analysis method based on LBP extraction and TCNN neural network classification. It improves the existing LBP method for feature extraction and adopts a new neural network architecture, TCNN, for texture classification, thereby improving the accuracy of brain texture feature extraction and classification. This invention has practical significance and promising prospects.

[0005] To achieve the above functions, this invention designs a brain texture analysis method based on LBP extraction and TCNN neural network, including the following steps S1-S5, to complete texture extraction and classification in brain structural magnetic resonance images:

[0006] Step S1: Acquire brain structural magnetic resonance images (MRI), preprocess them using MIPAV software, register the preprocessed brain structural MRI images onto the standard brain template space MNI, unify the coordinate space of each brain structural MRI image, and perform standardization processing.

[0007] Step S2: Construct a brain texture feature extraction model. For the brain structure magnetic resonance images obtained in step S1, the circular LBP method is used to extract texture features. Since the sampling points are distributed on a circle, it cannot be guaranteed that the coordinates of each sampling point are integers. For sampling points with integer coordinates, they are directly substituted into the formula for calculation. For sampling points with non-integer coordinates, the coordinates are rounded up and down, and their gray values ​​are calculated using bilinear interpolation.

[0008] Step S3: Based on the Transformer network and the CNN neural network, a TCNN neural network is formed, and a brain texture classification model is constructed. The brain structure magnetic resonance image after texture feature extraction is used as input, and the texture type corresponding to the texture in the brain structure magnetic resonance image is used as output to complete the texture classification.

[0009] Step S4: Train the brain texture classification model. Calculate the loss function of the brain texture classification model based on the original brain structure magnetic resonance image and the image after texture feature extraction, and update the weight parameters using the backpropagation algorithm.

[0010] Step S5: Repeat step S4 until the preset number of training iterations is reached to complete the training of the brain texture classification model. Apply the trained brain texture classification model to complete the texture extraction and classification in the brain structure magnetic resonance imaging.

[0011] As a preferred technical solution of the present invention: Step S1 specifically includes: preprocessing the acquired brain structural magnetic resonance images, including skull removal, correction, filtering and image enhancement; after preprocessing, registering the brain structural magnetic resonance images to the standard brain template space MNI, and adjusting and optimizing them by selecting the corresponding registration algorithm, adjusting the registration parameters, and adding registration constraints; after registration, performing Min-max normalization on the pixel values ​​of each sampling point in the registered brain structural magnetic resonance images; Min-Max normalization refers to performing a linear transformation on the original data to map the values ​​to the range [0, 1].

[0012] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0013] Step S2.1: For each sampling point in the brain structure magnetic resonance image obtained in step S1, divide the area into a circular neighborhood with radius R, centered on the sampling point. Select P equally divided points within the circular neighborhood. Use the lines connecting the central sampling point to each equally divided point as the trajectory for binary encoding. The formula for each sampling point on the trajectory is as follows:

[0014]

[0015]

[0016] Where p represents the p-th sampling point, x p ,y p Let x and y be the x and y coordinates of the p-th sampling point, respectively. c ,y c These are the x and y coordinates of the central sampling point c, respectively;

[0017] Step S2.2: For sampling points with non-integer coordinates, select the two closest sampling points p0 and p1 on the trajectory of sampling point p. Round the x and y coordinates of p0 and p1 to the nearest integers, and denote them as x0, x1, y0, and y1 respectively, obtaining four coordinates (x0, y0), (x1, y0), (x0, y1), and (x1, y1). Then calculate according to the following formula:

[0018]

[0019] Where, f(x) p ,y p () represents the pixel value of sampling point p;

[0020] Step S2.3: Using the gray value of the center sampling point as a threshold, compare the gray value of each sampling point on the trajectory with the gray value of the center sampling point to obtain a binary code; wherein the binary code is that the gray value of each sampling point on the trajectory is greater than the gray value of the center sampling point and is counted as 1, otherwise it is counted as 0. Arrange the obtained binary codes in a clockwise direction to obtain a binary number; calculate the frequency histogram of the binary numbers of all sampling points on the trajectory to obtain the LBP feature vector.

[0021] As a preferred technical solution of the present invention: In step S3, a brain texture classification model is constructed based on Transformer network and CNN neural network. The image is segmented into multiple patches, where i represents any position in the patch, and j represents any position in the 3×3 neighborhood centered at i. Its attention module introduces depthwise convolution, which is expressed as the following formula:

[0022]

[0023] Among them, Y i It is the output at position i. It is the local neighborhood of position i, w i-j X is the weight matrix between position i and position j. j It is the input at position j; in contrast, self-attention allows the receptive field to be not a local neighborhood and computes weights based on pairwise similarity.

[0024] The softmax function for the brain texture classification model is as follows:

[0025]

[0026] Among them, Y j Let j represent the output at position j, n represent a position in the global space where the input passes through, and l also represent a position in the global space where the input passes through. Representing the global space, X j The input is at position j, T represents the transpose of the matrix, and X... n The input is at position n, X l It is the input at position l;

[0027] Each patch is compared with other patches in the same image to generate an adaptive attention matrix. The global static convolutional kernel is added to the adaptive attention matrix before or after softmax initialization, as shown in the following formula:

[0028]

[0029] in, This represents the output of the function, where post represents the inversion formula, n represents a position in the global space where the input passes through, T represents the transpose of the matrix, l also represents a position in the global space where the input passes through, and X... j The input is at position j, X n The input is at position n, X l The input is at position l, w j-n It is the weight matrix between position j and position n.

[0030] As a preferred technical solution of the present invention: the backpropagation algorithm of the brain texture classification model in step S4 is to pass from the output layer to the input layer.

[0031] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0032] 1. The original LBP algorithm is improved by expanding the original 3×3 window neighborhood to an arbitrary neighborhood and replacing the square neighborhood with a circular neighborhood. The improved LBP method allows for any number of sampling points within a circular neighborhood of radius R, which can make texture extraction more accurate.

[0033] 2. A novel neural network architecture, TCNN, integrating CNN and Transformer, is proposed. It utilizes convolution operations and self-attention mechanisms for classification, which can take into account both local and global features without excessive computation, thus enhancing the training effect. Attached Figure Description

[0034] Figure 1 This is a flowchart of a brain texture analysis method based on LBP extraction and TCNN neural network provided according to an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0036] Reference Figure 1 The brain texture analysis method based on LBP extraction and TCNN neural network provided in this embodiment of the invention includes the following steps S1-S5, to complete texture extraction and classification in brain structural magnetic resonance images:

[0037] Step S1: Acquire brain structural magnetic resonance images (MRI), preprocess them using MIPAV software, register the preprocessed brain structural MRI images onto the standard brain template space MNI, unify the coordinate space of each brain structural MRI image, and perform standardization processing.

[0038] Step S1 specifically includes: preprocessing the acquired brain structural MRI images, including skull removal, correction, filtering, and image enhancement. After preprocessing, the brain structural MRI images are registered to the standard brain template space MNI, and adjusted and optimized. This involves selecting the appropriate registration algorithm, adjusting registration parameters, and adding registration constraints. After registration, the pixel values ​​of each sampling point in the registered brain structural MRI images are subjected to Min-max normalization to ensure their coordinate space is completely consistent with the standard brain template. After normalization, the processed brain structural MRI images are checked and verified to ensure image quality and accuracy. The formula for Min-max normalization is:

[0039]

[0040] Where ND represents the result of data normalization, OD represents the original data, MinA represents the minimum value among all data of attribute A, and MaxA represents the maximum value among all data of attribute A. Min-max normalization normalizes the pixel values ​​of each pixel in the brain structure magnetic resonance imaging to between 0 and 1, thus avoiding the influence of brightness differences between different images on texture features.

[0041] Step S2: Construct a brain texture feature extraction model. For the brain structure magnetic resonance image obtained in step S1, use the circular LBP method to extract texture features. Use bilinear interpolation to calculate the gray value of the brain structure magnetic resonance image. Obtain the feature vector corresponding to the brain structure magnetic resonance image based on the gray value to complete the texture feature extraction.

[0042] The specific steps of step S2 are as follows:

[0043] Step S2.1: For each sampling point in the brain structure magnetic resonance image obtained in step S1, divide the area into a circular neighborhood with radius R, centered on the sampling point. Select P equally divided points within the circular neighborhood. Use the lines connecting the central sampling point to each equally divided point as the trajectory for binary encoding. The formula for each sampling point on the trajectory is as follows:

[0044]

[0045]

[0046] Where p represents the p-th sampling point, x p ,y p Let x and y be the x and y coordinates of the p-th sampling point, respectively. c ,y c These are the x and y coordinates of the central sampling point c, respectively;

[0047] Step S2.2: When using the circular LBP algorithm, since the sampling points are distributed on a circle, it cannot be guaranteed that the coordinates of each sampling point are integers. For sampling points with non-integer coordinates, bilinear interpolation is required. If a sampling point on the trajectory does not fall at an integer position, its gray value is calculated using bilinear interpolation based on the gray values ​​of the two nearest integer pixel positions within the trajectory, denoted as p0 and p1. The horizontal and vertical coordinates of p0 and p1 are rounded up and down, respectively, and denoted as x0, x1, y0, y1, resulting in four coordinates: (x0, y0), (x1, y0), (x0, y1), (x1, y1). Then, the following formula is used for calculation:

[0048]

[0049] Where, f(x) p,y p () represents the pixel value of sampling point p;

[0050] Step S2.3: Using the gray value of the center sampling point as a threshold, compare the gray value of each sampling point on the trajectory with the gray value of the center sampling point to obtain a binary code; the binary code is set as follows: if the gray value of each sampling point on the trajectory is greater than the gray value of the center sampling point, it is counted as 1, otherwise it is counted as 0. Arrange the obtained binary codes in a clockwise direction to obtain a binary number; calculate the frequency histogram of the binary numbers of all sampling points on the trajectory to obtain the LBP feature vector; for different combinations of radius and number of neighborhoods, use statistical learning algorithms to perform classification or regression tasks to evaluate the performance of the improved LBP method.

[0051] Step S3: Based on the Transformer network and the CNN neural network, a TCNN neural network is formed, and a brain texture classification model is constructed. The brain structure magnetic resonance image after texture feature extraction is used as input, and the texture type corresponding to the texture in the brain structure magnetic resonance image is used as output to complete the texture classification.

[0052] To match the new dataset composed of brain structural MRI images with the image data format input to the brain texture classification model, the new dataset needs to be preprocessed and standardized to ensure that it has the same attributes and features as the original training dataset. Then, the parameters of the pre-trained brain texture classification model are initialized through random initialization. After the above steps are completed, the brain texture classification model is fine-tuned using the new dataset. The fine-tuning uses the backpropagation algorithm to update the parameters of the brain texture classification model to minimize the loss function. The fine-tuning process requires 30 iterations to obtain the corresponding evaluation accuracy to ensure the accuracy of the experiment. Then, after extracting the LBP features of the brain structural MRI images, new image patch embeddings are generated and input into the TCNN neural network based on the integration of Transformer and CNN.

[0053] In step S3, a brain texture classification model is constructed based on a Transformer network and a CNN neural network. The image is segmented into multiple patches, where i represents any position in the patch, and j represents any position in the 3×3 neighborhood centered at i. Its attention module introduces depthwise convolution, which is expressed as follows:

[0054]

[0055] Among them, Y i It is the output at position i. It is the local neighborhood of position i, w i-j X is the weight matrix between position i and position j. jIt is the input at position j; in contrast, self-attention allows the receptive field to be not a local neighborhood and computes weights based on pairwise similarity.

[0056] The softmax function for the brain texture classification model is as follows:

[0057]

[0058] Among them, Y j Let j represent the output at position j, n represent a position in the global space where the input passes through, and l also represent a position in the global space where the input passes through. Representing the global space, X j The input is at position j, T represents the transpose of the matrix, and X... n The input is at position n, X l It is the input at position l;

[0059] Each patch is compared with other patches in the same image to generate an adaptive attention matrix. The global static convolutional kernel is added to the adaptive attention matrix before or after softmax initialization, as shown in the following formula:

[0060]

[0061] in, This represents the output of the function, where post represents the inversion formula, n represents a position in the global space where the input passes through, T represents the transpose of the matrix, l also represents a position in the global space where the input passes through, and X... j The input is at position j, X n The input is at position n, X l The input is at position l, w j-n It is the weight matrix between position j and position n.

[0062] Step S4: Train the brain texture classification model. Calculate the loss function of the brain texture classification model based on the original brain structure magnetic resonance image and the image after texture feature extraction, and update the weight parameters using the backpropagation algorithm.

[0063] Step S4 involves calculating the output of the neural network, comparing it with the true labels, and calculating the loss function. The derivative of the loss function with respect to the network parameters is then calculated, and the chain rule is used to propagate the derivative from the output layer to the input layer, calculating the gradient of each parameter. Based on the calculated gradients, the neural network parameters are updated using gradient descent or other optimization algorithms. These steps are repeated until the specified stopping condition is met. The propagation pattern of multi-layer partial derivatives is as follows:

[0064] In step S4, the backpropagation algorithm of the brain texture classification model is passed from the output layer to the input layer. It only requires iteratively calculating the value of each node in each layer to obtain the partial derivative of the current layer, thereby obtaining the gradient of the weight matrix W of each layer. Then, the network parameters are iteratively optimized by using the gradient descent algorithm.

[0065] Step S5: Repeat step S4 until the preset number of training iterations is reached to complete the training of the brain texture classification model. Apply the trained brain texture classification model to extract and classify textures from brain structural magnetic resonance images. Specifically, input newly acquired brain structural magnetic resonance images into the brain texture feature extraction model to obtain their corresponding texture features; then input the texture features into the already trained brain texture classification model to obtain the corresponding classification results. When model validation is required, input multiple similar brain structural magnetic resonance images to be validated into the neural network and output the classification results. If the classification accuracy is better than the previous accuracy, it proves that the model can extract brain texture features more accurately and improve the accuracy of brain texture classification.

[0066] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A brain texture analysis method based on LBP extraction and TCNN neural network, characterized in that, The process includes the following steps S1-S5, which complete the texture extraction and classification from brain structural magnetic resonance images: Step S1: Acquire brain structural magnetic resonance images, preprocess them using MIPAV software, register the preprocessed brain structural magnetic resonance images onto the standard brain template space MNI, unify the coordinate space of each brain structural magnetic resonance image, and perform standardization processing using Min-Max. Step S2: Construct a brain texture feature extraction model. For the brain structure magnetic resonance images obtained in step S1, the circular LBP method is used to extract texture features. Since the sampling points are distributed on a circle, it cannot be guaranteed that the coordinates of each sampling point are integers. For sampling points with integer coordinates, they are directly substituted into the formula for calculation. For sampling points with non-integer coordinates, the coordinates are rounded up and down, and their gray values ​​are calculated using bilinear interpolation. Step S3: Construct a brain texture classification model based on the TCNN neural network. The model takes the brain structure magnetic resonance image after texture feature extraction as input and the texture type corresponding to the texture in the brain structure magnetic resonance image as output to complete the texture classification. In step S3, a brain texture classification model is constructed based on a Transformer network and a CNN neural network. The image is segmented into multiple patches, where i represents any position in the patch, and j represents any position in the 3×3 neighborhood centered at i. Its attention module introduces depthwise convolution, which is expressed as follows: Among them, Y u It is the output at position i. It is the local neighborhood of position i, w i-j X is the weight matrix between position i and position j. j It is the input at position j; in contrast, self-attention allows the receptive field to be not a local neighborhood and computes weights based on pairwise similarity. The softmax function for the brain texture classification model is as follows: Among them, Y j Let j represent the output at position j, n represent a position in the global space where the input passes through, and l also represent a position in the global space where the input passes through. Representing the global space, X j The input is at position j, T represents the transpose of the matrix, and X... n The input is at position n, X l It is the input at position l; Each patch is compared with other patches in the same image to generate an adaptive attention matrix. The global static convolutional kernel is added to the adaptive attention matrix before or after softmax initialization, as shown in the following formula: in, This represents the output of the function, where post represents the inversion formula, n represents a position in the global space where the input passes through, T represents the transpose of the matrix, l also represents a position in the global space where the input passes through, and X... j The input is at position j, X n The input is at position n, X l The input is at position l, w j-n It is the weight matrix between position j and position n; Step S4: Train the brain texture classification model. Calculate the loss function of the brain texture classification model based on the original brain structure magnetic resonance image and the image after texture feature extraction, and update the weight parameters using the backpropagation algorithm. Step S5: Repeat step S4 until the preset number of training iterations is reached to complete the training of the brain texture classification model. Apply the trained brain texture classification model to complete the texture extraction and classification in the brain structure magnetic resonance imaging.

2. The brain texture analysis method based on LBP extraction and TCNN neural network according to claim 1, characterized in that, Step S1 specifically includes: preprocessing the acquired brain structural magnetic resonance images, including skull removal, correction, filtering, and image enhancement. After preprocessing, the brain structural magnetic resonance images are registered onto the standard brain template space MNI, and adjusted and optimized. The corresponding registration algorithm is selected, the registration parameters are adjusted, and registration constraints are added. After registration, the pixel values ​​of each sampling point in the registered brain structural magnetic resonance images are subjected to Min-Max normalization. Min-Max normalization refers to performing a linear transformation on the original data to map the values ​​to the range [0, 1].

3. The brain texture analysis method based on LBP extraction and TCNN neural network according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S2.1: For each sampling point in the brain structure magnetic resonance image obtained in step S1, divide the area into a circular neighborhood with radius R, centered on the sampling point. Select P equally divided points within the circular neighborhood. Use the lines connecting the central sampling point to each equally divided point as the trajectory for binary encoding. The formula for each sampling point on the trajectory is as follows: Where p represents the p-th sampling point, x p ,y p Let x and y be the x and y coordinates of the p-th sampling point, respectively. c ,y c These are the x and y coordinates of the central sampling point c, respectively; Step S2.2: For sampling points with non-integer coordinates, select the two closest sampling points p0 and p1 on the trajectory of sampling point p. Round the x and y coordinates of p0 and p1 to the nearest integers, and denote them as x0, x1, y0, and y1 respectively, obtaining four coordinates (x0, y0), (x1, y0), (x0, y1), and (x1, y1). Then calculate according to the following formula: Where, f(x) p ,y p () represents the pixel value of sampling point p; Step S2.3: Using the gray value of the center sampling point as a threshold, compare the gray value of each sampling point on the trajectory with the gray value of the center sampling point to obtain a binary code; wherein the binary code is that the gray value of each sampling point on the trajectory is greater than the gray value of the center sampling point and is counted as 1, otherwise it is counted as 0. Arrange the obtained binary codes in a clockwise direction to obtain a binary number; calculate the frequency histogram of the binary numbers of all sampling points on the trajectory to obtain the LBP feature vector.

4. The brain texture analysis method based on LBP extraction and TCNN neural network according to claim 1, characterized in that, In step S4, the backpropagation algorithm for the brain texture classification model is passed from the output layer to the input layer.

Images