Liver fibrosis prediction method based on medical image recognition

Abstract

The invention proposes a liver fibrosis prediction method based on medical image recognition, and relates to the technical field of image recognition, and the method comprises the steps: S1, carrying out the backtracking collection of a multi-modal medical image set from the historical medical record data of definite fibrosis, and enabling the medical image set to correspond to the definite fibrosis degree, so as to generate an image-label pair; s2, preprocessing the medical image set, including denoising, contrast enhancement and liver region segmentation; s3, a prediction model is constructed, the prediction model takes a multi-path attention U-Net network as a framework, the preprocessed medical image set is divided into a training set and a verification set, iterative training is carried out on the prediction model until a loss function converges, and the trained prediction model is obtained; and S4, obtaining a new medical image, and inputting the new medical image into the trained prediction model to obtain a liver fibrosis prediction result. The method can overcome the problems of separation of segmentation and classification tasks, insufficient evaluation precision and the like in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method for predicting liver fibrosis based on medical image recognition. Background Art

[0002] Liver fibrosis is an important pathological stage in the development of chronic liver diseases, mainly manifested as abnormal deposition of collagen fibers and extracellular matrix in liver tissue. Among them, non-alcoholic fatty liver disease (NAFLD) has become one of the most common chronic liver diseases globally and is also an important cause of liver fibrosis. Research shows that about 20-30% of NAFLD patients may develop liver fibrosis and may further progress to cirrhosis, or even lead to liver failure and liver cancer. Therefore, accurately assessing the presence of liver fibrosis has important clinical significance for early diagnosis of diseases, formulation of treatment plans, and prognosis evaluation.

[0003] Currently, the methods for clinically evaluating the degree of liver fibrosis mainly include liver biopsy, serological index detection, and medical imaging evaluation. Among them, liver biopsy is considered the gold standard for evaluating the degree of liver fibrosis. However, as an invasive operation, liver biopsy has risks such as complex operation, patient discomfort, and postoperative complications, which limit its wide application in clinical practice. In addition, the sample size of liver biopsy is limited, and there may be problems with insufficient sample representativeness, thus affecting the accuracy of the evaluation results. This limitation is particularly prominent in the evaluation of NAFLD patients because NAFLD often shows the characteristic of uneven distribution.

[0004] With the development of medical imaging technology, non-invasive imaging methods have been increasingly widely used in the evaluation of liver fibrosis. Commonly used imaging techniques include MRI, CT, and B-ultrasound imaging. These imaging techniques can provide detailed information on the structure and function of the liver, providing a visual basis for the evaluation of the degree of fibrosis.

[0005] In recent years, with the rapid development of artificial intelligence, especially deep learning technology, automated liver fibrosis prediction methods based on medical images have received extensive attention. Through feature extraction and learning of a large amount of image data, deep learning models can, to a certain extent, achieve automatic recognition and evaluation of the liver and suspected fibrosis regions. However, existing automated evaluation methods still face many challenges, including the lack of fibrosis image samples in the medical field, low evaluation accuracy, and separation of segmentation and classification tasks. Summary of the Invention

[0006] In view of this, the present invention proposes a method for predicting liver fibrosis based on medical image recognition. By constructing medical image training data, preprocessing medical image data, constructing a prediction model with a multi-path attention U-Net network framework, and combining a segmentation and classification dual-task output module, accurate segmentation of the liver and fibrosis regions and accurate prediction of fibrosis risk classification are achieved, thereby overcoming problems such as the separation of segmentation and classification tasks and insufficient evaluation accuracy in the prior art.

[0007] The technical solution of the present invention is realized as follows:

[0008] The present invention provides a method for predicting liver fibrosis based on medical image recognition, including:

[0009] S1. Retrospectively collect a multi-modal medical image set from the historical medical record data of patients with diagnosed fibrosis, and correspond the medical image set with the diagnosed fibrosis degree to generate image-label pairs;

[0010] S2. Preprocess the medical image set, including denoising, contrast enhancement, and liver region segmentation;

[0011] S3. Construct a prediction model. The prediction model is based on a multi-path attention U-Net network framework. Divide the preprocessed medical image set into a training set and a validation set, and iteratively train the prediction model until the loss function converges to obtain a trained prediction model;

[0012] S4. Obtain a new medical image, input it into the trained prediction model, and obtain the liver fibrosis prediction result.

[0013] Preferably, step S2 includes:

[0014] S21. Smooth the medical image set using Gaussian filtering to obtain a denoised medical image set;

[0015] S22. Perform histogram equalization on the denoised medical image set to obtain a medical image set with enhanced contrast;

[0016] S23. Use superpixel segmentation to perform preliminary classification on the medical image set with enhanced contrast, and divide each medical image into several superpixel blocks with similar features;

[0017] S24. Perform binarization processing on the superpixel blocks of each medical image, identify and screen the target region through the connected component labeling algorithm, and obtain the liver segmentation result.

[0018] Preferably, step S23 includes:

[0019] S231. Select a medical image;

[0020] S232. Convert the medical image into a six-dimensional clustering space (L, a, b, x, y, T) that includes texture features, where (L, a, b) are color channels, (x, y) are pixel coordinates, and T is the texture feature;

[0021] S233. Set the number of superpixels K and calculate the initial clustering distance where N is the total number of pixels;

[0022] S234. Grid the medical image and uniformly sample on the grid to obtain the initial clustering centers C k , and define the comprehensive distance D as:

[0023]

[0024] where is the color distance; is the spatial distance; d t = ‖T i - T j ‖ is the texture distance; λ c , λ s , λ t are normalization coefficients;

[0025] S235. In the 2S×2S neighborhood of each clustering center C k , update the pixel attribution and clustering center according to the principle of the minimum comprehensive distance D;

[0026] S236. If the change in the clustering center is less than the threshold, stop the iteration and obtain the superpixel segmentation result of the medical image;

[0027] S237. Repeat steps S231 - S236 to perform superpixel segmentation on all medical images and divide all medical images into several superpixel blocks with similar features.

[0028] Preferably, step S24 includes:

[0029] S241. Set the adaptive threshold τ = μ + kσ according to the average gray value of the superpixel block, where μ is the pixel mean of the superpixel block, σ is the pixel standard deviation of the superpixel block, and k is an adjustment factor; perform binarization on the superpixel block using the adaptive threshold τ to obtain a binary image;

[0030] S242. Use the eight-connected algorithm to identify the connected regions in the binary image:

[0031] Traverse the binary image and check the eight neighboring pixels of each pixel (x, y):

[0032]

[0033] If the neighborhood pixel has the same value as the current pixel, they are assigned to the same connected component;

[0034] S243. Calculate the features of each connected component, including area A, perimeter P, shape factor S, aspect ratio R, and centroid position Among them, the area A is the total number of pixels in the connected component, the perimeter P is the number of pixels on the boundary of the connected component, and the shape factor reflects the circularity of the connected component, and the aspect ratio L max and L min are the maximum and minimum axis lengths of the connected component, and the centroid position

[0035] S244. Use the scoring function Q to screen the connected components:

[0036]

[0037] In the formula, A max is the maximum area, (x c , y c ) is the central coordinate of the possible position of the liver in the connected component, W and H are the width and height of the connected component, and α1, α2, α3, α4 are weight coefficients;

[0038] S245. According to the scoring function, select the connected component with the highest score value as the liver candidate region;

[0039] S246. Smooth the liver candidate region according to morphological operations, and use the active contour model to adjust the liver boundary:

[0040]

[0041] In the formula, C is the contour curve, k is the curvature, N is the normal vector, is the image gradient, and a and b are adjustment parameters;

[0042] S247. Mark the processed liver candidate region as 1, and mark other regions as 0 to generate a liver mask, and multiply the mask by the original superpixel block to obtain the liver segmentation result.

[0043] Preferably, the prediction model includes:

[0044] The backbone network consists of an encoder and a decoder, and a skip connection is set between the encoder and the decoder; the encoder part has multi-resolution branches, which are respectively used to extract high-resolution local features and low-resolution global features, and the multi-resolution branches are all composed of convolutional layers and pooling layers; the decoder part has a dynamic spatial attention fusion module, which weights and fuses the features of the multi-resolution branches of the encoder;

[0045] The feature enrichment mechanism has a feature sampling layer after the decoder output, which divides the two-dimensional feature map into several small blocks and converts them into a feature sequence; combines the feature sequence and the global feature to complete feature recombination and fusion, and forms a depth feature representation that is pixel-level segmented and globally compatible;

[0046] The dual-task output module, the first branch is the segmentation branch, which is used to output the pixel-level prediction results of the liver and fibrosis regions; the second branch is the classification branch, which uses the global feature vector to complete the classification prediction of fibrosis.

[0047] Preferably, the training process of the prediction model is as follows:

[0048] Divide the preprocessed medical image set into a training set and a validation set;

[0049] Use the Xavier initialization method to initialize the parameters of the prediction model, set the Adam optimizer, set the initial learning rate to η0, and adjust the learning rate according to the number of training epochs; define the loss function, including segmentation loss, grading loss and collaborative loss;

[0050] Training process:

[0051] Step 1: Randomly extract a batch of medical images from the training set;

[0052] Step 2: Input the batch of images into the prediction model, and successively pass through the encoder, skip connection, decoder and dual-task output module to obtain the predicted segmentation result and the predicted classification result;

[0053] Step 3: Calculate the loss according to the predicted segmentation result and the predicted classification result and the true label;

[0054] Step 4: Calculate the gradient through the backpropagation algorithm and update the model parameters using the optimizer;

[0055] Step 5: Adopt the gradient clipping method to limit the gradient norm within the threshold range;

[0056] Step 6: Repeat Step 1 - Step 5, and every h epochs, evaluate the model performance on the validation set and adjust the learning rate according to the loss on the validation set;

[0057] Until the loss function converges, the training ends and a trained prediction model is obtained.

[0058] Preferably, in step 6, the model performance is evaluated on the validation set, including:

[0059] Every h training rounds, input the model in batches in the validation set, and obtain the corresponding predicted segmentation results and classification results one by one;

[0060] Calculate the segmentation performance metrics, including: Dice coefficient, which is used to measure the overlap degree between the predicted segmentation and the real segmentation region; IoU intersection over union, which is used to evaluate the ratio of the intersection to the union of the segmentation regions;

[0061] Calculate the classification performance metrics, including: classification accuracy, which is used to measure the overall consistency between the predicted classification label and the real label; weighted F1 value, which is used to comprehensively evaluate the prediction performance of the classification task;

[0062] Record and calculate the loss of the validation set;

[0063] Monitor the training effect of the model according to the validation set performance metrics. If it is detected that the total validation loss has not decreased for m consecutive times, adjust the learning rate and perform iterative training again.

[0064] Preferably, the loss function during the training of the prediction model is as follows:

[0065]

[0066] L seg = L region + δ·L edge

[0067] L region = E[(P i,j - G i,j ) 2 · ω i,j

[0068]

[0069] In the formula, is the total loss; L seg is the segmentation loss; L class is the classification loss; L synergy is the collaborative loss; β1, β2, β3 are the weights of each loss; L region is the regional loss; L edge is the edge loss; δ is the weight coefficient of the edge loss; P i,j is the segmentation prediction probability of the (i, j) pixel; G i,j is the true label of the (i, j) pixel; ω i,j = 1 + λ1·A i,j is the adaptive weight coefficient, A​i,j is the attention weight for pixel (i,j), λ1 is a regulation parameter; E[·] represents expectation; C (i,j) is the degree of edge response; and are the edge gradient values of the predicted segmentation and the ground truth segmentation respectively; ω′ i,j = 1 + λ2·C i,j is the edge adaptive weight coefficient, λ2 is a regulation parameter; L CE is the cross-entropy loss; K is the total number of samples, y k is the ground truth label of the k-th sample, taking values 0 or 1, p k is predicted for the k-th sample; F seg is the feature representation of the segmentation branch; F class is the feature representation of the classification branch; is the average value of the attention weights of the fibrotic region, M is the total number of pixels.

[0070] Preferably, the medical image set includes MRI, CT, and B-ultrasound images.

[0071] The present invention has the following beneficial effects compared with the prior art:

[0072] (1) The present invention provides a method for predicting liver fibrosis based on medical image recognition. By preprocessing medical image data, constructing a prediction model with a multi-path attention U-Net network framework, and combining a dual-task output module of segmentation and classification, accurate segmentation of the liver and fibrotic regions and prediction of fibrosis risk are achieved. This method overcomes the problems of separation of segmentation and classification tasks and insufficient evaluation accuracy in the prior art;

[0073] (2) The present invention preprocesses medical image data by denoising, contrast enhancement, and liver region segmentation, improving the image quality and the distinguishability of the target region. Among them, Gaussian filtering effectively removes noise interference, histogram equalization enhances the contrast of the image, and superpixel segmentation combined with the connected component labeling algorithm realizes accurate extraction of the liver region;

[0074] (3) The present invention uses a multi-path attention U-Net network as the framework of the prediction model. Through the skip connection between the encoder and the decoder and the dynamic spatial attention fusion module, weighted fusion of multi-resolution features is achieved. This design can simultaneously extract high-resolution local features and low-resolution global features, enhancing the model's recognition ability for fibrotic regions. Especially in the case of complex tissue structures and blurred boundaries, the segmentation accuracy is significantly improved;

[0075] (4) The present invention designs a dual-task output module for segmentation and classification. Through the segmentation branch, pixel-level prediction of the liver and fibrosis regions is achieved, and through the classification branch, classification prediction is completed using the global feature vector. This module organically combines the segmentation task and the classification task, realizes information sharing and mutual promotion, and improves the accuracy and robustness of fibrosis prediction;

[0076] (5) The present invention defines a comprehensive loss function that includes segmentation loss, classification loss, and collaborative loss. The accuracy of the segmentation task is improved by combining region loss and edge loss; the ability to distinguish fibrosis risk in the classification task is enhanced through binary cross-entropy loss; and the feature consistency between the segmentation branch and the classification branch is constrained by the collaborative loss, promoting the collaborative optimization of the dual tasks, thereby effectively improving the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0078] Figure 1 is the flowchart of the method according to the embodiment of the present invention;

[0079] Figure 2 is the model structure diagram according to the embodiment of the present invention;

[0080] Figure 3 is the structure diagram of the encoding block according to the embodiment of the present invention;

[0081] Figure 4 is the structure diagram of the decoding block according to the embodiment of the present invention;

[0082] Figure 5 is the structure diagram of the feature recalibration module according to the embodiment of the present invention;

[0083] Figure 6 is the structure diagram of the dual-task output module according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0084] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0085] Such as Figure 1As shown, the present invention provides a method for predicting liver fibrosis based on medical image recognition, including:

[0086] S1. Retrospectively collect a multi-modal medical image set from the historical medical record data of patients with diagnosed fibrosis, and correspond the medical image set with the diagnosed fibrosis degree to generate image-label pairs;

[0087] S2. Preprocess the medical image set, including denoising, contrast enhancement, and liver region segmentation;

[0088] S3. Construct a prediction model. The prediction model is based on the multi-path attention U-Net network. Divide the preprocessed medical image set into a training set and a validation set, and iteratively train the prediction model until the loss function converges to obtain a trained prediction model;

[0089] S4. Obtain new medical images, input them into the trained prediction model, and obtain the liver fibrosis prediction results.

[0090] Specifically, in an embodiment of the present invention, the medical image set includes medical images of three imaging modalities: MRI, CT, and B-ultrasound images.

[0091] In this embodiment, the implementation process of step S1 is as follows:

[0092] Select patients diagnosed with liver cirrhosis as the research object. The diagnosis basis should conform to authoritative medical guidelines, such as pathological examinations, imaging diagnosis results, etc. Select liver cirrhosis cases diagnosed within the past five years to ensure the modernity and relevance of the image data. Ensure that the medical record data of the selected patients is complete, including various medical examination records before diagnosis. It should be noted that in this embodiment, the captured medical record data is data that has been consented by the patients and approved by relevant departments for medical research. When collecting data, ensure that the patient's privacy information is desensitized.

[0093] Collect multi-modal medical images, including but not limited to MRI, CT, and B-ultrasound images. Ensure that all relevant image data of each patient in the pre-diagnosis period is included in the collection scope. For each patient diagnosed with liver cirrhosis, retrospectively collect all relevant medical images at least one year before diagnosis. These images may be marked as abnormal or suspected fibrosis at the initial stage of the patient's symptoms or during physical examinations. Screen out the images marked as "abnormal" or "suspected fibrosis" before diagnosis.

[0094] Correspond each screened abnormal or suspected fibrosis image with the patient's finally diagnosed liver cirrhosis degree. The "labels" here mainly include two categories: "no fibrosis risk" and "suspected fibrosis".

[0095] Each image serves as an independent data sample, and its label is determined based on the patient's final diagnosis result. For example, if a patient is finally diagnosed with liver cirrhosis, the label corresponding to the image previously marked as suspected fibrosis is "suspected fibrosis". For images without abnormalities found during the diagnosis process, the corresponding label is "no fibrosis risk".

[0096] The collected images are reviewed by human experts, and a dual or multiple annotation mechanism is adopted. Different experts independently review the same image to ensure the consistency and accuracy of the labels. In case of disagreements, they are resolved through discussion or by a third-party expert.

[0097] All the collected images are converted into a unified digital format (such as DICOM format), and the image-label pairs are stored in a structured database. A multi-level backup mechanism is established to ensure the security and integrity of the data and prevent data loss or damage.

[0098] Specifically, in one embodiment of the present invention, step S2 includes:

[0099] S21. Smooth the medical image set using Gaussian filtering to obtain a denoised medical image set.

[0100] During the medical image processing, due to the influence of imaging equipment and environmental factors, the original images often contain different degrees of noise, which will affect subsequent feature extraction and analysis. The present invention uses a Gaussian filter to denoise the medical images, and its specific implementation process is as follows:

[0101] Design a two-dimensional Gaussian kernel function:

[0102]

[0103] where (x, y) represents the coordinates of the pixel relative to the center of the filter kernel, and σ is the standard deviation of the Gaussian distribution, which controls the smoothness degree. Select an appropriate kernel size according to actual needs, usually an odd number, such as 3×3, 5×5 or 7×7.

[0104] For each pixel point (i, j) in the medical image, extract its neighborhood pixels with this point as the center; perform a convolution operation on the Gaussian kernel and the neighborhood pixels:

[0105]

[0106] where I'(i, j) is the filtered pixel value, I(i + x, j + y) is the original pixel value, and k is the radius of the filter kernel.

[0107] Specifically, for MRI images: select a relatively small σ value (such as σ = 0.8 - 1.2) and kernel size (3×3 or 5×5) to preserve image details; for CT images: a moderate σ value (such as σ = 1.0 - 1.5) and kernel size (5×5) can be selected to balance the denoising effect and edge preservation; for elastography of ultrasound: due to the large amount of noise, a relatively large σ value (such as σ = 1.5 - 2.0) and kernel size (7×7) can be selected to enhance the smoothing effect.

[0108] Process the edge region of the image using boundary padding; zero padding, boundary replication, or mirror padding can be selected. In this embodiment, mirror padding is used to maintain the continuity of the boundary region.

[0109] Calculate the signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR) before and after filtering; observe the degree of edge preservation to ensure that image details are not overly blurred while removing noise; according to the evaluation results, appropriately adjust the filter parameters to obtain the best denoising effect.

[0110] Through the above Gaussian filtering process, the noise interference in medical images can be effectively reduced. At the same time, due to the isotropic property of Gaussian filtering, the edges and structural information of the image can be better preserved, providing image data with better quality for subsequent contrast enhancement and region segmentation. For medical images of different modalities, by appropriately adjusting the parameters of the Gaussian filter, the optimal denoising effect can be achieved, improving the accuracy and reliability of subsequent processing.

[0111] S22. Perform histogram equalization on the denoised medical image set to obtain a medical image set with enhanced contrast.

[0112] During the acquisition process of medical images, the contrast may be insufficient due to factors such as imaging conditions and equipment parameters. The present invention uses the histogram equalization method to enhance the contrast of the denoised medical images. The specific implementation process is as follows:

[0113] Calculate the gray-level histogram of the original image:

[0114]

[0115] where f(i, j) is the gray value of pixel (i, j), k is the gray level (0 - 255), and δ(x, y) is the Kronecker function;

[0116] Calculate the normalized histogram:

[0117]

[0118] where M×N is the total number of pixels in the image.

[0119] Calculate the cumulative distribution function of the gray level:

[0120]

[0121] Among them, k is the current gray level, and c(k) represents the proportion of pixels with gray values less than or equal to k.

[0122] Establish the mapping relationship between the original gray value and the new gray value:

[0123] s(k) = round((L - 1) × c(k))

[0124] Among them, L is the number of gray levels (usually 256), and round represents the rounding operation;

[0125] Perform gray value mapping on each pixel in the image:

[0126] g(i, j) = s(f(i, j))

[0127] Among them, g(i, j) is the pixel value after equalization.

[0128] To avoid detail loss caused by over-enhancement, local adaptive histogram equalization is adopted: the image is divided into several overlapping sub-blocks; histogram equalization is performed on each sub-block separately; the results of adjacent sub-blocks are fused using the bilinear interpolation method.

[0129] Introduce the contrast limit factor α to limit the peak value of the histogram: Allocate the exceeded part evenly to other gray levels; recalculate the cumulative distribution function and perform gray mapping.

[0130] For MRI images: Emphasize enhancing the contrast between tissues, and α takes a smaller value (such as 2 - 3); for CT images: Maintain the relative relationship of Hounsfield values, and α takes a moderate value (such as 3 - 4); for ultrasound elastography: Focus on improving the visibility of the fibrotic region, and α takes a larger value (such as 4 - 5).

[0131] Calculate the image entropy before and after enhancement: Calculate the contrast gain: Among them, std represents the standard deviation; adjust the processing parameters according to the evaluation results to ensure the enhancement effect without introducing artifacts.

[0132] Through the above histogram equalization processing, the contrast of medical images can be effectively improved, making the boundary between the liver region and the surrounding tissues clearer and the fibrotic features more prominent. This method can adaptively process medical images of different modalities, providing a better image basis for subsequent superpixel segmentation. At the same time, by introducing contrast limitation and local adaptive processing, the negative impacts brought by over-enhancement can be avoided.

[0133] S23. Use superpixel segmentation to perform preliminary classification on the contrast-enhanced medical image set, and divide each medical image into several superpixel blocks with similar features.

[0134] Specifically, step S23 includes:

[0135] S231. Select a medical image;

[0136] S232. Convert the medical image to a six-dimensional clustering space (L, a, b, x, y, T) that includes texture features, where (L, a, b) are color channels, where L is the luminance channel, a and b are chrominance channels, (x, y) are pixel coordinates, and T is the texture feature, which is extracted by methods such as local binary pattern (LBP) or gray-level co-occurrence matrix (GLCM);

[0137] S233. Set the number of superpixels K and calculate the initial clustering distance where N is the total number of pixels;

[0138] S234. Grid the medical image and uniformly sample on the grid to obtain the initial clustering centers C k , and define the comprehensive distance D as:

[0139]

[0140] where, is the color distance; is the spatial distance; d t = ‖T i - T j ‖ is the texture distance; λ c , λ s , λ t are normalization coefficients;

[0141] S235. Search within the 2S×2S neighborhood of each clustering center C k ; calculate the comprehensive distance D between each pixel in the neighborhood and the clustering center; update the pixel attribution according to the principle of the minimum distance; recalculate the position and features of the clustering center;

[0142] S236. If the change in the clustering center is less than the threshold, stop the iteration and obtain the superpixel segmentation result of the medical image;

[0143] S237. Repeat steps S231 - S236 to perform superpixel segmentation on all medical images, and divide all medical images into several superpixel blocks with similar features.

[0144] Specifically, calculate the statistical features for each superpixel block, including the average color value, spatial centroid, and texture descriptor, and record the boundary pixel set and adjacency relationship of the superpixel block.

[0145] Merge overly small superpixel blocks: Set an area threshold A min ; Merge superpixel blocks smaller than the threshold into the most similar adjacent block.

[0146] Through the above superpixel segmentation process, a medical image can be divided into local regions with similar features. These regions preserve the edge information of the image and have good spatial continuity.

[0147] S24. Perform binarization on the superpixel blocks of each medical image, identify and screen the target region through the connected component labeling algorithm, and obtain the liver segmentation result.

[0148] Specifically, step S24 includes:

[0149] S241. Set an adaptive threshold τ = μ + kσ according to the average gray value of the superpixel block, where μ is the pixel mean of the superpixel block, σ is the pixel standard deviation of the superpixel block, and k is an adjustment factor adjusted according to the characteristics of different modality images; use the adaptive threshold τ to perform binarization on the superpixel block to obtain a binary image; if the pixel value is greater than τ, set it to 1; if the pixel value is less than or equal to τ, set it to 0.

[0150] S242. Use the eight-connected algorithm to identify the connected regions in the binary image:

[0151] Traverse the binary image, for each pixel (x, y), check its eight neighboring pixels:

[0152]

[0153] If the neighboring pixel has the same value as the current pixel, they are classified into the same connected region; assign a unique identifier to each connected region.

[0154] S243. Calculate the features of each connected region, including area A, perimeter P, shape factor S, aspect ratio R, and centroid position where the area A is the total number of pixels in the connected region, the perimeter P is the number of pixels on the boundary of the connected region, the shape factor reflects the circularity of the connected region, and the aspect ratio L max and L min are the maximum and minimum axis lengths of the connected region, and the centroid position

[0155] S244. Use a scoring function Q to screen the connected regions:

[0156]

[0157] Wherein, A max is the maximum area, (x c , y c ) is the central coordinate of the possible position of the liver in the connected region, W and H are the width and height of the connected region, and α1, α2, α3, and α4 are weight coefficients.

[0158] S245. According to the scoring function, select the connected region with the highest score value as the liver candidate region.

[0159] S246. Smooth the liver candidate region according to morphological operations. The opening operation removes small protrusions, and the closing operation fills small holes. And use the active contour model to adjust the liver boundary:

[0160]

[0161] Wherein, C is the contour curve, κ is the curvature, N is the normal vector, is the image gradient, and a and b are adjustment parameters.

[0162] S247. Mark the processed liver candidate region as 1, and mark other regions as 0 to generate a liver mask. Multiply the mask by the original superpixel block to obtain the liver segmentation result.

[0163] Through the above processing steps, the present invention can achieve accurate extraction of the liver region. The use of the adaptive threshold ensures the adaptability of the binarization process to different image features; the connected component analysis combined with the multi-feature scoring function provides a reliable candidate region screening mechanism; the application of the active contour model further optimizes the accuracy of the segmentation boundary.

[0164] Specifically, as Figure 2 shown, in an embodiment of the present invention, the prediction model includes:

[0165] The backbone network is composed of an encoder and a decoder, and a skip connection is arranged between the encoder and the decoder; the encoder part is provided with multi-resolution branches for extracting high-resolution local features and low-resolution global features respectively, and the multi-resolution branches are all composed of convolutional layers and pooling layers; the decoder part is provided with a dynamic spatial attention fusion module for weighted fusion of the features of the multi-resolution branches of the encoder;

[0166] The feature enrichment mechanism is provided with a feature sampling layer after the decoder output, which divides the two-dimensional feature map into several small blocks and converts them into a feature sequence; combines the feature sequence and the global feature to complete feature recombination and fusion, and forms a depth feature representation that is compatible with pixel-level segmentation and global;

[0167] Dual-task output module. The first branch is the segmentation branch, which is used to output pixel-level prediction results of the liver and fibrosis regions; the second branch is the classification branch, which uses the global feature vector to complete the classification prediction of fibrosis.

[0168] Specifically, the encoder contains 5 encoding blocks (E1 - E5), and the structure of the encoding block is as Figure 3 shown. Each encoding block adopts a dual-path design: High-resolution branch = 3×3 convolutional layer → BN → ReLU → 3×3 convolutional layer → BN → ReLU → Spatial attention module; Low-resolution branch = 3×3 convolutional layer (stride = 2) → BN → ReLU → 3×3 convolutional layer → BN → ReLU → Spatial attention module.

[0169] The output channels of each encoding block are successively: 64, 128, 256, 512, 1024. The high-resolution branch maintains the original spatial resolution and extracts local detailed features; the low-resolution branch downsamples through stride convolution to extract global semantic features.

[0170] Structure of the spatial attention module: Feature map F → {AvgPool(F), MaxPool(F)} → 1×1 convolution → Sigmoid → Feature weighting.

[0171] The decoder contains 4 decoding blocks (D1 - D4), and the structure of the decoding block is as Figure 4 shown. The specific structure of each decoding block is: Decoding block = Transposed convolution upsampling → Dynamic spatial attention fusion → 3×3 convolutional layer → BN → ReLU → 3×3 convolutional layer → BN → ReLU. The output channels of each decoding block are successively: 512, 256, 128, 64, and the size of the feature map is increased to 2 times the original after upsampling.

[0172] Skip connection design: Connect corresponding levels: E1 → D4, E2 → D3, E3 → D2, E4 → D1.

[0173] The feature recalibration module is as Figure 5 shown. Its processing process is: Input feature → SE module → Spatial attention → 1×1 convolution → Output feature. SE module = GlobalAvgPool → FC → ReLU → FC → Sigmoid → Channel weighting. Specifically, the feature recalibration module is set on the paths of each skip connection.

[0174] The dynamic spatial attention fusion module is used to fuse the features of two branches, one from the skip connection and the other from the upsampling output. Its design is as follows:

[0175] Skip feature F h and upsampling feature F I :

[0176] Feature mapping: W h = MLP(F h ), W I = MLP(F I ); Attention weight: α = softmax([W h ; W I ); Weighted fusion: F = α1·F h + α2·F I .

[0177] The feature map F output by the decoder belongs to R H×W×C , and it is input to the feature sampling layer:

[0178] 1. Feature map partitioning: Divide F into N×N small blocks;

[0179] 2. Feature extraction: Perform average pooling on each small block;

[0180] 3. Sequence generation: Rearrange the pooling results into a sequence S;

[0181] 4. Position encoding: Add position information PE;

[0182] 5. Feature enhancement: S′ = S + PE.

[0183] The dual-task output module is as Figure 6 shown, where the segmentation branch: feature map → 1×1 convolution (number of channels = 2) → bilinear interpolation → Softmax → segmentation probability map; classification branch: global feature → global average pooling → FC(1024) → ReLU → Dropout(0.5) → FC(512) → ReLU → Dropout(0.5) → FC(2) → Softmax.

[0184] Through the above detailed network structure design, the present invention realizes: multi-resolution feature extraction, enhancing the model's perception ability of features at different scales; dynamic feature fusion, improving the feature utilization efficiency; feature serialization processing, enhancing the feature expression ability; dual-task collaborative learning, realizing the joint optimization of segmentation and classification. This structure design fully considers the characteristics of medical image analysis, can effectively extract and utilize the multi-scale features of images, and realizes accurate liver fibrosis segmentation and risk classification prediction.

[0185] Specifically, in an embodiment of the present invention, the training process of the prediction model is as follows:

[0186] Divide the preprocessed medical image set into a training set and a validation set;

[0187] Initialize the parameters of the prediction model using the Xavier initialization method, set the Adam optimizer, set the initial learning rate to η0, and adjust the learning rate according to the number of training epochs; define the loss functions, including segmentation loss, classification loss, and collaborative loss;

[0188] Training process:

[0189] Step 1: Randomly extract a batch of medical images from the training set;

[0190] Step 2: Input the batch of images into the prediction model, and successively pass through the encoder, skip connection, decoder, and dual-task output module to obtain the predicted segmentation result and the predicted classification result;

[0191] Step 3: Calculate the loss based on the predicted segmentation result and the predicted classification result and the ground truth labels;

[0192] Step 4: Calculate the gradients through the backpropagation algorithm and update the model parameters using the optimizer;

[0193] Step 5: Adopt the gradient clipping method to limit the gradient norm within the threshold range;

[0194] Step 6: Repeat Steps 1 - 5. Every h epochs, evaluate the model performance on the validation set and adjust the learning rate according to the loss on the validation set;

[0195] Until the loss function converges, the training ends and the trained prediction model is obtained.

[0196] In Step 6, evaluating the model performance on the validation set includes:

[0197] Every h training epochs, input the model in batches on the validation set, and obtain the corresponding predicted segmentation results and classification results for each image one by one;

[0198] Calculate the segmentation performance metrics, including: Dice coefficient, which is used to measure the overlap degree between the predicted segmentation and the ground truth segmentation region; IoU (Intersection over Union), which is used to evaluate the ratio of the intersection to the union of the segmentation regions;

[0199] Calculate the grading performance metrics, including: classification accuracy, which is used to measure the overall consistency between the predicted classification labels and the ground truth labels; weighted F1 value, which is used to comprehensively evaluate the prediction performance of the classification task;

[0200] Record and calculate the loss of the validation set;

[0201] Monitor the training effect of the model according to the validation set performance metrics. If it is detected that the total validation loss has not decreased for m consecutive times, adjust the learning rate and perform iterative training again.

[0202] The loss function during the training of the prediction model is as follows:

[0203]

[0204] L seg = L region + δ·L edge

[0205] L region = E[(P i,j - G i,j ) 2 ·ω i,j

[0206]

[0207] L class = L CE + γ·L ordinal

[0208]

[0209] Wherein, is the total loss; L seg is the segmentation loss; L class is the classification loss; L synergy is the collaborative loss; β1, β2, β3 are the weights of each loss; L region is the regional loss; L edge is the edge loss; δ is the weight coefficient of the edge loss; P i,j is the segmentation prediction probability of the (i, j) pixel; G i,j is the true label of the (i, j) pixel; ω i,j = 1 + λ1·A i,j is the adaptive weight coefficient, A i,j is the attention weight of the pixel (i, j), λ1 is the adjustment parameter; E[·] represents the expectation; C (i,j) is the edge response degree; and are the edge gradient values of the predicted segmentation and the true segmentation respectively; ω′ i,j = 1 + λ2·C i,j is the edge adaptive weight coefficient, λ2 is the adjustment parameter; L CE is the cross-entropy loss; K is the total number of samples, y k is the true label of the k-th sample, taking values 0 or 1, p k is the prediction for the k-th sample; F seg is the feature representation of the segmentation branch; F class is the feature representation of the classification branch; is the average value of the attention weights of the fibrotic regions, M is the total number of pixels.​

[0210] The present invention designs a multi-task joint loss function, which includes three parts: segmentation loss, classification loss, and collaboration loss. Among them, the segmentation loss ensures the accuracy of segmentation and the clarity of boundaries through the combination of region loss and edge loss, and introduces an adaptive weight mechanism to focus on key regions. The classification loss uses binary cross-entropy loss to classify two categories: "no fibrosis risk" and "suspected fibrosis". The collaboration loss promotes feature sharing and complementarity between the two tasks by minimizing the difference between the feature representations of the segmentation branch and the classification branch. The design of this multi-task joint loss function realizes the collaborative optimization of the segmentation and grading tasks, improves the comprehensive recognition ability of the model for liver fibrosis, and at the same time, by introducing the adaptive weight and attention mechanism, enhances the model's perception ability of key regions, and improves the accuracy and robustness of prediction.

[0211] In this embodiment, during training, η0 is set to 0.001, the momentum parameters β1 = 0.9, β2 = 0.999, and the numerical stability parameter ε = 10 -8 . In the loss function, β1 = 0.5, β2 = 0.3, β3 = 0.2. In the forward propagation stage, the randomly selected batchsize is set to 16. When gradient clipping, the threshold is set to 5.0. In the model evaluation stage, let h = 100. If the validation set loss does not decrease continuously for m = 5 times, the learning rate is adjusted according to the decay of η = η × 0.1. If the loss still does not improve, the early stopping mechanism is triggered. The training termination conditions, in addition to the convergence of the loss function, can also be set to reach the maximum number of training rounds (200 rounds), the validation set loss does not improve continuously for 15 times, or the performance index of the validation set reaches the preset threshold.

[0212] Specifically, in an embodiment of the present invention, step S4 uses the trained prediction model to predict liver fibrosis for new medical images, and the specific implementation process is as follows:

[0213] Perform the same preprocessing operations on the newly acquired medical images as in the training stage: input image → Gaussian filtering for denoising → histogram equalization to enhance contrast → superpixel segmentation → binarization processing → liver region segmentation.

[0214] Input the preprocessed medical images into the trained prediction model: extract multi-scale features through the encoder; transfer feature information through skip connections; the decoder reconstructs the feature map; the feature enrichment mechanism enhances the feature expression; the dual-task output module generates the prediction results.

[0215] Output of the segmentation result: generate pixel-level predictions of the liver and fibrosis regions; output the segmentation probability map to identify the liver region and the fibrosis region.

[0216] Classification result output: Generate a probability vector P = {p0, p1} of "no fibrosis risk" or "suspected fibrosis", where p0 represents the probability of "no fibrosis risk" and p1 represents the probability of "suspected fibrosis".

[0217] Post - processing of prediction results:

[0218] A. Processing of segmentation results: Threshold the segmentation probability map to generate a binary liver segmentation mask; Apply morphological operations to optimize the segmentation boundary; Generate the final segmentation mask.

[0219] B. Processing of classification results: Obtain the probability vector P = {p0, p1} output by the classification branch, and based on the maximum a posteriori probability principle, select the category corresponding to the maximum probability as the final risk prediction result.

[0220] Visual display: Annotate the liver region segmentation mask on the original medical image to ensure accurate display of the segmented region. Highlight the "suspected fibrosis" region using different colors or markers.

[0221] The method for predicting liver fibrosis based on medical image recognition provided by the present invention can be used as an auxiliary tool in the clinical diagnosis and treatment process. Especially for the follow - up management of NAFLD patients, the method of the present invention has the following potential application values:

[0222] The method of the present invention can analyze multi - modal medical images, which helps doctors to more comprehensively understand the structural change characteristics of liver tissue. This feature may contribute to the assessment of the liver fibrosis risk of NAFLD patients and provide objective reference information for clinicians. As a non - invasive assessment method, the method of the present invention can be used for regular follow - up examinations of NAFLD patients. By analyzing continuously acquired medical images, it can assist doctors in monitoring the dynamic changes of the patient's liver status, and this information can be used as one of the reference bases for clinical decisions. The automated analysis method of the present invention can provide standardized assessment results, and these objective data can be used as supplementary materials for clinical research, which helps to deeply understand the development law of NAFLD - related liver fibrosis.

[0223] It should be specifically noted that the method of the present invention is only used as an auxiliary tool in the clinical diagnosis and treatment process, and its prediction results should be combined with other diagnostic bases such as clinical symptoms and laboratory tests, and cannot be used as the sole basis for the diagnosis of NAFLD and its related complications. The formulation of specific treatment plans still requires professional physicians to make a comprehensive judgment according to the specific situation of the patients.

[0224] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for predicting liver fibrosis based on medical image recognition, characterized in that: include: S1. Retrospectively collect multimodal medical image sets from historical medical records of confirmed fibrosis, and match the medical image sets with the confirmed degree of fibrosis to generate image-label pairs; S2, preprocessing the medical image set, including denoising, contrast enhancement, and liver region segmentation; S3. Construct a prediction model. The prediction model uses a multi-path attention U-Net network as the framework. The preprocessed medical image set is divided into a training set and a validation set. The prediction model is iteratively trained until the loss function converges to obtain a trained prediction model. S4. Obtain new medical images and input them into the trained prediction model to obtain liver fibrosis prediction results.

2. The method for predicting liver fibrosis based on medical image recognition according to claim 1, characterized in that: Step S2 includes: S21, smoothing the medical image set using Gaussian filtering to obtain a denoised medical image set; S22, performing histogram equalization on the denoised medical image set to obtain a contrast-enhanced medical image set; S23, using superpixel segmentation to preliminarily classify the contrast-enhanced medical image set, and divide each medical image into a number of superpixel blocks with similar features; S24. Binarize the superpixel blocks of each medical image, identify and select the target area through the connected domain labeling algorithm, and obtain the liver segmentation result.

3. The method for predicting liver fibrosis based on medical image recognition according to claim 2, characterized in that: Step S23 includes: S231. Select a medical image; S232, converting the medical image into a six-dimensional clustering space (L, a, b, x, y, T) containing texture features, where (L, a, b) are color channels, (x, y) are pixel coordinates, and T is a texture feature; S233, set the number of superpixels K and calculate the initial clustering distance Where N is the total number of pixels; S234, grid the medical image, and uniformly sample on the grid to obtain the initial cluster center C k , define the comprehensive distance D as: in, is the color distance; is the spatial distance; d t =‖T i -T j ‖ is the texture distance; λ c , s , t is the normalization coefficient; S235, at each cluster center C k In the 2S×2S neighborhood, update the pixel attribution and cluster center according to the principle of minimum comprehensive distance D; S236, if the change of the cluster center is less than the threshold, the iteration is stopped to obtain the superpixel segmentation result of the medical image; S237. Repeat steps S231-S236 to perform superpixel segmentation on all medical images, and divide all medical images into a number of superpixel blocks with similar features.

4. The method for predicting liver fibrosis based on medical image recognition according to claim 2, characterized in that: Step S24 includes: S241, setting an adaptive threshold τ=μ+kσ according to the average gray value of the superpixel block, where μ is the pixel mean of the superpixel block, σ is the pixel standard deviation of the superpixel block, and k is an adjustment factor; binarizing the superpixel block using the adaptive threshold τ to obtain a binary image; S242, using the eight-connected algorithm, identifying the connected regions in the binary image: Traverse the binary image and for each pixel (x, y), check its eight neighboring pixels: If the neighboring pixel has the same value as the current pixel, they are classified as the same connected domain; S243. Calculate the features of each connected domain, including area A, perimeter P, shape factor S, aspect ratio R, and centroid position Among them, the area A is the total number of pixels in the connected domain, the perimeter P is the number of pixels at the border of the connected domain, and the shape factor Reflects the circularity of the connected domain, aspect ratio L max and L min is the maximum and minimum axis length of the connected domain, the centroid position S244. Use the scoring function Q to screen the connected domain: In the formula, A max is the maximum area, (x c ,y c ) is the central coordinate of the possible position of the liver in the connected domain, W and H are the width and height of the connected domain, α1, α2, α3, α4 are weight coefficients; S245, selecting the connected domain with the highest score value as the candidate liver region according to the scoring function; S246. Smoothing the liver candidate region according to the morphological operation and adjusting the liver boundary using the active contour model: In the formula, C is the contour curve, κ is the curvature, N is the normal vector, is the image gradient, a and b are adjustment parameters; S247, marking the processed liver candidate region as 1, marking other regions as 0, generating a liver mask, and multiplying the mask by the original superpixel block to obtain a liver segmentation result.

5. The method for predicting liver fibrosis based on medical image recognition according to claim 1, characterized in that: Predictive models include: The backbone network consists of an encoder and a decoder, with a jump connection between the encoder and the decoder. The encoder part is equipped with multi-resolution branches, which are used to extract high-resolution local features and low-resolution global features respectively. The multi-resolution branches are composed of convolutional layers and pooling layers. The decoder part is equipped with a dynamic spatial attention fusion module to perform weighted fusion on the features of the encoder's multi-resolution branches. Feature enrichment mechanism: a feature sampling layer is set after the decoder output to divide the two-dimensional feature map into several small blocks and convert them into feature sequences; feature sequences and global features are combined to complete feature reorganization and fusion, forming a deep feature representation that is compatible with pixel-level segmentation and globalization; The dual-task output module has a first branch, which is a segmentation branch, and is used to output pixel-level prediction results of the liver and fibrosis areas. The second branch is a classification branch, and the global feature vector is used to complete the classification prediction of fibrosis.

6. The method for predicting liver fibrosis based on medical image recognition according to claim 5, characterized in that: The training process of the prediction model is as follows: Divide the preprocessed medical image set into a training set and a validation set; Use the Xavier initialization method to initialize the parameters of the prediction model, set the Adam optimizer, set the initial learning rate to η0, and adjust the learning rate according to the training rounds; define the loss function, including segmentation loss, classification loss and collaborative loss; Training process: Step 1: Randomly select a batch of medical images from the training set; Step 2: Input the batch images into the prediction model, and pass through the encoder, skip connection, decoder and dual-task output module in sequence to obtain the predicted segmentation results and the predicted classification results; Step 3: Calculate the loss based on the predicted segmentation results, the predicted classification results and the true labels; Step 4: Calculate the gradient through the back-propagation algorithm and use the optimizer to update the model parameters; Step 5: Use the gradient clipping method to limit the gradient norm within the threshold range; Step 6: Repeat steps 1 to 5. Every h rounds, evaluate the model performance on the validation set and adjust the learning rate according to the loss of the validation set. The training ends when the loss function converges, and a trained prediction model is obtained.

7. The method for predicting liver fibrosis based on medical image recognition according to claim 6, characterized in that: In step 6, the model performance is evaluated on the validation set, including: Every h training rounds, the validation set is input into the model in batches, and the corresponding predicted segmentation results and classification results are obtained one by one; Calculate segmentation performance indicators, including: Dice coefficient, which is used to measure the overlap between the predicted segmentation and the true segmentation area; Intersection over Union (IoU), which is used to evaluate the ratio of the intersection and union of the segmented areas; Calculate classification performance indicators, including: classification accuracy, which is used to measure the overall consistency between the predicted classification label and the true label; weighted F1 value, which is used to comprehensively evaluate the prediction performance of the classification task; Record and calculate the loss of the validation set; The training effect of the model is monitored according to the performance indicators of the validation set. If it is detected that the total loss of m consecutive validations has not decreased, the learning rate is adjusted and iterative training is performed again.

8. The method for predicting liver fibrosis based on medical image recognition according to claim 5, characterized in that: The loss function when training the prediction model is as follows: L seg =L region +δ·L edge L region =E[(P i,j -G i,j ) 2 ·oh i,j ] In the formula, is the total loss; L seg is the segmentation loss; L class is the classification loss; L synergy is the collaborative loss; β1, β2, β3 are the weights of each loss; L region is the regional loss; L edge is the edge loss; δ is the weight coefficient of the edge loss; P i,j is the segmentation prediction probability of the (i, j)th pixel; G i,j is the true label of the (i, j)th pixel; ω i,j =1+λ1·A i,j is the adaptive weight coefficient, A i,j is the attention weight of pixel (i, j), λ1 is the adjustment parameter; E[·] represents the expectation; C (i,j) It is the degree of marginal reaction; and are the edge gradient values ​​of the predicted segmentation and the true segmentation respectively; ω′ i,j =1+λ2·C i,j is the edge adaptive weight coefficient, λ2 is the adjustment parameter; L CE is the cross entropy loss; K is the total number of samples, y k is the true label of the kth sample, which takes the value 0 or 1, p k The kth sample is predicted as; F seg is the feature representation of the segmentation branch; F class It is the feature representation of the classification branch; is the average attention weight of the fibrosis area, M is the total number of pixels.

9. The method for predicting liver fibrosis based on medical image recognition according to claim 1, characterized in that: The medical image collection includes MRI, CT and B-ultrasound images.

Images