A retinal disease segmentation method and device based on fundus image prior knowledge

Abstract

The application discloses a retinal lesion segmentation method and device based on fundus image prior knowledge, which comprises the following steps: establishing a characteristic expression according to the prior characteristics of an image; establishing a fundus image probability model; obtaining a posterior probability distribution expression related to a parameter to be solved according to the fundus image probability model, and solving the related parameter and an introduced hidden variable; converting a maximum problem of a logarithmic posterior function into a minimum problem related to the parameter to be solved; performing an iterative solving step; constructing a corresponding network module according to a closed-form solution calculation formula in an expression updated through iteration, performing a learnable proximal operator operation step in the closed-form solution by using a deep residual network; inputting training data into an expectation maximization network for a training step; and inputting test data into the optimized expectation maximization network for a forward inference to obtain a lesion segmentation result. The problems of how to embed the prior structure of the fundus image and how to design a segmentation network with explainability are solved.

Description

Technical Field

[0001] This application relates to the fields of medical image processing and deep learning technology, and in particular to a method and apparatus for segmenting retinal lesions based on prior knowledge of fundus images. Background Technology

[0002] Diabetic retinopathy (DR) is an eye disease caused by diabetes and has become one of the leading causes of vision loss in adults. The most common early clinical manifestations of diabetic retinopathy include microaneurysms (MA), hemorrhages (HE), soft exudates (SE), and hard exudates (EX). Fortunately, research shows that early detection and timely treatment can effectively prevent vision loss caused by DR. Early screening for DR is essential, and retinal fundus image analysis is currently the most widely used method for DR screening.

[0003] In recent decades, deep learning (DL)-based methods have outperformed traditional techniques in many computer vision tasks, and these methods have also been introduced into DR screening and proven effective. However, most methods lack interpretability to some extent, especially by ignoring the intrinsic prior structure of fundus images.

[0004] Therefore, how to embed prior structures from fundus images and design interpretable segmentation networks has become an unsolved problem. Summary of the Invention

[0005] In this application embodiment, a retinal lesion segmentation method based on prior knowledge of fundus images is provided, which solves the problem of how to embed prior structures of fundus images and design an interpretable segmentation network.

[0006] In a first aspect, embodiments of this application provide a method for retinal lesion segmentation based on prior knowledge of fundus images. The method includes: preprocessing raw fundus image data to obtain training data and testing data; wherein the preprocessing step includes reshaping, augmentation, and segmentation operations; establishing feature expressions based on prior features of the image; wherein the feature expressions are a sum of foreground and background features, the foreground features being lesion features, and the background features being non-lesion features; establishing expressions for the foreground and background features based on the feature expressions, and establishing a fundus image probability model based on the expressions for the foreground and background features; obtaining a posterior probability distribution expression related to parameters to be determined based on the fundus image probability model; wherein the parameters to be determined include feature maps used to fit the background and latent variables introduced in the modeling; converting the posterior probability distribution expression into a related log-posterior function expression, and converting the maximization problem of the log-posterior function into a problem related to the parameters to be determined. The problem involves minimizing parameters; iteratively solving the problem using the Expectation-Maximization (EM) algorithm and the Proximal Gradient Descent (PGR) algorithm; the iterative solution includes a first step and a second step; the first step is a lesion segmentation operation, subtracting the background portion obtained from the second step from the original fundus image to obtain the foreground portion as input for the first step; the second step is a background fitting operation for non-lesions, establishing an iterative update expression for the relevant parameters; obtaining the closed-form solution based on the quadratic approximation in the iterative update expression, expanding each step's calculation to obtain the corresponding network module, and using a deep residual network to execute learnable proximal operator operations in the closed-form solution; the first step involves performing the lesion segmentation operation using an existing segmentation network to obtain an Expectation-Maximization (EM) network; the training data is input into the EM network for training to obtain an optimized EM network; the test data is input into the trained EM network for forward inference to obtain the lesion segmentation result.

[0007] In conjunction with the first aspect, in one possible implementation, the feature expression established based on the prior features of the image is: X = F + B; where X is the fundus image, F is the foreground portion, B is the background portion, and X, F, B ∈ R. H×W×3 R is the real number field, H is the height of the fundus image, and W is the width of the fundus image.

[0008] In conjunction with the first aspect, in one possible implementation, the expression for the foreground part is: Here, Z is an introduced latent variable, with its height and width coordinates denoted as (h, w), and the element in the k-th channel represented as Z. hwk It satisfies Z hwk ∈{0,1} and This indicates the category to which each pixel belongs, where K represents the number of mixed components, with K-1 parts representing K-1 type lesions and the other part representing non-lesion components, and F... hw ∈R 3 This represents the colored pixel at coordinates (h, w) in the foreground region F, where R is the real number field. Let σ be the mean of the k-th group in the Gaussian mixture model. k Let μ be the variance of the k-th group in the Gaussian mixture model, and σ be μ' ... k and σ k For a vector k = 1, 2, ..., K, where I is the identity matrix and N represents a Gaussian distribution, the expression for the background part is: Among them, C l ∈R p×p×3 M represents the l-th convolutional kernel of size p. l ∈R H ×W It is the feature map associated with it, and L is the number of convolutional kernels in the model.

[0009] In conjunction with the first aspect, in one possible implementation, the established fundus image probability model is as follows: Among them, X hw ∈R 3 This represents the color pixel at coordinates (h, w) in the fundus image X, where C and M are C0, W ... l (l = 1, 2, ..., L) and M l The tensor formed by stacking (l = 1, 2, ..., L) has parameters C, μ, and σ shared across the entire dataset and can be learned from the training data. The feature map M and latent variable Z need to be re-estimated for each new sample. The resulting posterior probability distribution expression is: Where P(M) represents the prior distribution based on feature map M.

[0010] In conjunction with the first aspect, in one possible implementation, the log-posterior function expression is: The optimization process for feature map M is as follows: in, Let g(M) be the Hadamard product, and g(M) be the regularization term related to P(M).

[0011] In conjunction with the first aspect, in one possible implementation, the expression for the first step is: The closed-form solution is obtained based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. A deep residual network is used to execute learnable proximal operator operations in the closed-form solution. The first step involves the existing segmentation network performing lesion segmentation operations, thereby obtaining the expectation-maximization network. This includes: the expression for the iterative update of feature map M is: M t =argmin M Q(M,M t-1 ); where M t-1 This is the current update result; the quadratic approximation Q(M,M) t-1 The expression for ) is in, The quadratic approximation Q(M,M) t-1 The closed-form solution to the minimization problem of the feature map M is: Among them, prox g It is a proximal operator related to g. The first step involves performing lesion segmentation using an existing segmentation network; the segmentation network is expressed as follows: in, For the segmentation network used, α t For network parameters; F (t) The foreground portion serves as the input for the first step; the network module for solving the feature map M is... Where Conv(·) is the convolution operation, Conv T (·) represents the transpose convolution operation. Indicates the parameter is β t In a deep residual network, W(·) and U(·) are element-wise operators.

[0012] In conjunction with the first aspect, in one possible implementation, the step of inputting the training data into the expectation-maximization network to obtain an optimized expectation-maximization network includes: initializing the parameters μ and σ. The first step is performed using different segmentation networks, including UNet, UNet++, Lseg, and CE-Net. The output of the first step is used as part of the input to the second step, and the difference between the original fundus image and the output of the second step is used as the input to the first step. The loss function of the expectation-maximization network is calculated. The total loss function of the expectation-maximization network is: in, γ represents the segmentation result output at step t of the iterative steps in the first step. tru For the segmented real labels, As the background for the output of the t-th step in the second step of the iterative step, B tru For the real-world context, R(μ, σ) is the regularization term associated with μ and σ, and L ce For cross-entropy loss, λ t For L ce The weighting coefficients, L2 is the mean squared error loss, τ t Let L1 be the L2 weight coefficient, L1 be the 1-norm related to the feature map M, and ∈ be the L1 weight coefficient; the expression for cross-entropy loss is: Where β and β n The weights are the corresponding weight coefficients for the two terms; the network parameters and the optimized loss function of the expectation-maximization network are updated using the stochastic gradient descent algorithm; the training step is terminated after the preset iteration steps are reached, thereby obtaining the optimized expectation-maximization network.

[0013] Secondly, embodiments of this application provide a retinal lesion segmentation device based on prior knowledge of fundus images. The device includes: a preprocessing module for performing preprocessing steps on raw fundus image data to obtain training data and test data; wherein the preprocessing steps include reshaping operations, augmentation operations, and segmentation operations; a feature expression establishment module for establishing feature expressions based on prior features of the image; wherein the feature expressions are a summation of foreground and background parts, the foreground part representing lesion features, and the background part representing non-lesion features; a fundus image probability model establishment module for establishing expressions for the foreground and background parts based on the feature expressions, and establishing a fundus image probability model based on the expressions for the foreground and background parts; a probability distribution expression acquisition module for obtaining a posterior probability distribution expression related to parameters to be determined based on the fundus image probability model; wherein the parameters to be determined include feature maps used to fit the background and latent variables introduced in the modeling; and an optimization problem establishment module for converting the posterior probability distribution expression into a related log-posterior function expression, and maximizing the log-posterior function. The problem is transformed into a minimization problem related to the parameters to be solved. An iterative solution module is used to perform iterative solution steps using the expectation-maximization (EM) algorithm and the proximal gradient descent algorithm. The iterative solution steps include a first step and a second step. The first step is a lesion segmentation operation step, where the background portion obtained from the second step is subtracted from the original fundus image to obtain the foreground portion as input to the first step. The second step is a non-lesion background fitting operation step, establishing an iterative update expression for the relevant parameters. An expectation-maximization network module is obtained, which is used to obtain its closed-form solution based on the quadratic approximation in the iterative update expression. Each step of the calculation is expanded to obtain the corresponding network module. A deep residual network is used to perform learnable proximal operator operations in the closed-form solution. The first step involves performing the lesion segmentation operation step using an existing segmentation network, thereby obtaining the expectation-maximization network. An optimized network module is obtained, which is used to input the training data into the expectation-maximization network for training, obtaining the optimized expectation-maximization network. A lesion segmentation result module is obtained, which is used to input the test data into the trained expectation-maximization network for forward inference to obtain the lesion segmentation result.

[0014] In conjunction with the second aspect, in one possible implementation, the feature expression established based on the prior features of the image is: X = F + B; where X is the fundus image, F is the foreground portion, B is the background portion, and X, F, B ∈ R. H×W×3 R is the real number field, H is the height of the fundus image, and W is the width of the fundus image.

[0015] In conjunction with the second aspect, in one possible implementation, the expression for the foreground portion is: Here, Z is an introduced latent variable, with its height and width coordinates denoted as (h, w), and the element in the k-th channel represented as Z. hwk It satisfies Z hwk ∈{0,1} and This indicates the category to which each pixel belongs, where K represents the number of mixed components, with K-1 parts representing K-1 type lesions and the other part representing non-lesion components, and F... hw ∈R 3 This represents the colored pixel at coordinates (h, w) in the foreground region F, where R is the real number field. Let σ be the mean of the k-th group in the Gaussian mixture model. k Let μ be the variance of the k-th group in the Gaussian mixture model, and σ be μ' ... k and σ k For a vector k = 1, 2, ..., K, where I is the identity matrix and N represents a Gaussian distribution, the expression for the background part is: Among them, C l ∈R p×p×3 M represents the l-th convolutional kernel of size p. l ∈R H ×W It is the feature map associated with it, and L is the number of convolutional kernels in the model.

[0016] In conjunction with the second aspect, in one possible implementation, the established fundus image probability model is as follows: Among them, X hw ∈R 3 This represents the color pixel at coordinates (h, w) in the fundus image X, where C and M are C0, W ... l (1 = 1, 2, ..., L) and M l The tensor formed by stacking (l = 1, 2, ..., L) has parameters C, μ, and σ shared across the entire dataset and can be learned from the training data. The feature map M and latent variable Z need to be re-estimated for each new sample. The resulting posterior probability distribution expression is: Where P(M) represents the prior distribution based on feature map M.

[0017] In conjunction with the second aspect, in one possible implementation, the log-posterior function expression is: The optimization process for feature map M is as follows: in, Let g(M) be the Hadamard product, and g(M) be the regularization term related to P(M).

[0018] In conjunction with the second aspect, in one possible implementation, the expression for the first step is: The closed-form solution is obtained based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. A deep residual network is used to execute learnable proximal operator operations in the closed-form solution. The first step involves the existing segmentation network performing lesion segmentation operations, thereby obtaining the expectation-maximization network. This includes: the expression for the iterative update of feature map M is: M t =argmin M Q(M,M t-1 ); where M t-1 This is the current update result; the quadratic approximation Q(M, M) t-1 The expression for ) is Q(M, M t-1 )=f(M t-1 )+ <M-M t-1 , in, The quadratic approximation Q(M, M) t-1 The closed-form solution to the minimization problem of the feature map M is: Among them, prox g It is a proximal operator related to g. The first step involves performing lesion segmentation using an existing segmentation network; the segmentation network is expressed as follows: in, For the segmentation network used, α t For network parameters; F (t) The foreground portion serves as the input for the first step; the network module for solving the feature map M is... Where Conv(·) is the convolution operation, Conv T (·) represents the transpose convolution operation. Indicates the parameter is β t In a deep residual network, W(·) and U(·) are element-wise operators.

[0019] In conjunction with the second aspect, in one possible implementation, the training step involves inputting the data to be trained into the expectation-maximization network to obtain an optimized expectation-maximization network, including: initializing the parameters μ and σ. The first step is performed using different segmentation networks, including UNet, UNet++, Lseg, and CE-Net. The output of the first step is used as part of the input to the second step, and the difference between the original fundus image and the output of the second step is used as the input to the first step. The loss function of the expectation-maximization network is calculated. The total loss function of the expectation-maximization network is: in, γ represents the segmentation result output at step t of the iterative steps in the first step. tru For the segmented real labels, As the background for the output of the t-th step in the second step of the iterative step, B tru For the real-world context, R(μ, σ) is the regularization term associated with μ and σ, and L ce For cross-entropy loss, λ t For L ce The weighting coefficients, L2 is the mean squared error loss, τ t Let L1 be the L2 weight coefficient, L1 be the 1-norm related to the feature map M, and ∈ be the L1 weight coefficient; the expression for cross-entropy loss is: Where β and βn are the weight coefficients of the two terms; the network parameters and optimization loss function of the expectation-maximization network are updated using the stochastic gradient descent algorithm; the training step is terminated after the preset iteration steps are reached, thereby obtaining the optimized expectation-maximization network.

[0020] Thirdly, embodiments of this application provide a retinal lesion segmentation server based on prior knowledge of fundus images, including a memory and a processor; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable instructions, which, when executed by a computer, implement the method described in the first aspect or any possible implementation thereof.

[0022] One or more technical solutions provided in the embodiments of this application have at least the following technical effects:

[0023] This application provides a retinal lesion segmentation method based on prior knowledge of fundus images. The obtained expectation-maximization network includes a first step and a second step. The purpose of the first step is to segment the foreground portion, and the purpose of the second step is to learn a clean background image, decomposing the fundus image into the foreground portion (i.e., the lesion portion) and the background portion. The network used in the first step can be directly replaced by an existing segmentation network, achieving integration with existing methods in a plug-and-play manner. This application can segment various lesions more accurately, assisting doctors in diagnosis and treatment, thereby solving the problem of how to embed the prior structure of fundus images and design an interpretable segmentation network. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments of the present invention or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a retinal lesion segmentation method based on prior knowledge of fundus images provided in an embodiment of this application;

[0026] Figure 2 Schematic diagrams of related images and networks provided for embodiments of this application;

[0027] Figure 3 A schematic diagram of the T-iteration process of solving the posterior probability distribution expression using the expectation-maximization algorithm, and a schematic diagram of obtaining the t-th stage of the expectation-maximization network, provided for embodiments of this application.

[0028] Figure 4 A schematic diagram of the test results of the IDRID dataset provided in the embodiments of this application;

[0029] Figure 5 A schematic diagram of the test results of the DDR dataset provided in the embodiments of this application;

[0030] Figure 6 A schematic diagram illustrating the foreground and background portions obtained by inputting the data to be trained into the Desire Maximization Network for training, as provided in an embodiment of this application.

[0031] Figure 7 A schematic diagram illustrating a partial calculation result from the second step provided in the embodiments of this application;

[0032] Figure 8 A schematic diagram of a retinal lesion segmentation device based on prior knowledge of fundus images provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of a retinal lesion segmentation server based on prior knowledge of fundus images, provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0035] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0036] This application provides a method for segmenting retinal lesions based on prior knowledge of fundus images, such as... Figure 1 As shown, the method includes steps S101 to S109. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a retinal lesion segmentation method based on prior knowledge of fundus images. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0037] S101: Preprocessing steps are performed on the raw fundus image data to obtain training and testing data. The preprocessing steps include reshaping, augmentation, and segmentation operations.

[0038] Specifically, the reshaping operation involves reshaping the original 2848×4288 images of the IDRID dataset to 712×1072 pixels. For the DDR dataset, the samples, each with a different size, are all reshaped to 432×648 pixels. The augmentation operation employs vertical and horizontal flipping, as well as random angle rotation, to enhance the network's generalization ability and mitigate overfitting. The segmentation operation, to conserve GPU memory during training, divides the images into 304×304 pixels before feeding them into the network. This segmentation significantly reduces GPU memory usage, allowing for larger batch sizes and improving training speed and stability. These preprocessing steps effectively prepare the fundus image dataset, making it suitable for training and testing deep learning models.

[0039] It should be noted that the original fundus images used in this application are fundus images of diabetic patients.

[0040] Specifically, Figure 4 A schematic diagram of the test results of the IDRID dataset provided in the embodiments of this application, as shown below. Figure 4As shown, the four selected baseline networks are UNet, L-seg, CE-Net, and UNet++. (a) shows the original input image data, and (f) shows the ground truth of the segmentation results. (b) to (e) are visualizations of the segmentation using the baseline networks, and (g) to (j) show the segmentation results under the corresponding EM-Net framework. The figures clearly demonstrate the effectiveness of the method of this invention; performance has been significantly improved, enabling more accurate classification of lesion categories.

[0041] Specifically, Figure 5 A schematic diagram of the test results of the DDR dataset provided in the embodiments of this application, as shown below. Figure 5 As shown, the four selected baseline networks are UNet, L-seg, CE-Net, and UNet++. (a) shows the original input image data, and (f) shows the ground truth of the segmentation results. (b) to (e) are visualizations of the segmentation using the baseline networks, and (g) to (j) show the segmentation results under the corresponding EM-Net framework. The figures clearly demonstrate the effectiveness of the method of this invention, showing a significant performance improvement. It can also correctly provide segmentation results for categories that cannot be identified using the baseline networks (i.e., baseline methods).

[0042] S102: Establish a feature expression based on the prior features of the image. The feature expression is a summation of the foreground and background parts, where the foreground part represents lesion features and the background part represents non-lesion features.

[0043] The feature expression established based on the prior features of the image is: X = F + B. Where X is the fundus image, F is the foreground portion, B is the background portion, and X, F, B ∈ R. H×W×3 R is the real number field, H is the height of the fundus image, and W is the width of the fundus image.

[0044] Figure 2 These are schematic diagrams of related images and networks provided for embodiments of this application. Figure 2 (a) is a schematic diagram of a fundus image: it consists of a background portion without lesion features and a foreground portion containing only lesion features, such as... Figure 2 As shown in (a), although the fundus images are from different individuals, the background still exhibits similar pattern features (the main difference being the morphology of blood vessels). Therefore, this component can be encoded and represented in a deterministic manner. In contrast, foreground lesions are highly variable, with significant differences in shape, size, location, and brightness, making them suitable for characterization using a stochastic model. Figure 2 (b) is a traditional "black box" network architecture based on deep learning methods.

[0045] S103: Establish expressions for the foreground and background parts based on the feature expressions, and establish a fundus image probability model based on the expressions for the foreground and background parts.

[0046] Specifically, the expression for the foreground portion is as follows: Here, Z is an introduced latent variable, with its height and width coordinates denoted as (h, w), and the element in the k-th channel represented as Z. hwk It satisfies Z hwk ∈{0,1} and This indicates the category to which each pixel belongs, where K represents the number of mixed components, with K-1 parts representing K-1 type lesions and the other part representing non-lesion components, and F... hw ∈R 3 This represents the colored pixel at coordinates (h, w) in the foreground region F, where R is the real number field. Let σ be the mean of the k-th group in the Gaussian mixture model. k Let μ be the variance of the k-th group in the Gaussian mixture model, and σ be μ' ... k and σ k For a vector k = 1, 2, ..., K, I is the identity matrix, and N represents a Gaussian distribution. Specifically, the expression for the foreground part is modeled using a Gaussian mixture model, where each component characterizes a type of lesion. Each pixel belongs to a category including one type of non-lesion feature and four different types of lesion features. Non-lesion features can be camera noise with a mean close to 0.

[0047] Specifically, the expression for the background portion is: Among them, C l ∈R p×p×3 M represents the first convolutional kernel of size p, which characterizes the local features between different samples. l ∈R H×W It is the associated feature map, which encodes the location information of the local features, and L is the number of convolutional kernels in the model. Figure 2 (d) The R-channel convolution kernel C in the convolution dictionary model used for modeling background l , Figure 2 (e) is a schematic diagram of the convolutional dictionary representation of the modeling background part. The process of establishing the expression for the background part can be found in [reference needed]. Figure 2 (d) and Figure 2 (e).

[0048] S104: Obtain the posterior probability distribution expression related to the parameters to be determined based on the fundus image probability model. The parameters to be determined include the feature map used to fit the background and the latent variables introduced during modeling.

[0049] Specifically, the established probability model for fundus images is as follows: Among them, X hw ∈R 3 This represents the color pixel at coordinates (h, w) in the fundus image X, where C and M are C0, W ... l (l = 1, 2, ..., L) and M l A tensor formed by stacking (l = 1, 2, ..., L). Parameters C, μ, and σ are shared across the entire dataset and can be learned from the training data. Feature maps M and latent variables Z need to be re-estimated for each new sample.

[0050] Specifically, the key to solving the probabilistic model of fundus images lies in estimating the feature map M and the introduced latent variable Z.

[0051] Therefore, the obtained posterior probability distribution expression is: Here, P(M) represents the prior distribution based on feature map M, and Z is the introduced latent variable. Specifically, the maximum a posteriori problem for the posterior probability distribution expression can be solved using the expectation-maximization algorithm (EM algorithm).

[0052] Figure 3 This application provides a schematic diagram of the T-iteration process of solving the posterior probability distribution expression using the expectation-maximization algorithm, and a schematic diagram of obtaining the t-th stage of the expectation-maximization network. Figure 3 a is a schematic diagram of the T-iteration process of solving the posterior probability distribution expression using the expectation-maximization algorithm provided in the embodiments of this application.

[0053] S105: Transform the posterior probability distribution expression into the associated log-posterior function expression, and transform the maximization problem of the log-posterior function into a minimization problem related to the parameters to be solved.

[0054] Specifically, the expression for the logarithmic posterior function is:

[0055] The optimization process for feature map M is as follows: in, Let g(M) be the Hadamard product, and g(M) be the regularization term related to P(M).

[0056] S106: Iterative solution steps using the expectation-maximization algorithm and the proximal gradient descent algorithm. The iterative solution steps include a first step and a second step.

[0057] Specifically, the first step is the lesion segmentation operation, which involves subtracting the background from the original fundus image to obtain the foreground portion, which serves as the input for the first step. The second step is the background fitting operation for non-lesions, which involves establishing iterative update expressions for relevant parameters. Specifically, the second step involves solving the optimization process of the feature map M, establishing an iterative update expression for M.

[0058] S107: Obtain the closed-form solution based on the quadratic approximation in the iteratively updated expression. Expand each step of the calculation to obtain the corresponding network module. Use a deep residual network to execute learnable proximal operator operation steps in the closed-form solution. The first step is to perform lesion segmentation operation steps by the existing segmentation network, thereby obtaining the expectation-maximizing network.

[0059] Figure 2 (c) The desired network framework consists of E-Net and M-Net, where E-Net corresponds to the first step of this application and M-Net corresponds to the second step of this application.

[0060] Specifically, the expression for the first step is: The foreground portion is obtained by subtracting the background portion B from the original fundus image X, and used as the input for the first step, E{Z hwk The result is the segmentation. To simplify computation, existing segmentation networks can be used directly instead. This allows the designed network framework to be integrated seamlessly with existing methods in a plug-and-play manner.

[0061] Specifically, in the second step, the corresponding network module is constructed based on the closed-form solution calculation formula in the iteratively updated expression. The deep residual network is used to execute learnable proximal operator operation steps in the closed-form solution, and the lesion segmentation operation steps are executed in combination with the segmentation network in the first step, thereby obtaining the expectation maximization network.

[0062] Specifically, the second step of the calculation includes:

[0063] The expression for iteratively updating feature map M is: M t =argmin M Q(M,M t-1 ), where M t-1 This is the current update result.

[0064] The quadratic approximation Q(M, M) t-1 The expression for ) is in,

[0065] The quadratic approximation Q(M, M) t-1 The closed-form solution to the minimization problem of the feature map M is: Among them, prox g It is a proximal operator related to g.

[0066] Figure 3 (b) A schematic diagram illustrating the t-th stage of obtaining the desired maximization network, provided in an embodiment of this application. Figure 3 The expected value maximization network to be constructed consists of a first step and a second step. The first step corresponds to the E-step of the expected value maximization algorithm, and the second step corresponds to the M-step of the expected value maximization algorithm.

[0067] Specifically, the segmentation network is expressed as: in, For the segmentation network used, α t For network parameters, F (t) It is the foreground part, which serves as the input for the first step.

[0068] The network module for solving the feature map M is Where Conv(·) is the convolution operation, Conv T (·) represents the transpose convolution operation. Indicates the parameter is β t In a deep residual network, W(·) and U(·) are element-wise operators. Specifically, the network module for solving the feature map M can be understood as the various network modules obtained by expanding the algorithm in the second step. The key step is determining the proximal operator; other operations can be performed using common operators in neural networks. The proximal operator can be represented by a deep CNN (Convolutional Neural Network) module and learned end-to-end from the training data. Referring to previous work, a residual network ResNet is used here as the proximal network.

[0069] Specifically, Figure 7 A visual representation of some calculation results in the second step provided in the embodiments of this application. Through Figure 7 This allows for a more intuitive understanding of its physical meaning, thereby verifying the interpretability of the network.

[0070] Figure 7 (a) is the original fundus image that was input. Figure 7 (b) is the clean background image learned in the second step. Figure 7 (c) is the foreground image obtained by subtracting the background image from the original fundus image. Figure 7 (d) is a mean plot, where lesions of the same category share the same mean. It is not difficult to see that... Figure 7 (d) and Figure 7 (c) is similar, but smoother. Figure 7 (e) is the variance map, which is involved in the network module for solving the feature map M. The operation. Figure 7 (f) is a visualization of ε, which is the residual information extracted from the foreground part. It contains information that was incorrectly decoupled to the foreground, and therefore it is reasonable to add it to the estimation calculation of the background part.

[0071] S108: Input the data to be trained into the Expectation Maximization Network for training to obtain the optimized Expectation Maximization Network.

[0072] Figure 3 (b) is a schematic diagram of the t-th stage of the expectation maximization network provided in the embodiments of this application. The expectation maximization network of a completed stage can be referred to Figure 3 (b)

[0073] Specifically, the training process involves inputting the data to be trained into the Expectation-Maximization (EM) network to obtain an optimized Expectation-Maximization (EM) network, including:

[0074] Initialize parameters μ and σ:

[0075] The first step is performed using different segmentation networks. These different segmentation networks include UNet, UNet++, Lseg, and CE-Net.

[0076] The output of the first step is used as part of the input of the second step, and the difference between the original fundus image and the output of the second step is used as the input of the first step.

[0077] Specifically, data is input into the network to begin training until a preset number of iterations is reached, at which point training stops and the model is saved. Four networks that perform well in segmentation tasks were used as E-Net: UNet, UNet++, Lseg, and CE-Net. The Adam optimizer was used to train the network for a total of 70 epochs (i.e., the entire training dataset was trained once), with each epoch containing 2000 iterations. The initial learning rate was 0.001, and it decayed every 23 epochs by multiplying by a factor of 0.1. (Background Model) The hyperparameter L is set to 10, and the spatial size of the convolutional kernel Cl is set to 9×9. The number of network stages is set to 3, meaning the first and second steps iterate three times. It is worth noting, however, that the ultimate goal of the network is to output the lesion segmentation result (i.e., the output of E-Net).

[0078] Figure 6This is a schematic diagram illustrating the foreground and background portions obtained by inputting the data to be trained into the Desire Maximization Network for training, as provided in an embodiment of this application. Figure 6 As shown, only two stages are needed to estimate a relatively clean background image, and the corresponding foreground gradually becomes clear, leaving only the information of the lesion. This more explicit input reduces the pressure on the expectation-maximizing network, making the recognition task easier.

[0079] Furthermore, the loss function of the expectation-maximizing network is calculated, where the total loss function of the expectation-maximizing network is: in, γ represents the segmentation result output at step t of the iterative steps in the first step. tru For the segmented real labels, As the background for the output of the t-th step in the second step of the iterative step, B tru For a realistic context, R(μ, σ) is the regularization term associated with μ and σ. ce For cross-entropy loss, λ t For L ce The weighting coefficients, L2 is the mean squared error loss, τ t Let λ be the weight coefficient of L2, L1 be the 1-norm related to the feature map M, and ∈ be the weight coefficient of L1. Specifically, the coefficients vary for different stages, λ t and τ t Let the values ​​be [20, 5, 30] and [0.5, 2.5, 0] respectively. To ensure the sparsity of L1, the weight coefficients are set to 1. Specifically, the expression for the cross-entropy loss is: Where β and β n These are the weighting coefficients for the two items.

[0080] Specifically, the network parameters and optimized loss function of the expectation-maximization network are updated using the stochastic gradient descent algorithm. The training process terminates after a predetermined number of iterations, thus obtaining the optimized expectation-maximization network.

[0081] S109: Input the test data into the trained Expectation Maximization network for forward inference to obtain the lesion segmentation result.

[0082] Specifically, the process involves preparing the fundus image to be tested and its corresponding ground truth label, loading an optimized expectation-maximization network, inputting the fundus image to be tested into the model for forward inference, and the network output being the result image of retinal lesion segmentation. Simultaneously, relevant indicators can be calculated based on the ground truth values ​​for quantitative analysis.

[0083] The E-Net in the Expectation-Maximization Network (EM-Net) proposed in this invention, i.e., the network corresponding to the first step, was replaced with UNet, UNet++, L-seg, and CE-Net, respectively. Diabetic retinopathy segmentation tasks were then performed on the public datasets IDRID and DDR, and the numerical results are shown in Tables 1 and 2. Table 1 shows the AUC (Area Under the PR Curve) of the EM-Net framework combined with other segmentation baseline networks on the IDRID dataset, as provided in this embodiment of the application. AUC is an important metric for evaluating the performance of classification models. Table 2 shows the AUC of the EM-Net framework combined with other segmentation baseline networks on the DDR dataset, as provided in this embodiment of the application. Clearly, the EM-Net network architecture exhibits significant performance advantages under all four baseline methods, significantly improving mAUC. For example, using UNet++ as the baseline network, EM-Net improves performance by 3.15% and 2.86% on the IDRID and DDR datasets, respectively.

[0084] Specifically, Table 1 shows the IDRiD test results: EM-UNet++ has an mAUC of 0.7000, and UNet++ has an mAUC of 0.6685, with a difference of 0.0315. Table 2 shows the DDR test results: EM-UNet++ has an mAUC of 0.4389, and UNet++ has an mAUC of 0.4103, with a difference of 0.0286. Particularly for soft exudates (SE), due to their smaller sample size compared to other lesion types, learning is difficult. EM-Net significantly improved its segmentation accuracy, with a maximum improvement of 16.49% (EM-CE-Net). Reference Figure 4 and Figure 5 This allows us to more intuitively see the segmentation accuracy advantage of EM-Net.

[0085] Table 1

[0086] UNet 0.4573 0.6570 0.6348 0.8284 0.6444 EM-UNet 0.4744 0.6820 0.6389 0.8509 0.6616 L-seg 0.4420 0.6742 0.6717 0.8679 0.6639 EM-L-seg 0.5032 0.6630 0.6783 0.8358 0.6791 CE-Net 0.4859 0.6014 0.5724 0.8766 0.6341 EM-CE-Net 0.4733 0.6995 0.7373 0.8656 0.6939 UNet++ 0.4506 0.6576 0.6940 0.8718 0.6685 EM-UNet++ 0.5069 0.7020 0.6998 0.8913 0.7000

[0087] Table 2

[0088] UNet 0.0866 0.4693 0.1130 0.6287 0.3244 EM-UNet 0.1940 0.4307 0.2156 0.6537 0.3735 L-seg 0.1377 0.4687 0.1816 0.4968 0.3212 EM-L-seg 0.2365 0.4904 0.2869 0.6436 0.4144 CE-Net 0.1596 0.5085 0.2981 0.6568 0.4058 EM-CE-Net 0.1596 0.4982 0.3404 0.6494 0.4119 UNet++ 0.2110 0.4828 0.2742 0.6730 0.4103 EM-UNet++ 0.2057 0.5364 0.3237 0.6898 0.4389

[0089] To further demonstrate the effectiveness of the proposed method, EM-Net (with EM-UNet++ as the best performer) is compared with state-of-the-art segmentation results as shown in Tables 3 and 4. Table 3 shows the comparison results of different fundus lesion segmentation methods provided in the embodiments of this application on the IDRiD dataset. Table 4 shows the comparison results of different fundus lesion segmentation methods provided in the embodiments of this application on the DDR dataset. The first three rows of Table 3 correspond to the three leading teams in the IDRiD 2018 challenge. It is easy to observe that the average AUC value of EM-Net exceeds that of all the compared methods, and the framework proposed in this invention achieves the best performance in the segmentation of HE and EX lesions. This further demonstrates the effectiveness of this application in fundus image lesion segmentation.

[0090] Table 3

[0091] VRT 0.4951 0.6804 0.6995 0.7127 0.6469 PATech 0.4740 0.6490 - 0.8850 0.6693 iFLYTEK-MIG 0.5017 0.5588 0.6588 0.8741 0.6483 L-seg 0.4420 0.6742 0.6717 0.8679 0.6639 EAD-Net 0.2408 0.5649 0.6083 0.7818 0.5490 PMCNet 0.4694 0.6705 0.7111 0.8724 0.6808 SAA 0.4152 0.6704 0.7281 0.8812 0.6738 MSLF-Net 0.4393 0.6411 0.7597 0.8644 0.6761 SimCLR 0.4970 0.6810 0.6420 0.7960 0.6540 M2MRF 0.4843 0.6694 0.6900 0.8229 0.6667 CLC-Net 0.5460 0.6610 0.6720 0.8270 0.6765 EM-UNet++ 0.5069 0.7020 0.6998 0.8913 0.7000

[0092] Table 4

[0093] L-seg 0.1377 0.4687 0.1816 0.4968 0.3212 PMCNet 0.1994 0.3987 0.3164 0.5430 0.3644 RTNet 0.1176 0.3656 0.2943 0.5671 0.3362 SAA 0.1933 0.4456 0.3747 0.6269 0.4102 DMF 0.1194 0.3652 0.3260 0.6014 0.3530 EM-UNet++ 0.2057 0.5364 0.3237 0.6898 0.4389

[0094] This application demonstrates its advantages and feasibility by conducting a fair quantitative comparison using baseline methods and current representative techniques on different datasets and visualizing the experimental results. The proposed EM-Net network architecture is characterized by its stepwise mapping to the EM algorithm (Expectation-Maximization algorithm), designed to solve prior-based optimization problems. Each module of the network corresponds to each operator of the EM algorithm, thus possessing clear interpretability and specific physical meaning. This property facilitates simple analysis to deeply understand how the network operates internally during training and testing, exploring its working principles. Specifically, due to its good correspondence with prior-based models, this application, while achieving the studied fundus lesion segmentation task, generates a convolutional dictionary representing healthy fundus images and distribution parameters for conveying lesion knowledge. These information, as byproducts of the network, can be extended to other related tasks, demonstrating good generalization ability and potential usability.

[0095] This application also provides a retinal lesion segmentation device 800 based on prior knowledge of fundus images, such as... Figure 8 As shown, the device includes: a preprocessing module 801, a feature expression establishment module 802, a fundus image probability model establishment module 803, a probability distribution expression acquisition module 804, an optimization problem establishment module 805, an iterative solution module 806, an expectation-maximization network acquisition module 807, an optimized network acquisition module 808, and a lesion segmentation result acquisition module 809.

[0096] The preprocessing module 801 is used to perform preprocessing steps on the raw fundus image data to obtain training data and test data. The preprocessing steps include reshaping operation, augmentation operation, and segmentation operation.

[0097] The feature expression building module 802 is used to build feature expressions based on the prior features of the image. The feature expression is a sum of the foreground and background features, where the foreground features represent lesion features and the background features represent non-lesion features.

[0098] The fundus image probability model module 803 is used to establish expressions for the foreground and background parts based on feature expressions, and to establish a fundus image probability model based on the expressions for the foreground and background parts.

[0099] The probability distribution expression acquisition module 804 is used to obtain the posterior probability distribution expression related to the parameters to be determined based on the fundus image probability model. The parameters to be determined include the feature map used to fit the background and the latent variables introduced during modeling.

[0100] An optimization problem module 805 is established to transform the posterior probability distribution expression into a log-posterior function expression associated with it, and to transform the maximization problem of the log-posterior function into a minimization problem associated with the parameters to be solved.

[0101] The iterative solution module 806 is used to perform iterative solution steps using the expectation-maximization algorithm and the proximal gradient descent algorithm. The iterative solution steps include a first step and a second step. The first step is a lesion segmentation operation, which subtracts the background portion obtained in the second step from the original fundus image to obtain the foreground portion as input for the first step. The second step is a non-lesion background fitting operation, which establishes iterative update expressions for relevant parameters.

[0102] The expected-maximization network module 807 is used to obtain its closed-form solution based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. The deep residual network is used to perform learnable proximal operator operation steps in the closed-form solution. The first step is to perform lesion segmentation operation steps by the existing segmentation network, thereby obtaining the expected-maximization network.

[0103] The optimized network module 808 is used to input the data to be trained into the expectation-maximization network for training steps, thereby obtaining the optimized expectation-maximization network.

[0104] The lesion segmentation result acquisition module 809 is used to input the test data into the trained expectation maximization network for forward inference to obtain the lesion segmentation result.

[0105] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0106] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0107] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0108] like Figure 9As shown in the figure, this application embodiment also provides a retinal lesion segmentation server based on prior knowledge of fundus images, including a memory 901 and a processor 902; the memory 901 is used to store computer-executable instructions; the processor 902 is used to execute the computer-executable instructions to implement the retinal lesion segmentation method based on prior knowledge of fundus images described above in this application embodiment.

[0109] This application also provides a computer-readable storage medium storing executable instructions. When a computer executes the executable instructions, it can implement the retinal lesion segmentation method based on prior knowledge of fundus images described above in this application.

[0110] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0111] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations.

[0112] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for segmenting retinal lesions based on prior knowledge of fundus images, characterized in that, include: The raw fundus image data is preprocessed to obtain training data and test data; wherein, the preprocessing steps include reshaping operation, augmentation operation and segmentation operation; A feature expression is established based on the prior features of the image; wherein the feature expression is the sum of the foreground and background parts, the foreground part is the lesion feature, and the background part is the non-lesion feature; Based on the feature expression, expressions for the foreground and background parts are established, and a fundus image probability model is established based on the expressions for the foreground and background parts. The posterior probability distribution expression related to the parameters to be determined is obtained based on the fundus image probability model; wherein, the parameters to be determined include the feature map used to fit the background and the latent variables introduced by the modeling. The posterior probability distribution expression is transformed into its associated log-posterior function expression, and the maximization problem of the log-posterior function is transformed into a minimization problem related to the parameters to be solved. The iterative solution steps utilize the expectation-maximization algorithm and the proximal gradient descent algorithm; wherein, the iterative solution steps include a first step and a second step; The first step is a lesion segmentation operation step, which subtracts the background part obtained from solving the second step from the original fundus image to obtain the foreground part as the input of the first step; The second step is a background fitting operation for non-lesion areas, establishing iterative update expressions for relevant parameters; The closed-form solution is obtained based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. The deep residual network is used to execute learnable proximal operator operation steps in the closed-form solution. The first step is to perform lesion segmentation operation steps by the existing segmentation network, thereby obtaining the expectation maximization network. The training data is input into the expectation-maximization network for training, thereby obtaining the optimized expectation-maximization network. The test data is input into the trained expectation-maximization network for forward inference to obtain the lesion segmentation result.

2. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 1, characterized in that, The feature expression established based on the prior features of the image is: X = F + B; where X is the fundus image, F is the foreground portion, B is the background portion, and X, F, B ∈ R. H×W×3 R is the real number field, H is the height of the fundus image, and W is the width of the fundus image.

3. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 1, characterized in that, The expression for the foreground portion is: Here, Z is an introduced latent variable, with its height and width coordinates denoted as (h, w), and the element in the k-th channel represented as Z. hwk It satisfies Z hwk ∈{0,1} and This indicates the category to which each pixel belongs, where K represents the number of mixed components, with K-1 parts representing K-1 type lesions and the other part representing non-lesion components, and F... hw ∈R 3 This represents the colored pixel at coordinates (h, w) in the foreground region F, where R is the real number field. Let σ be the mean of the k-th group in the Gaussian mixture model. k Let μ be the variance of the k-th group in the Gaussian mixture model, and let μ,σ be the variance of μ. k and σ k For a vector k = 1, 2, ..., K, I is the identity matrix and N represents a Gaussian distribution; The expression for the background portion is: Among them, C l ∈R p×p×3 M represents the l-th convolutional kernel of size p. l ∈R H×W It is the feature map associated with it, and L is the number of convolutional kernels in the model.

4. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 1, characterized in that, The established probabilistic model for fundus images is as follows: Among them, X hw ∈R 3 This represents the color pixel at coordinates (h, w) in the fundus image X, where C and M are C0, W ... l (l=1,2,…,L) and M l The tensor formed by stacking (l=1,2,…,L) has parameters C, μ, σ that are shared across the entire dataset and can be learned from the training data. The feature map M and the latent variable Z need to be re-estimated for each new sample. The obtained posterior probability distribution expression is: Where P(M) represents the prior distribution based on feature map M.

5. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 1, characterized in that, The expression for the logarithmic posterior function is: The optimization process for feature map M is as follows: in, Let g(M) be the Hadamard product, and g(M) be the regularization term related to P(M).

6. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 1, characterized in that, The expression for the first step is: The closed-form solution is obtained based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. A deep residual network is used to execute learnable proximal operator operations in the closed-form solution. The first step involves the existing segmentation network performing lesion segmentation operations, thereby obtaining the expectation-maximizing network, including: The expression for iteratively updating feature map M is: M t =argmin M Q(M,M t-1 ); where M t-1 This is the current update result; The quadratic approximation Q(M,M) t-1 The expression for ) is in, The quadratic approximation Q(M,M) t-1 The closed-form solution to the minimization problem of the feature map M is: Among them, prox g It is a proximal operator related to g. The first step involves performing lesion segmentation using an existing segmentation network; the segmentation network is expressed as follows: in, For the segmentation network used, α t For network parameters; F (t) It is the foreground part, serving as the input for the first step; The network module for solving the feature map M is Where Conv(·) is the convolution operation, Conv T (·) represents the transpose convolution operation. Indicates the parameter is β t In a deep residual network, W(·) and U(·) are element-wise operators.

7. The retinal lesion segmentation method based on prior knowledge of fundus images according to claim 6, characterized in that, The process of inputting the data to be trained into the expectation-maximization network for training, to obtain an optimized expectation-maximization network, includes: Initialize parameters μ and σ: The first step is performed using different segmentation networks; wherein, the different segmentation networks include UNet, UNet++, Lseg, and CE-Net; The output of the first step is used as part of the input of the second step, and the difference between the original fundus image and the output of the second step is used as the input of the first step. Calculate the loss function of the expectation-maximizing network; The desired maximum loss function for the network is: in, γ represents the segmentation result output at step t of the iterative steps in the first step. tru For the segmented real labels, As the background for the output of the t-th step in the second step of the iterative step, B tru For the real-world context, R(μ,σ) is the regularization term associated with μ and σ, and L ce For cross-entropy loss, λ t For L ce The weighting coefficients, L2 is the mean squared error loss, τ t ∈ represents the weight coefficient of L2, L1 is the 1-norm related to the feature map M, and ∈ represents the weight coefficient of L1. The expression for cross-entropy loss is: Where β and β n These are the weighting coefficients for the two items; The network parameters and optimization loss function of the expectation-maximization network are updated using the stochastic gradient descent algorithm. The training process terminates after reaching the preset iteration steps, thereby obtaining the optimized expectation-maximization network.

8. A retinal lesion segmentation device based on prior knowledge of fundus images, characterized in that, include: The preprocessing module is used to perform preprocessing steps on the raw fundus image data to obtain training data and test data; wherein, the preprocessing steps include reshaping operation, augmentation operation and segmentation operation; A feature expression module is used to establish feature expressions based on prior features of an image; wherein, the feature expression is a summation of the foreground and background parts, the foreground part being lesion features and the background part being non-lesion features; A fundus image probability model module is used to establish expressions for the foreground and background parts based on the feature expressions, and to establish a fundus image probability model based on the expressions for the foreground and background parts. The probability distribution expression acquisition module is used to obtain the posterior probability distribution expression related to the parameters to be determined based on the fundus image probability model; wherein, the parameters to be determined include the feature map used to fit the background and the latent variables introduced by the modeling. An optimization problem module is established to convert the posterior probability distribution expression into a log-posterior function expression associated with it, and to convert the maximization problem of the log-posterior function into a minimization problem associated with the parameters to be solved. An iterative solution module is used to perform iterative solution steps using the expectation-maximization algorithm and the proximal gradient descent algorithm; wherein, the iterative solution steps include a first step and a second step; the first step is a lesion segmentation operation step, which subtracts the background part obtained from the second step from the original fundus image to obtain the foreground part as the input of the first step; the second step is a non-lesion background fitting operation step, which establishes iterative update expressions for relevant parameters; A network module for maximizing expectation is obtained, which is used to obtain its closed-form solution based on the quadratic approximation in the iteratively updated expression. Each step of the calculation is expanded to obtain the corresponding network module. A deep residual network is used to perform learnable proximal operator operation steps in the closed-form solution. The first step is to perform lesion segmentation operation steps by the existing segmentation network, thereby obtaining the network for maximizing expectation. An optimized network module is obtained, which is used to input the data to be trained into the expectation maximization network for training steps, thereby obtaining the optimized expectation maximization network; The lesion segmentation result acquisition module is used to input the test data into the trained expectation-maximization network for forward inference to obtain the lesion segmentation result.

9. A retinal lesion segmentation server based on prior knowledge of fundus images, characterized in that, Including memory and processor; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions, which, when executed by a computer, enable the implementation of the method as described in any one of claims 1-7.

Images