A Cell Image Instance Segmentation Method Based on Bayesian Inference and Boosting

Abstract

A cell image instance segmentation method based on Bayesian inference and Boosting. For the instance segmentation of cell images, since it contains very complex image features (low foreground-background contrast, large image noise, high cell distribution density, cell adhesion, multi-layer cell overlap, etc.), it is very difficult to accurately segment each cell individual from the pathological image. Some of the methods in the past research can only segment out some cells, or require a large amount of data sets and computing power. Therefore, we urgently need a method that can completely segment each cell without consuming a lot of computing power, and the segmentation effect can achieve the expected method. The present invention introduces a multi-level set energy function and a minimum description length to perform contour reasoning, and then adds multi-layer cascade Boosting for classification and regression, so that each target in the image can be identified and accurately segmented.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, in particular to the field of medical image or cell image processing, and proposes a cell image instance segmentation method based on Bayesian inference and Boosting driving. Background Art

[0002] With the booming development of computer vision and artificial intelligence, image processing, particularly the analysis and processing of digital pathology images, has garnered widespread attention from various industries. The quality of digital pathology image analysis and processing directly impacts doctors' diagnoses and the health and safety of patients. However, due to the large number and variety of digital pathology images, as well as their complex features (such as low foreground-background contrast, high cell density, overlapping layers of cells, and high image noise), the analysis and processing of digital pathology images is extremely complex and often yields less than ideal results. Therefore, developing a method or algorithm that effectively processes complex pathology images is a challenging and pressing issue in the field of image processing.

[0003] In recent years, the fields of computer vision, artificial intelligence, and image processing have achieved remarkable success, with many of these achievements finding widespread application in real-world applications. However, significant room for improvement and advancement remains in the processing of digital pathology images, whether cellular or glandular. While numerous methods exist for medical image analysis, most are limited to simple cases and exhibit poor robustness for more complex scenarios. For example, traditional image processing methods include graph theory, active contour models, and watershed algorithms; while machine learning methods include support vector machines, random forests, and the EM algorithm. In recent years, driven by the surge in popularity of artificial intelligence, deep learning technology has gained widespread favor among researchers. This rise has significantly improved the accuracy and precision of image segmentation, enabling convolutional neural networks to effectively represent images. However, deep learning's impressive performance relies on massive datasets, which require significant computing power and a lengthy training process. Therefore, there is an urgent need for a method that can effectively process complex pathology images without requiring extensive computing power. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a cell image instance segmentation method based on Bayesian inference and Boosting drive, construct a multi-level set energy function, and accurately describe the target contour through minimum description length optimization and multi-layer Boosting classification regression.

[0005] The present invention provides a cell image instance segmentation method based on Bayesian inference and Boosting driving.

[0006] Step S1: First, we need to construct these multi-level set energy functions for our segmentation model to control and optimize;

[0007] In step S2, combined with the minimum description length theory, we can perform a secondary optimization on the energy function optimized in step S1 to make the inferred contour smoother.

[0008] Step S3 uses the Boosting classification model to classify the objects in the image one by one according to the image features, and we use multiple Boosting modules to implement this process to prevent missing any image information and lock the target area;

[0009] In step S4, the Boosting model can not only perform classification, but also perform regression. It locates the target boundary through iterative regression, thereby enhancing its classification accuracy and creating favorable conditions for steps S1 and S2.

[0010] Step S5 uses evaluation indicators to perform qualitative and quantitative evaluation on the segmentation results;

[0011] As a further technical solution of the present invention, in step S1, a multi-level set energy function is constructed based on various features of a given image. The process of determining the final contour morphology is actually a process of minimizing these constructed energy functions. The specific method is:

[0012] Step S11. The multi-level set energy function can be expressed as:

[0013]

[0014] The first term on the right side of the equation represents the energy function required to infer the outline of a single cell; represents the zero level set function and Intersection, that is, the second term is the energy function of overlapping cells in the image; N is the total number of cells in the image; i , j Respectively expressed as i , j cells; the two terms on the right side of the equation can be expressed as follows:

[0015]

[0016] , , κ ∈R , is the regularization term, is the length term of the contour, H (·) is the Heaviside function, g =1, h c (·) represents the binary shape prior, and the negative sign is used to control the non-positive region of the level set function.

[0017] In order to minimize the value of the level set energy function, the most direct and effective way is to find the gradient, that is, to derive the energy function and then set it to zero. The principle followed is the gradient descent principle. The function value changes the most along the direction of the negative gradient. When the gradient value is zero, the function value no longer changes, and the whole process reaches convergence, that is, the right-hand unary function term in step S11 E u (·) and the binary function term E b When the negative gradient values ​​of (·) are all zero, the total energy value reaches the minimum, the evolution process ends, and the final contour is formed.

[0018] Furthermore, in step S2, a minimum description length optimization item is introduced based on step S1. The minimum description length principle says that we first give a set of data containing boundary point coordinate information. , which is achieved by using an encoding scheme ,in is the parameter estimated from the above data. Can be divided into two parts, one is the parameter description length , used for encoding model, the other part is the data description length , so the total description length It can be defined as the sum of the parameter description length and the data description length, so the representation of the minimum description length can be obtained.

[0019] The minimum description length is expressed as follows:

[0020]

[0021] The second term on the right side of the equation is equivalent to Therefore, it can be said that the process of using the minimum description length to infer the contour is also the process of using the Bayesian model to infer the contour.

[0022]

[0023] l It means we set l control points, Represents the variance, so that the corresponding probability distribution can be expressed in the form of a Gaussian distribution

[0024]

[0025] in , B i Representative 2 n i ×2 l The matrix, c i It represents the 2 coordinates of these control points. l ×1 column vector, the minimum description length is actually achieved by optimizing l , c i , These parameters are used to optimize the image. i The size of a cell nucleus is h i × w i , and the minimum description length of these parameters is estimated to be

[0026]

[0027] Combining the above formulas, it is not difficult to deduce l The optimal solution

[0028]

[0029] express yes l function, and is therefore affected by l The influence of , so that the minimum value of the total description length can be obtained

[0030]

[0031] Furthermore, in step S3, a Boosting algorithm is used to implement a method for classifying different features in an image. Each Boosting module is regarded as a weak learner, which is continuously iteratively trained and finally converted into a strong learner to better represent and learn the image. The Boosting algorithm includes a function model, an objective function, and an optimization algorithm.

[0032] In step S31, we first construct an accumulation function model as follows:

[0033]

[0034] in, MRespectively represent the number of iterations of the Boosting algorithm, C 0 (·) represents the initial function, Represents the function increment.

[0035] Step S32 takes the function model constructed in step S31 as an example, adopts the greedy strategy as the optimization algorithm to optimize each iteration process, and represents these weak learners as , so the optimization process can be obtained as follows:

[0036]

[0037] Considering that the learning process of the Boosting module is complex and lengthy, the image features represented by each training are inevitably different from the real ones. These differences can be expressed by constructing a loss function as the objective function. By optimizing the loss function, the model can be optimized. The loss function can use the binomial log-likelihood loss function:

[0038]

[0039] It is not difficult to see from the above formula that this is the process used to achieve two-category classification. For multi-category classification, you can refer to the above formula to make changes. y k ∈{-1,1}, ,and , p k ( x ) is as follows:

[0040]

[0041] Therefore, the parameter The optimization result can be described as:

[0042]

[0043] In addition, the Boosting classification process can also be vividly described as a probabilistic binary tree decision classification process. Taking binary classification as an example, the left and right halves of the tree are respectively p - ( x )and p + ( x ) to indicate that a threshold is set at the same time ε ,if p + ( x )-1 / 2> ε , then the prediction is the right half of the tree, on the contrary, ifp - ( x )-1 / 2> ε , then the prediction is the left half of the tree. Each iteration of training will generate a new binary tree on its current node, which is used to predict the current residual of each category on a probability scale until the end of training. Combined with the final estimate C M ( x ), it is not difficult to derive the expression of the probability of the left and right parts of the tree as:

[0044] , .

[0045] Just enough , the above is the whole process of Boosting classification.

[0046] Furthermore, in step S4, based on step S3, the excellent generalization regression of the Boosting algorithm can be used to make the model locate the target contour more accurately. The specific method is:

[0047] Step S41 is similar to step S3. Each boosting block is regarded as a weak regressor. The parameter iteration method still adopts the greedy strategy. Combined with the formula of step S3, the optimization process of the objective function is:

[0048]

[0049] According to the gradient descent method, it is not difficult to conclude that the direction of the fastest gradient descent, that is, the negative gradient direction, is the optimal parameter solution to be found, that is:

[0050]

[0051] It is not difficult to conclude that the model update changes to the following form:

[0052]

[0053] Furthermore, in step S5, the objective evaluation of the performance of the digital pathology image detection and segmentation algorithm is the key to the application of the algorithm in actual pathology diagnosis. The quality of the detection and segmentation results requires qualitative and quantitative evaluation. Therefore, after the above steps are implemented, this step needs to be implemented to evaluate the segmentation results of the model. Commonly used evaluation indicators include F1 score, Dice coefficient, accuracy (Acc), and intersection over union (mIoU). The calculation results of these indicators depend on the number of false negative (FN), false positive (FP), true negative (TN), and true positive (TP) cells in the image. The total number of cells N = FN + FP + TN + TP. The calculation formulas for these evaluation indicators are as follows:

[0054]

[0055]

[0056]

[0057]

[0058] Note: In the F1 score, PR and RC represent the precision and recall respectively, and their calculation formulas are as follows:

[0059] ,In the Dice coefficient, X and Y represent two different objects.

[0060] The advantages of the present invention are that the method of constructing multiple level set energy functions can effectively handle the overlapping and adherent cell clusters that are common in images. At the same time, the introduction of a multi-layer boosting module can help better locate the target area and target contour. Finally, the minimum description length optimization term is added to smooth the segmented contour after segmentation, and it does not rely on large data sets. The image segmentation research method of the present invention not only helps to improve the image segmentation effect, but also provides theoretical support for the segmentation of complex digital pathology images.

[0061] The present invention is mainly completed by using algorithm modeling and image analysis. The method has a clear logical process and strong repeatability, and has certain reference significance. Users can also adjust the model details and specific operations in the method according to their actual application needs, so as to apply it to other types of image analysis, thereby not only limiting the method of the present invention to specific image categories. The overall application prospects of the method are broad and have good use value. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The contour inference process for the minimum description length. Figure 2 is the decision process of the probabilistic binary tree. Figure 3 It is a multi-layer cascade Boosting prediction process. Figure 4 Schematic diagram of the method principle of the present invention. DETAILED DESCRIPTION

[0063] See also Figure 4 This embodiment provides a cell image instance segmentation method based on Bayesian inference and Boosting driving, which includes the following steps:

[0064] S1. Construct a multi-level set energy function based on the various features of the given image. By taking the derivative of these energy functions and setting them to 0, the final contour shape is determined. In fact, this is the process of minimizing these constructed energy functions. The specific method is as follows.

[0065] The multi-level set energy function in step S11 can be expressed as:

[0066] (1)

[0067] The first term on the right side of the equation represents the energy function required to infer the outline of a single cell; represents the zero level set function and Intersection, that is, the second term is the energy function of overlapping cells in the image; N is the total number of cells in the image; Represents the level set and Intersection area; i , j Respectively expressed as i , j cells; the two terms on the right side of the equation can be expressed as follows:

[0068] (2)

[0069] , , κ ∈ R , is the regularization term, is the length term of the contour, H (·) is the Heaviside function, g =1, h c (·) represents the binary shape prior, and the negative sign is used to control the non-positive region of the level set function.

[0070] After the energy function is constructed in step S12, the most direct and effective way to minimize the value of the level set energy function is to find the gradient, that is, to derive the energy function and then set it to zero. The principle followed is the gradient descent principle. The function value changes the most along the direction of the negative gradient. When the gradient value is zero, the function value no longer changes, and the whole process reaches convergence, that is, the right-hand unary function term in step S11 E u (·) and the binary function term E b When the negative gradient values ​​of (·) are all zero, the total energy value reaches the minimum, the evolution process ends, and the final contour is formed.

[0071] The derivative formula of formula (1) is:

[0072] (3)

[0073] According to formula (2), it is not difficult to derive their derivative equations as follows:

[0074] (4)

[0075] in, yes H The derivative of (·) is hour, , otherwise 0, usually .

[0076] By setting the two equations of formula (4) to 0, the minimum value of the energy function can be obtained, and the initial contour inference is completed.

[0077] S2, based on step S1, introduces a minimum description length optimization item. The minimum description length principle says that we first give a set of data containing boundary point coordinate information. , which is achieved by using an encoding scheme ,in is the parameter estimated from the above data. Can be divided into two parts, one is the parameter description length , used for encoding model, the other part is the data description length , so the total description length It can be defined as the sum of the parameter description length and the data description length, so the minimum description length can be expressed as:

[0078] (5)

[0079] The second term on the right side of the equation is equivalent to Therefore, it can be said that the process of using the minimum description length to infer the contour is also the process of using the Bayesian model to infer the contour.

[0080] l It means we set l control points, Represents the variance, so that the corresponding probability distribution can be expressed in the form of a Gaussian distribution

[0081] (6)

[0082] in , B i Representative 2 n i ×2 l The matrix, c i It represents the 2-bit coordinates of the control points. l ×1 column vector, the minimum description length is actually achieved by optimizing l , c i , These parameters are used to optimize the image. i The size of a cell nucleus is h i × w i , and the minimum description length of these parameters is estimated to be

[0083] (7)

[0084] Substituting formulas (6) and (7) into (5) yields l , c i , The minimum values ​​of these parameters , respectively expressed as

[0085] (9)

[0086] express yes l function, and is therefore affected by l The influence of , so that the minimum value of the total description length can be obtained

[0087] (10)

[0088] At this point, the process of optimizing the segmentation contour with the minimum description length is successfully completed. Figure 1It is the minimum description length complete contour inference process.

[0089] S3. Use the Boosting algorithm to implement a method for classifying different features in an image. Each Boosting module is regarded as a weak learner, which is continuously iteratively trained and finally converted into a strong learner to better represent and learn the image. The Boosting algorithm includes a function model, an objective function, and an optimization algorithm.

[0090] In step S31, we first construct an accumulation function model as follows:

[0091] (11)

[0092] in, M Respectively represent the number of iterations of the Boosting algorithm, C 0 (·) represents the initial function, Represents the function increment.

[0093] Step S32 takes the function model constructed in step S31 as an example, adopts the greedy strategy as the optimization algorithm to optimize each iteration process, and represents these weak learners as , so the optimization process can be obtained as

[0094] Considering that the learning process of the Boosting module is complex and lengthy, the image features represented by each training are inevitably different from the real ones. These differences can be expressed by constructing a loss function as the objective function. By optimizing the loss function, the model can be optimized. The loss function can use the binomial log-likelihood loss function:

[0095] (12)

[0096] It is not difficult to see from the above formula that this is the process used to achieve two-category classification. For multi-category classification, you can refer to the above formula to make changes. y k ∈{-1,1}, ,and , p k ( x ) is as follows:

[0097] (13)

[0098] Substituting formula (10) into formula (9) yields the parameter The optimization process is

[0099] (14)

[0100] Therefore, the parameter The optimization result can be described as:

[0101] (15)

[0102] The classification process of Boosting can also be vividly described as the process of probabilistic binary tree decision classification. Figure 2 As shown, taking the binary classification as an example, the left and right halves of the tree are respectively p - ( x )and p + ( x ) to indicate that a threshold is set at the same time ε ,if p + ( x )-1 / 2> ε , then the prediction is the right half of the tree, on the contrary, if p - ( x )-1 / 2> ε , then the prediction is the left half of the tree. Each iteration of training will generate a new binary tree on its current node, which is used to predict the current residual of each category on a probability scale until the end of training. Combined with the final estimate C M ( x ), it is not difficult to derive the expression of the probability of the left and right parts of the tree as:

[0103] , .

[0104] Just enough , the above is the whole process of Boosting classification.

[0105] S4. Based on step S3, the excellent generalization regression of the Boosting algorithm can be used to make the model locate the target contour more accurately. The specific method is:

[0106] Step S41 is similar to step S3. Each boosting block is regarded as a weak regressor. The parameter iteration method still adopts the greedy strategy. Combined with the formula of step S3, the optimization process of the objective function is:

[0107] (16)

[0108] According to the gradient descent method, it is not difficult to conclude that the direction of the fastest gradient descent, that is, the negative gradient direction, is the optimal parameter solution to be found, that is:

[0109] (17)

[0110] It can be deduced from formula (11), and it is not difficult to deduce that the model update changes to the following form:

[0111] (18)

[0112] Figure 3 That is the entire process of training M multi-layer Boosting models to achieve classification and regression.

[0113] S5. Objective evaluation of the performance of digital pathology image detection and segmentation algorithms is the key to applying the algorithms to actual pathology diagnosis. The quality of detection and segmentation results requires qualitative and quantitative evaluation. Therefore, after the above steps are implemented, this step needs to be implemented to evaluate the segmentation results of the model. Common evaluation indicators include F1 score, Dice coefficient, accuracy (Acc), and intersection over union (mIoU). The calculation results of these indicators depend on the number of false negative (FN), false positive (FP), true negative (TN), and true positive (TP) cells in the image. The total number of cells N = FN + FP + TN + TP. The calculation formulas for these evaluation indicators are as follows:

[0114]

[0115]

[0116]

[0117]

[0118] Note: In the F1 score, PR and RC represent the precision and recall respectively, and their calculation formulas are , ,In the Dice coefficient, X and Y represent two different objects.

[0119] The advantages of the present invention are that the method of constructing multiple level set energy functions can effectively handle the overlapping and adherent cell clusters that are common in images. At the same time, the introduction of a multi-layer boosting module can help better locate the target area and target contour. Finally, the minimum description length optimization term is added to smooth the segmented contour after segmentation, and it does not rely on large data sets. The image segmentation research method of the present invention not only helps to improve the image segmentation effect, but also provides theoretical support for the segmentation of complex digital pathology images.

[0120] The present invention is mainly completed by using algorithm modeling and analysis and evaluation indicators. The method has a clear logical process and strong repeatability, and has certain reference significance. Users can also adjust the model details and specific operations in the method according to their actual application needs, so as to apply it to other types of image analysis, thereby not only limiting the method of the present invention to specific image categories. The overall application prospect of the method is broad and has good use value.

Claims

1. A cell image instance segmentation method based on Bayesian inference and Boosting driven, characterized by The steps include: Step S1: First, we need to construct these multi-level set energy functions for our segmentation model to control and optimize. The multi-level set energy function can be expressed as: ； In step S2, combined with the minimum description length theory, we can perform a secondary optimization on the energy function optimized in step S1 to make the inferred contour smoother. Step S3 uses the Boosting classification model to classify the targets in the image one by one according to the image features, and we use multiple Boosting modules to implement this process to prevent missing any image information and lock the target area; in step S3, the Boosting algorithm is used to implement the method of classifying different features in the image, and each Boosting module is regarded as a weak learner, which is continuously iteratively trained and finally converted into a strong learner to better represent the image; the Boosting algorithm includes a function model, an objective function, an optimization algorithm, In step S31, we first construct an accumulation function model as follows: in, M Respectively represent the number of iterations of the Boosting algorithm, C 0 (·) represents the initial function, Represents function increment Step S32 takes the function model constructed in step S31 as an example, adopts the greedy strategy as the optimization algorithm to optimize each iteration process, and represents these weak learners as , so the optimization process can be obtained as follows: Considering that the learning process of the Boosting module is complex and lengthy, the image features represented are inevitably different from the real ones. These differences can be expressed by constructing a loss function as the objective function. The loss function can adopt the binomial log-likelihood loss function: It is not difficult to see that this is the process used to achieve two-category classification. For multi-category classification, you can refer to the above formula to make changes. y k ∈{-1,1}, ,and , p k ( x ) is represented as follows: Therefore, the parameter The optimization process can be described as: In addition, the Boosting classification process can also be vividly described as a probabilistic binary tree decision classification process. Taking binary classification as an example, the left and right halves of the tree are respectively p - ( x )and p + ( x ) to indicate that a threshold is set at the same time ε ,if p + ( x )-1 / 2> ε , then the prediction is the right half of the tree, on the contrary, if p - ( x )-1 / 2> ε , then the prediction is the left half of the tree, combined with the final estimated C M ( x ), it is not difficult to derive the expression of the probability of the left and right parts of the tree as: ， ； In step S4, the Boosting model can not only perform classification, but also perform regression. It locates the target boundary through iterative regression, thereby enhancing its classification accuracy and creating favorable conditions for steps S1 and S2. Step S5 uses evaluation indicators to perform qualitative and quantitative evaluation on the segmentation results.

2. The cell image instance segmentation method based on Bayesian inference and Boosting according to claim 1, characterized in that: In step S1, the multi-level set energy function is constructed according to the various features of the given image. The process of determining the final contour shape is actually the process of minimizing these constructed energy functions. The specific method is as follows: The multi-level set energy function in step S11 can be expressed as: The first term on the right side of the equation represents the energy function required to infer the outline of a single cell; represents the zero level set function and Intersection, that is, the second term is the energy function of overlapping cells in the image; N is the total number of cells in the image; i , j Respectively expressed as i , j cells; the two terms on the right side of the equation can be expressed as follows: , , κ ∈ R , is the regularization term, is the length term of the contour, H (·) is the Heaviside function, g =1, h c (·) represents the binary shape prior, and the negative sign is used to control the non-positive region of the level set function. In order to minimize the value of the level set energy function, the most direct and effective way is to find the gradient, that is, to derive the energy function and then set it to zero. The principle followed is the gradient descent principle. The function value changes the most along the direction of the negative gradient. When the gradient value is zero, the function value no longer changes, and the whole process reaches convergence, that is, the unary function term on the right side of step S11 E u (·) and the binary function term E b When the negative gradient values ​​of (·) are all zero, the total energy value reaches the minimum, the evolution process ends, and the final contour is formed.

3. The cell image instance segmentation method based on Bayesian inference and Boosting according to claim 1, characterized in that: In step S2, an optimization item of minimum description length is introduced based on step S1. The minimum description length principle says that we first give a set of data containing boundary point coordinate information. , which is achieved by using an encoding scheme ,in is the parameter estimated from the above data. Can be divided into two parts, one is the parameter description length , used for encoding model, the other part is the data description length , so the total description length It can be defined as the sum of the parameter description length and the data description length, so the minimum description length can be expressed. The minimum description length is expressed as follows: The second term on the right side of the equation is equivalent to Therefore, it can be said that the process of using the minimum description length to infer the contour is also the process of using the Bayesian model to infer the contour. l It means we set l control points, Represents the variance, so that the corresponding probability distribution can be expressed in the form of a Gaussian distribution in , B i Representative 2 n i ×2 l The matrix, c i It represents the 2-bit coordinates of the control points. l ×1 column vector, the minimum description length is actually achieved by optimizing l , c i , These parameters are used to optimize the image. Now we assume that i The size of a cell nucleus is h i × w i , and the minimum description length of these parameters is estimated to be Combining the above formulas, we can easily deduce express yes l function, and is therefore affected by l The influence of , so that the minimum value of the total description length can be obtained 。 4. The cell image instance segmentation method based on Bayesian inference and Boosting driving according to claim 1, characterized in that: In step S3, the Boosting algorithm classification can better lock the target area, and in step S4, the excellent generalization regression of the Boosting algorithm can make the model locate the target contour more accurately. The specific method is as follows: Step S41 is similar to step S3. The parameter iteration method still adopts the greedy strategy. Combined with the formula of step S3, the optimization process of the objective function is: According to the gradient descent method, it is not difficult to conclude that the direction in which the gradient descent is fastest is the optimal parameter solution to be found, that is: It is not difficult to conclude that the model update changes to the following form: 。 5. The cell image instance segmentation method based on Bayesian inference and Boosting driving according to claim 1, characterized in that: Step S5 is implemented to evaluate the segmentation results of the model. Common evaluation indicators include F1 score, Dice coefficient, accuracy Acc, and intersection over union (mIoU). The calculation formulas for these evaluation indicators are as follows: In the F1 score, PR and RC represent the precision and recall respectively, and their calculation formulas are as follows: In the Dice coefficient, X and Y represent two different objects. The calculation results of these indicators depend on the number of false negative FN, false positive FP, true negative TN and true positive TP cells in the image. The total number of cells N=FN+FP+TN+TP.

Images