Cell image segmentation method based on large language model and deep watershed algorithm

Abstract

The invention relates to a cell image segmentation method based on a large language model and a deep watershed algorithm, and the method is characterized in that the method comprises the following steps: obtaining a to-be-segmented cell image; performing semantic segmentation on the cell image by using a general segmentation model to distinguish a foreground and a background of the cell; and performing instance segmentation on the preprocessed cell image by adopting an improved deep watershed algorithm to identify and segment a single cell instance. Wherein the general segmentation model and the improved deep watershed algorithm are suitable for unstained and stained cell images, the problems of excessive segmentation and false positive segmentation of cell instances can be effectively solved, and the segmentation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cell segmentation, and in particular to a cell image segmentation method based on a large language model and a deep watershed algorithm. Background Art

[0002] In the field of bio-omics research and disease diagnosis, accurate segmentation of cell images is crucial for a deep understanding of the structure, function and pathological state of cells. Accurate cell segmentation can help researchers better understand the structure and function of cells, as well as play an important role in disease diagnosis and treatment. Due to the different characteristics of cells, including different contour shapes, adhesion and overlap, accurate cell segmentation becomes a challenge.

[0003] Traditional cell segmentation methods are mainly based on image processing and machine learning techniques, such as threshold segmentation, edge detection, region growing, etc. These methods can complete the cell segmentation task to a certain extent, but are often limited by their ability to handle complex cell structures and dynamic changes. For example, when cells are closely adhered or severely overlapped, these methods may not provide accurate segmentation results.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a cell image segmentation method based on a large language model and a deep watershed algorithm, aiming to solve the problems of over-segmentation and false positive segmentation in the cell segmentation method in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In the first aspect, a cell image segmentation method based on a large language model and a deep watershed algorithm comprises:

[0008] S1. Obtain the cell image to be segmented, and use the large language general model to perform semantic segmentation on the cell image to distinguish the cell foreground from the cell background;

[0009] S2. An improved deep watershed algorithm is used to perform instance segmentation on the preprocessed cell images to identify and segment individual cell instances.

[0010] Furthermore, the semantic segmentation of the cell image using the large language general model includes:

[0011] S11. For a cell image dataset obtained by an electron microscope, mark its cell boundaries;

[0012] S12, through the single sample learning method, the large language general model classifies each pixel of the original image according to the prompt word instruction;

[0013] S13. Based on the pre-trained large language general model, semantic segmentation is performed on the set of cell images collected under an electron microscope, aiming to divide the image into different semantic regions, including cell regions and background regions, by assigning each pixel to a predefined semantic category.

[0014] Furthermore, the step of using the improved deep watershed algorithm to perform instance segmentation on the preprocessed cell image includes:

[0015] S21. Connect the semantic segmentation result obtained in the first part with the original RGB format cell image to create a four-channel image. The purpose of the connection is to convert the background pixels in the cell image to 0, so that the network can focus on the boundaries of segmented clustered cell instances.

[0016] S22. Using the marked cell boundaries, calculate the energy drop direction map in the X and Y directions of each input image, and calculate the watershed energy level map; let I be a binary image, where the foreground pixel set is F and the background pixel set is B; for any pixel p∈I, the result of the Euclidean distance transform D(p) is defined as the Euclidean distance from p to the nearest foreground pixel; this can be expressed mathematically as:

[0017]

[0018] Where ||pf|| represents the Euclidean distance between pixels; for a two-dimensional image, if the coordinates of pixels p and f are (x p ,y p ) and (x f ,y f ), then the Euclidean distance is given by the formula:

[0019]

[0020] Therefore, the complete formula for the Euclidean distance transform D(p) can be written as:

[0021]

[0022] This formula indicates that for each pixel in the image, the Euclidean distance to all foreground pixels is calculated, and the minimum value is taken as the Euclidean distance transform; after obtaining the energy gradient field map of the cell instance, we use the function imgradientxy to calculate the descending direction map of the energy gradient field; this function calculates the gradient of a two-dimensional image in the horizontal x and vertical y directions, and returns two matrices representing the gradient values ​​in these respective directions;

[0023]

[0024] Energy i represents a pixel in the energy gradient field map, and Respectively represent the gradient of the pixel in the horizontal (x direction) and vertical (y direction);

[0025] S23. The original deep watershed network is composed of two parts: a direction prediction network and a watershed energy level prediction network. The direction prediction network is used to extract image features and predict the direction in which the energy of each pixel drops. The feature map of the middle convolutional layer is then extracted and fused with the energy drop prediction map. The energy level prediction network is then used to predict the watershed energy level map based on the fusion result.

[0026] Furthermore, the S23 includes:

[0027] S231, replace the VGG16 network with the VGG19 network to enhance the network depth and feature extraction capabilities; at the same time, change the third and fourth pooling layers to average pooling to reduce the size of the feature map while maintaining key features; delete the fifth pooling layer to simplify the network structure and improve computational efficiency;

[0028] S232, first change the third and fourth pooling layers to average pooling, and delete the fifth pooling layer; in feature fusion, the first path and the second path are a 5×5 convolution and two 1×1 convolutions, and the third path is four 5×5 convolutions. The feature fusion method is added to the network, the second feature extraction path is upsampled by a 4×4 convolution kernel, and the third feature extraction path is upsampled by an 8×8 convolution kernel;

[0029] S233. In order to maintain the spatial resolution and improve the feature fusion effect, the Concatenate fusion energy gradient field descent map is used to add the features of each layer to improve the feature fusion effect;

[0030] S234. The network performs three 1×1 convolutions to deepen the network, followed by upsampling, and finally outputs an energy drop direction map in the X and Y directions; this output map reflects the internal structure and boundary features of the cell instance, providing key information for accurate cell instance segmentation.

[0031] Furthermore, the S23 also includes:

[0032] S235. In order to train the watershed energy network to learn the energy drop direction of each pixel, a weighted training loss function combining the dice loss and the mean square error loss in the angle domain is adopted; the loss function is defined as:

[0033] La =0.8L direction +0.2L dice

[0034] Among them, L direction represents the direction loss, which is used to measure the difference between the predicted direction and the true direction, similar to “positioning training”; and L dice Represents Dice loss, which is used to evaluate the accuracy of the segmentation results, similar to "accuracy training"; L direction and L dice Combined, they form a powerful training system;

[0035] Specifically, L direction The calculation is as follows:

[0036]

[0037] Here, P obj represents the set of pixels that belong to the semantic class with object instances, which is related to the object that needs to be accurately segmented; w p It is a weighting factor proportional to the inverse of the square root of the area of ​​the object instance, which is used to emphasize the importance of small objects. Small objects need to be segmented more accurately during the segmentation process. This factor is calculated based on the area of ​​the object instance and is the inverse of the square root of the area of ​​the object instance. If an object is small, its pixels will get a larger weight. The loss is calculated by using the mean squared angular error instead of the cosine distance.

[0038] Dice loss function L dice It is a commonly used evaluation indicator in image segmentation tasks, which reflects the degree of overlap between the predicted label and the true label; its calculation formula is as follows:

[0039]

[0040] Among them, |A∩B| represents the intersection of the predicted segmentation result A and the true segmentation result B, |A| and |B| represent the set sizes of the predicted and true segmentation results respectively; the lower the Dice loss value, the better the segmentation performance;

[0041] Through the modified cross entropy loss function L b To train the direction prediction network, the function is defined as:

[0042]

[0043] Among them, t p，k and p，k denote the kth element of the target vector for pixel p and the kth channel of the network output, w p is a coefficient used to adjust the importance of small objects, ck is a scaling constant specific to each discretization class.

[0044] Further, in said S23, during the training process, the network is pre-trained using the real semantic segmentation and direction prediction as input and the discretized real modified watershed transform as the target;

[0045] S236. For the watershed energy gradient field prediction module, the input of this module is the energy drop direction map in the X and Y directions. The main function of this module is to learn and infer a watershed transformation map with 16 energy levels. This module is mainly composed of two groups of convolution blocks, each of which is followed by a 2×2 average pooling layer to reduce the size of the feature map and retain key features. Subsequently, a group of 1×1 convolutions and an 8×8 upsampling are performed to restore the spatial resolution of the feature map. Through the processing of the above network architecture, an energy gradient map of the cell instance is finally obtained. There are 16 energy levels in this map, and the 0 energy level corresponds to the background of the image, while higher energy levels correspond to areas closer to the inside of the cell instance object. This image reflects the internal structure and boundary characteristics of the cell instance. According to the set energy threshold, the cell instance is finally segmented to achieve accurate cell instance segmentation.

[0046] The technical solution adopted by the present invention has the following beneficial effects:

[0047] This paper proposes an innovative cell instance segmentation framework based on a large language general model and an improved deep watershed algorithm for instance segmentation, aiming to achieve accurate segmentation of cell images.

[0048] Optimization of the segmentation process: In the first part, the segmentation process is optimized by introducing the step of semantic segmentation. Although this step increases the complexity of the segmentation method, it significantly improves the final segmentation accuracy. Compared with the prior art, the first step of the present invention can effectively improve the accuracy of cell segmentation with only one labeled sample. The method of the present invention greatly reduces the implementation cost, thereby significantly improving the segmentation quality while maintaining efficient calculation.

[0049] Application of universal segmentation model: This paper adopts a universal segmentation model of single-sample learning to distinguish the foreground and background of cells. The goal is to enable the algorithm to focus on distinguishing individual cell instances without being disturbed by background information. This method proposed by the present invention helps to accurately delineate cell boundaries and improve the efficiency of instance segmentation algorithms;

[0050] Improvement of the deep watershed algorithm: In the second part, the improved deep watershed instance segmentation framework of the present invention performs accurate cell instance segmentation, paying special attention to the adhesion areas between cells and irregular over-segmentation. The energy gradient field method does not directly target the edge contour of the object, but the gradient field inside the object. Unlike most instance segmentation algorithms that rely on contour fitting, this algorithm improves the energy gradient field to achieve pixel-level instance segmentation. This method provides a more refined representation of the internal structure of the object and enhances the accuracy and robustness of the segmentation results.

[0051] This paper proposes a large language universal model using single-sample learning to distinguish the foreground and background of cells, improves the deep watershed transform network to segment instances between different cells, and uses this algorithm to successfully solve the over-segmentation problem when traditional networks such as the Unet series are fused with traditional watershed algorithms. It greatly improves the efficiency of subsequent instance segmentation models. This advancement solves the false positive problem of cell segmentation, improves the accuracy of instance segmentation, and effectively solves the problem of over-segmentation, which is significantly better than other state-of-the-art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The present invention provides a flowchart of semantic segmentation based on single sample learning and instance segmentation based on an improved deep watershed algorithm;

[0053] Figure 2 A schematic diagram of a semantic segmentation process based on a single-sample learning general large language model is provided for the present invention;

[0054] Figure 3 The present invention provides a schematic flow chart of an instance segmentation model of an improved deep watershed algorithm;

[0055] Figure 4 This is a schematic diagram of a cell sample of an experimental embodiment;

[0056] Figure 5 This is a schematic diagram of the direction prediction network architecture;

[0057] Figure 6 Schematic diagram of the watershed energy level prediction network architecture. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] See also Figure 1 , Figure 1The cell image segmentation method based on a large language model and a deep watershed algorithm proposed in the present invention is described in the text. The method is divided into two parts: semantic segmentation and instance segmentation.

[0060] Part 1: General semantic segmentation model based on single-shot learning

[0061] like Figure 2 As shown, in order to achieve accurate cell instance segmentation, the present invention first proposes a general segmentation model based on single sample learning to distinguish the foreground and background of cell images. It includes:

[0062] Data collection and data labeling: Cell image datasets are acquired through electron microscopy and cell boundaries are marked.

[0063] Single-sample learning: Through the single-sample learning method, the general segmentation model classifies each pixel of the original image according to the prompt word instruction. On the pre-trained general segmentation model, semantic segmentation is performed for the set of cell images collected under the electron microscope, aiming to divide the image into different semantic regions, including cell regions and background regions, by assigning each pixel to a predefined semantic category. The goal of this step is to identify the cell region in the image, rather than segmenting individual cells;

[0064] Part II: Instance Segmentation Model with Improved Deep Watershed Algorithm

[0065] like Figure 3 As shown, based on the semantic segmentation in the first part, the present invention further proposes an instance segmentation model of an improved deep watershed algorithm, aiming to solve the problem of accurate segmentation of overlapping cells. The key features and implementation steps of the framework are as follows:

[0066] Input image: Concatenate the semantic segmentation result (single-channel image) obtained in the first part with the original RGB format cell image (three-channel image) to create a four-channel image. The purpose of the connection is to convert the background pixels in the cell image to 0, so that the network can focus on segmenting the boundaries of clustered cell instances.

[0067] Input label: Using the marked cell boundaries, calculate the energy drop direction map in the X and Y directions of each input image, and calculate the watershed energy level map; let I be a binary image, where the foreground pixel set is F and the background pixel set is B; for any pixel p∈I, the result of the Euclidean distance transform D(p) is defined as the Euclidean distance from p to the nearest foreground pixel; this can be expressed mathematically as:

[0068]

[0069] Where ||pf|| represents the Euclidean distance between pixels; for a two-dimensional image, if the coordinates of pixels p and f are (x p ,y p ) and (x f ,y f ), then the Euclidean distance is given by the formula:

[0070]

[0071] Therefore, the complete formula for the Euclidean distance transform D(p) can be written as:

[0072]

[0073] This formula means that for each pixel in the image, the Euclidean distance to all foreground pixels is calculated, and the minimum value is taken as the Euclidean distance transform.

[0074] After obtaining the energy gradient field map of the cell instance, we use the function imgradientxy to calculate the descending direction map of the energy gradient field; this function calculates the gradient of a two-dimensional image in the horizontal x and vertical y directions and returns two matrices representing the gradient values ​​in these respective directions;

[0075]

[0076] Energy i A pixel in the energy gradient field map, and They represent the gradient of the pixel in the horizontal (x direction) and vertical (y direction) directions respectively.

[0077] Experimental Example: Figure 4 As shown, Figure 4 a is a small piece of the image obtained by the microscope after being cut. Figure 4 b is the result of semantic segmentation, Figure 4 c and 4d are the energy drop direction maps in the x and y directions obtained by calculating the semantic segmentation results. Figure 4 e is the energy gradient map obtained after calculating the semantic segmentation result.

[0078] Improvement of network architecture: The original deep watershed network is composed of two parts: the direction prediction network and the watershed energy level prediction network. The direction prediction network is used to extract image features and predict the direction of energy drop of each pixel. Then the feature map of the middle convolutional layer is extracted and fused with the energy drop prediction map. Then the energy level prediction network is used to predict the watershed energy level map based on the fusion result. The present invention modifies the direction prediction network structure to meet the special needs of cell image segmentation. Specifically including:

[0079] The modular network is trained to learn the energy falling direction for each pixel.

[0080] Replacement and adjustment of network architecture Figure 5 As shown in the figure: the VGG16 network is replaced with the VGG19 network to enhance the network depth and feature extraction capabilities. At the same time, the third and fourth pooling layers are changed to average pooling to reduce the size of the feature map while maintaining key features; the fifth pooling layer is deleted to simplify the network structure and improve computational efficiency.

[0081] Feature fusion and upsampling: First, the third and fourth pooling layers are changed to average pooling, and the fifth pooling layer is deleted; in feature fusion, the first and second paths are a 5×5 convolution and two 1×1 convolutions, and the third path is 4 5×5 convolutions. The feature fusion method is added to the network, the second feature extraction path is upsampled by a 4×4 convolution kernel, and the third feature extraction path is upsampled by an 8×8 convolution kernel;

[0082] Fusion of feature maps: In order to maintain spatial resolution and improve the feature fusion effect, Concatenate is used to fuse the energy gradient field descent map and add the features of each layer to improve the feature fusion effect.

[0083] Final output: The network performs three 1×1 convolutions to deepen the network, followed by upsampling, and finally outputs an energy drop direction map in the X and Y directions. This output map reflects the internal structure and boundary features of the cell instance, providing key information for accurate cell instance segmentation.

[0084] Loss function: In order to train the watershed energy network to learn the energy drop direction of each pixel, the present invention adopts a weighted training loss function that combines the dice loss and mean square error loss in the angle domain. The loss function is defined as:

[0085] L a =0.8L direction +0.2L dice

[0086] Among them, L direction represents the direction loss, which is used to measure the difference between the predicted direction and the true direction, similar to “positioning training”; and L dice Represents Dice loss, which is used to evaluate the accuracy of the segmentation results, similar to "accuracy training"; L direction and L dice Combined, they form a powerful training system;

[0087] Specifically, L direction The calculation is as follows:

[0088]

[0089] Here, P obj represents the set of pixels that belong to the semantic class with object instances, which is related to the object that needs to be accurately segmented; w p It is a weighting factor proportional to the inverse of the square root of the area of ​​the object instance, used to emphasize the importance of small objects; small objects need to be segmented more accurately during the segmentation process. This factor is calculated based on the area of ​​the object instance and is the inverse of the square root of the area of ​​the object instance. If an object is small, its pixels will get a larger weight; the loss is calculated by using the mean squared angular error instead of the cosine distance.

[0090] Dice loss function L dice It is a commonly used evaluation indicator in image segmentation tasks, which reflects the degree of overlap between the predicted label and the true label; its calculation formula is as follows:

[0091]

[0092] Watershed energy gradient field prediction module: The input of this module is the energy drop direction map in the X and Y directions. The main function of this module is to learn and infer the watershed transformation map with 16 energy levels. This process involves the following key steps:

[0093] Network architecture such as Figure 6 As shown: This module mainly consists of two groups of convolution blocks. Each group of convolution blocks is followed by a 2×2 average pooling layer to reduce the size of the feature map and retain key features. Subsequently, a group of 1×1 convolutions and an 8×8 upsampling are performed to restore the spatial resolution of the feature map.

[0094] Generation of energy gradient map: Through the processing of the above network architecture, the energy gradient map of the cell instance is finally obtained. There are 16 energy levels in this map, 0 energy level corresponds to the background of the image, and higher energy levels correspond to areas closer to the interior of the cell instance object. This image reflects the internal structure and boundary characteristics of the cell instance.

[0095] Segmentation of cell instances: According to the set energy threshold, the cell instances are finally segmented, thereby achieving accurate cell instance segmentation and solving the problems of over-segmentation and false positive segmentation of cell instances.

[0096] The present invention has achieved remarkable advanced results in experiments. Specifically, through comparative analysis, we found that the method using feature one (i.e., pre-semantic segmentation) improved the accuracy of cell segmentation by 8.4 percentage points and the recall by 9.7 percentage points compared to the method of directly performing instance segmentation. In addition, by comparing the improved deep watershed algorithm with the original deep watershed algorithm, we found that the improved algorithm improved the accuracy by 18.1 percentage points and the recall by 30.1 percentage points.

[0097] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the schemes disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims.

Claims

1. A cell image segmentation method based on a large language model and a deep watershed algorithm, characterized in that: include: S1. Obtain the cell image to be segmented, and use the large language general model to perform semantic segmentation on the cell image to distinguish the cell foreground from the cell background; S2. An improved deep watershed algorithm is used to perform instance segmentation on the preprocessed cell images to identify and segment individual cell instances.

2. The cell image segmentation method based on a large language model and a deep watershed algorithm according to claim 1, characterized in that: The use of a large language general model to perform semantic segmentation on a cell image comprises: S11. For a cell image dataset obtained by an electron microscope, mark its cell boundaries; S12, through the single sample learning method, the large language general model classifies the pixels of the original image according to the prompt word instructions; S13. Based on the pre-trained large language general model, semantic segmentation is performed on the set of cell images collected under an electron microscope, aiming to divide the image into different semantic regions, including cell regions and background regions, by assigning each pixel to a predefined semantic category.

3. The cell image segmentation method based on a large language model and a deep watershed algorithm according to claim 1, characterized in that: The step of using the improved deep watershed algorithm to perform instance segmentation on the preprocessed cell image includes: S21. Connect the semantic segmentation result obtained in the first part with the original RGB format cell image to create a four-channel image. The purpose of the connection is to convert the background pixels in the cell image to 0, so that the network can focus on the boundaries of segmented clustered cell instances. S22. Using the marked cell boundaries, calculate the energy drop direction map in the X and Y directions of each input image, and calculate the watershed energy level map; let I be a binary image, where the foreground pixel set is F and the background pixel set is B; for any pixel p∈I, the result of the Euclidean distance transform D(p) is defined as the Euclidean distance from p to the nearest foreground pixel; this can be expressed mathematically as: Where ||pf|| represents the Euclidean distance between pixels; for a two-dimensional image, if the coordinates of pixels p and f are (x p ,y p ) and (x f ,y f ), then the Euclidean distance is given by the formula: Therefore, the complete formula for the Euclidean distance transform D(p) can be written as: This formula indicates that for each pixel in the image, the Euclidean distance to all foreground pixels is calculated, and the minimum value is taken as the Euclidean distance transform; after obtaining the energy gradient field map of the cell instance, we use the function imgradientxy to calculate the descending direction map of the energy gradient field; this function calculates the gradient of a two-dimensional image in the horizontal x and vertical y directions, and returns two matrices representing the gradient values ​​in these respective directions; Energy i represents a pixel in the energy gradient field map, and Respectively represent the gradient of the pixel in the horizontal (x direction) and vertical (y direction); The Euclidean distance in this part can be replaced by other geometric distances, such as Manhattan distance, Chebyshev distance, and information entropy; S23. The original deep watershed network is composed of two parts: a direction prediction network and a watershed energy level prediction network. The direction prediction network is used to extract image features and predict the direction in which the energy of each pixel drops. The feature map of the middle convolutional layer is then extracted and fused with the energy drop prediction map. The energy level prediction network is then used to predict the watershed energy level map based on the fusion result.

4. The cell image segmentation method based on a large language model and a deep watershed algorithm according to claim 3, characterized in that: The S23 includes: S231, replace the VGG16 network with the VGG19 network to enhance the network depth and feature extraction capabilities; at the same time, change the third and fourth pooling layers to average pooling to reduce the size of the feature map while maintaining key features; delete the fifth pooling layer to simplify the network structure and improve computational efficiency; S232, first change the third and fourth pooling layers to average pooling, and delete the fifth pooling layer; in feature fusion, the first path and the second path are a 5×5 convolution and two 1×1 convolutions, and the third path is 4 5×5 convolutions. The feature fusion method is added to the network, the second feature extraction path is upsampled by a 4×4 convolution kernel, and the third feature extraction path is upsampled by an 8×8 convolution kernel; S233. In order to maintain the spatial resolution and improve the feature fusion effect, the Concatenate fusion energy gradient field descent map is used to add the features of each layer to improve the feature fusion effect; S234. The network performs three 1×1 convolutions to deepen the network, followed by upsampling, and finally outputs an energy drop direction map in the X and Y directions; this output map reflects the internal structure and boundary features of the cell instance, providing key information for accurate cell instance segmentation.

5. The cell image segmentation method based on a large language model and a deep watershed algorithm according to claim 3, characterized in that: The S23 also includes: S235. In order to train the watershed energy network to learn the energy drop direction of each pixel, a weighted training loss function combining the Dice loss and the mean square error loss in the angle domain is adopted; the loss function is defined as: L a =0.8L direction +0.2L dice Among them, L direction represents the direction loss, which is used to measure the difference between the predicted direction and the true direction; and L dice Represents Dice loss, which is used to evaluate the accuracy of the segmentation results; L direction and L dice Combined, they form a powerful training system; Specifically, L direction The calculation is as follows: Here, P obj represents the set of pixels that belong to the semantic class with object instances, which is related to the object that needs to be accurately segmented; w p It is a weighting factor proportional to the inverse of the square root of the area of ​​the object instance, which is used to emphasize the importance of small objects. Small objects need to be segmented more accurately during the segmentation process. This factor is calculated based on the area of ​​the object instance and is the inverse of the square root of the area of ​​the object instance. If an object is small, its pixels will get a larger weight. The loss is calculated by using the mean squared angular error instead of the cosine distance. Dice loss function L dice It is a commonly used evaluation indicator in image segmentation tasks, which reflects the degree of overlap between the predicted label and the true label; its calculation formula is as follows: Among them, |A∩B| represents the intersection of the predicted segmentation result A and the true segmentation result B, |A| and |B| represent the set sizes of the predicted and true segmentation results respectively; the lower the Dice loss value, the better the segmentation performance; Through the modified cross entropy loss function L b To train the direction prediction network, the function is defined as: Among them, t p，k and p，k denote the kth element of the target vector for pixel p and the kth channel of the network output, w p is a coefficient used to adjust the importance of small objects, c k is a scaling constant specific to each discretization class.

6. The cell image segmentation method based on a large language model and a deep watershed algorithm according to claim 5, characterized in that: In S23, during the training process, the network is pre-trained using the true semantic segmentation and direction prediction as input and the discretized true modified watershed transform as a target; S236. For the watershed energy gradient field prediction module, the input of this module is the energy drop direction map in the X and Y directions. The main function of this module is to learn and infer a watershed transformation map with 16 energy levels. This module is mainly composed of two groups of convolution blocks, each of which is followed by a 2×2 average pooling layer to reduce the size of the feature map and retain key features. Subsequently, a group of 1×1 convolutions and an 8×8 upsampling are performed to restore the spatial resolution of the feature map. Through the processing of the above network architecture, an energy gradient map of the cell instance is finally obtained. There are 16 energy levels in this map, and the 0 energy level corresponds to the background of the image, while higher energy levels correspond to areas closer to the inside of the cell instance object. This image reflects the internal structure and boundary characteristics of the cell instance. According to the set energy threshold, the cell instance is finally segmented to achieve accurate cell instance segmentation. The number of energy levels can be any positive integer.

Images