Hybrid Sampling-Based CT Cell Image Clarification Combining Method and System

Abstract

The present invention proposes a combined method and system for CT cell image clarification based on hybrid sampling. The method includes performing hybrid sampling on CT cell images of different layers respectively through an edge enhancement-based method and a visual segmentation model, mixing the sampling results and the segmentation results in proportion to obtain a hybrid sampling result, calculating the similarity of all cell images in the hybrid sampling result to obtain different-layer cell images of the same cell; using a clarity judgment index to compare and select different-layer cell images of different cells, and selecting the cell image with the highest clarity for each type; arranging and combining the cell images with the highest clarity together according to the cell positions of the original CT cell images to generate a high-resolution CT cell image. The present invention effectively avoids excessive calculation time while meeting the high quality of the sampling results, greatly improves the efficiency and quality of the image clarification task, and the present invention can also be used for image clarification and enhancement of movie film archives.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a combined method and system for CT cell image sharpening based on hybrid sampling. Background Art

[0002] Image sharpening is to improve the sharpness and quality of an image through a series of algorithms and techniques, making the details and information in the image more obvious and easy to understand. It is a key step from image processing to image analysis and the foundation of image engineering. Image sharpening has a wide range of applications in areas such as computer vision and biomedical image analysis. The most important step in the image sharpening task is to sample the image. The hybrid sampling method consists of a graphics sampling method based on edge enhancement and a sampling method based on a visual segmentation model, which highly coincides with the image sharpening task.

[0003] In the combined method for CT cell image sharpening, the edge detection algorithm is particularly important. The algorithm obtains the position and shape of cells by identifying the edges of the targets in the image, that is, the places where the image intensity changes the most.

[0004] Common methods include gradient-based edge detection methods, deep learning-based edge detection methods, etc. The gradient-based edge detection method can well reflect the details and complexity of the image by using operators such as Sobel and Canny. This method is simple and efficient, but it is sensitive to noise, resulting in incomplete cell edges extracted. The deep learning-based edge detection method can automatically learn and adapt to various complex edge characteristics, but it has a high time cost and requires a large amount of labeled data and computing resources. Another key problem is that the positions and sizes of the same cells in CT images of different layers are not consistent, and simple direct replacement will cause problems of dislocation and ghosting, greatly affecting the quality of image sharpening. In order to improve the efficiency and effect of image sharpening, it is very important to sample CT images efficiently and with high quality. Summary of the Invention

[0005] In view of the above situation, the main purpose of the present invention is to propose a combined method and system for CT cell image sharpening based on hybrid sampling to solve the above technical problems.

[0006] The present invention proposes a combined method for CT cell image sharpening based on hybrid sampling, and the method includes the following steps:

[0007] Step 1: Sample CT cell images of different layers in an edge-enhanced manner to obtain the sampling results of different cell images of different layers;

[0008] Step 2: Use a visual segmentation model to precisely segment cell objects in CT cell images of different layers, and obtain the segmentation results of different cell images of different layers;

[0009] Step 3: Mix the sampling results and the segmentation results in proportion to obtain a mixed sampling result, calculate the similarity of all cell images in the mixed sampling result, obtain the corresponding relationship of the same cell in different layers according to the similarity result, and obtain the cell images of different layers of the same cell;

[0010] Step 4: Use a clarity judgment index to compare different cell images of different layers, and select the cell image with the highest clarity for each type from the comparison;

[0011] Step 5: Arrange and combine the selected cell images with the highest clarity in pixel precision units according to the cell positions of the original CT cell images to generate a high-resolution CT cell image.

[0012] The present invention also proposes a CT cell image clarity combination system based on mixed sampling. Among them, the system applies the CT cell image clarity combination method based on mixed sampling as described above. The system includes:

[0013] Sampling module, used for:

[0014] Sample CT cell images of different layers in a way based on edge enhancement to obtain the sampling results of different cell images of different layers;

[0015] Use a visual segmentation model to precisely segment cell objects in CT cell images of different layers, and obtain the segmentation results of different cell images of different layers;

[0016] Retrieval module, used for:

[0017] Mix the sampling results and the segmentation results in proportion to obtain a mixed sampling result, calculate the similarity of all cell images in the mixed sampling result, obtain the corresponding relationship of the same cell in different layers according to the similarity result, and obtain the cell images of different layers of the same cell;

[0018] Comparison module, used for:

[0019] Use a clarity judgment index to compare different cell images of different layers, and select the cell image with the highest clarity for each type from the comparison;

[0020] Combination module, used for:

[0021] Arrange and combine the selected cell images with the highest clarity in pixel precision units according to the cell positions of the original CT cell images to generate a high-resolution CT cell image.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] The present invention combines edge enhancement and a visual segmentation model. It not only has the efficient calculation of the edge enhancement operator but also satisfies the accurate segmentation of the cell region by the visual segmentation model. Then, the results of the two are mixed to calculate the cell similarity between different levels. Finally, the cell region with the highest clarity is selected through a recognized image clarity determination index to generate a high-resolution CT image. This method significantly improves the efficiency and quality of image clarification and effectively solves the problem of excessive time consumption in previous methods.

[0024] The additional aspects and advantages of the present invention will be partly given in the following description, partly will become apparent from the following description, or can be learned through the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of a combined method for CT cell image clarification based on hybrid sampling proposed by the present invention;

[0026] Figure 2 It is a partial data graph intercepted from the test data that needs to be combined for image clarification in the present invention;

[0027] Figure 3 It is a partial result image of sampling the image sequence that needs to be clarified using the edge enhancement method with the test data in the present invention;

[0028] Figure 4 It is a result image of partitioning the image sequence that needs to be clarified using the test data in the present invention;

[0029] Figure 5 It is a partial result image of sampling the image sequence that needs to be clarified using the visual segmentation model with the test data in the present invention

[0030] Figure 6 It is a result image of clarity comparison and selection for the cell data of hybrid sampling using the test data in the present invention

[0031] Figure 7 It is a comparison graph of the effects before and after the combined clarification of the image that needs to be clarified using the test data in the present invention;

[0032] Figure 8 It is the overall framework of a combined system for CT cell image clarification based on hybrid sampling proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0034] Referring to the following description and the accompanying drawings, these and other aspects of the embodiments of the present invention will become clear. In these descriptions and drawings, some specific embodiments of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited by this.

[0035] In the process of clarifying CT cell images, problems such as noise interference, image misalignment, and high computational cost are mainly faced. Edge detection based on gradients is sensitive to noise, resulting in incomplete extraction of cell edges. Although deep learning-based methods are accurate, they have high time costs and require a large amount of labeled data. In addition, the positions and sizes of cells in different layers of CT images are inconsistent, and direct replacement will cause misalignment and ghosting, affecting image quality. To solve these problems, the present invention proposes a method for clarifying CT cell images based on hybrid sampling. This method combines edge enhancement and a visual segmentation model, which not only calculates efficiently with the edge enhancement operator but also satisfies the accurate segmentation of the cell region by the visual segmentation model. Then, the results of the two are combined to calculate the cell similarity between different levels. Finally, the cell region with the highest clarity is selected through a recognized image clarity determination index to generate a high-resolution CT image. This method significantly improves the efficiency and quality of image clarification and effectively solves the problem of excessive time consumption in previous methods.

[0036] In this embodiment, some test data is given for verification. The test data is 20 cell CT images with an original resolution exceeding 9391×9391 taken by the Celigo Image Cytometer produced by Nexcelom Corporation. As Figure 2 shown, Figure 2 is a partial data image intercepted that needs to be combined for image clarification. It should be noted that the resolution of these 20 images is only the resolution used for testing and does not represent the upper limit of the usage quantity and resolution of the present invention. At the same time, the CT image acquisition instrument is only a reference instrument, and other instruments are also applicable.

[0037] Please refer to Figure 1 , this embodiment provides a method for combining the clarification of CT cell images based on hybrid sampling. The method includes the following steps:

[0038] Step 1: Sample the CT cell images of different layers in an edge-enhanced manner to obtain the sampling results of different cell images of different layers;

[0039] In the said step 1, the CT cell images of different layers are sampled in a way based on edge enhancement to obtain the sampling results of different cell images of different layers. The specific method includes the following steps:

[0040] Based on the edge-enhanced sampling method, the Roberts edge enhancement operator is used to amplify the edges of the cells in the image. The Roberts operator detects edge lines through local difference calculation. The templates of the Roberts operator can be expressed in the horizontal and vertical directions as follows:

[0041] ;

[0042] Among them, 、 respectively represent the horizontal gradient and the vertical gradient of an image at the horizontal direction gradient and the vertical direction gradient at a certain position.

[0043] The sum of the Roberts operator pixels can be expressed as:

[0044] ;

[0045] Among them, represents edge amplification, respectively represent the horizontal coordinate and the vertical coordinate of the pixel;

[0046] The cell region in the image is subjected to a closing operation. The closing operation includes a dilation operation and an erosion operation. By using the dilation operation, the cell region in the image is expanded to fill the hollow part, and the closing operation is used to connect the gaps or breaks inside the cell edges to smooth the boundary without changing the area. The closing operation can be expressed as:

[0047] ;

[0048] Among them, represents the dilation operation, represents the erosion operation, represents the original image, represents the structuring element, represents the closing operation.

[0049] The calculation process of the erosion operation has the following relational expression:

[0050] ;

[0051] Among them, represents the coordinate where the structuring element moves to the image.

[0052] According to the set exclusion threshold, the closed region areas in the results of the closing operation are preliminarily screened and excluded to obtain the sampling results of different cells in different layers. The following relationship exists in the process of screening and exclusion:

[0053] ;

[0054] Among them, represents the pixel valid region in the closed region, represents the pixel value, and the range of the pixel value is 0 - 255, represents the total number of pixels in the closed region, represents the exclusion threshold, The value range of is , in this embodiment , Figure 3 represents the total area of the closed region. Some sampling results are as shown in

[0055] To maximize the optimization of the invention efficiency, for the results after preliminary screening, the image is divided into N partitions according to the N×N specification, and the cells at the intersection line are assigned to the (N 2 +1)-th partition. To optimize the invention efficiency, only the cells in the same partition are used to calculate the corresponding relationships between subsequent cells, and the cells with poor quality are further screened to obtain the final sampling results. The results are as shown in 2 +1)-th partition. To optimize the invention efficiency, only the cells in the same partition are used to calculate the corresponding relationships between subsequent cells, and the cells with poor quality are further screened to obtain the final sampling results. The results are as shown in Figure 4 , and the demonstration example is divided into 3×3.

[0056] Step 2: Use a visual segmentation model to accurately segment the cell objects in the CT cell images of different layers to obtain the segmentation results of different cell images in different layers;

[0057] In this embodiment, the visual segmentation model consists of: (1) Image encoder: Use ViT (Vision Transformer) to process the input image, which is minimally applicable to processing high-resolution inputs. (2) Prompt encoder: Process different forms of segmentation prompts and convert them into embedding representations. (3) Mask decoder: Combine the image embeddings from the image encoder and the prompt embeddings from the prompt encoder to predict the segmentation mask and generate the final segmentation result. Some sampling results are as shown in Figure 5 .

[0058] The process of guiding the model training using the focal loss and discriminator loss functions specifically includes the following steps;

[0059] Given a training set, the training set includes the given CT cell images and their corresponding true labels;

[0060] Input the given CT cell image into the visual segmentation model to obtain the prediction results, and construct the focal loss using the prediction results and the ground truth labels. The focal loss has the following relationship:

[0061] ;

[0062] ;

[0063] Among them, represents the cross-entropy, represents the focal loss, represents the balance factor used to adjust the importance of positive and negative samples, represents the predicted probability that the class label is 1, represents the ground truth label, represents the predicted probability of the model for the true class, represents the adjustable focusing parameter, and its value range is [0, 5]. In this embodiment, = 2.

[0064] Input the given CT cell image into the visual segmentation model to obtain the predicted segmentation results. Based on the objective function of the Dice coefficient, construct the discriminator loss using the prediction results and the ground truth labels. The discriminator loss has the following relationship:

[0065] ;

[0066] Among them, represents the Dice coefficient, which is used to measure the similarity or overlap degree between two volume elements, represents the th pixel value in the predicted segmentation result, and the pixel value range is 0 - 255, represents the th pixel value in the ground truth segmentation result, and the pixel value range is 0 - 255.

[0067] Add the focal loss and the discriminator loss as the total loss, and optimize the visual segmentation model by minimizing the total loss. The total loss has the following relationship:

[0068] ;

[0069] Among them, represents the total loss.

[0070] Step 3: Mix the sampling results and the segmentation results in proportion to obtain mixed sampling results. Calculate the similarity of all cell images in the mixed sampling results, and obtain the corresponding relationship of the same cell in different layers according to the similarity results, so as to obtain cell images of the same cell in different layers. An image is regarded as a pixel value matrix in imaging. In the present invention, the four-channel representation method is used to represent the sampling result as (x0, y0, h, w), where x0 and y0 represent the initial coordinates of the sampling matrix, and h and w represent the length and width of the matrix respectively.

[0071] In the said Step 3, the following relational expression exists in the calculation process of the similarity between different cells:

[0072] ;

[0073] ;

[0074] where represents the overlapping area, respectively represent the initial x coordinate, initial y coordinate, width, and height of the first region, respectively represent the initial x coordinate, initial y coordinate, width, and height of the second region.

[0075] Step 4: Use the clarity judgment index to compare and select the cell images of different cells in different layers, and select the cell image with the highest clarity for each type from the comparison and selection;

[0076] In this embodiment, the clarity judgment index adopted is a generally recognized image clarity judgment index such as NIQE or average gradient. The cell comparison and selection results are as Figure 6 shown.

[0077] NIQE is a natural image quality evaluator, which is based on a set of "quality perception" features and fits them into the MVG model. NIQE can be expressed as:

[0078] ;

[0079] where , represents the mean vector of the natural MVG model and the distorted image MVG model, , represents the covariance matrix of the natural MVG model and the distorted image MVG model.

[0080] Average Gradient: It is used to measure the clarity of an image and reflects the contrast of tiny details and texture transformation features in the image. The average gradient can be expressed as:

[0081] ;

[0082] Among them, represents the average gradient, where m and n are the width and height of the image respectively, represents the image gray value of the pixel at the position, and represent the change rates of the image in the horizontal and vertical directions.

[0083] Step 5: Arrange and combine the cell images with the highest clarity selected in pixel-precision units according to the cell positions of the original CT cell images to generate a high-resolution CT cell image. The comparison of the effects before and after the combination result of the CT cell image clarification is as Figure 7 shown.

[0084] Please refer to Figure 8 , this embodiment also provides a CT cell image clarification and combination system based on hybrid sampling. Among them, the system applies the CT cell image clarification and combination method based on hybrid sampling as described above. The system includes:

[0085] Sampling module, used for:

[0086] Sampling the CT cell images of different layers in an edge-enhanced manner to obtain the sampling results of different cell images of different layers;

[0087] Using a visual segmentation model to accurately segment the cell objects in the CT cell images of different layers to obtain the segmentation results of different cell images of different layers;

[0088] Retrieval module, used for:

[0089] Mixing the sampling results and the segmentation results proportionally to obtain a mixed sampling result, calculating the similarity of all cell images in the mixed sampling result, and obtaining the corresponding relationship of the same cell in different layers according to the similarity result to obtain the cell images of the same cell in different layers;

[0090] Comparison and selection module, used for:

[0091] Comparing and selecting the cell images of different layers of different cells using a clarity judgment index, and selecting the cell image with the highest clarity for each type from the comparison;

[0092] Combination module, used for:

[0093] Arranging and combining the cell images with the highest clarity selected in pixel-precision units according to the cell positions of the original CT cell images to generate a high-resolution CT cell image.

[0094] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0095] In the description of this specification, the description referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0096] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but should not be construed as a limitation to the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims

1. A CT cell image sharpening combination method based on mixed sampling, characterized in that: The method comprises the following steps: Step 1, sampling CT cell images of different layers using an edge enhancement-based method to obtain sampling results of different cell images of different layers; Step 2: Use a visual segmentation model to segment cell objects in CT cell images of different layers to obtain segmentation results of different cell images of different layers; Step 3: Mix the sampling result and the segmentation result in proportion to obtain a mixed sampling result, perform similarity calculation on all cell images in the mixed sampling result, obtain the correspondence between the same cell in different layers according to the similarity result, and obtain cell images of different layers of the same cell; Step 4, using the clarity judgment index to compare the cell images of different layers of different cells, and selecting the cell image with the highest clarity in each comparison; Step 5: Arrange and combine the selected cell images with the highest definition according to the cell positions of the original CT cell images with pixel accuracy to generate a high-resolution CT cell image.

2. The CT cell image sharpening combination method based on hybrid sampling according to claim 1, characterized in that: In step 1, the CT cell images of different layers are sampled in an edge-enhancement-based manner, and the method for obtaining the sampling results of different cell images of different layers specifically includes the following steps: Based on the edge enhancement sampling method, the edge enhancement operator is used to amplify the edges of cells in the image. The corresponding process has the following relationship: ； in, represents the edge gain, Represent the horizontal and vertical coordinates of the pixel respectively. represents the horizontal gradient, Represents the gradient in the vertical direction; The cell area in the image is closed, and the corresponding process has the following relationship: ； in, represents the expansion operation, represents the corrosion operation, represents the original image, Represents a structural element, Indicates a closing operation; According to the set exclusion threshold, the closed area in the closing operation result is initially screened and excluded to obtain the sampling results of different cells in different layers. The screening and exclusion process has the following relationship: ； in, Indicates the effective area of ​​pixels in the closed area. Represents pixel value, the range of pixel value is 0-255, Represents the total number of pixels in the closed area, represents the exclusion threshold, Represents the total area of ​​the enclosed region.

3. The CT cell image sharpening combination method based on hybrid sampling according to claim 2, characterized in that: The calculation process of the horizontal gradient has the following relationship: ； in, A function representing the pixel values ​​of an image.

4. The CT cell image sharpening combination method based on hybrid sampling according to claim 3, characterized in that: The calculation process of the gradient in the vertical direction has the following relationship: 。 5. The CT cell image sharpening combination method based on hybrid sampling according to claim 4, characterized in that: The calculation process of the corrosion operation has the following relationship: ； in, Indicates the coordinates where the structure element is moved to on the image.

6. The CT cell image sharpening combination method based on hybrid sampling according to claim 5, characterized in that: In step 2, the visual segmentation model includes: Image encoder for handling high-resolution input; A cue encoder for processing segmentation cues in different forms into embedded representations; A mask decoder is used to combine the image embedding from the image encoder and the hint embedding from the hint encoder to predict the segmentation mask and generate the final segmentation result.

7. The CT cell image sharpening combination method based on hybrid sampling according to claim 6, characterized in that: In the step 2, during the visual segmentation model training process, the focal loss and the discriminator loss function are used to guide the model training. The process of using the focal loss and the discriminator loss function to guide the model training specifically includes the following steps: Given a training set, the training set includes a given CT cell image and a corresponding true label; The given CT cell image is input into the visual segmentation model to obtain the prediction result, and the focus loss is constructed using the prediction result and the true label. The focus loss has the following relationship: ； ； in, represents the cross entropy, represents the focal loss, represents the balance factor, represents the predicted probability of the class label being 1, represents the true label, represents the model's predicted probability for the true category, Indicates adjustable focusing parameters; The given CT cell image is input into the visual segmentation model to obtain the predicted segmentation result. Based on the objective function of the Dice coefficient, the discriminator loss is constructed using the predicted result and the true label. The discriminator loss has the following relationship: ； in, Represents the Dice coefficient, which is used to measure the similarity or overlap between two volume elements. Indicates the predicted segmentation result pixel values, Indicates the true segmentation result pixel value; The focal loss and the discriminator loss are added as the total loss, and the visual segmentation model is optimized by minimizing the total loss. The total loss has the following relationship: ； in, Represents the total loss.

8. The CT cell image sharpening combination method based on hybrid sampling according to claim 7, characterized in that: In step 3, the calculation process of the similarity between different cells has the following relationship: ； ； in, represents the overlap area, They represent the initial x-coordinate, initial y-coordinate, width, and height of the first area respectively. They represent the initial x coordinate, initial y coordinate, width, and height of the second area respectively.

9. A CT cell image sharpening combined system based on hybrid sampling, characterized in that: The system applies the CT cell image sharpening combination method based on mixed sampling as described in any one of claims 1 to 8, and the system comprises: Sampling modules for: The CT cell images of different layers are sampled based on edge enhancement to obtain the sampling results of different cell images of different layers; The visual segmentation model is used to segment the cell objects in the CT cell images of different layers, and the segmentation results of different cell images of different layers are obtained; Retrieval module for: The sampling result and the segmentation result are mixed in proportion to obtain a mixed sampling result, similarity calculation is performed on all cell images in the mixed sampling result, and the correspondence between the same cell in different layers is obtained according to the similarity result to obtain cell images of different layers of the same cell; Comparison module, used for: The cell images of different layers of different cells are compared using the clarity judgment index, and the cell image with the highest clarity is selected from the comparison; Combination modules for: The selected cell images with the highest definition are arranged and combined according to the cell positions of the original CT cell images with pixel accuracy to generate a high-resolution CT cell image.

Images