Cell experiment image processing method and system based on multiple modes and storage medium

Abstract

The invention discloses a multimodal-based cell experiment image processing method and system and a storage medium, and the method comprises the steps: obtaining a single-channel gray image or a multi-channel fluorescence image containing cytoplasm and cell nucleus characteristics, carrying out the Gaussian filtering of a cytoplasm channel, and carrying out the median filtering of a cell nucleus channel; performing threshold segmentation on the preprocessed cytoplasm channel image by using a maximum between-class variance method to generate an initial cytoplasm mask; performing adaptive threshold segmentation on the preprocessed cell nucleus channel image to generate an initial cell nucleus mask; performing logic OR operation on the initial cytoplasm mask and the initial cell nucleus mask to generate an initial cell region; and finally, calculating each morphological parameter of each cell region and removing the cell regions of which the morphological parameters exceed a preset range to generate a segmentation result of the complete cell region, the morphological parameters and spatial distribution information. The problems that in the prior art, cytoplasm and cell nucleus noise processing segmentation precision is low, overlapped nucleus regions are difficult to separate, and morphological parameter analysis reliability is poor are solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for processing cell experimental images based on multimodality. Background Technology

[0002] In the field of cell experiment image processing, accurately segmenting the cytoplasm and nucleus and extracting their morphological parameters is crucial. However, existing technologies, particularly traditional methods, struggle to effectively filter noise from the cytoplasm and nucleus when processing images containing both cytoplasmic and nuclear features. This often results in incomplete noise smoothing in the cytoplasmic region or residual salt-and-pepper noise in the nuclear region due to a single filtering method. Furthermore, current technologies lack a mechanism to verify the morphological parameters of the initial segmentation results, failing to effectively remove cell regions with abnormal morphological parameters and failing to establish spatial constraints between the cytoplasm and nucleus. This frequently leads to unreasonable segmentation results where the nucleus extends beyond the cytoplasmic region, severely impacting the accuracy of morphological parameter calculations and the reliability of spatial distribution analysis. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies by providing a multimodal cell experiment image processing method, comprising the following steps: S1, acquire a single-channel grayscale image or multi-channel fluorescence image containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and medium filtering to the nuclear channel; S2, apply the maximum inter-class variance threshold segmentation method to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions, remove small noise regions through opening operation, and connect broken regions through closing operation to generate an initial cytoplasmic mask; apply adaptive threshold segmentation to the preprocessed nucleus channel image, combine distance transformation and local maximum point detection to generate nucleus seed points, and separate overlapping nucleus regions through watershed algorithm to generate an initial nucleus mask; S3: Perform a logical OR operation on the initial cytoplasmic mask and the initial nuclear mask to generate the initial cell region, and trim the region where the nucleus extends beyond the cytoplasm; calculate the morphological parameters of each cell region and remove the cell regions whose morphological parameters exceed the preset range, and generate the segmentation result containing the cytoplasmic mask, the nuclear mask and the complete cell region, as well as the morphological parameters and spatial distribution information.

[0004] Preferably, step S2 further includes: If the standard deviation of the image grayscale value is detected to be greater than the first preset threshold or the grayscale difference between the cell region and the background region in the image is less than the second preset threshold, the image is divided into multiple overlapping or non-overlapping sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels, where T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast. If the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background. If the average gradient magnitude of the detected image is less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

[0005] Preferably, step S2 further includes: S201, the image is divided into multiple sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels, where T(x,y) = μ(x,y) – C*σ(x,y), μ is the local average of the gray values ​​of all pixels in the sub-block, σ is the local standard deviation of the gray values ​​of the pixels in the sub-block, and C is a preset constant adjusted according to the image contrast. If the gray value of a pixel in a sub-block is greater than the local threshold, the sub-block is determined to be a cell region; otherwise, it is determined to be the background. An adjusted binary image is generated based on the determination results of all sub-blocks. S202, use the Sobel gradient operator to calculate the horizontal gradient Gx and the vertical gradient Gy respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width. S203, perform a logical OR operation between the adjusted binary image and the clear cell edge image. If the corresponding pixel in the binary image or edge image is the foreground, the fusion result is the foreground; otherwise, it is retained as the background. Use a circular structuring element of a set size to perform a first expansion and then erosion process to connect the broken cell edges. After filling the small hole areas in the image, a complete cell segmentation mask is generated.

[0006] Preferably, step S3 further includes: The morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask are calculated. The morphological evaluation indicators include compactness, roundness and nucleoplasmic ratio. The corresponding quality scores are obtained by calculating the morphological evaluation indicators using a preset quality scoring function. If the quality score is lower than the pre-screening score or the morphological indicators exceed the preset range multiple times, the threshold adjustment will be initiated. If the roundness is lower than the set value, the cytoplasmic threshold will be finely adjusted to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nucleus threshold will be reduced to shrink the nuclear segmentation range. The initial segmentation, morphological optimization, and spatial constraint verification are re-executed using the adjusted thresholds to generate corrected cytoplasmic and nuclear masks.

[0007] This invention also discloses a multimodal cell experiment image processing system, comprising: The image acquisition module is used to acquire single-channel grayscale images or multi-channel fluorescence images containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and medium filtering to the nuclear channel. The segmentation module applies maximum inter-class variance threshold segmentation to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions. It removes small noise regions through opening operations and connects broken regions through closing operations to generate an initial cytoplasmic mask. It applies adaptive threshold segmentation to the preprocessed nucleus channel image, combines distance transform and local maximum point detection to generate nucleus seed points, and separates overlapping nucleus regions through the watershed algorithm to generate an initial nucleus mask. The generation module performs a logical OR operation on the initial cytoplasmic mask and the initial nuclear mask to generate the initial cell region, and trims the region where the cell nucleus extends beyond the cytoplasm. After calculating the morphological parameters of each cell region and removing cell regions whose morphological parameters exceed the preset range, it generates a segmentation result containing the cytoplasmic mask, the nuclear mask, and the complete cell region, as well as morphological parameters and spatial distribution information.

[0008] Preferably, the segmentation module is further configured to, when detecting that the standard deviation of the image grayscale value is greater than a first preset threshold or the grayscale difference between the cell region and the background region in the image is less than a second preset threshold, divide the image into multiple overlapping or non-overlapping sub-blocks, calculate a local threshold T(x,y) for the pixels in each sub-block, T(x,y) = μ(x,y) –C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast; if the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background. When the average gradient magnitude of the image is detected to be less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

[0009] Preferably, the segmentation module is further configured to: This method is used to segment an image into multiple sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average of the gray values ​​of all pixels in the sub-block, σ is the local standard deviation of the gray values ​​of the pixels in the sub-block, and C is a preset constant adjusted according to the image contrast. If the gray value of a pixel in a sub-block is greater than the local threshold, the sub-block is identified as a cell region; otherwise, it is identified as background. An adjusted binary image is generated based on the identification results of all sub-blocks. The Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width. The adjusted binary image and the clear cell edge image are logically ORed. If the corresponding pixel in the binary image or the edge image is the foreground, the fusion result is the foreground; otherwise, it is retained as the background. A circular structuring element of a set size is used to perform a dilation and erosion process to connect the broken cell edges. After filling the small hole areas in the image, a complete cell segmentation mask is generated.

[0010] Preferably, the generation module is further used to calculate the morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask. The morphological evaluation indicators include compactness, roundness, and nucleocytoplasmic ratio. After calculating the morphological evaluation indicators using a preset quality scoring function, the corresponding quality score is obtained. If the quality score is lower than the pre-review score or the morphological indicators exceed the preset range multiple times consecutively, a threshold adjustment is initiated. If the roundness is lower than the set value, the cytoplasmic threshold is fine-tuned to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nuclear threshold is reduced to shrink the nuclear segmentation range. The initial segmentation, morphological optimization, and spatial constraint verification are re-executed using the adjusted thresholds to generate the corrected cytoplasmic mask and nuclear mask.

[0011] The present invention also discloses a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0012] The present invention also discloses a computer-readable storage medium storing a computer program, characterized in that: when the computer program is executed by a processor, it implements the steps of any of the methods described above.

[0013] The present invention provides a multimodal cell experiment image processing method, system, and storage medium. It acquires a single-channel grayscale image or a multi-channel fluorescence image containing cytoplasmic and nuclear features, separates the cytoplasmic and nuclear channels, applies Gaussian filtering to the cytoplasmic channel, and applies mean filtering to the nuclear channel. The preprocessed cytoplasmic channel image is then segmented using the maximum inter-class variance threshold method to generate candidate cytoplasmic regions. Small noise regions are removed by opening operations, and broken regions are connected by closing operations to generate an initial cytoplasmic mask. The preprocessed nuclear channel image is then segmented using adaptive thresholding, combined with distance transform and local maximum detection to generate nuclear seed points. Overlapping nuclear regions are separated using a watershed algorithm to generate an initial nuclear mask. The initial cytoplasmic mask and the initial nuclear mask are then logically ORed to generate initial cell regions, and regions where the nucleus extends beyond the cytoplasm are cropped. Finally, morphological parameters of each cell region are calculated, and cell regions with morphological parameters exceeding a preset range are removed. This process generates a segmentation result containing the cytoplasmic mask, nuclear mask, and complete cell regions, along with morphological parameters and spatial distribution information. This invention addresses the technical problems of insufficient targeting of cytoplasmic and nuclear noise processing, low segmentation accuracy, difficulty in separating overlapping nuclear regions, and poor reliability of morphological parameter analysis in existing technologies. By applying Gaussian filtering and median filtering to suppress different noises in the cytoplasmic and nuclear channels respectively, and combining the maximum inter-class variance method, adaptive threshold segmentation, and watershed algorithm, it achieves accurate segmentation of cytoplasm and nucleus and separation of overlapping nuclei. Furthermore, by verifying morphological parameters and eliminating abnormal regions through spatial constraints, it can generate high-precision segmentation results that include cytoplasmic masks, nuclear masks, and complete cell regions, accurately obtaining cell morphological parameters and spatial distribution information.

[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram illustrating the specific process of the multimodal cell experiment image processing method disclosed in this embodiment.

[0016] Figure 2 This is a schematic diagram illustrating the specific process of dynamically adjusting the segmentation threshold as disclosed in this embodiment.

[0017] Figure 3 This is a schematic diagram of the specific structure of the multimodal cell experiment image processing system disclosed in this embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a limitation of quantity, but rather indicate the presence of at least one.

[0020] In this embodiment, a multimodal cell experiment image processing method is disclosed, as shown in the attached figure. Figure 1 As shown, it can specifically include the following steps.

[0021] Step S1: Obtain a single-channel grayscale image or multi-channel fluorescence image containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and medium filtering to the nuclear channel.

[0022] Specifically, this step supports single-channel grayscale images or multi-channel fluorescence images, such as RGB images of cytoplasmic marker GFP or nuclear marker DAPI, as input images. The input image undergoes channel separation: if it is a fluorescence image, it is separated into a cytoplasmic channel (e.g., green channel) and a nuclear channel (e.g., blue channel). If it is a grayscale image, it is copied for dual-branch processing of cytoplasmic / nuclear analysis.

[0023] Noise filtering is performed. For cytoplasmic branches, a Gaussian filter with a kernel size of 3x3 and σ=1.5 is applied to smooth the noise. For nuclear branches, a medium-range filter, such as a 5x5 window, is applied to remove salt-and-pepper noise. If the fluorescence signal is weak, histogram equalization is performed on the fluorescence channels to enhance contrast.

[0024] Step S2: Apply the maximum inter-class variance threshold segmentation method to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions. Remove small noise regions through opening operation and connect broken regions through closing operation to generate an initial cytoplasmic mask. Apply adaptive threshold segmentation to the preprocessed nucleus channel image, combine distance transformation and local maximum point detection to generate nucleus seed points, and separate overlapping nucleus regions through watershed algorithm to generate an initial nucleus mask.

[0025] In this embodiment, when performing step S2, if low contrast may cause segmentation failure, the process can be switched to an alternative segmentation method step, as detailed below.

[0026] Step S101: If the standard deviation of the image grayscale value is detected to be greater than the first preset threshold or the grayscale difference between the cell region and the background region in the image is less than the second preset threshold, the image is divided into multiple overlapping or non-overlapping sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels, T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast. If the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background.

[0027] Specifically, to reduce image noise such as culture medium impurities and electronic noise, and to avoid interfering with edge detection, a median filter is applied to the input grayscale image, with the window size set to 3x3 or 5x5, which can be adjusted according to the noise intensity. Then, a median filter is performed, where for each pixel (x, y), the median of the grayscale values ​​of all pixels in its neighborhood is taken to replace the original pixel value, thereby smoothing the noise while preserving the edges.

[0028] If the standard deviation of the image grayscale value is greater than a first preset threshold, uneven illumination is determined. Alternatively, if the grayscale difference between the cell region and the background region in the image is less than a second preset threshold, the contrast between the cells and the background is determined to be low. A threshold is automatically calculated based on the local grayscale values ​​to segment the cells and background. The image is divided into multiple overlapping or non-overlapping sub-blocks, such as 16x16 pixels. For each sub-block, a local threshold T(x,y) is calculated: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels within the sub-block, σ is the local standard deviation of the pixel grayscale values ​​within the sub-block, and C is a preset constant adjusted according to the image contrast; the lower the contrast, the larger C is. If the pixel grayscale value of a sub-block is greater than the local threshold T(x,y), the sub-block is determined to be the foreground (i.e., the cell region), and is set to 255; otherwise, it is determined to be the background, and is set to 0. Finally, a preliminary binary image of the cells is generated, adapting to changes in local illumination.

[0029] Step S102: If the average gradient magnitude of the detected image is less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

[0030] When the average gradient magnitude of the image is less than a third preset threshold, the cell edges are considered blurred. Edge detection is then needed to enhance cell boundaries. By extracting cell contours, the problem of blurred or broken boundaries in adaptive thresholding segmentation is resolved. Specifically, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, and the gradient magnitude is calculated. To reduce computational complexity, this can be simplified to absolute value summation: G = |Gx| + |Gy|. The gradient direction θ is calculated, where θ = arctan(Gy / Gx), and is divided into four directions (0°, 45°, 90°, 135°). Non-maximum suppression is performed, meaning that for each pixel, only local maxima along the gradient direction are retained, suppressing non-edge pixels and refining edges.

[0031] Hysteresis thresholding is performed by setting two thresholds: a higher threshold Th and a lower threshold Tl. Typically, Th = 2*Tl. If the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only if it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

[0032] In another specific embodiment, the edge detection step and the threshold segmentation step described above can be used in combination, as follows: Step S201: Divide the image into multiple sub-blocks, and calculate a local threshold T(x,y) for the pixels in each sub-block, where T(x,y) = μ(x,y) –C*σ(x,y), where μ is the local average of the gray values ​​of all pixels in the sub-block, σ is the local standard deviation of the gray values ​​of the pixels in the sub-block, and C is a preset constant adjusted according to the image contrast. If the gray value of a pixel in a sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background. Generate an adjusted binary image based on the determination results of all sub-blocks. This step is similar to step S101 and will not be described in detail.

[0033] Step S202: The Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained. If the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge. If the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge. If the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width. This step is similar to step S102 and will not be described in detail.

[0034] Step S203: Perform a logical OR operation between the adjusted binary image and the clear cell edge image. If the corresponding pixel in the binary image or edge image is the foreground, the fusion result is the foreground; otherwise, it is retained as the background. Use a circular structuring element of a set size to perform a dilation and erosion process to connect the broken cell edges. After filling the small hole areas in the image, a complete cell segmentation mask is generated.

[0035] By combining region information from threshold segmentation and boundary information from edge detection, segmentation loopholes are repaired. Specifically, the binary image from adaptive threshold segmentation (i.e., the result of step S201) is logically ORed with the edge detection result (i.e., the result of step S202). If the corresponding pixel in either the binary or edge image is foreground, the fused result is the foreground; otherwise, it is retained as background. Alternatively, morphological closing operations can be performed: using a 3x3 circular structuring element, first dilate and then erode to connect broken cell edges and fill small holes. Finally, a complete cell segmentation mask is generated, reducing missed or over-segmentation. This method is well-suited for stem cell images with extremely low contrast, such as unstained transparent cytoplasm, or for global threshold segmentation errors caused by Otsu's failure.

[0036] Step S3: Perform a logical OR operation on the initial cytoplasmic mask and the initial nuclear mask to generate the initial cell region, and trim the region where the cell nucleus extends beyond the cytoplasm; calculate the morphological parameters of each cell region and remove the cell regions whose morphological parameters exceed the preset range, and generate the segmentation result containing the cytoplasmic mask, the nuclear mask and the complete cell region, as well as the morphological parameters and spatial distribution information.

[0037] Specifically, a logical OR operation is performed on the cytoplasmic mask and the nuclear mask to generate the initial cell region. Regions of the cell nucleus extending beyond the cytoplasm are trimmed to ensure spatial consistency between the nucleus and cytoplasm. Morphological filtering is then performed, calculating morphological parameters such as aspect ratio and nucleocytoplasmic ratio for each cell region, and removing abnormal regions with aspect ratios exceeding 1.5–4.0 or nucleocytoplasmic ratios exceeding 0.3–0.5. Finally, the results are output, generating a segmentation result that includes the cytoplasmic mask, the nuclear mask, and the complete cell region, outputting the morphological parameters and spatial distribution information of each cell.

[0038] In this embodiment, step S3 can also employ dynamic feedback to automatically handle complex situations such as noise interference and uneven illumination by dynamically adjusting the segmentation threshold based on image quality, thereby achieving high-precision cytoplasm / nucleus segmentation, as shown in the attached figure. Figure 2 As shown, it specifically includes the following contents.

[0039] Step S301: Calculate the morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask. The morphological evaluation indicators include compactness, roundness and nucleocytoplasmic ratio. After calculating the morphological evaluation indicators using a preset quality scoring function, obtain the corresponding quality score.

[0040] Specifically, the cytoplasmic mask and nuclear mask after morphological optimization and spatial constraint verification are obtained, along with their geometric feature parameters, including area A, perimeter P, and centroid coordinates (x, y).

[0041] Computational morphological evaluation metrics: Compactness The closer the value is to 1, the more regular the shape; roundness: The closer the value is to 1, the closer the shape is to a circle; nucleocytoplasmic ratio: , where An is the area of ​​the nuclear mask and Ac is the area of ​​the cytoplasmic mask.

[0042] Calculate the quality rating function and calculate the quality rating Q, where Q=α·R+β·(1-C)+γ·I, (N / C\in [0.2,0.8]) α, β, and γ are set constants, and I is a preset indicator function, which can be set to 0.4, 0.3, and 0.3. Finally, the quality score Q and morphological indicator anomaly alerts are output.

[0043] Step S302: If the quality score is lower than the pre-review score or the morphological index exceeds the preset range multiple times, the threshold adjustment is initiated. If the roundness is lower than the set value, the cytoplasmic threshold is finely adjusted to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nuclear threshold is reduced to shrink the nuclear segmentation range.

[0044] Specifically, if the quality score is lower than the pre-screening score or the morphological indicators exceed the preset range multiple times consecutively, threshold adjustment will be initiated. The segmentation threshold can be adjusted based on the type of abnormality: if the roundness is lower than the set value, the cytoplasmic threshold will be fine-tuned to prioritize preserving areas with clear edges; this set value could be 0.6. If the nucleocytoplasmic ratio is greater than the set value, the nuclear threshold will be lowered to shrink the nuclear segmentation range; this set value could be 0.8.

[0045] Step S303: Re-execute the initial segmentation, morphological optimization and spatial constraint verification using the adjusted threshold to generate the corrected cytoplasmic mask and nuclear mask.

[0046] Furthermore, multi-scale consistency verification and fusion steps can be added to further improve the accuracy of the final mask. Specifically: Step S304: Obtain multiple segmentation masks at different resolution scales and generate corresponding quality scores for each scale. For example, obtain the segmentation mask Ms (s=0,1,2) at the original resolution (1*), downsampling (0.5*), and upsampling (2*) scales, and the quality scores Qs for each scale.

[0047] Calculate the cross-scale similarity coefficient and identify divergence regions based on the cross-scale similarity coefficient. Cross-scale similarity coefficient D ij , (i,j=0,1,2), D can be... ij A value less than 0.7 is used as the threshold for identifying divergent regions.

[0048] Dynamic weight allocation is performed, assigning the highest quality score as the first weight value, and distributing the remaining weights equally among the other scales. Weighted fusion is then performed based on gradient features to generate the final multi-scale fused mask M. f , ,in This represents the gradient magnitude at the mask edge.

[0049] In this embodiment, the above segmentation process is repeated at different scales, such as the original image and the downsampled image, to verify the consistency of the results. If the differences between the multi-scale results are significant, the results of each scale are fused. Specifically, this may include the following:

[0050] Step S3041: Construct a multi-scale image group to generate an image group with three scales: original resolution, downsampled resolution, and upsampled resolution. Enhance the contrast of the downsampled low-resolution image to highlight the global structure. Perform edge-preserving filtering on the upsampled high-resolution image to reduce artifacts. Use the original resolution image to preserve details for verification.

[0051] Specifically, a three-scale image pyramid is generated, comprising the original resolution, downsampling (which can be done at 0.5x resolution), and upsampling (which can be done at 2x resolution). The 2x resolution pyramid can be generated using current super-resolution algorithms such as ESRGAN. If the original image resolution is lower than a preset minimum (e.g., 200*200 pixels), the downsampling scale is skipped, and the 1x and 2x upsampling scales are generated directly to avoid information loss. For downsampled images, histogram equalization is used to enhance contrast and highlight global structure; for upsampled images, bilateral filtering is used to preserve edges and reduce artifacts introduced by super-resolution; and for images sampled at the original resolution, original features are preserved for detail verification.

[0052] Step S3042: For images of different resolutions, different preset image processing strategies are adopted. For low-resolution images, a first-scale segmentation mask set is generated using a pre-trained lightweight model. For original-resolution images, a second-scale segmentation mask set is generated using a pre-trained deep learning model. For high-resolution images, a third-scale segmentation mask set is generated by refining cell boundaries using a boundary enhancement strategy.

[0053] Specifically, for low-resolution image segmentation: a lightweight U-Net model, such as MobileNetU-Net, is used to quickly obtain a global cell distribution mask and filter out background noise regions. If the cell area ratio in the segmentation result exceeds 30% of the image, it is considered a dense scene and is marked as a high-density region, triggering an oversegmentation prevention mechanism at subsequent scales.

[0054] Segmentation of the original resolution scale image: The segmentation method of Otsu threshold and morphological operation in steps S1 to S3 above can be combined with deep learning models such as U-Net to generate a fine mask with multimodal fusion, which focuses on preserving cell boundary details.

[0055] For high-resolution image segmentation: Canny edge detection can be fused with pseudo-edge maps generated by GAN to refine cell nuclear membranes and cytoplasmic boundaries at the sub-pixel level, generating high-precision edge masks. If severe artifacts exist in the upsampled image, the result at that scale is discarded, and only the original resolution and low-resolution results are retained.

[0056] Step S3043: Perform cross-scale feature alignment and fusion, unify the resolution of segmentation masks at each scale, calculate cross-scale similarity coefficients to identify divergent regions, generate dynamic weights based on image quality assessment results, perform weighted fusion of features at each scale, and generate fused segmentation results including cytoplasmic masks, nuclear masks, and complete cell regions.

[0057] Specifically, a unified feature space mapping is first performed to adjust the segmentation results at each scale to the original resolution. Through learnable linear transformation operations, cell features at different scales, such as area, centroid coordinates, and edge gradients, are mapped to a unified feature space. A linear modified activation function is then used to normalize the transformed features to ensure the dimensionality consistency of features across scales.

[0058] Divergence region detection is performed, and the similarity coefficient of segmentation results at each scale is calculated. If the similarity is lower than a preset threshold, it is determined to be a cross-scale divergence region. For the divergence region, edge gradient features at multiple resolutions are extracted, and the weights of features at different scales are dynamically allocated through an attention mechanism. Higher weights can be given to high-resolution edge detail features, followed by low-resolution global structural features, to enhance the feature complementarity of the divergence region.

[0059] A dynamic weight fusion strategy is implemented to construct a multi-dimensional evaluation system that includes image quality parameters such as structural similarity and segmentation intersection-union ratio. Dynamic fusion weights are generated based on the quality scores of images at each scale. Scales with higher quality scores occupy a larger weight proportion in the final segmentation result. Multi-scale segmentation masks are integrated through a weighted fusion algorithm to achieve an optimized combination of features at different resolutions.

[0060] In this embodiment, the segmentation results at each scale can also be verified based on prior biological knowledge such as the spatial relationship between the cell nucleus and cytoplasm and morphological parameters. For conflicting regions that violate the rules, the positions can be corrected and filtered by referring to valid results at other scales or global structural masks to ensure that the segmentation results conform to biological logic.

[0061] Specifically, a spatial constraint check is performed on the segmentation results at each scale, requiring the cell nucleus centroid to be located within the cytoplasmic mask. If a conflict occurs at a certain scale where the cell nucleus centroid deviates from the cytoplasmic region, the segmentation results from other scales that conform to the rules are prioritized. For conflicting regions, based on the global cell distribution structure extracted at the low-resolution scale, the cell nucleus centroid is projected onto the nearest valid cytoplasmic region to complete the position correction. Combined with morphological rule-based joint filtering, preset unified cell morphological standards across scales are used, such as a cell nucleus area between 100 and 500 pixels and a nucleocytoplasmic ratio between 0.2 and 0.8, to jointly screen the segmentation results at each scale. If the cell morphological parameters at a single scale exceed the preset range but other scales conform to the standards, the multi-scale fusion result is retained; if all scales violate the morphological rules, the corresponding region is marked as a suspicious region, triggering a manual review process. Temporal continuity constraints are applied: When processing time-series images, the trajectory continuity of the current frame's multi-scale segmentation result is checked against the previous frame, requiring that the displacement of the cell centroid does not exceed a set multiple of the cell diameter, such as 1.5 times; for areas with abnormal displacement, trajectory correction is performed by referring to the multi-scale segmentation results of adjacent frames; for areas where trajectory association cannot be established for 3 or more consecutive frames, the process is switched to single-frame independent segmentation mode, relying on the edge features of the high-resolution image to complete the segmentation.

[0062] Furthermore, it can monitor the processing efficiency of each scale in real time, automatically shut down scales that exceed the time limit and replace them with historical parameters, save intermediate results of multi-scale segmentation, backtrack the previous stable state parameters when the current frame result is abnormal, and generate reports containing key information for continuous abnormal scales to prompt manual inspection, thus ensuring system stability.

[0063] Specifically, the system monitors the processing time and resource consumption of images at each scale in real time. If the processing time at a certain scale, such as a 2x upsampling scale, exceeds a preset threshold, the processing flow at that scale is automatically terminated, and the optimal parameters obtained from historical training are used as replacements to ensure system processing efficiency. The system stores intermediate data from the last 10 multi-scale segmentation operations, including segmentation masks at each scale, quality scores, and biological rule verification logs. When an anomaly occurs in the fusion result of the current frame, the system backtracks to the stable state of the previous frame, copies its multi-scale fusion weight parameters, and downsamples the current frame image to the original resolution scale for reprocessing, avoiding global segmentation errors caused by a single scale failure.

[0064] By combining multi-dimensional scale analysis, intelligent fusion strategies, and biological rule constraints, the consistency and reliability of segmentation results are significantly improved, making it particularly suitable for complex experimental scenarios with unstable imaging quality and variable cell morphology.

[0065] In this embodiment, rules such as division event verification and proliferation rate constraints can be added to fit the stem cell growth process, and short-term noise can be filtered by combining the continuity of the time dimension to avoid single-frame misjudgment affecting the overall result; specifically including the following.

[0066] The cell trajectories are initialized based on the first frame image, creating an independent trajectory for each detected cell. Subsequent frames in the image set are analyzed and processed, and a matching relationship is established between the cells in the current frame and the cells in the previous frame based on spatial proximity, morphological similarity, and temporal continuity. After filtering out short-term noise trajectories based on survival time and proliferation rate, if multiple cells in the current frame still satisfy the condition that their total area is similar to that of the parent cell and their morphological characteristics conform to the genetic relationship, they are determined to be daughter cells produced by division.

[0067] In the cell fusion detection scenario of this embodiment, trajectory refers to the dynamic change path or feature evolution process of an individual cell or cell group in the spatiotemporal dimension. Examples include: spatial trajectory: the movement path of a cell in a microscopic field (e.g., migration, chemotaxis); temporal trajectory: the sequence of changes in cell morphological characteristics (e.g., area, fluorescence intensity, nucleocytoplasmic ratio) over time; and functional trajectory: the temporal representation of biological behaviors such as cell proliferation, differentiation, and fusion (e.g., the increasing curve of fusion degree over time). Trajectory initialization is the process of establishing initial tracking identifiers or state parameters for each individual cell in the spatiotemporal sequence analysis. This can include: individual identifier association, assigning a unique identifier to each segmented cell, such as the nucleus / cytoplasm region, in the first frame of the time series image to establish the basis for tracking in subsequent frames; and initial state parameter definition, recording the initial spatial position, morphological characteristics, fluorescence intensity value, etc., of the cell in the first frame as the starting benchmark for trajectory evolution.

[0068] Specifically, this step may include the following:

[0069] Step S401: Perform cell feature analysis on the first frame image of the time series, assign a unique identifier to each segmented cell in the image, and record the initial trajectory feature information of the cells in the first frame image. The initial trajectory feature information includes, but is not limited to, initial spatial location, morphological features, and fluorescence intensity value information.

[0070] Step S402 involves analyzing cell features in each subsequent frame image. If the centroid distance between a cell in the current frame and a cell in the previous frame is less than a preset threshold and the area change rate is within the pre-screening range, then a matching relationship is established between the cells in the current frame and the cells in the previous frame. If a single cell in the previous frame matches multiple cells in the current frame, and the total area of ​​the daughter cells and the area of ​​the mother cell have an error less than a set value and similar morphological features, then it is determined to be a division event, and the corresponding trajectory feature information is updated. Specifically, this may include the following:

[0071] When performing cell feature analysis on subsequent frames, the division events initially identified in the previous frame are verified. It is determined whether the list of unique identifiers for the detected child cells is the same as that in the previous frame, and whether the area growth trend and positional offset of the corresponding child cells meet the preset growth conditions. If the list of unique identifiers for the child cells is the same in two adjacent frames, and the area growth trend and positional offset of the corresponding child cells meet the preset growth conditions, then the division event is considered valid. Otherwise, the division event is marked as segmentation noise and removed from the division event record.

[0072] Step S403: For each cell trajectory, calculate indicators such as survival time, number of divisions, and net proliferation rate. The survival time is the time difference between the current frame and the initial frame; the number of divisions is the number of division events in the statistical trajectory; the net proliferation rate is calculated based on cell area change, survival time, and frame interval. Short-term noise trajectories with survival times less than a set number of frames are filtered out, and abnormal proliferation trajectories with net proliferation rates exceeding a set multiple of the average proliferation rate are marked. Specifically, for cell area reduction caused by apoptosis, a negative proliferation rate is allowed, and cell morphological characteristics such as nuclear condensation and cytoplasmic fragmentation are used to determine whether it is normal apoptosis, avoiding misjudgment as a detection error.

[0073] In this embodiment, the single-frame fusion degree can also be corrected by combining trajectory information. Long-standing trajectories, such as those with Tlive>12 frames, are given higher weights. For newly emerging trajectories, such as daughter cells generated by cell division, their morphological stability for three consecutive frames must be verified before being included in the fusion degree calculation. Abnormal trajectory areas after filtering are excluded. Finally, the time-fusion degree curve is output, along with the time points of the division events, the distribution of effective cell numbers and proliferation rates in each frame, and abnormal frame markers such as pollution warnings and detection confidence levels greater than a set value.

[0074] The above steps, through rules such as division event verification and proliferation rate constraints, conform to the growth dynamics of stem cells, and filter short-term noise by combining the continuity of the time dimension to avoid single-frame misjudgment affecting the overall results; they realize the dynamic tracking of cell proliferation, providing a reliable guarantee of fusion degree detection in the time dimension, and are especially suitable for stem cell production scenarios that require monitoring of cell growth dynamics.

[0075] This embodiment discloses a multimodal cell experiment image processing method. It acquires a single-channel grayscale image or a multi-channel fluorescence image containing cytoplasmic and nuclear features, separates the cytoplasmic and nuclear channels, applies Gaussian filtering to the cytoplasmic channel, and applies mean filtering to the nuclear channel. The preprocessed cytoplasmic channel image is then segmented using the maximum inter-class variance threshold method to generate candidate cytoplasmic regions. Small noise regions are removed using an opening operation, and broken regions are connected using a closing operation to generate an initial cytoplasmic mask. The preprocessed nuclear channel image is then segmented using an adaptive threshold method, combined with distance transform and local maximum detection to generate nuclear seed points. Overlapping nuclear regions are separated using a watershed algorithm to generate an initial nuclear mask. The initial cytoplasmic mask and the initial nuclear mask are then logically ORed to generate initial cell regions, and regions where the nucleus extends beyond the cytoplasm are cropped. Finally, morphological parameters of each cell region are calculated, and cell regions with morphological parameters exceeding a preset range are removed. This process generates a segmentation result containing the cytoplasmic mask, nuclear mask, and complete cell regions, along with morphological parameters and spatial distribution information. This invention addresses the technical problems of insufficient targeting of cytoplasmic and nuclear noise processing, low segmentation accuracy, difficulty in separating overlapping nuclear regions, and poor reliability of morphological parameter analysis in existing technologies. By applying Gaussian filtering and median filtering to suppress different noises in the cytoplasmic and nuclear channels respectively, and combining the maximum inter-class variance method, adaptive threshold segmentation, and watershed algorithm, it achieves accurate segmentation of cytoplasm and nucleus and separation of overlapping nuclei. Furthermore, by verifying morphological parameters and eliminating abnormal regions through spatial constraints, it can generate high-precision segmentation results that include cytoplasmic masks, nuclear masks, and complete cell regions, accurately obtaining cell morphological parameters and spatial distribution information.

[0076] In another embodiment, a multimodal cell experiment image processing system is also disclosed, as shown in the attached figure. Figure 3As shown, the system includes: an image acquisition module 1, used to acquire a single-channel grayscale image or a multi-channel fluorescence image containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and mean filtering to the nuclear channel; a segmentation module 2, used to apply maximum inter-class variance threshold segmentation to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions, remove small noise regions through opening operations, connect broken regions through closing operations, and generate an initial cytoplasmic mask; apply adaptive threshold segmentation to the preprocessed nuclear channel image, combine distance transformation and local maximum point detection to generate nuclear seed points, separate overlapping nuclear regions through a watershed algorithm, and generate an initial nuclear mask; a generation module 3, used to perform a logical OR operation between the initial cytoplasmic mask and the initial nuclear mask to generate an initial cell region, and crop the region where the cell nucleus extends beyond the cytoplasm; calculate the morphological parameters of each cell region and remove cell regions whose morphological parameters exceed a preset range, and generate a segmentation result containing the cytoplasmic mask, the nuclear mask, and the complete cell region, as well as morphological parameters and spatial distribution information.

[0077] In this embodiment, the segmentation module is further configured to divide the image into multiple overlapping or non-overlapping sub-blocks when the standard deviation of the image grayscale value is detected to be greater than a first preset threshold or the grayscale difference between the cell region and the background region in the image is less than a second preset threshold. For each sub-block, a local threshold T(x,y) is calculated: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast. If the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be background. When the average gradient magnitude of the image is detected to be less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

[0078] In this embodiment, the segmentation module is further configured to: segment the image into multiple sub-blocks, calculate a local threshold T(x,y) for each pixel within the sub-block, T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average of the gray values ​​of all pixels within the sub-block, σ is the local standard deviation of the gray values ​​of the pixels within the sub-block, and C is a preset constant adjusted according to the image contrast; if the gray value of a pixel in the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be background; and an adjusted binary image is generated based on the determination results of all sub-blocks. The Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width. The adjusted binary image and the clear cell edge image are logically ORed. If the corresponding pixel in the binary image or the edge image is the foreground, the fusion result is the foreground; otherwise, it is retained as the background. A circular structuring element of a set size is used to perform a dilation and erosion process to connect the broken cell edges. After filling the small hole areas in the image, a complete cell segmentation mask is generated.

[0079] In this embodiment, the generation module is also used to calculate the morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask. The morphological evaluation indicators include compactness, roundness, and nucleocytoplasmic ratio. After calculating the morphological evaluation indicators using a preset quality scoring function, the corresponding quality score is obtained. If the quality score is lower than the pre-review score or the morphological indicators exceed the preset range multiple times consecutively, a threshold adjustment is initiated. If the roundness is lower than the set value, the cytoplasmic threshold is fine-tuned to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nuclear threshold is reduced to shrink the nuclear segmentation range. The initial segmentation, morphological optimization, and spatial constraint verification are re-executed using the adjusted thresholds to generate the corrected cytoplasmic mask and nuclear mask.

[0080] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the multimodal cell experiment image processing system disclosed in the embodiments, since it corresponds to the multimodal cell experiment image processing method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to the method section.

[0081] In other embodiments, a server is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal cell experiment image processing method described in the above embodiments.

[0082] If the aforementioned multimodal cell experiment image processing system is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various embodiments of the multimodal cell experiment image processing method described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0084] In summary, the above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be covered by the present invention.

Claims

1. A multimodal cell experiment image processing method, characterized in that, Includes the following steps: S1, acquire a single-channel grayscale image or multi-channel fluorescence image containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and medium filtering to the nuclear channel; S2, apply the maximum inter-class variance threshold segmentation method to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions, remove small noise regions through opening operation, and connect broken regions through closing operation to generate the initial cytoplasmic mask. Adaptive threshold segmentation is applied to the preprocessed cell nucleus channel image. Cell nucleus seed points are generated by combining distance transformation and local maximum point detection. Overlapping nuclear regions are separated by the watershed algorithm to generate an initial cell nucleus mask. S3: Perform a logical OR operation on the initial cytoplasmic mask and the initial nuclear mask to generate the initial cell region, and trim the region where the nucleus extends beyond the cytoplasm; calculate the morphological parameters of each cell region and remove the cell regions whose morphological parameters exceed the preset range, and generate the segmentation result containing the cytoplasmic mask, the nuclear mask and the complete cell region, as well as the morphological parameters and spatial distribution information.

2. The cell experiment image processing method based on multimodality according to claim 1, characterized in that, Step S2 further includes: If the standard deviation of the image grayscale value is detected to be greater than the first preset threshold or the grayscale difference between the cell region and the background region in the image is less than the second preset threshold, the image is divided into multiple overlapping or non-overlapping sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels, where T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast. If the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background. If the average gradient magnitude of the detected image is less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

3. The cell experiment image processing method based on multimodality according to claim 1, characterized in that, Step S2 further includes: The image is divided into multiple sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average of the gray values ​​of all pixels in the sub-block, σ is the local standard deviation of the gray values ​​of the pixels in the sub-block, and C is a preset constant adjusted according to the image contrast. If the gray value of a pixel in a sub-block is greater than the local threshold, the sub-block is identified as a cell region; otherwise, it is identified as background. An adjusted binary image is generated based on the identification results of all sub-blocks. The Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width. The adjusted binary image and the clear cell edge image are logically ORed. If the corresponding pixel in the binary image or the edge image is the foreground, the fusion result is the foreground; otherwise, it is retained as the background. A circular structuring element of a set size is used to perform a dilation and erosion process to connect the broken cell edges. After filling the small hole areas in the image, a complete cell segmentation mask is generated.

4. The cell experiment image processing method based on multimodality according to claim 3, characterized in that, Step S3 further includes: The morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask are calculated. The morphological evaluation indicators include compactness, roundness and nucleoplasmic ratio. The corresponding quality scores are obtained by calculating the morphological evaluation indicators using a preset quality scoring function. If the quality score is lower than the pre-screening score or the morphological indicators exceed the preset range multiple times, the threshold adjustment will be initiated. If the roundness is lower than the set value, the cytoplasmic threshold will be finely adjusted to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nucleus threshold will be reduced to shrink the nuclear segmentation range. The initial segmentation, morphological optimization, and spatial constraint verification are re-executed using the adjusted thresholds to generate corrected cytoplasmic and nuclear masks.

5. A cell experiment image processing system based on multimodal processing, characterized in that, include: The image acquisition module is used to acquire single-channel grayscale images or multi-channel fluorescence images containing cytoplasmic and nuclear features, separate the cytoplasmic channel and the nuclear channel, apply Gaussian filtering to the cytoplasmic channel and medium filtering to the nuclear channel. The segmentation module is used to apply the maximum inter-class variance threshold segmentation method to the preprocessed cytoplasmic channel image to generate cytoplasmic candidate regions, remove small noise regions through opening operations, and connect broken regions through closing operations to generate an initial cytoplasmic mask. Adaptive threshold segmentation is applied to the preprocessed cell nucleus channel image. Cell nucleus seed points are generated by combining distance transformation and local maximum point detection. Overlapping nuclear regions are separated by the watershed algorithm to generate an initial cell nucleus mask. The generation module is used to perform a logical OR operation between the initial cytoplasmic mask and the initial nuclear mask to generate the initial cell region and trim the region of the cell nucleus that extends beyond the cytoplasm. After calculating the morphological parameters of each cell region and removing cell regions whose morphological parameters exceed the preset range, the segmentation results containing the cytoplasmic mask, nuclear mask and complete cell regions are generated, along with morphological parameters and spatial distribution information.

6. The cell experiment image processing system based on multimodality according to claim 5, characterized in that, The segmentation module is further configured to divide the image into multiple overlapping or non-overlapping sub-blocks when the standard deviation of the image grayscale value is detected to be greater than a first preset threshold or the grayscale difference between the cell region and the background region in the image is less than a second preset threshold. For each sub-block, a local threshold T(x,y) is calculated: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average grayscale value of all pixels in the sub-block, σ is the local standard deviation of the pixel grayscale value in the sub-block, and C is a preset constant adjusted according to the image contrast. If the pixel grayscale value of the sub-block is greater than the local threshold, the sub-block image is determined to be a cell region; otherwise, it is determined to be the background. When the average gradient magnitude of the image is detected to be less than a third preset threshold, the Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; hysteresis thresholding is performed: if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when it is connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width.

7. The cell experiment image processing system based on multimodality according to claim 6, characterized in that, The segmentation module is also configured to: This method is used to segment an image into multiple sub-blocks. For each sub-block, a local threshold T(x,y) is calculated for the pixels: T(x,y) = μ(x,y) – C*σ(x,y), where μ is the local average of the gray values ​​of all pixels in the sub-block, σ is the local standard deviation of the gray values ​​of the pixels in the sub-block, and C is a preset constant adjusted according to the image contrast. If the gray value of a pixel in a sub-block is greater than the local threshold, the sub-block is identified as a cell region; otherwise, it is identified as background. An adjusted binary image is generated based on the identification results of all sub-blocks. The Sobel gradient operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy, respectively, to obtain the gradient magnitude G and the gradient direction θ, where θ = arctan(Gy / Gx). For each pixel, only the local maximum value along the gradient direction is retained; if the pixel gradient magnitude is greater than the set higher threshold, it is retained as a cell contour edge; if the pixel gradient magnitude is lower than the set lower threshold, it is suppressed as a non-contour edge; if the pixel gradient magnitude is between the lower and higher thresholds, it is retained as a cell contour edge only when connected to a strong edge, otherwise it is suppressed, generating a clear cell edge image with a single pixel width; the adjusted binary image and the clear cell edge image are logically ORed, and if the corresponding pixel in the binary image or edge image is the foreground, the fusion result is the foreground, otherwise it is retained as the background; a circular structuring element of a set size is used to perform dilation and erosion processing to connect the broken cell edges, and after filling the small hole areas in the image, a complete cell segmentation mask is generated.

8. The cell experiment image processing system based on multimodality according to claim 7, characterized in that, The generation module is also used to calculate the morphological evaluation indicators of the generated initial cytoplasmic mask and initial nuclear mask. The morphological evaluation indicators include compactness, roundness, and nucleocytoplasmic ratio. After calculating the morphological evaluation indicators using a preset quality scoring function, the corresponding quality score is obtained. If the quality score is lower than the pre-review score or the morphological indicators exceed the preset range multiple times consecutively, the threshold adjustment is initiated. If the roundness is lower than the set value, the cytoplasmic threshold is fine-tuned to prioritize the preservation of areas with clear edges. If the nucleocytoplasmic ratio is greater than the set value, the nuclear threshold is reduced to shrink the nuclear segmentation range. The initial segmentation, morphological optimization, and spatial constraint verification are re-executed using the adjusted thresholds to generate the corrected cytoplasmic mask and nuclear mask.

9. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-4.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-4.

Images