Intelligent tumor margin identification method based on hcr amplified fluorescence image

Abstract

The present application relates to the technical field of image processing, in particular to a tumor margin intelligent recognition method based on HCR amplified fluorescence image, which aims at the problem that signal diffusion caused by defocus and margin recognition difficulty in HCR amplified fluorescence image, through evaluating the defocus degree of the image, calculating the fitting degree of the signal spot diffusion radius and the theoretically estimated radius, and combining the neighborhood difference analysis to generate the signal density distribution map, and finally recognizing different types of margins based on the signal density size. The present application can distinguish low-quality signals caused by defocus from real tumor signals, significantly improving the accuracy and reliability of tumor margin recognition.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for intelligent tumor margin recognition based on HCR magnified fluorescence images. Background Technology

[0002] Due to its high sensitivity, fluorescence in situ hybridization (HCR) technology has begun to be applied to rapid scanning detection and fluorescence imaging of tumor markers. However, to meet timeliness requirements, existing all-slide scanners typically only perform single-layer planar imaging and cannot perform time-consuming multi-layer Z-axis scanning. Because high-magnification objectives have extremely shallow depth of field, minute undulations on the slide surface can cause non-uniformly distributed local defocusing phenomena in the image. In defocused areas, the light signal of the fluorescent probe undergoes optical diffusion, manifested as an increased spot radius and a decrease in peak intensity.

[0003] Existing technologies typically employ thresholding, edge extraction, or morphological operations to obtain tumor margins in fluorescence images. These methods have significant limitations when dealing with the aforementioned defocused scenarios: fluorescence images contain both high-confidence, high-quality probe signals from tumor markers and low-quality, low-confidence signals caused by noise, non-specific impurities, and severe defocusing or fragmentation. The true tumor margin signal becomes larger and darker due to defocusing, mixing with the low-quality signal and becoming indistinguishable, thus leading to inaccurate and unreliable margins obtained by existing technologies. Summary of the Invention

[0004] To address the issues of inaccurate and unreliable tumor margin acquisition results in defocused scenarios, this invention provides an intelligent tumor margin recognition method based on HCR-magnified fluorescence images.

[0005] The intelligent tumor margin recognition method based on HCR magnified fluorescence images of the present invention adopts the following technical solution:

[0006] One embodiment of the present invention provides a method for intelligent identification of tumor resection margins based on HCR magnified fluorescence images, the method comprising the following steps:

[0007] The defocus value of each pixel is evaluated based on the average intensity of the texture edge of each sub-block in the fluorescence image. The brightness maxima searched in the fluorescence image are recorded as candidate signal points. The measured spot diffusion radius of each candidate signal point is fitted based on the gray value within the local window of each candidate signal point. The average difference between the measured spot diffusion radius of each candidate signal point and its neighboring candidate signal points is recorded as the neighborhood difference of each candidate signal point. The spot diffusion radius is predicted based on the defocus value of each candidate signal point to obtain the theoretical estimated spot radius of each candidate signal point. The difference between the theoretical estimated spot radius and the measured spot diffusion radius is recorded as the degree of fit of each candidate signal point. The degree of fit and neighborhood difference of all candidate signal points are smoothed and diffused to obtain the signal density of all pixels. For the edges extracted based on the signal density of all pixels, different types of cutting edges are identified based on the signal density on the edges.

[0008] Preferably, the specific steps for evaluating the defocus value of each pixel based on the average intensity of the texture edges of each sub-block in the fluorescence image are as follows:

[0009] For each sub-block in the fluorescence image, the gradient magnitude of each pixel within each sub-block is obtained, and the average gradient magnitude of all pixels within each sub-block is used as the average intensity of the texture edge. The defocus value of the center pixel of each sub-block is determined based on the difference between the preset standard clear texture intensity and the average intensity of the texture edge. The defocus value of all pixels is interpolated based on the defocus values ​​of the center pixels of all sub-blocks.

[0010] Preferably, the step of smoothly diffusing the matching degree and neighborhood difference of all candidate signal points to obtain the signal density of all pixels includes the following specific steps:

[0011] An environmental prediction matching score is obtained based on the degree of fit of each candidate signal point, and the environmental prediction matching score is negatively correlated with the degree of fit. A neighborhood consistency score is obtained based on the neighborhood difference of each candidate signal point, and the neighborhood consistency score is negatively correlated with the neighborhood difference. The weighted sum of the environmental prediction matching score and the neighborhood consistency score is used as the comprehensive confidence score of each candidate signal point. Smoothing diffusion is performed based on the comprehensive confidence scores of all candidate signal points to obtain the signal density of all pixels.

[0012] Preferably, the specific steps for identifying different types of cutting edges based on the signal density magnitude of the edges extracted from the edges based on the signal density of all pixels are as follows:

[0013] The signal density of all pixels constitutes a signal density distribution map; the signal density distribution map is segmented into a binary image, morphological closing operation is performed on the binary image, and the outer edge contour of each connected component is extracted from the binary image after morphological closing operation.

[0014] Edge pixels that are continuously distributed on the outer edge contour line and have an average signal density greater than a preset quality prompt threshold are marked as confirmed edges, while edge pixels that are continuously distributed on the outer edge contour line and have an average signal density less than or equal to a preset quality prompt threshold are marked as questionable edges.

[0015] Preferably, the radius of the structuring element used in performing the morphological closing operation is positively correlated with the average measured spot diffusion radius of all candidate signal points.

[0016] Preferably, the specific steps for obtaining the signal density are as follows:

[0017] For any given pixel, the attenuation of each candidate signal point at that pixel is obtained using the Gaussian kernel function. The product of this attenuation and the overall confidence score of each candidate signal point is recorded as the smooth diffusion result of the overall confidence score of each candidate signal point at that pixel. The mean of the smooth diffusion values ​​of all candidate signal points at that pixel is recorded as the signal density of that pixel.

[0018] Preferably, the specific steps for obtaining the measured light spot diffusion radius are as follows:

[0019] The gray values ​​of all pixels within the local window in the fluorescence image are fitted to a Gaussian model, and the standard deviation of the Gaussian model is used as the measured light spot diffusion radius.

[0020] Preferably, the specific steps for obtaining the neighborhood candidate signal points are as follows:

[0021] Construct a Delaunay triangulation with all candidate signal points as vertices. For any candidate signal point in the triangulation, the candidate signal points directly connected to that candidate signal point are considered as the neighboring candidate signal points of that candidate signal point.

[0022] Preferably, the theoretically estimated spot radius is obtained by linearly transforming the defocus value of each candidate signal point.

[0023] Preferably, after obtaining the comprehensive confidence score of each candidate signal point, only candidate signal points with a comprehensive confidence score greater than a preset confidence threshold are retained.

[0024] The beneficial effects of the technical solution of the present invention are:

[0025] This invention first assesses the defocus to understand the overall image quality background. Then, it compares the measured spot diffusion radius of candidate signal points with the theoretically estimated spot radius based on defocus prediction to determine the degree of agreement, which macroscopically judges whether the signal conforms to optical laws. Simultaneously, it calculates the neighborhood difference in spot radius between candidate signal points and their neighbors to evaluate the signal's reliability from a local consistency perspective. Finally, it smooths and diffuses these two key indicators (degree of agreement and neighborhood difference) to generate the signal density distribution of the entire image. This process allows the algorithm to effectively filter out low-reliability signals caused by severe defocus, noise, or non-specific binding, focusing instead on high-quality signals that conform to the optical diffusion model and are consistent with surrounding signals. Therefore, the accuracy, anti-interference, and reliability of the edges extracted based on this signal density distribution are fundamentally improved, effectively avoiding the problems of misclassifying low-quality signals as edges or missing real but diffuse signals.

[0026] This invention not only identifies the location of the cutting edge but also intelligently distinguishes the quality of the cutting edge segments, specifically identifying different types of cutting edges based on the signal density at the edge. This means that the signal density generated by the algorithm directly reflects the quality of the original signal supporting the cutting edge segment. Cutting edges corresponding to high signal density regions are supported by high-confidence, high-quality signals and can be labeled as one type of cutting edge (denoted as a "sure-fire" cutting edge); cutting edges corresponding to low signal density regions are supported by low-quality signals and can be labeled as another type of cutting edge (denoted as a "questionable" cutting edge). This distinguishing ability is not available in traditional methods. It upgrades a simple "presence / absence" cutting edge judgment to a "sure-fire" / "questionable" quality assessment.

[0027] In summary, this invention utilizes the average difference between the measured spot diffusion radius of each candidate signal point and its neighboring candidate signal points (i.e., neighborhood difference), and the difference between the theoretically estimated spot radius and the measured spot diffusion radius (i.e., degree of agreement) to describe whether a candidate signal point is reliable or effective from two perspectives: the macroscopic texture distribution environment and the speckled signal distribution in the neighborhood. Furthermore, the signal density obtained based on the neighborhood difference and degree of agreement is used to indirectly identify the cutting edge. This avoids the problem of inaccurate and unreliable results when directly using methods such as threshold segmentation, edge extraction, or morphological operations to obtain the cutting edge, due to the mixture of low-quality signals (noise, non-specific impurities, and low-confidence signals caused by severe defocusing or fragmentation) and high-quality signals (high-confidence signals generated by tumor markers). Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the steps of a method for intelligent tumor margin recognition based on HCR magnified fluorescence images, as provided in an embodiment of the present invention. Detailed Implementation

[0030] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the intelligent tumor margin recognition method based on HCR magnified fluorescence images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0032] The specific scheme of the intelligent tumor margin recognition method based on HCR magnified fluorescence image provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0033] Please see Figure 1 The diagram illustrates a flowchart of a method for intelligent tumor margin recognition based on HCR-magnified fluorescence images according to an embodiment of the present invention. The method includes the following steps:

[0034] Step S101: Evaluate the defocus value of each pixel based on the average intensity of the texture edge of each sub-block in the fluorescence image.

[0035] In this embodiment, the fluorescence image is a single-channel grayscale image, and the average intensity of the texture edge of each sub-block in the fluorescence image represents the local texture intensity of the fluorescence image.

[0036] In this embodiment, single-layer planar imaging is performed when acquiring fluorescence images using an all-glass slide scanner. Due to the extremely shallow depth of field of the high-magnification objective lens, minute undulations on the slide surface can cause non-uniformly distributed local defocusing phenomena in the image. In the defocused area, the light signal of the fluorescent probe will undergo optical dispersion. The local texture intensity of the fluorescence image is related to the severity of the defocusing phenomenon. Therefore, this embodiment uses the local texture intensity to evaluate the defocusing degree value of each pixel and uses the defocusing degree value to describe the severity of the defocusing phenomenon. A smaller defocusing degree value indicates focus, and the defocusing phenomenon is not serious. A larger defocusing degree value indicates defocusing, and the defocusing phenomenon is more serious.

[0037] Step S102: The maximum brightness points searched in the fluorescence image are recorded as candidate signal points. The measured light spot diffusion radius of each candidate signal point is fitted according to the gray value in the local window of each candidate signal point. The average difference between the measured light spot diffusion radius of each candidate signal point and the neighboring candidate signal points is recorded as the neighborhood difference of each candidate signal point.

[0038] Candidate signal points refer to pixels in a fluorescence image that are distinct from a flat background and can represent speckled signals, which include probe signals generated by tumor markers in the fluorescence image.

[0039] The light signal of the fluorescent probe will undergo optical diffusion at the candidate signal point, which manifests as an increase in the spot radius and a decrease in peak intensity. Based on this, this embodiment fits the measured spot diffusion radius of each candidate signal point according to the gray value in the local window of each candidate signal point, which is used to describe the influence radius of the observed (i.e., the optical diffusion presented in the fluorescence image), that is, the observation radius of the spot generated by defocus.

[0040] Furthermore, this embodiment considers that the speckled signals of candidate signal points not only include high-confidence, high-quality probe signals generated by tumor markers in fluorescence images, but also low-quality, low-confidence signals caused by noise, non-specific impurities, and severe defocusing or fragmentation. This makes it impossible to directly obtain the cutting edge using methods such as threshold segmentation, edge extraction, or morphological operations; otherwise, it would be easy to miss the true cutting edge signal that has become larger and darker due to defocusing, thus leading to inaccurate and unreliable cutting edges.

[0041] In this step, the smaller the neighborhood difference of each candidate signal point, the smaller the difference in the measured spot diffusion radius between each candidate signal point and its neighboring candidate signal points. However, since real biological signals on the same focal plane usually have similar optical diffusion characteristics, a smaller difference value means that the candidate signal point has higher credibility. Based on this, this embodiment uses the neighborhood difference of each candidate signal point to describe whether the candidate signal point is credible or effective. Subsequently, the cutting edge is indirectly obtained based on the neighborhood difference, avoiding the problem of inaccurate results caused by the mixing of low-quality and high-quality signals when directly using methods such as threshold segmentation, edge extraction, or morphological operations to obtain the cutting edge.

[0042] Step S103: Predict the spot diffusion radius based on the defocus value of each candidate signal point to obtain the theoretical estimated spot radius of each candidate signal point. The difference between the theoretical estimated spot radius and the measured spot diffusion radius is recorded as the degree of agreement for each candidate signal point.

[0043] Since the defocus value and the spot diffusion radius have a specific correlation (e.g., a linear relationship), that is, the larger the defocus value, the larger the spot diffusion radius, this embodiment predicts the spot diffusion radius based on the defocus value to obtain the theoretical estimated spot radius of each candidate signal point. It represents the influence radius of the optical diffusion situation predicted under the defocus value described by the local texture edge intensity in the fluorescence image, that is, the predicted radius of the spot generated by defocus.

[0044] Low-quality candidate signal points significantly affect the aforementioned correlations. When candidate signal points have high reliability, the predicted theoretical spot radius differs little from the measured spot diffusion radius, meaning the degree of agreement between each candidate signal point is low. Therefore, in this embodiment, the degree of agreement describes the reliability or effectiveness of candidate signal points from a macroscopic texture environment perspective.

[0045] Step S104: Smoothly diffuse the matching degree and neighborhood difference of all candidate signal points to obtain the signal density of all pixels. For the edges extracted based on the signal density of all pixels, identify different types of cutting edges based on the signal density on the edges.

[0046] The degree of fit and neighborhood difference of candidate signal points are used to describe whether candidate signal points are reliable or effective from two perspectives: the macroscopic texture distribution environment and the speckled signal distribution in the neighborhood.

[0047] The degree of agreement and neighborhood difference of all candidate signal points accurately describes the distribution of the real and effective signal (i.e. the signal that helps to extract the cutting edge). The diffusion described in this embodiment aims to obtain the signal density of all pixels including all candidate signal points. High signal density indicates that the signal is not only dense but also has reliable optical quality; low density indicates that the signal is sparse or the optical quality is questionable.

[0048] This embodiment identifies the cutting edge based on the signal density at the edge, enabling the obtained cutting edge to distinguish the optical quality of the signal and ensuring the accuracy and reliability of the cutting edge.

[0049] In summary, this embodiment utilizes the average difference between the measured spot diffusion radius of each candidate signal point and its neighboring candidate signal points (i.e., neighborhood difference), and the difference between the theoretically estimated spot radius and the measured spot diffusion radius (i.e., degree of agreement) to describe whether the candidate signal points are credible or effective from two perspectives: the macroscopic texture distribution environment and the speckled signal distribution in the neighborhood. Furthermore, the signal density obtained based on the neighborhood difference and degree of agreement is used to identify the cutting edge, so that the identified cutting edge can intuitively distinguish the cutting edge type supported by high confidence signals (denoted as a sure cutting edge) and the cutting edge type supported by low quality signals (denoted as a questionable cutting edge). This avoids the problem of inaccurate and unreliable results when directly using methods such as threshold segmentation, edge extraction, or morphological operations to obtain cutting edges, due to the mixture of low-quality signals (noise, non-specific impurities, and low confidence signals caused by severe defocusing or fragmentation) and high-quality signals (high confidence signals generated by tumor markers).

[0050] As a preferred example, the defocus value of each pixel is evaluated based on the average intensity of the texture edges of each sub-block in the fluorescence image, including the following methods:

[0051] For each sub-block in the fluorescence image, the gradient magnitude of each pixel within each sub-block is obtained using the Sobel operator, and the average gradient magnitude of all pixels within each sub-block is used as the average intensity of the texture edge.

[0052] A standard clear texture intensity is preset. This standard clear texture intensity refers to the average intensity of the texture edge obtained by photographing a standard clear tissue section. The fluorescence image obtained under the standard clear tissue section does not have defocus phenomenon.

[0053] The defocus value of the center pixel of each sub-block is determined based on the difference between the standard sharp texture intensity and the average intensity of the texture edge. The defocus values ​​of all pixels are then interpolated based on the defocus values ​​of the center pixels of all sub-blocks. This embodiment uses the bicubic interpolation algorithm for interpolation, which is well-known and will not be described in detail here.

[0054] As an example, the standard method for setting sharp texture intensity includes:

[0055] Prepare a standard reference slide, typically containing tissue with clearly defined, densely packed, and uniformly distributed cell structures. Place the standard reference slide on the microscope stage. Using a high-power objective lens (e.g., 40x or 60x), select an area in the field of view with clear cell boundaries and bright fluorescence. Manually and slowly adjust the Z-axis (focus adjustment knob) while observing the real-time image until the image contrast and sharpness reach visual optimality. At this point, the edges of structures such as cell membranes and nuclei are sharpest. Acquire and save this image at the optimal focal plane. Using the method described above, obtain the average intensity of the texture edges of each sub-block in the image. The maximum value of the average texture edge intensity of all sub-blocks in the image is taken as the standard sharp texture intensity.

[0056] In addition, the Z-axis was manually adjusted slowly and precisely while observing the real-time image. A single image was manually selected for acquisition, reaching its most visually blurred state. The average texture edge intensity of each sub-block in this image was obtained using the method described above, and the maximum value of the average texture edge intensity of all sub-blocks in the image was taken as the standard blurred texture intensity.

[0057] As an example, the method for obtaining each sub-block in a fluorescence image is to divide the fluorescence image into equal parts, with each sub-block being 65×65 in size.

[0058] In special cases, when the fluorescence image cannot be divided equally, resulting in some sub-blocks being smaller than 65×65, these sub-blocks are still retained.

[0059] As an example, the defocus value of the center pixel of each sub-block is determined based on the difference between the standard sharp texture intensity and the average intensity of the texture edge, including the following formula:

[0060] in This represents the defocus value of the center pixel of each sub-block. Indicates the standard clear texture intensity. This represents the average intensity of the texture edges for each sub-block. Indicates the standard blur texture intensity.

[0061] in The denominator is used for normalization and dimensionless removal. In this embodiment... Greater than (That is, the average intensity of the texture edges of a clear texture is greater than the average intensity of the texture edges of a blurry texture), if Less than or equal to ,illustrate , The settings are incorrect and need to be reset. In some embodiments, the settings can also be... Set to 0.

[0062] Special, Greater than At that time, As The value of , at this time =0.

[0063] In the above formula, when near When the defocus value approaches 0, it indicates that the texture is sharp (or in focus); when near When the result approaches 1, it indicates sparse texture (or out of focus).

[0064] As a preferred example, the brightness maxima points found in the fluorescence image are recorded as candidate signal points, including the following methods:

[0065] The fluorescence image is processed using the Difference of Gaussian (DoG) algorithm to obtain a difference map. This algorithm uses two Gaussian kernels of different scales to convolve and subtract the images to obtain the difference map, which can effectively enhance speckled signals and suppress flat backgrounds. This algorithm is a well-known technique and will not be described in detail in this embodiment. As an example, the scales of the two Gaussian kernels are 3×3 and 7×7, respectively.

[0066] In one optional example, the maxima in the difference plot are extracted as candidate signal points.

[0067] In a preferred example, a lower brightness threshold is set. Only gray values ​​greater than 1 in the difference image are retained. The extreme points are used as candidate signal points. This preferred example excludes candidate signal points with weak signal strength, which reduces the amount of subsequent calculations by reducing the number of candidate signal points, and avoids interference from weak noise signals.

[0068] As an example, The setting method is as follows: Calculate the mean grayscale value of all pixels in the difference image except for the maximum points (pixels with a grayscale value of 0 are not included in the mean calculation), and use 0.7 times this mean as the mean. .

[0069] As a preferred example, the measured spot diffusion radius of each candidate signal point is fitted based on the grayscale value within a local window of each candidate signal point. The method includes:

[0070] For a local window centered on any candidate signal point, the coordinates of each pixel within the local window are used as the independent variable, and the gray value of each pixel in the fluorescence image is used as the dependent variable. The independent and dependent variables of all pixels within the local window are fitted into a Gaussian model, and the standard deviation of the Gaussian model is used as the measured light spot diffusion radius.

[0071] In this example, the local window size is 13×13.

[0072] As an example, fitting a Gaussian model to the independent and dependent variables of all pixels within a local window includes the following methods:

[0073] The independent variable (i.e., relative coordinate) of the center pixel within the local window is denoted as (0, 0). The difference between the pixel coordinates of all pixels within the local window and the pixel coordinates of the center pixel is used as the independent variable (i.e., relative coordinate) of all pixels.

[0074] Construct the following Gaussian model: ,in This represents the independent variable (i.e., the relative coordinate) corresponding to any pixel. Represent the independent variable The corresponding dependent variable (i.e., the gray value of the pixel). This represents the peak value of the Gaussian model. Represents the baseline of the Gaussian model. This represents the standard deviation of the Gaussian model. , , These are the parameters to be fitted to the Gaussian model, and exp() represents an exponential function with the natural constant as the base.

[0075] Based on the independent and dependent variables of all pixels within a local window, the Levenberg-Marquardt algorithm is used to fit the Gaussian model, minimizing the error between the Gaussian model's output and the grayscale values ​​of all pixels within the local window. The fitted result... As the measured light spot diffusion radius.

[0076] As a preferred example, the method for obtaining the neighboring candidate signal points of each candidate signal point is as follows:

[0077] A Delaunay triangulation is constructed using all candidate signal points as vertices. This mesh has the property of being an empty circle, enabling it to adaptively connect spatially adjacent natural neighbors without requiring manually set distance thresholds.

[0078] For any vertex in the triangulation (i.e., any candidate signal point), the other vertices directly connected to that vertex are considered as neighboring candidate signal points.

[0079] As a preferred example, the neighborhood difference of each candidate signal point refers to the absolute value of the difference between the measured spot diffusion radius of each candidate signal point and the measured spot diffusion radii of all its neighboring candidate signal points, and then taking the average value.

[0080] Specifically, when the number of neighboring candidate signal points is equal to 0, the neighborhood difference is set to 0. This situation usually occurs at the edge of the image or in extremely sparse scattered signal areas.

[0081] As a preferred example, the spot diffusion radius is predicted based on the defocus value of each candidate signal point to obtain the theoretical estimated spot radius for each candidate signal point. The method includes:

[0082] Preset minimum spot radius and the divergence rate of the light spot ,in Characterizes the limiting resolution (point spread function radius) under ideal focusing conditions. This characterizes the rate at which the spot radius increases linearly with increasing defocusing degree. The theoretically predicted spot radius... , This represents the defocus value of each candidate signal point.

[0083] As an example, minimum spot radius and the divergence rate of the light spot The default method is:

[0084] Tissue slide samples containing standard-sized fluorescent microspheres (e.g., 0.2 μm in diameter) are prepared. These microspheres can be approximated as point light sources under ideal imaging conditions. The Z-axis stage of the microscope is controlled to move the focal point up and down in preset steps (e.g., 0.5 μm), acquiring a series of microsphere images at different defocus depths. For each microsphere image, the average intensity of the texture edges of each sub-block in the image is obtained. For the largest average texture edge intensity among all sub-blocks in the image, the defocus value is obtained based on this average texture edge intensity (this process is the same as step S101 and will not be repeated here). This defocus value is used as the abscissa. Additionally, the average radius of the microsphere spot in the microsphere image is measured, and the spot radius is used as the ordinate.

[0085] Linear regression fitting was performed on the x and y coordinates obtained from all microsphere images using the least squares method. The intercept of the fitted line is the minimum spot radius. The slope of the fitted straight line is the spot defocus diffusion rate. .

[0086] As a preferred example, the similarity and neighborhood differences of all candidate signal points are smoothly diffused to obtain the signal density of all pixels. The methods include:

[0087] An environmental prediction matching score is obtained based on the degree of agreement between each candidate signal point and the degree of agreement. The environmental prediction matching score is negatively correlated with the degree of agreement. The smaller the degree of agreement, the smaller the difference between the theoretically predicted spot radius and the measured spot diffusion radius of each candidate signal point. The signal of the candidate signal point has a higher reliability, and the larger the environmental prediction matching score is at this time.

[0088] The neighborhood consistency score is obtained based on the neighborhood difference of each candidate signal point. The neighborhood consistency score is negatively correlated with the neighborhood difference. The smaller the neighborhood difference, the smaller the difference between the measured spot diffusion radius of each candidate signal point and its surrounding neighboring candidate signal points. This means that the signal of the candidate signal point has a high degree of reliability, and the larger the neighborhood consistency score is.

[0089] The aforementioned environmental prediction matching score and neighborhood consistency score quantify the credibility or effectiveness of candidate signal points from two perspectives: the macroscopic texture distribution environment and the neighborhood speckled signal distribution, respectively.

[0090] The weighted sum of the environmental prediction matching score and the neighborhood consistency score is used as the comprehensive confidence score for each candidate signal point.

[0091] Smooth diffusion is performed based on the comprehensive confidence score of all candidate signal points to obtain the signal density of all pixels.

[0092] As an example, after obtaining the overall confidence score of all candidate signal points, before performing smooth diffusion, the process also includes setting a confidence threshold. (For example, set to 0.6). Only retain scores with a comprehensive confidence level greater than 0.6. Candidate signal points, less than or equal to Candidate signal points are removed. This example further ensures that the signals from candidate signal points can represent valid probe signals.

[0093] As an example, the formula for calculating the environmental prediction matching score is:

[0094]

[0095] This represents the environmental prediction matching score for each candidate signal point. This represents the measured light spot diffusion radius of each candidate signal point (i.e., the standard deviation of the Gaussian model fitted above). This represents the theoretically estimated spot radius for each candidate signal point. This is the first attenuation factor (e.g., a value of 1.5 pixels). Wherein This indicates the degree of agreement between each candidate signal point.

[0096] As an example, the formula for calculating the neighborhood consistency score is:

[0097] This represents the neighborhood consistency score for each candidate signal point. This represents the neighborhood difference of each candidate signal point. This is the second attenuation factor (for example, a value of 1.0 pixel).

[0098] As an example, the weighted sum of the environmental prediction matching score and the neighborhood consistency score is calculated, where the weights w1 and w2 of the environmental prediction matching score and the neighborhood consistency score are set as follows: 0.4. Specifically, when the number of neighboring candidate signal points for each candidate signal point is 0, it forces a single-dimensional decision based solely on macroscopic texture environment prediction to avoid bias. The corresponding neighborhood division is invalid and interferes with the result. In this case, w1 and w2 are set to respectively 0.

[0099] As an example, smooth diffusion is performed based on the comprehensive confidence score of all candidate signal points to obtain the signal density of all pixels. The methods include:

[0100] For any pixel (including candidate signal points), substitute the coordinates of the pixel and the coordinates of each candidate signal point into the standard Gaussian kernel function. The output of the standard Gaussian kernel function is recorded as the attenuation of each candidate signal point at that pixel. The product of this attenuation and the overall confidence score of each candidate signal point is recorded as the smoothed diffusion result of the overall confidence score of each candidate signal point at that pixel, or simply the smoothed diffusion value of each candidate signal point at that pixel. The mean of the smoothed diffusion values ​​of all candidate signal points at that pixel is recorded as the signal density of that pixel.

[0101] For all pixels, obtain the signal density of all pixels using the method described above.

[0102] This process transforms the quantification results of the signal reliability or validity of discrete candidate signal points into a continuous energy field. Areas with high density values ​​indicate not only dense signals but also reliable optical quality; areas with low density values ​​indicate sparse signals or questionable optical quality.

[0103] As an example, the formula for calculating the signal density is:

[0104]

[0105] This represents the coordinates of the i-th candidate signal point. Represents the coordinates of any pixel; This represents the overall confidence score of the i-th candidate signal point. Represents the standard Gaussian kernel function. h represents the bandwidth of the standard Gaussian kernel function; in this embodiment, h is set to 40. M represents the total number of candidate signal points. This represents the signal density of any pixel. This represents the smooth diffusion value of the i-th candidate signal point at any pixel.

[0106] As a preferred example, for edges extracted based on the signal density of all pixels, different types of cutting edges are identified based on the signal density magnitude on the edges, including the following methods:

[0107] The signal densities of all pixels constitute a signal density distribution map, and the gray value of each pixel in the signal density distribution map is the signal density of each pixel.

[0108] The Otsu's Method is used to automatically calculate the globally optimal segmentation threshold of the signal density distribution map. The signal density distribution map is segmented into a binary image. Regions with a gray value of 1 in the binary image represent gray values ​​greater than or equal to 1 in the density distribution map. In a binary image, a region with a gray value of 0 represents a region in the density distribution map where the gray value is less than [a certain value]. The area.

[0109] Furthermore, a morphological closing operation (dilation followed by erosion) is performed on the binary image. The outer edge contours of each connected component are then extracted from the resulting binary image. For each edge pixel on the outer edge contour, edge pixels continuously distributed along the contour with a mean signal density greater than a preset quality warning threshold are marked as confirmed edges, while edge pixels continuously distributed along the contour with a mean signal density less than or equal to the preset quality warning threshold are marked as questionable edges. In this example, the quality warning threshold is set to... 1.2 times that.

[0110] Visualize the confirmed and doubtful cutting edges on the display screen along all the outer edge contours. For example, duplicate the fluorescence image three times and stack these three fluorescence images into a three-channel color image (i.e., an RGB image). On the color image, set the pixels on the confirmed cutting edges to red, that is, set the pixel value of the pixel to (255, 0, 0), and display the pixels on the doubtful cutting edges in blue, that is, set the pixel value of the pixel to (0, 0, 255). Finally, display the color image on the display screen.

[0111] Thus far, this embodiment has completed the identification of the cutting edge, classifying it into two categories: a confirmed cutting edge and a questionable cutting edge. This allows for a direct distinction between a confirmed cutting edge supported by a high-confidence signal and a questionable cutting edge supported by a low-quality signal (such as defocus or breakage).

[0112] As an example, the specific steps for obtaining confirmed and questionable cutting edges include:

[0113] For any edge pixel on each outer edge contour line, obtain n edge pixels (including the edge pixel) that have the smallest path distance to the edge pixel along each outer edge contour line. If the average signal density of the n edge pixels is greater than the quality indication threshold, mark the edge pixel as the first pixel; otherwise, mark it as the second pixel.

[0114] This example uses n=7 as an example. The path distance refers to the number of edge pixels (including the two starting points) obtained along the outer edge contour line, starting from any two edge pixels.

[0115] For each edge pixel on the outer edge contour, mark them according to the above method. Then, the edge segment formed by the first pixel continuously distributed on the outer edge contour is recorded as the confirmed edge, and the edge segment formed by the second pixel continuously distributed is recorded as the doubtful edge.

[0116] Specifically, when the length of the confirmed edge is less than 5, it indicates that the confirmed edge is too short. In this case, the confirmed edge is re-established as the doubtful edge (which allows it to form a complete doubtful edge with the doubtful edges on both sides). When the length of the doubtful edge is less than 5, it indicates that the doubtful edge is too short. In this case, the doubtful edge is re-established as the confirmed edge (which allows it to form a complete confirmed edge with the confirmed edges on both sides). The purpose of this process is to ensure the continuity of the confirmed edge or the doubtful edge.

[0117] As an example, performing morphological closing operations on a binary image includes the following methods:

[0118] The radius of the structuring element (circular structuring element) used in morphological closing operations Set as dynamic value: , For preset coefficients (e.g.) ), This represents the average measured spot spread radius of all candidate signal points; note that... It needs to be rounded up.

[0119] In this example, when the overall field of view is severely out of focus, the light spot at the signal point generally becomes larger. As it increases, the closed operation radius is reduced. Automatic enlargement. This allows the algorithm to bridge large physical gaps between out-of-focus signals, connecting fragmented areas and helping to maintain the integrity of the connected domains of tissue blocks.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent tumor margin recognition based on HCR-magnified fluorescence images, characterized in that, The method includes the following steps: The defocus value of each pixel is evaluated based on the average intensity of the texture edge of each sub-block in the fluorescence image. The brightness maxima searched in the fluorescence image are recorded as candidate signal points. The measured spot diffusion radius of each candidate signal point is fitted based on the gray value within the local window of each candidate signal point. The average difference between the measured spot diffusion radius of each candidate signal point and its neighboring candidate signal points is recorded as the neighborhood difference of each candidate signal point. The spot diffusion radius is predicted based on the defocus value of each candidate signal point to obtain the theoretical estimated spot radius of each candidate signal point. The difference between the theoretical estimated spot radius and the measured spot diffusion radius is recorded as the degree of fit of each candidate signal point. The degree of fit and neighborhood difference of all candidate signal points are smoothed and diffused to obtain the signal density of all pixels. For the edges extracted based on the signal density of all pixels, different types of cutting edges are identified based on the signal density on the edges.

2. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The specific steps for evaluating the defocus value of each pixel based on the average intensity of the texture edges of each sub-block in the fluorescence image are as follows: For each sub-block in the fluorescence image, the gradient magnitude of each pixel within each sub-block is obtained, and the average gradient magnitude of all pixels within each sub-block is used as the average intensity of the texture edge. The defocus value of the center pixel of each sub-block is determined based on the difference between the preset standard clear texture intensity and the average intensity of the texture edge. The defocus value of all pixels is interpolated based on the defocus values ​​of the center pixels of all sub-blocks.

3. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The specific steps involved in smoothing and diffusing the matching degree and neighborhood differences of all candidate signal points to obtain the signal density of all pixels are as follows: An environmental prediction matching score is obtained based on the degree of fit of each candidate signal point, and the environmental prediction matching score is negatively correlated with the degree of fit. A neighborhood consistency score is obtained based on the neighborhood difference of each candidate signal point, and the neighborhood consistency score is negatively correlated with the neighborhood difference. The weighted sum of the environmental prediction matching score and the neighborhood consistency score is used as the comprehensive confidence score of each candidate signal point. Smoothing diffusion is performed based on the comprehensive confidence scores of all candidate signal points to obtain the signal density of all pixels.

4. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The specific steps for identifying different types of cutting edges based on the signal density of all pixels and the signal density on the edges are as follows: The signal density of all pixels constitutes a signal density distribution map; the signal density distribution map is segmented into a binary image, morphological closing operation is performed on the binary image, and the outer edge contour of each connected component is extracted from the binary image after morphological closing operation. Edge pixels that are continuously distributed on the outer edge contour line and have an average signal density greater than a preset quality prompt threshold are marked as confirmed edges, while edge pixels that are continuously distributed on the outer edge contour line and have an average signal density less than or equal to a preset quality prompt threshold are marked as questionable edges.

5. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 4, characterized in that, The radius of the structuring element used in performing the morphological closing operation is positively correlated with the average measured spot diffusion radius of all candidate signal points.

6. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 3, characterized in that, The specific steps for obtaining the signal density are as follows: For any given pixel, the attenuation of each candidate signal point at that pixel is obtained using the Gaussian kernel function. The product of this attenuation and the overall confidence score of each candidate signal point is recorded as the smooth diffusion result of the overall confidence score of each candidate signal point at that pixel. The mean of the smooth diffusion values ​​of all candidate signal points at that pixel is recorded as the signal density of that pixel.

7. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The specific steps for obtaining the measured light spot diffusion radius are as follows: The gray values ​​of all pixels within the local window in the fluorescence image are fitted to a Gaussian model, and the standard deviation of the Gaussian model is used as the measured light spot diffusion radius.

8. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The specific steps for obtaining the neighborhood candidate signal points are as follows: Construct a Delaunay triangulation with all candidate signal points as vertices. For any candidate signal point in the triangulation, the candidate signal points directly connected to that candidate signal point are considered as the neighboring candidate signal points of that candidate signal point.

9. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 1, characterized in that, The theoretically estimated spot radius is obtained by linearly transforming the defocus value of each candidate signal point.

10. The intelligent tumor margin recognition method based on HCR magnified fluorescence images according to claim 3, characterized in that, After obtaining the overall confidence score for each candidate signal point, only candidate signal points with an overall confidence score greater than the preset confidence threshold are retained.

Images