Fluorescence image processing method and device based on adaptive filtering and multi-scale and multi-mode

Abstract

The invention provides a fluorescence image processing method and device based on adaptive filtering and multi-scale and multi-mode, and the method comprises the steps: firstly converting a fluorescence image into a first gray image, carrying out the extraction of regions of interest, obtaining the pixel statistical information of each region of interest, and carrying out the extraction of the regions of interest according to the pixel statistical information of the regions of interest, according to the method, optimal filter parameters are matched, self-adaptive filtering is realized, the recognition precision and counting accuracy of fluorescent microspheres are remarkably improved, man-made interference in the processing process is reduced, meanwhile, multi-scale decomposition is further carried out on other modal images and the filtered first gray level image by adopting wavelet transform, and then according to the multi-scale decomposition result, the optimal filter parameters are obtained. Edge detection is carried out, and information fusion is carried out on edge results of the two, so that the identification capability of a system on a micro structure under a complex background can be effectively improved, and the reliability of experimental data and the consistency of interpretation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fluorescence image processing, and in particular to a fluorescence image processing method and device based on adaptive filtering and multi-scale and multi-modality. Background Art

[0002] Traditional fluorescence image processing techniques encounter multiple challenges in single-molecule immunoassays, including but not limited to noise interference, background complexity, and low contrast, which limit the ability to accurately identify and count tiny structures (such as fluorescent microspheres). Current solutions usually use filters with static parameters and basic edge detection methods, which cannot effectively cope with the changes and complexities under different image conditions. In addition, when processing multimodal (such as bright field and fluorescence) image information, existing technologies often cannot fully utilize the information advantages of different modalities, resulting in limited accuracy and reliability in fluorescence signal analysis. Therefore, it is necessary to develop a fluorescence image processing method that can adapt to different image conditions and fuse multi-scale and multi-modal images. Summary of the invention

[0003] Based on the technical problems existing in the prior art, the present invention provides a fluorescence image processing method based on adaptive filtering and multi-scale and multi-modal. The use of adaptive filtering technology will significantly improve the recognition accuracy and counting accuracy of fluorescent microspheres and reduce human interference in the processing process. At the same time, the edge detection technology of multi-scale and multi-modal information fusion will effectively enhance the system's ability to recognize tiny structures in complex backgrounds, thereby improving the reliability of experimental data and the consistency of interpretation.

[0004] The present invention provides a fluorescence image processing method based on adaptive filtering and multi-scale multi-modality, which comprises the following steps:

[0005] S1: Acquire a fluorescence image to be processed, and convert the fluorescence image into a first grayscale image;

[0006] S2: extracting a region of interest from the first grayscale image, and obtaining pixel statistics information of each region of interest;

[0007] S3: matching optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filtering the first grayscale with a filter configured with the optimal filter parameters to obtain a second grayscale image;

[0008] S4: performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample by using wavelet transform to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively;

[0009] S5: performing edge detection on the second grayscale image and the high-frequency sub-images of the other modality image at different scales, respectively, to obtain a first edge detection result and a second edge detection result;

[0010] S6: Fusing the first edge detection result and the second edge detection result to obtain a third edge detection result, and determining the fluorescent microspheres in the fluorescent image according to the third edge detection result.

[0011] According to a specific embodiment, in the fluorescence image processing method based on adaptive filtering and multi-scale multi-modality provided by the present invention, the way of obtaining pixel statistical information of each region of interest is: calculating the grayscale distribution and / or region size of each region of interest, and taking the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by grayscale mean and grayscale variance.

[0012] According to a specific embodiment, in the fluorescence image processing method based on adaptive filtering and multi-scale multi-modality provided by the present invention, the way to match the optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image is: according to the grayscale distribution and / or area size of each region of interest, respectively select adaptive filter parameters to maximize the denoising effect.

[0013] According to a specific embodiment, in the fluorescence image processing method based on adaptive filtering and multi-scale and multi-modality provided by the present invention, edge detection is performed on the high-frequency sub-images at different scales of the second grayscale image and the other modality images, respectively, and the first edge detection result and the second edge detection result are obtained in the following manner: pixel points whose wavelet coefficients of each high-frequency sub-image are greater than a set threshold are detected, and the detected pixel points are connected and refined as edge points, thereby obtaining a continuous edge contour.

[0014] According to a specific embodiment, in the fluorescence image processing method based on adaptive filtering and multi-scale multi-modality provided by the present invention, the first edge detection result and the second edge detection result are fused to obtain the third edge detection result in the following manner: edge feature vectors of the first edge detection result and the second edge detection result are extracted, and the distance between the edge feature vectors of the two is calculated. If the distance between the edge feature vectors is less than a set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour.

[0015] Based on the same inventive concept, the present invention also provides a fluorescence image processing device, which includes:

[0016] A preprocessing module, used for acquiring a fluorescent image to be processed and converting the fluorescent image into a first grayscale image;

[0017] A region of interest extraction module, used to extract the region of interest from the first grayscale image and obtain pixel statistical information of each region of interest;

[0018] A filter module, configured to match optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filter the first grayscale image using a filter configured with the optimal filter parameters to obtain a second grayscale image;

[0019] A wavelet transform module, used for performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample, to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively;

[0020] An edge detection module, used to perform edge detection on the second grayscale image and the high-frequency sub-images of the other modal image at different scales, respectively, to obtain a first edge detection result and a second edge detection result;

[0021] The information fusion module is used to fuse the first edge detection result and the second edge detection result to obtain a third edge detection result, and determine the fluorescent microspheres in the fluorescent image according to the third edge detection result.

[0022] According to a specific embodiment, in the fluorescence image processing device provided by the present invention, the region of interest extraction module is configured to: calculate the grayscale distribution and / or region size of each region of interest, and use the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by grayscale mean and grayscale variance.

[0023] According to a specific embodiment, in the fluorescence image processing device provided by the present invention, the filter module is configured to: select adaptive filter parameters according to the grayscale distribution and / or region size of each region of interest, so as to maximize the denoising effect.

[0024] According to a specific embodiment, in the fluorescence image processing device provided by the present invention, the edge detection module is configured to: detect pixel points whose wavelet coefficients of each high-frequency sub-image are greater than a set threshold, and connect and refine the detected pixel points as edge points to obtain a continuous edge contour.

[0025] According to a specific embodiment, in the fluorescence image processing device provided by the present invention, the information fusion module is configured to: extract the edge feature vectors of the first edge detection result and the second edge detection result, and calculate the distance between the edge feature vectors of the two. If the distance between the edge feature vectors is less than a set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: in the fluorescence image processing method based on adaptive filtering and multi-scale and multi-modal of the present invention, the fluorescence image is first converted into a first grayscale image, and then the region of interest is extracted to obtain pixel statistical information of each region of interest, and then the optimal filter parameters are matched according to the pixel statistical information of the region of interest to realize adaptive filtering, which will significantly improve the recognition accuracy and counting accuracy of fluorescent microspheres and reduce human interference in the processing process. At the same time, wavelet transform is further used to perform multi-scale decomposition on other modal images and the filtered first grayscale image, and then edge detection is performed according to the multi-scale decomposition results, and the edge results of the two are fused, which can effectively improve the system's ability to recognize tiny structures under complex backgrounds, thereby improving the reliability of experimental data and the consistency of interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the image processing process of the present invention;

[0028] Figure 2 Schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0030] like Figure 1 As shown, the fluorescence image processing method based on adaptive filtering and multi-scale multi-modality provided by the present invention comprises the following steps:

[0031] S1: Acquire a fluorescence image to be processed, and convert the fluorescence image into a first grayscale image;

[0032] S2: extracting a region of interest from the first grayscale image, and obtaining pixel statistics information of each region of interest;

[0033] S3: matching optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filtering the first grayscale with a filter configured with the optimal filter parameters to obtain a second grayscale image;

[0034] S4: performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample by using wavelet transform to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively;

[0035] S5: performing edge detection on the second grayscale image and the high-frequency sub-images of the other modality image at different scales, respectively, to obtain a first edge detection result and a second edge detection result;

[0036] S6: Fusing the first edge detection result and the second edge detection result to obtain a third edge detection result, and determining the fluorescent microspheres in the fluorescent image according to the third edge detection result.

[0037] Specifically, in the fluorescence image processing method based on adaptive filtering and multi-scale multi-modality provided by the present invention, the pixel statistical information of each region of interest is obtained by calculating the grayscale distribution and / or region size of each region of interest, and using the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by the grayscale value mean and grayscale value variance. Since the pixel information of the region of interest usually reflects the spatial scale of image analysis, in biomedical images, such as cells, tissue structures or lesion areas, the size can vary significantly, and the difference in pixel information of the region of interest will affect the capture of the details and features of the target.

[0038] Further, according to the pixel statistics of each region of interest in the first grayscale image, the optimal filter parameters are matched in the following manner: according to the grayscale distribution and / or region size of each region of interest, the adapted filter parameters are selected respectively to maximize the denoising effect. In implementation, the size of the filter is expressed by the size of its kernel. For example, in a convolutional neural network (CNN), the size of the filter is often represented by a two-dimensional matrix, such as k×k, where k represents the side length of the filter, and this size determines the size of the local area covered by the filter when it moves on the image; in other words, a large-sized local area may require a large filter to capture a wider range of information, while a small-sized local area can use a smaller filter to maintain resolution and details. At the same time, changes in grayscale values ​​will affect the parameters of the filter. For example, when the grayscale value mean and variance are high, the image tends to be bright as a whole, which may mean that the details and noise in the image are more concentrated in the high-brightness area. In this case, the size of the filter can be appropriately increased to capture and smooth these details, thereby reducing high-frequency noise. In addition, the filter function may need to be adjusted to focus more on retaining the structural features of the image while suppressing unnecessary high-frequency components. Conversely, when the grayscale value mean and variance are low, the filter size may need to be reduced to more finely process details in low-brightness areas.

[0039] Specifically, in the fluorescence image processing method based on adaptive filtering and multi-scale and multi-modal provided by the present invention, edge detection is performed on the high-frequency sub-images at different scales of the second grayscale image and the other modal images, and the first edge detection result and the second edge detection result are obtained in the following manner: pixel points whose wavelet coefficients of each high-frequency sub-image are greater than a set threshold are detected, and the detected pixel points are connected and refined as edge points, thereby obtaining a continuous edge contour.

[0040] Furthermore, after obtaining the edge detection results corresponding to the second grayscale image and other modal images (for example, the bright field image of the same sample), the first edge detection result and the second edge detection result are fused to obtain the third edge detection result in the following manner: extracting the edge feature vectors of the first edge detection result and the second edge detection result, and calculating the distance between the edge feature vectors of the two. If the distance between the edge feature vectors is less than the set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour. In implementation, the method of directly fusion of edges is weighted average fusion; and the regional growing method is used as follows: starting from one or more seed points, the neighboring pixels with similar features to the seed points are gradually merged into the growth area until certain growth conditions are met. In edge fusion, regional growing can be used to fuse adjacent images or multi-modal overlapping edge areas, so that the edges of the fused images are smoother and more natural.

[0041] like Figure 2 As shown, the present invention also provides a fluorescence image processing device, which includes:

[0042] A preprocessing module, used for acquiring a fluorescent image to be processed and converting the fluorescent image into a first grayscale image;

[0043] A region of interest extraction module, used to extract the region of interest from the first grayscale image and obtain pixel statistical information of each region of interest;

[0044] A filter module, configured to match optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filter the first grayscale image using a filter configured with the optimal filter parameters to obtain a second grayscale image;

[0045] A wavelet transform module, used for performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample, to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively;

[0046] An edge detection module, used to perform edge detection on the second grayscale image and the high-frequency sub-images of the other modal image at different scales, respectively, to obtain a first edge detection result and a second edge detection result;

[0047] The information fusion module is used to fuse the first edge detection result and the second edge detection result to obtain a third edge detection result, and determine the fluorescent microspheres in the fluorescent image according to the third edge detection result.

[0048] Specifically, in the fluorescence image processing device provided by the present invention, the region of interest extraction module is configured to: calculate the grayscale distribution and / or region size of each region of interest, and use the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by the grayscale value mean and the grayscale value variance.

[0049] The filter module is configured to select adaptive filter parameters according to the grayscale distribution and / or region size of each region of interest to maximize the denoising effect.

[0050] The edge detection module is configured to detect pixel points of each high-frequency sub-image whose wavelet coefficient is greater than a set threshold, and connect and refine the detected pixel points as edge points to obtain a continuous edge contour.

[0051] The information fusion module is configured to: extract edge feature vectors of the first edge detection result and the second edge detection result, and calculate the distance between the edge feature vectors of the two. If the distance between the edge feature vectors is less than a set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour.

[0052] It should be understood that the device disclosed in the present invention can be implemented in other ways. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the communication connection between modules can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical or other forms.

[0053] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one processing unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0054] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fluorescence image processing method based on adaptive filtering and multi-scale and multi-modal, characterized in that: The following steps are involved: S1: Acquire a fluorescence image to be processed, and convert the fluorescence image into a first grayscale image; S2: extracting regions of interest from the first grayscale image, and obtaining pixel statistics information of each region of interest; S3: matching optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filtering the first grayscale with a filter configured with the optimal filter parameters to obtain a second grayscale image; S4: performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample by using wavelet transform to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively; S5: performing edge detection on the second grayscale image and the high-frequency sub-images of the other modality image at different scales, respectively, to obtain a first edge detection result and a second edge detection result; S6: Fusing the first edge detection result and the second edge detection result to obtain a third edge detection result, and determining the fluorescent microspheres in the fluorescent image according to the third edge detection result.

2. The fluorescence image processing method based on adaptive filtering and multi-scale multi-modality as claimed in claim 1, characterized in that: The method for obtaining pixel statistical information of each region of interest is: calculating the grayscale distribution and / or region size of each region of interest, and using the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by the grayscale value mean and the grayscale value variance.

3. The fluorescence image processing method based on adaptive filtering and multi-scale multi-modality as claimed in claim 2, characterized in that: According to the pixel statistics of each region of interest in the first grayscale image, the optimal filter parameters are matched in the following manner: according to the grayscale distribution and / or region size of each region of interest, adaptive filter parameters are selected respectively to maximize the denoising effect.

4. The fluorescence image processing method based on adaptive filtering and multi-scale multi-modality as claimed in claim 3, characterized in that: Edge detection is performed on the second grayscale image and the high-frequency sub-images at different scales of the other modal images respectively, and the first edge detection result and the second edge detection result are obtained in the following manner: pixel points whose wavelet coefficients of each high-frequency sub-image are greater than a set threshold are detected, and the detected pixel points are connected and refined as edge points, so as to obtain a continuous edge contour.

5. The fluorescence image processing method based on adaptive filtering and multi-scale multi-modality as claimed in claim 4, characterized in that: The first edge detection result and the second edge detection result are fused to obtain the third edge detection result by extracting edge feature vectors of the first edge detection result and the second edge detection result, and calculating the distance between the edge feature vectors of the two. If the distance between the edge feature vectors is less than a set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour.

6. A fluorescence image processing device, characterized in that: include: A preprocessing module, used for acquiring a fluorescent image to be processed and converting the fluorescent image into a first grayscale image; A region of interest extraction module, used to extract the region of interest from the first grayscale image and obtain pixel statistical information of each region of interest; A filter module, configured to match optimal filter parameters according to the pixel statistical information of each region of interest in the first grayscale image, and filter the first grayscale image using a filter configured with the optimal filter parameters to obtain a second grayscale image; A wavelet transform module, used for performing multi-scale decomposition on the second grayscale image and at least one other modality image of the same sample, to obtain high-frequency sub-images of the second grayscale image and the other modality image at different scales respectively; An edge detection module, used to perform edge detection on the second grayscale image and the high-frequency sub-images of the other modal image at different scales, respectively, to obtain a first edge detection result and a second edge detection result; The information fusion module is used to fuse the first edge detection result and the second edge detection result to obtain a third edge detection result, and determine the fluorescent microspheres in the fluorescent image according to the third edge detection result.

7. A fluorescent image processing device as claimed in claim 6, characterized in that: The region of interest extraction module is configured to: calculate the grayscale distribution and / or region size of each region of interest, and use the grayscale value distribution and / or region size of each region of interest as the corresponding pixel statistical information; wherein the grayscale distribution is characterized by grayscale mean and grayscale variance.

8. A fluorescent image processing device as claimed in claim 7, characterized in that: The filter module is configured to select adaptive filter parameters according to the grayscale distribution and / or region size of each region of interest to maximize the denoising effect.

9. A fluorescent image processing device as claimed in claim 8, characterized in that: The edge detection module is configured to detect pixel points of each high-frequency sub-image whose wavelet coefficient is greater than a set threshold, and connect and refine the detected pixel points as edge points to obtain a continuous edge contour.

10. A fluorescent image processing device as claimed in claim 9, characterized in that: The information fusion module is configured to: extract edge feature vectors of the first edge detection result and the second edge detection result, and calculate the distance between the edge feature vectors of the two. If the distance between the edge feature vectors is less than a set threshold, the corresponding edges are directly fused; otherwise, the regional growing method is used for fusion to form a complete edge contour.

Images