Endoscopic electronic staining method based on spectral unmixing

Abstract

This invention discloses an endoscopic electronic staining method based on spectral unmixing. It separates the intrinsic tissue spectrum from background noise under LED white light using an improved spectral unmixing algorithm; constructs a multi-scale feature enhancement module to simultaneously enhance mucosal texture and vascular structure; designs a dynamic staining mapping function to achieve adaptive matching between staining parameters and tissue type; and finally, ensures the reliability of the staining effect through dual-index verification. This invention solves core problems such as tissue spectral overlap, low staining contrast, and poor adaptability under ordinary LED white light and CMOS imaging conditions. The method of this invention leverages algorithm optimization to unlock the potential of the imaging system, achieving software-defined staining effects.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of biomedical engineering and endoscopic imaging, specifically involving an electronic staining technology for human tissue based on ordinary LED white light illumination and CMOS sensors. It can be integrated into clinical endoscopic systems such as gastroscopes, colonoscopes, laryngoscopes, and hysteroscopes for early cancer screening, mucosal lesion identification, and vascular morphology observation. Background Technology

[0002] Endoscopic electronic staining technology uses image algorithms to simulate the effects of chemical staining, enhancing the visualization of fine tissue structures. It is a key technology for early cancer screening in areas such as the digestive tract and uterine cavity. Traditional electronic staining methods have significant limitations: one type relies on narrow-band filters or multi-wavelength dedicated light sources, increasing equipment cost and size, and narrow-band light has weak penetration, making it unsuitable for deep tissue observation; another type, although compatible with ordinary LED white light, is limited by the spectral response characteristics of CMOS sensors, resulting in problems such as tissue spectral overlap and low contrast between mucosa and blood vessels in the acquired images.

[0003] Existing algorithms also suffer from significant shortcomings: staining methods using fixed threshold segmentation exhibit poor generalization ability across different tissue types and are prone to producing false edges; schemes based on single-scale feature enhancement struggle to simultaneously preserve details of mucosal surface texture and deep blood vessels; some deep learning models rely on large amounts of paired labeled data, and the scarcity of clinical samples leads to insufficient model robustness. These issues result in bottlenecks in the promotion of existing technologies in primary healthcare institutions, including difficulties in equipment adaptation, operational complexity, and unstable staining results.

[0004] While recent studies have attempted to improve staining results through spectral separation and attention mechanisms, technological fragmentation remains a concern: spectral unmixing and feature enhancement are designed independently without synergistic optimization; staining parameters are mostly statically configured, failing to dynamically match tissue spectral differences and thus hindering the clinical demand for precise diagnosis. Therefore, developing a highly efficient electronic staining method adapted to ordinary LED white light and based on CMOS imaging is crucial for promoting the widespread adoption of endoscopic technology at the grassroots level. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing technologies and provide an endoscopic electronic staining method based on spectral unmixing: an improved spectral unmixing algorithm is used to separate the intrinsic spectrum of tissue under LED white light from background noise; a multi-scale feature enhancement module is constructed to simultaneously enhance mucosal texture and vascular structure; a dynamic staining mapping function is designed to achieve adaptive matching between staining parameters and tissue type; and finally, dual-index verification is used to ensure the reliability of the staining effect.

[0006] The technical solution adopted in this invention is: an endoscopic electronic staining method based on spectral unmixing, comprising the following steps: Step 1: CMOS Image Acquisition: Based on a conventional LED white light endoscope system, raw images of human tissue are acquired through a CMOS sensor, and imaging parameters are recorded simultaneously; Step 2: Image preprocessing: Noise suppression and white balance correction are performed on the original image to eliminate uneven lighting and sensor noise interference; Step 3: Tissue spectral unmixing: A weighted nonnegative matrix factorization algorithm is used to separate the intrinsic tissue spectrum from background interference and extract the pure tissue spectral features; Step 4: Multi-scale feature enhancement: Simultaneously enhance mucosal texture and vascular structure features through multi-scale convolution and channel attention mechanisms; Step 5: Dynamic staining mapping: Based on the spectral characteristics of the tissue, the staining parameters are dynamically adjusted to generate the target staining image and output it for clinical diagnosis.

[0007] Furthermore, in step 1, an endoscope system equipped with a common RGB three-color LED white light source is used. The CMOS sensor resolution is adapted to commonly used endoscope specifications, and the acquisition frame rate is synchronized with the endoscope main controller. During acquisition, the brightness of the LED light source is kept stable, and medical staff operate the endoscope to keep the distance between the lens and the tissue within the clinically standard range, avoiding image distortion caused by being too close or blurring caused by being too far. The acquired raw image is recorded as follows: , where (x,y) are pixel coordinates, and the image channels are RGB three channels.

[0008] Furthermore, step 2 eliminates interference through noise suppression and white balance correction to provide a clean input for subsequent processing, comprising two steps: (1) Adaptive noise suppression A joint spatial-frequency domain denoising algorithm is adopted, and the formula is: Symbol explanation: This is the denoised image; GF() is the guided filter operator used to preserve edge details; The noise weight is calculated from the local variance of the pixels; the larger the variance, the higher the weight. (2) Gray world white balance The formula for correcting color shift caused by color temperature fluctuations in LED light sources is: Symbol explanation: 'c' represents the R, G, and B channels; This represents the c-channel pixel value after white balance. The mean value of the c channel of the denoised image; This is the average of the three channel values, and this correction helps to balance the grayscale of the image.

[0009] Furthermore, step 3 employs a weighted nonnegative matrix factorization algorithm to separate the intrinsic tissue spectrum from background interference, utilizing the nonnegativity constraint of the tissue spectrum to improve unmixing accuracy. This includes two steps: spectral modeling and weighted unmixing. (1) Spectral matrix modeling The preprocessed image is converted into a spectral matrix form using the following formula: Symbol explanation: X is an M×3 spectral matrix, where M is the total number of pixels in the image, and each row represents the R, G, and B spectral response values ​​of a pixel; Reshape() is the matrix reshaping operator, which converts a two-dimensional image into a one-dimensional pixel spectral matrix; (2) Weighted nonnegative unmixing Introducing organizational feature weights to achieve accurate unmixing, the formula is as follows: Symbol explanation: W is an M×M tissue feature weight matrix, calculated from pixel gray-level gradients, with higher weights for edge regions; A is a 3×K spectral basis matrix; K is the number of tissue components, such as mucosa, blood vessels, and background; S is a K×M spectral abundance matrix, representing the proportion of each tissue component; N is an M×3 noise matrix; by iteratively optimizing A and S, the reconstruction error is minimized, and finally S is extracted as the pure tissue spectral feature.

[0010] Furthermore, step 4 involves constructing a multi-scale convolutional module combined with channel attention to simultaneously enhance organizational features at different scales, including two steps: multi-scale feature extraction and attention fusion. (1) Multi-scale feature extraction Feature extraction is performed using different convolutional kernel sizes, as shown in the formula: Symbol explanation: Here is the feature map corresponding to the kernel size k; Conv(·,k) is a k×k convolution operator; 3×3 convolution extracts micro-texture, and 5×5 and 7×7 convolution extract medium and macro-structural features respectively; (2) Channel attention fusion Adaptive weights are assigned to multi-scale features and then fused, as shown in the formula: Symbol explanation: This is the fused feature map; The attention weights for features at each scale are obtained by normalizing the channel response values ​​of the feature map using the Sigmoid function. The higher the response, the greater the weight, thus enhancing important features.

[0011] Furthermore, step 5 involves dynamically adjusting staining parameters based on tissue spectral abundance to generate a staining image that conforms to clinical observation practices. This includes two steps: staining parameter calculation and RGB mapping. (1) Calculation of dynamic parameters The staining intensity is adjusted according to the abundance of vascular components, using the following formula: Symbol explanation: These are pixel-level staining intensity parameters; σ represents the abundance of blood vessel components in this pixel (taken from the blood vessel column of the S matrix); σ is an adjustment coefficient used to control the range of parameter changes. The higher the blood vessel abundance, the greater the staining intensity. (2) RGB channel mapping The enhanced features are mapped to an RGB-colored image using the following formula: Symbol explanation: The pixel value of the c channel of the stained image; The c-channel response value of the fused feature map is used; the enhanced features are fused with the original image through dynamic weighting to ensure that the staining effect is natural and the details are highlighted.

[0012] The innovation of this invention lies in: abandoning the dependence on special light sources or filter hardware, and using a pure software solution of spectral demixing and algorithm enhancement to explore the spectral information potential of ordinary LED white light and CMOS imaging, so that the electronic staining effect breaks through hardware limitations, and at the same time, it can adapt to the imaging characteristics of different tissues through dynamic parameter adjustment.

[0013] This invention proposes an endoscopic electronic staining method based on spectral unmixing. Through technological innovations such as preprocessing noise reduction, weighted spectral unmixing, multi-scale feature enhancement, and dynamic staining mapping, it solves the core problems of tissue spectral overlap, low staining contrast, and poor adaptability under ordinary LED white light and CMOS imaging conditions. This method also optimizes the imaging system through algorithmic optimization to achieve software-defined staining effects.

[0014] The beneficial effects of this invention are: Excellent hardware compatibility: It is compatible with ordinary LED white light sources and universal CMOS sensors, requiring no modification to endoscope hardware and reducing the cost of clinical promotion.

[0015] Highly targeted staining: The dynamic staining mapping parameters are adaptively adjusted according to the spectral characteristics of the tissue, adapting to the detection needs of multiple sites and types of tissues.

[0016] Complete detail preservation: The multi-scale feature enhancement module simultaneously enhances micro-texture and macro-structure, solving the problem of "losing sight of one aspect while focusing on another" in traditional methods.

[0017] Outstanding robustness and convenience: The pure software solution reduces equipment dependence, the standardized process is adapted to primary healthcare operations, and the staining effect is stable and reliable.

[0018] High clinical value: It simulates the effect of chemical staining while avoiding reagent irritation, allows for repeated observation without the risk of tissue damage, and improves screening efficiency. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of the endoscopic electronic staining method of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: like Figure 1 As shown, an endoscopic electronic staining method based on spectral unmixing includes the following steps: Step 1: CMOS Image Acquisition The endoscope system employs a standard RGB three-color LED white light source. The CMOS sensor resolution is adapted to commonly used endoscope specifications, and the acquisition frame rate is synchronized with the endoscope's main controller. During acquisition, stable LED light source brightness is ensured, and medical personnel operate the endoscope to maintain the lens-to-tissue distance within the standard clinical range, avoiding image distortion caused by being too close or blurring caused by being too far. The acquired raw images are denoted as... , where (x,y) are pixel coordinates, and the image channels are RGB three channels.

[0021] Step 2: Image Preprocessing Noise suppression and white balance correction eliminate interference, providing a clean input for subsequent processing. The core process involves two steps: (1) Adaptive noise suppression A joint spatial-frequency domain denoising algorithm is adopted, and the formula is: Symbol explanation: This is the denoised image; GF() is the guided filter operator used to preserve edge details; The noise weight is calculated from the local variance of the pixels; the larger the variance, the higher the weight.

[0022] (2) Gray world white balance The formula for correcting color shift caused by color temperature fluctuations in LED light sources is: Symbol explanation: 'c' represents the R, G, and B channels; This represents the c-channel pixel value after white balance. The mean value of the c channel of the denoised image; This is the average of the three channel values, and this correction helps to balance the grayscale of the image.

[0023] Step 3: Tissue spectral unmixing A weighted nonnegative matrix factorization algorithm is used to separate the intrinsic spectrum of tissue from background interference. The nonnegativity constraint of the tissue spectrum is utilized to improve the unmixing accuracy. The core steps include spectral modeling and weighted unmixing. (1) Spectral matrix modeling The preprocessed image is converted into a spectral matrix form using the following formula: Symbol explanation: X is an M×3 spectral matrix, where M is the total number of pixels in the image, and each row represents the R, G, and B spectral response values ​​of a pixel; Reshape() is the matrix reshaping operator, which converts a two-dimensional image into a one-dimensional pixel spectral matrix.

[0024] (2) Weighted nonnegative unmixing Introducing organizational feature weights to achieve accurate unmixing, the formula is as follows: Notation: W is an M×M tissue feature weight matrix, calculated from pixel gray-level gradients, with higher weights for edge regions; A is a 3×K spectral basis matrix; K represents the number of tissue components, such as mucosa, blood vessels, and background; S is a K×M spectral abundance matrix, representing the proportion of each tissue component; N is an M×3 noise matrix. By iteratively optimizing A and S, the reconstruction error is minimized, and S is finally extracted as the pure tissue spectral feature.

[0025] Step 4: Multi-scale feature enhancement A multi-scale convolutional module combined with channel attention is constructed to simultaneously enhance organizational features at different scales. The core of this approach includes two steps: multi-scale feature extraction and attention fusion. (1) Multi-scale feature extraction Feature extraction is performed using different convolutional kernel sizes, as shown in the formula: Symbol explanation: is the feature map corresponding to the kernel size k; Conv(·,k) is a k×k convolution operator; 3×3 convolution extracts micro-texture, and 5×5 and 7×7 convolution extract medium and macro-structural features respectively.

[0026] (2) Channel attention fusion Adaptive weights are assigned to multi-scale features and then fused, as shown in the formula: Symbol explanation: This is the fused feature map; The attention weights for features at each scale are obtained by normalizing the channel response values ​​of the feature map using the Sigmoid function. The higher the response, the greater the weight, thus enhancing important features.

[0027] Step 5: Dynamic staining mapping Based on the dynamic adjustment of staining parameters according to tissue spectral abundance, staining images that conform to clinical observation habits are generated. The core process includes two steps: staining parameter calculation and RGB mapping. (1) Calculation of dynamic parameters The staining intensity is adjusted according to the abundance of vascular components, using the following formula: Symbol explanation: These are pixel-level staining intensity parameters; σ represents the abundance of blood vessel components in this pixel (taken from the blood vessel column of the S matrix); σ is an adjustment coefficient used to control the range of parameter changes. The higher the blood vessel abundance, the greater the staining intensity.

[0028] (2) RGB channel mapping The enhanced features are mapped to an RGB-colored image using the following formula: Symbol explanation: The pixel value of the c channel of the stained image; The c-channel response value of the fused feature map is used; the enhanced features are fused with the original image through dynamic weighting to ensure that the staining effect is natural and the details are highlighted.

[0029] This invention offers significant advantages, including broad hardware compatibility, stable staining results, and convenient clinical operation. It can be directly integrated into existing endoscopic equipment without the need for additional dedicated light sources or sensors, significantly reducing the technical application costs for primary healthcare institutions. By dynamically adapting to the spectral characteristics of different tissues, it achieves simultaneous enhancement of mucosal texture and vascular structure, providing reliable technical support for early detection and diagnosis of early cancer. In the future, it can be further expanded to dermatoscopy, laparoscopy, and other fields, providing a more comprehensive staining technology solution for precision medicine diagnosis, possessing significant clinical translational value and social significance. The above description is merely a preferred embodiment of the invention and is not intended to limit the invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An endoscopic electronic staining method based on spectral unmixing, characterized in that: Includes the following steps: Step 1: CMOS Image Acquisition: Based on a conventional LED white light endoscope system, raw images of human tissue are acquired through a CMOS sensor, and imaging parameters are recorded simultaneously; Step 2: Image preprocessing: Noise suppression and white balance correction are performed on the original image to eliminate uneven lighting and sensor noise interference; Step 3: Tissue spectral unmixing: A weighted nonnegative matrix factorization algorithm is used to separate the intrinsic tissue spectrum from background interference and extract the pure tissue spectral features; Step 4: Multi-scale feature enhancement: Simultaneously enhance mucosal texture and vascular structure features through multi-scale convolution and channel attention mechanisms; Step 5: Dynamic staining mapping: Based on the spectral characteristics of the tissue, the staining parameters are dynamically adjusted to generate the target staining image and output it for clinical diagnosis.

2. The endoscopic electron staining method based on spectral unmixing according to claim 1, characterized in that: In step 1, an endoscope system equipped with a standard RGB three-color LED white light source is used, and the frame rate is synchronized with the endoscope's main controller. During acquisition, the LED light source brightness is kept stable, and the endoscope is operated to maintain the distance between the lens and the tissue within the standard clinical range, avoiding image distortion caused by being too close or blurring caused by being too far. The acquired raw images are recorded as follows: , where (x,y) are pixel coordinates, and the image channels are RGB three channels.

3. The endoscopic electron staining method based on spectral unmixing according to claim 2, characterized in that: Step 2 involves noise suppression and white balance correction to eliminate interference and provide a clean input for subsequent processing. This includes two steps: (1) Adaptive noise suppression A joint spatial-frequency domain denoising algorithm is adopted, and the formula is: Symbol explanation: This is the denoised image; GF() is the guided filter operator used to preserve edge details; The noise weight is calculated from the local variance of the pixels; the larger the variance, the higher the weight. (2) Gray world white balance The formula for correcting color shift caused by color temperature fluctuations in LED light sources is: Symbol explanation: 'c' represents the R, G, and B channels; This represents the c-channel pixel value after white balance. The mean value of the c channel of the denoised image; This is the average of the three channel values, and this correction helps to balance the grayscale of the image.

4. The endoscopic electron staining method based on spectral unmixing according to claim 3, characterized in that: Step 3 employs a weighted nonnegative matrix factorization algorithm to separate the intrinsic tissue spectrum from background interference. The nonnegativity constraint of the tissue spectrum is utilized to improve unmixing accuracy. This process includes two steps: spectral modeling and weighted unmixing. (1) Spectral matrix modeling The preprocessed image is converted into a spectral matrix form using the following formula: Symbol explanation: X is an M×3 spectral matrix, where M is the total number of pixels in the image, and each row represents the R, G, and B spectral response values ​​of a pixel; Reshape() is the matrix reshaping operator, which converts a two-dimensional image into a one-dimensional pixel spectral matrix; (2) Weighted nonnegative unmixing Introducing organizational feature weights to achieve accurate unmixing, the formula is as follows: Symbol explanation: W is an M×M tissue feature weight matrix, calculated from pixel gray-level gradient, with higher weights for edge regions; A is a 3×K spectral basis matrix; K is the number of tissue components; S is a K×M spectral abundance matrix, representing the proportion of each tissue component; N is an M×3 noise matrix; by iteratively optimizing A and S, the reconstruction error is minimized, and finally S is extracted as the pure tissue spectral feature.

5. The endoscopic electron staining method based on spectral unmixing according to claim 4, characterized in that: Step 4 involves constructing a multi-scale convolutional module combined with channel attention to simultaneously enhance organizational features at different scales. This includes two steps: multi-scale feature extraction and attention fusion. (1) Multi-scale feature extraction Feature extraction is performed using different convolutional kernel sizes, as shown in the formula: Symbol explanation: Here is the feature map corresponding to the kernel size k; Conv(·,k) is a k×k convolution operator; 3×3 convolution extracts micro-texture, and 5×5 and 7×7 convolution extract medium and macro-structural features respectively; (2) Channel attention fusion Adaptive weights are assigned to multi-scale features and then fused, as shown in the formula: Symbol explanation: This is the fused feature map; The attention weights for features at each scale are obtained by normalizing the channel response values ​​of the feature map using the Sigmoid function. The higher the response, the greater the weight, thus enhancing important features.

6. The endoscopic electron staining method based on spectral unmixing according to claim 5, characterized in that: Step 5 involves dynamically adjusting staining parameters based on tissue spectral abundance to generate staining images that conform to clinical observation practices. This includes two steps: staining parameter calculation and RGB mapping. (1) Calculation of dynamic parameters The staining intensity is adjusted according to the abundance of vascular components, using the following formula: Symbol explanation: These are pixel-level staining intensity parameters; σ represents the abundance of blood vessels in this pixel; σ is an adjustment coefficient used to control the range of parameter changes. The higher the blood vessel abundance, the greater the staining intensity. (2) RGB channel mapping The enhanced features are mapped to an RGB-colored image using the following formula: Symbol explanation: The pixel value of the c channel of the stained image; The c-channel response value of the fused feature map; By using dynamic weighted fusion to enhance features and the original image, the staining effect is ensured to be natural and the details are highlighted.

Images