Endoscope narrow band image separation method, device, equipment and medium

Abstract

The application discloses an endoscope narrow-band image separation method and device, equipment and medium, relates to the field of medical image processing and computer vision, and comprises the following steps: acquiring a mixed illumination original image collected by double narrow-band light of an endoscope; inputting the image into a trained target generative adversarial network model, and generating a corresponding single-band separation image by a spectral decoupling generator in the model; the model is trained by triple images of the same part, and the triple images contain a mixed illumination image and real single-band images collected by single narrow-band light; then, according to the clinical characteristics of two bands, the separated images are respectively subjected to blood vessel feature enhancement optimization, and single-band target separation images are obtained, so that effective separation of mixed spectrum information can be realized in the double narrow-band light synchronous illumination collection mode.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of medical image processing and computer vision, and particularly to a method, apparatus, device, and medium for separating narrow-band endoscopic images. Background Technology

[0002] Narrow-band light imaging, a commonly used electronic staining technique in gastrointestinal endoscopy, enhances the contrast of microvessels and glandular structures on the mucosal surface by illuminating tissue with specific wavelengths of narrow-band light. This clearly reveals lesion boundaries, morphology, and vascular distribution characteristics, making it easier to detect early, minute lesions compared to traditional white light imaging. It has become a standard feature of gastrointestinal electronic endoscopes. Currently, mainstream narrow-band light imaging schemes fall into two main categories: one uses mechanical structures to switch narrow-band filters to acquire images of different wavelengths in a time-division manner; the other uses simultaneous illumination with dual narrow-band light and combines this with tone mapping for image processing. Both methods are widely used in clinical practice.

[0003] Narrowband light imaging using time-division acquisition requires alternating acquisition of images from different spectral bands, which leads to a decrease in the imaging frame rate, affecting video smoothness. Furthermore, it is prone to registration deviations between frames due to patient breathing, gastrointestinal peristalsis, or movement, causing inconvenience for doctors' diagnosis. While simultaneous illumination with dual narrowband light can maintain a stable frame rate, the spectral information of different bands will produce nonlinear aliasing within the sensor channel. Physical decoupling of band information cannot be achieved solely through tone mapping, making it difficult to present the optimal vascular contrast for a single band. It also leads to problems such as mutual interference between different band signals and the masking of subtle vascular information.

[0004] In summary, under the premise of simultaneous illumination by dual narrowband lights to acquire mixed illumination images, how to separate spectral information from endoscopic narrowband images with spectral overlap is a problem to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for separating narrow-band endoscopic images, which achieves spectral information separation of narrow-band endoscopic images with spectral aliasing under the premise of simultaneous illumination by dual narrow-band light to acquire mixed illumination images. The specific solution is as follows: In a first aspect, this application discloses a method for separating narrow-band endoscopic images, including: Acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area; The current original image is spectrally decoupled using the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band. The target adversarial generative network model is trained using triplet data paired with the same part. The triplet data includes historical mixed illumination images and real single band images. The real single band images are the original illumination images collected by the endoscope when the target part is illuminated by the corresponding narrow band light. The vascular features in each of the current separated images are enhanced and optimized based on the bands corresponding to each current separated image to obtain the target separated image corresponding to each single band.

[0006] Optionally, the process of creating the target adversarial generative network model includes: Construct a current adversarial generative network model that includes a spectral decoupling generator, a discriminator for each single band, and a model constraint; Triple data collected from the same location and paired together were used as training data. The training data is input into the current adversarial generative network model; The spectral decoupling generator in the current adversarial generative network model generates each predicted single-band image, and each discriminator outputs the discrimination result of the predicted single-band image. Based on the discrimination result, the loss value of the spectral decoupling generator is generated, and the model constraint is used to update the parameters of the spectral decoupling generator and the discriminator corresponding to each single band based on the loss value, so as to obtain a new current adversarial generative network model. The process then jumps back to the step of inputting the training data into the current adversarial generative network model until the preset stopping condition is met, and the new current adversarial generative network model obtained after parameter update is determined as the target adversarial generative network model.

[0007] Optionally, the spectral decoupling generator includes a shared feature extraction encoder constructed based on a multi-scale residual network and a dual-branch decoder; generating each predicted single-band image using the spectral decoupling generator in the current adversarial generative network model includes: The historical mixed lighting images in the training data are preprocessed to obtain standardized image data; The shared feature extraction encoder is used to extract multi-scale information from the standardized image data to obtain a feature tensor; The dual-branch decoder is used to separate the aliased shared feature tensors in the feature tensor into independent channel feature streams corresponding to each single band. The feature streams of each channel are mapped to the RGB channels to obtain the predicted single-band images corresponding to each single band.

[0008] Optionally, the discriminator adopts a PatchGAN architecture; the step of using each discriminator to output the discrimination result of the predicted single-band image includes: Each discriminator outputs a true / false score feature map corresponding to the predicted single-band image, and the mean of the true / false score feature maps is determined as the discrimination result corresponding to the predicted single-band image.

[0009] Optionally, generating the loss value of the spectral decoupling generator based on the discrimination result includes: The authenticity loss value of the spectral decoupling generator is determined based on the discrimination results of each predicted single-band image. The sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value is determined as the loss value of the spectral decoupling generator.

[0010] Optionally, determining the sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value as the loss value of the spectral decoupling generator includes: The predicted single-band images are weighted and fused according to the preset spectral mixing coefficients. The difference between the fusion result and the corresponding historical mixed illumination image is calculated to obtain the physical model loss value of the spectral decoupling generator. The feature differences between each predicted single-band image and the corresponding real single-band image are determined to obtain the feature matching loss value of the spectral decoupling generator. The real single-band image is input into the spectral decoupling generator to obtain the corresponding reconstructed single-band image. The difference between the reconstructed single-band image and the corresponding real single-band image is calculated to obtain the identity loss value of the spectral decoupling generator. The sum of the authenticity loss value, the physical model loss value, the feature matching loss value, and the identity loss value is determined as the loss value of the spectral decoupling generator.

[0011] Optionally, the dual narrowband includes a first single narrowband with a center wavelength of 420nm and a second single narrowband with a center wavelength of 540nm; the enhancement and optimization processing of vascular features in each of the current separated images based on the corresponding bands to obtain the target separated image corresponding to each single band includes: The current separated image corresponding to the first single narrow band is subjected to surface microvascular enhancement processing to obtain the first target separated image corresponding to the first single narrow band; Enhance the contrast of blood vessel contours in the current separated image corresponding to the second single narrow band to obtain the second target separated image corresponding to the second single narrow band.

[0012] Secondly, this application discloses an endoscopic narrow-band image separation device, comprising: The image acquisition module is used to acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area; The image separation module is used to perform spectral decoupling processing on the current original image using the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same part, the triplet data includes historical mixed illumination images and each real single band image, the real single band image is the original illumination image collected by the endoscope using the corresponding single narrowband light to illuminate the target part alone; The enhancement and optimization module is used to perform enhancement and optimization processing on the vascular features in each of the current separated images based on the bands corresponding to each current separated image, so as to obtain the target separated image corresponding to each single band.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed endoscopic narrow-band image separation method.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed endoscopic narrow-band image separation method.

[0015] The beneficial effects of this application are as follows: This application obtains the current original image acquired by an endoscope; wherein, the current original image is a mixed illumination image acquired by the endoscope using dual narrowband light to simultaneously illuminate the target area; the current original image is subjected to spectral decoupling processing by the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same location, the triplet data including historical mixed illumination images and each real single band image, the real single band image being the original illumination image acquired by the endoscope using the corresponding single narrowband light to individually illuminate the target area; the vascular features in each of the current separated images are enhanced and optimized based on the band corresponding to each of the current separated images to obtain the target separated image corresponding to each single band. Therefore, this application, by acquiring mixed illumination images of the target area simultaneously illuminated by dual narrowband lights using an endoscope, can complete the original image acquisition without reducing the imaging frame rate or causing image registration deviation. Relying on the spectral decoupling generator in the target adversarial generative network model trained with triplet data from the same location, spectral decoupling processing is performed on the mixed illumination image. This physically decouples the mixed image into independent single-band images, effectively avoiding signal interference and information overload between bands. Furthermore, based on the imaging characteristics corresponding to each single band, targeted enhancement and optimization processing is performed on the vascular features of the separated images, which can significantly improve the display clarity and feature recognition of vascular structures at different depths. This does not require time-division acquisition or simple tone mapping adjustment. While ensuring imaging smoothness, it achieves accurate separation of narrowband light information and improved visualization effects, providing more reliable image support for clinical observation and diagnosis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of an endoscopic narrow-band image separation method disclosed in this application; Figure 2 This application discloses a specific end-to-end narrowband image separation flowchart; Figure 3 This is a schematic diagram of a specific shared feature extraction encoder disclosed in this application; Figure 4 This is a schematic diagram of a specific 420nm decoder disclosed in this application; Figure 5 This is a schematic diagram of the structure of an endoscopic narrow-band image separation device disclosed in this application; Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Narrow-band light imaging, also known as electronic staining, uses specific narrow-band wavelengths of light to illuminate tissues, enhancing the contrast of microvessels and structures on the mucosal surface. This allows for clear visualization of lesion boundaries, morphology, and vascular distribution characteristics. Currently, most gastrointestinal electronic endoscopes using narrow-band light imaging utilize the strong absorption characteristics of hemoglobin at 410nm and 540nm narrow-band light. The blue-violet band (410nm) penetrates shallower to visualize the submucosal vascular network, while the green band (540nm) can better visualize the vessels in the intermediate layer, resulting in high-contrast images of mucosal vessels. Compared to traditional white light imaging, it can detect minute lesions earlier, improving the diagnostic accuracy of early-stage lesions.

[0020] Narrow-band imaging technologies include NBI (Narrow Band Imaging), BlueLight Imaging, and Versatile Intelligent Staining Technology. NBI uses a moving structure to drive a narrow-band filter to acquire 415nm and 540nm narrow-band light images in a time-division manner. BLI uses a 410nm narrow-band blue laser, supplemented by simultaneous illumination with short-wavelength green light, and obtains electronic staining images through real-time tone mapping. Currently, most electronic endoscopes use simultaneous illumination with 415nm narrow-band light and 540nm short-wavelength green LEDs, and then obtain the final narrow-band imaging effect through real-time tone mapping.

[0021] Because NBI technology requires time-division acquisition of two narrowband images, it inevitably needs to reduce the frame rate to half that of normal imaging. For example, when switching from a white light image to an NBI staining image, the frame rate drops from 60Hz to 30Hz, which affects the smoothness of the video. During the switching process, patient breathing, peristalsis, or operator movement can cause the two frames to be inaccurately registered, making it difficult for doctors to compare and diagnose. BLI and other electron chromatographic techniques use narrowband light mixed with short-wavelength green light for simultaneous illumination, and then acquire electron chromatographic images through image processing. In essence, they do not truly acquire images under the characteristic narrowband band. The spectral information of the two bands undergoes nonlinear aliasing within the RGB sensor channel. The corresponding image processing is essentially a holistic color adjustment, which cannot fundamentally decouple and separate the information of the two bands at the physical level, leading to the following problems: 1) It is impossible to observe and evaluate the optimal vascular contrast under a single band independently; 2) There is mutual interference in the mixed signal, and the delicate 420nm vascular signal may be overwhelmed by the strong 540nm signal.

[0022] Therefore, this application provides a corresponding endoscopic narrowband image separation scheme, which separates spectral information from endoscopic narrowband images with spectral overlap under the premise of simultaneous illumination of dual narrowband lights to acquire mixed illumination images.

[0023] See Figure 1 As shown in the figure, this application discloses a method for separating narrow-band endoscopic images, including: Step S11: Acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area.

[0024] In this embodiment, the dual narrowband includes a first single narrowband with a center wavelength of 420nm and a second single narrowband with a center wavelength of 540nm.

[0025] Acquire the current raw image captured by the endoscope. This current raw image is a dual-band mixed illumination image acquired by the endoscope using a first single narrowband light with a center wavelength of 420nm and a second single narrowband light with a center wavelength of 540nm to simultaneously illuminate the target area of ​​the digestive tract endoscope. The image is a 1920×1080 resolution RGB (Red, Green, Blue) image, without motion artifacts and collected from the same location, which can be used as input data for spectral decoupling processing in the subsequent deep learning-based end-to-end hybrid narrowband image separation method.

[0026] Step S12: Use the spectral decoupling generator in the target adversarial generative network model to perform spectral decoupling processing on the current original image to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same part, the triplet data includes historical mixed illumination images and each real single band image, the real single band image is the original illumination image collected by the endoscope using the corresponding single narrowband light to illuminate the target part alone.

[0027] The original image is spectrally decoupled using the spectral decoupling generator in the target adversarial generative network model, resulting in the separated images corresponding to the 420nm and 540nm single bands. , This spectral decoupling generator is built on a U-Net convolutional neural network architecture, comprising an input layer submodule, a shared feature extraction encoder submodule, a dual-branch decoder submodule, and an output layer submodule. The target adversarial generative network model is trained using effective triplet images acquired continuously from the same location without significant motion artifacts. The triplet data includes dual-band mixed illumination images and real single-band images acquired only under 420nm illumination. Real single-band images acquired with only 540nm illumination The model is also equipped with independent 420nm discriminators, 540nm discriminators, and loss function and model constraint modules to ensure that the spectral decoupling generator can accurately output separation results consistent with real single-band images.

[0028] In this embodiment, the creation process of the target adversarial generative network model includes: constructing a current adversarial generative network model including a spectral decoupling generator, discriminators corresponding to each single band, and a model constraint; determining the collected triplet data of the same part as training data; inputting the training data into the current adversarial generative network model; generating each predicted single band image using the spectral decoupling generator in the current adversarial generative network model, outputting the discrimination result of the predicted single band image using each discriminator, generating the loss value of the spectral decoupling generator based on the discrimination result, and updating the parameters of the spectral decoupling generator and the discriminators corresponding to each single band using the model constraint based on the loss value to obtain a new current adversarial generative network model; and returning to the step of inputting the training data into the current adversarial generative network model until a preset stopping condition is met, and determining the new current adversarial generative network model obtained after parameter update as the target adversarial generative network model.

[0029] For example Figure 2 As shown, a spectral decoupling generator and a 420nm discriminator are first constructed. 540nm discriminator And the loss function and model constraint module The current adversarial generative network model uses effective triplet images without significant motion artifacts, acquired continuously from the same site using an endoscope with switchable illumination modes, as training data. These triplet images include dual-band mixed illumination images. Images with only 420nm illumination With only 540nm illumination image The training data is then input into the current adversarial generative network model, which generates the corresponding predicted 420nm single-band image through a spectral decoupling generator. Compared with the predicted 540nm single-band image Then, using 420nm discriminators with PatchGAN architecture respectively Compared with 540nm discriminator The predicted single-band image is discriminated and the discrimination result is output, providing a realistic gradient for the spectral decoupling generator module and forcing the prediction of the 420nm single-band image. Distribution towards real images Distribution convergence and prediction of 540nm single-band image Distribution towards real images The distribution converges; the model constraint calculates the loss value based on the total loss function, which includes adversarial loss, physical model loss, identity loss and feature matching loss. Based on this loss value, the parameters of the spectral decoupling generator and the two discriminators are updated synchronously to obtain the current adversarial generative network model with updated parameters. The training data input and parameter iteration update steps are repeated until the total loss function reaches the preset stopping condition of the Pareto optimal equilibrium point. Finally, the model with converged parameters is determined as the target adversarial generative network model.

[0030] In this embodiment, the spectral decoupling generator includes a shared feature extraction encoder built based on a multi-scale residual network and a dual-branch decoder. Generating each predicted single-band image using the spectral decoupling generator in the current generative adversarial network model includes: preprocessing the historical mixed illumination images in the training data to obtain standardized image data; extracting multi-scale information from the standardized image data using the shared feature extraction encoder to obtain a feature tensor; separating the aliased shared feature tensor in the feature tensor into independent channel feature streams corresponding to each single band using the dual-branch decoder; and mapping each channel feature stream to RGB channels to obtain the predicted single-band image corresponding to each single band.

[0031] The spectral decoupling generator adopts a U-Net convolutional neural network-based structure, comprising an input layer submodule, a shared feature extraction encoder submodule built based on a three-layer multi-scale residual network Res2Net, a dual-branch decoder submodule consisting of a 420nm decoder and a 540nm decoder, and an output layer submodule, such as... Figure 3 As described above, when generating predicted single-band images using the spectral decoupling generator in the current adversarial generative network model, the historical mixed illumination images in the training data are first standardized and preprocessed through the input layer submodule, mapping the pixel values ​​of each RGB channel from [0,255] to the [-1,1] interval to obtain standardized image data. Then, a shared feature extraction encoder based on the multi-scale residual network Res2Net is used to extract multi-scale features from the standardized image data to obtain a deep feature tensor, such as... Figure 4 As shown, the aliased shared feature tensor in the feature tensor is then separated into independent channel feature streams corresponding to two single bands, 420nm and 540nm, using a dual-branch decoder. Finally, the output layer submodule maps each independent channel feature stream to the RGB three channels, thereby generating predicted single-band images corresponding to 420nm and 540nm respectively. , .

[0032] In this embodiment, the discriminator adopts the PatchGAN architecture; the step of using each discriminator to output the discrimination result of the predicted single-band image includes: using each discriminator to output the discrimination true and false score feature map corresponding to the predicted single-band image, and determining the mean of the discrimination true and false score feature map as the discrimination result corresponding to the predicted single-band image.

[0033] The discriminator comprises independent 420nm and 540nm discriminators, both employing the PatchGAN architecture. When each discriminator outputs its prediction result for a single-band image, the 420nm discriminator analyzes the predicted 420nm single-band image. The system performs discrimination and outputs the corresponding true / false score feature maps. The mean of these true / false score feature maps is used as the discrimination result for the 420nm predicted single-band image. The specific formula is as follows: ; in, For the 420nm discriminator to predict 420nm single-band images The feature map for distinguishing between true and false scores. The mean of the feature map.

[0034] Similarly, the 540nm discriminator predicts 540nm single-band images. The system performs discrimination and outputs the corresponding true / false score feature maps. The mean of these true / false score feature maps is used as the discrimination result for the 540nm predicted single-band image. The discrimination result output by the discriminator is used to provide a realism gradient for the spectral decoupling generator, forcing the generated predicted single-band image distribution to converge with the real single-band image distribution.

[0035] In this embodiment, generating the loss value of the spectral decoupling generator based on the discrimination result includes: determining the authenticity loss value of the spectral decoupling generator based on the discrimination result of each of the predicted single-band images; and determining the sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value as the loss value of the spectral decoupling generator.

[0036] Based on the discrimination results of the predicted single-band images at 420nm and 540nm, the adversarial discrimination loss for each band is calculated. The sum of the adversarial discrimination losses for the two bands is then determined as the authenticity loss value of the spectral decoupling generator. This authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value are then weighted and summed according to preset weight coefficients. Finally, the weighted sum is determined as the total loss value of the spectral decoupling generator. The details are as follows: ; in, , , , These are the realism loss value and the physical model loss value of the spectral decoupling generator, respectively. Feature matching loss value and identity loss value The corresponding weighting coefficients.

[0037] Multi-dimensional loss constraints enable the generator to output single-band images that visually resemble real single-band images, while ensuring the physical rationality and feature fidelity of spectral separation, effectively improving the clinical diagnostic applicability of the separated images.

[0038] In this embodiment, determining the sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value as the loss value of the spectral decoupling generator includes: weighting and fusing each predicted single-band image according to a preset spectral mixing coefficient; calculating the difference between the fusion result and the corresponding historical mixed illumination image to obtain the physical model loss value of the spectral decoupling generator; determining the feature difference between each predicted single-band image and the corresponding real single-band image to obtain the feature matching loss value of the spectral decoupling generator; inputting the real single-band image into the spectral decoupling generator to obtain the corresponding reconstructed single-band image; calculating the difference between the reconstructed single-band image and the corresponding real single-band image to obtain the identity loss value of the spectral decoupling generator; and determining the sum of the authenticity loss value, the physical model loss value, the feature matching loss value, and the identity loss value as the loss value of the spectral decoupling generator.

[0039] The formula for obtaining the physical model loss value of the spectral decoupling generator is: ; in, , Let be the spectral mixing coefficient, where In other words, the 420nm predicted single-band image is generated according to the preset spectral mixing coefficients. Compared with the 540nm predicted single-band image Perform weighted fusion and compare the weighted fusion result with the corresponding historical mixed illumination image. L1 norm difference calculations are performed to obtain the physical model loss value of the spectral decoupling generator.

[0040] The formula for obtaining the feature matching loss value of the spectral decoupling generator is: ; in, This refers to the feature extraction part of a specified layer in the training network model, that is, extracting each predicted single-band image using the feature extraction part of the specified layer in the training network model. , Corresponding real single-band image , The feature information is used to determine the L2 norm feature difference between each predicted single-band image and the corresponding real single-band image, so as to obtain the feature matching loss value of the spectral decoupling generator.

[0041] The formula for obtaining the identity loss value of the spectral decoupling generator is: ; in, Indicates will The reconstructed single-band image output when inputting the spectral decoupling generator. Indicates will The reconstructed single-band image output when inputting into the spectral decoupling generator; that is, inputting the 420nm real single-band image and the 540nm real single-band image into the spectral decoupling generator respectively to obtain the reconstructed single-band image of the corresponding band. The L1 norm difference between the reconstructed single-band image and the corresponding real single-band image is calculated to obtain the identity loss value of the spectral decoupling generator.

[0042] Model Constraints The iterative spectral decoupling generator G and the discriminator are forced to work through gradient directional propagation. / The training parameters make the loss function... The Pareto optimal equilibrium point is reached, thus completing the end-to-end training of the adversarial network.

[0043] Step S13: Enhance and optimize the vascular features in each of the current separated images based on the corresponding bands to obtain the target separated image corresponding to each single band.

[0044] In this embodiment, the step of enhancing and optimizing the vascular features in each of the current separated images based on the bands corresponding to each of the current separated images to obtain the target separated image corresponding to each single band includes: performing surface microvascular enhancement processing on the current separated image corresponding to the first single narrow band to obtain the first target separated image corresponding to the first single narrow band; and enhancing the contrast of the vascular contours in the current separated image corresponding to the second single narrow band to obtain the second target separated image corresponding to the second single narrow band.

[0045] The current separated image corresponding to the first single narrow band with a center wavelength of 420nm. Microvascular morphology enhancement or edge enhancement algorithms based on bilateral filtering are used to enhance the surface microvessels, fully highlighting the capillary network of the gastrointestinal mucosa. This yields the first target separation image corresponding to the first narrow band, allowing doctors to clearly observe the morphology and distribution of surface microvessels, significantly improving the detection capability of minute lesions. The current separation image corresponds to the second narrow band with a center wavelength of 540nm. The adaptive contrast stretching or CLAHE contrast stretching algorithm is used to optimize the image processing, which enhances the contour clarity and overall contrast of medium-depth blood vessels. This results in a second target separation image corresponding to the second single narrow band, which can clearly present the intermediate layer vascular structure and avoid weak signals being overwhelmed by strong signals. The two band-specific enhancement processing work together to retain the diagnostic advantages of each band while eliminating spectral aliasing interference, significantly improving the clinical diagnostic accuracy and observation convenience of endoscopic images.

[0046] The beneficial effects of this application are as follows: This application obtains the current original image acquired by an endoscope; wherein, the current original image is a mixed illumination image acquired by the endoscope using dual narrowband light to simultaneously illuminate the target area; the current original image is subjected to spectral decoupling processing by the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same location, the triplet data including historical mixed illumination images and each real single band image, the real single band image being the original illumination image acquired by the endoscope using the corresponding single narrowband light to individually illuminate the target area; the vascular features in each of the current separated images are enhanced and optimized based on the band corresponding to each of the current separated images to obtain the target separated image corresponding to each single band. Therefore, this application, by acquiring mixed illumination images of the target area simultaneously illuminated by dual narrowband lights using an endoscope, can complete the original image acquisition without reducing the imaging frame rate or causing image registration deviation. Relying on the spectral decoupling generator in the target adversarial generative network model trained with triplet data from the same location, spectral decoupling processing is performed on the mixed illumination image. This physically decouples the mixed image into independent single-band images, effectively avoiding signal interference and information overload between bands. Furthermore, based on the imaging characteristics corresponding to each single band, targeted enhancement and optimization processing is performed on the vascular features of the separated images, which can significantly improve the display clarity and feature recognition of vascular structures at different depths. This does not require time-division acquisition or simple tone mapping adjustment. While ensuring imaging smoothness, it achieves accurate separation of narrowband light information and improved visualization effects, providing more reliable image support for clinical observation and diagnosis.

[0047] See Figure 5 As shown in the figure, this application discloses an endoscopic narrow-band image separation device, comprising: Image acquisition module 11 is used to acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area; Image separation module 12 is used to perform spectral decoupling processing on the current original image using the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same part, the triplet data includes historical mixed illumination images and each real single band image, the real single band image is the original illumination image collected by the endoscope using the corresponding single narrowband light to illuminate the target part alone; The enhancement and optimization module 13 is used to perform enhancement and optimization processing on the vascular features in each of the current separated images based on the bands corresponding to each of the current separated images, so as to obtain the target separated image corresponding to each single band.

[0048] Furthermore, embodiments of this application also provide an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the endoscopic narrow-band image separation method performed by the electronic device disclosed in any of the foregoing embodiments.

[0050] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0051] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0052] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0053] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the endoscopic narrowband image separation method executed by the electronic device as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0054] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned endoscopic narrow-band image separation method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0055] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0056] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the art.

[0057] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The present invention provides a detailed description of an endoscopic narrow-band image separation method, apparatus, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for separating narrow-band endoscopic images, characterized in that, include: Acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area; The current original image is spectrally decoupled using the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band. The target adversarial generative network model is trained using triplet data paired with the same part. The triplet data includes historical mixed illumination images and real single band images. The real single band images are the original illumination images collected by the endoscope when the target part is illuminated by the corresponding single narrowband light. The vascular features in each of the current separated images are enhanced and optimized based on the bands corresponding to each current separated image to obtain the target separated image corresponding to each single band.

2. The endoscopic narrow-band image separation method according to claim 1, characterized in that, The creation process of the target adversarial generative network model includes: Construct a current adversarial generative network model that includes a spectral decoupling generator, a discriminator for each single band, and a model constraint; Triple data collected from the same location and paired together were used as training data. The training data is input into the current adversarial generative network model; The spectral decoupling generator in the current adversarial generative network model generates each predicted single-band image, and each discriminator outputs the discrimination result of the predicted single-band image. Based on the discrimination result, the loss value of the spectral decoupling generator is generated, and the model constraint is used to update the parameters of the spectral decoupling generator and the discriminator corresponding to each single band based on the loss value, so as to obtain a new current adversarial generative network model. The process then jumps back to the step of inputting the training data into the current adversarial generative network model until the preset stopping condition is met, and the new current adversarial generative network model obtained after parameter update is determined as the target adversarial generative network model.

3. The endoscopic narrow-band image separation method according to claim 2, characterized in that, The spectral decoupling generator includes a shared feature extraction encoder built on a multi-scale residual network and a dual-branch decoder; the generation of each predicted single-band image using the spectral decoupling generator in the current adversarial generative network model includes: The historical mixed lighting images in the training data are preprocessed to obtain standardized image data; The shared feature extraction encoder is used to extract multi-scale information from the standardized image data to obtain a feature tensor; The dual-branch decoder is used to separate the aliased shared feature tensors in the feature tensor into independent channel feature streams corresponding to each single band; The feature streams of each channel are mapped to the RGB channels to obtain the predicted single-band images corresponding to each single band.

4. The endoscopic narrow-band image separation method according to claim 2, characterized in that, The discriminator adopts the PatchGAN architecture; The step of using each discriminator to output the discrimination result of the predicted single-band image includes: Each discriminator outputs a true / false score feature map corresponding to the predicted single-band image, and the mean of the true / false score feature maps is determined as the discrimination result corresponding to the predicted single-band image.

5. The endoscopic narrow-band image separation method according to claim 2, characterized in that, The loss value generated based on the discrimination result for the spectral decoupling generator includes: The authenticity loss value of the spectral decoupling generator is determined based on the discrimination results of each predicted single-band image. The sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value is determined as the loss value of the spectral decoupling generator.

6. The endoscopic narrow-band image separation method according to claim 5, characterized in that, The step of determining the loss value of the spectral decoupling generator as the sum of the authenticity loss value, the physical model loss value of the spectral decoupling generator, the feature matching loss value, and the identity loss value includes: The predicted single-band images are weighted and fused according to the preset spectral mixing coefficients. The difference between the fusion result and the corresponding historical mixed illumination image is calculated to obtain the physical model loss value of the spectral decoupling generator. The feature differences between each predicted single-band image and the corresponding real single-band image are determined to obtain the feature matching loss value of the spectral decoupling generator. The real single-band image is input into the spectral decoupling generator to obtain the corresponding reconstructed single-band image. The difference between the reconstructed single-band image and the corresponding real single-band image is calculated to obtain the identity loss value of the spectral decoupling generator. The sum of the authenticity loss value, the physical model loss value, the feature matching loss value, and the identity loss value is determined as the loss value of the spectral decoupling generator.

7. The endoscopic narrow-band image separation method according to any one of claims 1 to 6, characterized in that, The dual narrowband includes a first single narrowband with a center wavelength of 420nm and a second single narrowband with a center wavelength of 540nm; the enhancement and optimization processing of vascular features in each of the current separated images based on the corresponding bands to obtain the target separated image corresponding to each single band includes: The current separated image corresponding to the first single narrow band is subjected to surface microvascular enhancement processing to obtain the first target separated image corresponding to the first single narrow band; Enhance the contrast of blood vessel contours in the current separated image corresponding to the second single narrow band to obtain the second target separated image corresponding to the second single narrow band.

8. An endoscopic narrow-band image separation device, characterized in that, include: The image acquisition module is used to acquire the current raw image captured by the endoscope; wherein, the current raw image is a mixed illumination image captured by the endoscope using dual narrow-band light to simultaneously illuminate the target area; The image separation module is used to perform spectral decoupling processing on the current original image using the spectral decoupling generator in the target adversarial generative network model to obtain the current separated image corresponding to each single band; wherein, the target adversarial generative network model is trained using triplet data paired with the same part, the triplet data includes historical mixed illumination images and each real single band image, the real single band image is the original illumination image collected by the endoscope using the corresponding single narrowband light to illuminate the target part alone; The enhancement and optimization module is used to perform enhancement and optimization processing on the vascular features in each of the current separated images based on the bands corresponding to each current separated image, so as to obtain the target separated image corresponding to each single band.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the endoscopic narrow-band image separation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the endoscopic narrow-band image separation method as described in any one of claims 1 to 7.

Images