Early cancerization risk early warning method and device based on multi-band image fusion

Abstract

The invention relates to an early cancerization risk early warning method and device based on multiband image fusion. The method comprises the following steps: respectively acquiring N endoscopic images shot at the same time at the same position under different band light sources; for a light source of any wave band, performing image fusion on the N endoscopic images of the light source to obtain a fusion representation image; registering and preprocessing the fused representation images corresponding to all the waveband light sources to obtain processed fused representation images corresponding to all the waveband light sources; inputting the processed fusion representation images corresponding to all the waveband light sources into a pre-trained deep learning model, and obtaining an output result of the deep learning model, namely a multi-waveband fusion image; and carrying out visual display on the multi-band fusion image. And obtaining an early canceration risk analysis result for the multi-band fusion image, and performing early warning when an early canceration risk exists. According to the invention, the accuracy of early canceration risk early warning results can be improved based on multiband endoscope image fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an early cancer risk warning method and device based on multi-band image fusion. Background Art

[0002] Endoscopic examination is the core means for early cancer screening of the digestive tract. By combining high-resolution imaging, optical staining technology and artificial intelligence-assisted analysis, it can accurately capture the characteristics of tiny mucosal lesions.

[0003] The problems existing in the current endoscopic examination are that the contrast of the existing endoscopic images is relatively low, which is likely to cause the changes of early cancer (such as tiny) blood vessels to be ignored, thereby delaying the treatment progress of patients.

[0004] Multi-band image fusion can effectively improve the image contrast by integrating the image information obtained from different spectra or sensors, especially showing remarkable performance in complex scenes or low-light conditions.

[0005] How to apply the multi-band image fusion technology to endoscopic examination to better capture the cancer details in endoscopic images is an urgent problem to be solved in the current market. Summary of the Invention

[0006] The purpose of the present invention is to at least solve one of the deficiencies of the prior art, and provide an early cancer risk warning method and device based on multi-band image fusion.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions:

[0008] Specifically, an early cancer risk warning method based on multi-band image fusion is proposed, including the following:

[0009] Step 110: Obtain N endoscopic images taken at the same position and the same time period under light sources of different bands respectively;

[0010] Step 120: For any light source band, perform image fusion on its N endoscopic images to obtain a fused representation image of this light source band, and finally obtain the fused representation images corresponding to all light source bands;

[0011] Step 130: Perform registration and preprocessing on the fused representation images corresponding to all light source bands to obtain the processed fused representation images corresponding to all light source bands;

[0012] Step 140: Input the processed fused representation images corresponding to all light source bands into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, a multi-band fused image;

[0013] Step 150: Visually display the multi-band fusion image;

[0014] Step 160: Obtain the early cancer risk analysis result for the multi-band fusion image, and issue a warning when there is an early cancer risk.

[0015] Furthermore, specifically, for any band light source, image fusion is performed on its N endoscopic images to obtain the fusion characterization image of this band light source, including,

[0016] Construct an endoscopic image sequence of the current band light source in the order of shooting time before and after. In the endoscopic image sequence, define matrix_i(i) as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel at the j-th row and k-th column in matrix_i(i);

[0017] For any pixel point matrix_i(i,ji,ki), calculate its relative two-dimensional neighborhood pixel point's dominant feature performance degree adv(i,ji,ki). The calculation formula is as follows,

[0018] ;

[0019] Where, represents the average value of the two-dimensional neighborhood pixel points of the pixel point matrix_i(i,ji,ki), represents the maximum value among the two-dimensional neighborhood pixel points of the pixel point matrix_i(i,ji,ki), represents the minimum value among the two-dimensional neighborhood pixel points of the pixel point matrix_i(i,ji,ki);

[0020] The process of image fusion is as follows,

[0021] Step 210: Perform preliminary fusion on the N endoscopic images in the way of equal-weight weighted average to obtain a preliminary fusion image;

[0022] Step 220: Initialize i and set i = 1;

[0023] Step 230: Calculate the dominant feature performance region of matrix_i(i) based on the dominant feature performance degree adv(i,ji,ki);

[0024] Step 240: Replace the region of the same pixel point position in the preliminary fusion image with the dominant feature performance region of matrix_i(i) to obtain an updated preliminary fusion image;

[0025] Step 250: Determine whether i is equal to N. If it is equal to N, end the loop and output the updated preliminary fusion image as the fusion characterization image. If it is not equal to N, increment the value of i by 1 and go back to Step 230 to continue running.

[0026] Further, specifically, calculate the dominant feature manifestation region of matrix_i(i) based on the dominant feature manifestation degree adv(i, ji, ki), including

[0027] For matrix_1(1), traverse matrix_1(1), find the two pixel points with the largest dominant feature manifestation degree and denote them as pix_1 and pix_2. Connect pix_1 and pix_2 to obtain the straight line pix_1,2. In matrix_1(1), find the pixel point with the largest dominant feature manifestation degree that is not on the straight line pix_1,2 and denote it as pix_3. Then, the region formed by connecting pix_1, pix_2, and pix_3 pairwise is the dominant feature manifestation region of matrix_1(1).

[0028] For matrix_i(i), calculate the average pixel value of the pixel points in the dominant feature manifestation region of matrix_i - 1(i - 1) and denote it as , then traverse matrix_i(i), find any two pixel points whose dominant feature manifestation degree is greater than and denote them as pix_4 and pix_5. Connect pix_4 and pix_5 to obtain the straight line pix_4,5. In matrix_i(i), find the pixel point with the largest dominant feature manifestation degree that is not on the straight line pix_4,5 and denote it as pix_6. Then, the region formed by connecting pix_4, pix_5, and pix_6 pairwise is the dominant feature manifestation region of matrix_i(i).

[0029] Further, specifically, in Step 130, register and preprocess the fusion characterization images corresponding to all band light sources to obtain the processed fusion characterization images corresponding to all band light sources, including

[0030] Register the fusion characterization images corresponding to all band light sources through the Elastic Fusion or DeepReg network, and unify the resolutions of the fusion characterization images corresponding to all band light sources to obtain the registered fusion characterization images corresponding to all band light sources;

[0031] Independently perform Z-score normalization on the registered fusion characterization images corresponding to all band light sources to eliminate the brightness difference and obtain the processed fusion characterization images corresponding to all band light sources.

[0032] Further, specifically, visualizing the multi-band fusion image includes:

[0033] When the deep learning model outputs the results, it automatically marks abnormal blood vessels and mucosal color changes, and generates a high-contrast image for the marked positions for visual display.

[0034] Further, specifically, in the step 160,

[0035] The method for obtaining the early cancer risk analysis result for the multi-band fusion image includes obtaining the analysis result input by a doctor or the analysis result obtained by automatic analysis based on AI.

[0036] The present invention also provides an early cancer risk warning device based on multi-band image fusion, including:

[0037] A data acquisition module, configured to acquire N endoscopic images taken at the same position and the same time period under different band light sources respectively;

[0038] A fusion characterization image calculation module, configured to perform image fusion on the N endoscopic images for any band light source to obtain the fusion characterization image of this band light source, and finally obtain the fusion characterization images corresponding to all band light sources;

[0039] A preprocessing module, configured to perform registration and preprocessing on the fusion characterization images corresponding to all band light sources to obtain the processed fusion characterization images corresponding to all band light sources;

[0040] A multi-band fusion image calculation module, configured to input the processed fusion characterization images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, the multi-band fusion image;

[0041] A visual display module, configured to visually display the multi-band fusion image;

[0042] A risk warning module, configured to obtain the early cancer risk analysis result for the multi-band fusion image, and give a warning when there is an early cancer risk.

[0043] The present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of the early cancer risk warning method based on multi-band image fusion as described above.

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention proposes an early cancer risk warning method and device based on multi-band image fusion. First, multiple images are collected for different band light sources respectively, and the fusion representation images corresponding to all band light sources are obtained by fusing the images in the regions with the best performance of multiple images. Then, after preprocessing the fusion representation images corresponding to all band light sources, they are input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are carried out based on the output result. On the one hand, for images of different band light sources, higher-quality fusion representation images can be obtained through multi-image fusion to accelerate the analysis of the subsequent deep learning model, and to a certain extent, improve the accuracy of the analysis results of the subsequent deep learning model. On the other hand, the fusion image obtained by fusing the endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous region, facilitating subsequent risk analysis and warning.

[0046] (2) The present invention estimates the region with the best performance of dominant features based on the idea of the pixel point with the maximum degree of dominant feature manifestation. For non-first images in the endoscopic image sequence, the region with the best performance of dominant features is estimated based on the idea of pixel points with a larger average pixel value of pixel points in the region with the best performance of dominant features compared to the previous endoscopic image, ensuring that the entire process is in an optimized iteration to make the image quality of the finally obtained fusion representation image of the current band light source the best. Description of the Drawings

[0047] By elaborating on the embodiments shown in combination with the drawings, the above and other features of the present disclosure will become more obvious. The same reference numerals in the drawings of the present disclosure represent the same or similar sampling monitoring points. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0048] Figure 1 Shown is the flowchart of the early cancer risk warning method based on multi-band image fusion of the present invention;

[0049] Figure 2 Shown is the flowchart of fusing N endoscopic images of an arbitrary band light source to obtain the fusion representation image of the band light source;

[0050] Figure 3 Shown is the structural diagram of the early cancer risk warning device based on multi-band image fusion of the present invention. Detailed Embodiments

[0051] The concept, specific structure, and technical effects of the present invention will be clearly and completely described below in conjunction with embodiments and the accompanying drawings to fully understand the purpose, solution, and effects of the present invention. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0052] Example 1. Referring to Figure 1 , the present invention proposes an early cancer risk warning method based on multi-band image fusion, including the following:

[0053] Step 110: Obtain N endoscopic images taken at the same position and the same time period (the shooting interval in this same time period is very short, and the changes in the shooting object can be ignored) under different band light sources respectively.

[0054] Step 120: For any band light source, perform image fusion on its N endoscopic images to obtain the fusion characterization image of this band light source, and finally obtain the fusion characterization images corresponding to all band light sources.

[0055] Step 130: Perform registration and preprocessing on the fusion characterization images corresponding to all band light sources to obtain the processed fusion characterization images corresponding to all band light sources.

[0056] Step 140: Input the processed fusion characterization images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, the multi-band fusion image.

[0057] Step 150: Visualize the multi-band fusion image.

[0058] Step 160: Obtain the early cancer risk analysis result for the multi-band fusion image, and give a warning when there is an early cancer risk.

[0059] In this Example 1, first, multiple images are collected respectively for different band light sources, and image fusion is performed on the dominant regions of the multiple images to obtain the fusion characterization images corresponding to all band light sources; then, the fusion characterization images corresponding to all band light sources are preprocessed and input into a pre-established deep learning model for multi-band image fusion to obtain the output result, and subsequent early cancer risk judgment and warning are carried out according to the output result. On the one hand, for images of different band light sources, higher-quality fusion characterization images can be obtained through multi-image acquisition and fusion, which can speed up the analysis of the subsequent deep learning model and improve the accuracy of the analysis result of the subsequent deep learning model to a certain extent; on the other hand, the fusion image obtained by fusing the endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous region, which is convenient for subsequent risk analysis and warning.

[0060] Reference Figure 2 , as a preferred embodiment of the present invention, specifically, for any band light source, N endoscopic images thereof are subjected to image fusion to obtain a fusion characterization image of the band light source, including,

[0061] Construct an endoscopic image sequence of the current band light source in the order of shooting time before and after. In the endoscopic image sequence, define matrix_i(i) as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel at the j-th row and k-th column in matrix_i(i);

[0062] For any pixel matrix_i(i,ji,ki), calculate its dominant feature performance degree adv(i,ji,ki) relative to its two-dimensional neighborhood pixel points. The calculation formula is as follows,

[0063] ;

[0064] Wherein, represents the average value of the two-dimensional neighborhood pixel points of the pixel matrix_i(i,ji,ki), represents the maximum value among the two-dimensional neighborhood pixel points of the pixel matrix_i(i,ji,ki), represents the minimum value among the two-dimensional neighborhood pixel points of the pixel matrix_i(i,ji,ki). Here, considering that the image matrix is a two-dimensional matrix, its two-dimensional neighborhood is the 8-neighborhood;

[0065] The process of image fusion is as follows,

[0066] Step 210: Perform preliminary fusion on N endoscopic images in the way of equal-weight weighted average to obtain a preliminary fusion image;

[0067] Step 220: Initialize i and set i = 1;

[0068] Step 230: Calculate the dominant feature performance region of matrix_i(i) based on the dominant feature performance degree adv(i,ji,ki);

[0069] Step 240: Replace the region at the same pixel position in the preliminary fusion image with the dominant feature performance region of matrix_i(i) to obtain an updated preliminary fusion image;

[0070] Step 250: Determine whether i is equal to N. If it is equal to N, end the loop and output the updated preliminary fusion image as the fusion characterization image. If it is not equal to N, increase the value of i by 1 and go back to step 230 to continue running.

[0071] In this preferred embodiment, for the endoscopic images in the endoscopic image sequence of the current band light source, first, the dominant feature manifestation degree of each pixel point in each endoscopic image is determined by comparing the 8-neighborhood pixel points with each other. Then, based on the dominant feature manifestation degree, the dominant feature manifestation region of the image is calculated. The dominant feature manifestation regions of all the endoscopic images in the endoscopic image sequence are successively used to replace the corresponding pixel point positions in the preliminary fusion image (i.e., the image obtained by simple equal-weight weighted averaging). In this way, the more expressive pixel points in each endoscopic image can be retained, and more image details can be retained compared with the preliminary fusion image. On the one hand, the random noise of the image is reduced. On the other hand, when the subsequent deep learning model performs image fusion, due to the more obvious details, the fusion efficiency can be accelerated and the fusion accuracy can be improved.

[0072] As a preferred embodiment of the present invention, specifically, calculating the dominant feature manifestation region of matrix_i(i) based on the dominant feature manifestation degree adv(i,ji,ki) includes:

[0073] For matrix_1(1), traverse matrix_1(1), find the two pixel points with the largest dominant feature manifestation degree and denote them as pix_1 and pix_2. Connect pix_1 and pix_2 to obtain the straight line pix_1,2. In matrix_1(1), find the pixel point with the largest dominant feature manifestation degree that is not on the straight line pix_1,2 and denote it as pix_3. Then, the region formed by connecting pix_1, pix_2, and pix_3 pairwise is the dominant feature manifestation region of matrix_1(1);

[0074] For matrix_i(i), calculate the average pixel value of the pixel points in the dominant feature manifestation region of matrix_i-1(i-1) and denote it as , and then traverse matrix_i(i), find any two pixel points with a dominant feature manifestation degree greater than and denote them as pix_4 and pix_5. Connect pix_4 and pix_5 to obtain the straight line pix_4,5. In matrix_i(i), find the pixel point with the largest dominant feature manifestation degree that is not on the straight line pix_4,5 and denote it as pix_6. Then, the region formed by connecting pix_4, pix_5, and pix_6 pairwise is the dominant feature manifestation region of matrix_i(i).

[0075] In this preferred embodiment, through the above method, for the first image in the endoscopic image sequence, the region with the most prominent feature manifestation is estimated based on the pixel point with the maximum degree of prominent feature manifestation. For non-first images in the endoscopic image sequence, the region with the most prominent feature manifestation is estimated based on the pixel points whose average pixel value is greater than that of the pixel points in the region with the most prominent feature manifestation in the previous endoscopic image. This ensures that the entire process is in an optimized iteration, resulting in the best image quality for the fused representation image of the current band light source.

[0076] As a preferred embodiment of the present invention, specifically, in step 130, registering and preprocessing the fused representation images corresponding to all band light sources to obtain the processed fused representation images corresponding to all band light sources includes:

[0077] Registering the fused representation images corresponding to all band light sources through Elastic Fusion or DeepReg network, and unifying the resolution of the fused representation images corresponding to all band light sources to obtain the registered fused representation images corresponding to all band light sources;

[0078] Independently performing Z-score normalization on the registered fused representation images corresponding to all band light sources to eliminate the brightness difference and obtain the processed fused representation images corresponding to all band light sources.

[0079] In this preferred embodiment, considering the registration and normalization issues in the analysis of deep learning networks, the fused representation images corresponding to all band light sources are processed through the above method to ensure the accuracy and stability of the subsequent deep learning network processing.

[0080] As a preferred embodiment of the present invention, specifically, visualizing the multi-band fused image includes:

[0081] When the deep learning model outputs the results, it automatically marks abnormal blood vessels and mucosal color changes, and generates high-contrast images for the marked positions for visual display.

[0082] In this preferred embodiment, AI automatically marks suspicious lesions (such as abnormal blood vessels and mucosal color changes) and generates high-contrast images similar to "staining" to assist doctors in making quick judgments.

[0083] As a preferred embodiment of the present invention, specifically, in step 160,

[0084] The method for obtaining the early cancer risk analysis result for the multi-band fused image includes obtaining the analysis result input by the doctor or the analysis result obtained based on AI automatic analysis.

[0085] In this preferred embodiment, considering subsequent application issues, the subsequent early cancer risk analysis results can be the results actively input by a doctor or the results obtained by calling an AI model for analysis.

[0086] In addition, as a preferred embodiment, the pre-construction process of the deep learning network is as follows.

[0087] I. Model architecture design

[0088] Decomposition and reconstruction filter adaptive module

[0089] Deep stacked convolutional neural network (DSCNN): By stacking basic convolutional units, it automatically learns the decomposition filters of multi-band images, replacing traditional manually designed multi-scale transforms (such as wavelets, NSCT). DSCNN decomposes multi-band images into high-frequency (detail) and low-frequency (contour) components, realizing end-to-end adaptive optimization of filter parameters.

[0090] Residual connection and multi-layer feature extraction: Introduce residual modules during the decomposition process to retain the global information of the original image and avoid the problem of gradient disappearance in deep networks.

[0091] Fusion rule adaptive module

[0092] Deep gated convolutional neural network (DGCNN): Dynamically adjusts the weights of features in different bands through a gating mechanism, replacing traditional fixed fusion rules (such as mean, maximum). DGCNN adaptively generates a fusion decision map according to the spatial distribution of the input features, highlighting key regions (such as lesion boundaries).

[0093] Hybrid attention mechanism: Combines channel attention (focusing on important features) and spatial attention (locating significant regions) to enhance the selective fusion of multi-band features. For example, in the fusion of infrared and visible light, the attention module can strengthen edge details.

[0094] Colorization and super-resolution module

[0095] Adaptive colorization network (DCNN): If the original multi-band image is grayscale, a pseudo-color image can be generated through a lightweight CNN (such as a U-Net variant) to improve the human eye observation effect.

[0096] Multi-contrast super-resolution fusion: For low-resolution bands (such as infrared), combined with the high-resolution features of visible light, the detail clarity of the fused image is improved through a super-resolution network (such as SRResNet).

[0097] II. Training strategy optimization

[0098] End-to-end adaptive training

[0099] Generative Adversarial Network (GAN): Construct a generator (fusion network) and a discriminator (quality assessment), and optimize the fusion result through adversarial training. The discriminator can be designed as a multi-task structure to simultaneously evaluate clarity, information content, and spectral fidelity.

[0100] Unsupervised loss function: Includes feature reconstruction loss (L1 / L2 norm), perceptual loss (similarity in the feature space of pre-trained CNN), and adversarial loss. For example, the PMGI model optimizes the fused image through gradient and intensity ratio maintenance loss.

[0101] Dataset construction and data augmentation

[0102] Multi-band paired data: Collect multi-band images of the same scene (such as white light, narrow-band light, infrared), and align pixel-level features through annotation tools.

[0103] Simulation data augmentation: Simulate the real endoscopic environment by adding noise, blur, or lighting changes to improve the generalization ability of the model.

[0104] The proposed deep learning network has the following advantages

[0105] Multi-band synchronous fusion

[0106] Supports inputs of more than two bands (such as white light + ultraviolet + infrared), extracts multi-scale features through parallel convolutional branches, and then dynamically fuses them through a gating mechanism.

[0107] Multi-task joint optimization

[0108] Integrates decomposition, fusion, super-resolution, and colorization tasks in a single model, reducing redundant calculations and improving real-time performance.

[0109] Hardware adaptation optimization

[0110] Adopts a lightweight network structure (such as MobileNet) or knowledge distillation technology to compress the model size to adapt to the computing power limitations of embedded devices (such as endoscopic terminals).

[0111] IV. Performance evaluation and application

[0112] Evaluation metrics

[0113] Objective metrics: Entropy value (information content), QAB / F index (fusion quality), SAM / ERGAS (spectral fidelity), and LPIPS (perceptual similarity).

[0114] Subjective evaluation: Judge the visibility of the lesion area and the naturalness of the image through doctor scoring.

[0115] In application, it can perform early tumor detection: fuse white light and fluorescence images to enhance the contrast of subtle lesions on the mucosal surface.

[0116] And vascular visualization: Combining infrared and visible light to highlight the vascular structure to guide minimally invasive surgery.

[0117] Referring to Figure 3 , Example 2, the present invention also proposes an early cancer risk warning device based on multi-band image fusion, including:

[0118] A data acquisition module for respectively acquiring N endoscopic images taken at the same position and the same time period under different band light sources;

[0119] A fusion characterization image calculation module for performing image fusion on the N endoscopic images for any band light source to obtain a fusion characterization image of the band light source, and finally obtaining fusion characterization images corresponding to all band light sources;

[0120] A preprocessing module for registering and preprocessing the fusion characterization images corresponding to all band light sources to obtain processed fusion characterization images corresponding to all band light sources;

[0121] A multi-band fusion image calculation module for inputting the processed fusion characterization images corresponding to all band light sources into a pre-trained deep learning model and obtaining the output result of the deep learning model, that is, a multi-band fusion image;

[0122] A visualization display module for visually displaying the multi-band fusion image;

[0123] A risk warning module for obtaining an early cancer risk analysis result for the multi-band fusion image and giving a warning when there is an early cancer risk.

[0124] In this Example 2, consistent with the early cancer risk warning method based on multi-band image fusion proposed by the present invention, first, multiple images are respectively collected for different band light sources, and image fusion is performed on the dominant regions of the multiple images to obtain fusion characterization images corresponding to all band light sources; then, the fusion characterization images corresponding to all band light sources are preprocessed and input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are performed according to the output result. On the one hand, for different band light source images, higher-quality fusion characterization images can be obtained through multi-image acquisition and fusion to accelerate the analysis of the subsequent deep learning model, and the accuracy of the analysis result of the subsequent deep learning model can be improved to a certain extent; on the other hand, the fusion image obtained by fusing the endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous region, facilitating subsequent risk analysis and warning.

[0125] Embodiment 3. The present invention further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the early cancer risk warning method based on multi-band image fusion are implemented.

[0126] In this Embodiment 3, which is consistent with the early cancer risk warning method based on multi-band image fusion proposed by the present invention, first, multiple images are respectively collected for different band light sources, and image fusion is performed on the dominant display areas of the multiple images to obtain a fused representation image corresponding to all band light sources; then, after preprocessing the fused representation images corresponding to all band light sources, they are input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are performed based on the output result. On the one hand, for different band light source images, higher-quality fused representation images can be obtained through multi-image acquisition and fusion, which can accelerate the analysis of the subsequent deep learning model and, to a certain extent, improve the accuracy of the analysis results of the subsequent deep learning model; on the other hand, the fused image obtained by fusing the endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous area, facilitating subsequent risk analysis and warning.

[0127] In addition, in each embodiment of the present invention, each functional module can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0128] If the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0129] Although the description of the present invention has been quite detailed and several of the described embodiments have been particularly described, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather it should be regarded as providing a broad interpretation of these claims in light of the prior art by reference to the appended claims, so as to effectively cover the intended scope of the present invention. In addition, the present invention has been described above in terms of embodiments foreseeable by the inventors for the purpose of providing a useful description, and non-substantive modifications to the present invention that are not currently foreseeable may still represent equivalent modifications of the present invention.

[0130] As mentioned above, these are only the preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. As long as the same means are used to achieve the technical effects of the present invention, they should fall within the protection scope of the present invention. Within the protection scope of the present invention, various different modifications and changes can be made to its technical solutions and / or embodiments.

Claims

1. An early cancer risk warning method based on multi - band image fusion, characterized in that, including the following: Step 110: Obtain N endoscopic images taken at the same position and the same time period under light sources of different bands respectively; Step 120: For any band light source, perform image fusion on its N endoscopic images to obtain a fused representation image of this band light source, and finally obtain the fused representation images corresponding to all band light sources; Step 130: Perform registration and preprocessing on the fused representation images corresponding to all band light sources to obtain the processed fused representation images corresponding to all band light sources; Step 140: Input the processed fused representation images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, a multi-band fused image; Step 150: Visualize the multi-band fused image; Step 160: Obtain the early cancer risk analysis result for the multi-band fused image, and give an alarm when there is an early cancer risk.

2. The early cancer risk warning method based on multi - band image fusion according to claim 1, characterized in that, Specifically, for any band light source, performing image fusion on its N endoscopic images to obtain a fused representation image of this band light source includes: Construct an endoscopic image sequence of the current band light source in the order of shooting time before and after. In the endoscopic image sequence, define matrix_i(i) as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel at the j-th row and the k-th column in matrix_i(i); For any pixel matrix_i(i,ji,ki), calculate its dominant feature performance degree adv(i,ji,ki) relative to the two-dimensional neighborhood pixels. The calculation formula is as follows: ； Among them, represents the average value of the two-dimensional neighborhood pixel points of the pixel point matrix_i(i, ji, ki), represents the maximum value among the two-dimensional neighborhood pixel points of the pixel point matrix_i(i, ji, ki), represents the minimum value among the two-dimensional neighborhood pixel points of the pixel point matrix_i(i, ji, ki); The process of image fusion is as follows: Step 210: Perform preliminary fusion on the N endoscopic images in the way of equal-weight weighted average to obtain a preliminary fusion image; Step 220: Initialize i, and let i = 1; Step 230: Calculate the dominant feature performance region of matrix_i(i) based on the dominant feature performance degree adv(i,ji,ki); Step 240: Replace the region of the same pixel position in the preliminary fusion image with the dominant feature performance region of matrix_i(i) to obtain an updated preliminary fusion image; Step 250: Judge whether i is equal to N. If i is equal to N, end the loop and output the updated preliminary fusion image as the fused representation image. If i is not equal to N, increase the value of i by 1 and go back to Step 230 to continue running.

3. The early cancer risk warning method based on multi - band image fusion according to claim 2, characterized in that, Specifically, calculating the dominant feature performance region of matrix_i(i) based on the dominant feature performance degree adv(i,ji,ki) includes: For matrix_1(1), traverse matrix_1(1) to find the two pixel points with the greatest degree of dominant feature expression, denoted as pix_1 and pix_2. Connect pix_1 and pix_2 to obtain the straight line pix_1,2. In matrix_1(1), find the pixel point with the greatest degree of dominant feature expression that is not on the straight line pix_1,2, denoted as pix_3. Then, the region formed by connecting pix_1, pix_2, and pix_3 pairwise is the dominant feature expression region of matrix_1(1). For matrix_i(i), calculate the average pixel value of the pixels in the dominant feature manifestation area of matrix_i-1(i-1), denoted as , then traverse matrix_i(i) to find any two pixels whose dominant feature manifestation degree is greater than . Denote them as pix_4 and pix_5, connect pix_4 and pix_5 to obtain the straight line pix_4,5. In matrix_i(i), find the pixel with the maximum dominant feature manifestation degree that is not on the straight line pix_4,5, denoted as pix_6. Then, the area formed by connecting pix_4, pix_5, and pix_6 pairwise is the dominant feature manifestation area of matrix_i(i).

4. The early cancer risk warning method based on multi - band image fusion according to claim 1, characterized in that, Specifically, in step 130, registering and preprocessing the fusion representation images corresponding to all band light sources to obtain the processed fusion representation images corresponding to all band light sources includes registering the fusion representation images corresponding to all band light sources through an Elastic Fusion or DeepReg network, and unifying the resolutions of the fusion representation images corresponding to all band light sources to obtain the registered fusion representation images corresponding to all band light sources; independently performing Z-score normalization on the registered fusion representation images corresponding to all band light sources to eliminate brightness differences and obtain the processed fusion representation images corresponding to all band light sources.

5. The early cancer risk warning method based on multi - band image fusion according to claim 1, characterized in that, Specifically, visualizing the multi-band fusion image includes when the deep learning model outputs the results, automatically marking abnormal blood vessels and mucosal color changes, and generating a high-contrast image for the marked positions for visual display.

6. The early cancer risk warning method based on multi - band image fusion according to claim 1, characterized in that, Specifically, in step 160 The ways to obtain the early cancer risk analysis results for the multi-band fusion image include obtaining the analysis results input by a doctor or the analysis results automatically analyzed based on AI.

7. An early cancer risk warning device based on multi - band image fusion, characterized in that, including a data acquisition module for respectively acquiring N endoscopic images taken at the same position and the same time under different band light sources; a fusion representation image calculation module for, for any band light source, performing image fusion on its N endoscopic images to obtain the fusion representation image of this band light source, and finally obtaining the fusion representation images corresponding to all band light sources; a preprocessing module for registering and preprocessing the fusion representation images corresponding to all band light sources to obtain the processed fusion representation images corresponding to all band light sources; a multi-band fusion image calculation module for inputting the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model, and obtaining the output result of the deep learning model, i.e., the multi-band fusion image; a visual display module for visually displaying the multi-band fusion image; a risk warning module for obtaining the early cancer risk analysis results for the multi-band fusion image and giving a warning when there is an early cancer risk.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images