Early cancer risk warning method and device based on multi-band image fusion

Abstract

The present invention relates to an early cancer risk warning method and device based on multi-band image fusion, comprising the following steps: obtaining N endoscopic images taken at the same location and time under different wavelength light sources; fusing the N endoscopic images for any wavelength light source to obtain a fused representation image; registering and preprocessing the fused representation images corresponding to all wavelength light sources to obtain processed fused representation images corresponding to all wavelength light sources; inputting the processed fused representation images corresponding to all wavelength light sources into a pre-trained deep learning model, and obtaining the output of the deep learning model, namely a multi-band fused image; visually displaying the multi-band fused image; obtaining an early cancer risk analysis result for the multi-band fused image, and issuing a warning when an early cancer risk exists. The present invention can improve the accuracy of early cancer risk warning results based on multi-band endoscopic image fusion.

Description

Technical Field

[0001] The present invention relates to the technical field related to image processing, and in particular to an early cancer risk warning method and device based on multi-band image fusion. Background Art

[0002] Endoscopic examination is the core means of screening for early gastrointestinal cancer. It can accurately capture the characteristics of subtle mucosal lesions by combining high-resolution imaging, optical staining technology and artificial intelligence-assisted analysis.

[0003] Problems with current endoscopic testing include the low contrast of existing endoscopic images, which can easily lead to the neglect of early cancerous (such as tiny) vascular changes, thereby slowing down the patient's treatment progress.

[0004] Multi-band image fusion can effectively improve image contrast by integrating image information obtained from different spectra or sensors, especially in complex scenes or low-light conditions.

[0005] How to apply multi-band image fusion technology to endoscopic detection to better capture the details of cancer in endoscopic images is an issue that urgently needs to be addressed in today's market. Summary of the Invention

[0006] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a method and device for early cancer risk warning based on multi-band image fusion.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] Specifically, an early cancer risk warning method based on multi-band image fusion is proposed, including the following:

[0009] Step 110: Obtain N endoscopic images taken at the same position and during the same period of time under light sources of different wavelengths;

[0010] Step 120: For any wavelength light source, perform image fusion on N endoscopic images to obtain a fused representation image of the wavelength light source, and finally obtain fused representation images corresponding to all wavelength light sources;

[0011] Step 130: registering and preprocessing the fused representation images corresponding to all wavelength light sources to obtain processed fused representation images corresponding to all wavelength light sources;

[0012] Step 140: Input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, the multi-band fusion image;

[0013] Step 150: Visually display the multi-band fusion image;

[0014] Step 160: Obtain an early cancer risk analysis result for the multi-band fusion image, and issue an early warning when there is an early cancer risk.

[0015] Furthermore, specifically, for a light source of any wavelength band, N endoscopic images thereof are fused to obtain a fused representation image of the light source of the wavelength band, including:

[0016] Construct an endoscopic image sequence of the current band light source in the order of shooting time. In the endoscopic image sequence, matrix_i(i) is defined as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel point in the j-th row and k-th column in matrix_i(i);

[0017] For any pixel point matrix_i(i,ji,ki), calculate its dominant feature expression adv(i,ji,ki) relative to the two-dimensional neighboring pixel points. The calculation formula is as follows:

[0018] ;

[0019] in, Represents the average value of the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the maximum value among the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the minimum value among the two-dimensional neighborhood pixels of pixel point matrix_i(i,ji,ki);

[0020] The process of image fusion is as follows:

[0021] Step 210: Perform preliminary fusion of N endoscopic images in an equal-weighted weighted average manner to obtain a preliminary fused image;

[0022] Step 220: Initialize i, set i=1;

[0023] Step 230: Calculate the dominant feature expression area of matrix_i(i) based on the dominant feature expression degree adv(i,ji,ki);

[0024] Step 240: Replace the dominant feature expression region of matrix_i(i) with the region at the same pixel position in the preliminary fused image to obtain an updated preliminary fused image;

[0025] Step 250 , determine whether i is equal to N. If it is equal to N, end the loop and output the updated preliminary fusion image as the fusion representation image. If it is not equal to N, increase the value of i by 1 and go to step 230 to continue.

[0026] Further, specifically, the dominant feature expression area of matrix_i(i) is calculated based on the dominant feature expression degree adv(i,ji,ki), including:

[0027] For matrix_1(1), traverse matrix_1(1) and find the two pixels with the largest dominant feature expression, record them as pix_1 and pix_2, connect pix_1 and pix_2 to get the straight line pix_1,2, find the pixel with the largest dominant feature expression in matrix_1(1) that is not on the straight line pix_1,2, record it as pix_3, then the area formed by connecting pix_1, pix_2 and pix_3 is the dominant feature expression area of matrix_1(1);

[0028] For matrix_i(i), calculate the average pixel value of the pixel points in the dominant feature expression area of matrix_i-1(i-1) and record it as , then traverse matrix_i(i) and find out the dominant feature expression greater than Any two pixels are denoted as pix_4 and pix_5, and a straight line pix_4,5 is obtained by connecting pix_4 and pix_5. In matrix_i(i), find the pixel with the largest dominant feature expression that is not on the straight line pix_4,5 and denote it as pix_6. Then the area formed by connecting pix_4, pix_5 and pix_6 is the dominant feature expression area of matrix_i(i).

[0029] Furthermore, specifically, in step 130, the fused representation images corresponding to all wavelength light sources are registered and preprocessed to obtain processed fused representation images corresponding to all wavelength light sources, including:

[0030] The fused representation images corresponding to all band light sources are registered through Elastic Fusion or DeepReg network, and the resolution of the fused representation images corresponding to all band light sources is unified to obtain the fused representation images corresponding to all band light sources after registration;

[0031] The fused representation images corresponding to all band light sources after registration are independently Z-score normalized to eliminate brightness differences and obtain the processed fused representation images corresponding to all band light sources.

[0032] Further, specifically, visually displaying the multi-band fusion image includes:

[0033] When outputting results, the deep learning model automatically marks abnormal blood vessels and mucosal color changes, and generates high-contrast images for the marked locations for visual display.

[0034] Further, specifically, in step 160,

[0035] Methods for obtaining early cancer risk analysis results for the multi-band fusion image include obtaining analysis results input by a doctor or analysis results obtained based on AI automatic analysis.

[0036] The present invention also proposes an early cancer risk warning device based on multi-band image fusion, comprising:

[0037] A data acquisition module is used to acquire N endoscopic images taken at the same position and at the same time under light sources of different wavelengths;

[0038] The fusion representation image calculation module is used to fuse N endoscopic images of any wavelength light source to obtain a fusion representation image of the wavelength light source, and finally obtain a fusion representation image corresponding to all wavelength light sources;

[0039] A preprocessing module is used to register and preprocess the fused representation images corresponding to all band light sources to obtain processed fused representation images corresponding to all band light sources;

[0040] A multi-band fusion image calculation module is used to input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model and obtain the output result of the deep learning model, namely the multi-band fusion image;

[0041] A visualization display module, used for visually displaying the multi-band fusion image;

[0042] The risk warning module is used to obtain the early cancer risk analysis result for the multi-band fusion image and issue a warning when there is an early cancer risk.

[0043] The present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the early cancer risk warning method based on multi-band image fusion.

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention proposes an early cancer risk warning method and device based on multi-band image fusion. First, multiple images are collected for different band light sources, and the image fusion is performed based on the dominant expression areas of the multiple images to obtain a fused representation image corresponding to all band light sources. Then, the fused representation images corresponding to all band light sources are pre-processed and input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are performed based on the output result. On the one hand, for images of different band light sources, a higher quality fused representation image can be obtained by fusing multiple collected images, which speeds up the analysis of the subsequent deep learning model and improves the accuracy of the analysis results of the subsequent deep learning model to a certain extent. On the other hand, the fused image obtained by fusing endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous area, facilitating subsequent risk analysis and warning.

[0046] (2) The present invention estimates the dominant feature expression area based on the idea of the pixel point with the largest dominant feature expression degree. For images other than the first image in the endoscopic image sequence, the dominant feature expression area is estimated based on the idea of the pixel point with a greater dominant feature expression degree than the average pixel value of the pixel points in the dominant feature expression area of the previous endoscopic image, ensuring that the entire process is in an optimized iteration, so that the image quality of the fusion representation image of the current band light source is the best. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar sampling monitoring points. Obviously, the drawings described below are only some embodiments of the present disclosure. Those skilled in the art can derive other drawings based on these drawings without inventive work. In the drawings:

[0048] Figure 1 Shown is a flow chart of the early cancer risk warning method based on multi-band image fusion of the present invention;

[0049] Figure 2 The figure shows a flow chart for performing image fusion on N endoscopic images of a light source of any wavelength band to obtain a fused representation image of the light source of that wavelength band;

[0050] Figure 3 Shown is a structural diagram of the early cancer risk warning device based on multi-band image fusion of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0052] Example 1, reference Figure 1 The present invention proposes an early cancer risk warning method based on multi-band image fusion, which includes the following:

[0053] Step 110: Acquire N endoscopic images taken at the same location and during the same time period (the interval between shots during the same time period is very short, and changes in the shot object are negligible) under light sources of different wavelengths.

[0054] Step 120: For any wavelength light source, perform image fusion on N endoscopic images to obtain a fused representation image of the wavelength light source, and finally obtain fused representation images corresponding to all wavelength light sources;

[0055] Step 130: registering and preprocessing the fused representation images corresponding to all wavelength light sources to obtain processed fused representation images corresponding to all wavelength light sources;

[0056] Step 140: Input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, the multi-band fusion image;

[0057] Step 150: Visually display the multi-band fusion image;

[0058] Step 160: Obtain an early cancer risk analysis result for the multi-band fusion image, and issue an early warning when there is an early cancer risk.

[0059] In this embodiment 1, multiple images are first collected for different light sources of different wavelengths, and image fusion is performed based on the dominant regions of the multiple images to obtain fused representation images corresponding to all light sources of all wavelengths; then, the fused representation images corresponding to all light sources of all wavelengths are preprocessed and input into a pre-established deep learning model for multi-band image fusion to obtain output results, and subsequent early cancer risk judgment and warning are performed based on the output results. On the one hand, for images of light sources of different wavelengths, the fusion of multiple collected images can obtain higher-quality fused representation images, accelerate the analysis of the subsequent deep learning model, and improve the accuracy of the analysis results of the subsequent deep learning model to a certain extent; on the other hand, the fused image obtained by fusing endoscopic images of all light sources of all wavelengths based on the deep learning model can highlight the characteristics of the cancerous area, facilitating subsequent risk analysis and warning.

[0060] Reference Figure 2 As a preferred embodiment of the present invention, specifically, for a light source of any wavelength band, N endoscopic images thereof are fused to obtain a fused representation image of the light source of the wavelength band, including:

[0061] Construct an endoscopic image sequence of the current band light source in the order of shooting time. In the endoscopic image sequence, matrix_i(i) is defined as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel point in the j-th row and k-th column in matrix_i(i);

[0062] For any pixel point matrix_i(i,ji,ki), calculate its dominant feature expression adv(i,ji,ki) relative to the two-dimensional neighboring pixel points. The calculation formula is as follows:

[0063] ;

[0064] in, Represents the average value of the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the maximum value among the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the minimum value of the two-dimensional neighborhood pixel points of pixel point matrix_i(i,ji,ki). Considering that the image matrix is a two-dimensional matrix, its two-dimensional neighborhood is an 8-neighborhood;

[0065] The process of image fusion is as follows:

[0066] Step 210: Perform preliminary fusion of N endoscopic images in an equal-weighted weighted average manner to obtain a preliminary fused image;

[0067] Step 220: Initialize i, set i=1;

[0068] Step 230: Calculate the dominant feature expression area of matrix_i(i) based on the dominant feature expression degree adv(i,ji,ki);

[0069] Step 240: Replace the dominant feature expression region of matrix_i(i) with the region at the same pixel position in the preliminary fused image to obtain an updated preliminary fused image;

[0070] Step 250 , determine whether i is equal to N. If it is equal to N, end the loop and output the updated preliminary fusion image as the fusion representation image. If it is not equal to N, increase the value of i by 1 and go to step 230 to continue.

[0071] In this preferred embodiment, for the endoscopic images in the endoscopic image sequence of the current band light source, the dominant feature expression degree of the pixel points in each endoscopic image is first determined by mutual comparison between the 8-neighborhood pixel points, and then the dominant feature expression area of the image is calculated based on the feature expression degree, and the dominant feature expression areas of all endoscopic images in the endoscopic image sequence are replaced in turn with the corresponding pixel positions in the preliminary fusion image (i.e., the image of a simple equal-weighted weighted average). In this way, the more expressive pixels in each endoscopic image can be retained, and more image details can be retained compared to the preliminary fusion image. On the one hand, the random noise of the image is reduced, and on the other hand, when the subsequent deep learning model performs image fusion, the details are more obvious, which can speed up the fusion efficiency and improve the fusion accuracy.

[0072] As a preferred embodiment of the present invention, specifically, the dominant feature expression area of matrix_i(i) is calculated based on the dominant feature expression degree adv(i,ji,ki), including:

[0073] For matrix_1(1), traverse matrix_1(1) and find the two pixels with the largest dominant feature expression, record them as pix_1 and pix_2, connect pix_1 and pix_2 to get the straight line pix_1,2, find the pixel with the largest dominant feature expression in matrix_1(1) that is not on the straight line pix_1,2, record it as pix_3, then the area formed by connecting pix_1, pix_2 and pix_3 is the dominant feature expression area of matrix_1(1);

[0074] For matrix_i(i), calculate the average pixel value of the pixel points in the dominant feature expression area of matrix_i-1(i-1) and record it as , then traverse matrix_i(i) and find out the dominant feature expression greater than Any two pixels are denoted as pix_4 and pix_5, and a straight line pix_4,5 is obtained by connecting pix_4 and pix_5. In matrix_i(i), find the pixel with the largest dominant feature expression that is not on the straight line pix_4,5 and denote it as pix_6. Then the area formed by connecting pix_4, pix_5 and pix_6 is the dominant feature expression area of matrix_i(i).

[0075] In this preferred embodiment, through the above-mentioned method, for the first image in the endoscopic image sequence, the dominant feature expression area is estimated with the idea of the pixel point with the largest dominant feature expression degree, and for the non-first image in the endoscopic image sequence, the dominant feature expression area is estimated with the idea of the pixel point with a dominant feature expression degree greater than the average pixel value of the pixel points in the dominant feature area of the previous endoscopic image, so as to ensure that the whole process is in an optimized iteration, so that the image quality of the fusion representation image of the current band light source obtained finally is the best.

[0076] As a preferred embodiment of the present invention, specifically, in step 130, the fused representation images corresponding to all wavelength light sources are registered and preprocessed to obtain the processed fused representation images corresponding to all wavelength light sources, including:

[0077] The fused representation images corresponding to all band light sources are registered through Elastic Fusion or DeepReg network, and the resolution of the fused representation images corresponding to all band light sources is unified to obtain the fused representation images corresponding to all band light sources after registration;

[0078] The fused representation images corresponding to all band light sources after registration are independently Z-score normalized to eliminate brightness differences and obtain the processed fused representation images corresponding to all band light sources.

[0079] In this preferred embodiment, considering the alignment and standardization issues of the deep learning network during analysis, the fused representation images corresponding to all band light sources are processed in the above manner to ensure that the subsequent deep learning network processing process is accurate and stable.

[0080] As a preferred embodiment of the present invention, specifically, visually displaying the multi-band fusion image includes:

[0081] When outputting results, the deep learning model automatically marks abnormal blood vessels and mucosal color changes, and generates high-contrast images for the marked locations for visual display.

[0082] In this preferred embodiment, AI automatically marks suspicious lesions (such as abnormal blood vessels, mucosal color changes) and generates high-contrast images similar to "staining" to assist doctors in making quick judgments.

[0083] As a preferred embodiment of the present invention, specifically, in step 160,

[0084] Methods for obtaining early cancer risk analysis results for the multi-band fusion image include obtaining analysis results input by a doctor or analysis results obtained based on AI automatic analysis.

[0085] In this preferred embodiment, considering subsequent application issues, the subsequent early cancer risk analysis results can come from the analysis results actively input by the doctor or the results obtained by calling the AI model for analysis.

[0086] In addition, as a preferred embodiment, the deep learning network pre-construction process is as follows:

[0087] 1. Model Architecture Design

[0088] Decomposition and reconstruction filter adaptive module

[0089] Deep Stacked Convolutional Neural Network (DSCNN): By stacking basic convolutional units, it automatically learns decomposition filters for multi-band images, replacing traditional hand-designed multi-scale transforms (such as wavelets and NSCT). DSCNN decomposes multi-band images into high-frequency (detail) and low-frequency (contour) components, enabling end-to-end adaptive optimization of filter parameters.

[0090] Residual connection and multi-layer feature extraction: Introducing the residual module in the decomposition process to retain the global information of the original image and avoid the gradient vanishing problem in deep networks.

[0091] Fusion rule adaptive module

[0092] Deep Gated Convolutional Neural Network (DGCNN): This method dynamically adjusts the weights of different band features through a gating mechanism, replacing traditional fixed fusion rules (such as mean and maximum values). Based on the spatial distribution of input features, DGCNN adaptively generates a fusion decision graph to highlight key areas (such as lesion boundaries).

[0093] Hybrid attention mechanism: Combining channel attention (focusing on important features) and spatial attention (locating salient regions) to enhance the selective fusion of multi-band features. For example, in the fusion of infrared and visible light, the attention module can enhance edge details.

[0094] Colorization and super-resolution modules

[0095] Adaptive Colorization Network (DCNN): If the original multi-band image is grayscale, a lightweight CNN (such as a U-Net variant) can be used to generate a pseudo-color image to improve the human eye's observation effect.

[0096] Multi-contrast super-resolution fusion: For low-resolution bands (such as infrared), combined with the high-resolution features of visible light, the super-resolution network (such as SRResNet) is used to improve the detail clarity of the fused image.

[0097] 2. Training Strategy Optimization

[0098] End-to-end adaptive training

[0099] Generative Adversarial Network (GAN): This constructs a generator (fusion network) and a discriminator (quality assessment), optimizing the fusion results through adversarial training. The discriminator can be designed as a multi-task structure to simultaneously evaluate clarity, information content, and spectral fidelity.

[0100] Unsupervised loss functions include feature reconstruction loss (L1 / L2 norm), perceptual loss (feature space similarity of pre-trained CNNs), and adversarial loss. For example, the PMGI model optimizes the fused image through gradient and intensity ratio maintenance loss.

[0101] Dataset construction and data enhancement

[0102] Multi-band paired data: Collect multi-band images of the same scene (such as white light, narrowband light, and infrared) and align pixel-level features using annotation tools.

[0103] Simulation data enhancement: Simulate the real endoscopy environment by adding noise, blur or lighting changes to improve the model's generalization ability.

[0104] The deep learning network proposed above has the following advantages:

[0105] Multi-band synchronous fusion

[0106] It supports input of more than two bands (such as white light + ultraviolet + infrared), extracts multi-scale features through parallel convolution branches, and then dynamically fuses them through a gating mechanism.

[0107] Multi-task joint optimization

[0108] Integrate decomposition, fusion, super-resolution, and colorization tasks into a single model to reduce redundant calculations and improve real-time performance.

[0109] Hardware adaptation optimization

[0110] Use lightweight network structures (such as MobileNet) or knowledge distillation technology to compress the model size to adapt to the computing power limitations of embedded devices (such as endoscope terminals).

[0111] 4. Performance Evaluation and Application

[0112] Evaluation Metrics

[0113] Objective indicators: entropy (information content), QAB / F index (fusion quality), SAM / ERGAS (spectral fidelity) and LPIPS (perceptual similarity).

[0114] Subjective evaluation: Doctors score the visibility of the lesion area and the naturalness of the image.

[0115] When applied, it can detect tumors at an early stage by fusing white light and fluorescence images to enhance the contrast of subtle lesions on the mucosal surface.

[0116] and vascular visualization: combining infrared and visible light to highlight vascular structures to guide minimally invasive surgery.

[0117] Reference Figure 3 In Example 2, the present invention further proposes an early cancer risk warning device based on multi-band image fusion, comprising:

[0118] A data acquisition module is used to acquire N endoscopic images taken at the same position and at the same time under light sources of different wavelengths;

[0119] The fusion representation image calculation module is used to fuse N endoscopic images of any wavelength light source to obtain a fusion representation image of the wavelength light source, and finally obtain a fusion representation image corresponding to all wavelength light sources;

[0120] A preprocessing module is used to register and preprocess the fused representation images corresponding to all band light sources to obtain processed fused representation images corresponding to all band light sources;

[0121] A multi-band fusion image calculation module is used to input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model and obtain the output result of the deep learning model, namely the multi-band fusion image;

[0122] A visualization display module, used for visually displaying the multi-band fusion image;

[0123] The risk warning module is used to obtain the early cancer risk analysis result for the multi-band fusion image and issue a warning when there is an early cancer risk.

[0124] In this embodiment 2, consistent with the early cancer risk warning method based on multi-band image fusion proposed by the present invention, multiple images are first collected for different band light sources, and image fusion is performed based on the dominant expression areas of the multiple images to obtain a fused representation image corresponding to all band light sources; then, the fused representation images corresponding to all band light sources are preprocessed and input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are performed based on the output result. On the one hand, for images from different band light sources, the fusion of multiple collected images can obtain a higher quality fused representation image, accelerate the subsequent deep learning model analysis, and improve the accuracy of the subsequent deep learning model analysis results to a certain extent; on the other hand, the fused image obtained by fusing endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous area, facilitating subsequent risk analysis and warning.

[0125] Example 3. The present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the early cancer risk warning method based on multi-band image fusion.

[0126] In this embodiment 3, consistent with the early cancer risk warning method based on multi-band image fusion proposed by the present invention, multiple images are first collected for different band light sources, and image fusion is performed based on the dominant expression areas of the multiple images to obtain a fused representation image corresponding to all band light sources; then, the fused representation images corresponding to all band light sources are preprocessed and input into a pre-established deep learning model for multi-band image fusion to obtain an output result, and subsequent early cancer risk judgment and warning are performed based on the output result. On the one hand, for images from different band light sources, the fusion of multiple collected images can obtain a higher quality fused representation image, accelerate the subsequent deep learning model analysis, and improve the accuracy of the subsequent deep learning model analysis results to a certain extent; on the other hand, the fused image obtained by fusing endoscopic images of all band light sources based on the deep learning model can highlight the characteristics of the cancerous area, facilitating subsequent risk analysis and warning.

[0127] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0128] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0129] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0130] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. An early cancer risk warning method based on multi-band image fusion, characterized by: These include: Step 110: Acquire N endoscopic images taken at the same position and during the same time period under light sources of different wavelengths; Step 120: For any wavelength light source, perform image fusion on N endoscopic images to obtain a fused representation image of the wavelength light source, and finally obtain fused representation images corresponding to all wavelength light sources; Step 130: registering and preprocessing the fused representation images corresponding to all wavelength light sources to obtain processed fused representation images corresponding to all wavelength light sources; Step 140: Input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model, and obtain the output result of the deep learning model, that is, the multi-band fusion image; Step 150: Visually display the multi-band fusion image; Step 160: Obtain an early cancer risk analysis result for the multi-band fusion image, and issue an early warning when there is an early cancer risk; Specifically, for a light source of any wavelength band, N endoscopic images are fused to obtain a fused representation image of the light source of the wavelength band, including: Construct an endoscopic image sequence of the current band light source in the order of shooting time. In the endoscopic image sequence, matrix_i(i) is defined as the image matrix of the i-th endoscopic image, and matrix_i(i,ji,ki) represents the pixel value of the pixel point in the j-th row and k-th column in matrix_i(i); For any pixel point matrix_i(i,ji,ki), calculate its dominant feature expression adv(i,ji,ki) relative to the two-dimensional neighboring pixel points. The calculation formula is as follows: ； in, Represents the average value of the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the maximum value among the two-dimensional neighborhood pixels of pixel matrix_i(i,ji,ki), Represents the minimum value among the two-dimensional neighborhood pixels of pixel point matrix_i(i,ji,ki); The process of image fusion is as follows: Step 210: Perform preliminary fusion of N endoscopic images in an equal-weighted weighted average manner to obtain a preliminary fused image; Step 220: Initialize i, set i=1; Step 230: Calculate the dominant feature expression area of matrix_i(i) based on the dominant feature expression degree adv(i,ji,ki); Step 240: Replace the dominant feature expression region of matrix_i(i) with the region at the same pixel position in the preliminary fused image to obtain an updated preliminary fused image; Step 250: Determine whether i is equal to N. If so, end the loop and output the updated preliminary fusion image as the fusion representation image. If not, increment the value of i by 1 and go back to step 230 to continue. Specifically, the dominant feature expression area of matrix_i(i) is calculated based on the dominant feature expression degree adv(i,ji,ki), including: For matrix_1(1), traverse matrix_1(1) and find the two pixels with the largest dominant feature expression, record them as pix_1 and pix_2, connect pix_1 and pix_2 to get the straight line pix_1,2, find the pixel with the largest dominant feature expression in matrix_1(1) that is not on the straight line pix_1,2, record it as pix_3, then the area formed by connecting pix_1, pix_2 and pix_3 is the dominant feature expression area of matrix_1(1); For matrix_i(i), calculate the average pixel value of the pixel points in the dominant feature expression area of matrix_i-1(i-1) and record it as , then traverse matrix_i(i) and find out the dominant feature expression greater than Any two pixels are denoted as pix_4 and pix_5, and a straight line pix_4,5 is obtained by connecting pix_4 and pix_5. In matrix_i(i), find the pixel with the largest dominant feature expression that is not on the straight line pix_4,5 and denote it as pix_6. Then the area formed by connecting pix_4, pix_5 and pix_6 is the dominant feature expression area of matrix_i(i).

2. The early cancer risk warning method based on multi-band image fusion according to claim 1 is characterized in that: Specifically, in step 130, the fused representation images corresponding to all wavelength light sources are registered and preprocessed to obtain processed fused representation images corresponding to all wavelength light sources, including: The fused representation images corresponding to all band light sources are registered through Elastic Fusion or DeepReg network, and the resolution of the fused representation images corresponding to all band light sources is unified to obtain the fused representation images corresponding to all band light sources after registration; The fused representation images corresponding to all band light sources after registration are independently Z-score normalized to eliminate brightness differences and obtain the processed fused representation images corresponding to all band light sources.

3. The early cancer risk warning method based on multi-band image fusion according to claim 1 is characterized in that: Specifically, the multi-band fusion image is visualized, including: When outputting results, the deep learning model automatically marks abnormal blood vessels and mucosal color changes, and generates high-contrast images for the marked locations for visual display.

4. The early cancer risk warning method based on multi-band image fusion according to claim 1 is characterized in that: Specifically, in step 160, Methods for obtaining early cancer risk analysis results for the multi-band fusion image include obtaining analysis results input by a doctor or analysis results obtained based on AI automatic analysis.

5. An early cancer risk warning device based on multi-band image fusion, characterized in that: The method according to any one of claims 1 to 4 is applied, comprising: A data acquisition module is used to acquire N endoscopic images taken at the same position and at the same time under light sources of different wavelengths; The fusion representation image calculation module is used to fuse N endoscopic images of any wavelength light source to obtain a fusion representation image of the wavelength light source, and finally obtain a fusion representation image corresponding to all wavelength light sources; A preprocessing module is used to register and preprocess the fused representation images corresponding to all band light sources to obtain the processed fused representation images corresponding to all band light sources; A multi-band fusion image calculation module is used to input the processed fusion representation images corresponding to all band light sources into a pre-trained deep learning model and obtain the output result of the deep learning model, namely the multi-band fusion image; A visualization display module, used for visually displaying the multi-band fusion image; The risk warning module is used to obtain the early cancer risk analysis result for the multi-band fusion image and issue a warning when there is an early cancer risk.

6. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images