Colorectal polyp identification and classification method, system, electronic device and storage medium

Abstract

The present application provides a method, system, electronic device and storage medium for colorectal polyp identification and classification, and relates to the field of polyp identification and classification. The method includes: obtaining a multi-source image of the colorectal region of a patient through an endoscopic device, and evaluating and enhancing the image quality. Then, extracting a derived image, and normalizing it together with the enhanced multi-source image to obtain a target image. Next, performing multi-scale feature extraction and polyp region detection on the target image to obtain a polyp region image. The polyp region image is then input into a contour segmentation model to obtain a polyp contour image, and polyp feature data is extracted. Finally, the polyp feature data and the patient's clinical information are input into a prediction model to obtain a polyp lesion risk probability value. The above method is used to improve the accuracy of colorectal polyp identification and classification.

Description

Technical Field

[0001] The present invention relates to the field of polyp identification and classification, and in particular to a method, system, electronic device and storage medium for colorectal polyp identification and classification. Background Art

[0002] Colorectal polyps are common diseases of the digestive tract. If not discovered and treated in time, they may deteriorate into colorectal cancer, threatening the patient's life and health. Traditional colorectal polyp diagnosis mainly relies on the doctor's experience and naked eye observation, which has problems such as strong subjectivity, low efficiency, and high missed diagnosis rate.

[0003] In order to improve the accuracy and efficiency of colorectal polyp diagnosis, researchers began to explore computer-aided diagnosis technology. A common method is to detect and classify polyp areas based on endoscopic images. This method automatically locates the polyp area and determines its type by performing image segmentation, feature extraction, pattern recognition and other processing on endoscopic images, providing doctors with diagnostic references. However, existing polyp diagnosis methods based on endoscopic images generally have the following shortcomings: First, the quality of endoscopic images is uneven, with problems such as blur, noise, and uneven lighting, which directly affect the detection and classification accuracy of polyp areas. Summary of the invention

[0004] The present application provides a method, system, electronic device and storage medium for identifying and classifying colorectal polyps, so as to improve the accuracy of identifying and classifying colorectal polyps.

[0005] In a first aspect, the present application provides a method for identifying and classifying colorectal polyps, the method comprising:

[0006] Taking photos of the colorectal area of ​​the patient under different lighting conditions by an endoscopic device to obtain a multi-source image, and performing quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image, and when the assessment value is less than a preset threshold, performing image enhancement processing on the multi-source image to obtain an image-enhanced multi-source image;

[0007] Performing image extraction based on the image-enhanced multi-source image to obtain a derived image, and performing standardization processing on the derived image and the image-enhanced multi-source image to obtain a target image;

[0008] Performing multi-scale feature extraction on the target image to obtain a multi-scale feature map, and performing polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image;

[0009] Inputting a preset contour segmentation image into the polyp region image to obtain a polyp contour image;

[0010] Obtain the patient's clinical information and perform feature extraction on the contour image to obtain polyp feature data;

[0011] The polyp characteristic data and the patient's clinical information are input into the preset polyp lesion risk prediction model to obtain the polyp lesion risk probability value.

[0012] In the above technical solution, an endoscopic device is used to take photos of the colorectal area of ​​the patient, obtain multi-source images, and perform quality assessment on the multi-source images. By setting a comparison mechanism between the quality assessment value and the preset threshold, it is dynamically determined whether the image quality meets the requirements. When the quality assessment value is less than the preset threshold, the multi-source image is automatically enhanced, which effectively improves the image quality, overcomes the problem of unstable endoscopic image quality, and lays a good foundation for subsequent processing.

[0013] After obtaining the multi-source image after image enhancement, the method of the present invention further extracts a variety of derived images, such as grayscale images, gradient images, wavelet coefficient images, saliency maps, etc. These derived images depict the characteristics of the original image from different angles, providing a richer and more comprehensive image information representation. The grayscale image highlights the brightness and contrast characteristics of the image by removing the color information of the image, making the difference between the polyp area and the background area more obvious. The gradient map reflects the texture and edge characteristics of the image by calculating the gradient amplitude of the image pixels in the horizontal and vertical directions, and captures the direction and change trend of the polyp boundary. The wavelet coefficient map extracts the local texture and detail features of the image at different frequency sub-bands and spatial positions by performing multi-scale and multi-directional wavelet transform on the image, and comprehensively depicts the morphological and structural information of the polyp. The saliency map automatically highlights the salient areas related to polyps in the image by calculating the visual saliency and attention of the image area, providing prior guidance for subsequent polyp positioning and segmentation.

[0014] By extracting multiple derivative images, the intrinsic features of the image are mined from multiple dimensions such as brightness, texture, edge, frequency domain, and saliency, which not only makes up for the shortcomings of a single image representation, but also enhances the discriminability and complementarity of image features. Compared with relying solely on the original image, the derived image provides a richer and more fine-grained feature representation, enabling subsequent feature learning and classification to capture more comprehensive and detailed imaging characteristics of polyps, which helps to improve the accuracy and reliability of polyp detection. At the same time, the extraction of derived images also greatly reduces the complexity of image analysis, converting high-dimensional, highly redundant original images into low-dimensional, highly discriminative feature maps, reducing computational overhead and improving analysis efficiency.

[0015] After obtaining the target image, this method adopts a multi-scale feature extraction strategy to generate a multi-scale feature map. Through feature extraction, the characteristic performance of polyps at different scales is fully explored, and the multi-level and multi-angle characteristics of polyps are comprehensively characterized, overcoming the limitations of the single-scale feature representation of existing methods. Based on the multi-scale feature map, this method further performs polyp region detection, adopts a selective search strategy to adaptively generate candidate regions, and judges and optimizes the candidate regions through classifiers and regressors, and finally obtains an accurate polyp region image. Selective search extracts candidate regions of different scales and positions from the bottom up through steps such as region segmentation, similarity calculation, and region merging, avoiding the redundancy and inefficiency of traditional sliding window and brute force search methods, and greatly improving the accuracy and speed of polyp region detection.

[0016] After accurately locating the polyp area, the method performs contour segmentation on the polyp area image to obtain the polyp contour image, and extracts the features of the contour image to obtain the polyp feature data. At the same time, the method also obtains the patient's clinical information, combines the polyp feature data with the patient's clinical information, constructs a polyp lesion risk prediction model, outputs the polyp lesion risk probability value, and realizes the risk assessment and prognosis prediction of polyps.

[0017] In a second aspect of the present application, a colorectal polyp identification and classification system is provided, the system comprising:

[0018] An image acquisition module is used to take a picture of the colorectal area of ​​the patient through an endoscope device, obtain a multi-source image through exploration, and perform a quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image. When the assessment value is less than a preset threshold, the multi-source image is enhanced to obtain an enhanced multi-source image.

[0019] An image preprocessing module is used to perform image extraction based on the image-enhanced multi-source image to obtain a derived image, and to perform standardization processing on the derived image and the image-enhanced multi-source image to obtain a target image;

[0020] A polyp region detection module is used to perform multi-scale feature extraction on a target image to obtain a multi-scale feature map, and perform polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image;

[0021] A polyp segmentation module is used to input a preset contour segmentation image into the polyp region image to obtain a polyp contour image;

[0022] A polyp feature data acquisition module is used to obtain clinical information of the patient and perform feature extraction on the contour image to obtain polyp feature data;

[0023] The lesion risk prediction module is used to input the polyp characteristic data and the patient's clinical information into a preset polyp lesion risk prediction model to obtain a polyp lesion risk probability value.

[0024] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

[0025] In the fourth aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the above method.

[0026] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0027] 1. After obtaining the multi-source image after image enhancement, the present invention further extracts a variety of derived images, such as grayscale images, gradient images, wavelet coefficient images, saliency images, etc. These derived images depict the features of the original image from different angles, providing a richer and more comprehensive image information representation. At the same time, the method also performs standardization processing on the derived images and the multi-source images after image enhancement to obtain a unified and standardized target image, which is convenient for subsequent feature extraction and analysis. The comprehensive use of multi-source images, derived images and standardization processing greatly enhances the richness and consistency of image feature representation.

[0028] 2. After obtaining the target image, the method of this application adopts a multi-scale feature extraction strategy to generate a multi-scale feature map. Through feature extraction, the characteristic performance of polyps at different scales is fully explored, and the multi-level and multi-angle characteristics of polyps are comprehensively portrayed, overcoming the limitations of the single-scale feature representation of existing methods. Based on the multi-scale feature map, the method further performs polyp region detection, adopts a selective search strategy to adaptively generate candidate regions, and judges and optimizes the candidate regions through classifiers and regressors, and finally obtains an accurate polyp region image. Selective search extracts candidate regions of different scales and positions from the bottom up through steps such as region segmentation, similarity calculation, and region merging, avoiding the redundancy and inefficiency of traditional sliding windows and brute force search methods, and greatly improving the accuracy and speed of polyp region detection.

[0029] 3. After accurately locating the polyp area, the method of the present application performs contour segmentation on the polyp area image to obtain the polyp contour image, and extracts the features of the contour image to obtain the polyp feature data. At the same time, the method also obtains the patient's clinical information, combines the polyp feature data with the patient's clinical information, constructs a polyp lesion risk prediction model, outputs the polyp lesion risk probability value, and realizes the risk assessment and prognosis prediction of polyps. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a flow chart of a method for identifying and classifying colorectal polyps provided in an embodiment of the present application;

[0031] Figure 2 An architectural diagram of a colorectal polyp identification and classification system provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0033] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0034] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0035] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0036] In order to facilitate understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced.

[0037] Colorectal polyps are common diseases of the digestive tract. If not discovered and treated in time, they may deteriorate into colorectal cancer, seriously threatening the patient's life and health. Traditional colorectal polyp diagnosis mainly relies on doctors' clinical experience and naked eye observation. There are problems such as strong subjectivity, low efficiency, and high missed diagnosis rate, which makes it difficult to meet the growing clinical needs.

[0038] In order to improve the accuracy and efficiency of colorectal polyp diagnosis, researchers have begun to explore computer-aided diagnosis technology to seek more objective, efficient and reliable diagnostic methods. Among them, polyp area detection and classification based on endoscopic images is a research direction that has attracted much attention. This method uses computer vision and artificial intelligence technology to perform a series of processing such as image segmentation, feature extraction, and pattern recognition on endoscopic images to automatically locate polyp areas and determine their types, providing doctors with objective and accurate diagnostic references.

[0039] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0040] Based on the above background technology, please refer to Figure 1 , Figure 1 A flow chart of a method for identifying and classifying colorectal polyps provided in an embodiment of the present application. The system can be implemented by a computer program or can be run as an independent tool application. Specifically, in an embodiment of the present application, the method can be applied on a server, but can also be applied on electronic devices such as servers. A method for identifying and classifying colorectal polyps includes the following steps:

[0041] S101, photographing the colorectal area of ​​a patient under different lighting conditions by an endoscope device to obtain a multi-source image, and performing quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image, and when the assessment value is less than a preset threshold, performing image enhancement processing on the multi-source image to obtain an image-enhanced multi-source image;

[0042] Specifically, the colorectal area of ​​the patient is photographed and explored by an endoscopic device to obtain multi-source images. With its tiny size and flexible control, the endoscopic device can penetrate deep into the colorectum, directly observe and image the mucosal surface and intestinal structure, and obtain high-resolution images reflecting tissue morphology and pathological changes. By adjusting the light source type and intensity of the endoscope, the same area is imaged multiple times under different lighting conditions, and multiple modal images such as white light, narrow spectrum, and autofluorescence can be obtained, that is, multi-source images. Multi-source images comprehensively record the imaging information of the colorectal area under different spectral and polarization conditions, including the apparent features such as color and texture provided by white light images, the mucosal surface and subsurface vascular network highlighted by narrow spectrum images, and the tissue metabolism and structural abnormalities reflected by autofluorescence images. These images of different modalities depict the imaging characteristics of polyps from multiple perspectives such as morphology and physiological pathology, greatly enrich the expression of polyp characteristics, and provide a more reliable data basis for subsequent feature extraction and classification, which is conducive to improving the accuracy and generalization ability of polyp detection.

[0043] After obtaining multi-source images, quality assessment is performed to quantify quality indicators such as image clarity, contrast, and integrity to obtain quality assessment values. Quality assessment can promptly discover and identify various quality problems that occur during image acquisition, such as motion blur, underexposure, overexposure, and tissue reflection. When the quality assessment value is lower than the preset threshold, it means that the image quality does not meet the requirements of subsequent analysis and image enhancement is required. Image enhancement improves the visual quality and information integrity of the image through image processing algorithms such as contrast adjustment, edge sharpening, and denoising filtering, making the image clearer, more uniform, and more layered. The multi-source image after image enhancement improves image quality and reduces image noise and distortion while retaining the original image information, providing a purer and more consistent input for subsequent feature extraction, which helps to improve the distinguishability and robustness of feature representation, thereby improving the performance of polyp detection.

[0044] Quality assessment and image enhancement are key links in image preprocessing. Through automated quality control and optimization mechanisms, the quality and availability of input images are guaranteed, the impact of image quality differences on subsequent analysis is reduced, and the adaptability and reliability of the system are improved. Compared with directly using the original acquired images, the multi-source images after quality assessment and image enhancement have obvious improvements in clarity, contrast, signal-to-noise ratio, etc., highlighting the key features of polyps and reducing the interference of irrelevant backgrounds. It provides more ideal input for subsequent feature extraction, segmentation, classification, etc., and lays a good data foundation. At the same time, quality assessment can be used as a feedback basis for the image acquisition process, guiding the parameter adjustment and imaging optimization of endoscopic operations, forming a closed-loop control, realizing the coordinated optimization of image acquisition and analysis, and further enhancing the performance and stability of the system.

[0045] Based on the above embodiment, as an optional embodiment, performing quality assessment on multi-source images to obtain quality assessment values ​​of the multi-source images includes:

[0046] Image parameters are extracted from multi-source images to obtain image parameters, and quality assessment is performed based on the image parameters and preset quality indicators to obtain quality assessment values ​​of the multi-source images.

[0047] Specifically, first, image parameters are extracted from multi-source images. This step automatically analyzes various parameter indicators of multi-source images, such as image clarity, contrast, signal-to-noise ratio, artifact level, etc., through image processing algorithms. Specifically, edge detection, frequency domain analysis and other methods can be used to calculate the gradient information, spectrum characteristics, etc. of the image, and quantitatively characterize the clarity and texture details of the image. At the same time, the contrast and uniformity of the image are calculated through histogram analysis, regional statistics and other methods to evaluate the dynamic range and grayscale distribution of the image. In addition, the noise level and artifact severity of the image can be quantitatively analyzed through noise estimation, artifact detection and other technologies. Through image parameter extraction, various quality indicators of multi-source images can be comprehensively and objectively characterized, providing a reliable data basis for quality assessment.

[0048] After obtaining the image parameters, the embodiment of the present invention further performs a quality assessment based on the image parameters and preset quality indicators. The preset quality indicators are quality evaluation standards formulated for different modal images based on clinical experience and expert consensus, such as the signal-to-noise ratio threshold of CT images, the artifact limit of MRI images, etc. By comparing the extracted image parameters with the preset quality indicators, it can be determined whether the multi-source images meet the quality requirements and quantitatively evaluate their quality levels. For example, if the signal-to-noise ratio of the CT image is lower than the preset threshold, it is judged to be of unqualified quality; if the artifact level of the MRI image exceeds the preset limit, it is judged to have serious artifacts. Through parameter comparison and threshold judgment, the quality of multi-source images can be automatically and accurately evaluated, avoiding the subjectivity and inconsistency of manual evaluation.

[0049] After quality assessment, the quality assessment value of the multi-source image can be obtained, which reflects the quality of the image. The quality assessment value can be a continuous value, such as 0-100 points, or a discrete level, such as excellent, good, medium, poor, etc. Through the quality assessment value, clinicians can intuitively and quantitatively understand the quality level of multi-source images and judge whether they meet the diagnosis and analysis requirements. For images with low quality assessment values, re-capture or image enhancement processing can be performed to improve image quality and ensure the reliability of subsequent analysis. For images with higher quality assessment values, they can directly enter the subsequent feature extraction and disease risk prediction process to improve the efficiency of prediction.

[0050] Based on the above embodiment, as an optional embodiment, performing image enhancement processing on the multi-source image to obtain the multi-source image after image enhancement includes:

[0051] S201, performing super-resolution reconstruction on the multi-source image to obtain a high-resolution multi-source image;

[0052] Specifically, through super-resolution reconstruction technology, low-resolution multi-source images are converted into high-resolution images to improve the clarity and detail information of the image. Super-resolution reconstruction is an advanced image processing technology that can restore or generate high-resolution images from low-resolution images, break through the physical limitations of traditional image acquisition equipment, and obtain image quality beyond the original resolution. The embodiment of the present invention adopts a super-resolution reconstruction method based on deep learning, and realizes end-to-end image super-resolution reconstruction through a convolutional neural network. Specifically, first, a super-resolution reconstruction network consisting of multiple convolutional layers, upsampling layers and residual blocks is constructed, and end-to-end training is performed on large-scale high- and low-resolution image pairs to learn the nonlinear mapping relationship from low-resolution images to high-resolution images. During the training process, advanced loss functions such as adversarial loss and perceptual loss are used to make the reconstructed high-resolution images as close to the real high-resolution images as possible in terms of visual quality and feature representation. Through deep learning, the super-resolution reconstruction network can automatically extract and utilize the multi-scale and multi-directional features of the image to generate high-resolution images with rich details and clear textures.

[0053] In the application stage, the low-resolution multi-source images to be processed are input into the trained super-resolution reconstruction network. After layer-by-layer feature extraction, upsampling and nonlinear transformation, the reconstructed high-resolution multi-source images can be obtained. Compared with the original low-resolution images, the reconstructed high-resolution images have significantly improved visual quality, with clearer and sharper edges, richer and more realistic detail textures, and significantly improved overall sharpness and contrast. Through super-resolution reconstruction, ultra-high-resolution diagnostic images can be obtained from existing imaging equipment, breaking through the imaging limits of hardware equipment, extending the service life of equipment, and reducing medical costs. After super-resolution reconstruction, high-resolution multi-source images with image enhancement can be obtained. These high-resolution images provide clearer and more accurate imaging basis for clinical diagnosis and analysis.

[0054] S202, performing defogging processing on the high-resolution multi-source image by using a dark channel prior method to obtain a defogged multi-source image;

[0055] Specifically, the dark channel prior method is based on the statistical laws of a large number of fog-free images, and finds that the dark channel values ​​of most fog-free images are close to zero, that is, in the local area of ​​each pixel, there are always some pixels whose color channel values ​​are close to zero. In foggy images, due to the presence of haze, the dark channel value is high, resulting in the overall whiteness of the image and blurred details. Using this prior knowledge, the dark channel prior method first estimates the global atmospheric illumination value of the image, by finding the pixels in the top 0.1% of brightness on the dark channel map, and taking the average value of the pixel values ​​at the corresponding position on the original foggy image as the atmospheric illumination value. Then, using the atmospheric illumination value and the foggy image, the transmittance map of the image is estimated. The transmittance reflects the attenuation degree of the pixel by haze. Finally, according to the estimated atmospheric illumination and transmittance map, the fog-free image is solved through the atmospheric scattering model to achieve image defogging.

[0056] In the dehazing process, the dark channel prior method also uses optimization strategies such as guided filtering and soft cutout to improve the accuracy and robustness of transmittance estimation and avoid image distortion caused by excessive dehazing. Through guided filtering, the gradient information of the original image can be used to refine the transmittance map, so that the haze area can be smoothed while maintaining the edge. Through soft cutout technology, the contrast and saturation of the transmittance map can be adaptively adjusted to avoid problems such as too bright dark areas and too dark bright areas, making the dehazed image more natural and realistic.

[0057] After dehazing, a high-resolution multi-source image with dehazing enhancement can be obtained. Compared with the original image, the visual quality of the dehazing image is significantly improved, the haze and blur are effectively removed, the image is clearer and brighter, and the detail texture is more distinct. Through dehazing, the visual obstacles introduced by factors such as imaging environment and equipment noise in medical images can be eliminated, and the true appearance of the image can be restored.

[0058] S203, performing spectral enhancement processing on the dehazed multi-source image to obtain an image-enhanced multi-source image.

[0059] Specifically, an embodiment of the present invention adopts a spectral enhancement method based on multi-scale Top-Hat transform. Top-Hat transform is an image enhancement technology based on mathematical morphology. By performing opening and closing operations on the image, bright details and dark details in the image are extracted to achieve contrast enhancement and background correction. In spectral enhancement, Top-Hat transform is first performed on each band of the multi-source image to obtain a bright detail map and a dark detail map of each band. Then, the bright detail map and the dark detail map are scale-fused to extract the multi-scale spectral features of the image at different scales. Finally, the multi-scale spectral features are weighted-fused with the original image to obtain a spectrally enhanced image.

[0060] In the process of spectral enhancement, the size and shape of the structural elements of the Top-Hat transform are key parameters that directly affect the enhancement effect. The embodiments of the present invention use multi-scale structural elements to extract the spectral features of the image at different scales, which can not only highlight the local fine texture but also retain the global structural information. At the same time, according to the characteristics of different modal images, the shape of the structural elements is adaptively selected, such as linear structural elements for vascular images and disc-shaped structural elements for lung images, so as to better match the morphological characteristics of the tissue structure. In addition, when fusing spectral features, an adaptive weighting strategy is adopted to automatically adjust the weights according to the signal-to-noise ratio and contrast of each band, highlighting the bands with good quality and large amount of information, and suppressing the influence of noise bands.

[0061] After spectral enhancement, a high-resolution multi-source image can be obtained. Compared with the dehazed image, the spectral enhanced image has a significant improvement in spectral resolution and contrast. The spectral features of different tissue structures are more distinct, the edge contours are clearer, the texture details are richer, and the overall visual effect is more vivid and realistic. Through spectral enhancement, the spectral information contained in the multi-source image can be maximized.

[0062] S102, performing image extraction based on the image-enhanced multi-source image to obtain a derived image, and performing standardization processing on the derived image and the image-enhanced multi-source image to obtain a target image;

[0063] Specifically, after obtaining the multi-source image after image enhancement, the embodiment of the present invention further performs image extraction based on the multi-source image after image enhancement. Specifically, a variety of derivative images, such as grayscale images, gradient images, edge images, texture images, wavelet coefficient images, etc., are extracted from the multi-source image after image enhancement by using methods such as image transformation and image decomposition. These derivative images depict the information of the original image from different feature angles, such as the grayscale image reflects the brightness information of the image, the gradient image and the edge image reflect the structural information of the image, the texture image reflects the texture characteristics of the image, and the wavelet coefficient image reflects the frequency characteristics of the image at different scales. By extracting a variety of derivative images, the feature information contained in the multi-source image after image enhancement can be mined from multiple dimensions to obtain a richer and more comprehensive image representation.

[0064] After image extraction, the derived images and the multi-source images after image enhancement are standardized. Standardization includes operations such as image size normalization and pixel value normalization, the purpose of which is to eliminate the scale differences and intensity differences between different images and unify the image data to the same scale and value range. For example, the size of all images is uniformly adjusted to 256*256, and the pixel values ​​are uniformly normalized to the interval [0,1]. Through standardization, the effects of image scale and intensity changes can be eliminated, making different images more comparable, which is convenient for subsequent feature extraction and analysis. At the same time, standardization can also speed up calculations and improve the efficiency of model training and reasoning. After standardization, a unified and standardized target image can be obtained, which provides a unified image representation basis for subsequent processing.

[0065] Based on the above embodiment, as an optional embodiment, image extraction is performed based on the multi-source image after image enhancement to obtain a derived image, including:

[0066] S301, extracting grayscale images from multi-source images to obtain grayscale images;

[0067] Specifically, grayscale image extraction can be achieved through weighted averaging. For RGB color images, the pixel values ​​of the three color channels are taken respectively and assigned different weights. Usually, the weights of the red, green, and blue channels are 0.299, 0.587, and 0.114, respectively, which is determined based on the difference in the sensitivity of the human eye to light of different wavelengths. The weighted average of the three channels is used as the pixel value of the grayscale image to obtain the grayscale image. For RGBA images, it is necessary to remove the transparency channel first and then perform weighted averaging. For multi-channel medical images, such as CT, MRI, etc., a specific channel combination can be selected for grayscale to highlight the tissue structure of interest.

[0068] In the process of grayscale image extraction, attention should be paid to the range of grayscale values ​​and quantization accuracy. In order to facilitate storage and processing, the pixel values ​​of grayscale images are usually normalized to the range of 0-255 and quantized using 8 bits. For some medical images, such as CT images, the original grayscale value range is very large, and the window width and window position need to be adjusted to map the grayscale range of interest to 0-255 to improve contrast and detail expression. In addition, in the grayscale process, information loss caused by quantization error should be avoided. If necessary, a higher-precision quantization method can be used, such as 16-bit quantization.

[0069] S302, calculating the gradient magnitude and gradient direction of the multi-source image to obtain the gradient magnitude and gradient direction of the multi-source image, and constructing a gradient map based on the gradient magnitude and gradient direction of the multi-source image;

[0070] Specifically, gradient calculation can be implemented through a variety of operators, such as Sobel operator, Prewitt operator, Scharr operator, etc. Taking the Sobel operator as an example, two 3*3 convolution kernels are used to convolve the image to obtain the horizontal component and vertical component of the gradient. The horizontal convolution kernel is [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], and the vertical convolution kernel is [[-1, -2, -1], [0, 0, 0], [1, 2, 1]]. Take the sum of the squares of the convolution results and then take the square root to obtain the gradient amplitude map. Take the inverse tangent of the horizontal component and the vertical component to obtain the gradient direction map. Considering the computational efficiency and noise suppression, the gradient amplitude map can also be mean filtered or median filtered, and the gradient direction map can be quantized or smoothed.

[0071] In the process of gradient calculation, attention should be paid to the processing of edge pixels and the normalization of gradient values. For edge pixels of an image, since their neighborhood is incomplete, convolution cannot be performed directly, and edge extension, such as zero padding and mirror padding, is required. The calculated gradient amplitude needs to be normalized and mapped to the range of 0-255 for visualization and subsequent processing. At the same time, in order to highlight significant gradient changes, the gradient amplitude can be transformed nonlinearly, such as logarithmic transformation, gamma transformation, etc., to further enhance the contrast and detail expression of the gradient map. After the gradient is calculated, the gradient amplitude map and gradient direction map can be obtained, and they are combined into a gradient map. The gradient map intuitively shows the distribution and trend of brightness changes in the image, where the edges and contours show obvious highlights, and flat areas correspond to low gradient values.

[0072] S303, performing morphological transformation on the multi-source images to obtain a morphological image;

[0073] Specifically, morphological transformation requires the selection of appropriate structural elements and morphological operators. The structural element is a small binary template used to detect local morphological features in an image. Common structural elements include rectangles, disks, crosses, and other shapes. Morphological operators define the interaction between structural elements and images. Common operators include erosion, dilation, opening, closing, top-hat transformation, and bottom-hat transformation. The erosion operation shrinks the bright areas in the image and expands the dark areas, which can remove small bright noise and burrs; the dilation operation is the opposite, expanding the bright areas and shrinking the dark areas, which can fill small dark noise and holes. The opening operation and the closing operation are a serial combination of erosion and dilation. The opening operation first erodes and then dilates, which can remove bright details and retain the overall contour; the closing operation first dilates and then erodes, which can fill dark details and smooth the boundaries. The top-hat transformation and the bottom-hat transformation extract the bright details and dark details in the image respectively, highlighting the local texture and structural information.

[0074] In the process of morphological transformation, it is necessary to adaptively select the shape, size and combination of operators of the structural element according to the characteristics of the image and the purpose of analysis. For different medical images, such as CT, MRI, ultrasound, etc., the imaging principles and tissue structures are quite different, and different morphological strategies need to be adopted. For example, for vascular images, thin linear structural elements and open operations can be used to extract tubular structures; for lung images, disc-shaped structural elements and closed operations can be used to remove the interference of bronchi and blood vessels. At the same time, it is also necessary to dynamically adjust the scale of the structural element and the number of morphological operations according to the size, shape, density and other characteristics of the lesion to achieve the best enhancement and segmentation effect.

[0075] S304, performing multi-scale decomposition on the multi-source images by using wavelet basis functions to obtain a wavelet coefficient map;

[0076] Specifically, the wavelet transform decomposes the image into low-frequency approximate coefficients and high-frequency detail coefficients by convolving the image with the wavelet basis function. Commonly used wavelet basis functions include Haar wavelet, Daubechies wavelet, Symlet wavelet, Coiflet wavelet, etc. Different wavelet basis functions have different time-frequency characteristics and support lengths. In view of the characteristics of medical images, the embodiment of the present invention adopts a biorthogonal wavelet basis function, such as Cohen-Daubechies-Feauveau (CDF) 9 / 7 wavelet, to achieve multi-scale decomposition of the image. The CDF 9 / 7 wavelet has excellent orthogonality, symmetry, and compact support properties. It can maximize the concentration of image energy and reduce the redundancy of wavelet coefficients while ensuring complete reconstruction. By selecting a suitable number of decomposition layers, such as 3-5 layers, a wavelet coefficient map covering different frequency bands and spatial scales can be obtained. In each layer of decomposition, the image is decomposed into a low-frequency approximate subband LL and a high-frequency detail subband LH, HL, and HH, which correspond to the edge and texture features in the horizontal, vertical, and diagonal directions, respectively.

[0077] In the process of wavelet decomposition, attention should be paid to the extension method of the image boundary and the normalization of the wavelet filter. Since the wavelet transform is a convolution process, the convolution result of the image boundary pixels depends on its neighboring pixels, and the boundary pixels need to be extended, such as zero extension, symmetric extension, periodic extension, etc., to ensure the complete reconstruction of the inverse transform. At the same time, in order to make the wavelet coefficients have the same energy distribution at different scales, the wavelet filter needs to be normalized to meet the completeness condition. Through boundary extension and filter normalization, the stability and reversibility of the wavelet transform can be ensured, laying the foundation for subsequent processing.

[0078] After wavelet decomposition, wavelet coefficient maps at different scales can be obtained. Wavelet coefficient maps intuitively reflect the structural characteristics of the original image in different frequency bands and directions. High-frequency coefficients correspond to image details such as edges, contours, and textures, while low-frequency coefficients correspond to image backgrounds, smooth areas, and other main information.

[0079] S305, performing saliency detection on the multi-source image using a frequency domain analysis method to obtain salient regions in the multi-source image, and marking the salient regions in the multi-source image to obtain a saliency map;

[0080] Specifically, the frequency domain analysis method converts the image from the spatial domain to the frequency domain through Fourier transform, and then uses the phase and amplitude information of the spectrum to construct a saliency measurement model. The phase spectrum reflects the position and structural information of the object in the image, and plays a leading role in cognitive saliency. The amplitude spectrum reflects the energy distribution and texture characteristics of the image, and has an important influence on visual saliency. In view of the characteristics of medical images, the embodiment of the present invention adopts a phase spectrum weighted amplitude spectrum model, comprehensively utilizes global and local frequency information, and improves the accuracy and robustness of saliency detection. First, the multi-source image is Fourier transformed to obtain the amplitude spectrum and phase spectrum of the frequency spectrum. Then, the gradient amplitude of the phase spectrum is calculated to obtain a phase saliency map. Next, the amplitude spectrum is weighted with the phase saliency map to obtain a weighted amplitude spectrum. Finally, the weighted amplitude spectrum is Gaussian filtered and inverse Fourier transformed to obtain a saliency map in the spatial domain.

[0081] S306: Use the grayscale image, the gradient image, the wavelet coefficient image and the saliency image as derived images.

[0082] Specifically, the embodiment of the present invention uses a series of image processing and analysis steps to use grayscale images, gradient images, wavelet coefficient images and saliency images as derived images of multi-source images. These four derived images depict the characteristics and attributes of the original image from different angles, providing rich and diverse information representation for subsequent medical image analysis.

[0083] Based on the above embodiment, as an optional embodiment, the derived image and the multi-source image after image enhancement are subjected to standardization processing to obtain a target image, including:

[0084] S401, performing image registration on the derived image and the multi-source image after image enhancement to obtain a registered image;

[0085] Specifically, the embodiment of the present invention performs image registration on the derived image and the multi-source image after image enhancement to obtain the registered image. During the acquisition process of medical images, due to factors such as changes in patient position, respiratory movement, and differences in scanning parameters, there are often geometric deformations and spatial position deviations between images of different modalities and different phases. Image registration is to align these images to the same coordinate space through geometric transformation so that they can achieve the best match in anatomical structure. Common medical image registration methods include feature-based registration, grayscale-based registration, and deformation model-based registration. Among them, feature-based registration extracts anatomical structure features in the image, such as corners, edges, contours, etc., establishes a correspondence between features, and then estimates the optimal geometric transformation parameters. Grayscale-based registration maximizes the similarity metrics between images, such as correlation coefficients, mutual information, etc., iteratively optimizes the transformation parameters, and makes the grayscale distribution of the registered images as consistent as possible. Deformation model-based registration estimates the non-rigid transformation field by constructing a deformation model of the anatomical structure, such as thin plate splines, B-splines, Demons, etc., to achieve compensation for local deformation.

[0086] In view of the characteristics of derived images and multi-source images, the embodiments of the present invention adopt a multi-step, multi-scale registration strategy. First, the image is preprocessed, such as noise suppression, background removal, etc., to improve the stability of the registration. Then, on a global scale, the grayscale information of the image is used to perform a rough registration through a rigid transformation or an affine transformation to eliminate large positional deviations and rotational dislocations. Then, on a local scale, the gradient information of the image is used to perform a fine registration through a non-rigid transformation to compensate for local nonlinear deformations. Finally, the registration results are regularized and smoothed using a deformation model to obtain the final registered image. During the registration process, a multi-resolution pyramid strategy is used to register images in different scale spaces, and the registration accuracy is gradually refined. Through iterative optimization, the residual energy between the registered images is minimized to obtain the optimal transformation parameters. After the image is registered, the geometrically corrected derived image and multi-source image can be obtained, which are aligned in the spatial coordinate system and the anatomical structure is optimally matched. Registration eliminates the positional deviation and geometric deformation in the image acquisition process, laying a spatial foundation for subsequent processing.

[0087] S402, using a histogram equalization method to perform color standardization on the registered image to obtain a target image.

[0088] Specifically, the embodiment of the present invention first performs grayscale normalization on the registered image and maps its grayscale value to the interval [0, 1]. Then, the grayscale histogram of the image is calculated and the frequency of occurrence of each grayscale level is counted. Next, the grayscale histogram is transformed by cumulative distribution function to obtain a grayscale histogram after equalization. Finally, according to the grayscale histogram after equalization, the grayscale values ​​of the image are mapped and interpolated to obtain a target image after color standardization. Histogram equalization increases the dynamic range of low-frequency grayscale levels by stretching the grayscale histogram, making the contrast and details of the image more obvious. At the same time, the distribution of the grayscale histogram after equalization is more uniform at each grayscale level, reducing the skewness and concentration of the grayscale distribution, which is beneficial to the quantitative analysis and threshold segmentation of the image.

[0089] During the implementation process, attention should be paid to the scope of application and parameter selection of histogram equalization. For medical images with relatively concentrated grayscale values ​​in the lesion area, such as tumor CT images, directly performing global histogram equalization may over-stretch the grayscale range, resulting in reduced contrast of normal tissues and blurred lesion boundaries. Therefore, a local adaptive histogram equalization method can be used to perform adaptive contrast enhancement based on the local grayscale distribution of the image. At the same time, in order to avoid grayscale distortion and artifacts during the histogram equalization process, the grayscale histogram after equalization can be smoothed and regularized, such as using median filtering, bilateral filtering and other methods to maintain the continuity and consistency of the grayscale distribution.

[0090] S103, performing multi-scale feature extraction on the target image to obtain a multi-scale feature map, and performing polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image;

[0091] Specifically, after obtaining the target image after standardization, an embodiment of the present invention performs multi-scale feature extraction on the target image. Specifically, a multi-scale convolutional neural network, such as ResNet, Inception, etc., is used to perform convolution and pooling operations on the target image layer by layer to extract feature maps of different scales and different abstract levels. For example, for a target image of 256*256, through 5 convolutional layers and pooling layers of different scales, 5 feature maps of 128*128, 64*64, 32*32, 16*16, and 8*8 can be obtained, whose sizes are halved in sequence. These feature maps are 128*128, 64*64, 32*32, 16*16, and 8*8, respectively. These feature maps depict the feature representation of the target image at different scales from low-level to high-level, from local to global. Through multi-scale feature extraction, the feature information of the target image at different scales can be fully explored, a more comprehensive and effective feature representation can be obtained, and the limitations of single-scale features can be overcome.

[0092] After obtaining the multi-scale feature map, the embodiment of the present invention further performs polyp region detection on the target image based on the multi-scale feature map. Specifically, a selective search algorithm is used to adaptively extract candidate regions from the multi-scale feature map. Selective search gradually merges similar regions from the bottom layer to the top layer through steps such as initial segmentation, region merging, and similarity calculation to generate candidate regions of different scales and positions. Compared with traditional sliding window and brute force search methods, selective search can adaptively generate candidate regions of different scales and shapes, avoiding a large amount of redundant and invalid searches, and improving the efficiency and accuracy of detection. Through selective search, a series of potential polyp candidate regions can be obtained from the multi-scale feature map.

[0093] Based on the above embodiment, as an optional embodiment, polyp region detection is performed on the target image based on the multi-scale feature map to obtain a polyp region image, including:

[0094] S501, extracting a candidate region containing a polyp from the multi-scale feature map based on a selective search method;

[0095] Specifically, the embodiment of the present invention first preprocesses the multi-scale feature map, such as smoothing filtering, edge enhancement, etc., to highlight the morphological and texture features of the polyp. Then, a selective search algorithm is used to perform regional segmentation and merging on the multi-scale feature map. The algorithm first uses an image over-segmentation algorithm, such as the Felzenszwalb algorithm, fast moving clustering, etc., to divide the image into a large number of small regions. Then, by defining similarity metrics between regions, such as color similarity, texture similarity, scale similarity, etc., adjacent similar regions are merged to generate larger-scale candidate regions. During the merging process, the level and granularity of the merge are controlled by the threshold of the similarity metric to generate a set of candidate regions at multiple scales. Finally, the candidate regions are screened and sorted, and several candidate regions that are most similar to the polyp features are selected according to the size, shape, texture and other features of the region as the focus of subsequent detection and identification.

[0096] During the implementation process, attention should be paid to the parameter setting and optimization of selective search. The selection of similarity metrics and weight allocation will directly affect the quality and quantity of candidate regions. For colorectal polyps, the shape characteristics of the region, such as circularity, convexity, etc., and texture characteristics, such as entropy, variance, etc., can be considered. At the same time, in order to improve the diversity and coverage of candidate regions, a variety of over-segmentation algorithms and similarity metrics can be used to generate a set of candidate regions with different characteristics, and then fuse and optimize them. In addition, prior knowledge can be used to screen and optimize candidate regions. For example, based on clinical experience such as the size range and location distribution of polyps, candidate regions that are too large or too small or abnormally located can be eliminated to improve the pertinence and efficiency of detection.

[0097] Through selective search, candidate regions containing polyps can be automatically extracted from multi-scale feature maps, providing key targets for subsequent detection and identification. This method makes full use of the multi-scale and multi-level features of the image, and can adaptively generate candidate regions of different sizes and positions, improving the flexibility and robustness of detection. At the same time, guided by similarity metrics and prior knowledge, candidate regions can effectively cover the real polyp area, reducing the risk of missed detection and false detection.

[0098] Based on the above embodiment, as an optional embodiment, extracting a candidate region containing a polyp from a multi-scale feature map based on a selective search method includes:

[0099] S601, performing initial segmentation on the multi-scale feature map to obtain a plurality of small-area multi-scale feature maps, and calculating the similarity between the small-area multi-scale feature map and the adjacent areas to obtain the adjacent similarity between each small-area multi-scale feature map and the adjacent small-area multi-scale feature map;

[0100] Specifically, an embodiment of the present invention uses a graph theory-based method to perform initial segmentation on a multi-scale feature map. First, the multi-scale feature map is regarded as an undirected graph, each pixel is used as a node of the graph, and the edges between adjacent pixels are established according to feature similarity. By setting a similarity threshold, the sparsity of the edges can be controlled to obtain graph structures of different granularities. Then, a graph segmentation algorithm, such as the Kruskal algorithm or the Prim algorithm based on the minimum spanning tree, is used to divide the graph into multiple connected subgraphs, each of which corresponds to a small area. During the segmentation process, by adjusting the similarity threshold, small areas of different sizes and numbers can be obtained, which not only ensures the consistency within the small area, but also avoids the computational burden caused by excessive segmentation. The small areas obtained by segmentation constitute the initial segmentation results of the multi-scale feature map, which are recorded as multiple small-area multi-scale feature maps.

[0101] After obtaining the multi-scale feature map of the small area, it is necessary to calculate the similarity between each small area and its adjacent areas, that is, the adjacent similarity. Adjacent areas refer to small areas that are adjacent in the image space, which are usually determined by judging the connectivity of the region boundaries. In order to calculate the adjacent similarity, it is necessary to extract the feature descriptor of each small area to comprehensively reflect the texture, color, shape and other characteristics of the area. Common feature descriptors include color histogram, gradient histogram, LBP texture feature, etc. Then, the similarity measurement method is used to calculate the similarity between the feature descriptors of the two small areas to obtain the adjacent similarity. Commonly used similarity metrics include Euclidean distance, cosine similarity, chi-square distance, etc. The appropriate measurement method is selected according to the type and distribution characteristics of the feature descriptor. By calculating the similarity between each small area and its adjacent areas, the adjacent similarity matrix between small areas is obtained, which reflects the similarity relationship between areas in the initial segmentation result.

[0102] S602, based on adjacent similarities, similar regions are merged from bottom to top to obtain a candidate region set, and based on prior knowledge of polyps, regions containing polyps are screened out from the candidate region set to obtain candidate regions containing polyps.

[0103] Specifically, an embodiment of the present invention uses a graph theory-based method to perform initial segmentation on a multi-scale feature map. First, the multi-scale feature map is regarded as an undirected graph, each pixel is used as a node of the graph, and the edges between adjacent pixels are established according to feature similarity. By setting a similarity threshold, the sparsity of the edges can be controlled to obtain graph structures of different granularities. Then, a graph segmentation algorithm, such as a minimum spanning tree, is used to divide the graph into multiple connected subgraphs, each of which corresponds to a small area. During the segmentation process, by adjusting the similarity threshold, small areas of different sizes and numbers can be obtained, which not only ensures the consistency within the small area, but also avoids the computational burden caused by excessive segmentation. The small areas obtained by segmentation constitute the initial segmentation results of the multi-scale feature map, which are recorded as multiple small-area multi-scale feature maps.

[0104] After obtaining the multi-scale feature map of the small area, it is necessary to calculate the similarity between each small area and its adjacent areas, that is, the adjacent similarity. Adjacent areas refer to small areas that are adjacent in the image space, which are usually determined by judging the connectivity of the region boundaries. In order to calculate the adjacent similarity, it is necessary to extract the feature descriptor of each small area to comprehensively reflect the texture, color, shape and other characteristics of the area. Common feature descriptors include color histogram, gradient histogram, LBP texture feature, etc. Then, the similarity measurement method is used to calculate the similarity between the feature descriptors of the two small areas to obtain the adjacent similarity. Commonly used similarity metrics include Euclidean distance, cosine similarity, chi-square distance, etc. The appropriate measurement method is selected according to the type and distribution characteristics of the feature descriptor. By calculating the similarity between each small area and its adjacent areas, the adjacent similarity matrix between small areas is obtained, which reflects the similarity relationship between areas in the initial segmentation result.

[0105] The initial segmentation of the multi-scale feature map and the calculation of adjacent similarities can lay the foundation for the regional merging of selective search. The initial segmentation ensures the fine-grained regional division and makes full use of local features. The adjacent similarity quantitatively characterizes the correlation between regions and provides a reference for the merging strategy. In the subsequent merging process, similar regions can be merged from the bottom up according to the adjacent similarity to obtain a set of regions of different scales and different semantics, and then the candidate regions containing polyps can be screened out to achieve the goal of selective search. This step fully exploits the local and regional features of the multi-scale feature map through graph segmentation and similarity calculation, provides important support for the extraction of candidate regions, and is one of the key links to achieve accurate polyp positioning.

[0106] After obtaining the multi-scale feature map of a small area and its adjacent similarity, an embodiment of the present invention merges similar areas from bottom to top according to the adjacent similarity to obtain a set of candidate areas, and based on the prior knowledge of polyps, screens out areas containing polyps in the candidate area set to obtain candidate areas of polyps. Area merging is the core step of selective search. By continuously merging similar areas, candidate areas of different scales and different semantics are generated, gradually approaching the true location of the target polyp. The introduction of prior knowledge can further improve the pertinence and effectiveness of candidate areas, exclude areas that obviously do not meet the characteristics of polyps, and reduce the amount of computation for subsequent processing.

[0107] Specifically, the embodiment of the present invention adopts a greedy merging strategy based on similarity to merge adjacent similar regions from bottom to top. First, all adjacent region pairs are sorted according to adjacent similarity, and the region pairs with the greatest similarity are selected for merging to generate new regions. Then, the similarity between the new region and its adjacent regions is calculated, and the adjacent similarity table is updated. The above process is repeated until the similarity is lower than the set threshold or the number of regions reaches a predetermined value. In the merging process, by setting a similarity threshold, the process and granularity of the merge are controlled, so that the merged region is more complete and consistent in semantics. At the same time, in order to improve the efficiency of merging, data structures such as priority queues and union-find sets can be used to optimize the selection and update of region pairs. After the merging is completed, a set of candidate regions containing regions of different scales and different positions is obtained, providing a variety of options for polyp positioning. Next, based on the prior knowledge of polyps, the regions that are more likely to contain polyps are screened out from the candidate region set. Prior knowledge comes from the observation and summary of a large amount of clinical polyp data, mainly including the size, shape, location and other characteristics of polyps. For example, polyps are usually round or oval, with a diameter between 3-10 mm, and are located on the surface of the colon mucosa. Using this prior knowledge, we can preliminarily screen the candidate regions, eliminate regions that are too large or too small, irregular in shape, or abnormal in location, and obtain a subset of candidate regions that are more likely to contain polyps. During the screening process, we can set thresholds or rules to quantify the constraints of prior knowledge on candidate regions and improve the degree of automation of screening. At the same time, in order to improve the robustness of screening, we can use fuzzy reasoning, probabilistic graphical models, and other methods to evaluate and rank candidate regions by integrating multiple prior knowledge to obtain candidate regions with higher confidence.

[0108] S502, extracting features from each candidate region containing a polyp to obtain a multi-scale feature vector, and judging the multi-scale feature vector using a preset classifier to obtain a judgment result;

[0109] Specifically, the embodiment of the present invention first preprocesses the candidate region, such as image enhancement, denoising, etc., to improve the quality and stability of feature extraction. Then, a variety of feature extraction algorithms are used to extract the features of the candidate region from different scales and angles. In terms of morphological features, geometric parameters such as the area, perimeter, circularity, and convexity of the candidate region are extracted to characterize the size and shape characteristics of the polyp. In terms of texture features, methods such as grayscale co-occurrence matrix and Gabor filtering are used to extract statistical parameters such as texture energy, entropy, and inertia of the candidate region to characterize the roughness and regularity of the polyp surface. In terms of color features, color parameters such as the average color and hue distribution of the candidate region are extracted to characterize the color characteristics of the polyp. Through multi-scale and multi-dimensional feature extraction, a high-dimensional feature vector of the candidate region is obtained to fully characterize its visual characteristics. Next, the multi-scale feature vector of the candidate region is input into a preset classifier for polyp discrimination. The classifier learns and classifies the feature vector through machine learning algorithms such as support vector machines, random forests, neural networks, etc. The training samples of the classifier come from real polyp data annotated by clinical experts. By learning the feature distribution of the samples, the mapping relationship between the features and the polyp categories is established. In the discrimination process, the classifier maps the feature vector of the candidate area to the category label of polyp or non-polyp according to the learned mapping relationship, and obtains the judgment result of whether the candidate area is a real polyp. In order to improve the accuracy and reliability of the discrimination, multiple classifiers can be used for integrated discrimination, and the judgment results of multiple classifiers can be combined by voting, weighting, etc. to obtain the final discrimination result.

[0110] S503, determining a candidate region with polyps based on the judgment result, and performing regression processing on the candidate region with polyps through a preset regressor to obtain a polyp region image.

[0111] Specifically, the embodiment of the present invention first selects candidate areas with confidence levels higher than a preset threshold as candidate areas with polyps based on the judgment results of the classifier. The setting of the threshold requires a balance between the detection rate and the false alarm rate, and is usually adjusted according to clinical needs and sample characteristics to ensure that as many real polyps as possible are detected while controlling the false alarm rate within an acceptable range. Then, regression processing is performed on each candidate area with polyps. The regression processing uses machine learning algorithms, such as bounding box regression and pixel-level regression, to refine the position and range of the candidate area to make it more consistent with the boundary of the real polyp. The training samples of the regressor come from polyp data that are finely annotated by clinical experts. By learning the boundary features and position mapping relationship of the samples, a mapping model between the candidate area and the precise polyp area is established. During the regression process, the regressor adjusts and optimizes the bounding box or pixel label of the candidate area according to the learned mapping model to obtain a regional image that accurately outlines the boundary of the polyp.

[0112] S104, inputting a preset contour segmentation image into the polyp region image to obtain a polyp contour image;

[0113] Specifically, after obtaining the polyp region image, the embodiment of the present invention further performs contour segmentation on the polyp region image to obtain more detailed polyp contour information. Specifically, the polyp region image is input into a preset contour segmentation model, and the precise contour of the polyp region is obtained through operations such as feature extraction, pixel classification, and edge optimization. The contour segmentation model can use semantic segmentation networks such as U-Net and DeepLab, and through end-to-end learning, directly predict the category (polyp or background) to which each pixel belongs from the polyp region image to obtain a binary polyp contour image. Compared with traditional edge detection and contour extraction methods, contour segmentation based on deep learning can better utilize the contextual information of the image, adaptively extract the contours of polyps with different morphologies and textures, and has stronger robustness and generalization capabilities.

[0114] In the process of contour segmentation, in order to further improve the segmentation accuracy, the embodiments of the present invention introduce an attention mechanism and a multi-scale fusion strategy. Specifically, in the encoder part of the contour segmentation network, the attention mechanism is used to adaptively weight the features of different regions, highlight the significant features of the polyp region, and suppress the interference information of the background region. In the decoder part, through multi-scale feature fusion, the feature information at different scales is comprehensively utilized, which can not only capture the overall morphology of the polyp, but also retain the local fine contours. Through the attention mechanism and multi-scale fusion, the accuracy of polyp contour segmentation can be effectively improved, especially for polyp regions with irregular morphology and blurred edges, more accurate contour extraction can be achieved.

[0115] S105, obtaining clinical information of the patient and performing feature extraction on the contour image to obtain polyp feature data;

[0116] Specifically, after obtaining the polyp contour image, the embodiment of the present invention further obtains the patient's clinical information and performs feature extraction on the contour image to obtain comprehensive, multi-dimensional polyp feature data. Specifically, the patient's age, gender, family history, medical history and other clinical information are obtained through electronic medical record systems or manual entry, etc., as important reference indicators for polyp risk prediction. At the same time, multi-level and multi-angle feature extraction is performed on the polyp contour image, including morphological features, texture features, color features, etc.

[0117] In terms of morphological features, the area, perimeter, circularity, irregularity and other shape features of the polyp are calculated through contour analysis and morphological operations to characterize the size and morphological characteristics of the polyp. In terms of texture features, gray-level co-occurrence matrix, local binary pattern and other methods are used to extract the texture information of the polyp surface, such as roughness, contrast, entropy, etc., to reflect the details and unevenness of the polyp surface. In terms of color features, the color space conversion and statistical analysis of the polyp area are performed, and the color histogram, color moment, etc. are calculated to characterize the color distribution and abnormality of the polyp. Through multi-dimensional feature extraction, the morphology, texture, color and other characteristics of the polyp can be fully characterized.

[0118] S106, inputting the polyp characteristic data and the patient's clinical information into a preset polyp lesion risk prediction model to obtain a polyp lesion risk probability value.

[0119] Specifically, after obtaining the polyp characteristic data and the patient's clinical information, the embodiment of the present invention inputs them into a preset polyp lesion risk prediction model to obtain a quantitative polyp lesion risk probability value. Specifically, the polyp lesion risk prediction model is based on large-scale clinical data and expert knowledge, and is obtained through training of a machine learning algorithm. It can comprehensively consider the polyp characteristics and the patient's clinical information to make an intelligent prediction of the risk of malignant transformation of polyps. The prediction model can use classic machine learning algorithms such as logistic regression, support vector machine, random forest, etc., or it can use more complex and nonlinear models such as deep neural networks. Through end-to-end learning, it can directly mine implicit patterns related to risks from the characteristic data, and has stronger expression and generalization capabilities.

[0120] In the process of risk prediction, the prediction model adaptively combines polyp characteristics and clinical information through feature selection and feature fusion, assigns different weights to different features, and highlights key features related to risk. For example, morphological features such as the size and irregularity of the polyp, as well as clinical factors such as advanced age and family history of colorectal cancer, are usually closely related to the risk of malignant transformation of polyps and are given greater weights in the prediction model. Some irrelevant or redundant features will be automatically filtered out or downgraded to avoid introducing noise and interference. Through feature selection and fusion, the prediction model can capture the risk pattern of polyps more accurately and robustly, and improve the reliability of the prediction.

[0121] When constructing a polyp lesion risk prediction model, this method uses the patient's pathological conclusion as a label, and the polyp characteristic data and clinical information as input to train machine learning models, such as logistic regression, support vector machine, random forest, etc. By learning a large amount of labeled training data, the model can automatically discover the association rules and discrimination patterns between polyp characteristics, clinical information and lesion risks. During the training process, the model optimizes the objective function and adjusts the model parameters to make the prediction results as consistent as possible with the pathological labels. After sufficient training and verification, the model can learn the general rules and individual differences in the risk prediction of polyp lesions, and has the ability to perform risk assessment and prognosis prediction for new patient samples.

[0122] When new patient data is input into the trained risk prediction model, the model calculates the risk probability value of polyp lesions based on the patient's polyp characteristics and clinical information, taking into account the morphological and epigenetic characteristics of the polyps and the individual factors of the patient. The risk probability value quantifies the possibility of malignant transformation of the patient's polyps. In summary, this method uses polyp contour segmentation and feature extraction to finely depict the morphological and epigenetic characteristics of polyps, and combines the patient's clinical information to construct a comprehensive and personalized risk prediction model for polyp lesions.

[0123] On the other hand, the present invention also provides a colorectal polyp identification and classification system, such as Figure 2 The system comprises:

[0124] The image acquisition module 1 is used to take a picture of the colorectal area of ​​the patient through an endoscope device, obtain a multi-source image through exploration, and perform quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image. When the assessment value is less than a preset threshold, the multi-source image is subjected to image enhancement processing to obtain an image-enhanced multi-source image;

[0125] The image preprocessing module 2 is used to perform image extraction based on the multi-source image after image enhancement to obtain a derived image, and perform standardization processing on the derived image and the multi-source image after image enhancement to obtain a target image;

[0126] The polyp region detection module 3 is used to perform multi-scale feature extraction on the target image to obtain a multi-scale feature map, and perform polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image;

[0127] The polyp segmentation module 4 is used to input a preset contour segmentation image into the polyp region image to obtain a polyp contour image;

[0128] The polyp characteristic data acquisition module 5 is used to acquire the patient's clinical information and perform feature extraction on the contour image to obtain polyp characteristic data;

[0129] The lesion risk prediction module 6 is used to input the polyp characteristic data and the patient's clinical information into a preset polyp lesion risk prediction model to obtain a polyp lesion risk probability value.

[0130] Please refer to Figure 3 The application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0131] The communication bus 302 is used to realize the connection and communication between these components.

[0132] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0133] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0134] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field~Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0135] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read~Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage system located away from the aforementioned processor 301. Refer to Figure 3 The memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a flocculant addition analysis method.

[0136] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program of the colorectal polyp identification and classification method stored in the memory 305. When executed by one or more processors 301, the electronic device 300 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application. In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0137] In the several implementations provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The road assessment method of the embodiment shown in the figure can be found in the specific implementation process. Figure 1 The specific description of the illustrated embodiment will not be repeated here.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0142] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0143] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for identifying and classifying colorectal polyps, characterized in that: The method comprises: Taking photos of the colorectal area of ​​the patient under different lighting conditions by an endoscopic device to obtain a multi-source image, and performing quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image, and when the assessment value is less than a preset threshold, performing image enhancement processing on the multi-source image to obtain an image-enhanced multi-source image; Performing image extraction based on the image-enhanced multi-source image to obtain a derived image, and performing standardization processing on the derived image and the image-enhanced multi-source image to obtain a target image; Performing multi-scale feature extraction on the target image to obtain a multi-scale feature map, and performing polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image; Inputting a preset contour segmentation image into the polyp region image to obtain a polyp contour image; Acquiring clinical information of the patient, and performing feature extraction on the contour image to obtain polyp feature data; The polyp characteristic data and the clinical information of the patient are input into a preset polyp lesion risk prediction model to obtain a polyp lesion risk probability value.

2. The method according to claim 1, characterized in that The performing image enhancement processing on the multi-source images to obtain the multi-source images after image enhancement includes: Performing super-resolution reconstruction on the multi-source image to obtain a high-resolution multi-source image; Defogging the high-resolution multi-source image by using a dark channel prior method to obtain a defogged multi-source image; The spectral enhancement process is performed on the dehazed multi-source image to obtain the image-enhanced multi-source image.

3. The method according to claim 1, characterized in that The performing quality assessment on the multi-source images to obtain quality assessment values ​​of the multi-source images includes: Image parameters are extracted from the multi-source images to obtain image parameters, and quality assessment is performed based on the image parameters and preset quality indicators to obtain quality assessment values ​​of the multi-source images.

4. The method according to claim 1, characterized in that: The step of performing image extraction based on the multi-source image after image enhancement to obtain a derived image comprises: Performing grayscale image extraction on the multi-source image to obtain a grayscale image; Calculating the gradient magnitude and the gradient direction of the multi-source image to obtain the gradient magnitude and the gradient direction of the multi-source image, and constructing a gradient map based on the gradient magnitude and the gradient direction of the multi-source image; Performing morphological transformation on the multi-source images to obtain a morphological image; Performing multi-scale decomposition on the multi-source images by using wavelet basis functions to obtain a wavelet coefficient map; Performing saliency detection on the multi-source image by using a frequency domain analysis method to obtain salient regions in the multi-source image, and marking the salient regions in the multi-source image to obtain a saliency map; The grayscale image, the gradient image, the wavelet coefficient image and the saliency image are used as the derived image.

5. The method according to claim 1, characterized in that The step of performing standardization processing on the derived image and the multi-source image after image enhancement to obtain a target image includes: Performing image registration on the derived image and the multi-source image after image enhancement to obtain a registered image; The color of the registered image is standardized by using a histogram equalization method to obtain the target image.

6. The method according to claim 1, characterized in that The performing polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image includes: Extracting a candidate region containing a polyp from the multi-scale feature map based on a selective search method; Performing feature extraction on each candidate region containing polyps to obtain a multi-scale feature vector, and judging the multi-scale feature vector by a preset classifier to obtain a judgment result; Based on the judgment result, a candidate region with polyps is determined, and regression processing is performed on the candidate region with polyps through a preset regressor to obtain the polyp region image.

7. The method according to claim 6, characterized in that The method of extracting a candidate region containing a polyp from the multi-scale feature map based on a selective search method includes: Performing an initial segmentation on the multi-scale feature map to obtain a plurality of small-area multi-scale feature maps, and calculating the similarity between the small-area multi-scale feature map and an adjacent area to obtain the adjacent similarity between each small-area multi-scale feature map and an adjacent small-area multi-scale feature map; Based on the adjacent similarities, similar regions are merged from bottom to top to obtain a candidate region set, and based on prior knowledge of polyps, regions containing polyps are screened out from the candidate region set to obtain the candidate regions containing polyps.

8. A colorectal polyp identification and classification system, characterized in that: The system comprises: An image acquisition module is used to take a picture of the colorectal area of ​​the patient through an endoscope device to obtain a multi-source image through exploration, and to perform quality assessment on the multi-source image to obtain a quality assessment value of the multi-source image. When the assessment value is less than a preset threshold, the multi-source image is subjected to image enhancement processing to obtain an image-enhanced multi-source image; An image preprocessing module, used for performing image extraction based on the image-enhanced multi-source image to obtain a derived image, and performing standardization processing on the derived image and the image-enhanced multi-source image to obtain a target image; A polyp region detection module, used to perform multi-scale feature extraction on the target image to obtain a multi-scale feature map, and perform polyp region detection on the target image based on the multi-scale feature map to obtain a polyp region image; A polyp segmentation module, used for inputting a preset contour segmentation image into the polyp region image to obtain a polyp contour image; A polyp feature data acquisition module is used to acquire clinical information of the patient and perform feature extraction on the contour image to obtain polyp feature data; The lesion risk prediction module is used to input the polyp characteristic data and the clinical information of the patient into a preset polyp lesion risk prediction model to obtain a polyp lesion risk probability value.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor, a memory and a transceiver, the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

Images