A data augmentation method for colonoscopy images

Abstract

The application discloses a colonoscopy image data enhancement method, and belongs to the technical field of computer vision, which comprises the following steps: acquiring multiple original colonoscopy images, fusing the gray information and the saturation information to obtain a set of coordinate points of the reflection area of the original colonoscopy images; calculating the set of coordinate points of the small reflection area from the set of coordinate points of the reflection area and restoring the set of coordinate points to obtain a small reflection area restoration image; converting the source image and the target domain image to the LAB color space; calculating the standard deviation and the mean value of each color channel of the source image and the target domain image except the reflection area and the dark area; migrating the color domain of the source image to the target domain corresponding to the target domain image; restoring the dark area of the migration image to obtain a color domain migration enhancement image; and composing the original colonoscopy image, the small reflection area restoration image and the color domain migration enhancement image into a colonoscopy enhancement image data set, wherein the colonoscopy enhancement image data set is used for subsequent image analysis.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and specifically relates to a data augmentation method for colonoscopy images. Background Technology

[0002] Colonoscopy is an important standard for screening intestinal diseases, and colonoscopy screening currently relies mainly on manual examination by doctors. However, due to limitations such as doctors' experience, physical strength, and equipment imaging, manual examination still has a significant possibility of missing lesions.

[0003] With the development of artificial intelligence (AI) technology, the analytical performance of convolutional neural networks (CNNs) in the field of vision has been significantly improved, even surpassing human performance in some visual analysis tasks. Therefore, CNNs are widely used for intelligent analysis of colonoscopy lesions, such as polyp detection, classification, and bleeding detection. Although the corresponding algorithms have demonstrated certain analytical performance, such as lesion recall rate, there is still a significant gap compared to expert-level examination results. A major constraint of existing AI algorithms for clinical colonoscopy analysis is their adaptability to cross-domain scenarios. Cross-domain mainly refers to the differences in images generated when images are acquired by different devices. AI algorithms primarily learn from manually constructed image datasets, and single datasets can easily lead AI algorithms into local optima. Existing image data augmentation algorithms mainly use methods such as image rotation, cropping, scaling, and random color transformation, but there is a lack of technical methods specifically tailored to the characteristics of colonoscopy images.

[0004] In summary, in colonoscopy scenarios, when artificial intelligence analyzes images of colonoscopy lesions, the lack of a single reference image data leads to poor stability and low reliability of the analysis results. Summary of the Invention

[0005] The purpose of this invention is to provide a data augmentation method for colonoscopy images, which can solve the technical problems of single reference image data, poor stability and low reliability of analysis results in the process of existing artificial intelligence colonoscopy lesion analysis.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] This invention provides a data augmentation method for colonoscopy images, comprising:

[0008] S101: Acquire multiple raw colonoscopy images, and convert the color space of each raw colonoscopy image to grayscale space and hue-saturation-brightness color space respectively to obtain the grayscale information and saturation information of the raw colonoscopy image;

[0009] S102: The grayscale information and saturation information are fused, and the set of coordinate points of the reflective area of ​​the original colonoscopy image is obtained according to the first preset intensity threshold and the second preset intensity threshold.

[0010] S103: Calculate the set of coordinate points of the small reflective area from the set of coordinate points of the reflective area according to the third preset intensity threshold;

[0011] S104: Restore the coordinates of the small reflective area to obtain the restored image of the small reflective area;

[0012] S105: Select the source image and the target domain image, and extract the reflective and dark regions by fusing multiple feature spaces. The source image is the original colonoscopy image or the image restored from the small reflective area.

[0013] S106: Convert the source image and the target domain image to the LAB color space to obtain the component features of the source image and the component features of the target domain image. The target domain image has a higher resolution than the source image and has the same samples as the source image. The component features include the L component intensity value, the A component intensity value and the B component intensity value.

[0014] S107: Based on the component features of the source image and the component features of the target domain image, calculate the standard deviation and mean of each color channel in the image region other than the reflective and dark regions in the source image, and calculate the standard deviation and mean of each color channel in the image region other than the reflective and dark regions in the target domain image.

[0015] S108: Based on the standard deviation and mean of the source image and the standard deviation and mean of the target image, the color domain of the source image is transferred to the target domain corresponding to the target image to obtain the transferred image;

[0016] S109: Restore the dark areas of the transferred image to obtain a color gamut transferred enhanced image;

[0017] S110: The original colonoscopy image, the small reflective area recovery image, and the color domain transfer enhanced image are combined into a colonoscopy enhanced image data set, which is used for subsequent colonoscopy image analysis.

[0018] In this embodiment of the invention, grayscale and saturation information from multiple original colonoscopy images are fused, and a set of coordinate points for small reflective areas is extracted using a preset intensity threshold. These points are then used to restore the small reflective areas, resulting in a restored image of the original colonoscopy images. This data enhancement increases the size of the colonoscopy image enhancement dataset. Furthermore, by selecting the original colonoscopy images and the restored small reflective area images and the target domain image for multi-feature fusion, reflective and dark areas are extracted and converted to the LAB color space. The original colonoscopy images and the restored small reflective area images are then color-domain transferred to the target domain image, followed by dark area restoration, resulting in a color-domain-transferred enhanced image. This further expands the size of the colonoscopy enhanced image dataset. The resulting colonoscopy enhanced image dataset provides richer reference image data for artificial intelligence analysis of colonoscopy images, improving the stability and reliability of the analysis results. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of a data enhancement method for colonoscopy images provided in an embodiment of the present invention.

[0020] The realization of the objective, functional characteristics and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] The data enhancement method for colonoscopy images of the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0023] Reference Figure 1 The diagram illustrates a flowchart of a data augmentation method for colonoscopy images provided by an embodiment of the present invention.

[0024] This invention provides a data augmentation method for colonoscopy images, comprising:

[0025] S101: Acquire multiple raw colonoscopy images, and convert the color space of each raw colonoscopy image to grayscale space and hue-saturation-brightness color space respectively to obtain the grayscale information and saturation information of the raw colonoscopy image.

[0026] In one possible implementation, S101 specifically includes:

[0027] S1011: Acquire colonoscopy video data;

[0028] S1012: Capture frames from colonoscopy video data to obtain multiple raw colonoscopy images.

[0029] S102: The grayscale information and saturation information are fused, and the set of coordinate points of the reflective area of ​​the original colonoscopy image is obtained according to the first preset intensity threshold and the second preset intensity threshold.

[0030] In one possible implementation, S102 specifically includes:

[0031] S1021: Obtain the set of coordinate points P of the reflective area in the original colonoscopy image according to the following formula 1:

[0032] P = {x | x ∈ D, G(x) > Th1 & S(x) < Th2} Formula 1

[0033] Where P represents the set of coordinate points of the reflective area, D represents the set of coordinate points of the original colonoscopy image, G(x) represents the gray intensity value of the gray space at coordinate x in the original colonoscopy image, S(x) represents the intensity value of the saturation space at coordinate x in the original colonoscopy image, Th1 represents the first preset intensity threshold, Th2 represents the second preset intensity threshold, and the symbol & represents the logical AND operation.

[0034] Optionally, the first preset intensity threshold and the second preset intensity threshold are set to 240 and 0.3, respectively.

[0035] S103: Calculate the set of coordinate points of the small reflective area from the set of coordinate points of the reflective area according to the third preset intensity threshold.

[0036] In one possible implementation, S103 specifically includes:

[0037] S1031: Calculate the set of coordinate points S of the small reflective area:

[0038]

[0039] Where S represents the set of coordinate points of the small reflective area, P sub This represents a connected subset of the coordinate points of the reflective area in the original colonoscopy image. The symbol |.| is used to calculate the number of coordinate points contained in the set, and Th3 represents the third preset intensity threshold.

[0040] Optionally, the third preset intensity threshold is set to 1000.

[0041] S104: Restore the coordinates of the small reflective area to obtain the restored image of the small reflective area.

[0042] In a possible implementation, S104 specifically includes:

[0043] S1041: Scan the original colonoscopy image from top to bottom and from left to right;

[0044] S1042: When the scanned pixel point coordinates belong to the set of coordinates of the small specular region, fill the scanned pixel points, and use the filled scanned pixel points as sub-normal pixel points.

[0045] Among them, S1042 specifically includes:

[0046] S1042A: With the scanned pixel point as the center, expand the search area layer by layer in a "return" shape. The first-layer search area is the 8-neighborhood of the scanned pixel point, and the second-layer search area is one circle outside the first-layer search area;

[0047] S1042B: When K normal / sub-normal pixel points are found, calculate the mean color vector of the K normal / sub-normal pixel points, and fill the mean color vector into the currently scanned pixel point to be filled.

[0048] Optionally, the preset value of K is 6.

[0049] It should be noted that by filling the scanned pixel points within the range of the set of coordinates of the small specular region, the color domain effect of the small specular region is made close to that of the surrounding neighborhood pixel points, and the deviation between the restored image of the small specular region and the display effect of the real image is smaller.

[0050] S105: Select the source image and the target domain image, and extract the specular region and the dark region by fusing multiple feature spaces. Among them, the source image is the original colonoscopy image or the restored image of the small specular region.

[0051] In a possible implementation, S105 specifically includes:

[0052] S1051: The method for calculating the set of coordinate points of the dark region of the source image and the target domain image is: <00​​​​​​​​​​​​​​​​={x|x∈D, G(x)<Th4} Formula 4

[0057] Among them, P d This represents the set of coordinate points of the acquired dark region, D is the set of coordinate points of the entire target domain image, G(x) is the gray intensity value at coordinate x in the target domain image, and Th4 is the fourth preset intensity value.

[0058] It should be noted that the process of extracting the reflective areas of the source image and the target domain image is the same as the method of extracting the reflective areas of the original colonoscopy image. The target of extraction is the coordinate set of the corresponding source image or target domain image.

[0059] S106: Convert the source image and the target domain image to the LAB color space to obtain the component features of the source image and the component features of the target domain image. The target domain image has a higher resolution than the source image and has the same samples as the source image. The component features include the L component intensity value, the A component intensity value and the B component intensity value.

[0060] It should be noted that the number of pixels in the target domain image sample is N times that of the source image sample.

[0061] S107: Based on the component features of the source image and the component features of the target domain image, calculate the standard deviation and mean of each color channel in the image region other than the reflective and dark regions in the source image, and calculate the standard deviation and mean of each color channel in the image region other than the reflective and dark regions in the target domain image.

[0062] In one possible implementation, S107 specifically includes:

[0063] S1071: Select an original colonoscopy image or a reconstructed image of a small reflective area. The source image variance and source image mean are calculated as follows: the standard deviation of each color channel in the image region excluding reflective and dark areas in the source image.

[0064] Source image standard deviation:

[0065]

[0066] Wherein, sets L(x), A(x), and B(x) represent the L component intensity value, A component intensity value, and B component intensity value of the corresponding pixel set in the LAB color space, respectively; sets σ(L(x)), σ(A(x)), and σ(B(x)) represent the variances of sets L(x), A(x), and B(x), respectively; sets σ(L(x)), σ(A(x)), and σ(B(x)) constitute the L component intensity value, A component intensity value, and B component intensity value of the variance of each color channel of the source image, respectively; and x represents all pixels in the source image except for the reflective and dark areas.

[0067] Source image mean:

[0068]

[0069] Where μ(L(x)), μ(A(x)) and μ(B(x)) represent the calculation of the mean of sets L(x), A(x) and B(x), respectively, and x represents the pixel points in the source image after removing both the reflective and dark areas of the source image. The sets μ(L(x)), μ(A(x)) and μ(B(x)) constitute the L component intensity value, A component intensity value and B component intensity value of the mean of each color channel of the source image, respectively.

[0070] S1072: Select N target domain images, and calculate the variance and mean of the target domain images respectively;

[0071] Standard deviation of target domain image:

[0072]

[0073] Where x represents the pixel in the target domain image after removing the reflective and dark areas, and the sets σ(L(x)), σ(A(x)), and σ(B(x)) constitute the L component intensity value, A component intensity value, and B component intensity value of the variance of each color channel of the target domain image, respectively. i P i and P di Let represent the set of coordinate points of the entire target domain image, the set of coordinate points of the reflected area of ​​the target domain image, and the set of coordinate points of the dark area of ​​the target domain image, respectively.

[0074] Mean of the target domain image:

[0075]

[0076] Among them, the sets μ(L(x)), μ(A(x)) and μ(B(x)) constitute the L component intensity value, A component intensity value and B component intensity value of the mean of each color channel of the target domain image, respectively.

[0077] S108: Based on the standard deviation and mean of the source image and the standard deviation and mean of the target image, the color domain of the source image is transferred to the target domain corresponding to the target image to obtain the transferred image.

[0078] In one possible implementation, S108 specifically includes:

[0079]

[0080] Where T(x) represents the L, A, and B component intensity values ​​of the transferred image at coordinate x in the LAB color space, σ0, σ1, and σ2 represent the L, A, and B component intensity values ​​of the variance of each color channel in the source image, respectively, and μ0, μ1, and μ2 represent the L, A, and B component intensity values ​​of the mean of each color channel in the source image, respectively. M0 ,、σ M1 and σ M2 Let μ represent the intensity values ​​of the L component, A component, and B component of the variance of each color channel in the target domain image, respectively. M0 μ M1 and μ M2 These represent the L component intensity value, A component intensity value, and B component intensity value of the mean values ​​of each color channel in the target domain image, respectively.

[0081] In this process, the background pixels in the target image and the source image do not belong to any object category. The main task is to perform color domain migration on the sample data.

[0082] It should be noted that excluding reflective and dark areas before migrating the color gamut of the source image to the corresponding target gamut of the target image can effectively avoid the influence of special image imaging areas on the resulting migrated image during the image migration process.

[0083] For colonoscopy scenarios, the imaging characteristics of colonoscopy images are fully considered. The dataset is enriched mainly through color gamut directional migration to avoid random color changes from deviating from the actual imaging characteristics, thus laying the foundation for obtaining a more stable and reliable intelligent analysis model for colonoscopy images.

[0084] S109: Restore the dark areas of the migrated image to obtain a color gamut migrated enhanced image.

[0085] It should be noted that restoring a migrated image means filling the regions in the migrated image that belong to the set of coordinate points of dark areas with pixels.

[0086] Optionally, during the filling process, when the 20th normal or subnormal pixel is found, the average color vector of the 20 found pixels is calculated, and the average color vector is filled into the scanned pixel to be filled.

[0087] S110: The original colonoscopy image, the small reflective area recovery image, and the color domain transfer enhancement image are combined to form an enhanced image data set.

[0088] The enhanced image dataset is used for subsequent image analysis.

[0089] It should be noted that the original colonoscopy images obtained, along with the restored images of small reflective areas and color-enhanced images obtained based on the original colonoscopy images, can provide a rich dataset for artificial intelligence to use as a reference when analyzing lesions in colonoscopy images.

[0090] In this embodiment of the invention, grayscale and saturation information from multiple original colonoscopy images are fused, and a set of coordinate points for small reflective areas is extracted using a preset intensity threshold. These points are then used to restore the small reflective areas, resulting in a restored image of the original colonoscopy images. This data enhancement increases the size of the colonoscopy image enhancement dataset. Furthermore, by selecting the original colonoscopy images and the restored small reflective area images and the target domain image for multi-feature fusion, reflective and dark areas are extracted and converted to the LAB color space. The original colonoscopy images and the restored small reflective area images are then color-domain transferred to the target domain image, followed by dark area restoration, resulting in a color-domain-transferred enhanced image. This further expands the size of the colonoscopy enhanced image dataset. The resulting colonoscopy enhanced image dataset provides richer reference image data for artificial intelligence analysis of colonoscopy images, improving the stability and reliability of the analysis results.

[0091] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A data augmentation method of colonoscopy images, characterized in that, The method comprises: S101: Obtain a plurality of original colonoscopy images, convert the color space of each of the original colonoscopy images to a gray space and a hue-saturation-brightness color space respectively, and obtain gray information and saturation information of the original colonoscopy images; S102: Fuse the gray information and the saturation information, and obtain a set of coordinates of a reflective light area of the original colonoscopy images according to a first preset intensity threshold and a second preset intensity threshold; S103: Calculate a set of coordinates of a small reflective light area from the set of coordinates of the reflective light area according to a third preset intensity threshold; S104: Restore the coordinates of the small reflective light area to obtain a small reflective light area restoration image; S105: Select a source image and a target domain image, and extract a reflective light area and a dark color area by fusing a plurality of feature spaces, wherein the source image is the original colonoscopy image or the small reflective light area restoration image; S106: Convert the source image and the target domain image to a LAB color space to obtain component features of the source image and component features of the target domain image, wherein the target domain image has a higher resolution than the source image and has the same sample as the source image, and the component features comprise L component intensity values, A component intensity values and B component intensity values; S107: Calculate the standard deviation and the mean of each color channel of an image area other than the reflective light area and the dark color area in the source image according to the component features of the source image and the component features of the target domain image, and calculate the standard deviation and the mean of each color channel of an image area other than the reflective light area and the dark color area in the target domain image; S108: Migrate the color domain of the source image to a target domain corresponding to the target domain image according to the standard deviation and the mean of the source image and the standard deviation and the mean of the target domain image to obtain a migration image; S109: Restore the dark color area of the migration image to obtain a color domain migration enhanced image; S110: Group the original colonoscopy image, the small reflective light area restoration image and the color domain migration enhanced image to form a colonoscopy enhanced image data set, wherein the colonoscopy enhanced image data set is used for subsequent colonoscopy image analysis.

2. The image data enhancement method of claim 1, wherein, The S101 specifically comprises: S1011: Obtain colonoscopy video data; S1012: Frame the colonoscopy video data to obtain a plurality of original colonoscopy images.

3. The image data enhancement method of claim 1, wherein, The S102 specifically comprises: S1021: Obtain a set of coordinates of a reflective light area of the original colonoscopy image according to the following formula 1: wherein P represents the set of coordinate points of the reflective region, D represents the set of coordinate points of the original colonoscopy image, G(x) represents the intensity value of the gray scale space at coordinate x point x in the original colonoscopy image, S(x) represents the intensity value of the saturation space at coordinate x point x in the original colonoscopy image, wherein represents the first preset intensity threshold value, represents the second preset intensity threshold value, and the symbol & represents a logical AND operation.

4. The image data enhancement method of claim 1, wherein, The S103 specifically comprises: S1031: Calculate a set of coordinates of a small reflective light area: wherein S represents a set of coordinate points of a small reflection area, represents a connected subset of the set of coordinate points of the reflection area of the original colonoscopy image, and the symbol is used to calculate the number of coordinate points contained in the set, represents a third preset intensity threshold, and P represents the set of coordinate points of the reflection area.

5. The image data enhancement method of claim 1, wherein, The S104 specifically comprises: S1041: Scan the original colonoscopy image from top to bottom and from left to right; S1042: In the case that the scanned pixel point coordinates belong to the set of coordinates of the small reflective light area, fill the scanned pixel points, and take the filled scanned pixel points as secondary normal pixel points.

6. The image data enhancement method of claim 5, wherein, The S1042 specifically comprises: S1042A: expanding the search region layer by layer in a "back" shape with the scanning pixel point as the center, wherein the first layer is the 8-neighborhood of the pixel point, and the second layer is the first layer search region expanded by one circle; S1042B: in the case of searching K normal or sub-normal pixel points, calculating the color vector mean of the K normal or sub-normal pixel points, and filling the color vector mean to the Kth to-be-filled highlight pixel point.

7. The image data enhancement method of claim 6, wherein, The preset value of K is 6.

8. The image data enhancement method of claim 1, wherein, The S105 specifically includes: S1051: the method for calculating the source image dark color region coordinate point set is: wherein, represents a set of acquired dark region coordinate points, D represents a set of full image coordinate points of the source image, G(x) represents a gray intensity value at a pixel coordinate point x of the source image, and wherein is a fourth preset intensity value. The method for calculating the target domain image dark color region coordinate point set is: wherein, represents a set of acquired dark region coordinate points, is a set of full image coordinate points of the target domain image, is a pixel point coordinate of the target domain image x gray intensity value, wherein is a fourth preset intensity value.

9. The image data enhancement method of claim 8, wherein, The S107 specifically includes: S1071: selecting an original colonoscopy image or a small highlight area recovery image, and the calculation methods of the source image standard deviation and the source image mean are respectively: the standard deviation of each color channel of the image region of the source image except the highlight region and the dark color region; The source image standard deviation: wherein the set L(x ), A ( x ), B ( x ) respectively represent the LAB color space component intensity values, A component intensity values, B component intensity values of the corresponding pixel point set, the set and respectively represent the standard deviations of the set L ( x ), A ( x ) and B ( x ), the set and respectively constitute the L component intensity values, A component intensity values and B component intensity values of the standard deviations of each color channel of the source image, x P represents the set of coordinates of the light reflection region, and P represents the set of coordinates of the dark color region. The source image mean: wherein, and respectively represent the mean values of the calculated sets L ( x ), A ( x ), B ( x ), x represents the pixel point coordinates of the source image excluding both the light reflection area and the dark color area of the source image, the set and respectively constitute the L component intensity value, the A component intensity value and the B component intensity value of the mean value of each color channel of the source image; S1072: selecting N target domain images, and the calculation methods of the target domain image standard deviation and the target domain image mean are respectively; The target domain image standard deviation: wherein, x represents the pixel point coordinates of the target domain image excluding the highlight area and the dark color area of the target domain image, and the set and respectively constitute the L component intensity value, the A component intensity value and the B component intensity value of the standard deviation of each color channel of the target domain image, and respectively represent the full image coordinate point set of the i-th target domain image, the highlight area image coordinate point set of the i-th target domain image, and the dark color area image coordinate point set of the i-th target domain image. The target domain image mean: wherein the set and constitute the L-component intensity value, the A-component intensity value and the B-component intensity value of the mean value of each color channel of the target domain image, respectively.

10. The image data enhancement method of claim 9, wherein, The S108 specifically includes: wherein, L, A, B represent the L, A, B component intensity values of the LAB color space of the migration image at the migration image pixel coordinate x L, A, B represent the L, A, B component intensity values of the LAB color space of the migration image at the migration image pixel coordinate L, A, B represent the L, A, B component intensity values of the standard deviation of each color channel of the source image, L, A, B represent the L, A, B component intensity values of the mean of each color channel of the source image, L, A, B represent the L, A, B component intensity values of the standard deviation of each color channel of the target domain image, L, A, B represent the L, A, B component intensity values of the mean of each color channel of the target domain image.

Images