A melanoma data enhancement method based on image fusion

Abstract

The application discloses a melanoma data enhancement method based on image fusion, which comprises the following steps: acquiring a melanoma data set and uniformly sizing images in the melanoma data set; traversing the uniformly sized images, and performing the following operations on each image: cutting the image in half to obtain two cut images; randomly flipping one of the two cut images, denoting the randomly flipped cut image as a first image and the cut image without random flipping as a second image; randomly acquiring a ratio number in the range of [0, 1], and fusing the first image and the second image according to the ratio number to form a fused image; and saving all the fused images to complete data enhancement of the images in the melanoma data set. The method realizes image data enhancement by fusing the cut images according to the ratio number, so as to balance data categories, expand data volume, improve model robustness and accurately identify melanoma.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a melanoma data enhancement method based on image fusion. BACKGROUND

[0002] As a branch of information fusion, image fusion is a hotspot in current information fusion research. The data form of image fusion is an image containing brightness, color, temperature, distance and other scene features. These images can be given in the form of one or a column. Image fusion is to fuse the information of two or more images into one image, so that the fused image contains more information and can be more convenient for people to observe or computer to process. The goal of image fusion is to reduce the uncertainty and redundancy of the output on the basis of maximizing the relevant information under the actual application target. It can expand the time and space information contained in the image, reduce uncertainty, increase reliability and improve the robustness of the system.

[0003] At present, two popular methods for realizing medical image fusion are CNN and GAN methods. Most CNN methods for medical image fusion only participate in feature fusion, and usually use a pre-trained convolutional neural network to measure the activity level of medical image pixels and generate a fusion weight map. Then, the generated weight map is combined with a traditional decomposition and reconstruction strategy (such as Laplacian pyramid) to realize medical image fusion. The GAN method models the salient information in the medical source image through an adversarial learning mechanism. For example, the salient information of functional medical images is intensity distribution, and the salient information of structural medical images is spatial texture. GFPPC-GAN introduces GAN into the fusion of GFP and PC images, and uses adversarial learning between PC images and fusion images to strengthen the preservation of structural information. However, most existing image fusion technologies are based on the fusion of two different images, which cannot fully utilize the information in the same image and increase information disturbance to realize data enhancement, which is not conducive to improving recognition accuracy. Therefore, the present application proposes a data enhancement method based on image self-fusion. SUMMARY

[0004] The purpose of the present application is to solve the above problems, and to provide a melanoma data enhancement method based on image fusion. The method realizes image data enhancement, so as to achieve the purposes of balancing data categories and expanding data quantity, and improving model robustness, and can accurately identify melanoma.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] The melanoma data enhancement method based on image fusion provided by the present application comprises the following steps:

[0007] Step 1, obtain a melanoma dataset and unify the size of images in the melanoma dataset;

[0008] Step 2, traverse the images after unifying the size, and perform the following operations on each image:

[0009] Step 2.1, cut the image in half to obtain two cropped images, the process is as follows:

[0010] Step 2.1.1, randomly obtain a decimal number [0, 1];

[0011] Step 2.1.2, determine whether the decimal number is greater than 0.5, if yes, cut the image along the horizontal symmetry axis, otherwise, cut the image along the vertical symmetry axis;

[0012] Step 2.2, randomly flip one of the two cropped images, and record the randomly flipped cropped image as the first image and the cropped image without random flipping as the second image;

[0013] Step 2.3, randomly obtain a ratio number n [0, 1], and fuse the first image and the second image to form a fused image according to the ratio number n, wherein the ratio of the first image is n and the ratio of the second image is 1-n;

[0014] Step 3, save all fused images and complete the data augmentation of images in the melanoma dataset.

[0015] Preferably, the melanoma dataset is the ISIC melanoma dataset.

[0016] Preferably, unifying the size of images in the melanoma dataset specifically refers to cutting each image to 512x512 size at the center.

[0017] Preferably, one of the two cropped images is randomly flipped, and the flipping manner is random horizontal flipping or vertical flipping.

[0018] Compared with the prior art, the beneficial effects of the present application are: the method realizes image data augmentation by cutting the image itself and fusing according to the ratio number, thereby achieving data class balance and expanding data volume, improving model robustness, and accurately identifying melanoma. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The melanoma data augmentation method flowchart based on image fusion of the present application;

[0020] Figure 2 The flowchart of step 2 of the present application;

[0021] Figure 3 The image cutting in half schematic diagram of the present application;

[0022] Figure 4 This is a schematic diagram of random image flipping according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0025] like Figures 1-4 As shown, a melanoma data augmentation method based on image fusion includes the following steps:

[0026] Step 1: Obtain the melanoma dataset and unify the image size in the melanoma dataset.

[0027] In one embodiment, the melanoma dataset is the ISIC melanoma dataset. The melanoma dataset is the 2019 ISIC melanoma dataset, containing 25,331 melanoma images (original images) of various sizes, of which 20,809 are benign and 4,522 are malignant.

[0028] In one embodiment, unifying the image size in the melanoma dataset specifically involves cropping each image to a size of 512×512 centered on the image.

[0029] Step 2: Iterate through the images after unifying their size, and perform the following operations on each image:

[0030] Step 2.1: Crop the image in half to obtain two cropped images, as follows:

[0031] Step 2.1.1: Randomly obtain a decimal number in the range [0, 1].

[0032] Step 2.1.2: Determine if the decimal is greater than 0.5. If yes, crop the image along the horizontal axis of symmetry; otherwise, crop the image along the vertical axis of symmetry.

[0033] Step 2.2: Randomly flip one of the two cropped images and denote the randomly flipped cropped image as the first image and the cropped image that is not randomly flipped as the second image.

[0034] Step 2.3, randomly obtain a decimal number n in [0, 1], and fuse the first image and the second image to form a fused image according to the decimal number n, wherein the proportion of the first image is n and the proportion of the second image is 1-n.

[0035] In an embodiment, one of the two cropped images is randomly flipped, and the flipping manner is random horizontal flipping or vertical flipping.

[0036] If the randomly obtained decimal number is greater than 0.5, the image is cut into two parts along the horizontal symmetry axis to obtain two 512x256 images; if the randomly obtained decimal number is less than or equal to 0.5, the image is cut into two parts along the vertical symmetry axis to obtain two 256x512 images, as shown in FIG. 2B. Figure 3 As shown in FIG. 2C, one of the cropped images is randomly flipped, and the flipping manner can be random horizontal or vertical flipping. Figure 4

[0037] Step 3, save all the fused images, and complete the data augmentation of the images in the melanoma dataset.

[0038] Iterate through the images of the uniform size, and repeat step 2 for the remaining 25330 images in the melanoma dataset, so as to obtain the fused images corresponding to the original images.

[0039] The method can achieve the purposes of balancing data categories, expanding data volume, and improving model robustness by cutting the image itself and fusing according to the proportion number, so as to accurately identify melanoma.

[0040] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered as within the scope of the present application.

[0041] The above-described embodiments only express the specific and detailed embodiments described in the present application, but should not be construed as limiting the scope of the patent application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of the patent application of the present application should be subject to the appended claims.​

Claims

1. A melanoma data augmentation method based on image fusion, characterized in that: The melanoma data augmentation method based on image fusion comprises the following steps: Step 1, obtaining a melanoma data set and unifying the size of images in the melanoma data set; Step 2, traversing the images after unifying the size, and performing the following operations on each image: Step 2.1, cutting the image in half to obtain two cut images, the process is as follows: Step 2.1.1, randomly obtaining a decimal number [0, 1]; Step 2.1.2, judging whether the decimal number is greater than 0.5, if yes, cutting the image along the horizontal symmetry axis, otherwise, cutting the image along the vertical symmetry axis; Step 2.2, randomly flipping one of the two cut images, denoted as the first image, and the other cut image without random flipping as the second image; Step 2.3, randomly obtaining a ratio number n [0, 1], and fusing the first image and the second image to form a fused image according to the ratio number n, wherein the ratio of the first image is n and the ratio of the second image is 1-n; Step 3, saving all fused images and completing the data augmentation of images in the melanoma data set.

2. The image fusion-based melanoma data augmentation method of claim 1, wherein: The melanoma data set is ISIC melanoma data set.

3. The image fusion-based melanoma data augmentation method of claim 1, wherein: The unifying the size of images in the melanoma data set specifically refers to cutting each image to 512*512 size by center cutting.

4. The image fusion-based melanoma data augmentation method of claim 1, wherein: The random flipping of one of the two cut images is random horizontal flipping or vertical flipping.

Images