Method for generating skin symptom image

Abstract

The present invention relates to a method for generating a skin symptom image, comprising: a first step of preparing, as a start image, an image in which skin is captured; a second step of randomly determining the symptom locations and number of skin symptoms to be added; a third step of generating a skin symptom mask for the symptom locations to which the skin symptom have been determined to be added; and a fourth step of covering the start image with the generated skin symptom mask at the same location of the start image so as to generate a skin symptom image, wherein the third step is performed by applying, to the start image, an image processing algorithm that is assigned differently by skin symptom with respect to the selected skin symptom. In the present invention, image processing is performed on the captured skin image so as to reproduce skin symptoms, and thus an image to which more natural skin symptoms are added can be generated. Furthermore, since additional locations and number, the shape and the like of skin symptoms are randomly applied, a plurality of random skin symptom images can be generated.

Description

Method for generating skin symptom images

[0001] The present invention relates to a method for generating a virtual image, and more particularly, to a method for generating an image of skin having a skin symptom.

[0002] In general, symptoms caused by skin diseases are usually confirmed and diagnosed by a dermatologist, but recently, with the use of artificial intelligence through machine learning in the field of medical diagnosis, including medical image analysis, efforts are being made to apply artificial intelligence to the analysis of skin symptoms.

[0003] Machine learning, a type of artificial intelligence technology, analyzes large amounts of input data to probabilistically classify objects or predict values ​​within specific ranges. Unlike traditional programming methods, which derive outcomes determined by clear rules, machine learning operates by empirically analyzing large amounts of input data to derive probabilistic outcomes.

[0004] Accordingly, securing a large amount of training data as input data for machine learning is crucial. In the field of medical image analysis, which uses images captured under similar conditions, such as CT or MRI, a technology has been developed to create virtual medical images with lesions and use them as training data (Korean Patent Publication No. 10-2020-0089146). However, this technology cannot be applied to skin symptoms due to differences in shooting conditions, such as the performance of the imaging device and lighting.

[0005] Meanwhile, images of virtual skin symptoms can be used for a variety of purposes beyond AI training data. They can be generated by adding virtual skin symptoms to existing skin images. While this approach could potentially utilize image synthesis technology (Korean Patent No. 10-2236904), the fact that it synthesizes separate images poses a problem: it fails to reflect the conditions under which the original skin images were captured, resulting in a natural-looking skin symptom.

[0006] The present invention is intended to solve the problems of the prior art described above, and its purpose is to provide a new method for generating a virtual image in which more natural skin symptoms are randomly assigned to an existing skin image.

[0007] In order to achieve the above object, the method for generating a skin symptom image according to the present invention comprises the following steps: a first step of preparing an image in which skin is photographed as a starting image; a second step of randomly determining a symptom location and number to which skin symptoms are added; a third step of generating a skin symptom mask for the symptom location determined to be added to the skin symptom; and a fourth step of generating a skin symptom image by covering the generated skin symptom mask at the same location of the starting image, wherein the third step is characterized in that it is performed by applying an image processing algorithm assigned differently for each skin symptom to the starting image for the selected skin symptom.

[0008] The image processing algorithm set for each of the above skin symptoms is preferably assigned differently to each image group grouped into multiple image groups by reflecting the image shooting conditions and skin color, and the third step is preferably performed by analyzing the starting image, classifying it into one of the multiple image groups, and applying the image processing algorithm assigned to the classified image group to the starting image.

[0009] The second step above can be performed by reflecting the type of skin symptom in determining the location and number of symptoms.

[0010] The second step is performed by setting multiple regions of interest in which skin symptoms may occur in the starting image, randomly selecting one or more of the multiple regions of interest, and dividing the selected regions of interest into one or more to create a segmented patch, and the size and number of segments of the selected regions of interest can be randomly selected.

[0011] At this time, a process of randomly selecting one or more regions of interest from among multiple regions of interest can be performed to reflect the type of skin symptom.

[0012] And, it is preferable that the third step is performed by applying an image processing algorithm set for each skin symptom to the segmented patch for a randomly selected skin symptom.

[0013] In addition, it is preferable that the third step further includes a process of reflecting the appearance of skin symptoms on the split patch.

[0014] And, it is preferable that the process of reflecting the appearance of the skin symptom be performed by randomly selecting an appearance template from an appearance database containing multiple appearance templates for each skin symptom.

[0015] Furthermore, the above-mentioned appearance template may be obtained by segmenting a portion of a reference image in which skin symptoms have been expressed and transforming it into a binary map.

[0016] It is desirable to further include a process of assigning weights to the appearance of the reflected skin symptoms so that transparency increases from the center to the periphery.

[0017] At this time, the process of assigning weights can apply Euclidean distance transformation or Gaussian corner blur.

[0018] The above image processing algorithm may convert one or more of a color value (Hue), a saturation value (Saturation), and a brightness value (Value) at each pixel.

[0019] The present invention, configured as described above, has the effect of generating an image with more natural skin symptoms added by performing image processing on a skin photographed image to reproduce skin symptoms.

[0020] Additionally, since image processing is performed by reflecting the shooting conditions of the captured image, it has the effect of creating an image with natural skin symptoms added regardless of the shooting conditions.

[0021] Furthermore, since the additional location, number, and shape of skin symptoms are randomly applied, there is an effect of being able to generate a large number of random skin symptom images.

[0022] FIG. 1 is a drawing for explaining a method for generating a skin symptom image according to an embodiment of the present invention.

[0023] Figure 2 is a diagram showing the difference in the expression of skin symptoms according to the color temperature of the image and skin color.

[0024] FIG. 3 is a drawing explaining a process of analyzing a starting image and performing image processing according to an image group in a method for generating a skin symptom image according to an embodiment of the present invention.

[0025] FIG. 4 is an example of appearance templates for redness in a method for generating a skin symptom image according to an embodiment of the present invention.

[0026] FIG. 5 is a drawing for explaining a process of generating redness in a method for generating a skin symptom image according to an embodiment of the present invention.

[0027] An embodiment of the present invention is described in detail with reference to the attached drawings.

[0028] However, the embodiments of the present invention may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. The shapes and sizes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated by the same symbols in the drawings are the same elements.

[0029] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where it is "directly connected" but also the cases where it is "electrically connected" with another element in between. Furthermore, when a part is said to "include" or "comprise" a component, this does not mean that it excludes other components, but rather that it can include or comprise other components, unless otherwise specifically stated.

[0030] Additionally, terms such as "first," "second," etc. are intended to distinguish one component from another and should not be construed as limiting the scope of the rights. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0031]

[0032] FIG. 1 is a drawing for explaining a method for generating a skin symptom image according to an embodiment of the present invention.

[0033]

[0034] In the method for generating a skin symptom image according to this embodiment, first, an image of the skin is prepared as a starting image (step 1).

[0035] This embodiment creates a skin symptom image by adding skin symptoms to a normal image of the skin. To this end, an image of the skin is prepared as a starting image. Preferably, the starting image is an image of a person with clean skin and no skin symptoms.

[0036]

[0037] Next, determine the symptom location where skin symptoms are added to the starting image (Step 2).

[0038] The present invention provides randomness to the locations and number of skin symptoms added to generate diverse images with virtual skin symptoms. The randomness of the locations and number of symptoms can be achieved by applying complete random probability, or by adjusting certain probabilities to reflect the characteristics of the skin symptoms. For example, the probability of a specific location can be increased by reflecting information on the location where the skin symptom is most commonly expressed. Furthermore, the random probability can be adjusted based on data distribution probabilities when determining the number of symptoms.

[0039] In this embodiment, the symptom location and number are randomly selected using the following method.

[0040] First, multiple regions of interest (ROI) are set in the starting image, and one or more of the multiple regions of interest are randomly selected and cropped to create a segmented patch.

[0041] Here, the region of interest (ROI) refers to locations where skin symptoms may occur, and can be determined based on the starting image. For example, in images of a human face, the ROI can be set by extracting features such as the eyes, nose, and mouth. In addition, in images of other body parts, the ROI can be set based on each feature.

[0042] Accordingly, one or more regions of interest are randomly selected from the multiple regions of interest established. As discussed above, the randomness of the location and number of regions of interest selected can be completely random, or some of the probability can be adjusted to reflect the characteristics of the skin condition. Through this region of interest selection process, the location and number of skin conditions added to the starting image can be randomly determined.

[0043] Meanwhile, in cases such as when creating a learning data set, images in which skin symptoms do not appear can be included in the learning data set by including cases in which the region of interest is not selected.

[0044] The selected region of interest is then divided into one or more cropped patches to create segmented patches. These cropped patches represent the locations where skin symptoms will be added. The entire region of interest or a portion of the region can be divided. The size and number of segments within the selected region of interest can be randomly selected, further increasing the randomness of the location of skin symptoms.

[0045]

[0046] And a skin symptom mask is created for the symptom location where the skin symptom is determined to be added (Step 3).

[0047] Specifically, in this embodiment, a skin symptom mask is generated using each of one or more segmented patches selected from the region of interest.

[0048] The generated skin symptom masks are images of various skin symptoms, such as redness, dark circles, pigmentation, and acne. These skin symptoms can reflect the previously discussed region of interest settings. Different skin symptoms can be assigned to different segmented patches, but it is preferable to apply a single skin symptom to segmented patches belonging to the same region of interest.

[0049] In this embodiment, the process of creating a skin symptom mask proceeds in the order of image processing, appearance assignment, and weight assignment.

[0050] First, image processing is a step that performs image processing tailored to the given skin symptom for the selected segmented patch.

[0051] Image processing is performed by transforming one or more of the hue, saturation, and value values ​​of each pixel in the base image (i.e., the segmented patch), and is performed according to an image processing algorithm designed for each skin condition to be created. For example, dark circles and pigmentation can be created by lowering the brightness value, while rosacea and acne can be created by increasing the color value to the red series.

[0052] Meanwhile, this embodiment generates an image with a virtual skin symptom using an image of the skin. Generally, the color of the image, etc. of the image, is different due to environmental influences such as the shooting conditions of the camera filter, depth of field, exposure, and lighting, as well as skin color. In addition, the appearance of the captured skin symptom also differs depending on the shooting conditions and skin color. For example, in the case of the same skin symptom, rosacea, it appears as a bright pink in an image with a low white balance color temperature (green), and appears as a red color in an image with a high color temperature (red). Therefore, if a skin symptom is generated regardless of the characteristics of the original image, such as the white balance, it is inevitable that there will be a difference from the actual skin symptom captured, and if such an image is used as training data, it will cause the accuracy of the training model to decrease.

[0053] Figure 2 is a diagram showing the difference in the expression of skin symptoms according to the color temperature of the image and skin color.

[0054] The present invention applies a feature extraction method and a clustering algorithm to images of various conditions to classify the images into multiple image groups in order to impart natural skin symptoms according to the shooting conditions of the starting images in which the skin is captured. For example, the multiple groups can be grouped by classifying photos in which skin symptoms are captured in the form of the same color into N image groups through images in which skin symptoms are captured. In order to identify the color of the symptom, a preprocessing algorithm for segmenting the facial skin can be applied, and the color (Hue) can be extracted from the HSV color space corresponding to the corresponding location.

[0055] These multiple image groups reflect the shooting conditions and skin tone, and each image group is assigned a separate image processing algorithm for each skin condition. The image processing algorithm may be optimized to reflect the shooting conditions and skin tone that formed the basis for grouping each image group.

[0056] The starting image is analyzed, classified into one of the plurality of image groups, and an image processing algorithm assigned to the group is applied to perform image processing on the segmented patch.

[0057] FIG. 3 is a drawing explaining a process of analyzing a starting image and performing image processing according to an image group in a method for generating a skin symptom image according to an embodiment of the present invention.

[0058] Appearance assignment is the step of cutting out the segmented patch on which image processing has been performed into the appearance of a skin symptom.

[0059] Skin symptoms manifest themselves in different forms (appearances) on the skin depending on the symptom, and even the same symptom can manifest in various appearances. In the previous step, HSV corresponding to the skin symptom was expressed through image processing. However, since this image processing was performed on the entire segmented patch, the appearance of the skin symptom was not reflected. Appearance assignment is the step of assigning a range and shape appropriate to the skin symptom. To this end, in this embodiment, a database consisting of multiple appearance templates for each skin symptom is prepared, and a configuration is made to randomly select from among the multiple appearance templates. Specifically, in this embodiment, an appearance template was acquired by segmenting the area where the skin symptom appeared from a reference image and transforming it into a binary map. Appearance templates were prepared for each skin symptom, and a database containing more than 50 appearance templates for each region of interest was constructed. Then, for the image-processed segmented patch, a stencil mask with an appearance was created by cutting it into the shape of a binary map, which is a randomly selected appearance template from the appearance database.

[0060] FIG. 4 is an example of appearance templates for redness in a method for generating a skin symptom image according to an embodiment of the present invention.

[0061] In this embodiment, image processing is performed first and then appearance assignment is performed, but this is not limited to this and the order of these may be changed.

[0062] Weighting is a process of processing the stencil mask shape so that it has greater transparency from the center to the periphery so that the cut stencil mask can naturally match the original starting image.

[0063] The stencil mask generated through the above process is assigned the color and shape of the skin condition, but if it is directly pasted onto the starting image, the boundary line becomes prominent. To resolve this problem, the shape is processed so that it gradually becomes more transparent from the center to the periphery. Algorithms such as the Euclidian distance transform (EDT) and Gaussian edge blur can be applied, but are not particularly limited. At this time, the degree of weighting, i.e., the amount of change in transparency between the center and the periphery, can be adjusted differently depending on the type of disease.

[0064]

[0065] Finally, a skin symptom image is created by overlaying the generated skin symptom mask onto the same location of the starting image (Step 4).

[0066] FIG. 5 is a drawing for explaining a process of generating redness in a method for generating a skin symptom image according to an embodiment of the present invention.

[0067] ① is a segmented patch selected from the region of interest, ② is the result of image processing for the segmented patch for the flushing symptom, ③ is an appearance template for the flushing symptom, ④ is the appearance with weights applied to the appearance template, and ⑤ shows the result of attaching a skin symptom mask for the flushing symptom to the same location.

[0068] Through the above process, the skin symptom mask created in the form of a stencil mask has an HSV close to an actual skin symptom because it has been image-processed with an image processing algorithm selected according to the image characteristics of the starting image, and because it has been performed up to the appearance selected from the appearance database and weighting, when attached to the same location of the starting image, the skin symptom mask covers the top of the starting image, thereby creating an image with natural skin symptoms.

[0069] In addition, since skin diseases are randomly assigned to randomly selected segmented patches among multiple regions of interest, various skin symptom images with random locations and numbers of skin symptoms can be generated, and since skin symptom masks are generated with appearances randomly selected from multiple appearance databases, various skin symptom images with random appearances can be generated.

[0070]

[0071] The present invention has been described above through preferred embodiments. However, the above-described embodiments are merely illustrative of the technical idea of ​​the present invention. Those skilled in the art will understand that various changes may be made without departing from the technical idea of ​​the present invention. Therefore, the scope of protection of the present invention should be interpreted not by specific embodiments, but by the matters described in the claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present invention.

Claims

1. The first step is to prepare the skin image as the starting image; The second step is to randomly determine the location and number of additional skin symptoms; A third step of creating a skin symptom mask for the symptom location where the skin symptom is determined to be added; and A fourth step of generating a skin symptom image by covering the generated skin symptom mask at the same location as the starting image is included. A method for generating a skin symptom image, characterized in that the third step is performed by applying an image processing algorithm assigned differently for each skin symptom to the starting image for the selected skin symptom.

2. In claim 1, The image processing algorithm set for each of the above skin symptoms is assigned differently to each image group grouped into multiple image groups, reflecting the image shooting conditions and skin color. A method for generating a skin symptom image, characterized in that the third step is performed by analyzing the starting image, classifying it into one of a plurality of image groups, and applying an image processing algorithm assigned to the classified image group to the starting image.

3. In claim 1, The second step is a method for generating a skin symptom image, characterized in that the process of determining the location and number of symptoms is performed to reflect the type of skin symptom.

4. In claim 1, The second step above is, After setting multiple regions of interest where skin symptoms may occur in the above starting image, one or more regions of interest are randomly selected, and the selected regions of interest are divided into one or more to create a split patch, which is then performed. A method for generating a skin symptom image, characterized in that the size and number of segments for dividing a selected region of interest are randomly selected.

5. In claim 4, A method for generating a skin symptom image, characterized in that a process of randomly selecting one or more regions of interest from among multiple regions of interest is performed to reflect the type of skin symptom.

6. In claim 4, A method for generating a skin symptom image, characterized in that the third step is performed by applying an image processing algorithm set for each skin symptom to the segmented patch for the selected skin symptom.

7. In claim 6, A method for generating a skin symptom image, characterized in that the third step further includes a process of reflecting the appearance of the selected skin symptom on the split patch.

8. In claim 7, The process of reflecting the appearance of skin symptoms is: A method for generating a skin symptom image, characterized in that the method is performed by randomly selecting an appearance template from an appearance database containing multiple appearance templates for each skin symptom.

9. In claim 8, A method for generating a skin symptom image, characterized in that the above-mentioned external shape template is obtained by segmenting a part where skin symptoms are expressed from a reference image where skin symptoms are expressed and transforming it into a binary map.

10. In claim 7, A method for generating a skin symptom image, characterized in that it further includes a process of assigning weights to the appearance of a reflected skin symptom so that transparency increases from the center to the periphery.

11. In claim 10, A method for generating a skin symptom image, wherein the weighting process is characterized by applying a Euclidean distance transform or a Gaussian edge blur.

12. In claim 1, A method for generating a skin symptom image, wherein the image processing algorithm converts at least one of a color value (Hue), a saturation value (Saturation), and a brightness value (Value) in each pixel.

Images