Image conversion device, control method for image conversion device, control program, and recording medium

Abstract

To control color tone of a generated image generated by image conversion.SOLUTION: An image converting apparatus according to the present invention includes: an acquiring unit for acquiring an input image of a first image style in which an object part of a first object is captured; a generating unit having (1) a neural network for generating a target image of a second image style different from the first image style from the input image, and (2) a hue information control unit for acquiring hue information indicating hue of the object part in the reference image of the second image style in which the object part of a second object different from the first object is captured to input it to the neural network; an input control unit for inputting the input image and the reference image into the generating unit; and an output control unit for controlling the output of the generated target image.SELECTED DRAWING: Figure 2

Claims

1. an acquisition unit that acquires an input image in a first image style, the input image being a three-dimensional image that mimics a three-dimensional shape of a target portion of a second object; (1) a neural network that generates a target image in a second image style different from the first image style from the input image; and (2) a color information control unit that acquires color information indicating a color tone of the target portion in a reference image in the second image style, the reference image being a two-dimensional image showing the target portion of a first object different from the second object, and inputs the color information to the neural network. an input control unit that inputs the input image and the reference image to the generation unit; an output control unit that controls output of the target image generated by the generation unit, The neural network comprises: generating, as the target image, an image in which the shape of the target portion of the second object captured in the input image is maintained and the color tone of the target portion of the second object is controlled based on the color tone information; Image conversion device.

2. The neural network comprises: a first generator for generating a first transformed image in the second image style from a first input image in the first image style; a second generator for generating a second transformed image in the first image style from a second input image in the second image style; a first classifier capable of classifying the image of the first image style based on the shape and color of the target part; a second classifier capable of classifying the image of the second image style based on the shape and color of the target part; Equipped with the first generator is further operable to generate a third transformed image in the first image style from the second transformed image; the second generator is further operable to generate a fourth transformed image in the first image style from the first transformed image; The first classifier is a first color error between color information of the target area in the first input image and color information of the target area in a reference image of the second image style; (1) a first error between the first input image and the fourth converted image regarding a shape of the target portion; (2) a second error between the second transformed image and the image of the first image style relating to the shape of the target area; and (3) a sixth error related to the shape of the target portion between a second evaluation image generated when the second converted image is input to the second generator and the second converted image; and identifying an image of the first image style based on at least one of The second classifier is a second color tone error between color tone information of the target area in the second input image and color tone information of the target area in a reference image of the first image style; (1) a fourth error related to the shape of the target portion between the second input image and the third converted image; (2) a fifth error between the first transformed image and the image of the second image style relating to the shape of the target area; and (3) a third error, regarding the shape of the target portion, between a first evaluation image generated when the first converted image is input to the first generator and the first converted image; and identifying the image of the first image style based on at least one of 2. The image conversion device according to claim 1.

3. the neural network is trained to minimize a first combined error that combines the first color error, the first error, the second error, and the third error, and a second combined error that combines the second color error, the fourth error, the fifth error, and the sixth error.

3. The image conversion device according to claim 2.

4. the reference image is a partial image obtained by extracting a region of the target part from a whole image obtained by capturing an image of the entire first object, the input image is an image of the target portion of the second object, The generation unit further generates a composite image by combining the target image with a region of the whole image corresponding to the target part.

4. An image conversion device according to claim 1.

5. the first object and the second object are living organisms suffering from a disease at the target site, The disease in the second subject is more advanced than the disease in the first subject.

5. An image conversion device according to claim 1.

6. The first object and the second object are living beings that have undergone an intervention on the target site, the time since the second subject underwent a medical intervention is greater than the time since the first subject underwent the intervention; 5. An image conversion device according to claim 1.

7. the first object and the second object are living organisms, The target area is the whole body, joints, skin, face, eyes, nose, mouth, ears, and / or hair; 7. An image conversion device according to claim 1.

8. A method for controlling an image conversion device, comprising: the image conversion device includes a neural network that generates a target image in a second image style different from an input image in a first image style, the input image being a three-dimensional image that imitates a three-dimensional shape of a target portion of a second object, the target image being different from the first image style; acquiring the input image; an input step of inputting, into the neural network, color tone information indicating a color tone of the target portion in the input image and a reference image which is a two-dimensional image in the second image style in which the target portion of a first object different from the second object is shown; A generating step of causing the neural network to generate a target image; and an output step of outputting the target image generated by the neural network, The neural network comprises: generating, as the target image, an image in which a shape of the target portion of the second object captured in the input image is maintained and a color tone of the target portion of the second object is controlled based on the color tone information; A method for controlling an image conversion device.

9. A control program for causing a computer to function as an image conversion device as described in any one of claims 1 to 7, comprising the acquisition unit, the generation unit, the color information control unit, the input control unit, and the output control unit.

10. A computer-readable recording medium having the control program according to claim 9 recorded thereon.

Images