Computer program and simulation device for performing hair dyeing simulations

Abstract

To provide a technique of executing a hair dyeing simulation to the hair of a person by using a method different from a conventional technique.SOLUTION: A simulation device comprises: a classification unit which classifies a hair region included in an object image obtained by imaging a person into a plurality of hair color regions corresponding to a plurality of hair colors; a first generation unit which executes first coloring processing corresponding to hair dyeing with a specific hair dyeing agent to each of the plurality of hair color regions, generates a first colored image including a plurality of first processed regions corresponding to the plurality of hair color regions and in which coloring amounts of the first coloring processing to each of the plurality of hair color regions are different from each other; and a first display control unit which causes a display unit to display the first colored image.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] This specification relates to a technique for performing hair dyeing simulation on human hair.

Background Art

[0002] Non-Patent Document 1 discloses a web server that performs hair dyeing simulation on human hair.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] This specification provides a technique for performing hair dyeing simulation on human hair using a method different from the conventional ones.

Means for Solving the Problems

[0006] A first aspect disclosed herein is a computer program for performing a hair dyeing simulation on human hair. The computer program may function as follows: a classification unit that classifies hair regions included in a target image obtained by photographing a person into multiple hair color regions corresponding to multiple hair colors; a first generation unit that performs a first coloring process corresponding to hair dyeing with a specific hair dye on each of the multiple hair color regions to generate a first colored image containing multiple first processed regions corresponding to the multiple hair color regions, wherein the amount of coloring in the first coloring process for each of the multiple hair color regions is different from that of the multiple hair color regions; and a first display control unit that displays the first colored image on a display unit. With this configuration, the computer can appropriately perform a hair dyeing simulation according to the actual hair color of a person because it utilizes different amounts of coloring corresponding to each classified hair color region.

[0007] In a second aspect disclosed herein, in the first aspect described above, the plurality of hair color regions include a first hair color region corresponding to a first hair color specified by a first pixel value range, and a second hair color region corresponding to a second hair color specified by a second pixel value range different from the first pixel value range, wherein the first coloring process for the first hair color region includes changing the pixel value of each pixel constituting the first hair color region by a first fixed value, and the first coloring process for the second hair color region may include changing the pixel value of each pixel constituting the second hair color region by a second fixed value different from the first fixed value. With this configuration, the computer can utilize different coloring amounts corresponding to each classified hair color region.

[0008] In a third aspect disclosed herein, in the first or second aspect described above, the computer program may further function as a second generation unit that generates a second colored image including a plurality of second processed regions corresponding to the plurality of first processed regions, by performing a second coloring process on each of the plurality of first processed regions included in the first colored image, wherein the amount of coloring in the second coloring process for each of the plurality of first processed regions is different from that of the plurality of first processed regions, and a second display control unit that causes the second colored image to be displayed on a display unit. With this configuration, the computer can appropriately perform the second hair dyeing simulation because it utilizes the different amounts of coloring corresponding to each first processed region included in the first hair dyeing simulation result (i.e., the first colored image).

[0009] In a fourth aspect disclosed herein, in the third aspect described above, the amount of coloring of the first hair dyeing treatment for a specific hair color region among the plurality of hair color regions and the amount of coloring of the second coloring treatment for a specific first treated region corresponding to the specific hair color region among the plurality of first treated regions may be different. With this configuration, the computer can perform a second hair dyeing simulation using the amount of coloring corresponding to the hair color of the first hair dyeing simulation result.

[0010] In a fifth aspect disclosed herein, in the fourth aspect described above, the computer program may further cause the computer to function as a third generation unit that generates a third colored image including a plurality of third processed regions corresponding to the plurality of second processed regions, by performing a third coloring process on each of the plurality of second processed regions included in the second colored image, the amount of coloring of the third coloring process on each of the plurality of second processed regions being different from the amount of coloring of the first coloring process on the particular hair color region, the amount of coloring of the second coloring process on the particular first processed region, and the amount of coloring of the third coloring process on the particular second processed region corresponding to the particular first processed region among the plurality of second processed regions, and a third display control unit that causes the third colored image to be displayed on a display unit. With this configuration, the computer can properly perform the third hair dyeing simulation by utilizing the different coloring amounts corresponding to each second processed region in the second hair dyeing simulation result (i.e., the second colored image). In particular, the computer can perform the third hair dyeing simulation by utilizing the coloring amounts corresponding to the hair color in the second hair dyeing simulation result.

[0011] In a sixth aspect disclosed herein, in any one of the first to fifth aspects described above, the particular hair dye is a gray hair dye, and the first coloring treatment is a gray hair dyeing treatment. With this configuration, the computer can appropriately perform a gray hair dyeing simulation according to the actual hair color of a person.

[0012] In a seventh aspect disclosed herein, in the sixth aspect described above, the classification unit may classify the hair region into the multiple hair color regions according to deep learning using a plurality of training data, wherein the deep learning is such that glossy regions included in the hair region are not determined to be hair color regions corresponding to gray hair. With this configuration, the computer does not determine glossy regions to be hair color regions corresponding to gray hair, so it can appropriately perform a gray hair dyeing simulation according to the actual hair color of a person.

[0013] An eighth aspect disclosed herein is a computer program for performing a hair dyeing simulation on human hair. The computer program may function as follows: a classification unit that classifies hair regions included in a target image obtained by photographing the human into multiple hair color regions corresponding to multiple hair colors; a generation unit that uses the target image to perform N coloring processes corresponding to N hair dyeings (where N is an integer of 2 or more) with a specific hair dye to generate N colored images; and a display control unit that displays at least one of the N colored images on a display unit. The generation unit performs a first coloring process on the target image and then performs an Nth coloring process on the (N-1)th colored image generated by the (N-1)th coloring process, and in each of the N coloring processes, the amount of coloring for each of the multiple processing target regions corresponding to the multiple hair color regions may be different from each other. With this configuration, the computer can appropriately perform a hair dyeing simulation according to the actual hair color of the human, since it uses different amounts of coloring according to each classified hair color region in each of the N coloring processes.

[0014] In the ninth aspect disclosed herein, in the eighth aspect described above, the amount of coloring applied to the target area corresponding to a specific hair color area among the plurality of hair color areas may differ between the (N-1)th coloring treatment and the Nth coloring treatment. With this configuration, the computer can perform the Nth hair dyeing simulation using the amount of coloring corresponding to the hair color of the (N-1)th hair dyeing simulation result.

[0015] A computer-readable storage medium for storing the above computer program is also novel and useful. A simulation device implemented by the above computer program, a method executed by the simulation device, and a communication system comprising the simulation device and other devices are also novel and useful. [Brief explanation of the drawing]

[0016] [Figure 1] This shows the configuration of the communication system. [Figure 2] This shows a flowchart of the simulation process performed by the simulation server. [Figure 3] An example of the shooting screen, the captured image, and the resized image is shown. [Figure 4] A diagram illustrating the process of defining the hair region is shown. [Figure 5] A diagram illustrating the process of identifying glossy areas is shown. [Figure 6] This diagram illustrates the process of classifying hair regions into multiple hair color regions. [Figure 7] This shows an example of three colored images generated by three coloring processes. [Modes for carrying out the invention]

[0017] (Configuration of communication system 2: Figure 1) As shown in FIG. 1, the communication system 2 includes a simulation server 10 (hereinafter simply referred to as "server 10") and a mobile terminal 100. The server 10 and the mobile terminal 100 are connected to the Internet 6 and can communicate with each other via the Internet 6.

[0018] (Configuration of Server 10) In this embodiment, the server 10 is installed on the Internet 6 by a manufacturer that manufactures and sells hair dyes, and performs a white hair dyeing simulation on the hair region included in the human image captured by the mobile terminal 100. The server 10 includes a communication interface 12 and a control unit 30. Each of the units 12 and 30 is connected to a bus line (reference numeral omitted). The communication interface 12 is connected to the Internet 6.

[0019] The control unit 30 includes a CPU 32 and a memory 34. The CPU 32 executes various processes according to programs 36 and 38 stored in the memory 34. The memory 34 is composed of a volatile memory, a non-volatile memory, etc. The OS program 36 is a program for realizing the basic operations of the server 10. Each application program 38 is a program for executing various processes related to the white hair dyeing simulation. Each application program 38 includes a program realized by an existing Library or an existing Model, and a program independently developed by the above manufacturer.

[0020] In addition to the programs 36 and 38, the memory 34 stores learning data 40 and two types of conversion tables 42 and 44. The learning data 40 is data used for deep learning to identify the hair region and the gloss region. The RGB→Lab conversion table 42 is a table for converting pixel values in the RGB color space into pixel values in the Lab color space. The Lab→RGB conversion table 44 is a table for converting pixel values in the Lab color space into pixel values in the RGB color space.

[0021] (Configuration of Mobile Terminal 100) The mobile terminal 100 is a portable user terminal such as a tablet PC or smartphone. In a modified example, a stationary user terminal such as a desktop PC may be used instead of the mobile terminal 100. The mobile terminal 100 comprises a communication interface 112, an operation unit 114, a display unit 116, a camera 118, and a control unit 130. Each of the units 112 to 130 is connected to a bus line (symbol omitted).

[0022] The communication interface 112 is connected to the internet 6. The operation unit 114 is equipped with buttons that accept user instructions depending on how it is operated by the user. The display unit 116 is a display for displaying various information. The display unit 116 functions as a so-called touch panel. That is, the display unit 116 also functions as an operation unit operated by the user. The camera 118 is a device for taking still images or videos, and in this embodiment, it is used to take still images of people.

[0023] The control unit 130 comprises a CPU 132 and a memory 134. The CPU 132 performs various processes according to programs 136 and 138 stored in the memory 134. The memory 134 is composed of volatile memory, non-volatile memory, etc. The OS program 136 is a program that realizes the basic operation of the mobile terminal 100. The browser program 138 is a program that accesses a web server (for example, server 100) and displays various images on the display unit 116.

[0024] (Simulation process: Figure 2) Referring to Figure 2, the content of the simulation processing executed by the CPU 32 of the server 10 according to programs 36 and 38 will be explained. Although the operation of the mobile terminal 100 will not be described in detail below, the mobile terminal 100 accesses the server 10 according to the browser program 138 and displays the image on the display unit 116.

[0025] In S10, the CPU 32 of the server 10 monitors for access from the mobile terminal 100. When the browser program 138 of the mobile terminal 100 is running and the URL of the server 10 (abbreviation for UniformResource Locator) is entered into the mobile terminal 100, the CPU 32 receives an access request from the mobile terminal 100. In this case, the CPU 32 determines YES in S10 and proceeds to S12.

[0026] In S12, the CPU 32 transmits shooting screen data representing the shooting screen 200 to the mobile terminal 100. As a result, as shown in Figure 3, the shooting screen 200 is displayed on the display unit 116 of the mobile terminal 100. The shooting screen 200 includes a message 202 prompting the user to take a picture, and a display area 204 for displaying the image currently being captured by the camera 118. The display area 204 includes a circular area 206 indicating the reference position of the face, and a shooting button 208. When the user selects the shooting button 208, the captured image 210 is stored in the memory 134. The captured image 210 includes multiple pixels represented by pixel values ​​in the RGB color space. In this embodiment, the pixel values ​​in the RGB color space are in the range of 0 to 255, but in modified examples, they may be in a range other than 255.

[0027] In S14, the CPU 32 receives captured image data representing the captured image 210 from the mobile terminal 100. This allows the CPU 32 to read the captured image 210 and perform various processing steps. For example, the CPU 32 reads the captured image 210 using an application program 38 implemented by the Pillow Imaging Library. Details of this library are disclosed, for example, at the URL "https: / / ja.wikipedia.org / wiki / Python_Imaging_Library".

[0028] In S16, the CPU 32 performs a resizing process on the captured image 210. The CPU 32 performs the resizing process using an application program 38 implemented by a library called OpenCV, for example. Details of this library are disclosed, for example, at the URL "https: / / ja.wikipedia.org / wiki / OpenCV". Specifically, if the number of pixels in at least one of the height and width directions of the captured image 210 is greater than 1200, the CPU 32 reduces the captured image 210 so that the number of pixels in both the height and width directions becomes 1200, thereby generating a resized image 220. On the other hand, if the number of pixels in both the height and width directions of the captured image 210 is 1200 or less, the CPU 32 does not resize the captured image 210. However, for convenience, in the following, the image that was not resized in this way will also be referred to as "resized image 220". Note that the pixel threshold for resizing is not limited to 1200; it may be greater than or less than 1200.

[0029] In S20, the CPU 32 identifies the background region, face region, and hair region contained in the resized image 220. Different application programs 38 are used to identify each region. Specifically, the application program 38 for identifying the background region is implemented by the MODNet Model. Details of this model are disclosed, for example, at the URL "https: / / github.com / ZHKKKe / MODNet". A paper detailing this model can also be downloaded from the URL "https: / / arxiv.org / abs / 2011.11961v4". The authors, university, and title of this paper are Zhanghan Ke et al., Cornell University, "MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition". The application program 38 for identifying the face region is implemented by a library called Face Toolbox. Details of this library are disclosed, for example, at the URL "https: / / github.com / shaoanlu / face_toolbox_keras". The application program 38 for identifying hair regions is implemented by a library called PyTorch-hair-segmentation. Details of this library are disclosed, for example, at the URL "https: / / github.com / YBIGTA / pytorch-hair-segmentation".

[0030] As shown in Figure 4, the application program 38 implemented by the MODNet Model performs deep learning using training data 40A, which includes annotations indicating background regions, from the training data 40 representing a vast number of human images. In Figure 4, the annotations indicating background regions in the training data 40A are represented by hatching. The CPU 32 identifies the background region 220A contained in the resized image 220 according to the deep learning implemented by the application program 38.

[0031] The application program 38 implemented by Face Toolbox performs deep learning using training data 40B, which includes annotations indicating face regions from the training data 40. In Figure 4, the annotations indicating face regions in training data 40B are represented by hatching. The CPU 32 identifies the face regions 220B contained in the resized image 220 according to the deep learning implemented by the application program 38.

[0032] The application program 38, implemented by PyTorch-hair-segmentation, performs deep learning using training data 40C, which includes annotations indicating hair regions, from the training data 40. In Figure 4, the annotations indicating hair regions in training data 40C are represented by hatching. The CPU 32 identifies the hair regions 220C contained in the resized image 220 according to the deep learning implemented by the application program 38.

[0033] In S22, the CPU 32 uses the regions identified in S20 to determine the hair regions included in the resized image 220. Our research has shown that in the process of identifying hair regions in S20, there is a possibility that areas that are actually background or face may be mistakenly recognized as hair. Therefore, in S22, if there is an overlap (i.e., pixels) between the hair region 220C identified in S20 and the background region 220A identified in S20, the CPU 32 removes the overlap from the hair region 220C. Similarly, if there is an overlap (i.e., pixels) between the hair region 220C identified in S20 and the face region 220B identified in S20, the CPU 32 removes the overlap from the hair region 220C. This allows the CPU 32 to determine the hair regions 222 included in the resized image 220. The CPU 32 may also execute the process in S22 using an application program 38 implemented by a library called pymatting, for example. Details of the library are disclosed, for example, at the URL "https: / / github.com / pymatting / pymatting". In this way, the server 10 of this embodiment uses different application programs to identify the background region 220A, the face region 220B, and the hair region 220C, and excludes the parts included in the background region 220A and the face region 220B from the hair region 220C, thereby enabling the hair region 222 to be appropriately determined (i.e., extracted).

[0034] In S24, the CPU 32 performs a color space conversion process on the resized image 220, which includes the hair region 222 determined in S22. Specifically, for each of the multiple pixels included in the resized image 220, the CPU 32 uses the RGB→Lab conversion table 42 to convert the pixel value in the RGB color space to the pixel value in the Lab color space. This allows the CPU 32 to generate a Lab image 230 (see Figure 5) containing multiple pixels represented by the pixel values ​​in the Lab color space. The Lab image 230 includes the hair region 232 corresponding to the hair region 222 in the resized image 220. The L value among the pixel values ​​in the Lab color space indicates brightness; the smaller the L value, the closer the color is to black, and the larger the L value, the closer the color is to white. In this embodiment, the pixel values ​​in the Lab color space are in the range of 0 to 255, but in modified examples, they may be in a range other than 255. In the modified version, the RGB→Lab conversion table 42 may not be used, and instead, the application program 38 implemented by the library called OpenCV may be used to perform the color space conversion process in S24. The same applies to the color space conversion process in S36, which will be described later.

[0035] In S30, the CPU 32 identifies glossy regions in the Lab image 230. Glossy regions are areas that appear white due to the shine caused by light hitting the hair. In this embodiment, the application program 38 for identifying glossy regions was developed by the manufacturer that installed the server 10, but in modified examples, it may be implemented using an existing Model or Library. As shown in Figure 5, the application program 38 for identifying glossy regions performs deep learning using training data 40D, which includes annotations indicating glossy regions from the training data 40. In Figure 5, the annotations indicating glossy regions in training data 40D are represented by hatching. The CPU 32 identifies glossy regions 234 in the Lab image 230 according to the deep learning implemented by the application program 38.

[0036] In S32, the CPU 32 classifies the hair region 232 contained in the Lab image 230 into multiple hair color regions corresponding to multiple hair colors. In this embodiment, the multiple hair colors are black, white, light brown, and dark brown. The CPU 32 classifies each pixel contained in the hair region 232 into multiple hair color regions based solely on its L value, regardless of its a and b values. Specifically, the CPU 32 determines each pixel with an L value in the range of 0 to 41 as a black region, each pixel with an L value in the range of 42 to 90 as a dark brown region, each pixel with an L value in the range of 91 to 139 as a light brown region, and each pixel with an L value in the range of 140 to 255 as a white region (i.e., a region indicating gray hair). In this way, the CPU 32 can classify the hair region 232 into four hair color regions. However, the CPU 32 does not determine each pixel constituting the glossy region 234 identified in S30 as a white region. In other words, glossy region 234 is not classified into any of the multiple hair color regions.

[0037] In the example in Figure 6, since humans only have black and white hair, the CPU 32 classifies the hair region 232 into the black region 236 and the white region 238. The CPU 32 does not classify the glossy region 234 into either region 236 or 238. Note that the numerical range of the L value for classifying into multiple hair color regions is not limited to the above, and other ranges may be used. The CPU 32 may also consider the numerical ranges of the a and b values ​​when classifying into multiple hair color regions.

[0038] In S34, the CPU 32 performs a coloring process corresponding to a specific hair dye on each hair color region 236,238 identified in S32, and generates a colored image. Specifically, the CPU 32 performs three coloring processes using the Lab image 230 to generate three colored images. The CPU 32 performs the first coloring process on the Lab image 230 to generate the first colored image 240 (see Figure 7). Next, the CPU 32 performs the second coloring process on the first colored image 240 to generate the second colored image 250. Finally, the CPU 32 performs the second coloring process on the second colored image 250 to generate the third colored image. In this way, the CPU 32 generates multiple simulation results (i.e., colored images 240, 250, 260) corresponding to the same person using hair dye multiple times.

[0039] In the first coloring process, the CPU 32 changes the pixel value of each pixel constituting each hair color region 236,238 in the Lab image 230 by the fixed value shown below "First time" in Figure 7. This fixed value corresponds to the effect of gray hair dyeing by the specific gray hair dye mentioned above. The fixed values ​​used in the second and third processes are similar. For example, for each pixel constituting the black region 236 in the Lab image 230, the CPU 32 does not change the a and b values, but decreases the L value by 4. Also, for each pixel constituting the white region 238 in the Lab image 230, the CPU 32 decreases the L value by 14, increases the a value by 2, and decreases the b value by 1. As a result, the CPU 32 generates a first colored image 240 that includes a first processed region 246 corresponding to the black region 236 and a first processed region 248 corresponding to the white region 238. For example, if Lab image 230 contains a region corresponding to dark brown (or light brown), the CPU 32 reduces the L value by 9 (or 11), increases the a value by 1, and decreases the b value by 1 for each pixel constituting that region. In this way, the CPU 32 utilizes different coloring amounts corresponding to each classified hair color region, enabling it to appropriately perform the first gray hair dyeing simulation according to the actual hair color of a person.

[0040] In contrast, a comparative example could be considered in which, instead of classifying the hair region into separate hair color regions as in this embodiment, a different coloring amount is used for each pixel value constituting the hair region. In this comparative example, since it is necessary to calculate the coloring amount for each pixel value constituting the hair region, the coloring process may take a long time. In contrast, in this embodiment, the CPU 32 performs classification of the hair color regions and performs the coloring process using a fixed value corresponding to the hair color region, so it is not necessary to calculate the coloring amount for each pixel value constituting the hair region. Therefore, the coloring process can be performed more quickly compared to the comparative example.

[0041] In particular, during the first coloring process, the CPU 32 does not change the pixel values ​​of each pixel constituting the glossy region 234. Therefore, the first colored image 240 includes a glossy region 234 that has the same color as the glossy region 234 in the Lab image 230. Similarly, in the second and third coloring processes described later, no coloring process is performed on the glossy region 234. In this way, the CPU 32 does not determine the glossy region 234 as white and does not perform the coloring process corresponding to the white region 238 on the glossy region 234, so that a gray hair dyeing simulation corresponding to the actual hair color of a person can be appropriately performed. In a modified example, the CPU 32 may classify the glossy region 234 as either black, dark brown, or light brown, and perform a coloring process on the glossy region 234 according to the classification. Generally speaking, the CPU 32 just needs to avoid determining the glossy region 234 as white.

[0042] In the second coloring process, the CPU 32 changes the pixel value of each pixel constituting each of the first processed regions 246,248 included in the first colored image 240 by the fixed value shown below "Second time" in Figure 7. For example, for each pixel constituting the first processed region 246 corresponding to the black region 236, the CPU 32 does not change the a and b values, but decreases the L value by 5. Also, for each pixel constituting the first processed region 248 corresponding to the white region 238, the CPU 32 decreases the L value by 15, increases the a value by 2, and decreases the b value by 1. As a result, the CPU 32 generates a second colored image 250 that includes a second processed region 256 corresponding to the first processed region 246 (i.e., the second processed region 256 corresponding to the black region 236) and a second processed region 258 corresponding to the first processed region 248 (i.e., the second processed region 258 corresponding to the white region 238). In this way, the CPU 32 utilizes different coloring amounts corresponding to each of the first processed regions 246,248 included in the first gray hair dyeing simulation result (i.e., the first colored image 240), so that the second gray hair dyeing simulation can be executed appropriately.

[0043] In particular, the amount of coloring in the first coloring process corresponding to the white region 238 (i.e., L value minus 14) and the amount of coloring in the second coloring process corresponding to the first processed region 248 (i.e., L value minus 15) are different. In this way, the CPU 32 can execute the second gray hair dyeing simulation using the amount of coloring corresponding to the hair color from the result of the first gray hair dyeing simulation.

[0044] In the third coloring process, the CPU 32 changes the pixel value of each pixel constituting each of the second processed regions 256,258 in the second colored image 250 by the fixed value shown below "Third time" in Figure 7. For example, for each pixel constituting the second processed region 256 corresponding to the black region 236, the CPU 32 does not change the a and b values, but decreases the L value by 6. Also, for each pixel constituting the second processed region 258 corresponding to the white region 238, the CPU 32 decreases the L value by 16, increases the a value by 2, and decreases the b value by 1. As a result, the CPU 32 generates a third colored image 260 that includes a third processed region 266 corresponding to the second processed region 256 (i.e., the third processed region 266 corresponding to the black region 236) and a third processed region 268 corresponding to the second processed region 258 (i.e., the third processed region 268 corresponding to the white region 238). In this way, the CPU 32 utilizes different coloring amounts corresponding to each of the second processed regions 256 and 258 included in the second gray hair dyeing simulation result (i.e., the second colored image 250), so that the third gray hair dyeing simulation can be executed appropriately.

[0045] In particular, the amount of coloring in the second coloring process corresponding to the first processed region 248 (i.e., L value minus 15) and the amount of coloring in the third coloring process corresponding to the second processed region 258 (i.e., L value minus 16) are different. In this way, the CPU 32 can use the amount of coloring corresponding to the hair color from the second gray hair dyeing simulation result to execute the third gray hair dyeing simulation.

[0046] The fixed values ​​used in the above coloring process correspond to the effect of dyeing gray hair with the specific gray hair dye described above. Therefore, if a hair dyeing simulation is performed using a hair dye different from the specific gray hair dye described above, the values ​​corresponding to the effect of dyeing hair with that different hair dye will be used as the fixed values. This different hair dye includes gray hair dyes different from the specific gray hair dye described above, and hair dyes for purposes other than gray hair dyeing.

[0047] In S36 of Figure 2, the CPU 32 performs a color space conversion process on the three colored images 240, 250, and 260 generated in S34. Specifically, for each of the multiple pixels contained in the first colored image 240, the CPU 32 uses the Lab→RGB conversion table 44 to convert the pixel values ​​in the Lab color space to the pixel values ​​in the RGB color space. As a result, the CPU 32 can generate a first simulation image containing multiple pixels represented by pixel values ​​in the RGB color space. Similarly, the CPU 32 generates a second simulation image from the second colored image 250 and a third simulation image from the third colored image 260.

[0048] In S40, the CPU 32 transmits three simulation image data, representing the three simulation images generated in S36, to the mobile terminal 100. This causes the CPU 32 to display the three simulation images (i.e., three colored images 240, 250, and 260) on the display unit 116 of the mobile terminal 100. By viewing the three simulation images, the user can see how gray hair is dyed through three applications of the hair dye.

[0049] Furthermore, when the CPU 32 displays the simulation image on the display unit 116, it may also display the product names of recommended hair dyes on the display unit 116. Here, a configuration may be adopted in which the recommended hair dyes change according to the simulation results.

[0050] (Correspondence) Application program 38 and server 10 are examples of a "program" and a "simulation device," respectively. The display unit 116 of the mobile terminal 100 is an example of a "display unit." Lab image 230 is an example of a "target image." Black and white are examples of a "first hair color" and a "second hair color," respectively. The L value ranges of 0 to 41 and 140 to 255 are examples of a "first pixel value range" and a "second pixel value range," respectively. The L value of minus 4 in the black area during the first coloring process is an example of a "first fixed value." The L value of minus 14, a value of plus 2, and b value of minus 1 in the white area during the second coloring process are examples of "second fixed values." The white area 238, the first processed area 248, and the second processed area 258 are examples of a "specific hair color area," a "specific first processed area," and a "specific second processed area," respectively. Regions 236 and 238 are examples of "processing areas" for the first coloring process, regions 246 and 248 are examples of "processing areas" for the second coloring process, and regions 256 and 258 are examples of "processing areas" for the third coloring process.

[0051] The processes in S30 and S32 in Figure 2 are examples of processes executed by the "Classification Unit". The process in S34 is an example of processes executed by the "First Generation Unit", "Second Generation Unit", and "Third Generation Unit" (or "Generation Unit"). The process in S40 is an example of processes executed by the "First Display Control Unit", "Second Display Control Unit", and "Third Display Control Unit" (or "Display Control Unit").

[0052] Although specific examples of the present invention have been described in detail above, these are merely illustrative and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples illustrated above. Modifications of the above embodiments are listed below.

[0053] (Modification 1) The first to third coloring treatments do not have to be gray hair dyeing treatments, but may be other types of hair dyeing treatments such as hair color, hair bleach, hair manicure, or color treatment.

[0054] (Modification 2) The technology of the above embodiment does not necessarily have to be implemented by the server 10, but may be implemented, for example, by an application program installed on a mobile terminal 100. In this case, instead of S12 to S14 in Figure 2, a process to display the shooting screen on the display unit 116 is executed, and instead of S40, a process to display three simulation images on the display unit 116 is executed. In this modification, the above application program is an example of a "computer program".

[0055] (Modification 3) In S14 of Figure 2, the CPU 32 may receive captured image data representing a video instead of captured image data representing a still image. In this case, the CPU 32 may perform processing from S16 onwards on the video. Generally speaking, the "target image" includes both still images and videos.

[0056] (Modification 4) The second and third coloring processes do not need to be performed. That is, the server 10 may perform only one coloring process. In this modification, the "second generation unit," "second display control unit," "third generation unit," and "third display control unit" can be omitted.

[0057] (Modification 5) The process in S30 of Figure 2 does not need to be executed. Generally speaking, the "classification unit" does not need to perform the process to prevent the glossy area from being identified as an area corresponding to gray hair.

[0058] (Modified Version 6) In S36 of Figure 2, the CPU 32 does not have to generate a second simulation image from the second colored image 250, for example. In this case, in S40, the CPU 32 displays only the first and third simulation images on the display unit 116 of the mobile terminal 100. In another modified version, in S36 of Figure 2, the CPU 32 does not have to generate the first and second simulation images from the first and second colored images 240, 250, for example. In this case, in S40, the CPU 32 displays only the third simulation image on the display unit 116 of the mobile terminal 100. Generally speaking, the "display control unit" only needs to display at least one colored image out of the N colored images on the display unit. In yet another modified version, the CPU 32 may perform the coloring process from the fourth time onward. Generally speaking, the "generation unit" only needs to perform the coloring process N times (where N is an integer of 2 or more) to generate N colored images.

[0059] (Modification 7) Multiple hair colors are not limited to the four hair colors of the above embodiment (i.e., black, dark brown, light brown, and white). For example, dark brown and light brown may be treated as one brown. Similarly, black and brown may be treated as one color, or brown and white may be treated as one color. The numerical range of the L value for classifying into multiple hair color regions can be arbitrarily set within the range of 0 to 255, but may also be as follows: For example, the upper limit of the L value classified as black is not limited to 41, but may be any of 21 to 61, preferably 31 to 51. The lower limit of the L value classified as white is not limited to 140, but may be any of 120 to 160, preferably 130 to 150. The lower and upper limits of the L value classified as brown, which combines dark brown and light brown, are not limited to 42 and 139, respectively, but may be any of 22 to 62, preferably 32 to 52, or 119 to 159, preferably 129 to 149. Furthermore, if the classification of hair color regions is performed in a color space different from the Lab color space, as in Modification 8 described later, the range within that different color space obtained by converting the above L value range may be used. This modification is also equivalent to using the above L value range.

[0060] (Modification 8) In S24 of Figure 2, the CPU 32 may perform a color space conversion process from the RGB color space to the Lab color space using, for example, a mathematical formula, instead of using the RGB→Lab conversion table 42. In this case, the CPU 32 may perform a first color conversion process from the RGB color space to the XYZ color space using, for example, a first mathematical formula, and then perform a second color conversion process from the XYZ color space to the Lab color space using a second mathematical formula. In another modification, the CPU 32 may perform the processes S30 to S34 in the RGB color space without performing the process in S24. In this case, the process in S36 can also be omitted.

[0061] (Modification 9) In steps S20 and S22 of Figure 2, the CPU 32 may identify hair regions without using deep learning. For example, the CPU 32 scans each pixel in the resized image 220 to identify two regions corresponding to the eyes of a person. Since the eyes are mainly composed of the whites and blacks, the CPU 32 can identify the two regions corresponding to the eyes of a person by searching for white pixels and black pixels in their vicinity. Next, the CPU 32 identifies a region corresponding to the skin of a person's face by searching for skin-colored pixels in the vicinity of these two regions. Next, the CPU 32 identifies a hair region corresponding to human hair by searching for black pixels in the vicinity of these regions. The process described herein is merely an example, and the process for identifying hair regions is not particularly limited.

[0062] (Modification 10) Focusing on the process at S30 in Figure 2, it can be said that the above embodiment describes the following technology: A computer program for performing a hair dyeing simulation on human hair, wherein the computer functions as the following parts: a classification unit that classifies hair regions included in a target image obtained by photographing the human into glossy regions and non-glossy regions according to deep learning using multiple training data; a generation unit that performs a coloring process on the non-glossy regions to generate a colored image, wherein the same process as the coloring process is not performed on the glossy regions; and a display control unit that displays the colored image on a display unit.

[0063] (Modification 11) In the above embodiment, each process in Figure 2 is realized by the CPU 32 executing the program 38. Alternatively, any of the processes in Figure 2 may be realized by hardware such as a logic circuit.

[0064] Furthermore, the technical elements described herein or in the drawings demonstrate technical usefulness individually or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technologies illustrated herein or in the drawings achieve multiple objectives simultaneously, and achieving even one of these objectives constitutes technical usefulness in itself. [Explanation of Symbols]

[0065] 2: Communication system, 6: Internet, 10: Simulation server, 12: Communication interface, 30: Control unit, 32: CPU, 34: Memory, 36: OS program, 38: Application program, 40: Training data, 42: RGB to Lab conversion table, 44: Lab to RGB conversion table, 100: Mobile terminal, 112: Communication interface, 114: Operation unit, 116: Display unit, 118: Camera, 130: Control unit, 132: CPU, 134: Memory, 136: OS program, 138: Browser program, 210: Captured image, 220: Resized image, 230: Lab image, 240~260: Colored image

Claims

1. A computer program for performing hair dyeing simulations on human hair, The computer consists of the following parts, namely: A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A first generation unit that performs a first coloring process, which is a gray hair dyeing process corresponding to dyeing with a gray hair dye, on each of the plurality of hair color regions, and generates a first colored image including a plurality of first processed regions corresponding to the plurality of hair color regions, wherein the amount of coloring in the first coloring process for each of the plurality of hair color regions is different from that of the first generation unit, A first display control unit that causes the first colored image to be displayed on the display unit, A computer program that functions as such.

2. The plurality of hair color regions include the black region corresponding to black as defined by a first pixel value range, the brown region corresponding to brown as defined by a second pixel value range different from the first pixel value range, and the white region corresponding to white as defined by a third pixel value range different from the first and second pixel value ranges. The first coloring process for the black region includes changing the pixel value of each pixel constituting the black region by a first fixed value, The first coloring process for the brown region includes changing the pixel value of each pixel constituting the brown region by a second fixed value different from the first fixed value, The computer program according to claim 1, wherein the first coloring process for the white region includes changing the pixel value of each pixel constituting the white region by a third fixed value different from the first and second fixed values.

3. The aforementioned computer program further uses the computer, A second generation unit that generates a second colored image including a plurality of second processed regions corresponding to the plurality of first processed regions, by performing a second coloring process, which is a gray hair dyeing process corresponding to a second hair dyeing with the gray hair dye, on each of the plurality of first processed regions included in the first colored image, wherein the amount of coloring in the second coloring process for each of the plurality of first processed regions is different from that of the plurality of first processed regions, A second display control unit that displays the second colored image on the display unit, A computer program according to claim 1, which functions as such.

4. The amount of coloring applied to a specific hair color region among the plurality of hair color regions in the first coloring treatment, The computer program according to claim 3, wherein the amount of coloring of the second coloring treatment on a specific first processed region corresponding to a specific hair color region among the plurality of first processed regions is different from the amount of coloring of the second coloring treatment on a specific first processed region.

5. The aforementioned computer program further uses the computer, A third generation unit that generates a third colored image including a plurality of third processed regions corresponding to the plurality of second processed regions, wherein the amount of coloring in the third coloring process for each of the plurality of second processed regions is different from that of the plurality of second processed regions, and the amount of coloring in the first coloring process for a specific hair color region, the amount of coloring in the second coloring process for a specific first processed region, and the amount of coloring in the third coloring process for a specific second processed region corresponding to a specific first processed region among the plurality of second processed regions are different from that of the third generation unit, A third display control unit that displays the aforementioned third colored image on the display unit, A computer program according to claim 4, which functions as such.

6. The computer program according to claim 1, wherein the classification unit is a deep learning using multiple training data, and classifies the hair region into the multiple hair color regions according to the deep learning that prevents the determination of glossy regions included in the hair region as hair color regions corresponding to gray hair.

7. A computer program for performing hair dyeing simulations on human hair, The computer consists of the following parts, namely: A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A generation unit that uses the aforementioned target image to perform N coloring processes, which are N hair dyeing processes corresponding to N hair dyeing processes (where N is an integer of 2 or more), to generate N colored images. A display control unit that displays at least one of the N colored images on the display unit. To make it function as, The generation unit performs a first coloring process on the target image and then performs an Nth coloring process on the (N-1)th colored image generated by the (N-1)th coloring process. A computer program in which, in each of the N coloring processes, the amount of coloring applied to each of the multiple processing target regions corresponding to the multiple hair color regions is different from that of the other.

8. The computer program according to claim 7, wherein the amount of coloring applied to the processing target area corresponding to a specific hair color area among the plurality of hair color areas is different between the (N-1)th coloring process and the Nth coloring process.

9. A simulation device that performs hair dyeing simulations on human hair, A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A first generation unit that performs a first coloring process, which is a gray hair dyeing process corresponding to dyeing with a gray hair dye, on each of the plurality of hair color regions, and generates a first colored image including a plurality of first processed regions corresponding to the plurality of hair color regions, wherein the amount of coloring in the first coloring process for each of the plurality of hair color regions is different from that of the first generation unit, A first display control unit that causes the first colored image to be displayed on the display unit, A simulation device equipped with the following features.

10. A simulation device that performs hair dyeing simulations on human hair, A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A generation unit that uses the aforementioned target image to perform N coloring processes, which are N hair dyeing processes corresponding to N hair dyeing processes (where N is an integer of 2 or more), to generate N colored images. A display control unit that causes at least one of the N colored images to be displayed on the display unit, Equipped with, The generation unit performs a first coloring process on the target image and then performs an Nth coloring process on the (N-1)th colored image generated by the (N-1)th coloring process. A simulation device in which, in each of the N coloring processes, the amount of coloring applied to each of the multiple processing target regions corresponding to the multiple hair color regions is different from that of the other.

11. A simulation system for performing a hair dyeing simulation on human hair, Server and A terminal device is provided, The aforementioned terminal device is The system includes a target image transmission unit that transmits a target image obtained by photographing the aforementioned person to the server. The aforementioned server, A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A first generation unit that performs a first coloring process, which is a gray hair dyeing process corresponding to dyeing with a gray hair dye, on each of the plurality of hair color regions, and generates a first colored image including a plurality of first processed regions corresponding to the plurality of hair color regions, wherein the amount of coloring in the first coloring process for each of the plurality of hair color regions is different from that of the first generation unit, The system includes a colored image transmission unit that transmits the first colored image to the terminal device, The aforementioned terminal device further, Display unit and The system includes a first display control unit that causes the first colored image received from the server to be displayed on the display unit, Simulation system.

12. A simulation system for performing a hair dyeing simulation on human hair, Server and A terminal device is provided, The aforementioned terminal device is The system includes a target image transmission unit that transmits a target image obtained by photographing the aforementioned person to the server. The aforementioned server, A classification unit that classifies hair regions included in a target image obtained by photographing the aforementioned human into multiple hair color regions corresponding to multiple hair colors, wherein the multiple hair color regions include black regions, brown regions, and white regions. A generation unit that uses the aforementioned target image to perform N coloring processes, which are N hair dyeing processes corresponding to N hair dyeing processes (where N is an integer of 2 or more), to generate N colored images. The system includes a colored image transmission unit that transmits at least one colored image from the N colored images to the terminal device, The generation unit performs a first coloring process on the target image and then performs an Nth coloring process on the (N-1)th colored image generated by the (N-1)th coloring process. In each of the N coloring processes, the amount of coloring applied to each of the multiple processing target regions corresponding to the multiple hair color regions is different from that of the other. The aforementioned terminal device further, Display unit and The system includes a display control unit that causes the display unit to display the at least one colored image received from the server. Simulation system.

Images