Method for operating information processing device, information processing device, and program

Abstract

To improve evaluation accuracy for a surface state of a face.SOLUTION: A method for operating an information processing device includes: a first step of extracting a linear region having a feature that is different from those of the other regions in a predetermined portion of an image of the face of a person from the predetermined portion; and a second step of determining a surface state of the face on the basis of a size and an extension direction of the linear region.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an operation method of an information processing device, an information processing device, and a program. [Background technology]

[0002] The surface condition of a person's facial skin changes depending on various factors. Changes in the surface condition are manifested, for example, as an increase in wrinkles, and are a cosmetic concern. Various techniques for evaluating such facial surface conditions have been proposed (for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-007568 [Patent Document 2] Japanese Patent Application Laid-Open No. 2015-062569 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in the accuracy of assessing facial surface condition.

[0005] In view of the above, the following discloses an operation method of an information processing device that can improve the accuracy of evaluating the surface state of a face. [Means for solving the problem]

[0006] In order to solve the above problem, the operating method of an information processing device in the present disclosure includes a first step of extracting a linear region from a predetermined portion of an image of a person's face, the linear region having characteristics different from other regions in the predetermined portion, and a second step of determining the surface condition of the face based on the size and extension direction of the linear region.

[0007] In addition, the information processing device of the present disclosure includes a memory unit that stores judgment information that associates the surface condition of a person's face with the size and extension direction of a linear region that is extracted from a specified part of an image of the person's face and has characteristics different from other areas in the specified part, and a control unit that extracts a first linear region from an image of a first face and uses the judgment information to judge the surface condition of the first face that corresponds to the first linear region.

[0008] Furthermore, the program of the present disclosure is a program that, when executed by an information processing device, causes the information processing device to perform a first step of extracting, from a specified portion of an image of a person's face, a linear region having characteristics different from other regions in the specified portion, and a second step of determining the surface condition of the face based on the size and extension direction of the linear region. [Effects of the Invention]

[0009] According to the operation method of the information processing device and the like in the present disclosure, it is possible to improve the accuracy of evaluating the surface condition of the face. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 2] FIG. 10 is a flowchart illustrating an example of an operation procedure of the server device. [Figure 3A] FIG. 10 is a diagram showing examples of facial parts and linear regions. [Figure 3B] FIG. 10 is a diagram showing examples of facial parts and linear regions. [Figure 3C] FIG. 10 is a diagram showing examples of facial parts and linear regions. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described.

[0012] [System Configuration] FIG. 1 is a diagram illustrating an example of a configuration of an embodiment of the present invention. The information processing system 1 includes a server device 10 and a terminal device 12 that are connected to each other via a network 11 so that they can communicate with each other. In the information processing system 1, the server device 10 performs information processing, including machine learning, using various information sent from the terminal device 12. The terminal device 12 is, for example, one or more personal computers. The personal computer may include a tablet terminal device, a smartphone, etc. The server device 10 corresponds to the "information processing device" in this embodiment. The server device 10 is, for example, one or more server computers. The server device 10 may be a single server computer or multiple server computers that cooperate to perform the operations of this embodiment and provide services. The network 11 is, for example, a local area network (LAN), the Internet, an ad hoc network, a metropolitan area network (MAN), a mobile communication network, or other networks, or any combination thereof.

[0013] In this embodiment, the server device 10 performs information processing to evaluate changes over time in features of a predetermined region extracted from a person's face image (hereinafter referred to as a face image). The face image is a captured image obtained by capturing a person's face. The server device 10 executes a process of extracting a linear region (hereinafter referred to as a linear region extraction process) from a predetermined region of the face image, the linear region having features different from other regions in the predetermined region. Here, the linear region is, for example, a wrinkle. The server device 10 also executes a process of determining the facial surface condition based on the size and extension direction of the linear region (hereinafter referred to as a condition determination process). For example, using the knowledge that the shape of wrinkles in the eyelid region has a certain degree of correlation with age, the server device 10 determines the facial surface condition, i.e., the aging state, of the person based on the size and extension direction of wrinkles in the eyelid region of the face. As such, this embodiment enables quantitative evaluation of the facial surface condition without relying on subjectivity. This improves the accuracy of determining the facial surface condition.

[0014] [Configuration example of server device 10] The server device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the server device 10 is configured with two or more server computers, these components are appropriately arranged in the two or more server computers.

[0015] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10 and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101 and communicates information with a terminal device 12 via the network 11.

[0016] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, that function as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read-only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable read-only memory (EEPROM). The storage unit 102 stores information used in the operation of the control unit 103 and information obtained by the operation of the control unit 103. The storage unit 102 previously stores surface condition determination information 108, which the control unit 103 references to determine the facial surface condition. The surface condition determination information 108 is generated in advance based on any experiment, information analysis, etc., and is sent, for example, from the terminal device 12 to the server device 10 and stored in the storage unit 102.

[0017] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0018] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing the processor to function as the control unit 103. Alternatively, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Alternatively, the control program may be stored in a non-transitory recording / storage medium readable by the control unit 103, and read by the control unit 103 from the medium.

[0019] The input unit 105 includes one or more input interfaces. The input interfaces are, for example, physical keys, capacitance keys, a pointing device, a touch screen integrated with a display, or a microphone that accepts voice input. The input unit 105 accepts an operation to input information used in the operation of the server device 10 and sends the input information to the control unit 103.

[0020] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the server device 10.

[0021] [Configuration example of terminal device 12] The terminal device 12 includes a communication unit 121 , a storage unit 122 , a control unit 123 , an input unit 125 , and an output unit 126 .

[0022] The communication unit 121 includes a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 121 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0023] The storage unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 122 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 122 stores information used in the operation of the control unit 123 and information obtained by the operation of the control unit 123.

[0024] The control unit 123 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), etc., or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 123 may have one or more dedicated circuits such as an FPGA, an ASIC, etc. The control unit 123 performs overall control of the operation of the terminal device 12 by operating according to a control / processing program or operating according to an operating procedure implemented as a circuit. The control unit 123 then transmits and receives various information to and from the server device 10, etc. via the communication unit 121, and performs the operation according to this embodiment.

[0025] The functions of the terminal device 12 are realized by a processor included in the control unit 123 executing a control program. The control program is a program for causing the processor to function as the control unit 123. Alternatively, some or all of the functions of the terminal device 12 may be realized by a dedicated circuit included in the control unit 123. Alternatively, the control program may be stored in a non-transitory recording / storage medium readable by the control unit 123, and read by the control unit 123 from the medium.

[0026] The input unit 125 includes one or more input interfaces. The input interfaces include, for example, physical keys, capacitive keys, a pointing device, and a touch screen integrated with a display. The input interfaces also include a microphone for receiving voice input and a camera for capturing captured images. The input interfaces may also include a scanner or camera for scanning image codes, and an IC card reader. The input unit 125 receives an operation for inputting information used in the operation of the control unit 123 and sends the input information to the control unit 123. The input unit 125 also sends images captured by the camera to the control unit 123.

[0027] The output unit 126 includes one or more output interfaces. The output interfaces include, for example, a display and a speaker. The display is, for example, an LCD or an organic EL display. The output unit 126 outputs information obtained by the operation of the control unit 123.

[0028] [Operation procedure of server device 10] Fig. 2 is a flowchart for explaining an example of the operation of server device 10. The procedure in Fig. 2 is a procedure executed by control unit 103 when determining the surface condition of a person's face, i.e., the aging state, based on the size and extension direction of wrinkles in the eyelid region of the face. For example, a user operates terminal device 12 to send an instruction to server device 10 to operate server device 10, and control unit 103 executes the procedure in Fig. 2 in response to the instruction.

[0029] In step S20, the control unit 103 acquires a facial image. The facial image is generated by capturing an image of a person's face. The image of the person's face is captured by, for example, the terminal device 12. For example, the control unit 103 receives the facial image of the person sent from the terminal device 12 via the communication unit 101 and stores it in the storage unit 102. The control unit 103 may acquire the facial image from open data.

[0030] In step S21, the control unit 103 extracts an eyelid region from the face image. The control unit 103 detects a face region in the face image using any algorithm, such as landmark point detection, and detects and extracts the eyelid region of the face. The eyelid region is a region of any size and any shape, such as a substantially elliptical or rectangular shape. The eyelid region is preferably detected from a face image with the eyelids closed, but may also be detected with the eyelids half-open. For example, as shown in the example of FIG. 3A, a face region 31 is detected in a face image 31, and a right eyelid region 32 is extracted. Here, the right eyelid region 32 is exemplified, but the control unit 103 may extract the left eyelid region.

[0031] In step S22, control unit 103 extracts a linear region from the eyelid region. The linear region is, for example, a wrinkle, which is a longitudinal region having characteristics, such as color or brightness, that differ from other regions in the eyelid region. Control unit 103 extracts the wrinkles using any image processing method, such as a Frangi vessel enhancement filter. For example, as shown in the example of FIG. 3B, wrinkles 33a, 33b, 33c, 33d, 33e, and 33f are extracted from eyelid region 32. Note that before extracting the wrinkles, control unit 103 may apply image processing, such as an average filter, to the image of the eyelid region to remove information other than wrinkles, such as moles and blemishes.

[0032] In step S23, control unit 103 derives the direction of each wrinkle. The wrinkle direction is, for example, the extension direction of the wrinkle, which is the direction toward center line 30 of facial region 31. Center line 30 may be a line passing through the bridge of the nose detected from facial region 31 by an arbitrary algorithm. For example, as shown in the example of FIG. 3C , directions 34a, 34b, 34c, 34d, 34e, and 34f are derived by an arbitrary algorithm for wrinkles 33a, 33b, 33c, 33d, 33e, and 33f, respectively.

[0033] In step S24, control unit 103 derives the wrinkles in the reference direction. The direction of each wrinkle is the angle formed when an extension of the direction of each wrinkle intersects with center line 30, assuming that the downward direction of center line 30 is 0 degrees, as shown in FIG. 3C . For example, FIG. 3C shows angles θ1, θ2, and θ3 corresponding to wrinkle directions 34b, 34c, and 34d, respectively. The reference direction is, for example, the direction along the right nasolabial fold in FIG. 3C , and direction 34b corresponding to angle θ2 of approximately 60 degrees corresponds to the reference direction. Such a reference direction is set by any experiment, statistical analysis, or the like.

[0034] In step S25, control unit 103 derives the wrinkle intensity in the reference direction. The wrinkle intensity corresponds to the relative area of ​​all wrinkles detected in the eyelid region in the reference direction. For example, the wrinkle intensity is the ratio of the wrinkle area in the reference direction to the sum of the areas of all wrinkles. Alternatively, the wrinkle intensity may be the degree of deviation of the wrinkle area in the reference direction from the average area of ​​all wrinkles. The intensity of each wrinkle is derived, for example, by morphological processing.

[0035] In step S26, control unit 103 determines the surface condition of the eyelid region based on the wrinkle intensity in the reference direction. Control unit 103, for example, references surface condition determination information 108. Here, the surface condition of the eyelid region is an aging state, i.e., a person's age. Surface condition determination information 108 includes, for example, information associating wrinkle intensity in the reference direction with age. Control unit 103 determines age by selecting from surface condition determination information 108 a value that most closely resembles the wrinkle intensity derived in step S25 and extracting the age corresponding to the selected value. Surface condition determination information 108 is generated, for example, by collecting information on wrinkle intensity in the reference direction for multiple subjects in advance using the procedure described above, deriving a representative value (e.g., an average value) of wrinkle intensity in the reference direction for each age of the subjects, and associating the representative value of wrinkle intensity with age. Alternatively, surface condition determination information 108 may include a history of wrinkle intensity in a reference direction for the same person. In this case, for example, control unit 103 derives the amount of change in wrinkle intensity per unit elapsed time from the history, and can determine the progression of aging when the most recent amount of change exceeds a representative value (e.g., average value) of past amounts of change. Information indicating the determination result is sent from server device 10 to terminal device 12 and displayed on terminal device 12, allowing the user to confirm the result.

[0036] According to the above-described procedure, it is possible to quantitatively determine the surface condition of a person's face, that is, the state of aging, based on the size and extension direction of wrinkles in the eyelid area using a facial image of the person.

[0037] In the above description, the server device 10 corresponds to the "information processing device." However, the server device 10 and the terminal device 12 may cooperate to configure the "information processing device," or the terminal device 12 may correspond to the "information processing device."

[0038] In the above-described embodiment, the processing / control program that defines the operation of the terminal device 12 may be stored in the memory unit 102 of the server device 10 or in the memory unit of another server device, and may be downloaded to the terminal device 12 via the network 11, or may be stored in a computer-readable non-transitory recording / storage medium and read by the terminal device 12 from the medium.

[0039] Although the embodiments have been described above based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined or divided into one. [Explanation of symbols]

[0040] 10: Server device 11: Network 12: Terminal device 101, 121: Communications Department 102, 122: Storage section 103, 123: control unit 105, 125: Input section 106, 126: Output section 108 Surface condition determination information

Claims

1. A method for operating an information processing device, comprising: A first step of extracting a linear region from a predetermined portion of an image of a person's face, the linear region having characteristics different from other regions in the predetermined portion; a second step of determining the surface condition of the face based on the size and extension direction of the linear region; A method of operation including:

2. In claim 1, the predetermined area is an eyelid area, the linear regions are wrinkles, In the second step, the facial surface condition is determined based on the size of wrinkles having a reference extension direction. How it works.

3. a storage unit that stores determination information that associates the surface condition of a face with the size and extension direction of a linear region that is extracted from a predetermined portion of an image of a person's face and has characteristics that differ from other regions in the predetermined portion; a control unit that extracts a first linear region from a first face image and determines a surface state of the first face corresponding to the first linear region using the determination information; An information processing device having the above.

4. In claim 3, the predetermined area is an eyelid area, the linear regions are wrinkles, the control unit determines the first facial surface condition based on the size of a wrinkle having a reference extension direction; Information processing device.

5. When executed by an information processing device, the information processing device A first step of extracting a linear region from a predetermined portion of an image of a person's face, the linear region having characteristics different from other regions in the predetermined portion; a second step of determining the surface condition of the face based on the size and extension direction of the linear region; Run the program.

6. In claim 5, the predetermined area is an eyelid area, the linear regions are wrinkles, In the second step, the facial surface condition is determined based on the size of wrinkles having a reference extension direction. program.

Images