Imaging guidance device, imaging guidance method, and program

Abstract

To provide an imaging guidance device capable of guiding the imaging state of a user to acquire facial images suitable for skin analysis, an imaging guidance method, and a program.SOLUTION: The imaging guidance device includes: a storage unit that stores conditions to be targeted for the characteristics of a regional image, which is the image obtained by photographing a region; a regional information acquisition unit that acquires a piece of regional information concerning a region of a face to be analyzed of a subject; a regional image extraction unit that extracts a regional image of the facial part to be analyzed from the facial image that includes the facial part to be analyzed; and a subject guidance unit that guides the imaging state of the subject based on the acquired regional information, the extracted regional image, and target values of characteristics corresponding to the regional information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an imaging guidance device, an imaging guidance method, and a program. [Background technology]

[0002] In recent years, image-based facial recognition using a captured image of a user's face or a portion of the image has been introduced as a contactless method of biometric authentication. In such facial recognition, a facial image of the user's entire face is captured, and the user is guided to a predetermined position. For example, when the captured image of the user is displayed on a camera screen on a display, a face frame is superimposed on the camera screen, and the user is guided to move their face so that it fits into the face frame. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-280277 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-268248 Summary of the Invention [Problem to be solved by the invention]

[0004] In this way, in facial authentication, it is desirable that the user's entire face be captured at a predetermined illuminance or higher. Meanwhile, facial images captured by a camera are also used in skin analysis. For example, a staff member at a cosmetics counter operates a camera to capture a customer's skin, and the captured skin image is transmitted to a personal computer (PC) in the store and to a server on a network for skin analysis. Alternatively, a customer operates a smartphone to capture a picture of their own skin, and the captured skin image is transmitted to a server on a network for skin analysis.

[0005] However, when performing skin analysis, there are camera angles, lighting positions, and other factors that are appropriate for skin analysis, making it difficult for users without specialized knowledge to determine whether a facial image suitable for skin analysis has been captured. Particularly in online environments, staff cannot directly move the imaging device or correct the user's facial orientation, making it difficult to obtain a facial image suitable for skin analysis. Furthermore, since the required image for skin analysis varies depending on the subject and area of ​​analysis, it is difficult to provide an intuitive indicator that is easy for even amateurs to understand, such as in facial recognition, where a sufficiently large and bright image of the entire face is sufficient. For example, when analyzing wrinkles, if lighting is irradiated parallel to the wrinkles of the target, the wrinkles may not be properly imaged. Therefore, there is a problem in that it is not possible to meet the demand for obtaining a facial image suitable for skin analysis.

[0006] Non-limiting examples of the present disclosure contribute to providing an image capture guidance device, an image capture guidance method, and a program that can guide a user's image capture state so as to acquire a face image suitable for skin analysis. [Means for solving the problem]

[0007] An imaging guidance device according to one embodiment of the present disclosure includes a memory unit that stores, for each facial part, conditions to be set as targets for the characteristics of a part image, which is an image of the part; a part information acquisition unit that acquires part information regarding the facial part of the subject that is to be analyzed; a part image extraction unit that extracts part images of the facial part of the subject that is to be analyzed from a face image including the facial part of the subject that is to be analyzed; and a subject guidance unit that guides the imaging state of the subject based on the acquired part information, the extracted part images, and target values ​​of the characteristics corresponding to the part information.

[0008] An imaging guidance method according to one embodiment of the present disclosure is executed by a computer and includes storing, for each facial part, conditions to be targeted for the characteristics of a part image, which is an image of the part; acquiring part information regarding the facial part of a subject that is to be analyzed; extracting the part image of the facial part that is to be analyzed from a facial image that includes the facial part that is to be analyzed; and guiding the imaging state of the subject based on the acquired part information, the extracted part image, and target values ​​of the characteristics that correspond to the part information.

[0009] A program according to one embodiment of the present disclosure causes a computer to perform the following steps: store, for each facial part, conditions to be targeted for the characteristics of a part image, which is an image of the part; acquire part information regarding the facial part of a subject that is to be analyzed; extract part images of the facial part that is to be analyzed from a facial image that includes the facial part that is to be analyzed; and guide the imaging state of the subject based on the acquired part information, the extracted part images, and target values ​​of the characteristics corresponding to the part information. [Effects of the Invention]

[0010] According to an embodiment of the present disclosure, it is possible to provide an image capture guidance device, an image capture guidance method, and a program that can guide a user's image capture state so as to acquire a face image suitable for skin analysis. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic diagram of an imaging guidance device 100 according to an embodiment of the present disclosure; [Figure 2A] FIG. 1 is a diagram showing a procedure for skin analysis using the imaging guidance device 100 according to an embodiment of the present disclosure. [Figure 2B] FIG. 1 is a diagram showing a procedure for skin analysis using the imaging guidance device 100 according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a block diagram showing a hardware configuration of an imaging guidance device 100 according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a block diagram showing a functional configuration of an imaging guidance device 100 according to an embodiment of the present disclosure. [Figure 5] 1 is a flowchart illustrating an imaging state guidance process according to an embodiment of the present disclosure. [Figure 6] 10 is a flowchart showing an imaging state guidance process according to a first modified example of the present disclosure. [Figure 7] 10 is a flowchart showing an imaging state guidance process according to a second modification of the present disclosure. [Figure 8A] 10 is a flowchart showing an imaging state guidance process according to a third modified example of the present disclosure. [Figure 8B] 10 is a flowchart showing an imaging state guidance process according to a third modified example of the present disclosure. [Figure 9] 10 is a flowchart showing an imaging state guidance process according to a fourth modified example of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a schematic diagram of an imaging guidance device 100 according to an embodiment of the present disclosure. As shown in FIG. 1, after acquiring a user's facial image, the imaging guidance device 100 guides the user to an imaging position appropriate for the facial region to be analyzed using audio guidance, text guidance, image guidance, etc., and outputs the user's skin analysis results based on the user's facial image at the imaging position. The imaging guidance device 100 may be a terminal device such as a personal computer or tablet installed in a cosmetics counter, for example, and may be used to perform a user's skin analysis based on a facial image captured by a camera. Alternatively, the imaging guidance device 100 may be a server installed on a network, connected online to a user's terminal device such as a smartphone or camera, and perform a user's skin analysis based on the facial image received from the terminal device.

[0013] The imaging guidance device 100 is not limited to the above configuration and may be any other information processing device. The configuration of the imaging guidance device 100 will be described in detail later.

[0014] (Skin analysis procedure) The procedure for performing skin analysis will be described below. Figures 2A and 2B are diagrams showing the procedure for skin analysis using the imaging guidance device 100 according to the embodiment of the present disclosure.

[0015] (1) The imaging guidance device 100 receives a face image captured by the user's terminal device.

[0016] (2) Based on the received facial image, the imaging guidance device 100 generates guidance information for guiding the user's imaging state and displays the guidance information on the display unit. The staff member operating the imaging guidance device 100 refers to the guidance information and gives instructions to the user, such as changing the direction of the face or adjusting the brightness of the lighting, to obtain a facial image suitable for skin analysis. Details of the guidance information will be described later.

[0017] (3) When the user changes the direction of their face according to the instructions of the staff member operating the imaging guidance device 100, a facial image suitable for skin analysis is displayed on the display unit of the imaging guidance device 100, and the facial image is captured as shown in FIG. 2B. The imaging guidance device 100 starts skin analysis based on the captured right cheek image. The facial image may be captured by remotely operating a facial image capture device (not shown) held by the staff member or the user, or by the user operating the facial image capture device in response to instructions from the staff member. In this case, the imaging guidance device 100 may notify the staff member whether or not the capture is ready, i.e., whether or not a facial image suitable for skin analysis can be captured. The imaging guidance device 100 may also be configured to instruct the facial image capture device to capture a facial image when it determines that a facial image suitable for skin analysis can be captured. This guidance method can be used not only for instructions from the staff member to the user, but also when the user operates the imaging device alone.

[0018] (4) Next, the imaging guidance device 100 analyzes the amount of wrinkles on the cheeks, the amount of spots, the length of the nasolabial folds, etc. using a known skin analysis method, stores the skin analysis results, and displays the skin analysis results on the user's terminal device via an online conference tool or the like. By using an online conference tool, the staff and the user can share the skin analysis results in real time. In addition to sharing the skin analysis results via an online conference tool or the like, the imaging guidance device 100 may also send the skin analysis result data to the user's terminal device so that the user can check the skin analysis results at any time. The skin analysis results may include an image of the right cheek superimposed with an image of wrinkles, an image of a barometer showing the degree of spots, etc., as shown in FIG. 2B .

[0019] (5) The terminal device that receives the skin analysis result information provides the skin analysis result to the user. This allows the staff operating the image capture and guidance device 100 and the user to share the skin analysis result. The staff operating the image capture and guidance device 100 can recommend to the user the most suitable cosmetics, skin care products, etc. based on the skin analysis result.

[0020] (Facial image suitable for skin analysis) Facial images suitable for skin analysis may include:

[0021] (1) If the skin image is too bright due to localized light exposure and sebum, it may be difficult to properly analyze the condition of wrinkles, pores, etc. Therefore, a facial image in which the area to be analyzed is not too bright can be an image suitable for skin analysis. Also, even if the cheeks are bright, a facial image in which the area to be analyzed is a part other than the cheeks and the part is not bright can be an image that can be used for skin analysis.

[0022] (2) Facial images captured in dark places or with the shadow of a terminal device cast on them may not be able to accurately analyze the condition of wrinkles, pores, blemishes, etc. due to low illumination. For this reason, facial images that do not have dark areas in the areas where skin analysis is to be performed are suitable for skin analysis. In this case, as long as the areas where skin analysis is to be performed are sufficiently bright, it is acceptable for other areas to be dark.

[0023] (3) Facial images captured using lighting equipment that radiates light so that it evenly illuminates the entire face are suitable for facial recognition, but omnidirectional light can obscure blemishes, wrinkles, and other imperfections. Therefore, facial images in which light of a predetermined level of brightness is irradiated from a certain direction on the area to be analyzed, rather than irradiated uniformly across the entire face, can be suitable for skin analysis. Even if the entire face is not uniformly bright, as long as the area to be analyzed is irradiated with light of a predetermined level of brightness, and other areas are irradiated with light of a lower or higher level of brightness, the image can still be suitable for skin analysis. In other words, as long as the light irradiating the area to be analyzed is appropriate, the condition of the light irradiating other areas is not important. Specifically, for example, even if areas other than the area to be analyzed are dark, or if blemishes and wrinkles other than the area to be analyzed are obscured, the image can still be suitable for skin analysis of the target area.

[0024] Note that the images suitable for skin analysis exemplified above are merely examples, and whether a facial image is suitable for skin analysis may vary depending on the target area and the content of the analysis. For example, when analyzing the amount of sebum on the skin based on the reflection of light from sebum, a facial image in which the area to be analyzed is shining, as opposed to the example described above, may be suitable for skin analysis. Therefore, when configuring an imaging guidance device 100 that can accommodate multiple skin analysis purposes, the imaging guidance device 100 stores the target area for each skin analysis purpose and the characteristics of the facial image suitable for skin analysis, and reads this information according to the purpose of skin analysis specified by the staff or user to determine whether the image is suitable for skin analysis.

[0025] In order to obtain a facial image suitable for skin analysis in this way, it is effective to guide the user's imaging state by adjusting the camera position, facial direction, lighting position, lighting brightness, etc.

[0026] Next, an example of the hardware configuration of the imaging guidance device 100 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the hardware configuration of the imaging guidance device 100 according to an embodiment of the present disclosure.

[0027] The imaging guidance device 100 is realized by a computing device such as a smartphone, a tablet, or a personal computer, and may have a hardware configuration such as that shown in Fig. 3. That is, the imaging guidance device 100 has a storage device 101, a processor 102, a user interface device 103, and a communication device 104, which are interconnected via a bus B.

[0028] The programs or instructions that realize the various functions and processes described below in the imaging guidance device 100 may be downloaded from any external device via a network, etc., or may be provided from a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or flash memory.

[0029] Storage device 101 may be implemented by random access memory, flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 101 may also include a non-transitory storage medium.

[0030] The processor 102 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the imaging guidance device 100 described below in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the storage device 101.

[0031] The user interface device 103 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a touch panel, and realizes an interface between the user and the imaging guidance device 100. For example, the user operates the imaging guidance device 100 by operating a GUI (Graphical User Interface) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0032] The communication device 104 is realized by various communication circuits that execute communication processes with external devices, the Internet, a communication network such as a LAN (Local Area Network), and the like.

[0033] However, the above-described hardware configuration is merely an example, and the imaging guidance device 100 according to the present disclosure may be realized by any other appropriate hardware configuration. For example, the functions of the imaging guidance device 100 may be distributed and realized by multiple computers. In this case, information is exchanged between the functions of different computers via a communication network (LAN or the Internet) connecting the computers.

[0034] 4 is a block diagram showing a functional configuration of an imaging guidance device 100 according to an embodiment of the present disclosure. The imaging guidance device 100 includes an image acquisition unit 110, a display control unit 111, a body part information acquisition unit 112, a body part image extraction unit 113, and a subject guidance unit 114.

[0035] (Image acquisition unit 110) The image acquisition unit 110 acquires a facial image capturing a user's facial expression. Specifically, a camera captures an image of the user's face, and the image acquisition unit 110 acquires the image of the user's face captured by the camera. The camera captures an image of a measurement target area and generates an RGB image of the measurement target area. For example, the camera may be a monocular camera that generates a monocular RGB image. Instead of using a camera, an image may be acquired by capturing video received via a network.

[0036] (Display control unit 111) The display control unit 111 displays the facial image on the display of the imaging and guidance device 100. When a target area for skin analysis is selected from the displayed facial image, part information about the selected facial part is transmitted to the part information acquisition unit 112 and the part image extraction unit 113. The target area for skin analysis is selected by, for example, a touch operation on the screen of the user's terminal device, or by operating a mouse connected to the imaging and guidance device 100.

[0037] (Part information acquisition unit 112) The part information acquisition unit 112 acquires part information (e.g., information indicating the position of the cheek) regarding the selected face part (target part for skin analysis) based on facial feature points of face parts such as the cheeks, between the eyebrows, and forehead. Furthermore, the part information acquisition unit 112 acquires the target brightness value of the selected face part from a server or the like. For example, the part information may include, for each face part to be analyzed, the position of the part on the face and the target brightness value of the part.

[0038] (region image extraction unit 113) The part image extraction unit 113 analyzes facial feature points from a face image including the facial part of the analysis target, and extracts a part image (an image of spots, wrinkles, pores, etc.) of the facial part of the analysis target based on the position of the part in the part information (for example, an area defined by one or more facial feature points that identify the position). The part image extraction unit 113 transmits the extracted part image to the subject guidance unit 114.

[0039] (Subject guidance section 114) The subject guidance unit 114 acquires the part information and the part image, and generates guidance information based on the part information and the part image. Specifically, the subject guidance unit 114 identifies the distribution of brightness (degree of brightness) of the part image extracted by the part image extraction unit 113. For example, a high-brightness area is considered to be close to the light, and a low-brightness area is considered to be far from the light. This makes it possible to identify the direction from which light is irradiating the face. Furthermore, the subject guidance unit 114 uses a well-known face direction estimation technique to calculate the direction of the user's face based on the positional relationship of facial feature points obtained from the face image. This makes it possible to identify whether the face is facing directly toward the camera (the face is looking straight at the camera) or is facing obliquely toward the camera.

[0040] Having thus identified the lighting direction, the user's facial orientation, and the like, the subject guiding unit 114 calculates the user's facial orientation and lighting direction such that the luminance value of the target area for skin analysis acquired by the area information acquisition unit 112 falls within the aforementioned target luminance value range. The subject guiding unit 114 generates guidance information for guiding the user's imaging state based on the calculated facial orientation and lighting direction. For example, if the luminance value of the target area is below the lower limit of the target luminance value, the subject guiding unit 114 may guide the user to move their face closer to the direction of the lighting light source. Alternatively, if the luminance value of the target area exceeds the upper limit, the subject guiding unit 114 may guide the user to move away from the direction of the lighting light source.

[0041] (Guidance information) The guidance information is information that guides the user in the imaging state (camera position, face direction, lighting position, etc.) so that an image suitable for skin analysis can be obtained. The guidance information may include the following information. (1) Guidance to guide the user to turn their face in a specific direction. (2) Guidance to guide the user to point the terminal device in a specific direction. (3) Guidance prompting the user to adjust the distance from the user's terminal device to the user's face. (4) Guidance to encourage users to adjust the direction and brightness of the light illuminating their face.

[0042] The guidance may be, for example, instructions such as "Slowly move your smartphone to the right," "Park your smartphone in that position," "Rotate your face to the left," "Park your face in that position," or "Slightly dim the lights." Various modes of outputting the guidance are conceivable. For example, the guidance may be displayed as text or images on a terminal used by a staff member, and the staff member may read the guidance to the user. This configuration is useful when the user does not have a screen on which the guidance information can be viewed, such as when the user captures a facial image using the rear camera of the smartphone. The guidance information may also be provided to the user's own imaging device (such as a smartphone). This configuration is useful when the user has a screen on which the guidance information can be viewed, such as when the user captures a facial image using the smartphone's front camera. The guidance may also be provided as an image. For example, if the user has a screen on which the guidance information can be viewed, an arrow or the like may be displayed indicating the direction in which the face or imaging device (such as a smartphone) should be moved. If the user's current facial image is visible to the user, a face frame guide or the like may be displayed.

[0043] Next, an imaging state guidance process according to an embodiment of the present disclosure will be described with reference to Fig. 5. The imaging state guidance process is executed by the imaging guidance device 100 described above, and more specifically, may be realized by one or more processors 102 of the imaging guidance device 100 executing one or more programs or instructions stored in one or more storage devices 101. For example, the face type diagnosis process may be started by a user of the imaging guidance device 100 launching an application or the like related to the process.

[0044] 5 is a flowchart illustrating an imaging state guidance process according to an embodiment of the present disclosure. As shown in FIG. 5, in step S101, the imaging guidance device 100 acquires a face image. Specifically, a camera captures an image of a user's face, and the imaging guidance device 100 acquires the image of the user's face captured by the camera.

[0045] Next, in step S102, the imaging guidance device 100 acquires part information (e.g., information indicating the position of the cheek) regarding the part of the skin analysis selected from the face image. For example, the part information may include the position of the part and the target brightness value for each part of the face to be analyzed.

[0046] Next, in step S103, the imaging guidance device 100 analyzes facial feature points from a face image including the facial part to be analyzed, and extracts a part image (image of spots, wrinkles, pores, etc.) of the facial part to be analyzed based on the facial feature points and the position of the part in the part information.

[0047] Next, in step S104, the imaging guidance device 100 identifies the brightness distribution of the part to be analyzed, the direction of the user's face, and the like, based on the part information and the part image.

[0048] Next, in step S105, the imaging guidance device 100 calculates the orientation of the user's face, the direction of lighting, and the like, so that the luminance value of the target area for skin analysis falls within the range of target luminance values.

[0049] Finally, in step S106, the subject guiding unit 114 generates and outputs guidance information for guiding the imaging state of the user based on the calculated face orientation and lighting direction.

[0050] The skin analysis process of the imaging guidance device 100 may include the following aspects.

[0051] (First Modification) Facial wrinkles have different optimal lighting conditions depending on the region. For example, forehead wrinkles are often formed by horizontal lines, so forehead wrinkles can be analyzed accurately by illuminating the face from above or below. On the other hand, brow wrinkles are often formed by vertical lines, so forehead wrinkles can be analyzed accurately by illuminating the face from the left or right. Therefore, in this modification, the region information further includes a target lighting direction for each facial region. The subject guidance unit 114 may be configured to guide the user's imaging state so that the luminance value indicating the luminance of the facial region to be analyzed falls within a target range and so that the lighting direction falls within a target direction range, depending on the facial region to be analyzed.

[0052] Fig. 6 is a flowchart showing an imaging state guidance process according to a first modified example of the present disclosure. The difference from Fig. 5 is that step S105A has been added in Fig. 6. Below, a description of the same processes as those in the steps shown in Fig. 5 will be omitted, and different processes will be described.

[0053] In step S105A, the subject guiding unit 114 refers to a table in which target areas of skin analysis are associated with facial orientations, and reads information about the facial orientation corresponding to the target areas of skin analysis acquired by the area information acquiring unit 112. Then, the subject guiding unit 114 guides the user so that the luminance value of the target area of ​​skin analysis falls within a target luminance value range and the lighting direction falls within a target orientation range. Alternatively, the subject guiding unit 114 may change the position of the lighting on the face, the position of the mobile device capturing the user's image, etc.

[0054] In this way, the subject guidance unit 114, which has calculated the angle of the user's face, etc., generates and outputs guidance information in step S106 according to the target area of ​​skin analysis so that the brightness value indicating the brightness of the target area falls within the target range and the lighting direction falls within the target orientation range.

[0055] For example, the subject guiding unit 114 generates guidance information such as "Tilt your face down about 45 degrees" when analyzing wrinkles on the forehead, and "Rotate your face about 45 degrees to the left" when analyzing wrinkles between the eyebrows. By guiding the user's imaging state for each target area of ​​skin analysis in this way, it is possible to adjust the lighting to suit the area to be analyzed, thereby enabling accurate analysis of the analysis area.

[0056] (Second Modification) In the first modified example, the imaging state of the user is changed depending on the facial part to be analyzed, but for example, when wrinkles on the forehead and wrinkles between the eyebrows are analyzed simultaneously, it is effective to guide the user to move the face up, down, left, and right so that the brightness values ​​of each part fall within the range of the target brightness value. Therefore, the subject guidance unit 114 may be configured to guide the imaging state of the user so that multiple parts to be analyzed (for example, the entire face) fall within the range of predetermined conditions.

[0057] Fig. 7 is a flowchart showing an imaging state guidance process according to a second modified example of the present disclosure. The flowchart differs from Fig. 6 in that step S105B is added instead of step S105A. Below, a description of the same processes as those in the steps shown in Fig. 5 will be omitted, and different processes will be described.

[0058] In step S105B, the subject guidance unit 114 reads information about the position, target luminance value, and target orientation corresponding to each target area by referencing the area information for each area to be analyzed. The subject guidance unit 114 then guides the user so that all luminance values ​​for each target area of ​​the skin analysis fall within the target luminance value range and the lighting direction falls within the target orientation range. For example, the subject guidance unit 114 may guide the user to change the position of the lighting on the face, the position of the mobile device capturing the user's image, etc.

[0059] In this way, the subject guidance unit 114, which has calculated the angle of the user's face, etc., generates and outputs guidance information in step S106 so that all the luminance values ​​of each target area of ​​the skin analysis fall within the target range and the lighting direction falls within the target orientation range.

[0060] In general, it is more difficult to keep all the luminance values ​​of multiple body parts within the target range than it is to keep the luminance value of a single body part within the target range, so the target range for each body part in this modification may be wider than the target range when only a single body part is analyzed.

[0061] For example, when simultaneously analyzing wrinkles on the forehead and between the eyebrows, the subject guiding unit 114 generates guidance information such as, "Turn your face slowly to the left while tilting your face slightly downward." In this way, by guiding the user's imaging state so that all the luminance values ​​of the target areas for skin analysis fall within the range of target luminance values ​​and the lighting direction falls within the range of target directions, it is possible to perform skin analysis of multiple areas in a short time compared to when the user's imaging state is changed for each area to be analyzed.

[0062] (Third Modification) Even if the imaging state of the user is guided by a certain guidance method, there are cases where an appropriate image cannot be obtained by just one guidance, and in such cases, it is preferable to change the guidance mode. Therefore, the subject guidance unit 114 may be configured to compare the imaging state of the user before and after guidance, and change the method of guiding the imaging state of the user depending on the comparison result.

[0063] 8A and 8B are flowcharts showing an imaging state guidance process according to a third modified example of the present disclosure. The flowchart differs from that of FIG. 5 in that processes such as step S107 have been added. Below, a description of the same processes as those in the steps shown in FIG. 5 will be omitted, and different processes will be described.

[0064] For example, when wrinkles on the forehead are analyzed, the subject guiding unit 114 outputs the guidance information and then calculates the change in the luminance value of the forehead in step S107.

[0065] If the brightness value is changing to approach the target range (step S108, YES), the direction of the user's face, the direction of the light from the lighting, etc. are appropriate, so the guidance information continues to be output without changing the content of the guidance information (step S109).

[0066] If the forehead luminance value has not changed to approach the target range (step S108, NO), the orientation of the user's face, the direction of light from the lighting, etc. are not appropriate, so in step S110, the subject guidance unit 114 calculates the amount of change in the difference between the forehead luminance value and the target range.

[0067] In step S111, the subject guiding unit 114, which has calculated the difference, changes the guiding method depending on the amount of change in the calculated difference. For example, guidance messages may be prepared in advance in a table or the like, and the subject guiding unit 114 may use different guidance messages depending on the amount of change in the difference to guide the user. Alternatively, the guidance information may be changed from text format to image format or audio format.

[0068] A series of processes is executed until the brightness value of the forehead falls within the target range. As a result, even if a facial image suitable for skin analysis cannot be obtained with a single guidance, a facial image suitable for skin analysis can be displayed in a short time by changing the guidance mode.

[0069] The change of the guidance information in this modification may be a process of subdividing the target guidance content. For example, if the guidance content is intended to have the user look diagonally upward to the right, and the brightness value is not approaching the target range, the guidance may be divided into two stages: "Turn your face to the right" and "Turn your face up." The change of the guidance information in this modification may be a process of switching the equivalent guidance content to a different expression. For example, if the guidance "Turn your face to the right" is being given but the brightness value is not approaching the target range, the expression may be changed to "Move the imaging device to the left." The subdivision of the guidance content before or after the change, or the target to be moved or rotated, may be appropriately changed.

[0070] Furthermore, a predetermined time limit may be set for determining whether the brightness value is changing to approach the target range. If the brightness value does not improve immediately after guidance is given, it is possible that the user has not yet performed an action and the guidance content itself has been accurately conveyed. Therefore, changing the guidance content simply because the brightness value has not approached the target range may confuse the user. On the other hand, if the brightness value has not approached the target range for a long time, it is highly likely that the user has not accurately understood the guidance content, and therefore it is highly likely that switching the guidance content will improve the situation.

[0071] Furthermore, in changing the guidance information in this modified example, even if the brightness value is out of the target range, as long as the change is within a predetermined range, the guidance content may not be changed. This is because the brightness value of the face may temporarily change in a direction away from the target range due to the surrounding environment, hand shake, etc., even if the user tries to move as instructed.

[0072] (Fourth Modification) When performing a skin analysis, it is preferable to guide the user's imaging state so that it is the same as the imaging state (face direction, lighting, distribution of brightness values, standard deviation of brightness values, etc.) when a skin analysis was performed in the past.

[0073] Therefore, the subject guidance unit 114 may be configured to store past imaging states that guided the user's imaging state in chronological order, and when guiding a new imaging state, guide the subject's imaging state so that it is equal to the stored past imaging state.

[0074] Fig. 9 is a flowchart showing an imaging state guidance process according to a fourth modified example of the present disclosure. The difference from Fig. 5 is that Fig. 9 adds a process of step S102A instead of the process of step S102. Below, a description of the same processes as those in the steps shown in Fig. 5 will be omitted, and different processes will be described.

[0075] In step S102A, the part information acquisition unit 112 acquires part information on the part for skin analysis selected from the face image, and reads out part information (which may include the position of the facial part to be analyzed, a target brightness value, a target standard deviation, etc.) in the stored previous imaging state. Then, in step S106, when guiding a new imaging state, the subject guiding unit 114 guides the imaging state of the subject so that it is equal to the imaging state read out in step S102A.

[0076] This makes it possible to perform skin analysis in an environment similar to the imaging conditions used when skin analysis was performed in the past, allowing the staff operating the imaging guidance device 100 to recognize changes in the skin over time, indicating how much the condition of the analysis area has improved compared to past skin analysis results, and allowing them to recommend the most appropriate cosmetics, care products, etc. to the user based on the analysis results.

[0077] As described above, the imaging guidance device according to an embodiment of the present disclosure includes a memory unit that stores, for each facial part, the conditions that should be set as targets for the characteristics of the part image, which is an image of the part; a part information acquisition unit that acquires part information related to the facial part of the subject that is the target of analysis; a part image extraction unit that extracts the part image of the facial part that is the target of analysis from a face image that includes the facial part that is the target of analysis; and a subject guidance unit that guides the imaging state of the subject based on the acquired part information, the extracted part image, and target values ​​of the characteristics corresponding to the part information.

[0078] With this configuration, even when a user without specialized knowledge takes a facial image by himself, he can be guided in the direction of the face and the imaging device so as to capture an image suitable for skin analysis. As a result, even when a user takes a facial image for skin analysis by himself in an online environment, it is possible to obtain a facial image that can be used for skin diagnosis with the same accuracy as when a professional staff member takes a facial image for skin analysis.

[0079] In the above-described embodiment, an online environment was used as an example, but in an environment where it is difficult for staff to directly move the imaging device or the user's face, the same effect can be achieved by using the imaging guidance device of this embodiment.

[0080] (Other variations) In the above-described embodiment, characteristics related to luminance are mainly used to evaluate whether a facial image is suitable for skin analysis. However, whether a facial image is suitable for skin analysis may also be evaluated based on other characteristics, such as the size of the area to be analyzed or the position of the area to be analyzed within the facial image. That is, in the above-described embodiment and modified examples, a desirable value of another characteristic, instead of luminance, may be stored in association with each area, and guidance may be provided to bring that characteristic closer to the desirable value depending on the area to be analyzed.

[0081] In the above-described embodiment, guidance for the movement or rotation of the face and guidance for the movement or rotation of the imaging device (e.g., a smartphone) may be interchanged. This is because movement or rotation of the imaging device in one direction and movement or rotation of the face in the opposite direction appear as the same change on the face image. Furthermore, by utilizing this property, in the second modified example, when changing the guidance content, the moving or rotating target may be switched between the imaging device and the user's face. This is particularly useful when the user's face has moved or rotated to the limit of the human body's range of motion or when the imaging device is fixed, and the movement or rotation of either the face or the imaging device is restricted. However, when illumination is irradiated from a biased direction, the movement or rotation of the imaging device and the movement or rotation of the face will produce different shadows, so interchanging the moving or rotating target is not appropriate. Therefore, when analysis of the face image indicates that illumination is irradiated from a biased direction, switching to guidance content that swaps the moving or rotating target may be suppressed even in the second modified example. Similarly, when guiding brightness or lighting conditions, the same results can be obtained whether the face, image capture device, or lighting is moved, so the target of guidance can be switched as needed.

[0082] It should be noted that, for example, the following aspects are also understood to fall within the technical scope of the present disclosure.

[0083] (1) An imaging guidance device according to an embodiment of the present disclosure includes a memory unit that stores, for each facial part, conditions to be set as targets for the characteristics of a part image, which is an image of the part; a part information acquisition unit that acquires part information regarding the facial part of the subject that is the subject of analysis; a part image extraction unit that extracts the part image of the facial part that is the subject of analysis from a face image that includes the facial part that is the subject of analysis; and a subject guidance unit that guides the imaging state of the subject based on the acquired part information, the extracted part image, and target values ​​of the characteristics corresponding to the part information.

[0084] (2) The characteristic is a brightness value indicating the brightness of the part, and the condition is that the brightness value falls within a target range. The subject guidance unit guides the imaging state of the subject so that the brightness value falls within the target range according to the part of the face being analyzed.

[0085] (3) The characteristic is the way the light hits the part, and the subject guidance unit guides the imaging state of the subject so that the way the light that illuminates the face hits the face is changed depending on the part of the face being analyzed.

[0086] (4) The lighting direction is the direction in which the lighting hits the part, and the condition is that the direction in which the lighting hits the part falls within a range of a predetermined target direction. The subject guidance unit guides the imaging state of the subject so that the direction in which the lighting hits the face falls within the range of the target lighting direction, depending on the part of the face being analyzed.

[0087] (5) The part information acquisition unit acquires part information regarding multiple parts of the analysis target, the memory unit records target characteristic values ​​for each of the multiple parts of the analysis target, and the subject guidance unit guides the imaging state of the subject so that the multiple parts of the analysis target approach the target characteristic values.

[0088] (6) The target value of the characteristic is the target range of the characteristic value, and the condition of the characteristic is that the characteristic falls within the target range. The memory unit records different target ranges for the same characteristic depending on whether there is one or multiple parts to be analyzed, and the subject guidance unit changes the target range used for guidance depending on whether there is one or multiple parts to be analyzed.

[0089] (7) The subject guiding unit compares the imaging state of the subject before and after the guidance, and changes the method of guiding the imaging state of the subject according to the comparison result.

[0090] (8) The subject guidance unit stores the previous imaging state of the subject, and guides the imaging state of the subject so that it becomes equal to the stored previous imaging state.

[0091] (9) An imaging guidance method according to one embodiment of the present disclosure is executed by a computer, and includes storing, for each facial part, conditions to be targeted for the characteristics of a part image, which is an image of the part; acquiring part information regarding the facial part of the subject that is to be analyzed; extracting the part image of the facial part that is to be analyzed from a facial image that includes the facial part that is to be analyzed; and guiding the imaging state of the subject based on the acquired part information, the extracted part image, and target values ​​of the characteristics corresponding to the part information.

[0092] (10) A program according to one embodiment of the present disclosure causes a computer to perform the following operations: memorizing, for each facial part, conditions to be set as targets for the characteristics of a part image, which is an image of the part; acquiring part information regarding the facial part of the subject that is the target of analysis; extracting the part image of the facial part that is the target of analysis from a facial image that includes the facial part that is the target of analysis; and guiding the imaging state of the subject based on the acquired part information, the extracted part image, and target values ​​of the characteristics corresponding to the part information.

[0093] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims. [Explanation of symbols]

[0094] 100 Imaging guidance device 101 Storage device 102 processors 103 User Interface Device 104 Communication equipment 110 Image acquisition unit 111 Display control unit 112 Part information acquisition unit 113 Part Image Extraction Unit 114 Subject guidance section

Claims

1. a storage unit that stores, for each facial region, a condition that should be set as a target for the characteristics of a region image that is an image of the region; a region information acquisition unit that acquires region information regarding a region of the subject's face that is to be analyzed; a part image extraction unit that extracts part images of the facial parts to be analyzed from a face image including the facial parts to be analyzed; a subject guidance unit that guides an imaging state of the subject based on the acquired region information, the extracted region image, and a target value of the characteristic corresponding to the region information; An imaging guidance device having the same.

2. the characteristic is a luminance value indicating the luminance of the part, the condition is that the luminance value falls within a target range; The imaging guidance device according to claim 1 , wherein the subject guidance unit guides the imaging state of the subject so that the luminance value falls within a target range according to a part of the face to be analyzed.

3. the characteristic is how the part is illuminated, 3. The imaging guidance device according to claim 1, wherein the subject guidance unit guides the imaging state of the subject so as to change the way in which light that illuminates the face hits the face depending on the part of the face to be analyzed.

4. The lighting direction is a direction in which the lighting is irradiated onto the part, the condition is that the direction in which the light is irradiated onto the part falls within a range of a predetermined target direction; 4. The imaging guidance device according to claim 3, wherein the subject guidance unit guides the imaging state of the subject so that the direction in which the illumination shines on the face falls within a range of target directions of the illumination, depending on the part of the face to be analyzed.

5. the part information acquisition unit acquires part information relating to a plurality of parts of the analysis target, the storage unit records target values ​​of the characteristics for each of the plurality of parts to be analyzed, The imaging guidance device according to claim 2 , wherein the subject guidance unit guides the imaging state of the subject so that the plurality of parts of the analysis target approach target values ​​of the characteristics.

6. the target value of the characteristic is a target range of values ​​for the characteristic; the condition for the characteristic is that the characteristic falls within the target range; the storage unit records different target ranges for the same characteristic depending on whether the analysis target is one or multiple parts, The imaging guidance device according to claim 5 , wherein the subject guidance unit changes a target range used for guidance depending on whether the number of the analysis target regions is one or more.

7. 6. The imaging guidance device according to claim 1, wherein the subject guidance unit compares characteristics of the area image before and after guidance, and changes a method for guiding the imaging state of the subject depending on whether a comparison result indicates that the characteristics have changed to approach the target value.

8. The imaging guidance device according to claim 1 , wherein the subject guidance unit stores a past imaging state of the subject and guides the imaging state of the subject to be equal to the stored past imaging state.

9. storing, for each facial region, a condition to be set as a target for the characteristics of a region image, which is an image obtained by photographing the region; acquiring region information relating to a region of the subject's face to be analyzed; extracting a regional image of the facial region to be analyzed from a face image including the facial region to be analyzed; guiding an imaging state of the subject based on the acquired region information, the extracted region image, and a target value of the characteristic corresponding to the region information; 10. A computer-implemented imaging guidance method comprising:

10. storing, for each facial region, a condition to be set as a target for the characteristics of a region image, which is an image obtained by photographing the region; acquiring region information relating to a region of the subject's face to be analyzed; extracting a regional image of the facial region to be analyzed from a face image including the facial region to be analyzed; guiding an imaging state of the subject based on the acquired region information, the extracted region image, and a target value of the characteristic corresponding to the region information; A program that causes a computer to execute the following.

Images