Skin condition measurement system

Abstract

To provide a skin state measurement system which acquires data related to the skin state of a user with each imaging by a plurality of pieces of imaging means and improves the quality of information about the skin state of the user shown to the user on the basis of a plurality of pieces of skin image data.SOLUTION: A skin state measurement system images a prescribed range of the body of a user by a first imaging unit 11, images a prescribed portion of the skin of the user by a second imaging unit 12, acquires information about the skin state of the user by a control unit on the basis of the skin image data of the obtained prescribed range of the user and skin image data of the prescribed portion, improves the quality of the information about the skin state of the user on the basis of the image data obtained in the plurality of times of imaging when deriving the information about the skin state by analyzing the image data for the skin, performs processing such as display and analysis of the skin state with high accuracy, and allows the user to more correctly grasp the skin state by properly and efficiently implementing the diagnosis of the skin state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a skin condition measuring system capable of providing a user with information related to the user's skin condition. [Background technology]

[0002] There has long been a service in which stores are equipped with skin condition measuring devices and analytical equipment to measure customers' skin conditions, analyze the measurement data to obtain analysis results, and then recommend and sell to customers skin care cosmetics and beauty tools that correspond to the skin condition indicated by the analysis results.

[0003] In response to this, in recent years, systems have been proposed that use small, inexpensive devices that can be used personally by the user to detect the user's skin condition with a sensor, diagnose the skin condition based on the obtained data, output the diagnosis results on a display, etc., and set up and execute electronic treatments based on the diagnosis results. An example of such a conventional system for diagnosing skin conditions is disclosed in Japanese Patent Application Laid-Open No. 2004-141259. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-141259 A Summary of the Invention [Problem to be solved by the invention]

[0005] Conventional systems for diagnosing skin conditions have the configuration shown in the above-mentioned patent documents, in which a sensor is placed in contact with the user's skin, the skin condition detected by the sensor is diagnosed for the area of ​​skin that the sensor contacts, and the diagnosis results are output, allowing the user to select an appropriate type of treatment and its execution time according to their own skin condition.

[0006] However, in the conventional systems described above, the detection range of the sensor is small relative to the area of ​​skin where the skin condition is to be grasped, and the skin that is detected by the sensor is only a part of the entire area, so it is usually necessary to detect only a part of the skin area where the skin condition is to be grasped, and the diagnosis based on the detection results does not reflect the skin condition of the entire skin area, making it difficult to ensure the accuracy of the diagnosis.

[0007] The present invention has been made to solve the above-mentioned problems, and aims to provide a skin condition measurement system that obtains data related to a user's skin condition by capturing images using multiple imaging means, and improves the quality of information related to the user's skin condition that is presented to the user based on multiple sets of skin image data. [Means for solving the problem]

[0008] The skin condition measuring system disclosed in the present invention comprises a first imaging means capable of imaging a predetermined area of ​​the user's body skin, a second imaging means capable of imaging a predetermined part of the user's skin narrower than the predetermined area of ​​the body, and a control unit that acquires information regarding the skin condition based on skin image data of the predetermined area of ​​the body obtained by imaging with the first imaging means and skin image data of the predetermined part obtained by imaging with the second imaging means.

[0009] Thus, according to the disclosure of the present invention, a specified area of ​​the user's body is imaged with a first imaging means, and a specified part of the user's skin is imaged with a second imaging means, and the control unit obtains information regarding the user's skin condition based on the skin image data of the specified area of ​​the body and the skin image data of the specified part obtained, respectively.By analyzing the image data about the skin and deriving information regarding the skin condition, the quality of the information regarding the user's skin condition based on the image data obtained from multiple images can be improved, processing such as displaying and analyzing the skin condition can be performed with high accuracy, diagnosis of the skin condition can be carried out appropriately and efficiently, and the user can be allowed to understand their skin condition more accurately.

[0010] In addition, in the skin condition measuring system disclosed in the present invention, the second imaging means may, as necessary, image a predetermined part of the skin that overlaps with the predetermined body area.

[0011] As disclosed in the present invention, a specific part of the user's skin imaged by the second imaging means overlaps a specific area of ​​the user's body imaged by the first imaging means, and two sets of skin image data are obtained for the same specific part in the specific area of ​​the user's body.This makes it possible to understand the relationship between the image data obtained by the different imaging means from the common specific parts in the image data, and to infer the detailed skin condition of each part of the specific area of ​​the body other than the overlapping parts.In this way, detailed information regarding the skin condition can be obtained by analysis using each image data.This information regarding the skin condition allows the user to correctly understand their skin condition, further encouraging the user to take appropriate skin care measures, etc.

[0012] In addition, in the skin condition measuring system disclosed in the present invention, the second imaging means may, as necessary, image a predetermined part of the skin that does not overlap with the predetermined body area.

[0013] As disclosed in the present invention, the specified part of the user's skin imaged by the second imaging means does not overlap with the specified area of ​​the user's body imaged by the first imaging means, and by obtaining two sets of skin image data, one for the specified area of ​​the user's body and one for the specified area of ​​the skin outside this range, it is possible to derive information regarding the skin condition by comparing the two sets of skin image data. For example, by using the skin condition based on the skin image data of the narrower specified area of ​​the skin as a standard and comparing it with the skin condition of each part of the specified area of ​​the body based on the skin image data of the specified area of ​​the body, it is possible to determine the relative skin condition. In this way, information regarding the skin condition can be easily obtained by analysis using the two sets of skin image data, allowing the user to reliably understand their skin condition and further encouraging the user to take appropriate care and other measures regarding the skin in the specified area of ​​the body.

[0014] In addition, in the skin condition measuring system disclosed in the present invention, the imaging of the specified area of ​​the body by the first imaging means and the imaging of the specified part of the skin by the second imaging means are performed at different times, as necessary.

[0015] Thus, according to the disclosure of the present invention, imaging of a specified area of ​​the user's body by a first imaging means and imaging of a specified part of the user's skin by a second imaging means are performed at different times, eliminating the need for devices constituting each imaging means to be present around the user at the same time and perform each imaging respectively. This eliminates the need to arrange the imaging means as separate devices for simultaneous imaging, and makes it possible to reuse the devices, for example, by imaging each imaging target sequentially using an imaging device in which imaging means are arranged together, and the number of imaging devices can be minimized to reduce device costs.

[0016] In addition, in the skin condition measuring system according to the present disclosure, the first imaging means and the second imaging means are disposed in a predetermined terminal device, as necessary.

[0017] Thus, according to the disclosure of the present invention, by providing a first imaging means together with a second imaging means in a single terminal device and being able to perform imaging using both the first imaging means and the second imaging means on a single terminal device, imaging of the skin of a specified area of ​​the user's body and imaging of the specified part of the skin can be handled by a single terminal device, and the costs associated with imaging can be reduced by using only a single imaging device.

[0018] In addition, in the skin condition measuring system disclosed in the present invention, the first imaging means is switchable between a state of imaging the skin of a specified area of ​​the body and a state of imaging the specified part, as necessary, and also serves as the second imaging means.

[0019] Thus, according to the disclosure of the present invention, by using a first imaging means that can be switched between a state of imaging the specified body area and a state of imaging the specified part, and having this first imaging means also function as a second imaging means, and by imaging the specified body area and the specified part with one imaging means in one terminal device and obtaining image data for each, imaging can be performed with the minimum necessary configuration, and information related to the skin condition can be obtained at low cost.

[0020] In addition, in the skin condition measuring system disclosed in the present invention, as necessary, the first imaging means is a specified imaging device that is part of or connected to the user's terminal device and images at least the specified range of the user's body, and image data obtained by imaging is capable of being transmitted to another person's terminal device, and communication is performed between the user's terminal device and the other person's terminal device, and an image of the user including the specified range of the body is displayed on the other person's terminal device based on the transmitted image data of the user, while the control unit acquires information regarding the skin condition based on the skin image data of the specified range of the body obtained by imaging with the first imaging means and the skin image data of the specified part obtained by imaging with the second imaging means.

[0021] Thus, according to the disclosure of the present invention, image data of the user is transmitted to another person's terminal device from a first imaging means constituting an imaging device that is part of the user's terminal device or connected to this terminal device, and while the image of the user is displayed on the other person's terminal device, the control unit acquires information regarding the skin condition based on the skin image data of the specified area and the specified part of the body, and in some cases, it is possible to handle not only the image of the user but also information regarding the skin condition in communication between the user's terminal device and the other person's terminal device, so that information regarding the user's skin condition can be referenced while communication involving image display is being performed between the user's terminal device and the other person's terminal device, such as a video call, and the image of the user is displayed on the other person's terminal device, and the information regarding the skin condition can be utilized while communicating with the user via communication.

[0022] In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the first imaging means is a specified imaging device that is part of or connected to another person's terminal device and images at least the specified range of the user's body, and while the image data of the user obtained by the first imaging means is used on the other person's terminal device for a main purpose other than obtaining information regarding the user's skin condition, the control unit obtains information regarding the skin condition based on the skin image data of the specified range of the body obtained by imaging with the first imaging means and the skin image data of the specified part obtained by imaging with the second imaging means.

[0023] Thus, according to the disclosure of the present invention, a user is imaged with a first imaging means constituting an imaging device that is part of another person's terminal device or connected to this terminal device, and the user's image data is used for a specified purpose on the other person's terminal device, while the control unit acquires information regarding the skin condition based on skin image data of a specified area of ​​the body from the first imaging means and skin image data of the specified part from the second imaging means, and makes this information regarding the skin condition available.By doing so, image data captured and acquired for purposes such as personal authentication by the imaging device used together with the other person's terminal device can be reused for skin diagnosis, and there is no need to prepare a new first imaging means, and information regarding the skin condition based on each image data can be acquired efficiently and at low cost.

[0024] In addition, in the skin condition measuring system disclosed in the present invention, as necessary, the first imaging means is a predetermined imaging device that is part of or connected to the user's terminal device and images at least the predetermined range of the user's body, the second imaging means is capable of performing imaging by user operation, the control unit initially acquires information regarding the skin condition based on skin image data of the predetermined range of the body obtained by imaging with the first imaging means, and uses the information to set a predetermined area of ​​the skin to be imaged by the second imaging means and notifies the user of the information of the set predetermined area via the user's terminal device, and after the control unit obtains skin image data of the predetermined area in conjunction with imaging of the predetermined area of ​​the skin by the second imaging means, it again acquires information regarding the skin condition based on the skin image data of the predetermined range of the body and the skin image data of the predetermined area.

[0025] Thus, according to the disclosure of the present invention, an image of a user is captured by a first imaging means constituting an imaging device that is part of a user's terminal device or that is connected to the terminal device, and the control unit first acquires information regarding the skin condition based on the obtained skin image data of the predetermined area of ​​the user's body, and then sets a predetermined area of ​​the skin to be imaged by the second imaging means based on this information, and the information regarding this predetermined area is notified to the user from the user's terminal device to cause the user to image the predetermined area of ​​the skin with the second imaging means. Once skin image data of the predetermined area is obtained by imaging with the second imaging means, the control unit acquires information regarding the skin condition again and makes this information regarding the skin condition available, so that the predetermined area of ​​the skin to be imaged by the second imaging means is set according to the skin condition of the predetermined area of ​​the user's body. Areas where more detailed information on the skin condition is required, such as areas where there is a problem with the skin condition such as blemishes or signs of a problem, are first extracted, and the relevant areas are imaged by the second imaging means as the predetermined area of ​​the skin. Using the obtained skin image data, more detailed information regarding the skin condition focusing on the problem area can be obtained, and the user can properly grasp the skin condition.

[0026] In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the control unit acquires information regarding the skin condition of the specified part of the skin based on skin image data of the specified part obtained by imaging with the second imaging means, and then acquires information regarding the relative skin condition of each part of the specified range of the body based on the information regarding the skin condition of the specified part as information regarding the skin condition.

[0027] Thus, according to the disclosure of the present invention, a second imaging means captures an image of a predetermined part of the skin that does not overlap with the predetermined body area, and a control unit acquires information regarding the skin condition of the predetermined skin area based on the obtained skin image data of the predetermined area. The control unit further acquires information regarding the skin condition based on the skin image data of the predetermined body area as relative skin condition information for each part of the predetermined body area based on the information regarding the skin condition of the predetermined area, and makes the information regarding the skin condition available. This makes it possible to display the skin condition of the predetermined body area of ​​the user relatively based on the skin condition of the predetermined area. If the predetermined skin area is a location that is suitable as a standard for evaluation, for example, a part that is hidden from sunlight, such as the surface of the arm close to the torso or the underside of the chin, which is less affected by ultraviolet rays and has a small degree of deterioration, the skin condition of each part of the predetermined body area can be displayed as the degree of deterioration from the reference area, and the skin condition can be represented in a form that is easy for the user to understand, allowing the user to more accurately grasp the skin condition.

[0028] In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the first imaging means captures a mirror image of the specified area of ​​the body reflected on a mirror surface located in front of the user, and the second imaging means captures an image of a specified part of the user's skin simultaneously with the image captured by the first imaging means, and the control unit acquires information related to the skin condition based on skin image data of the specified area of ​​the body obtained by capturing the mirror image by the first imaging means and skin image data of the specified part obtained by capturing the image by the second imaging means.

[0029] Thus, according to the disclosure of the present invention, a first imaging means captures an image of the mirror image reflected on a mirror surface in front of the user to obtain skin image data of the specified area of ​​the body, and simultaneously, a second imaging means captures an image to obtain skin image data of the specified area, and a control unit obtains information regarding the skin condition from each piece of skin image data and makes this information regarding the skin condition available.By capturing images simultaneously with each imaging means, skin image data of the specified area of ​​the body and skin image data of the specified area can be obtained at once, the time from capturing images to obtaining information regarding the skin condition can be shortened, the burden on the user related to capturing images can be reduced, the user can be encouraged to continue activities such as capturing images to obtain information regarding the skin condition, and information regarding the skin condition can be obtained without wasting time, making it possible to quickly grasp the user's skin condition.

[0030] Furthermore, the skin condition measuring system disclosed in the present invention is capable of imaging the specified part of the user with the first imaging means disposed on one surface of a specified terminal device, the second imaging means disposed in the terminal device, and the first imaging means in the terminal device facing the mirror surface, as necessary.

[0031] Thus, according to the disclosure of the present invention, a first imaging means is arranged on one surface of a specified terminal device, and this first imaging means is capable of imaging the specified part of the user with the first imaging means facing the mirror surface, and a second imaging means is arranged in the same terminal device, and a single device can be used to image the specified area of ​​the user's body and the specified part, so that skin image data of the specified area of ​​the body and skin image data of the specified part can be obtained together by imaging with each imaging means of the terminal device, and when imaging, the user only needs to handle the terminal device equipped with each imaging means, which significantly reduces the user's burden related to imaging and allows the user to naturally continue activities to obtain information about their skin condition.

[0032] In addition, in the skin condition measuring system disclosed in the present invention, the control unit records information about a specific part of the skin to be imaged by the set second imaging means, as necessary, and when setting a new specific part of the skin to be imaged by the second imaging means, the control unit uses the information about the skin condition that was previously acquired based on the skin image data of the specific area of ​​the body, and also refers to the information about the recorded specific part to set the specific part from among parts of the specific area of ​​the body that have not been recorded as a specific part.

[0033] As described above, according to the disclosure of the present invention, the control unit first acquires information regarding the skin condition based on skin image data of the predetermined area of ​​the user's body obtained by imaging with the first imaging means, and when setting the predetermined area of ​​the skin to be imaged by the second imaging means based on the acquired information, the control unit records the information of the set predetermined area for each such setting, and makes the information of the set predetermined area available for later reference. If recorded information of a predetermined area exists, at a new imaging opportunity with the first imaging means, the control unit sets the predetermined area of ​​the skin to be imaged by the second imaging means from among areas of the predetermined area of ​​the body that have not been recorded as a predetermined area based on the information regarding the skin condition acquired based on the skin image data of the predetermined area of ​​the body, so that areas of the predetermined area of ​​the user's body that have not yet been imaged by the second imaging means are always imaged as the predetermined area of ​​the skin to be imaged by the second imaging means. By imaging the set predetermined area of ​​the skin with the second imaging means, detailed information regarding the skin condition centered on the new predetermined area can be acquired using the obtained skin image data, and the user can appropriately grasp the skin condition. In addition, when imaging is performed multiple times with the second imaging means for each imaging with the first imaging means, skin image data of different areas without overlapping is obtained for each imaging with the second imaging means, making it possible to obtain more detailed and accurate information regarding the skin condition based on the skin image data.

[0034] In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the control unit records skin image data obtained by imaging the specified area by the second imaging means, together with information about the specified area of ​​the skin to be imaged by the second imaging means, in a manner that makes it possible to determine the date and time when the skin image data was obtained, and in parallel with setting the specified area of ​​the skin to be imaged by the second imaging means at a new imaging opportunity with the first imaging means, the control unit refers to the recorded information about the specified area and skin image data, and acquires information regarding the skin condition based on the recorded skin image data of the specified area, whose data acquisition date and time falls within a predetermined past period corresponding to the user's skin turnover cycle, the skin image data of the specified area obtained by imaging the specified area of ​​the skin set as the area to be imaged by the second imaging means, and skin image data of the specified area of ​​the body.

[0035] According to the disclosure of the present invention, information on a specific part of the skin to be imaged by the second imaging means set by the control unit is recorded, and skin image data obtained by imaging the set specific part by the second imaging means is recorded. When a newly set specific part of the skin is imaged by the second imaging means to obtain skin image data of the specific part, the previously recorded information on the specific part and the skin image data are referenced, and the skin image data of the specific part whose data acquisition date and time is included in a previous specific period corresponding to the turnover cycle of the user's skin is deemed to have been imaged at the same timing as the new imaging. Information on the skin condition is obtained based on the previously recorded skin image data acquired during the specific period, the skin image data of the specific part obtained by the latest imaging, and the skin image data of the specific range of the body. Each time the specific part of the skin is imaged repeatedly by the second imaging means within the specific period corresponding to the turnover cycle of the skin, the skin image data of the specific part is recorded sequentially, thereby obtaining a state in which each part of the specific range of the body is widely covered. The information on the skin condition obtained based on these and the skin image data of the specific range of the body reflects the user's skin condition in detail and with good accuracy, allowing the user to properly grasp the skin condition.

[0036] In addition, in the skin condition measuring system disclosed in the present invention, while the control unit records information on a specific area of ​​the skin to be imaged by the second imaging means and skin image data of the specific area obtained by imaging the specific area by the second imaging means, as necessary, the control unit deletes recorded skin image data obtained before the previous specific period corresponding to the turnover cycle of the user's skin and recorded information of the specific area corresponding to the skin image data for each new imaging opportunity.

[0037] Thus, according to the disclosure of the present invention, by deleting skin image data recorded by the control unit that is taken before a predetermined period corresponding to the skin turnover cycle from the time when new imaging is to be performed, accurate information regarding the skin condition can be derived using only skin image data from a period considered to be the same skin condition, together with skin image data obtained by imaging with the second imaging means, and the amount of recording can be limited, thereby reducing the load associated with recording. [Brief description of the drawings]

[0038] [Figure 1] 1 is a schematic configuration diagram of a terminal device in a skin condition measuring system according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing the configuration of a skin condition measuring system according to a first embodiment of the present invention. [Diagram 3] 4 is an explanatory diagram showing a state in which a conversion lens is attached to a second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. FIG. [Figure 4] FIG. 2 is an explanatory diagram of a display screen of the terminal device immediately before imaging by a first imaging unit in the skin condition measuring system according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is an explanatory diagram of a display screen of the terminal device immediately before imaging by the second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. [Figure 6] 5A to 5C are explanatory diagrams illustrating an imaging state by a second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. [Figure 7]FIG. 2 is an explanatory diagram showing a display state of diagnostic information of a skin condition based on images captured by each imaging unit in the skin condition measuring system according to the first embodiment of the present invention. [Figure 8] FIG. 11 is an explanatory diagram of the display screen of the terminal device immediately before capturing an image by the second imaging unit at a new imaging opportunity in the skin condition measuring system according to the first embodiment of the present invention. [Figure 9] 5A to 5C are explanatory diagrams illustrating an image capturing state by another terminal device in the skin condition measuring system according to the first embodiment of the present invention. [Figure 10] FIG. 11 is a schematic configuration diagram of a skin condition measuring system according to a second embodiment of the present invention. [Figure 11] FIG. 11 is an explanatory diagram of information transmission and reception between a terminal device and a server in a skin condition measuring system according to a second embodiment of the present invention. [Figure 12] FIG. 11 is an explanatory diagram of the state of information transmission and reception between each terminal device of a user and another person and a server in a first alternative example of the skin condition measuring system according to the second embodiment of the present invention. [Figure 13] FIG. 11 is an explanatory diagram of an imaging state of another person's terminal device having a first imaging unit in a second alternative example of the skin condition measuring system according to the second embodiment of the present invention. [Figure 14] FIG. 11 is an explanatory diagram of the state of information transmission and reception between each terminal device and the server in a second alternative example of the skin condition measuring system according to the second embodiment of the present invention. [Figure 15] 13A to 13C are explanatory diagrams showing an image capturing state by a second imaging unit in a skin condition measuring system according to a third embodiment of the present invention. [Figure 16] FIG. 13 is an explanatory diagram of a display screen of a terminal device immediately before imaging by each imaging unit in another example of a skin condition measuring system according to the third embodiment of the present invention. [Figure 17] FIG. 13 is an explanatory diagram of a usage state of a terminal device in a skin condition measuring system according to a fourth embodiment of the present invention. [Figure 18] FIG. 11 is a block diagram showing the configuration of a skin condition measuring system according to a fourth embodiment of the present invention. [Figure 19] 13A to 13C are explanatory diagrams illustrating image capturing states by each image capturing unit in a skin condition measuring system according to a fourth embodiment of the present invention. [Figure 20] 13A and 13B are explanatory diagrams of an image capturing state by another terminal device in the skin condition measuring system according to the fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] (First embodiment of the present invention) Hereinafter, a skin condition measuring system according to a first embodiment of the present invention will be described with reference to FIGS. In each of the above figures, the skin condition measuring system 1 of this embodiment comprises a first imaging unit 11 as the first imaging means capable of imaging the skin of a specified area of ​​the user's body, a second imaging unit 12 as the second imaging means capable of imaging a specified part of the user's skin narrower than the specified area of ​​the body, and a control unit 13 that acquires information regarding the skin condition based on skin image data of the specified area of ​​the body obtained by imaging with the first imaging unit 11 and skin image data of the specified part obtained by imaging with the second imaging unit 12, and the first imaging unit 11, second imaging unit 12, and control unit 13 are configured to be arranged in a single terminal device 10.

[0040] In this embodiment, the user, who is the subject of skin condition measurement, enters into a contract to receive skin condition diagnosis services with a business operator (hereinafter referred to as the provider) that provides skin condition diagnosis services using the skin condition measurement system 1.

[0041] This allows the user to receive a skin condition diagnosis using the skin condition measuring system 1, obtain the skin condition diagnosis results from the skin condition measuring system 1, and, if necessary, enjoy the benefits of obtaining advice information according to the diagnosis results. Furthermore, the provider can obtain data regarding the user's skin condition, and can enjoy the benefit of realizing a sales promotion effect for the skin care products and skin care services provided by a specified provider (e.g., a sponsor of the provider) by recommending these products to the user through advertising, etc. Skin care products are products used for skin care, such as lotions, beauty serums, supplements, facial massagers, and facial cleansers. Skin care services are services for providing skin care, such as aesthetic services, etc.

[0042] The first imaging unit 11 is disposed in the terminal device 10 as the first imaging means, and is capable of capturing an image of a predetermined area of ​​the user's body, such as the face, for example, to measure the user's skin condition.

[0043] The second imaging unit 12, as the second imaging means, is arranged in the same terminal device 10 as the first imaging unit 11, and is capable of imaging a predetermined part of the user's skin, such as a part of the cheek or forehead, which is narrower than the predetermined area of ​​the user's body and overlaps with this predetermined area of ​​the body, in order to measure the user's skin condition. The second imaging section 12 is equipped with a conversion lens 19, so that it is possible to obtain image data having a higher resolution per unit area of ​​the imaging target than the first imaging section 11.

[0044] The first imaging unit 11 and the second imaging unit 12 are capable of performing imaging by user operation via the terminal device 10. The imaging of the predetermined body range by the first imaging unit 11 and the imaging of the predetermined part of the skin by the second imaging unit 12 are performed at different times and with different relative positional relationships between the imaging units and the imaging target.

[0045] Although the first imaging unit 11 and the second imaging unit 12 are each part of the user's terminal device 10, this is not limited thereto, and either or both of the first imaging means and the second imaging means may be independent imaging devices that are connected to the user's terminal device, perform imaging, and obtain skin image data.

[0046] In addition, the second imaging unit 12 is one of multiple camera units in a terminal device 10 such as a smartphone that is capable of imaging the user's skin, and is not limited to a configuration in which it is used with a conversion lens 19 suitable for imaging the skin attached, but can also be a camera unit in a terminal device such as a smartphone that is capable of directly imaging a specific part of the skin, and can be configured to not require a conversion lens.

[0047] Furthermore, the terminal device 10 is configured to have a first imaging unit 11 constituting the first imaging means and a second imaging unit 12 constituting the second imaging means. Alternatively, the terminal device may be configured to have one imaging unit that can switch between a state of imaging the skin of a specified area of ​​the user's body and a state of imaging the specified part; in other words, the imaging unit that is the first imaging means capable of imaging the skin of a specified area of ​​the user's body can also be configured to image the specified part as necessary and serve as the second imaging means.

[0048] In addition, the second imaging unit 12 constituting the second imaging means is configured to be provided in the same terminal device 10 as the first imaging unit 11 constituting the first imaging means. Alternatively, the second imaging means may be provided separately from the first imaging means, for example, a terminal device having a camera unit constituting the first imaging means may be used in combination with another device having a camera unit constituting the second imaging means.

[0049] In addition, the second imaging unit 12 constituting the second imaging means is configured to be provided on a different surface of the terminal device 10 from the first imaging unit 11 constituting the first imaging means, but the second imaging unit 12 and the first imaging unit 11 may be provided on the same surface of the terminal device 10, in other words, any one of a plurality of imaging units provided on the same surface of the terminal device may be the first imaging means and the others may be the second imaging means.

[0050] In addition, the second imaging unit 12 constituting the second imaging means can capture images at a position closer to the user than the first imaging unit 11 constituting the first imaging means, and the second imaging unit 12 can be configured to automatically capture an image of the specified part of the skin when the second imaging unit 12 is brought closer to the user together with the terminal device 10 or when the user approaches the second imaging unit 12 and the distance between the second imaging unit 12 and the specified part of the skin reaches a specified value.

[0051] In addition, one or more lighting means for irradiating light in conjunction with each of the first imaging unit 11 and the second imaging unit 12 may be provided, and illumination corresponding to when a predetermined range of the body is imaged by the first imaging unit 11 and illumination corresponding to when a predetermined part is imaged by the second imaging unit 12 may be performed. Furthermore, the lighting means corresponding to the second imaging unit 12 may be configured to irradiate unpolarized light as illumination for detecting the texture of the user's skin, and to irradiate linearly polarized light as illumination for detecting blemishes on the user's skin, when the predetermined part of the user is imaged.

[0052] In this way, by capturing images by selectively using unpolarized light and linearly polarized light as illumination for the second imaging unit 12, skin texture detection and blemish detection can be performed appropriately. Alternatively, circularly polarized light may be irradiated as illumination, and specifically, right-handed circularly polarized light may be used as illumination corresponding to skin texture detection, and left-handed circularly polarized light may be used as illumination corresponding to skin blemish detection. Alternatively, the color (wavelength) of light irradiated as illumination may be made to correspond to skin texture detection and blemish detection, respectively, and a light source capable of irradiating skin-colored light for texture detection and white light for blemish detection may be provided in conjunction with the second imaging unit 12.

[0053] The terminal device 10 has the first imaging unit 11 and the second imaging unit 12, and is a device capable of imaging a specified range of the user's body and the specified part of the body and acquiring skin image data, and also has the control unit 13.

[0054] The terminal device 10 is a known portable terminal device such as a mobile phone, a so-called smartphone, or a tablet device having wireless communication means 14 for connecting to and communicating with an IP communication network, and among these, an application program can be executed on the device, and a display unit 16 can display various information. The terminal device 10 is equipped with two camera units, which can function as the first imaging unit 11 and the second imaging unit 12.

[0055] The display unit 16 of the terminal device 10 is capable of displaying an image representing at least the predetermined area of ​​the user's body. The display unit 16 is a touch panel type display device, and also serves as an input device.

[0056] The terminal device 10 is an information processing device that realizes the functions and processing of the control unit 13 by executing a pre-stored program, and is not limited to a portable device as long as it is equipped with the first imaging unit 11 and the second imaging unit 12 and can measure the skin condition, and may be a computer such as a notebook or desktop type with a camera.

[0057] The control unit 13 of the terminal device 10 is capable of analyzing the skin image data obtained by imaging using an application program for diagnosing a skin condition, and acquiring diagnostic information, which the user can refer to. In this case, the control unit 13 of the terminal device 10 executes the application program to display the acquired diagnostic information as content on the display unit 16, thereby enabling the user to refer to the information.

[0058] The control unit 13 acquires information regarding the skin condition based on skin image data obtained by imaging, i.e., skin image data of the specified area of ​​the body obtained by imaging with the first imaging unit 11 and skin image data of the specified part obtained by imaging with the second imaging unit 12.

[0059] This control unit 13 analyzes the skin image data of the predetermined body area and the skin image data of the predetermined part to obtain diagnostic information on the skin condition of the predetermined body area of ​​the user as information on the skin condition. The function of the control unit 13 to analyze each skin image data and obtain diagnostic information on the user's skin condition is realized by the control unit 13 executing processing according to an application program.

[0060] As the above process, the control unit 13 evaluates the user's skin based on the skin image data, and obtains an index value indicating the user's skin condition. For example, the control unit 13 evaluates at least one of the following items as the skin condition: skin age, fineness of skin texture, degree of skin whiteness, degree of openness of skin pores, amount of skin blemishes, amount of moisture contained in the skin (degree of moisturization), amount of oil contained in the skin, etc., and obtains an index value indicating the skin condition from the evaluation result.

[0061] In this embodiment, the control unit 13 performs predetermined image processing on the images captured by the first imaging unit 11 and the second imaging unit 12 to obtain an index value indicating the skin condition of the user. For example, the control unit 13 performs edge detection on the captured image, detects skin grooves in the image, identifies the area of ​​the skin ridge surrounded by the detected skin grooves, and calculates an index value indicating the fineness of the skin texture (index value indicating the skin condition) such that the smaller the size of the identified skin ridge area is and the more uniform it is, the larger the index value will be.

[0062] In this embodiment, the control unit 13 determines information indicating that the diagnostic result has been obtained as an index value indicating the user's skin condition as diagnostic information. Then, the control unit 13 performs control so that the acquired diagnostic information can be appropriately displayed on the display unit 16 as display content of the display unit 16. For example, the control unit 13 can create a graph for some index values ​​indicating the user's skin condition from the diagnostic result, and display this on the display unit 16 as display content.

[0063] The control unit 13 can also accept data including information on the skin condition, skin care action information, skin images, and the time when the skin was imaged on the terminal device 10, associate the received index value indicating the skin condition, the skin care action information, the skin image, and information on the time when the skin was imaged, and record the data temporarily in the recording unit 17 for recording.The control unit 13 can also diagnose the user's skin condition based on the recorded index value of the skin condition and the information on the time when the skin was imaged.

[0064] Specifically, the control unit 13 identifies the transition of the skin condition index value over time based on the skin condition index value recorded in the recording unit 17 and information on the time when the skin condition measurement was performed, and diagnoses whether the skin condition has improved based on the state of the identified transition. For example, the control unit 13 identifies the transition of the skin condition index value over time during a period (e.g., about 28 days ± 1 to 7 days) required for the user's skin metabolism (turnover) immediately before the treatment, and diagnoses the skin condition.

[0065] In addition, the control unit 13 may identify the time-dependent changes in the skin condition index values ​​during a period different from the period required for the user's turnover immediately before the treatment (for example, a period of a specified length immediately before the treatment (for example, one week, ten days, one month, etc.), a specified period, etc.) and diagnose the skin condition.

[0066] When the index value of the skin condition changes over time in a direction indicating an improvement in the skin condition, the control unit 13 gives a diagnosis result that the user's skin condition has improved. On the other hand, when the index value of the skin condition does not change over time in a direction indicating an improvement in the skin condition (changing in a direction indicating deterioration or showing no change), the control unit 13 gives a diagnosis result that the user's skin condition has not improved.

[0067] The control unit 13 may also perform a skin condition diagnostic process that takes into account the user's attributes such as age, etc. In addition, the control unit 13 may change the diagnostic criteria for a part or all of the diagnostic information based on the analysis of the skin image data of the predetermined part obtained by the second imaging unit 12 depending on the location where the second imaging unit 12 captures the image.

[0068] The control unit 13 performs control so that images including graphs and texts indicating diagnostic information are appropriately displayed on the display unit 16 of the terminal device 10. In addition, the control unit 13 may perform control so that images including a transition graph indicating the transition of skin condition, a group of images that are skin images, a group of action information indicating information on skin care actions corresponding to the skin images, and an advertisement indicating an advertisement (with a link) for a specific skin care product or skin care service are appropriately displayed on the display unit 16.

[0069] Furthermore, the control unit 13 may create content indicating the diagnosis result as part of the diagnosis information, and cause the display unit 16 to display the content so that the content can be referenced from the terminal device 10. For example, the control unit 13 may create content including bar objects indicating index values ​​of the skin condition for each of the level of moisturization, the level of pore opening, the level of whitening, and the skin age (aging), and cause the display unit 16 to display the content.

[0070] In addition, the control unit 13 acquires, as other information regarding the skin condition, a display image of a specified area of ​​the user's body based on skin image data of the specified area of ​​the body obtained by imaging with the first imaging unit 11, and a display image of the specified area of ​​the user based on skin image data of the specified area of ​​the skin obtained by imaging with the second imaging unit 12.

[0071] Each time a new image is captured by the first imaging unit 11 and the second imaging unit 12, the control unit 13 obtains a display image of the specified area or part of the user's body based on the skin image data of the specified area or part of the body obtained by the imaging, and displays this display image on the display unit 16.

[0072] In addition, the control unit 13 can associate a portion of information in the display image of the specified area of ​​the body that corresponds to the specified part of the skin with the display image of the specified part, and cause the display unit 16 to display the display image of the specified part simultaneously with the display of a part or all of the display image of the specified area of ​​the body.

[0073] In addition, the control unit 13 can also perform image correction processing (e.g., color calibration) on a part or all of the display image of the specified body range derived from the image data of the specified body range, based on the display image of the specified part derived from the image data of the specified part.

[0074] Furthermore, the control unit 13 can execute a process for synthesizing a predetermined guide design portion, which indicates that the position is to be imaged by the second imaging unit 12, on a predetermined position on the display image of the predetermined body area derived from the image data of the predetermined body area, and display the processed image on the display unit 16. As a result, the guide design portion is displayed as a part of the display image of the predetermined body area displayed on the display unit 16, and the user can visually recognize and understand the position in the predetermined body area that should be imaged by the second imaging unit 12.

[0075] Furthermore, the control unit 13 temporarily acquires diagnostic information on the skin condition of the predetermined area of ​​the user's body as information on the skin condition based on the skin image data of the predetermined area of ​​the body obtained by imaging with the first imaging unit 11, and uses this diagnostic information to set a predetermined area of ​​the skin that is included in the predetermined area of ​​the body and should be imaged by the second imaging unit 12. The control unit 13 can notify the user of the information on the set predetermined area via the user's terminal device 10.

[0076] Then, when skin image data of a specified area is obtained as a result of the second imaging unit 12 imaging the specified area of ​​the skin based on user operation referring to the information on the specified area, the control unit 13 again obtains diagnostic information on the skin condition of the specified area of ​​the body based on the skin image data of the specified area of ​​the body and the skin image data of the specified area.

[0077] In setting the predetermined part of the skin to be imaged by the second imaging unit 12 by the control unit 13, the predetermined part of the skin to be imaged may be suggested using AI based on the current overall face image, which is known from the diagnostic information of the skin condition once acquired based on the facial skin image data as the predetermined body range. In this case, in addition to the diagnostic information, necessary information can be appropriately obtained by inputting information from the user, inputting information based on cooperation with other IOT devices, or accessing so-called big data on the network. For example, in addition to the past and present wrinkles and age of wrinkles and age spots, statistical data on the location and age at which wrinkles and age spots appear in Japanese people can be obtained and used. In addition, statistical data based on the location where the user lives (e.g., Okinawa, Hokkaido, etc.) and weather-related information based on the GPS of the area where the user lives (e.g., ultraviolet light amount, temperature, humidity, weather, etc.) may be obtained. Furthermore, various types of information about the user may be obtained and used, such as information based on GPS on whether the user spends more time outdoors or indoors (proportion), information about the user's lifestyle (e.g., sleep, exercise, bathing, stress, etc.), the user's biometric information (blood pressure, body temperature, blood flow, electrocardiogram, etc.), the user's eating habits (likes fried food, dislikes vegetables, etc.), information about cosmetics and skin care devices used by the user, genetic information related to the user (e.g., age spots and wrinkles on the faces of grandparents or parents, etc.), the user's skin color, skin temperature, etc. (including temporary changes due to exposure to ultraviolet rays, etc.), and information related to improving the user's facial impression (e.g., physiognomy, facial muscles, smile, etc.).

[0078] When using AI to suggest specific areas of the skin, it can, in particular, based on various information, suggest areas (including hidden blemishes) where the user is likely to develop blemishes or wrinkles in the future, suggest areas that will improve the impression of the face if improved through care, and suggest areas where favorable results can be expected if improvements are started now in order to keep the user beautiful several years or decades into the future.

[0079] In addition, the control unit 13 uses AI to suggest an effective skin care and makeup plan when it has roughly grasped the blemishes, wrinkles, dark circles under the eyes, freckles, etc. on the entire face as diagnostic information of the user's facial skin condition based on the facial skin image data of the specified body area obtained by imaging with the first imaging unit 11. For example, it may suggest that blemishes in certain areas should be treated with cosmetics, and blemishes in other areas should be covered with foundation for the time being since they cannot be improved immediately.

[0080] The control unit 13 has a recording unit 17 which, when a specific area of ​​the skin to be imaged by the second imaging unit 12 is set using diagnostic information on the skin condition of the specific area of ​​the body that has been acquired based on skin image data of the specific area of ​​the body, records information about the set specific area.

[0081] The recording unit 17 records information about the specified area of ​​the skin to be imaged by the second imaging unit 12, as well as skin image data obtained by imaging this specified area by the second imaging unit 12, in a manner that makes it possible to determine the date and time when this skin image data was obtained.

[0082] When setting a new specified area of ​​the skin to be imaged by the second imaging unit 12, the control unit 13 uses the diagnostic information of the skin condition that was previously acquired based on the skin image data of the specified area of ​​the body that was previously obtained, and if information of a previously set specified area has been recorded in the recording unit 17, it also refers to the information of the specified area already recorded in the recording unit 17 and sets the specified area from among parts of the specified area of ​​the body that have not been recorded as a specified area.

[0083] When imaging is performed multiple times by the second imaging unit 12 for each imaging by the first imaging unit 11 to obtain skin image data for multiple predetermined areas, the settings of the predetermined areas of the skin to be imaged are repeated so as not to overlap with each other for each imaging by the second imaging unit 12. Furthermore, for each imaging by the second imaging unit 12, information on the set predetermined areas is notified to the user through the terminal device 10 and is recorded in the recording unit 17 to be reflected in the next setting.

[0084] In addition, the control unit 13 has a mechanism for referencing information on the specific area of ​​the skin that has already been recorded in the recording unit 17 and skin image data of this specific area, in parallel with setting a specific area of ​​the skin to be imaged by the second imaging unit 12 during a new imaging opportunity by the first imaging unit 11.

[0085] The control unit 13 then regards skin image data of the recorded specified area, whose data acquisition date and time falls within a specified period of the past corresponding to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture.

[0086] In this case, the control unit 13 acquires diagnostic information (information related to the skin condition) of the skin condition of the specified area of ​​the body based on the recorded skin image data acquired during the specified period, the skin image data of the specified area of ​​the skin set at this new imaging opportunity obtained by imaging the specified area by the second imaging unit 12, and the skin image data of the specified area of ​​the body.

[0087] On the other hand, the control unit 13 may delete from the recording unit 17 the information on the specified area and the skin image data of the specified area, if the data acquisition date and time is not included in the specified period and is older than the turnover cycle of the user's skin, on the basis that the data is from a period when the skin turnover does not overlap with the user's skin in the new imaging opportunity by the first imaging unit 11, and therefore cannot be regarded as having been captured at the same time as the new imaging.

[0088] Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, the entire face of the user is set as the predetermined body range to be imaged by the first imaging unit 11, and the predetermined part included therein and imaged by the second imaging unit 12 is the main part of the skin of the face (cheeks, eyes, mouth, etc.). The terminal device 10 is a smartphone, and can perform imaging by using a camera unit (so-called in-camera) provided on the side where the display unit 16 is located as the first imaging unit 11, while using a camera unit (so-called out-camera) provided on the opposite side of the display unit 16 of the terminal device 10 as the second imaging unit 12. When imaging the predetermined part, a conversion lens 19 is attached in front of the second imaging unit 12 in the terminal device 10, so that the imaging range of the second imaging unit 12 is narrower than that of the first imaging unit 11, while imaging can be performed at a higher resolution, and an extremely narrow range of skin on the face is imaged as the predetermined part.

[0089] In order to capture an image using each imaging unit of the terminal device 10, first, the conversion lens 19 is attached to the terminal device 10 by the user. When a user operates an application program on the terminal device 10 that already exists in the terminal device and provides a user interface for using each imaging unit, the user is able to use each imaging unit for imaging through the terminal device 10.

[0090] When the user performs an operation to start imaging on the user interface displayed on the terminal device 10, the first imaging unit 11 of the terminal device 10 starts imaging the face as the predetermined body area of ​​the user.

[0091] In detail, when the terminal device 10 executes an application program for measuring skin condition, the first imaging unit 11 as the first imaging means starts imaging the user's face, and the control unit 13 of the terminal device 10 continuously acquires image data. In this case, the control unit 13 of the terminal device 10 causes the user to maintain a state in which the terminal device 10 is held so that the face is displayed in a predetermined orientation and size by displaying a predetermined guide on the display unit 16. For example, the control unit 13 displays on the display unit 16 an image of a frame that fits the contour of the face on the screen of the display unit 16 in a state where the face is correctly imaged, superimposed on the image captured by the first imaging unit 11, and prompts the user to adjust the relative positional relationship between the first imaging unit 11 and the face using the image of the frame as a guide while looking at the display unit 16 so that the face fits within the frame.

[0092] When the user's face is imaged by the first imaging unit 11, the control unit 13 causes the display unit 16 to display an image of the face based on the image data obtained by imaging. The control unit 13 causes the display unit 16 to display a predetermined guide display or the like, to have the user maintain a state in which the terminal device 10 is held so that the face is displayed in a predetermined orientation and size. Furthermore, as the first imaging unit 11 of the terminal device 10 captures an image of the user's face, the image of the face is displayed on the display unit 16 based on the image data obtained by the capture.

[0093] The user refers to the facial image displayed on the display unit 16, and performs a predetermined operation on the user interface of the terminal device 10 at a timing when the face as the predetermined body area is deemed to be appropriately captured by the first imaging unit 11, to determine a still image of the face. The data of the still image of the face at the determination moment is handled by the control unit 13 as skin image data of the predetermined body area obtained by imaging by the first imaging unit 11. Note that the control unit 13 may automatically determine the facial image at a timing when the face as the predetermined body area is appropriately captured by the first imaging unit 11, regardless of the user's operation.

[0094] In this way, when the skin image data based on the image of the face as the specified body area by the first imaging unit 11 is confirmed, the second imaging unit 12 may transition to a state suitable for imaging the specified part of the skin.

[0095] The control unit 13 acquires a display image of the face based on the skin image data obtained by imaging with the first imaging unit 11, and displays this on the display unit 16 while analyzing the skin image data, and initially acquires diagnostic information on the skin condition of the user's face.

[0096] In addition, the control unit 13 refers to the information already recorded in the recording unit 17 and determines whether or not there is information about the specified area that was set during a previous image capture, that is, information about the specified area that was recorded when the control unit 13 set the specified area of ​​the skin to be captured by the second imaging unit 12 using diagnostic information about the facial skin condition that was previously obtained in the same manner as described above.

[0097] If information on a previously set specified area has been recorded, the control unit 13 uses the diagnostic information based on the recorded information to identify areas within the specified body range other than those previously set as specified areas, and determines areas where detailed images need to be obtained by imaging with the second imaging unit 12 for a more appropriate diagnosis of the skin condition, for example, areas where confirmation of the presence or absence of blemishes is required due to the detection of signs of the presence of blemishes or the like on a part of the skin.

[0098] On the other hand, when information about the specified areas to be imaged by the second imaging unit 12 that was previously set has not been recorded in the recording unit 17, such as when imaging the user's skin for the first time, the control unit 13 uses the diagnostic information about the facial skin condition previously obtained for the entire specified area of ​​the body to determine areas that require detailed images to be obtained by imaging with the second imaging unit 12 in order to more appropriately diagnose the skin condition, as described above.

[0099] The control unit 13 sets the portion thus determined as the predetermined portion to be imaged by the second imaging unit 12, and records information about the set predetermined portion in the recording unit 17. The control unit 13 then generates, for this predetermined portion, for example, a predetermined guide design portion (mark) indicating the predetermined portion on the face to be imaged by the second imaging unit 12, performs processing to synthesize this guide design portion onto a display image of the face based on the facial skin image data, and causes the synthesized image to be displayed on the display unit 16 (see FIG. 5). By looking at this display unit 16, the user recognizes a specific area to be imaged by the second imaging unit 12, and adjusts the position of the terminal device 10 having the second imaging unit 12 so that this area can be imaged.

[0100] When the guide design section indicates the position to be imaged by the second imaging unit 12, the user can easily understand the specific part of the face, which is the specified area of ​​the body, that should be imaged by the second imaging unit 12, and imaging can be performed appropriately using the second imaging unit 12 of the terminal device 10 whose position has been adjusted by the user.

[0101] Instead of setting a predetermined part to be imaged by the second imaging unit 12 and synthesizing an image showing this predetermined part with an image of the face and displaying it, the control unit 13 can also display guidance or provide audio guidance to the user from the terminal device 10 so that the user can image a part of the face, which is the predetermined body range of the user, that the user is concerned about, using the second imaging unit 12. Alternatively, a part where the skin condition is generally measured may be selected in advance and registered as an initial measurement position, and a process may be performed to synthesize a predetermined guide design part showing one or more parts of the skin, which are the initial position, with the display image of the face, and the synthesized image may be displayed on the display unit 16.

[0102] In this case too, the user recognizes any parts of the face that concern them, or checks the initial measurement positions on the face indicated by the guide design displayed on the display unit 16, and then adjusts the position of the terminal device 10 equipped with the second imaging unit 12 so that these can be imaged (see Figure 6).

[0103] The user adjusts the position of the terminal device 10 held in the hand, and when the user determines that the second imaging unit 12 is facing the specified part of the face, etc., the user performs a specified operation on a user interface such as a shutter button of the terminal device 10 to cause the second imaging unit 12 to capture an image.

[0104] In addition, the user may bring the second imaging unit 12 together with the terminal device 10 closer to the specified area, or bring the specified area closer to the second imaging unit 12, so that when the distance between the second imaging unit 12 and the specified area of ​​the skin reaches a specified value, the second imaging unit 12 automatically captures an image of the specified area of ​​the skin, or the terminal device 10 may notify the user that it is the optimal time to capture an image, prompting the user to operate the terminal device 10 related to the image capture.

[0105] In this imaging stage, the second imaging unit 12 images the predetermined part of the skin under a predetermined lighting condition by an external light source (for example, a built-in light source of a conversion lens) or a built-in light source of the terminal device 10, and outputs skin image data. The control unit 13 acquires this skin image data.

[0106] For example, the built-in light source of the conversion lens 19 attached to the terminal device 10, which can adjust the light emitted as illumination, optimizes the light illumination state on the area to be imaged through verification of skin images obtained in trial imaging by the second imaging unit 12, and then imaging is performed by the second imaging unit 12.The control unit 13 can acquire image data for a texture diagnosis image obtained by imaging the skin under illumination with unpolarized light, and a blemish diagnosis image obtained by imaging the skin under illumination with linearly polarized light.

[0107] The control unit 13 records the acquired skin image data of the specific area in the recording unit 17 together with the previously recorded information of the specific area in a state in which the date and time when the skin image data was acquired can be determined. Here, when imaging is performed multiple times at once using the second imaging unit 12, for each imaging operation using the second imaging unit 12, the control unit 13 sets the specific area of ​​the skin to be imaged so as not to overlap with the previous area based on the information recorded in the recording unit 17, and imaging using the second imaging unit 12 is repeatedly performed by the user in accordance with the information on the specific area notified to the user via the terminal device 10.

[0108] In addition, when information of a previously set specific area is recorded in the recording unit 17, the control unit 13 sets the specific area of ​​the skin to be imaged by the second imaging unit 12, and while this specific area is being imaged by the second imaging unit 12, it refers to the recording unit 17 and regards the skin image data of the specific area, whose data acquisition date and time is included in a past specific period corresponding to the turnover cycle of the user's skin, as having been imaged at the same time as the new imaging by the second imaging unit 12.

[0109] Then, after acquiring skin image data of the specified area of ​​the skin from the second imaging unit 12, the control unit 13 analyzes this skin image data, the previously recorded skin image data of the specified area acquired during the specified period which is assumed to have been acquired at the same time, and the facial image data acquired previously by imaging with the first imaging unit 11, to acquire diagnostic information on the skin condition of the user's face as the specified area of ​​the body.

[0110] In addition, if information about a previously set specific area has not been recorded in the recording unit 17, the control unit 13 acquires skin image data of the specific area of ​​the skin from the second imaging unit 12, and then analyzes this skin image data and the facial skin image data previously acquired by imaging with the first imaging unit 11 to obtain formal diagnostic information about the condition of the user's facial skin.

[0111] Then, the control unit 13 causes the display unit 16 to display the diagnostic information on the facial skin condition simultaneously with the facial image or in place of the facial image (see FIG. 7). The display of diagnostic information on the facial skin condition on the display unit 16 is not limited to text-based information display, and the information can also be provided by specifically indicating areas and symptoms of the skin that require attention with marking images on the facial image, etc. For example, the position and size of hidden blemishes derived as diagnostic information analyzed from the detailed image data obtained by the second imaging unit 12 can be displayed with a mark of a predetermined color superimposed on the corresponding area on the facial image, and can be used as reference information when carrying out skin care such as blemish prevention. When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 11 and the second imaging unit 12 is completed.

[0112] After this, in order to grasp the progress of the skin condition, the user will repeat the same procedure at predetermined intervals, capturing images of the face as the predetermined area of ​​the body using the first imaging unit 11 and capturing images of the predetermined part of the body using the second imaging unit 12, and the control unit 13 will obtain diagnostic information on the skin condition based on the obtained skin image data for each image capture and display it on the display unit 16.

[0113] While repeating such imaging, the control unit 13 records information on the predetermined part set in the recording unit 17, and when the next imaging is performed, a new predetermined part that does not overlap with the previous one is set by referring to the recorded information (see FIG. 8), so that the face area as the predetermined body range is gradually filled with the part for which detailed skin image data is obtained by imaging with the second imaging unit 12 for each imaging, and multiple high-precision skin image data are continuously arranged in a map shape, so that the analysis of the skin condition based on the skin image data can be performed with higher precision, and accurate and reliable diagnostic information can be derived. Here, when imaging is performed multiple times with the second imaging unit 12 for each imaging with the first imaging unit 11, detailed skin image data covering each part of the face can be obtained in a short period of time (for example, about one week), but if such imaging action is burdensome for the user, the imaging may be distributed to reduce the burden on the user. For example, imaging can be performed once a week with the first imaging unit 11 and once with the second imaging unit 12, with the cheek being imaged as the specified part in one week and the forehead being imaged as the specified part in the following week, and imaging can be repeated every week so that imaging by the second imaging unit 12 covers the entire face in about a month.

[0114] In addition, each time an image of the specified area is captured by the second imaging unit 12, the control unit 13 records information about the specified area and skin image data obtained by capturing the specified area by the second imaging unit 12. At the time of a new imaging opportunity, the control unit 13 can determine that, among the information and skin image data recorded in the recording unit 17, data acquisition dates and times that exceed a specified period of time in the past corresponding to the turnover cycle of the user's skin do not overlap with the turnover of the user's skin at this new imaging opportunity, and delete such old information and skin image data so that new recording can be performed repeatedly without difficulty.

[0115] Furthermore, when a new image is captured by the second imaging unit 12, the control unit 13 refers to the information recorded in the recording unit 17 so that a new specified part that does not overlap with the previous part is captured by the second imaging unit 12; however, without being limited to this, the second imaging unit 12 can be configured to repeatedly capture images of the same specified part many times while the first imaging unit 11 captures the specified area of ​​the body and the second imaging unit 12 captures the specified part. From the diagnostic information on the skin condition and the display image obtained by the control unit 13 based on the skin image data focusing on the same part of the skin, the detailed progress of the skin condition in a specific part can be grasped, and the degree of improvement due to skin care can be easily confirmed.

[0116] In this case, the control unit 13 can record images of the face or the specified part for display on the display unit 16 based on the skin image data obtained by repeated imaging in the recording unit 17 after each imaging, and if necessary, the images can be arranged in chronological order and displayed as a list, or displayed in chronological order in sequential switching, which is preferable as it allows the user to easily grasp changes in the skin.

[0117] In addition, the facial skin image data obtained by imaging with the first imaging unit 11, which is the basis for the display image and diagnostic information on the skin condition for the face as the specified body area, is skin image data obtained by imaging with the first imaging unit 11 before imaging with the second imaging unit 12. Alternatively, after the control unit 13 acquires the skin image data of the specified area from the second imaging unit 12, it may perform image correction processing on the facial skin image data based on the skin image data of the specified area, and then generate a display facial image based on this corrected facial skin image data, and display the facial image on the display unit 16, or the corrected facial skin image data may be analyzed together with the skin image data of the specified area to acquire diagnostic information on the facial skin condition.

[0118] In this case, since the imaging range is limited and imaging conditions are easier to prepare, image correction processing is performed on the image data of the specified body range based on the image data of the specified part based on imaging by the second imaging unit 12, and the skin color of the specified body range shown in this image data can be matched to that of the skin in the image data of the specified part. Therefore, regardless of the imaging conditions in the first imaging unit 11, the image data of the specified body range obtained by imaging can be made in a state that makes it easy to display the image and perform analysis related to diagnosing the skin condition, thereby encouraging the user to correctly understand the skin condition.

[0119] In this way, the skin condition measuring system of this embodiment images the face as a specified area of ​​the user's body with the first imaging unit 11, and images a specified part of the user's face with the second imaging unit 12, and the control unit 13 obtains information regarding the user's skin condition based on the obtained skin image data of the face and the skin image data of the specified part. Therefore, when analyzing the skin image data to derive information regarding the skin condition, the quality of information regarding the user's skin condition, such as diagnostic information for the skin condition based on skin image data obtained by multiple images, can be improved, and processing such as displaying and analyzing the skin condition can be performed with high accuracy, the skin condition diagnosis can be carried out appropriately and efficiently, the reliability of the obtained information can be increased, and the user can understand their skin condition more accurately.

[0120] Furthermore, since the first imaging unit 11 is provided in the terminal device 10 together with the second imaging unit 12 and imaging can be performed using both the first imaging unit 11 and the second imaging unit 12 on a single terminal device 10, imaging of the face as the specified range of the user's body and imaging of the specified part of the face can be handled by a single terminal device 10, and the cost associated with imaging can be reduced by using only one device for imaging.

[0121] In the skin condition measuring system according to the embodiment, the control unit 13 recognizes the image captured by the second imaging unit 12 as an image of the set predetermined part on the assumption that the user correctly captures the part indicated by the terminal device 10 as the predetermined part of the skin to be captured by the second imaging unit 12, but the control unit can also extract parts having characteristics common to those appearing in the image captured by the second imaging unit (e.g., the shape of characteristic parts such as skin texture and blemishes) in the image of the predetermined body area obtained by the first imaging unit, and set such parts as the predetermined part and recognize the image captured by the second imaging unit as the image of the predetermined part, and correctly identify the position of the image captured by the second imaging unit in the predetermined body area, thereby making the information on the skin condition based on the image data accurate. In this way, when the control unit does not particularly set the predetermined part of the skin to be captured by the second imaging unit, the image captured by the second imaging unit may be captured before or after the image captured by the first imaging unit.

[0122] In addition, in the skin condition measuring system according to the embodiment, the first imaging unit 11, the second imaging unit 12, and the control unit 13 are configured to be provided in a portable terminal device 10 such as a smartphone, but the present invention is not limited to this, and the terminal device may be a computer such as a notebook type or desktop type with a camera, as long as it is an information processing device that realizes the functions and processing of the control unit 13 by executing a program stored in advance and is equipped with the first imaging unit 11 and the second imaging unit 12 and can measure the skin condition. The terminal device may also be another information processing device specialized for a specific purpose such as beauty, for example, an electronic device that incorporates a camera and a display into the mirror part (half mirror) of a dressing table or a makeup or beauty table mirror, and that can display, in addition to being used as a normal mirror, an image of a user captured by a camera, an image processed therefrom, information obtained by analyzing the image of a user, etc., on the mirror surface as necessary.

[0123] In the case of a terminal device 50 having such a mirror unit 55, as shown in FIG. 9, multiple camera units constituting a first imaging unit 51 and a second imaging unit 52, or a camera unit serving as both a first imaging unit and a second imaging unit, can be provided on the upper part of the mirror unit 55, all facing directly towards the user. In addition, a light source 53 for illumination during imaging may be provided together with the first imaging unit 51 and the second imaging unit 52. In particular, a light source unit that can switch between a state in which unpolarized light is irradiated as illumination for detecting the user's skin texture and a state in which linearly polarized light is irradiated as illumination for detecting blemishes on the user's skin when imaging the specified area, or a light source unit that can switch between a state in which right-handed circularly polarized light is irradiated for skin texture detection and a state in which left-handed circularly polarized light is irradiated for skin blemish detection, is provided near the second imaging unit 52. This is preferable because it allows the second imaging unit 52 to properly detect skin texture and blemishes by selectively irradiating unpolarized light and linearly polarized light, or right-handed circularly polarized light and left-handed circularly polarized light, as illumination.

[0124] A hood 54 that covers the first and second imaging units to block the incidence of unnecessary light from the surroundings may be provided above the mirror unit where the first and second imaging units 51 and 52 are located. When a light source 53 for illumination during imaging is provided near the first and second imaging units 51 and 52, the hood 54 can also be used as a reflector that directs illumination light toward the user.

[0125] In addition, when a hood 54 is provided, the size of the hood can be set so that when a part of the user's face (e.g., the forehead) touches the tip of the hood, the face as the specified area of ​​the user's body is positioned relative to the first imaging unit 51 in a manner suitable for imaging by it, or so that when the specified area of ​​the user or the skin nearby touches the tip of the hood, the specified area of ​​the user is positioned relative to the second imaging unit 52 in a manner suitable for imaging by them.

[0126] In such a terminal device 50, when imaging is performed by the second imaging unit 52, the distance between the imaging unit in focus and the imaging target is usually shorter than in the case of the first imaging unit 51 because the second imaging unit 52 images a narrower range in detail than in the case of imaging by the first imaging unit 51, so the user needs to bring the predetermined part that is to be the imaging target of the second imaging unit 52 closer to the terminal device 50. In order to facilitate imaging by the second imaging unit 52, the distance between the terminal device 50 and the user is measured by a predetermined measuring means of the terminal device 50, and when the user brings the predetermined part closer to the terminal device 50 and the distance becomes suitable for imaging by the second imaging unit 52, imaging by the second imaging unit 52 is performed, or when the distance between the terminal device 50 and the user is too far or too close during imaging, a notification is issued to the user through a notification means of the terminal device 50 to encourage the user to move closer to or away from the terminal device 50 to maintain an appropriate distance. In addition, so that the user does not have to move his or her body to change the distance between the user and the terminal device 50 for each image capture by the first imaging unit 51 and the second imaging unit 52, the distance between the terminal device 50 and the user's image capture target portion in each focus state of the first imaging unit 51 and the second imaging unit 52 may be set to be the same. In this case, imaging can be performed simultaneously by the first imaging unit 51 and the second imaging unit 52, reducing the burden on the user related to imaging.

[0127] (Second embodiment of the present invention) A skin condition measuring system according to the second embodiment of the present invention will be described with reference to FIGS. In each of the figures, the skin condition measuring system 2 of this embodiment comprises a first imaging unit 21, a second imaging unit 22, and a control unit 93, just like the first embodiment, but differs in that the first imaging unit 21 and the second imaging unit 22 are arranged in a terminal device 20, while the control unit 93 is arranged in a server 90 that is set up to be able to communicate data with the terminal device 20 via a network 100.

[0128] In this embodiment, as in the first embodiment, the user who is the subject of skin condition measurement has entered into a contract with the provider of skin condition measuring system 2 to receive skin condition diagnosis services, and is able to use skin condition measuring system 2, thereby obtaining skin condition diagnostic information from skin condition measuring system 2.

[0129] The first imaging section 21 is disposed in the terminal device 20 as the first imaging means, similarly to the first embodiment, and is capable of capturing an image of a predetermined range of the user's body. As in the first embodiment, the second imaging unit 22 is arranged in the same terminal device 20 as the first imaging unit 21 as the second imaging means, and is capable of imaging a specific part of the user's skin.

[0130] These first imaging unit 21 and second imaging unit 22 have the same configuration as that shown in the first embodiment, including the fact that they each form part of the user's terminal device 20 as camera units of the terminal device 20 capable of capturing images of the user's skin, and detailed explanations will be omitted.

[0131] The terminal device 20 has the first imaging unit 21 and the second imaging unit 22, and is a device capable of capturing an image of a predetermined range of the user's body and the predetermined part, and acquiring skin image data. As in the first embodiment, the terminal device 20 is a known portable terminal device such as a mobile phone, a so-called smartphone, or a tablet device having wireless communication means 24 for connecting to and communicating with an IP communication network, and among these, an application program can be executed on the device and a display unit 26 can display various information. The terminal device 20 is equipped with two camera units, which can function as the first imaging unit 21 and the second imaging unit 22.

[0132] The display unit 26 of the terminal device 20 is capable of displaying an image representing at least the predetermined body area of ​​the user. The display unit 26 is a touch panel type display device and also serves as an input device. More specifically, the display unit 26 displays the display image of the predetermined body area and the display image of the predetermined part acquired by the control unit 93 of the server 90, and also displays diagnostic information on the skin condition of the predetermined body area acquired by the control unit 93.

[0133] In addition, the terminal device 20 is an information processing device that realizes various functions and processes by executing pre-stored programs, and as long as it is capable of communicating via the network 100 and is equipped with the first imaging unit 21 and the second imaging unit 22 and can measure the skin condition, it is not limited to a portable device as in the first embodiment, and may be a computer such as a notebook type or desktop type, or another information processing device specialized for a specific purpose.

[0134] This terminal device 20 launches an application program for measuring skin condition, images the user's face as the specified area of ​​the body with the first imaging unit 21, obtains skin image data of the face, and images the specified area of ​​the skin with the second imaging unit 22, obtains skin image data of the specified area, and transmits these skin image data to the control unit 93 of the server 90.

[0135] Furthermore, by executing an application program, the terminal device 20 communicates with the server 90 to request that the diagnostic information acquired by the server 90 be made available for reference from the terminal device 20, and acquires the diagnostic information as content from the server 90 in response to the request, and displays the content on the display unit 26. In addition, in order to make the diagnostic information available for reference, the terminal device 20 can receive diagnostic information data transmitted from the server 90 and display the data, or can download and display the diagnostic information data. Furthermore, the terminal device 20 receives from the server 90 a display image of the predetermined body region or a display image of the predetermined part acquired by the control unit 93 of the server 90, and causes the display unit 26 to display them.

[0136] The server 90 is arranged to be capable of data communication with the terminal device 20 via the network 100, receives the skin image data of the specified area of ​​the body and the skin image data of the specified part acquired by the terminal device 20, and as a control unit, analyzes the skin image data to acquire diagnostic information on the user's skin condition, acquires display images based on each piece of skin image data, transmits the diagnostic information and display images to the terminal device 20, and causes the display unit 26 of the terminal device 20 to display the diagnostic information and the skin image, respectively.

[0137] This server 90 is a computer that functions as a server and is equipped with a CPU, etc. The computer executes processes according to a recorded program, thereby realizing the functions of the server.

[0138] It should be noted that the server 90 in this embodiment is not necessarily limited to a single physical computer, but may be a group of devices in which multiple computer devices are connected to a network and work together to provide one or more requested services as a whole, or may be a virtual device that, although it is only a portion of the functions of a computer device, has the function of providing one or more requested services as if they were a single device and can be treated as a single device by the service recipient.

[0139] The server 90 includes a control unit 93 that acquires diagnostic information on the skin condition and information on an image to be displayed on the terminal device, as information on the skin condition, based on skin image data obtained by imaging with the first imaging unit 21 and the second imaging unit 22 of the terminal device 20, and a recording unit 97 that records the information. In other words, the server 90 realizes the functions of the control unit 93 and the recording unit 97 by the computer that constitutes the server 90 executing processes according to a program.

[0140] The control unit 93 in the server 90 acquires information about the skin condition based on image data of the skin obtained by imaging. This control unit 93 is capable of acquiring information regarding the skin condition of a specified area of ​​the body based on skin image data of the specified area of ​​the body obtained by imaging with the first imaging unit 21 received from the terminal device 20, and is also capable of acquiring information regarding the skin condition of the specified area based on skin image data of the specified area obtained by imaging with the second imaging unit 22 also received from the terminal device 20.

[0141] In detail, the control unit 93 acquires diagnostic information on the skin condition of the predetermined body region by analyzing the skin image data of the predetermined body region and the skin image data of the predetermined part as information on the skin condition. The function of the control unit 93 to analyze each image data and acquire diagnostic information on the user's skin condition is based on the same mechanism as in the first embodiment, except that the server 90 executes the process according to the application program, and therefore a detailed description will be omitted.

[0142] Then, the control unit 93 can transmit the diagnostic information on the skin condition of the predetermined body area to the terminal device 20. The terminal device 20 can display the diagnostic information on the skin condition of the predetermined body area on the display unit 26 based on the control of the control unit 93.

[0143] In addition, the control unit 93 acquires, as other information regarding the skin condition, a display image of a specified area of ​​the body derived from skin image data of the specified area of ​​the body, and also acquires a display image of the specified area derived from skin image data of the specified area.

[0144] Then, the control unit 93 can transmit information about such a display image to the terminal device 20. Based on the control of the control unit 93, the terminal device 20 can display an image of the predetermined body range, specifically, the face, on the display unit 26, while displaying an image of the predetermined part with a larger magnification than the image of the predetermined body range in a state where the image is replaced in the same area as the display area on the screen of the display unit 26 where the image is displayed, or in an area different from the display area, simultaneously and in parallel with the display of the image of the face.

[0145] In addition, the control unit 93 can also perform image correction processing (e.g., color calibration) on a part or all of the display image of a specified body area derived from the skin image data of the specified body area, based on the display image of the specified area derived from the skin image data of the specified area.

[0146] Furthermore, the control unit 93 executes a process of synthesizing a predetermined guide design portion, which indicates that the predetermined part is to be imaged by the second imaging unit 22, on the display image of the predetermined body area derived from the skin image data of the predetermined body area at a position on the image corresponding to the predetermined part to be imaged by the second imaging unit 22, and can transmit the processed display image to the terminal device 20 to display it on the display unit 26 of the terminal device 20. As a result, the guide design portion is displayed as a part of the display image of the predetermined body area displayed on the display unit 26 of the terminal device 20, and the user can visually recognize and understand the predetermined part in the predetermined body area to be imaged by the second imaging unit 22.

[0147] In addition, the control unit 93 first obtains diagnostic information on the skin condition of the specified area of ​​the body as information related to the skin condition, based on the skin image data of the specified area of ​​the body obtained by imaging with the first imaging unit 21 received from the terminal device 20, and from this diagnostic information sets the specified area of ​​the skin to be imaged by the second imaging unit 22. It then transmits the information related to this specified area of ​​the skin to the terminal device 20 and shows it to the user on the display unit 26 of the terminal device 20, thereby making it possible to image the specified area of ​​the skin by the second imaging unit 22.

[0148] Then, when the second imaging unit 22 images the specified area of ​​the skin based on the user's operation referring to the information on the specified area, the skin image data of the specified area is sent to the server 90. The control unit 93 then acquires new diagnostic information on the skin condition of the specified area of ​​the body based on the newly acquired skin image data of the specified area and the skin image data of the specified area of ​​the body.

[0149] The recording unit 97 records information about a specific area of ​​the skin to be imaged by the second imaging unit 12 set by the control unit 93, using diagnostic information about the skin condition of the specific area of ​​the body that was initially acquired by the control unit 93 based on skin image data of the specific area of ​​the body.

[0150] This recording unit 97 records information about the specific part of the skin to be imaged by the second imaging unit 22, as well as skin image data obtained by imaging this specific part by the second imaging unit 22, in a manner that makes it possible to determine the date and time when this skin image data was obtained.

[0151] Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, as in the first embodiment, the entire face of the user is the predetermined body range to be imaged by the first imaging unit 21, and the predetermined part included therein and imaged by the second imaging unit 22 is the main part of the skin of the face. The terminal device 20 is a smartphone, and the camera unit on the display unit 26 side is used as the first imaging unit 21, while the camera unit on the opposite side to the display unit 26 is used as the second imaging unit 22, and each is capable of performing imaging and communicating with the server 90 through the network 100. When imaging the predetermined part, a conversion lens 29 is attached in front of the second imaging unit 22 in the terminal device 20, so that the imaging range of the second imaging unit 22 is narrower than that of the first imaging unit 21, while imaging can be performed at a higher resolution, and an extremely narrow range of skin on the face is imaged as the predetermined part.

[0152] As in the first embodiment, in order to capture images using the imaging units of the terminal device 20, a conversion lens (not shown) is first attached to the terminal device 20 by the user. Next, the user operates the terminal device 20 to access a specific web page prepared by a skin condition measurement service providing server (e.g., server 90), and this access allows the terminal device 20 to use online services such as web applications provided by the server, and an application program that provides a user interface for using each imaging unit is launched in the terminal device 20, allowing the user to use each imaging unit through the terminal device 20 to capture images for skin condition measurement.

[0153] In addition, a user may operate terminal device 20 to start an application program on terminal device 20 that is pre-recorded in the terminal device and provides a user interface for using each imaging unit, so that the user can use each imaging unit to capture skin images through terminal device 20.

[0154] When the user performs an operation to start measurement on the user interface displayed on the display unit 26 of the terminal device 20, the terminal device 20 executes imaging of the face as the predetermined area of ​​the user's body.

[0155] In detail, in the terminal device 20, a user interface is displayed on the display unit 26 by executing an application program for measuring a skin condition. When a user operates this user interface, the first imaging unit 21 serving as a first imaging means in the terminal device 20 starts capturing an image of the user's face. The skin image data is continuously acquired by the terminal device 20 and transmitted to the server 90.

[0156] Then, the control unit 93 of the server 90 causes the display unit 26 of the terminal device 20 to display an image of the face based on the image data obtained by imaging. Also, the control unit 93 causes the display unit 26 of the terminal device 20 to display a predetermined guide display or the like, to have the user maintain a state in which the terminal device 20 is held so that the face is displayed in a predetermined orientation and size.

[0157] The user refers to the facial image displayed on the display unit 26, and performs a predetermined operation on the user interface of the terminal device 20 to determine a still image of the face at the timing when the face as the predetermined body area is deemed to be appropriately captured by the first imaging unit 21. Data of the still image of the face at the determination moment is transmitted to the server 90, and is handled by the control unit 93 as skin image data of the predetermined body area obtained by imaging by the first imaging unit 21.

[0158] The control unit 93 acquires a display image of the face based on the skin image data obtained by imaging with the first imaging unit 21, transmits this to the terminal device 20, and displays it on the display unit 26 while analyzing the skin image data and temporarily acquiring diagnostic information about the skin condition of the user's face.

[0159] In addition, the control unit 93 refers to the information already recorded in the recording unit 97 and determines whether or not there is information about the specified area that was set during a previous image capture, that is, information about this set specified area that was recorded when the control unit 13 set the specified area of ​​the skin to be captured by the second imaging unit 12 using diagnostic information on the facial skin condition that was previously obtained in the same manner as described above.

[0160] If information on previously set specified areas has been recorded, the control unit 93 uses the diagnostic information based on the recorded information to identify areas within the specified body range other than those previously set as specified areas, and determines areas where detailed images need to be obtained by imaging with the second imaging unit 22 for more appropriate diagnosis of the skin condition, for example, areas where signs of the presence of blemishes or the like have been found on part of the skin and confirmation of the presence or absence of blemishes is required.

[0161] On the other hand, when information about the specified area to be imaged by the second imaging unit 22 that was previously set is not recorded in the recording unit 97, such as when imaging the user's skin for the first time, the control unit 93 uses the diagnostic information to determine, for the entire specified range of the body, areas that require detailed images to be obtained by imaging with the second imaging unit 22 in order to more appropriately diagnose the skin condition, as described above.

[0162] The control unit 93 sets the portion thus determined as the predetermined portion to be imaged by the second imaging unit 22, and records information about the set predetermined portion in the recording unit 97. The control unit 93 then generates a predetermined guide design portion indicating the predetermined portion, performs processing to synthesize the guide design portion onto a display image of the face based on the facial skin image data, and transmits the synthesized image to the terminal device 20, which then causes the display unit 26 of the terminal device 20 to display it. By looking at this display unit 26, the user recognizes a specific area to be imaged by the second imaging unit 22, and adjusts the position of the terminal device 20 including the second imaging unit 22 so that this area can be imaged.

[0163] In addition, as in the first embodiment, instead of setting a predetermined part to be imaged by the second imaging unit 22 and synthesizing an image of a guide design part or the like showing the predetermined part with a display image of the face and displaying it, the control unit 93 can transmit an instruction to the terminal device 20 to guide the user to take an image of a part of the face as the predetermined body range of the user, which part is concerned about the skin condition after viewing the face display by the display unit 26, and cause the terminal device 20 to execute a guide display or voice guide for the user. Alternatively, a part where the skin condition is generally measured is selected in advance and registered as an initial measurement position, and a process of synthesizing a predetermined guide design part showing one or more parts of the skin as the initial measurement position with a display image of the face is performed, and the synthesized image is transmitted to the terminal device 20 and displayed on the display unit 26.

[0164] The user adjusts the position of the terminal device 20 held in the hand, and when the user determines that the second imaging unit 22 is facing the specified part of the face, etc., the user performs a specified operation on a user interface such as the shutter button of the terminal device 20 to capture an image using the second imaging unit 22.

[0165] In this imaging stage, the second imaging unit 22 images the predetermined part of the skin under a predetermined lighting condition by an external light source (for example, a built-in light source of a conversion lens) or a built-in light source of the terminal device 20, and outputs skin image data. The skin image data of the predetermined part is transmitted from the terminal device 20 to the server 90 and acquired by the control unit 93.

[0166] For example, a built-in light source of a conversion lens attached to terminal device 20, which can adjust the light emitted as illumination, optimizes the light illumination state on the area to be imaged through verification of skin images obtained in trial imaging by second imaging unit 22, and then imaging is performed. Each piece of skin image data, an image for texture diagnosis in which the skin is imaged under illumination with unpolarized light irradiation, and an image for blemish diagnosis in which the skin is imaged under illumination with linearly polarized light irradiation, are transmitted from terminal device 20 to server 90 and acquired by control unit 93.

[0167] The control unit 93 records the acquired skin image data of the specific area in the recording unit 97 in a manner that makes it possible to determine the date and time when this skin image data was acquired, in addition to the information of the specific area that was previously recorded. In addition, when information on a previously set specific area is recorded in the recording unit 97, the control unit 93 refers to the recording unit 97 and regards the skin image data of the specific area, whose data acquisition date and time is included in a past specific period corresponding to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture by the second imaging unit 22.

[0168] Then, after acquiring skin image data of the specified area of ​​the skin from the second imaging unit 22, the control unit 93 analyzes this skin image data, the previously recorded skin image data of the specified area acquired during the specified period which is deemed to have been acquired at the same time, and the facial image data acquired previously by the first imaging unit 21, to acquire formal diagnostic information on the skin condition of the face as the specified area of ​​the user's body.

[0169] In addition, if information about a previously set specific area has not been recorded in the recording unit 97, the control unit 93 acquires skin image data of the specific area of ​​the skin from the second imaging unit 22, and then analyzes this skin image data and the facial skin image data previously acquired by imaging with the first imaging unit 21, to obtain formal diagnostic information about the condition of the user's facial skin.

[0170] Then, the control unit 93 transmits diagnostic information on the facial skin condition to the terminal device 20 (see FIG. 11), and displays the diagnostic information on the display unit 26 of the terminal device 20 simultaneously with the facial image or in place of the facial image.

[0171] In addition, the control unit 93 can generate a display image of the specified part of the skin based on image data of the specified part, transmit information about the display image of the specified part to the terminal device 20, and display the image of the specified part in any area on the screen of the display unit 26 of the terminal device 20 as necessary.

[0172] The display of diagnostic information on the facial skin condition on the display unit 26 of the terminal device 20 is not limited to text-based information display, as in the first embodiment, but can also be provided in a format in which areas of the skin requiring attention or symptoms are specifically indicated by marking images or the like on the facial image, based on display control of the diagnostic information by the control unit 93. For example, image information can be displayed in which the positions and sizes of hidden blemishes derived as diagnostic information analyzed from the fine image data obtained by the second imaging unit 22 are indicated by marks of a predetermined color superimposed on the corresponding areas on the facial image, and this can be used as reference information when carrying out skin care such as blemish prevention. When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 21 and the second imaging unit 22 is completed.

[0173] After this, in order to grasp more detailed skin conditions, for example, the skin condition of each part of the face and the progress of skin condition with care, the user repeats the same procedure as described above, at predetermined intervals, capturing images of the face as the predetermined area of ​​the body with the first imaging unit 21 and capturing images of the predetermined part with the second imaging unit 22, and the control unit 93 of the server 90 which has acquired each piece of skin image data acquires diagnostic information on the skin condition based on each piece of skin image data for each image capture, and displays it on the display unit 26 of the terminal device 20.

[0174] While repeating such imaging, the control unit 93 records information about the specified areas set in the recording unit 97, and when the next imaging is performed, the recorded information is referenced to set a new specified area that does not overlap with the previous one. As in the first embodiment, the face area as the specified body range is gradually filled in with areas for which detailed skin image data has been obtained by imaging with the second imaging unit 22 with each imaging, and multiple high-precision skin image data can be connected in a map-like manner, allowing analysis of the skin condition based on the skin image data to be performed with higher precision, and accurate and reliable diagnostic information can be derived.

[0175] Furthermore, while the control unit 93 records information about the specified area and skin image data obtained by imaging the specified area by the second imaging unit 22 each time an image of the specified area is captured by the second imaging unit 22, at the time of a new imaging opportunity, if any of the information about the specified area and skin image data recorded in the recording unit 97 has an acquisition date and time that exceeds a specified period of time in the past that corresponds to the turnover cycle of the user's skin, the control unit 93 can determine that this data acquisition date and time does not overlap with the turnover of the user's skin at the new imaging opportunity, and can erase such old information and skin image data so that new recording can be performed repeatedly without difficulty.

[0176] In addition, the facial skin image data obtained by imaging with the first imaging unit 21, which is the basis for the display image and diagnostic information on the skin condition of the face as the specified body area acquired by the server 90, is the skin image data obtained by imaging with the first imaging unit 21 before imaging with the second imaging unit 22. Alternatively, the control unit 93 of the server 90 may acquire the skin image data of the specified area acquired by the second imaging unit 22 from the terminal device 20, and then perform image correction processing on the facial skin image data based on the skin image data of the specified area, generate a display image of the face based on this corrected facial skin image data, and transmit this display image of the face to the terminal device 20 to be displayed on the display unit 26 of the terminal device 20, or may analyze the corrected facial skin image data together with the skin image data of the specified area to acquire diagnostic information on the facial skin condition.

[0177] In this case, since the imaging range is limited and it is easier to prepare the imaging conditions, image correction processing is performed on the image data of the specified body range based on the image data of the specified part based on the imaging of the second imaging unit 22, and the color of the skin image of the specified body range shown in this image data can be matched to that of the skin image in the image data of the specified part. Therefore, regardless of the imaging conditions of the first imaging unit 21, the image data of the specified body range obtained by imaging can be made in a state that makes it easy to display the image and perform analysis related to diagnosing the skin condition, thereby encouraging the user to correctly understand the skin condition.

[0178] In this way, the skin condition measuring system of this embodiment images the face as a specified area of ​​the user's body with the first imaging unit 21, and images a specified part of the user's face with the second imaging unit 22, and based on the obtained skin image data of the face and the skin image data of the specified part, the server 90 obtains diagnostic information on the skin condition and an image for display as information related to the user's skin condition. Therefore, when analyzing the skin image data to derive information related to the skin condition, the quality of information related to the user's skin condition, such as diagnostic information on the skin condition based on the skin image data obtained by multiple images, can be improved, and processing related to the display and analysis of the skin condition on the server 90 can be performed with high accuracy, allowing the skin condition diagnosis to be carried out appropriately and efficiently, while also increasing the reliability of the information obtained, allowing the user to understand the skin condition more accurately.

[0179] In the skin condition measuring system according to the embodiment, the facial skin image data obtained by imaging with the first imaging unit 21 as the first imaging means is transmitted only from the terminal device 20 to the server 90 and used for processing in the control unit 93 to obtain information related to the skin condition, such as diagnostic information. However, this is not limited to this. As shown in FIG. 12, the facial skin image data as the specified body range can be transmitted together with the server 90 to another person's terminal device 60, and an image of the user including the face can be displayed on the other person's terminal device 60 based on the transmitted skin image data in parallel with the control unit 93 of the server 90 obtaining information related to the skin condition based on the skin image data.

[0180] This can be applied to a usage situation, such as a video call, in which an image based on imaging at each terminal device is continuously displayed on the other party's terminal device while communicating between the user's terminal device 20 and the other party's terminal device 60. As a specific example, a user can capture an image of his / her face with the first imaging unit 21 of his / her own terminal device 20 to obtain diagnostic information, and while viewing the diagnostic information finally acquired by the server 90, make a video call with a specific party through the terminal device, and the image of the user displayed on the other party's terminal device 60 is based on the facial skin image data obtained by imaging with the first imaging unit 21.

[0181] As an application example of this, a configuration can be adopted in which a user receives skin counseling via a video call with a counselor via a terminal device, and while receiving support from the counselor via the call, the user operates the terminal device 20 to capture images of the face and the specified parts of the body as the specified range of the body using the first imaging unit 21 and the second imaging unit 22 of the terminal device 20, and the skin image data obtained by the imaging is transmitted from the terminal device 20 to the server 90, and the control unit 93 of the server 90 obtains diagnostic information about the skin condition.

[0182] In this case, the skin condition diagnostic information acquired by the control unit 93 of the server 90 can be transmitted to the counselor, i.e., to the terminal device 60 of a person other than the user, so that the user and the counselor can share the diagnostic information and reach a common understanding of the user's skin condition, and the counselor can then provide detailed counseling on the user's skin based on the diagnostic information. Also, the skin condition diagnostic information may not be transmitted from the server 90 to the user's terminal device 20, but may be viewable only on the counselor's terminal device 60, which prevents the user from being given preconceived ideas due to the diagnostic information and makes it easier for the user to smoothly accept the counselor's advice. Furthermore, if the second imaging unit as the second imaging means is disposed on the same side as the first imaging unit in the user's terminal device in the same orientation, or is shared by the first imaging unit of the terminal device, or is an imaging unit independent of the user's terminal device, then when the user images the specified part of his or her face with the second imaging unit, the user can adjust the distance between the second imaging unit and the specified part of the user's face or appropriately point the second imaging unit at the specified part while receiving support for imaging by phone from a counselor who can check the state of the image of the user through the display screen of the terminal device, which is preferable.

[0183] Furthermore, in the skin condition measuring system according to the embodiment, the first imaging unit 21 as the first imaging means is disposed in the user's terminal device 20, and is configured to image the face of the user as the specified range of the user's body with the first imaging unit 21 by operation of the user, but this is not limited to this. The first imaging means may be a specified imaging device that is part of or connected to another person's terminal device and images at least the specified range of the user's body, and the image data of the user obtained by such first imaging means is used on the other person's terminal device for a main purpose other than obtaining information regarding the user's skin condition. The server 90 may receive skin image data of the specified range of the user's body obtained by imaging with the first imaging means, and the control unit 93 of the server 90 may obtain information regarding the user's skin condition, such as diagnostic information, based on the skin image data of the specified range of the body and the skin image data of the specified part obtained by imaging with the second imaging means.

[0184] As a specific example, as shown in Figures 13 and 14, an image of the user is captured by an imaging unit of another person's terminal device 61, and the main purpose of personal authentication processing to identify the user is performed from the obtained image data. The image data obtained by capturing the image on the other person's terminal device 61 is then reused by server 90 as skin image data of a specified area of ​​the user's body, and the control unit 93 analyzes this together with skin image data of the specified area obtained separately from a second imaging means, to obtain diagnostic information on the skin condition.

[0185] In this case, image data of the user acquired by imaging with the imaging unit of another person's terminal device 61 can be reused for skin diagnosis, so there is no need to prepare a new first imaging means for imaging the specified area of ​​the user's body, and information regarding the skin condition, such as diagnostic information based on each image data, can be acquired efficiently and at low cost.

[0186] (Third embodiment of the present invention) In the skin condition measuring system according to the first embodiment, the second imaging unit 12 as the second imaging means is configured to be capable of imaging a predetermined part of the face (e.g., part of the cheek, forehead, etc.) that overlaps with the face as the predetermined area of ​​the user's body. In addition, as a third embodiment, the second imaging means can also be configured to image a predetermined part of the skin that does not overlap with the predetermined area of ​​the user's body.

[0187] In this case, the control unit acquires information regarding the skin condition of the specified part of the skin based on skin image data of the specified part obtained by imaging with the second imaging means, and then acquires information regarding the relative skin condition of each part of the specified body range based on the information regarding the skin condition of the specified part as information regarding the skin condition of the specified body range.

[0188] As a specific example, the face of the user as the predetermined body range is imaged by the first imaging unit 31 as the first imaging means, and a predetermined part of the skin that does not overlap with the face, for example, the surface of the arm close to the torso or the underside of the chin, which is less exposed to sunlight and less affected by ultraviolet rays, is imaged by the second imaging unit 32 as the second imaging means provided in the terminal device 30 (see FIG. 15). Note that the imaging by the second imaging unit 32 may be performed before or after the imaging by the first imaging unit 31.

[0189] A control unit (not shown) acquires information on the skin condition of the predetermined part of the skin based on the skin image data of the predetermined part obtained by imaging with the second imaging unit 32. Furthermore, when the control unit acquires information on the skin condition of the face based on the skin image data of the face as the predetermined body range obtained by imaging with the first imaging unit 31, the control unit acquires information on the skin condition of the face as information on the relative skin condition of each part of the face based on the previously acquired information on the skin condition of the predetermined part.

[0190] This allows the skin condition of the user's face to be shown relatively to the skin condition of the specified area, which is less affected by ultraviolet rays and has a smaller degree of skin deterioration, and is therefore suitable as a standard for skin evaluation.The degree of deterioration of the skin condition of each part of the face in the specified area of ​​the body can be shown in a form that is easy for the user to understand, and the user can more accurately grasp the facial skin condition as a difference from the user's ideal skin condition.The skin condition of the specified area used as the standard can be used as a target state, and the user can be encouraged to take appropriate care of their facial skin.

[0191] In addition, the terminal device may be configured to suggest to the user skin care that brings the facial skin condition closer to the target skin condition of the specified part based on information on the facial skin condition, such as relative diagnostic information on the skin condition based on the information on the skin condition of the specified part acquired by the control unit. In this case, an external AI may be used to perform a more detailed analysis based on various information on the skin condition in addition to the diagnostic information, leading to a more effective skin care suggestion.

[0192] In the above embodiment, the first imaging unit 31 as the first imaging means and the second imaging unit 32 as the second imaging means are arranged in one terminal device 30, and the imaging by each imaging unit is performed at different timing, so that imaging can be easily performed with one terminal device 30, but in addition to this, a configuration in which the second imaging means is provided separately from the first imaging means, for example, a configuration in which another imaging device 36 having a camera unit forming the second imaging means is used in combination with a terminal device 35 having a camera unit forming the first imaging means, may be used. In this case, imaging can be performed simultaneously by the first imaging means and the second imaging means (see FIG. 16), shortening the time from imaging to obtaining information on the skin condition, reducing the burden on the user related to imaging, and making it easier for the user to continue activities such as imaging to obtain information on the skin condition.

[0193] (Fourth embodiment of the present invention) A skin condition measuring system according to a fourth embodiment of the present invention will be described with reference to FIGS. In each of the above figures, the skin condition measuring system 4 of this embodiment is equipped with a first imaging unit 41, a second imaging unit 42, and a control unit 43, like the first embodiment, but differs in that the terminal device 40 in which the first imaging unit 41, the second imaging unit 42, and the control unit 43 are arranged is configured to simultaneously capture images using the first imaging unit 41 and the second imaging unit 42, and the first imaging unit 41 is configured to capture a mirror image of the user reflected on a mirror surface 80 in front of the user.

[0194] In this embodiment, as in the first embodiment, the user who is the subject of skin condition measurement has entered into a contract with the provider of skin condition measuring system 4 to receive skin condition diagnosis services, and is able to use skin condition measuring system 4, thereby obtaining skin condition diagnostic information from skin condition measuring system 4.

[0195] The first imaging unit 41 is disposed in the terminal device 40 as the first imaging means, and is capable of imaging a predetermined region of the user's body, such as the face, for measuring the user's skin condition. However, the first imaging unit 41 images a mirror image of the predetermined region of the user's body reflected on a mirror surface 80 located in front of the user.

[0196] The second imaging unit 42, as the second imaging means, is arranged in the same terminal device 40 as the first imaging unit 41, and is capable of imaging a specified part of the user's skin, such as a part of the cheek or forehead, which is narrower than the specified area of ​​the user's body and overlaps with this specified area of ​​the body, in order to measure the user's skin condition.

[0197] This second imaging unit 42 is arranged on the other side of the surface on the terminal device 40 opposite to the surface on which the first imaging unit 41 is arranged, and is capable of imaging the specified part of the user when the first imaging unit 41 of the terminal device 40 is directed toward the mirror surface 80 in front of the user. Further, the second imaging section 42 is capable of acquiring image data having a higher resolution per unit area of ​​the imaging target than the first imaging section 41 .

[0198] The first imaging unit 41 and the second imaging unit 42 are capable of performing imaging by user operation via the terminal device 40, and imaging of the specified range of the body by the first imaging unit 41 and imaging of the specified part of the skin by the second imaging unit 42 are performed at the same time.

[0199] The terminal device 40 has the first imaging unit 41 and the second imaging unit 42, and is a device capable of imaging a specified range of the user's body and the specified part of the body and acquiring skin image data, and also has the control unit 43.

[0200] The terminal device 40 is a handheld device capable of executing application programs, and has a display unit 46 capable of displaying various types of information. The display unit 46 of the terminal device 40 is capable of displaying an image of the user. The display unit 46 is a touch panel type display device, and also serves as an input device.

[0201] Terminal device 40 is an information processing device that realizes the functions and processes of control unit 43 by executing a pre-stored program, and is not limited to a handheld device, but may be a fixed device as long as it is positioned between the user and the mirror surface 80 in front of the user and can simultaneously capture images using first imaging unit 41 and second imaging unit 42. Also, considering that mirror surface 80 may be that of a mirror often installed in places where there is moisture, such as bathrooms, washstands, and vanities, terminal device 40 is preferably made of a waterproof structure since it will be used in the same space.

[0202] The control unit 43 of the terminal device 40 can use an application program for diagnosing a skin condition to analyze skin image data obtained by imaging and obtain diagnostic information, and the obtained diagnostic information can be referred to by the user. In this case, the control unit 43 of the terminal device 40 executes the application program to display the obtained diagnostic information as content on the display unit 46, thereby making it possible for the user to refer to the information.

[0203] The control unit 43 acquires information regarding the skin condition based on skin image data obtained by imaging, i.e., skin image data of the specified range of the body obtained by imaging with the first imaging unit 41 and skin image data of the specified part obtained by imaging with the second imaging unit 42.

[0204] This control unit 43 analyzes the skin image data of the predetermined body area and the skin image data of the predetermined part to obtain diagnostic information on the skin condition of the predetermined body area of ​​the user as information on the skin condition. The function of the control unit 43 to analyze each skin image data and obtain diagnostic information on the user's skin condition in this manner is based on the same mechanism as in the first embodiment, which is realized by the control unit 43 executing processing according to an application program, and detailed description will be omitted.

[0205] In addition, as other information regarding the skin condition, the control unit 43 obtains a display image of a specified area of ​​the body based on skin image data of the specified area of ​​the body, and obtains a display image of the specified area based on skin image data of the specified area.

[0206] The control unit 43 then causes the display unit 46 of the terminal device 40 to display a display image of the specified body area, specifically the face, while at the same time replacing the image in the same display area on the screen of the display unit 46 in which the display image is displayed, or causes the control unit 43 to display a display image of the specified part of the body with a larger magnification than the display image of the specified body area, simultaneously and in parallel with the display of the display image of the face, in an area other than the display area.

[0207] In addition, the control unit 43 can use diagnostic information on the skin condition of the specified area of ​​the user's body as information on the skin condition previously obtained to set a specified area of ​​the skin to be imaged by the second imaging unit 42 included in the specified area of ​​the body, and notify the user of the information on this specified area via the terminal device 40.

[0208] In this case, when skin image data of a specified area of ​​the skin is obtained by the second imaging unit 42 imaging the specified area of ​​the skin based on user operation referring to information about the specified area, the control unit 43 can obtain diagnostic information about the skin condition of the specified area of ​​the body based on the skin image data of the specified area and the skin image data of the specified area of ​​the body obtained by simultaneous imaging.

[0209] In addition, the control unit 43 has a recording unit 47 that records information about a specific area of ​​the skin to be imaged by the second imaging unit 42 when the control unit 43 uses the acquired diagnostic information on the skin condition of the specific area of ​​the body to set the specific area of ​​the skin to be imaged by the second imaging unit 42.

[0210] The recording unit 47 records information about the specific area of ​​the skin to be imaged by the second imaging unit 42, as well as skin image data obtained by imaging this specific area by the second imaging unit 42, in a manner that makes it possible to determine the date and time when this skin image data was obtained.

[0211] When information on a previously set specified area is recorded in the recording unit 47, the control unit 43 uses the diagnostic information on the skin condition of the specified area of ​​the body previously acquired during a new imaging opportunity with the first imaging unit 41, and also refers to the information on the specified area already recorded in the recording unit 47 to set a new specified area of ​​the skin to be imaged by the second imaging unit 42 from among areas of the specified area of ​​the body that have not been recorded as a specified area.

[0212] In addition, the control unit 43 has a mechanism for referencing information on the specific area of ​​the skin that has already been recorded in the recording unit 47, the skin image data of this specific area, and the date and time that data was obtained, in parallel with setting the specific area of ​​the skin to be imaged by the second imaging unit 42 during a new imaging opportunity by the first imaging unit 41.

[0213] The control unit 43 then regards skin image data of the recorded specified area, whose data acquisition date and time falls within a specified period of the past corresponding to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture.

[0214] In this case, the control unit 43 acquires diagnostic information (information related to the skin condition) of the skin condition of the specified area of ​​the body based on the recorded skin image data acquired during the specified period, the skin image data of the specified area of ​​the skin set at this new imaging opportunity obtained by imaging the specified area by the second imaging unit 42, and the skin image data of the specified area of ​​the body.

[0215] On the other hand, the control unit 43 may delete from the recording unit 47 the information on the specific area and the skin image data of the specific area, if the data acquisition date and time is not included in the specified period and is older than the turnover cycle of the user's skin, on the basis that the data is from a period when the skin turnover does not overlap with the user's skin in the new imaging opportunity by the first imaging unit 41, and therefore cannot be regarded as having been captured at the same time as the new imaging.

[0216] Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, as in the first embodiment, the entire face of the user is set as a predetermined body range imaged by the first imaging unit 41, and a predetermined part included therein and imaged by the second imaging unit 42 is a main part of the facial skin. It is assumed that the first imaging unit 41 and the second imaging unit 42 of the terminal device 40 are each capable of performing imaging.

[0217] First, a user operates an application program on terminal device 40 that already exists in the terminal device and that provides a user interface for using each imaging unit, and the user is then able to use each imaging unit for imaging via terminal device 40.

[0218] The user then holds the terminal device 40 so that the first imaging unit 41 as the first imaging means faces the mirror image of the face as a specified range of the user's body reflected on the mirror surface 80 located in front of the user, and the second imaging unit 42 as the second imaging means faces a part of the user's face.

[0219] When the user is holding the terminal device 40 in this manner, the first imaging unit 41 captures an image of the mirror image of the face as the specified range of the body reflected on the mirror surface 80 located in front of the user, while the second imaging unit 42 captures an image of a part of the user's face, and the control unit 43 continuously acquires the image data obtained by each imaging unit.

[0220] In addition, as the first imaging unit 41 and the second imaging unit 42 capture images, the images captured by each imaging unit are displayed on the display unit 46 of the terminal device 40 based on the image data obtained by the capture. In this case, the control unit 43, by displaying a predetermined guide on the display unit 46, etc., causes the user to maintain a state in which the terminal device 40 is held and a distance between the user and the mirror surface 80 so that at least a mirror image of the face is captured by the first imaging unit 41 in a predetermined orientation and size, and the control unit 43 obtains a display image of the face based on the facial skin image data obtained by imaging by the first imaging unit 41, and causes this display image of the face to be appropriately displayed on the display unit 46.

[0221] Then, the control unit 43 executes guidance display, audio guidance, etc. on the terminal device 40 to the user so that the user will capture an image of a part of the face of the user that is of concern to the user using the second imaging unit 42. Alternatively, a part where the skin condition is generally measured is selected in advance and registered as an initial measurement position, and a process is performed to synthesize a predetermined guide design part showing one or more parts of the skin that are the initial measurement positions onto a display image of the face, and the synthesized image is displayed on the display unit 46.

[0222] In response to this, the user can recognize any parts of the face that are of concern, or check the initial measurement positions on the face indicated by the guide design displayed on the display unit 46, and then fine-tune the position and orientation of the terminal device 40 equipped with the second imaging unit 42 so that these can be imaged.

[0223] The user aims the first imaging unit 41 and the second imaging unit 42 at the respective imaging targets, and referring to the mirror image of the face and the image display of a predetermined part of the face on the display unit 46, performs a predetermined operation on the user interface such as the shutter button of the terminal device 40 at the timing when each imaging target is deemed to be appropriately captured by the corresponding imaging unit, thereby determining the mirror image of the face and the still image of the predetermined part. The data of the mirror image of the face and the still image of the predetermined part at the determination moment are handled by the control unit 43 as the skin image data of the predetermined body area and the skin image data of the predetermined part.

[0224] In this stage of determining the image, the second imaging unit 42 may be configured to image the predetermined part of the skin under a predetermined illumination condition using an external light source or a built-in light source of the terminal device 40. For example, in the terminal device 40, a built-in light source capable of adjusting the light emitted as illumination optimizes the illumination state of light on the predetermined part of the face of the subject to be imaged through verification of the skin image obtained by trial imaging by the second imaging unit 42, and then imaging is performed under illumination, and the control unit 43 can acquire each of image data of an image for texture diagnosis obtained by imaging the skin under illumination using unpolarized light irradiation and an image for blemish diagnosis obtained by imaging the skin under illumination using linearly polarized light irradiation.

[0225] The control unit 43 acquires display images of the face and the specified parts based on the skin image data obtained by the first imaging unit 41 and the second imaging unit 42, and displays these on the display unit 46 while analyzing the skin image data and acquiring diagnostic information on the skin condition of the user's face.

[0226] Then, the control unit 43 causes the diagnostic information on the facial skin condition to be displayed on the display unit 46 either simultaneously with the facial image or in place of the facial image. The control unit 43 also records information on the position of the predetermined part on the face and the skin image data of the predetermined part in the recording unit 47 in a state in which the date and time when the skin image data was obtained can be determined.

[0227] The display of diagnostic information on the facial skin condition on the display unit 46 is not limited to displaying text-based information, but can also provide information by specifically indicating areas and symptoms of the skin that require attention using marking images on the facial image, etc. For example, the position and size of hidden blemishes derived as diagnostic information analyzed from the detailed image data obtained by the second imaging unit 42 can be displayed using marks of a predetermined color superimposed on the corresponding areas on the facial image, and can be used as reference information when carrying out skin care such as blemish prevention. When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 41 and the second imaging unit 42 is completed.

[0228] After this, in order to grasp the progress of the skin condition, the user will repeat the same procedure at predetermined intervals, capturing images of the face as the specified area of ​​the body with the first imaging unit 41 and capturing images of the specified part with the second imaging unit 42, and the control unit 43 will obtain diagnostic information on the skin condition based on the obtained skin image data for each image capture and display it on the display unit 46.

[0229] In this case, the control unit 43 refers to the information already recorded in the recording unit 47 and determines whether or not there is information on the specific part of the face that was set during a previous image capture, that is, information on this set specific part that was recorded when the control unit 43 set the specific part of the skin to be captured by the second imaging unit 42 using skin condition diagnostic information, etc.

[0230] If information on previously set specified areas has been recorded, the control unit 43 uses the previous diagnosis information for each part of the specified body range other than the areas previously set as specified areas based on the recorded information to determine areas where detailed images need to be obtained by imaging with the second imaging unit 42 for more appropriate diagnosis of the skin condition, for example, if signs of the presence of blemishes or the like have been found in part of the skin and confirmation is required for the presence or absence of blemishes.

[0231] On the other hand, if information about the specified area to be imaged by the previously set second imaging unit 42 has not been recorded in the recording unit 47, the control unit 43 uses the previous skin condition diagnosis information for the entire specified area of ​​the body to determine areas that require detailed images to be obtained by imaging with the second imaging unit 42 in order to more appropriately diagnose the skin condition, as described above.

[0232] The control unit 43 sets the portion determined as described above as the predetermined portion to be imaged by the second imaging unit 42, and records information about the set predetermined portion in the recording unit 47. Then, the control unit 43 generates, for this predetermined portion, for example, a predetermined guide design portion (mark) indicating the predetermined portion on the face to be imaged by the second imaging unit 42, performs processing to synthesize this guide design portion with a display image of the face based on the facial skin image data, and causes the display unit 46 to display the synthesized image. By looking at this display unit 46, the user recognizes the specific area to be imaged by the second imaging unit 42, and fine-tunes the position and orientation of the terminal device 40 so that this area can be imaged by the second imaging unit 42.

[0233] While repeating this process of imaging, the control unit 43 records information about the specified areas set in the recording unit 47, and when imaging next time, the recorded information is referred to to set a new specified area that does not overlap with the previous one. In this way, the face area as the specified body range is gradually filled in with areas for which detailed skin image data has been obtained by imaging with the second imaging unit 42 with each imaging, and multiple high-precision skin image data can be connected in a map-like manner, allowing analysis of the skin condition based on the skin image data to be performed with greater precision, resulting in accurate and reliable diagnostic information.

[0234] In this way, each time an image of the specified area is captured by the second imaging unit 42, the control unit 43 records information about the specified area and skin image data obtained by capturing the specified area by the second imaging unit 42. At the time of a new imaging opportunity, for any information or skin image data recorded in the recording unit 47 whose data acquisition date and time exceeds a specified period of time in the past corresponding to the user's skin turnover cycle, the control unit 43 can determine that the skin turnover does not overlap with the user's skin at the time of this new imaging opportunity, and can erase such old information or skin image data so that new recording can be performed repeatedly without difficulty.

[0235] It should be noted that when a new image is captured by the second imaging unit 42, the control unit 43 refers to the information recorded in the recording unit 47 so that a new specified part that does not overlap with the previous part is captured by the second imaging unit 42. However, without being limited to this, while the first imaging unit 41 repeatedly captures the specified area of ​​the body and the second imaging unit 42 repeatedly captures the specified part, the second imaging unit 42 can also be configured to repeatedly capture the same specified part many times. From the diagnostic information on the skin condition and the display image obtained by the control unit 43 based on the skin image data focusing on the same part of the skin, the detailed progress of the skin condition in a specific part can be grasped, and the degree of improvement due to skin care can be easily confirmed.

[0236] In this way, the skin condition measuring system of this embodiment captures an image of the mirror image reflected on the mirror surface 80 in front of the user with the first imaging unit 41 to obtain skin image data of the specified area of ​​the body, and simultaneously captures an image with the second imaging unit 42 to obtain skin image data of the specified area, and the control unit 43 obtains information regarding the skin condition from each piece of skin image data and makes this information regarding the skin condition available. Therefore, by capturing images simultaneously with each imaging unit, skin image data of the specified area of ​​the body and skin image data of the specified area can be obtained at once, shortening the time from capturing images to obtaining information regarding the skin condition and reducing the burden on the user related to capturing images, and also allowing information regarding the skin condition to be obtained without taking much time, making it possible to quickly grasp the user's skin condition.

[0237] Furthermore, a first imaging unit 41 is arranged on one surface of the terminal device 40, and with this first imaging unit 41 facing the mirror surface 80, it is possible to image the specified part of the user, and a second imaging unit 42 is arranged in the same terminal device 40, and one device can handle imaging of the specified range of the user's body and the specified part. Therefore, skin image data of the specified range of the body and skin image data of the specified part can be obtained together by imaging with each imaging unit of the terminal device 40, and the user need only handle the terminal device 40 equipped with each imaging unit when imaging, which significantly reduces the user's burden related to imaging and enables the user to continue activities to obtain information about their skin condition without difficulty.

[0238] In the skin condition measuring system according to the embodiment, the terminal device 40 in which the first imaging unit 41 as the first imaging means and the second imaging unit 42 as the second imaging means are arranged is configured as a device that exclusively captures images using the first imaging unit 41 and the second imaging unit 42 to obtain diagnostic information on the skin condition. However, this is not limited to this, and a communication terminal device such as a smartphone that can be used for various purposes and that has multiple camera units can be configured to be used as a terminal device in which the multiple camera units function as the first imaging means and the second imaging means, respectively, if the multiple camera units are configured to capture images for each camera unit simultaneously and obtain image data individually.

[0239] In addition, in the terminal device 40, the second imaging unit 42 as the second imaging means is configured to be disposed on the opposite surface of the surface on which the first imaging unit 41 as the first imaging means in the terminal device 40 is disposed, but this is not limiting, and the second imaging means disposed in the terminal device may be disposed in any position of the terminal device as long as the second imaging means is capable of imaging the predetermined part of the user in a state in which the first imaging means in the terminal device is directed toward the mirror surface 80 that reflects the user in front of the user. In particular, when the imaging target direction can be changed by adding a conversion lens or the like, the second imaging means may be disposed on the surface on which the first imaging means in the terminal device is disposed. In addition, the second imaging unit 42 as the second imaging means is not limited to being provided in the same terminal device 40 as the first imaging unit 41 as the first imaging means, but may be provided separately from the first imaging means, for example, a terminal device having a camera unit constituting the first imaging means may be used in combination with another device having a camera unit constituting the second imaging means.

[0240] In the skin condition measuring system according to the embodiment, the second imaging unit 42 as the second imaging means is capable of imaging a predetermined part of the face (e.g., a part of the cheek, forehead, etc.) that overlaps with the face as the predetermined body area of ​​the user, while the imaging range of the second imaging unit 42 is a part hidden by the terminal device 40 in the mirror image of the face reflected on the mirror surface 80 in front of the user, and does not overlap with the imaging range of the first imaging unit 42 as the first imaging means that images the mirror image of the face, so that no part is commonly imaged by the first imaging unit 41 and the second imaging unit 42. In addition, as shown in FIG. The second imaging unit 49 in 5 can be configured to be able to image the specified part of the user's face even from a position a little distance away, and can also be configured to be able to image the specified part of the face even when the terminal device 45 is positioned away from the front of the face, for example, diagonally in front of or to the side of the face, within a range in which the first imaging unit 48 can image the mirror image of the face reflected on the mirror surface 80 in front of the user, so that the imaging range of the first imaging unit 48, which can image a mirror image of the entire face, overlaps with the imaging range of the second imaging unit 49, which can image the specified part of the face, and there are parts that are commonly imaged by the first imaging unit 48 and the second imaging unit 49.

[0241] In this case, the relationship between the skin image data obtained by the two different imaging units can be understood from the common specific areas in the skin image data, and the detailed skin condition of each part of the face as a specific area of ​​the body other than the overlapping areas can be inferred. Detailed information regarding the skin condition can be obtained by analysis of the skin image data in the control unit, and this information regarding the skin condition allows the user to correctly understand their skin condition, further encouraging the user to take appropriate skin care measures, etc.

[0242] In addition, in the skin condition measuring system according to the embodiment, the first imaging unit 41 as the first imaging means in the terminal device is directed toward a mirror surface 80 reflecting the user in front of the user, and captures a mirror image of the user's face as the specified body area reflected on the mirror surface 80 to obtain skin image data of the face. This skin image data is then analyzed by the control unit 43 to obtain information regarding the skin condition. No particular processing is performed on the image, and when the image of the face is displayed on the display unit 46, it is displayed as it was captured. However, the control unit can also be configured to process images of the image data obtained by the first imaging means, such as the mirror image of the specified body area (face) of the user, or the part of the user's body other than the specified body area or the wall behind the user that is reflected as a background, to the extent that a third party cannot discern the displayed content even if they view the image. This can reliably prevent leakage of personal information of the user that is unrelated to obtaining information regarding the skin condition, such as diagnostic information.

[0243] In addition, in the skin condition measuring system according to the embodiment, the control unit 43 analyzes both the skin image data of the face as the specified range of the body obtained by imaging with the first imaging unit 41 as the first imaging means, and the skin image data of the specified part obtained by imaging with the second imaging unit 42 as the second imaging means, to obtain diagnostic information on the skin condition. Alternatively, the skin image data of the specified range of the user's body obtained by imaging with the first imaging means can be used only to identify the position of the specified part imaged by the second imaging means of the terminal device based on the position of the terminal device contained therein, and only the skin image data of the specified part obtained by imaging with the second imaging means can be used for the image analysis by the control unit focusing on the skin. This ensures that the specified part imaged by the second imaging means in the specified range of the user's body can be correctly identified, and information on the skin condition, such as diagnostic information, can be obtained accurately based on the skin characteristics of each part of the specified range of the body, while further simplifying the process of obtaining information on the skin condition of the control unit based on each skin image data, reducing the load on the control unit and speeding up processing. [Explanation of symbols]

[0244] 1, 2, 4 Skin condition measurement system 10, 20, 30, 40 Terminal Equipment 11, 21, 31, 41 First imaging unit 12, 22, 32, 42 Second imaging unit 13, 23, 43 Control section 14, 24 Wireless communication means 16, 26, 46 display section 17, 27, 47 Recording section 19 Conversion lens 35 Terminal Equipment 36 Imaging device 45 Terminal Equipment 48 First Imaging Unit 49 Second Imaging Unit 50 Terminal Equipment 51 First Imaging Unit 52 Second imaging unit 53 Light source for lighting 54 Food 55 Mirror section 60, 61 Terminal equipment 80 Mirror surface 90 Servers 93 Control Unit 97 Recording Section 100 Network

Claims

[Claim 1] A first imaging means capable of imaging a predetermined area of ​​the skin of a user's body; A second imaging means capable of imaging a predetermined part of the user's skin which is narrower than the predetermined body area; a control unit that acquires information about a skin condition based on skin image data of the predetermined body area obtained by imaging with the first imaging means and skin image data of the predetermined part obtained by imaging with the second imaging means, the second imaging means is capable of imaging at a position closer to the user than the first imaging means and capable of acquiring image data having a higher resolution per unit area of ​​the imaging target than the first imaging means; The control unit is configured to analyze the skin image data of the predetermined body area and the skin image data of the predetermined part to obtain diagnostic information on the skin condition of the predetermined body area of ​​the user as information on the skin condition, The control unit acquires information on the skin condition of the predetermined part of the skin based on skin image data of the predetermined part obtained by imaging with the second imaging means, and then acquires, as information on the skin condition, information on the relative skin condition of each part of the predetermined body range based on the information on the skin condition of the predetermined part. A skin condition measuring system.

Images