Information processing method, information processing device, program, and information processing system

Abstract

For example, an information processing method, an information processing device, and a program for visualizing skin elasticity and / or skin distortion are provided. [Solution] The information processing method executed on the computer of the present disclosure includes an acquisition step of acquiring a plurality of skin images, the plurality of skin images being images captured over time of pressure treatment on the skin of a target, an assignment step of assigning feature points to each skin image, a matching step of matching the same feature points in each skin image, and a setting step of calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from the plurality of skin images, and the distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the distance difference as an index value of the distortion of skin tissue caused by the pressure treatment at the first feature point of the target skin image.

Description

[Technical field]

[0001] The present disclosure relates to an information processing method, an information processing device, a program, and an information processing system. [Background technology]

[0002] As a method for measuring the morphological characteristics of the skin, a method for visualizing or quantifying the texture, wrinkles, etc. from a skin surface image captured by a camera is widely used. In addition, as a method for measuring the physical properties of the skin, the measurement of elasticity with an elastometer, the measurement of plasticity, etc. are generally known (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-268460 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the measurement of elasticity and the measurement of plasticity using the elasticity meter have a problem in that a dedicated device for measuring skin is required.

[0005] Therefore, an object of the present disclosure is to provide a new information processing method that can be used to visualize, for example, skin elasticity and / or skin distortion. [Means for solving the problem]

[0006] In order to achieve the above object, the present disclosure provides an information processing method executed on a computer (hereinafter also referred to as "information processing method"), which includes an acquisition step of acquiring a plurality of skin images, the plurality of skin images being images of a pressure treatment applied to the skin of a subject over time; an assignment step of assigning feature points to each skin image; a matching step of matching the same feature points in each skin image; The method includes a setting step of calculating a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0007] The information processing device of the present disclosure includes an acquisition unit that acquires a plurality of skin images, The plurality of skin images are images obtained by capturing pressure treatment of a subject's skin over time; an adding unit that adds feature points to each skin image; a matching unit for matching the same feature points in each skin image; and a setting unit that calculates a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and sets the magnitude of the distance difference as an index value of the distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0008] The program of the present disclosure is provided to a computer. An acquisition process for acquiring a plurality of skin images, The plurality of skin images are images obtained by applying pressure to the skin of a subject over time; an assignment process for assigning feature points to each skin image; a matching process for matching the same feature points in each skin image; and executing a setting process to calculate a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and set the magnitude of the difference in distance as an index value of the distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0009] The present disclosure includes a terminal and a server, The terminal and the server can be connected via a communication network, The server includes an information processing device according to the present disclosure, The server sets an index value for distortion of skin tissue from a plurality of skin images transmitted from the terminal. Effect of the Invention

[0010] According to the present disclosure, for example, skin elasticity and / or skin distortion can be visualized. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of an information processing system including an information processing device according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the hardware configuration of the information processing device according to the first embodiment. [Diagram 3] FIG. 3 is a flowchart showing an information processing method and program according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of processing of a skin image according to the first embodiment. [Diagram 5] FIG. 5 is a block diagram showing an example of an information processing system including an information processing device according to the second embodiment. [Figure 6] FIG. 6 is a flowchart showing an information processing method according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Definition> In this specification, "skin tissue" refers to skin tissue composed of epidermis, dermis, and subcutaneous tissue. The epidermis is the outermost tissue of the skin and refers to a tissue including the stratum corneum and the stratum germinativum. The dermis is the main tissue of the skin that is between the epidermis and the subcutaneous tissue and refers to a tissue rich in collagen. The subcutaneous tissue is the innermost tissue in the skin tissue and refers to a tissue that supports the epidermis and the dermis.

[0013] In this specification, a "skin image" is an image of skin tissue. The skin image is preferably an image of the epidermis taken from the outside of the skin tissue.

[0014] In this specification, "external skin preparation" means a preparation used for application to the skin. The external skin preparation is preferably an external skin preparation applied to the skin of the body surface such as the face or body. The form of the external skin preparation may be an ampoule, capsule, powder, granule, liquid, gel, foam, emulsion, sheet, mist, spray, etc., depending on the form of use. The form of use may be, for example, medicine(s); quasi-drug(s); external skin preparations for topical or whole body use; medicinal and / or cosmetic preparations for application to the scalp and hair; bath additives for use in bath water; other preparations; etc. Examples of the topical or systemic skin preparations include basic cosmetics such as lotions, milky lotions, creams, ointments, lotions, oils and packs; face washes or skin cleansers such as solid soap, liquid soap and hand wash; massage preparations, cleansing preparations, hair removal preparations, depilatories, shaving treatments, aftershave lotions, pre-shave lotions, shaving creams, makeup cosmetics such as foundations, lipsticks, blushers, eye shadows, eyeliners and mascaras; perfumes, nail beautifying preparations, nail beautifying enamel, nail beautifying enamel removers, poultices, plasters, tapes, sheets, patches, aerosols, toothpastes and mouthwashes such as mouthwash. Examples of the medicinal and / or cosmetic preparations to be applied to the scalp and hair include shampoos, rinses, hair treatments, pre-hair treatments, permanent solutions, hair dyes, hair styling products, hair tonics, hair growth and hair care products, poultices, plasters, tapes, sheets, aerosols, etc. Examples of the other preparations include underarm odor prevention or deodorants, antiperspirants, sanitary products, sanitary cotton, wet tissues, etc.

[0015] Hereinafter, the present disclosure will be described in detail with reference to the drawings, giving examples. However, the present disclosure is not limited by the following description. In addition, in the following Figs. 1 to 6, the same parts may be given the same reference numerals, and their description may be omitted. In addition, in the drawings, for convenience of explanation, the structure of each part may be appropriately simplified and the dimensional ratio of each part may be shown schematically, different from the actual one. In addition, each embodiment and description can be combined with each other, unless otherwise specified, and the description can be used. In addition, when the expression "~" is used in this specification, it is used to mean including the numerical value or physical value before and after it. In addition, in this specification, the expression "A and / or B" includes "A only", "B only", and "both A and B".

[0016] (Embodiment 1) The first embodiment relates to an information processing device and an information processing method of the present disclosure.

[0017] The present embodiment is an example of an information processing device and an information processing method of the present disclosure. Fig. 1 is a schematic diagram of an information processing device 10 of the first embodiment. As shown in Fig. 1, the information processing device 10 mainly includes an acquisition unit 11, an assignment unit 12, an association unit 13, a setting unit 14, and a display unit 15. In the present embodiment, the information processing device 10 is configured as a server in which the program of the present disclosure is installed.

[0018] The information processing device 10 may be incorporated in a personal computer (PC) in which the program of the present disclosure is installed, or in a server or the like as a system. The information processing device 1 of the present embodiment may also be a system composed of one or more computers or servers capable of executing the program of the present disclosure, that is, a cloud computing system. The personal computers may constitute a computer cluster. Although not shown, the information processing device 10 may be configured to be connectable to an external terminal of a system administrator via a communication line network, so that the system administrator can manage the information processing device 10 from the external terminal.

[0019] 2 illustrates a block diagram of a hardware configuration of the information processing device 10. The information processing device 10 includes, for example, a CPU (Central Processing Unit) 101, which is a computing element, a memory 102, a bus 103, a storage device 104, an input device 107, a display 108, and a communication device (communication unit) 109. Each unit of the information processing device 1 is connected to each other via the bus 103 by its respective interface (I / F).

[0020] The CPU 101 cooperates with other components through a controller (system controller, I / O controller, etc.) and controls the entire information processing device 10. In the information processing device 10, the CPU 101 executes the program 105 of the present disclosure, the skin image 106, and other programs, and also reads or writes various information. Specifically, in this embodiment, the CPU 101 functions as an acquisition unit 11, an assignment unit 12, an association unit 13, a setting unit 14, and a display unit 15. The information processing device 10 includes the CPU 101 as a calculation device (calculation element), but may include other calculation devices such as a GPU (Graphics Processing Unit) and an APU (Accelerated Processing Unit), or may include a combination of the CPU and these. Note that the CPU 101 functions, for example, as each unit (estimation unit, proposal unit, and output unit) other than the storage unit in other embodiments.

[0021] The memory 102 includes a main memory. The main memory is also called a primary storage device. When the CPU 101 performs processing, the memory 102 reads various operation programs, such as a program 105 of the present disclosure, stored in a storage device 104 (auxiliary storage device) described below. The CPU 101 then reads data from the memory 102, decodes it, and executes the program. The main memory is a RAM (random access memory). The memory 102 may further include a ROM (read only memory).

[0022] The bus 103 can also be connected to external devices. Examples of the external devices include an external storage device (external database, etc.), a printer, etc. The information processing device 10 can be connected to a communication network, for example, by a communication device 109 connected to the bus 103, and can also be connected to external devices such as an external server and a terminal via the communication network.

[0023] The storage device 104 is also called an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 104 includes an operating program including the program 105 of the present disclosure. The storage device 104 may store data including a skin image 106 and the like. The storage device 104 includes, for example, a storage medium and a drive for reading and writing data from and to the storage medium. The storage medium is not particularly limited, and may be, for example, an internal or external type, and may be, for example, a hard disk (HD), a floppy disk (FD), a CD-ROM, a CD-R, a CD-RW, an MO, a DVD, a flash memory, a memory card, and the like, and the drive is not particularly limited. The storage device 104 may be, for example, a hard disk drive (HDD) in which the storage medium and the drive are integrated.

[0024] The information processing device 10 further includes an input device 107 and a display 108 which is an output device. The input device 107 may be, for example, a pointing device such as a touch panel, a track pad, or a mouse; a keyboard; an imaging means such as a camera or a scanner; a card reader such as an IC card reader or a magnetic card reader; or a voice input means such as a microphone. The display 108 may be, for example, a display device such as an LED (light emitting diode) display or a liquid crystal display. In this embodiment, the input device 107 and the display 108 are configured separately, but the input device 107 and the display 108 may be configured as an integrated device, such as a touch panel display. In addition, in the information processing device 1, the input device 107 and the display 108 are of any configuration, and may be omitted or may include one or more of them.

[0025] Next, an example of processing in the information processing device 10 of this embodiment will be described with reference to the flowchart in Fig. 3 and the schematic diagram of processing of the skin image 106 in Fig. 4, taking as an example a case where the skin image 106 is stored in advance in the storage device 104. Note that Fig. 4 describes an example where a weight is placed on the skin of the target and a pressing process is performed on the skin of the target, but any pressure process can be adopted for the pressure process on the skin of the target. As shown in Fig. 3, the information processing method and program of the first embodiment performs each process of S1 (acquiring a skin image), S2 (adding feature points), S3 (associating feature points), S4 (setting a distortion index value), and S5 (displaying a skin image) in this order.

[0026] In step S1, the acquisition unit 11 acquires a plurality of (n, n is an integer equal to or greater than 2) skin images 106 (S1). Specifically, the acquisition unit 11 acquires skin images 106 stored in the storage device 104 of the information processing device 10. The number of skin images 106 acquired by the acquisition unit 11 may be a plurality of images, for example, 2 to 100 images, 2 to 50 images, 2 to 20 images, or 2 to 10 images.

[0027] The skin image 106 is an image obtained by capturing pressure treatment applied to the skin of a subject over time. The pressure treatment is, for example, a pressing (pressurizing) treatment applied to the skin, or a suction or depressurization treatment applied to the skin. The pressing treatment applied to the skin can be performed, for example, by placing a weight on the skin of the subject; pressing the skin of the subject with an instrument such as a stick; or the like. The suction or depressurization treatment applied to the skin can be performed, for example, by sucking the skin of the subject with an aspirator; or the like.

[0028] The multiple skin images 106 include an image captured before the start of pressure processing on the skin of the target, and an image captured after the start of pressure processing on the skin of the target. The multiple skin images 106 include an image captured before the start of pressure processing on the skin of the target, and by using the image as a reference skin image described later, a state in which the skin of the target has little distortion can be used as a reference, so that an index value of the distortion of the skin tissue of the target caused by the pressure processing can be calculated and set more accurately. The skin image 106 can be obtained, for example, by capturing an image of the location of the skin of the target and / or its surroundings where the pressure processing is performed on the skin of the target using an imaging device such as a camera from before the start of pressure processing on the skin of the target to any timing after the start of the pressure processing. The multiple skin images 106 are preferably, for example, skin images 106 captured so as to include the same region of the skin of the target. Such multiple skin images 106 can be obtained, for example, by fixing the position of the skin of the target and the position of the imaging device by a fixing means or the like, and capturing images with the imaging device. This allows the information processing device 10 to suitably set an index value for the distortion of the skin tissue of the subject caused by the pressure processing. Furthermore, by including, for example, a portion of the skin of the subject where the pressure processing is performed, the portion where the pressure processing is performed can be used as a reference point (reference coordinates) when processing the skin image 106, and each process can be performed. The imaging device may capture a moving image. In the case of the moving image, the frame rate of the moving image is, for example, 10 to 500 fps (frames per second), or 100 to 300 fps.

[0029] Next, in step S2, the assigning unit 12 assigns feature points to each of the n skin images 106 (S2). In the n skin images 106, the same part of the skin has similar feature changes between the skin images 106. Therefore, in step S2, the feature changes of the unprocessed skin images 106 (P1 to Pn) shown in FIG. 4(A) are detected using a corner detection method such as the Shi-Tomasi corner detection method, SIFT (Scale-Invariant Feature Transform), KAZE algorithm such as AKAZE (Accelerated KAZE), etc., and feature points F are assigned to the parts (pixels) having the feature changes as shown in FIG. 4(B). The corner detection method, KAZE algorithm, etc. can be implemented using, for example, a program registered in OpenCV. The frequency (density) of the feature points can be set, for example, according to the detection accuracy of the distortion of the skin of the subject. In the information processing device 10 of embodiment 1, for example, in step S2, if the frequency (density) of the feature points to be added is increased, the number of feature points per area of ​​the subject's skin increases relatively, allowing the distortion of the subject's skin tissue to be confirmed in more detail.

[0030] Next, in step S3, the matching unit 13 matches the same feature points in each skin image 106. That is, in step S3, the feature points between each skin image 106 are compared, and the same feature point F is matched (associated) as the same feature point (F1 to Fm, m is an integer of 2 or more) as shown in FIG. 4(C). Specifically, in this embodiment, when a pressing process is performed on the skin of the target, the skin of the target moves over time toward the pressed location according to the distance from the pressed location. Therefore, in step S3, the movement of the feature point assigned in step S2 is recognized, and the movement of the feature point is tracked, so that the same feature point is identified and linked between the skin image 106 having the tracked feature point and the next captured skin image 106 (between Pk and Pk+1, k is an integer of 1 to n-1). Note that when the pressure process is a decompression process or a suction process, the same movement of the skin of the target also occurs. In step S3, the same process is performed on the other skin images 106 to establish correspondence between feature points (F1 to Fn) between all of the skin images 106 (P1 to Pn). The correspondence between feature points (F1 to Fn) in step S3 can be performed by an optical flow estimation method such as the Lucas-Kanade method, optical flow, the Farneback method, or the Mean-Shift method. The optical flow estimation method can be performed by using, for example, a program registered in OpenCV.

[0031] Next, in step S4, the setting unit 14 assigns an index value of the distortion of the skin tissue caused by the pressure processing to each of the feature points (F1 to Fn) between the multiple skin images 106 set in step S3. When the pressing process is performed on the skin of the subject, the skin of the subject moves toward the pressing point, starting from the point closest to the pressing point. Therefore, when comparing the distance between two feature points in any two skin images 106, if any one feature point moves by the pressing process, a difference occurs in the distance between the two feature points between the two skin images 106. In other words, when the pressing process on the skin of the subject causes distortion in the skin tissue at the pressing point, the distortion of the skin tissue at the pressing point propagates to the surrounding skin tissue. Therefore, distortion also occurs in the skin tissue around the pressing point, and the feature point present on the skin tissue where the distortion occurs moves when other feature points are used as a reference. Therefore, in step S4, the distance of the reference feature points is set, and the change in the distance relative to the distance is calculated as an index value of the distortion of the skin tissue, and set (assigned) to the feature points. Specifically, in step S4, a reference skin image 106 is first set in order to calculate the distance between the reference feature points. Then, in step S4, the distance between the feature points in the reference skin image 106 is used as an index to calculate the change (amount of movement) in the distance between the same feature points in other skin images 106, and the obtained distance is used as the index value of the distortion of the skin tissue. Note that the above-mentioned propagation of the distortion of the skin tissue is similar in the case of a decompression process or a suction process, and can be applied to any pressure process.

[0032] More specifically, in step S4, a reference skin image 106 (Ps) is set as a reference for the distance between feature points from a plurality of skin images 106 (P1 to Pn). The reference skin image 106 (Ps) (s is an integer of 1 or more and n-1 or less) can be any skin image 106 from the plurality of skin images 106, but is preferably a skin image 106 captured before the start of the pressure treatment, and more preferably a skin image 106 captured before the start of the pressure treatment and is the skin image 106 (P1) captured earliest among the skin images 106, since this allows for more accurate detection of the distortion of the skin tissue. Next, in step S4, the distance between feature points is calculated in the reference skin image 106 (Ps). Specifically, in step S4, a feature point (first feature point F1) for detecting the distortion of the skin tissue and a second feature point (F2) for use in calculating the distortion index are set in the reference skin image 106 (Ps), and the distance (Ls) between the first feature point F1 and the second feature point F2 is calculated. Next, in step S4, the distance (Lt) between the corresponding first feature point (F1) and second feature point (F2) of the target skin images 106 (PS+1 to Pn) captured after the reference skin image 106 (Ps) is calculated in the same manner. The number of target skin images 106 (PS+1 to Pn) may be one, multiple, or all. Then, in step S4, the magnitude of the difference in distance between the first feature point (F1) and the second feature point (F2) in the reference skin image 106 (Ps) and the target skin image 106 (PS+1 to Pn) (absolute value of the difference, |Lt-Ls|) is set as an index value of the distortion of the skin tissue caused by the pressure processing of the first feature point F1 in the target skin image 106 (PS+1 to Pn). Furthermore, in step S4, a feature point other than F1 (e.g., F2 to Fn) is reset as the first feature point in the reference skin image 106 (Ps), and index values ​​of the distortion of the skin tissue in each of the other feature points (F2 to Fn) are set in a similar manner.

[0033] Then, in step S5, the display unit 15 displays the feature points of the target skin image and the distortion index value of each feature point on the target skin image (PS+1 to Pn) on the display 108. At this time, as shown in FIG. 4(D), in step S5, the display of each feature point may be changed based on the distortion index value of each feature point. Specifically, as shown in FIG. 4(D), in step S4, the numerical range of the distortion index value of the first feature point is linked to the display change (degree of grayscale shading in FIG. 4(D)), and when displayed on the display 108, the display of the first feature point is changed in accordance with the display change. The display change may be, for example, the display of the color, size, brightness, etc. of the feature point.

[0034] Then, the information processing method of the first embodiment ends the process.

[0035] (Effects of the First Embodiment) In the information processing device 10 of the first embodiment, after adding feature points to a plurality of skin images 106, the feature points are associated between the skin images 106. Then, in the information processing device 10 of the first embodiment, the movement of the first feature point during the pressure processing is detected by calculating the magnitude of the difference in distance between the same first feature point and the same second feature point in two skin images 106 using information on the associated feature points. The information processing device 10 of the first embodiment can thus indirectly detect the distortion of the skin tissue caused by the pressure processing on the skin of the subject at the first feature point. Also, when pressure is applied to the skin of the subject, if the elasticity of the skin of the subject is relatively high, the skin tissue of the subject can absorb the pressure, so that the distortion of the skin is relatively suppressed and the movement of the skin of the subject is relatively reduced. On the other hand, when pressure is applied to the skin of the subject, if the elasticity of the skin of the subject is relatively low, the skin tissue of the subject cannot absorb the pressure, so that the distortion of the skin is relatively large and the movement of the skin of the subject is relatively large. Therefore, the information processing device 10 of the first embodiment can also estimate the elasticity of the skin of the subject through, for example, an index value of the distortion of the skin tissue caused by the pressure processing. Also, since the elasticity of the skin correlates with the tendency of the skin to sag, the information processing device 10 of the first embodiment can also estimate the tendency of the skin to sag through, for example, an index value of the distortion of the skin tissue caused by the pressure processing.

[0036] The information processing device 10 of the first embodiment includes the display unit 15, but the display unit 15 has an arbitrary configuration and may or may not be present.

[0037] In the information processing device 10 of the first embodiment, the second feature point is one for one first feature point F1, but the present disclosure is not limited thereto. The second feature point may be one, multiple, or all feature points other than the first feature point F1 (e.g., F2 to Fn). In the information processing device 10 of the first embodiment, by increasing the number of second feature points, even when a distortion of the skin tissue not caused by the pressure processing occurs, the reflection of the distortion of the skin tissue in the index value can be suppressed, so that an index value that more accurately reflects the distortion of the skin tissue caused by the pressure processing can be set. In the information processing device 10 of the first embodiment, when the index value of the distortion is set using a plurality of second feature points, the plurality of second feature points may be, for example, arbitrarily selected from feature points present in the skin image 106, or may be feature points (excluding the first feature point) present within a pre-specified distance from the first feature point. By setting the second feature points to feature points (excluding the first feature point) that are present within a pre-specified distance from the first feature point, calculation costs can be reduced, and even if a distortion of skin tissue not caused by the pressure processing occurs, the reflection of the distortion of the skin tissue in the index value can be suppressed, so that an index value that more accurately reflects the distortion of skin tissue caused by the pressure processing can be set. The pre-specified distance can be set according to the size of the skin in the skin image 106, for example, within 25 to 200 pixels, within 50 to 150 pixels, or within 75 to 125 pixels based on the first feature point. In the information processing device 10 of the first embodiment, when the index value of the distortion is set using the second feature points, the maximum value of the differences in the distances, the average value of the differences in the distances, the median value of the differences in the distances, the sum value of the distances, or the like can be set as the index value of the distortion.

[0038] Furthermore, in the information processing device 10 of the first embodiment, feature points included in a plurality or all of the skin images 106 (Ps to Pn) among the reference skin image 106 (Ps) and the skin images 106 (PS+1 to Pn) captured after the reference skin image 106 (Ps) may be set as the first feature points and / or the second feature points. In this way, the information processing device 10 of the first embodiment may perform calculations only for feature points that can be tracked in a plurality or all of the skin images 106, for example, thereby reducing calculation costs. Furthermore, the information processing device of the first embodiment may suppress or nullify the influence of distortion caused by feature points whose presence or absence changes due to the influence of changes in the imaging area such as camera shake when capturing the skin images 106, by tracking only feature points that exist in the plurality of skin images 106, for example.

[0039] (Embodiment 2) This embodiment is another example of the information processing device of the present disclosure. FIG. 5 is a block diagram showing an information processing system 100 including an information processing device 10A of this embodiment. As shown in FIG. 5, the information processing device of this embodiment includes an information processing device 10A and a terminal 20. The information processing device 10A and the terminal 20 are configured to be mutually connectable via a communication line network 30 outside the system. The information processing device 10A of this embodiment includes an estimation unit 16, a proposal unit 17, and an output unit 18 in addition to the configuration of the information processing device 10 of the first embodiment. The hardware configuration of the information processing device 10A is the same as that of the information processing device 10 of FIG. 2, except that the CPU 101 has the configuration of the information processing device 10A of FIG. 5 instead of the configuration of the information processing device 10 of FIG. 1. Except for these points, the configuration of the information processing device 10A of the second embodiment is the same as the configuration of the information processing device 10 of the first embodiment, and the description thereof can be used. In this embodiment, the information processing device 10A and the terminal 20 included in the information processing system 100 may each be one or more. In this embodiment, the information processing device 10A is configured as a server in which the program of the present disclosure is installed.

[0040] The terminal 20 may be a personal computer (PC); a mobile phone, a smartphone, a tablet, Mobile terminals such as reddit terminals; smart watches, smart glasses, wearable terminals; etc. The hardware configuration of the terminal 20 is similar to, for example, the configuration of the information processing device 10, and the description thereof can be used.

[0041] The communication network (network) 30 is not particularly limited and may be a known network, for example, a wired or wireless network. Examples of the communication network 30 include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), and WiFi (Wireless Fidelity).

[0042] Next, an example of processing of the information processing system 100 including the information processing device 10A of the present embodiment will be described with reference to the flowchart of Fig. 6. As shown in Fig. 6, the information processing method and program of the second embodiment includes S6 (estimation of skin elasticity), S7 (suggestion of external skin preparation), and S8 (output to terminal) in addition to steps S1 to S5 of the information processing method of the first embodiment. Note that, in the information processing method of the second embodiment, S5 to S7 are performed in this order, but S5 to S7 may be performed after S1 to S4 and before S8, and the order of their execution can be set arbitrarily.

[0043] First, the information processing system 100 of this embodiment performs steps S1 to S5 in the same manner as the information processing device 10 of the first embodiment.

[0044] Next, in step S6, the estimation unit 16 estimates the sagging tendency or elasticity of the subject's skin based on the strain index value. As described above, the strain index value correlates with the elasticity and sagging tendency of the subject's skin. For this reason, in step S6, for example, the strain index value, the elasticity and / or the sagging tendency of the skin are estimated based on the correlation between the strain index value and the elasticity and / or the sagging tendency of the skin. The elasticity and / or the sagging tendency of the skin can be evaluated, for example, by Cutometer CT580 (manufactured by Integral Co., Ltd.).

[0045] Next, in step S7, a skin topical agent for the subject is proposed based on the index value of the distortion. As described above, the index value of the distortion correlates with the elasticity and sagging tendency of the skin of the subject. In addition, there are various skin topical agents for recovering the elasticity and sagging of the skin according to the elasticity and sagging tendency of the skin of the subject. Therefore, by presetting the skin topical agent to be proposed according to the index value of the distortion or the numerical range of the index value, in step S7, a skin topical agent suitable for recovering the elasticity and sagging of the skin of the subject can be proposed based on the index value of the distortion of the subject.

[0046] Then, in step S8, the feature points of the target skin image obtained in step S4, the distortion index value of each feature point, the sagging tendency or elasticity of the target skin obtained in step S6, and the suggested product obtained in step S7 are output to terminal 20 via communication network 30. Then, terminal 20 displays the output image and data on a display unit such as a display of terminal 20.

[0047] Then, the information processing system of the second embodiment ends the process.

[0048] (Effects of the second embodiment) The information processing system 100 of the second embodiment can estimate the elasticity and sagging tendency of the subject's skin by using the index value of the distortion of the subject's skin, and can also suggest an appropriate topical skin preparation accordingly. Therefore, the information processing system 100 of the second embodiment can grasp the condition of the subject's skin and suggest an appropriate topical skin preparation according to the condition of the subject's skin.

[0049] In the information processing system 100 of the second embodiment, the information processing device 10A includes both the estimation unit 16 and the proposal unit 17, but may include only one of them. Moreover, the information processing device 10A may not include, for example, the output unit 18. Moreover, the information processing system 100 of the second embodiment may use the information processing device 10 instead of the information processing device 10A. In this case, for example, the description of the information processing device 10 of the first embodiment can be used for the information processing device 10.

[0050] In the information processing system 100 of the second embodiment, a plurality of skin images 106 stored in the information processing device 10A of the second embodiment are used, but the present disclosure is not limited to this, and may be implemented, for example, by capturing a plurality of skin images 106 by the terminal 20. In this case, the terminal 20 transmits the plurality of skin images 106 captured by the terminal 20 to the information processing device 10A, and the acquisition unit 11 of the information processing device 10A acquires the received plurality of skin images 106 and performs processing.

[0051] (Embodiment 3) The program of this embodiment is a program for making a computer execute the information processing method of the present disclosure. In the program of this embodiment, the "processing" can also be referred to as, for example, a "procedure" or an "instruction". The program of this embodiment may be recorded, for example, in a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include a random access memory (RAM), a read-only memory (ROM), a hard disk (HD), a solid state drive (SSD), an optical disk, and a floppy disk (FD).

[0052] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0053] <Additional Notes> Some or all of the above-described embodiments and examples may be described as follows, but are not limited to the following: <Information processing method> (Appendix 1) An acquisition step of acquiring a plurality of skin images; the plurality of skin images being images of a pressure treatment applied to the skin of a subject over time; an assignment step of assigning feature points to each skin image; a matching step of matching the same feature points in each skin image; a setting step of calculating a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting a magnitude of the difference in distance as an index value of a distortion of skin tissue caused by the pressure treatment at the first feature point of the target skin image. An information processing method carried out on a computer. (Appendix 2) The information processing method described in Appendix 1, wherein in the setting process, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, the distance difference is calculated, and the magnitude of the maximum distance difference among the obtained distance differences is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Appendix 3) The information processing method described in Appendix 1 or 2, wherein in the setting process, a feature point other than the first feature point that exists within a predetermined distance from the first feature point of the reference skin image is set as a second feature point. (Appendix 4) The information processing method described in Appendix 1 or 2, wherein in the setting process, all feature points other than the first feature point that are present within a predetermined distance from the first feature point of the reference skin image are set as second feature points. (Appendix 5) The information processing method according to any one of appendices 1 to 4, wherein in the setting step, each feature point of the reference skin image is set as a first feature point. (Appendix 6) An information processing method described in any one of Appendices 1 to 5, wherein in the setting process, a feature point contained in a plurality of skin images, including the reference skin image and a skin image captured after the reference skin image, is set as the first feature point. (Appendix 7) An information processing method described in any one of Appendices 1 to 5, wherein in the setting process, a feature point contained in all skin images, including the reference skin image and skin images captured after the reference skin image, is set as the first feature point. (Appendix 8) An information processing method according to any one of appendices 1 to 5, further comprising a display step of displaying, on the target skin image, feature points of the target skin image and an index value of distortion for each feature point. (Appendix 9) 9. The information processing method according to claim 8, wherein the display step changes the display of each feature point based on the distortion index value of each feature point. (Appendix 10) The information processing method according to any one of appendices 1 to 9, wherein the reference skin image is a skin image captured before the pressure treatment is started. (Appendix 11) 11. The information processing method according to any one of claims 1 to 10, wherein the magnitude of the difference in distance is an absolute value of the difference in distance. (Appendix 12) 12. The information processing method according to any one of appendices 1 to 11, wherein the pressure processing applied to the skin of the subject is a pressing processing applied to the skin of the subject. (Appendix 13) 13. An information processing method according to any one of appendices 1 to 12, comprising an estimation step of estimating the tendency for the subject's skin to sag or elasticity based on the distortion index value. (Appendix 14) An information processing method according to any one of appendices 1 to 13, further comprising a suggestion step of suggesting a topical skin preparation for the subject based on the index value of the distortion. <Information processing device> (Appendix 15) an acquisition unit that acquires a plurality of skin images; The plurality of skin images are images obtained by capturing pressure treatment of a subject's skin over time; an adding unit that adds feature points to each skin image; a matching unit for matching the same feature points in each skin image; a setting unit that calculates a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and the second feature point of a target skin image captured after the reference skin image, and sets a magnitude of the difference in distance as an index value of a distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image. Information processing device. (Appendix 16) The information processing device described in Appendix 15, wherein the setting unit calculates the distance difference by setting a plurality of feature points other than the first feature point of the reference skin image as second feature points, and sets the magnitude of the maximum distance difference among the obtained distance differences as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Appendix 17) The information processing device according to claim 15 or 16, wherein the setting unit sets a feature point other than the first feature point that exists within a predetermined distance from the first feature point of the reference skin image as a second feature point. (Appendix 18) The information processing device according to claim 15 or 16, wherein the setting unit sets all feature points other than the first feature point that are present within a predetermined distance from the first feature point of the reference skin image as second feature points. (Appendix 19) 19. The information processing device according to any one of appendices 15 to 18, wherein the setting unit sets each feature point of the reference skin image as a first feature point. (Appendix 20) The information processing device according to any one of appendices 15 to 19, wherein the setting unit sets a feature point contained in a plurality of skin images, including the reference skin image and a skin image captured after the reference skin image, as the first feature point. (Appendix 21) The information processing device described in any one of Appendices 15 to 20, wherein the setting unit sets a feature point contained in all skin images, including the reference skin image and skin images captured after the reference skin image, as the first feature point. (Appendix 22) 22. The information processing device according to any one of appendices 15 to 21, further comprising a display unit that displays, on the target skin image, feature points of the target skin image and an index value of distortion for each feature point. (Appendix 23) 23. The information processing device of claim 22, wherein the display unit changes the display of each feature point based on the distortion index value of each feature point. (Appendix 24) 26. The information processing device of any one of appendices 15 to 25, wherein the reference skin image is a skin image captured before the start of the pressure treatment. (Appendix 25) 25. The information processing device according to any one of claims 1 to 24, wherein the magnitude of the difference in distance is an absolute value of the difference in distance. (Appendix 26) 26. The information processing device according to any one of appendices 15 to 25, wherein the pressure processing on the skin of the target is a pressing processing on the skin of the target. (Appendix 27) 27. The information processing device according to any one of appendix 15 to 26, further comprising an estimation unit that estimates a tendency for the skin to sag or elasticity of the subject based on the index value of the distortion. (Appendix 28) 28. An information processing method according to any one of appendices 15 to 27, further comprising a suggestion unit that suggests a topical skin preparation for the target based on the index value of the distortion. <Program> (Appendix 29) On the computer, An acquisition process for acquiring a plurality of skin images, The plurality of skin images are images obtained by applying pressure to the skin of a subject over time; an assignment process for assigning feature points to each skin image; a matching process for matching the same feature points in each skin image; a setting process for calculating a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance at the first feature point of the target skin image as an index value of the distortion of skin tissue caused by the pressure processing. (Appendix 30) The program described in Appendix 29, wherein in the setting process, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, the distance difference is calculated, and the magnitude of the maximum distance difference among the obtained distance differences is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Appendix 31) The program described in Appendix 29 or 30, wherein in the setting process, a feature point other than the first feature point that exists within a predetermined distance from the first feature point of the reference skin image is set as a second feature point. (Appendix 32) The program described in Appendix 29 or 30, wherein in the setting process, all feature points other than the first feature point that are present within a predetermined distance from the first feature point of the reference skin image are set as second feature points. (Appendix 33) The program according to any one of appendices 29 to 32, wherein in the setting process, each feature point of the reference skin image is set as a first feature point. (Appendix 34) A program described in any one of Appendices 29 to 33, wherein in the setting process, a feature point contained in a plurality of skin images, including the reference skin image and a skin image captured after the reference skin image, is set as the first feature point. (Appendix 35) A program described in any one of Appendices 29 to 34, wherein in the setting process, a feature point contained in all skin images, including the reference skin image and skin images captured after the reference skin image, is set as the first feature point. (Appendix 36) A program described in any one of appendices 29 to 35, comprising a display process for displaying feature points of the target skin image and a distortion index value for each feature point on the target skin image. (Appendix 37) 37. The program according to claim 36, wherein the display process changes the display of each feature point based on a distortion index value of each feature point. (Appendix 38) 38. The program of any one of appendices 29 to 37, wherein the reference skin image is a skin image captured before the pressure treatment is started. (Appendix 39) 39. The program of any one of appendixes 29 to 38, wherein the magnitude of the distance difference is an absolute value of the distance difference. (Appendix 40) 40. The program described in any one of Appendices 29 to 39, wherein the pressure treatment to the skin of the subject is a pressing treatment to the skin of the subject. (Appendix 41) 41. The program of any one of appendices 29 to 40, further comprising an estimation process for estimating the sagging or elasticity of the subject's skin based on the index value of the distortion. (Appendix 42) 42. A program described in any one of appendices 29 to 41, comprising a suggestion process for suggesting a topical skin preparation for the target based on the index value of the distortion. <Recording Media> (Appendix 43) A computer-readable recording medium having a program according to any one of appendices 29 to 42 recorded thereon. <System> (Appendix 44) A terminal and a server are included, The terminal and the server can be connected via a communication network, The server includes an information processing device according to any one of Supplementary Notes 15 to 28; The server sets an index value for distortion of skin tissue from a plurality of skin images transmitted from the terminal. [Explanation of symbols]

[0054] 10. Information processing device 11 Acquisition Department 12 Granting Department 13 Mapping section 14 Setting section 15 Display section 101 CPU 102 Memory 103 Bus 104 Storage device 105 Programs 106 Skin images 107 Input Devices 108 Display 109 Communication Devices

Claims

1. An acquisition step of acquiring a plurality of skin images; the plurality of skin images being images of a pressure treatment applied to the skin of a subject over time; an assignment step of assigning feature points to each skin image; a matching step of matching the same feature points in each skin image; a setting step of calculating a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting a magnitude of the difference in distance as an index value of a distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image, In the setting step, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, and the distance difference is calculated. The magnitude of the maximum distance difference among the obtained distance differences is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. An information processing method carried out on a computer.

2. The information processing method according to claim 1 , wherein in the setting step, a feature point other than the first feature point that exists within a pre-specified distance from the first feature point of the reference skin image is set as a second feature point.

3. An information processing method as described in claim 1 or 2, wherein in the setting process, a feature point contained in all skin images, including the reference skin image and skin images captured after the reference skin image, is set as the first feature point.

4. The information processing method according to claim 1 , wherein in the setting step, each feature point of the reference skin image is set as a first feature point.

5. The information processing method according to claim 1 , wherein in the setting step, feature points included in a plurality of skin images, among the reference skin image and a skin image captured after the reference skin image, are set as the first feature points.

6. The information processing method according to claim 1 , further comprising the step of displaying, on the target skin image, feature points of the target skin image and an index value of distortion for each feature point.

7. The information processing method according to claim 6 , wherein in the display step, display of each feature point is changed based on an index value of distortion of each feature point.

8. The information processing method according to claim 1 , wherein the reference skin image is a skin image captured before the pressure treatment is started.

9. The information processing method according to claim 1 , wherein the magnitude of the difference in distance is an absolute value of the difference in distance.

10. The information processing method according to claim 1 , wherein the pressure processing on the skin of the target is a pressing processing on the skin of the target.

11. The information processing method according to claim 1 , further comprising an estimation step of estimating a tendency for the subject's skin to sag or elasticity based on the index value of the distortion.

12. The information processing method according to claim 1 , further comprising a suggestion step of suggesting an external skin preparation for the subject based on the index value of the distortion.

13. an acquisition unit that acquires a plurality of skin images; The plurality of skin images are images obtained by capturing pressure treatment of a subject's skin over time; an adding unit that adds feature points to each skin image; a matching unit for matching the same feature points in each skin image; a setting unit that calculates a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between a corresponding first feature point and a second feature point of a target skin image captured after the reference skin image, and sets a magnitude of the difference in distance as an index value of a distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image, the setting unit calculates the distance difference by setting a plurality of feature points other than the first feature point of the reference skin image as second feature points, and sets the magnitude of the maximum distance difference among the obtained distance differences as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. Information processing device.

14. On the computer, An acquisition process for acquiring a plurality of skin images, The plurality of skin images are images obtained by applying pressure to the skin of a subject over time; an assignment process for assigning feature points to each skin image; a matching process for matching the same feature points in each skin image; a setting process for calculating a difference between a distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from a plurality of skin images, and a distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting a magnitude of the difference in distance as an index value of a distortion of skin tissue caused by the pressure processing at the first feature point of the target skin image; In the setting process, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, and the distance difference is calculated. The magnitude of the maximum distance difference among the obtained distance differences is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. program.

15. A terminal and a server are included, The terminal and the server can be connected via a communication network, The server includes the information processing device according to claim 13 , The server sets an index value for distortion of skin tissue from a plurality of skin images transmitted from the terminal.

Images