Information processing method, information processing device, program, and information processing system

Abstract

For example, the present invention provides an information processing method, information processing device, and program for visualizing skin elasticity and / or skin strain. [Solution] The information processing method performed on a computer according to the present disclosure includes an acquisition step of acquiring a plurality of skin images, wherein the plurality of skin images are images captured over time of pressure processing on target skin; an assignment step of assigning feature points to each skin image; an assignment step of associating the same feature points in each skin image; and a setting step of calculating the difference between the distance between a first feature point of a reference skin image set from the plurality of skin images and a second feature point different from the first feature point, and the distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image.

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing apparatus, a program, and an information processing system.

Background Art

[0002] As a method for measuring morphological characteristics of the skin, a method for visualizing or quantifying texture and wrinkles from a skin surface image captured by a camera is widely used. Further, as a method for measuring physical properties of the skin, measurement of elasticity by an elastometer, measurement of plasticity, etc. are generally known (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the measurement of elasticity by the elastometer and the measurement of plasticity have a problem that a dedicated device for skin measurement is required.

[0005] Therefore, an object of the present disclosure is to provide a new information processing method that can be used for visualizing, for example, skin elasticity and / or skin distortion.

Means for Solving the Problems

[0006] In order to achieve the above object, an information processing method (hereinafter, also referred to as "information processing method") executed on a computer of the present disclosure is an acquisition step of acquiring a plurality of skin images, wherein the plurality of skin images are images obtained by imaging pressure treatment on a target skin over time, an assignment step of assigning feature points to each skin image, Each skin image is associated with the same feature points in a matching process, The method includes a setting step of calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0007] The information processing device disclosed herein is an acquisition unit that acquires multiple skin images, The aforementioned multiple skin images include a portion that is an image captured over time of pressure treatment applied to the target skin, A unit that assigns feature points to each skin image, Each skin image has a matching section that associates the same feature points, The system includes a setting unit that calculates the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and sets the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0008] The program disclosed herein is installed on a computer. This is an acquisition process that obtains multiple skin images. The aforementioned multiple skin images are images captured over time as pressure is applied to the target skin, and the process involves this. A process to assign feature points to each skin image, A matching process is performed to associate the same feature points in each skin image, The system performs a setting process that calculates the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and sets the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image.

[0009] This disclosure includes the terminal and server, The aforementioned terminal and the aforementioned server are connected via a communication network. The aforementioned server includes an information processing device of the Disclosure, The server sets an index value for the distortion of skin tissue from multiple skin images transmitted from the terminal. [Effects of the Invention]

[0010] According to this disclosure, for example, skin elasticity and / or skin distortion can be visualized. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing an example of an information processing system equipped with an information processing device according to Embodiment 1. [Figure 2] Figure 2 is a block diagram showing the hardware configuration of the information processing device in Embodiment 1. [Figure 3] Figure 3 is a flowchart showing the information processing method and program in Embodiment 1. [Figure 4] Figure 4 is a schematic diagram showing an example of skin image processing according to Embodiment 1. [Figure 5] Figure 5 is a block diagram showing an example of an information processing system equipped with an information processing device according to Embodiment 2. [Figure 6] Figure 6 is a flowchart showing the information processing method in Embodiment 2. [Modes for carrying out the invention]

[0012] <Definition> In this specification, "skin tissue" means the tissue of the skin composed of the epidermis, dermis, and subcutaneous tissue. The epidermis is the outermost tissue of the skin and means the tissue including the stratum corneum and the germinative layer. The dermis is the main tissue of the skin that constitutes the layer between the epidermis and the subcutaneous tissue and means the tissue rich in collagen. The subcutaneous tissue is the innermost tissue in the skin tissue and means the tissue that supports the epidermis and the dermis.

[0013] In this specification, "skin image" is an image obtained by imaging skin tissue. Preferably, the skin image is an image obtained by imaging the epidermis from the outside of the skin tissue.

[0014] In this specification, “topical skin preparation” means a preparation used for application to the skin. The topical skin preparation is preferably a preparation applied to the skin surface of the body, such as the face or body. Depending on the form of use, the topical skin preparation may take the form of an ampoule, capsule, powder, granules, liquid, gel, foam, emulsion, sheet, mist, spray, etc. Examples of the forms of use include pharmaceuticals; quasi-drugs; topical or systemic topical skin preparations; medicinal and / or cosmetic preparations applied to the scalp and hair; bath preparations used by adding them to bathwater; other preparations; etc. Examples of topical or systemic skin preparations include basic cosmetics such as lotions, emulsions, creams, ointments, oils, and packs; facial cleansers or skin cleansers such as bar soaps, liquid soaps, and hand washes; massage agents, cleansing agents, depilatory agents, hair removal agents, shaving agents, aftershave lotions, preshave lotions, shaving creams; makeup cosmetics such as foundations, lipsticks, blushes, eyeshadows, eyeliners, and mascaras; perfumes; nail care products, nail polish, nail polish removers; poultices, plasters, tapes, sheets, patches, aerosols; and mouthwashes and other gargles. Examples of medicinal and / or cosmetic preparations applied to the scalp and hair include shampoos, conditioners, hair treatments, pre-hair treatments, permanent solutions, hair dyes, hair styling products, hair tonics, hair growth and nourishing products, poultices, plasters, tapes, sheets, aerosols, etc. Examples of other preparations include deodorants or antiperspirants, antiperspirants, sanitary products, sanitary cotton, wet wipes, etc.

[0015] Hereinafter, the present disclosure will be described in detail with reference to the drawings by way of examples. However, the present disclosure is not limited by the following description. In the following FIGS. 1 to 6, the same parts may be denoted by the same reference numerals and the description thereof may be omitted. In the drawings, for convenience of explanation, the structure of each part may be appropriately simplified and shown, and the dimensional ratios and the like of each part may be different from the actual ones and may be shown schematically. Also, each embodiment and the description can be combined with each other and the description thereof can be incorporated unless otherwise specified. In this specification, when the expression "~" is used, it is used in the sense of including the numerical values or physical values before and after it. Also, in this specification, the expression "A and / or B" includes "only A", "only B", and "both A and B".

[0016] (Embodiment 1) Embodiment 1 relates to an information processing apparatus and an information processing method of the present disclosure.

[0017] This embodiment is an example of the information processing apparatus and the information processing method of the present disclosure. FIG. 1 is a schematic diagram of the information processing apparatus 10 of Embodiment 1. As shown in FIG. 1, the information processing apparatus 10 mainly includes an acquisition unit 11, an assignment unit 12, a correspondence unit 13, a setting unit 14, and a display unit 15. In this embodiment, the information processing apparatus 10 is configured as a server in which the program of the present disclosure is installed.

[0018] The information processing apparatus 10 may be incorporated into a server or the like as a personal computer (PC) or a system in which the program of the present disclosure is installed. Also, the information processing apparatus 1 of this embodiment may be a system composed of one or more computers or servers capable of executing the program of the present disclosure, that is, a cloud computing system. The personal computer may form a computer cluster. Although not shown, the information processing apparatus 10 may be configured to be connectable to an external terminal of a system administrator via a communication network, and the system administrator may be configured to manage the information processing apparatus 10 from the external terminal.

[0019] Figure 2 illustrates a block diagram of the hardware configuration of the information processing device 10. The information processing device 10 includes, for example, a CPU (central processing unit) 101, memory 102, bus 103, storage device 104, input device 107, display 108, and communication device (communication unit) 109, which are all arithmetic elements. Each part of the information processing device 1 is connected via the bus 103 through its respective interface (I / F).

[0020] The CPU 101 operates in conjunction with other components via controllers (system controller, I / O controller, etc.) and is responsible for the overall control of the information processing device 10. In the information processing device 10, the CPU 101 executes the program 105, skin image 106, and other programs of this disclosure, and also reads or writes various types of information. Specifically, in this embodiment, the CPU 101 functions as an acquisition unit 11, an assignment unit 12, an mapping unit 13, a setting unit 14, and a display unit 15. The information processing device 10 includes the CPU 101 as an arithmetic unit (arithmetic element), but may also include other arithmetic units such as a GPU (Graphics Processing Unit) or an APU (Accelerated Processing Unit), or a combination of the CPU and these. The CPU 101 may also function as, for example, the other units (estimation unit, proposal unit, and output unit) in other embodiments, excluding the storage unit.

[0021] Memory 102 includes main memory. This main memory is also called primary memory. When the CPU 101 performs processing, memory 102 reads various operational programs, such as the program 105 of this disclosure, which are stored in the storage device 104 (auxiliary storage device) described later. The CPU 101 then reads and decodes the data from memory 102 and executes the program. This main memory is RAM (random access memory). Memory 102 may further include ROM (read-only memory).

[0022] Bus 103 can also be connected to external devices. Examples of such external devices include external storage devices (external databases, etc.) and printers. The information processing device 10 can be connected to a communication network, for example, by a communication device 109 connected to bus 103, and can also be connected to external devices such as external servers and terminals via the communication network.

[0023] The storage device 104 is also called an auxiliary storage device, for example, in relation to the main memory (primary memory). As described above, the storage device 104 includes an operating program, including the program 105 of this disclosure. The storage device 104 may store data, including skin images 106, etc. The storage device 104 includes, for example, a storage medium and a drive for reading from and writing to the storage medium. The storage medium is not particularly limited and may be internal or external, and examples include HD (hard disk), FD (floppy disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, etc., and the drive is not particularly limited. The storage device 104 may be, for example, a hard disk drive (HDD) in which the storage medium and the drive are integrated.

[0024] The information processing device 10 further includes an input device 107 and an output device, a display 108. The input device 107 may include, for example, a pointing device such as a touch panel, trackpad, or mouse; a keyboard; imaging means such as a camera or scanner; a card reader such as an IC card reader or magnetic card reader; or an audio input means such as a microphone. The display 108 may include, for example, an LED (light-emitting diode) display or a liquid crystal display. In this embodiment, the input device 107 and the display 108 are configured separately, but the input device 107 and the display 108 may be configured as an integrated unit, such as a touch panel display. Furthermore, in the information processing device 1, the input device 107 and the display 108 are of any configuration and may be omitted, or one or more of them may be included.

[0025] Next, an example of processing in the information processing device 10 of this embodiment will be explained using the flowchart in Figure 3 and the schematic diagram of the processing of the skin image 106 in Figure 4, taking the case where the skin image 106 is stored in the storage device 104 in advance as an example. In Figure 4, the example is given of a case where a weight is placed on the target skin and pressure processing is performed on the target skin, but any pressure processing can be used for the pressure processing on the target skin. As shown in Figure 3, the information processing method and program of Embodiment 1 perform the following processes in this order: S1 (acquisition of skin image), S2 (assignment of feature points), S3 (association of feature points), S4 (setting of distortion index value), and S5 (display of skin image).

[0026] In step S1, the acquisition unit 11 acquires multiple (n, where n is an integer of 2 or more) skin images 106 (S1). Specifically, the acquisition unit 11 acquires the skin images 106 stored in the storage device 104 of the information processing device 10. The number of skin images 106 acquired by the acquisition unit 11 can be multiple, for example, 2 to 100, 2 to 50, 2 to 20, or 2 to 10.

[0027] Skin image 106 is an image captured over time of pressure treatment applied to the target skin. The pressure treatment is, for example, pressing (pressurizing) the skin or suction or depressurization of the skin. The pressing treatment can be performed, for example, by placing a weight on the target skin; pressing the target skin with an instrument such as a rod; etc. The suction or depressurization of the skin can be performed, for example, by applying suction to the target skin with a suction device; etc.

[0028] The multiple skin images 106 include images taken before the start of pressure processing on the target skin and images taken after the start of pressure processing on the target skin. The multiple skin images 106 include images taken before the start of pressure processing on the target skin, and by using these images as reference skin images as described later, a state with minimal distortion in the target skin can be used as a reference, allowing for more accurate calculation and setting of the index value of the distortion of the target skin tissue caused by the pressure processing. The skin images 106 can be obtained, for example, by using an imaging device such as a camera to image the area and / or surrounding area of ​​the target skin where pressure processing is performed, from before the start of pressure processing on the target skin to any timing after the start of pressure processing. Preferably, the multiple skin images 106 are images 106 taken so as to include the same area of ​​the target skin. Such multiple skin images 106 can be obtained, for example, by fixing the position of the target skin and the position of the imaging device with fixing means, and taking images with the imaging device. This allows the information processing device 10 to suitably set an index value for the distortion of the target skin tissue caused by the pressure processing. Furthermore, by including, for example, the area of ​​the target skin that is subjected to pressure processing in the skin image 106, the area where the pressure processing is performed can be used as a reference point (reference coordinate) during processing of the skin image 106, and each processing can be carried out accordingly. The imaging by the imaging device may also be video imaging. In the case of video, the frame rate of the video is, for example, 10 to 500 fps (frames per second) or 100 to 300 fps.

[0029] Next, in step S2, the assignment unit 12 assigns feature points to each of the n skin images 106 (S2). In the n skin images 106, the same area of ​​skin has similar characteristic changes across the skin images 106. Therefore, in step S2, the characteristic changes in the unprocessed skin images 106 (P1~Pn) shown in Figure 4(A) are detected using corner detection methods such as the Shi-Tomasi corner detection method, KAZE algorithms such as SIFT (Scale-Invariant Feature Transform) and AKAZE (Accelerated KAZE), and feature points F are assigned to the areas (pixels) having the characteristic changes, as shown in Figure 4(B). The corner detection method, KAZE algorithm, etc. can be implemented using, for example, programs registered in OpenCV. The frequency (density) of the feature points can be set, for example, according to the detection accuracy of the distortion of the target skin. In the information processing device 10 of Embodiment 1, for example, if the frequency (density) of the feature points to be assigned is increased in step S2, the number of feature points per unit area of ​​the target skin increases relatively, so that the distortion of the target skin tissue can be checked in more detail.

[0030] Next, in step S3, the matching unit 13 associates the same feature points in each skin image 106. That is, in step S3, the feature points between each skin image 106 are compared and the same feature point F is associated (linked) as the same feature point (F1 to Fm, where m is an integer of 2 or more) as shown in Figure 4(C). Specifically, in this embodiment, when a pressing process is performed on the target skin, the target skin moves over time toward the pressed area, depending on the distance from the pressed area. Therefore, in step S3, the movement of the feature points assigned in step S2 is recognized and the movement of the feature points is tracked to identify and link the same feature points between the skin image 106 having the tracked feature points and the next captured skin image 106 (between Pk and Pk+1, where k is an integer from 1 to n-1). Note that the same movement of the target skin occurs when the pressure process is a depressurization process or a suction process. Furthermore, in step S3, a similar process is performed between other skin images 106 to establish a correspondence between feature points (F1 to Fn) across all skin images 106 (P1 to Pn). The correspondence of feature points (F1 to Fn) in step S3 can be performed using optical flow estimation methods such as the Lucas-Kanade method, Optical Flow, Farneback method, or Mean-Shift method. These optical flow estimation methods can be implemented using, for example, programs registered in OpenCV.

[0031] Next, in step S4, the setting unit 14 uses the feature points (F1 to Fn) between the multiple skin images 106 set in step S3 to assign an index value to each feature point indicating the distortion of the skin tissue caused by the pressure processing. When the target skin is subjected to pressure processing, the target skin moves toward the pressure point, starting from the point closest to the pressure point. Therefore, when comparing the distance between two feature points in any two skin images 106, if either feature point moves due to the pressure processing, a difference will occur in the distance between the two feature points in the two skin images 106. In other words, when the skin tissue is distorted at the pressure point due to the pressure processing on the target skin, the distortion of the skin tissue at the pressure point propagates to the surrounding skin tissue. Therefore, distortion also occurs in the skin tissue surrounding the pressure point, and feature points located on the distorted skin tissue will move relative to other feature points. Therefore, in step S4, the distance between reference feature points is set, the variation in distance relative to the said distance is calculated as an index value of the distortion of the skin tissue, and this index value is set (assigned) to the feature points. Specifically, in step S4, first, a reference skin image 106 is set in order to calculate the distance between reference feature points. Then, in step S4, the distance between feature points in the reference skin image 106 is used as an index to calculate the change (amount of movement) in the distance between the same feature points in other skin images 106, and the obtained distance is used as an index value of the distortion of the skin tissue. The propagation of the distortion of the skin tissue described above is the same in the case of decompression processing or suction processing, and can be applied to any pressure processing.

[0032] More specifically, in step S4, a reference skin image 106(Ps) is set from multiple skin images 106(P1~Pn) to serve as the reference for the distance between feature points. The reference skin image 106(Ps) (where s is an integer between 1 and n-1) can be any of the multiple skin images 106, but preferably it is a skin image 106 captured before the start of the pressure processing, and more preferably it is a skin image 106 captured before the start of the pressure processing, and among those skin images 106, it is the earliest captured skin image 106(P1). Next, in step S4, the distance between the reference feature points is calculated in the reference skin image 106(Ps). Specifically, in step S4, a feature point (first feature point F1) for detecting distortion of the skin tissue and a second feature point (F2) for calculating the distortion index are set in the reference skin image 106(Ps), and the distance (Ls) between the first feature point F1 and the second feature point F2 is calculated. Next, in step S4, the distance (Lt) between the corresponding first feature point (F1) and second feature point (F2) of the target skin images 106(PS+1~Pn) taken after the reference skin image 106(Ps) is calculated in the same manner. The target skin images 106(PS+1~Pn) may be one image, multiple images, or all of them. Then, in step S4, the magnitude of the difference in distance between the first feature point (F1) and the second feature point (F2) in the reference skin image 106(Ps) and the target skin image 106(PS+1~Pn) (absolute value of the difference, |Lt-Ls|) is set as an index value of the distortion of the skin tissue caused by the pressure treatment of the first feature point F1 in the target skin image 106(PS+1~Pn). Furthermore, in step S4, feature points other than F1 (for example, F2~Fn) in the reference skin image 106(Ps) are reset as first feature points, and similarly, index values ​​of the distortion of the skin tissue at each of the other feature points (F2~Fn) are set.

[0033] Then, in step S5, the display unit 15 displays the feature points of the target skin image (PS+1~Pn) and the distortion index value of each feature point on the display 108. At this time, as shown in Figure 4(D), in step S5, the display of each feature point may be changed based on the distortion index value of each feature point. Specifically, as shown in Figure 4(D), in step S4, the numerical range of the distortion index value of the first feature point is linked to the display change (in Figure 4(D), the degree of grayscale intensity), and when displaying on the display 108, the display of the first feature point is changed in accordance with the display change. Examples of the display change include the display of the color, size, and brightness of the feature point.

[0034] Then, the information processing method of Embodiment 1 terminates.

[0035] (Effects of Embodiment 1) In the information processing device 10 of Embodiment 1, feature points are assigned to multiple skin images 106, and then the feature points are associated with each skin image 106. The information processing device 10 of Embodiment 1 then uses the information of the associated feature points to calculate the magnitude of the difference in distance between the same first feature point and second feature point in two skin images 106, thereby detecting the movement of the first feature point during the pressure processing. In this way, the information processing device 10 of Embodiment 1 can indirectly detect the distortion of the skin tissue caused by the pressure processing on the target skin at the first feature point. Furthermore, when pressure is applied to the target skin, if the elasticity of the target skin is relatively high, the target skin tissue can absorb the pressure, so the distortion of the skin is relatively suppressed, and the movement of the target skin is relatively reduced. On the other hand, when pressure is applied to the target skin, if the elasticity of the target skin is relatively low, the target skin tissue cannot absorb the pressure, so the distortion of the skin is relatively large, and the movement of the target skin is relatively large. Therefore, the information processing device 10 of Embodiment 1 can also estimate the elasticity of the target skin, for example, through an index value of the strain of the skin tissue caused by the pressure processing. Furthermore, since the elasticity of the skin is correlated with the tendency of the skin to sag, the information processing device 10 of Embodiment 1 can also estimate the tendency of the target skin to sag, for example, through an index value of the strain of the skin tissue caused by the pressure processing.

[0036] The information processing device 10 of Embodiment 1 includes a display unit 15, but the display unit 15 has an arbitrary configuration and may or may not be present.

[0037] In the information processing device 10 of Embodiment 1, there was one second feature point for each first feature point F1. However, the disclosure is not limited thereto, and the second feature point may be one, multiple, or all feature points other than the first feature point F1 (for example, F2 to Fn). In the information processing device 10 of Embodiment 1, by increasing the number of second feature points, even if distortion of the skin tissue occurs that is not caused by the pressure processing, the reflection of the distortion of the skin tissue in the index value can be suppressed, so that an index value that more accurately reflects the distortion of the skin tissue caused by the pressure processing can be set. In the information processing device 10 of Embodiment 1, when setting the index value of the distortion using multiple second feature points, the multiple second feature points may be, for example, arbitrarily selected from feature points present in the skin image 106, or they may be feature points (excluding the first feature point) that are within a predetermined distance from the first feature point. By setting the plurality of second feature points to feature points located within a predetermined distance from the first feature point (excluding the first feature point), computational costs can be reduced, and even if distortion of the skin tissue occurs that is not caused by the pressure processing, the reflection of the distortion of the skin tissue in the index value can be suppressed, so that an index value that more accurately reflects the distortion of the skin tissue caused by the pressure processing can be set. The predetermined distance can be set, for example, according to the size of the skin in the skin image 106, and for example, with respect to the first feature point, it can be within 25 to 200 pixels, within 50 to 150 pixels, or within 75 to 125 pixels. In the information processing device 10 of Embodiment 1, when setting the distortion index value using a plurality of second feature points, the maximum value of the difference between the multiple distances, the average value of the difference between the multiple distances, the median value of the difference between the multiple distances, the sum of the multiple distances, etc., can be set as the distortion index value.

[0038] Furthermore, in the information processing device 10 of Embodiment 1, feature points included in multiple or all of the skin images 106(Ps~Pn) among the reference skin image 106(Ps) and skin images 106(PS+1~Pn) captured after the reference skin image 106(Ps) may be set as the first feature points and / or the second feature points. This allows the information processing device 10 of Embodiment 1 to perform calculations only on feature points that can be tracked in multiple or all of the skin images 106, thereby reducing computational costs. In addition, by tracking only the feature points present in multiple skin images 106, the information processing device of Embodiment 1 can suppress or neutralize the influence of feature points whose presence or absence changes due to changes in the imaging area, such as camera shake during the imaging of the skin image 106, on the index value of distortion.

[0039] (Embodiment 2) This embodiment is another example of an information processing device of the present disclosure. Figure 5 is a block diagram showing an information processing system 100 equipped with the information processing device 10A of this embodiment. As shown in Figure 5, the information processing device of this embodiment includes the information processing device 10A and a terminal 20. The information processing device 10A and the terminal 20 are configured to be interconnected via an external communication network 30. In addition to the configuration of the information processing device 10 of Embodiment 1, the information processing device 10A of this embodiment includes an estimation unit 16, a proposal unit 17, and an output unit 18. The hardware configuration of the information processing device 10A is the same as that of the information processing device 10 of Figure 2, except that the CPU 101 has the configuration of the information processing device 10A of Figure 5 instead of the configuration of the information processing device 10 of Figure 1. Except for these points, the configuration of the information processing device 10A of Embodiment 2 is the same as the configuration of the information processing device 10 of Embodiment 1, and its description can be applied accordingly. In this embodiment, the information processing system 100 may consist of one or multiple information processing devices 10A and terminal 20. In this embodiment, the information processing device 10A is configured as a server on which the program of this disclosure is installed.

[0040] Terminal 20 includes personal computers (PCs); mobile phones, smartphones, and tablets. Mobile devices such as redline devices; smartwatches, smart glasses, wearable devices; etc. The hardware configuration of terminal 20 is similar to that of information processing device 10, for example, and the explanation for that device can be applied.

[0041] The communication network 30 is not particularly restricted and can use any publicly known network, such as a wired or wireless network. Examples of communication networks 30 include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), WiFi (Wireless Fidelity), etc.

[0042] Next, an example of the processing of the information processing system 100 equipped with the information processing device 10A of this embodiment will be explained using the flowchart in Figure 6. As shown in Figure 6, the information processing method and program of Embodiment 2 include steps S6 (estimation of skin elasticity), S7 (proposal of topical skin preparations), and S8 (output to terminal) in addition to steps S1 to S5 of the information processing method of Embodiment 1. In the information processing method of Embodiment 2, steps S5 to S7 are performed in this order, but steps S5 to S7 only need to be performed after steps S1 to S4 and before step S8, and the order of execution can be set arbitrarily.

[0043] First, the information processing system 100 of this embodiment performs steps S1 to S5 in the same manner as the information processing device 10 of Embodiment 1.

[0044] Next, in step S6, the estimation unit 16 estimates the sagging or elasticity of the target skin based on the strain index value. As mentioned above, the strain index value correlates with the elasticity and sagging of the target skin. Therefore, in step S6, for example, the strain index value is used to estimate the elasticity and / or sagging of the skin based on the correlation between the strain index value and the elasticity and / or sagging of the skin. The elasticity and / or sagging of the skin can be evaluated, for example, using a Cutometer CT580 (manufactured by Integral Co., Ltd.).

[0045] Next, in step S7, a topical skin agent for the target is proposed based on the strain index value. As mentioned above, the strain index value correlates with the elasticity and susceptibility to sagging of the target skin. Furthermore, various topical skin agents exist to restore the elasticity and sagging of the target skin, depending on its elasticity and susceptibility to sagging. Therefore, by pre-setting the proposed topical skin agents according to the strain index value or the numerical range of the index value, step S7 can propose a topical skin agent suitable for restoring elasticity and sagging to the target skin based on the strain index value.

[0046] Then, in step S8, the feature points of the target skin image obtained in step S4, the index value of the distortion of each feature point, the tendency of the target skin to sag or elasticity obtained in step S6, and the proposed product obtained in step S7 are output to the terminal 20 via the communication network 30. The terminal 20 then displays the outputted images and data on its display unit, such as a display screen.

[0047] Then, the information processing system of Embodiment 2 terminates its processing.

[0048] (Effects of Embodiment 2) The information processing system 100 of Embodiment 2 can estimate the elasticity and susceptibility to sagging of the target skin by utilizing an index value of the distortion of the target skin, and can also suggest a topical skin preparation accordingly. Therefore, the information processing system 100 of Embodiment 2 can grasp the condition of the target skin and suggest a topical skin preparation according to the condition of the target skin.

[0049] In the information processing system 100 of Embodiment 2, the information processing device 10A includes both the estimation unit 16 and the proposal unit 17, but it may also include only one of them. Furthermore, the information processing device 10A may not include, for example, the output unit 18. In addition, the information processing system 100 of Embodiment 2 may use the information processing device 10 instead of the information processing device 10A. In this case, the description of the information processing device 10 of Embodiment 1 can be applied, for example.

[0050] In the information processing system 100 of Embodiment 2, a plurality of skin images 106 stored in the information processing device 10A of Embodiment 2 were used. However, the disclosure is not limited thereto, and for example, a plurality of skin images 106 may be captured by a terminal 20 and used for the implementation. In this case, the terminal 20 transmits the plurality of skin images 106 captured by the terminal 20 to the information processing device 10A, and the acquisition unit 11 of the information processing device 10A acquires the received plurality of skin images 106 and performs processing.

[0051] (Embodiment 3) The program of this embodiment is a program that causes a computer to execute the information processing method described herein. In the program of this embodiment, "processing" can also be expressed as, for example, "procedure" or "instruction". The program of this embodiment may also be recorded on, for example, a computer-readable storage medium. The storage medium is, for example, a non-transitory computer-readable storage medium. The storage medium is not particularly limited and includes, for example, random access memory (RAM), read-only memory (ROM), hard disk (HD), solid state drive (SSD), optical disc, floppy disk (FD), and the like.

[0052] While the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure may be made that will be understood by those skilled in the art within the scope of the present disclosure.

[0053] <Note> Some or all of the above embodiments and examples may be described as follows, but are not limited to the following. <Information Processing Methods> (Note 1) This is an acquisition process that obtains multiple skin images. The aforementioned multiple skin images are images captured over time as pressure is applied to the target skin, and A step of assigning feature points to each skin image, Each skin image is associated with the same feature points in a matching process, The method includes a setting step of calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image. A method of information processing performed on a computer. (Note 2) The information processing method described in Appendix 1, wherein in the setting step, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, the difference in distance is calculated, and the magnitude of the largest difference in distance among the obtained differences in distance is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Note 3) The information processing method according to Appendix 1 or 2, wherein in the setting step, a feature point other than the first feature point that is located within a predetermined distance from the first feature point of the reference skin image is designated as the second feature point. (Note 4) The information processing method according to Appendix 1 or 2, wherein in the setting step, all feature points other than the first feature point that are within a predetermined distance from the first feature point of the reference skin image are designated as second feature points. (Note 5) The information processing method according to any one of the appendices 1 to 4, wherein in the setting step, each feature point of the reference skin image is designated as a first feature point. (Note 6) The information processing method according to any one of the appendices 1 to 5, wherein in the setting step, the feature points included in a plurality of skin images, including the reference skin image and skin images taken after the reference skin image, are designated as the first feature points. (Note 7) The information processing method according to any one of the appendices 1 to 5, wherein in the setting step, the feature points included in all skin images among the reference skin image and skin images taken after the reference skin image are defined as the first feature points. (Note 8) The information processing method according to any one of the appendices 1 to 5, comprising a display step of displaying feature points of the target skin image and an index value of the distortion of each feature point on the target skin image. (Note 9) The information processing method described in Appendix 8, wherein the display step involves changing the display of each feature point based on the distortion index value of each feature point. (Note 10) The information processing method according to any one of the appendices 1 to 9, wherein the reference skin image is a skin image taken before the start of the pressure treatment. (Note 11) The magnitude of the difference in distance is the absolute value of the difference in distance, as described in any of the appendices 1 to 10 of the information processing method. (Note 12) The information processing method according to any one of the appendices 1 to 11, wherein the pressure treatment applied to the target skin is a pressing treatment applied to the target skin. (Note 13) An information processing method according to any one of Appendix 1 to 12, comprising an estimation step of estimating the susceptibility to sagging or elasticity of the target skin based on the aforementioned strain index value. (Note 14) An information processing method according to any one of the appendices 1 to 13, comprising a suggestion step of suggesting the target topical skin preparation based on the aforementioned strain index value. <Information Processing Device> (Note 15) This is an acquisition unit that acquires multiple skin images. The aforementioned multiple skin images include a portion that is an image captured over time of pressure treatment applied to the target skin, A unit that assigns feature points to each skin image, Each skin image has a matching section that associates the same feature points, The system includes a setting unit that calculates the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and sets the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image. Information processing device. (Note 16) The information processing apparatus according to Appendix 15, wherein the setting unit calculates the difference in distances by setting a plurality of feature points other than the first feature point of the reference skin image as second feature points, and sets the magnitude of the largest difference in distance among the obtained differences in distances as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Note 17) The information processing device according to Appendix 15 or 16, wherein the setting unit defines a feature point other than the first feature point that is located within a predetermined distance from the first feature point of the reference skin image as a second feature point. (Note 18) The setting unit is an information processing device according to Appendix 15 or 16, wherein all feature points other than the first feature point that are located within a predetermined distance from the first feature point of the reference skin image are designated as second feature points. (Note 19) The setting unit is an information processing device according to any one of appendices 15 to 18, wherein each feature point of the reference skin image is designated as a first feature point. (Note 20) The setting unit is an information processing device according to any one of appendices 15 to 19, wherein the setting unit defines the first feature point as a feature point included in a plurality of skin images, including the reference skin image and skin images taken after the reference skin image. (Note 21) The setting unit is an information processing device according to any one of appendices 15 to 20, wherein the feature points included in all skin images among the reference skin image and skin images taken after the reference skin image are defined as the first feature points. (Note 22) An information processing apparatus according to any one of appendices 15 to 21, comprising a display unit that displays feature points of the target skin image and an index value of the distortion of each feature point on the target skin image. (Note 23) The information processing device described in Appendix 22, wherein the display unit changes the display of each feature point based on the distortion index value of each feature point. (Note 24) The information processing apparatus according to any one of appendices 15 to 25, wherein the reference skin image is a skin image taken before the start of the pressure treatment. (Note 25) The magnitude of the difference in distance is the absolute value of the difference in distance, as described in any of the appendices 1% to 24. (Note 26) The information processing apparatus according to any one of appendices 15 to 25, wherein the pressure treatment applied to the target skin is a pressing treatment applied to the target skin. (Note 27) An information processing apparatus according to any one of appendices 15 to 26, comprising an estimation unit that estimates the susceptibility to sagging or elasticity of the target skin based on the aforementioned strain index value. (Note 28) An information processing method according to any one of the appendices 15 to 27, including a suggestion section that proposes the target topical skin preparation based on the aforementioned strain index value. <Program> (Note 29) On the computer, This is an acquisition process that obtains multiple skin images. The aforementioned multiple skin images are images captured over time as pressure is applied to the target skin, and the process involves this. A process to assign feature points to each skin image, A matching process is performed to associate the same feature points in each skin image, A program that performs a setting process which involves calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image. (Note 30) The program described in Appendix 29, wherein in the setting process, a plurality of feature points other than the first feature point of the reference skin image are used as second feature points, the difference in distance is calculated, and the magnitude of the largest difference in distance among the obtained differences in distance is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image. (Note 31) The program described in Appendix 29 or 30, wherein the setting process includes setting a second feature point to any feature point other than the first feature point of the reference skin image that is located within a predetermined distance from the first feature point. (Note 32) The program described in Appendix 29 or 30, wherein the setting process includes setting all feature points other than the first feature point that are within a predetermined distance from the first feature point of the reference skin image as second feature points. (Note 33) The program described in any of Appendix 29 to 32, wherein the setting process uses each feature point of the reference skin image as the first feature point. (Note 34) The program described in any of Appendix 29 to 33, wherein the setting process involves selecting a feature point included in a plurality of skin images from the reference skin image and skin images taken after the reference skin image as the first feature point. (Note 35) The program described in any of Appendix 29 to 34, wherein the setting process includes defining the first feature point as a feature point included in all skin images, including the reference skin image and skin images taken after the reference skin image. (Note 36) A program according to any one of Appendix 29 to 35, which includes a display process that displays feature points of the target skin image and an index value of the distortion of each feature point on the target skin image. (Note 37) The program described in Appendix 36 modifies the display of each feature point based on the distortion index value of each feature point in the aforementioned display process. (Note 38) The program according to any one of appendices 29 to 37, wherein the reference skin image is a skin image taken before the start of the pressure treatment. (Note 39) The magnitude of the difference in distance is the absolute value of the difference in distance, as described in any of the programs in Appendix 29 to 38. (Note 40) The program described in any one of Appendix 29 to 39, wherein the pressure treatment applied to the target skin is a pressing treatment applied to the target skin. (Note 41) A program according to any one of Appendix 29 to 40, which includes an estimation process for estimating the susceptibility or elasticity of the target skin based on the aforementioned strain index value. (Note 42) A program according to any one of the appendices 29 to 41, which includes a suggestion process to propose the target topical skin preparation based on the aforementioned strain index value. <Recording medium> (Note 43) A computer-readable recording medium on which a program described in any of the appendices 29 to 42 is recorded. <System> (Note 44) Including terminals and servers, The aforementioned terminal and the aforementioned server are connected via a communication network. The server includes an information processing device as described in any of appendices 15 to 28. The server is an information processing system that sets an index value for skin tissue distortion from multiple skin images transmitted from the terminal. [Explanation of symbols]

[0054] 10 Information Processing Devices 11 Acquisition Department 12. Assignment section 13 Correspondence section 14. Settings section 15 Display 101 CPU 102 memory 103 Bus 104 Storage device 105 Programs 106 Skin images 107 Input device 108 displays 109 Communication devices

Claims

1. This is an acquisition process that obtains multiple skin images. The aforementioned multiple skin images are images captured over time as pressure is applied to the target skin, and the process is as follows: A process of assigning feature points to each skin image, A matching process is performed to associate the same feature points in each skin image. The method includes a setting step of calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first feature point and second feature point of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image. A method of information processing performed on a computer.

2. The information processing method according to claim 1, wherein in the setting step, a plurality of feature points other than the first feature point of the reference skin image are set as second feature points, the difference in distance is calculated, and the magnitude of the largest difference in distance among the obtained differences in distance is set as an index value of the skin distortion caused by the pressure processing at the first feature point of the target skin image.

3. The information processing method according to claim 1 or 2, wherein in the setting step, a feature point other than the first feature point that is located within a predetermined distance from the first feature point of the reference skin image is defined as the second feature point.

4. The information processing method according to claim 1 or 2, wherein in the setting step, each feature point of the reference skin image is defined as a first feature point.

5. The information processing method according to claim 1 or 2, wherein in the setting step, feature points included in a plurality of skin images, including the reference skin image and skin images taken after the reference skin image, are defined as the first feature points.

6. The information processing method according to claim 1 or 2, further comprising a display step of displaying feature points of the target skin image and an index value of the distortion of each feature point on the target skin image.

7. The information processing method according to claim 6, wherein in the display step, the display of each feature point is changed based on the distortion index value of each feature point.

8. The information processing method according to claim 1 or 2, wherein the reference skin image is a skin image taken before the start of the pressure treatment.

9. The information processing method according to claim 1 or 2, wherein the magnitude of the difference in distance is the absolute value of the difference in distance.

10. The information processing method according to claim 1 or 2, wherein the pressure treatment applied to the target skin is a pressing treatment applied to the target skin.

11. The information processing method according to claim 1 or 2, comprising an estimation step of estimating the susceptibility to sagging or elasticity of the target skin based on the aforementioned strain index value.

12. The information processing method according to claim 1 or 2, comprising a suggestion step of suggesting the target topical skin preparation based on the aforementioned strain index value.

13. This is an acquisition unit that acquires multiple skin images. The aforementioned multiple skin images include a portion that is an image captured over time of pressure treatment applied to the target skin, A unit that assigns feature points to each skin image, Each skin image has a matching section that associates the same feature points, The system includes a setting unit that calculates the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and sets the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image. Information processing device.

14. On the computer, This is an acquisition process that obtains multiple skin images. The aforementioned multiple skin images are images captured over time as pressure is applied to the target skin, and the process involves this. A process to assign feature points to each skin image, A matching process is performed to associate the same feature points in each skin image, A program that performs a setting process which involves calculating the difference between the distance between a first feature point and a second feature point different from the first feature point of a reference skin image set from multiple skin images, and the distance between the corresponding first and second feature points of a target skin image captured after the reference skin image, and setting the magnitude of the difference in distance as an index value of the distortion of the skin tissue caused by the pressure processing at the first feature point of the target skin image.

15. Including terminals and servers, The aforementioned terminal and the aforementioned server are connected via a communication network. The server includes the information processing device described in claim 13, The server is an information processing system that sets an index value for skin tissue distortion from multiple skin images transmitted from the terminal.

Images