Method for generating data for human skin tactile sensation, device for generating data for human skin tactile sensation, method for evaluating human skin tactile sensation, device for evaluating human skin tactile sensation, device for presenting human skin tactile sensation, and method for presenting human skin tactile sensation

Abstract

This method for generating data on the tactile perception of human skin comprises a step for acquiring an image of skin, which is human skin or a skin model, and a step for using the acquired image of the skin and generating data on the tactile perception of human skin deduced as vibrations generated when the human skin or skin model is touched.

Description

[Technical field]

[0001] The present invention relates to a method for generating data for human skin tactile sensation, a device for generating data for human skin tactile sensation, a method for evaluating human skin tactile sensation, a device for evaluating human skin tactile sensation, and a method for presenting human skin tactile sensation. [Background technology]

[0002] A conventional method for evaluating human skin is to photograph the skin and analyze the image of the skin surface to evaluate the apparent condition of the skin.

[0003] In recent years, in the cosmetics field, there has been a demand for tactile devices that allow multiple people to share the condition of their skin. However, conventional methods that simply analyze images of two-dimensional skin patterns do not anticipate converting the images into acceleration information (vibrations), making it difficult to reproduce a skin feel that can be shared by multiple people.

[0004] On the other hand, in order to present different textures even with the same material, a tactile reproduction device has been proposed that can change the tactile sensation when touching a material with a fingertip by applying depth-wise vibration to the contact material (e.g., Non-Patent Document 1).

[0005] However, Non-Patent Document 1 considers a hard contact material whose surface does not change, whereas in the case of skin, the material itself has softness and viscoelasticity that causes it to deform. Therefore, when reproducing the tactile sensation caused by vibration for a contact material that does not change is applied to a skin model, errors will occur in the reproduced tactile sensation.

[0006] Furthermore, in non-patent document 2, in order to analyze the sensation of soft skin felt by the finger, the friction between a finger and an artificial skin model is considered, and it is shown that the tendency for the contact area and normal force to change differs depending on the surface hardness of the finger and the artificial skin model that is the object of contact.

[0007] Furthermore, Patent Document 1 proposes that, in order to evaluate skin to which a cosmetic has been applied, a relationship be constructed in advance by measuring the vibrations generated when the skin is touched with a finger using a vibration sensor as an evaluation criterion, and by correlating the sensory evaluation of the tactile sensation of the applied cosmetic with the change in frequency of the measured vibration. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] WO2019 / 030803 publication [Non-patent literature]

[0009] [Non-Patent Document 1] Shuhei Asano, Shogo Okamoto, and Yoji Yamada. Increase and decrease of texture roughness by vibrotactile stimulation. IEEE Trans. Human-Mach. Syst. 45, 393-398, 2015. [Non-Patent Document 2] Kouki Inoue, Shogo Okamoto, Yasuhiro Akiyama, Yoji Yamada, Effect of material hardness on friction between fingertips and dry and lubricated joint skin, IEEE Transactions on Haptics, vol.13, no.1, pp.123-129, 2020 Summary of the Invention [Problem to be solved by the invention]

[0010] Here, when trying to apply the friction technology of Non-Patent Document 2 to reproducing the feel of skin, it is necessary to distinguish between cases depending on the surface hardness of the finger and the contact object, and since the surface hardness of each person's fingertips differs, it is difficult to share various skin feel among multiple people.

[0011] In addition, in the method of Patent Document 1, a vibration sensor was used to collect vibrations when actually touching the skin with a finger as a sample for evaluation criteria, but there was a risk that the collected vibrations would be influenced by the characteristics of the person touching (such as the hardness of the finger and the way of touching).

[0012] In view of the above circumstances, the present invention aims to provide a method for generating human skin tactile data that can collect information on human skin without touching the skin or using a special tactile sensor, minimize errors due to the characteristics of the person touching the skin, and generate human skin tactile data that can be shared by multiple people. [Means for solving the problem]

[0013] In order to solve the above problems, in one aspect of the present invention, acquiring an image of skin, the image being human skin or a skin model; and generating human skin tactile data estimated as vibrations that occur when the human skin or skin model is touched using the acquired skin image. To provide a method for generating data for human skin tactile sensation. Effect of the Invention

[0014] According to one aspect, in a method for generating human skin tactile data that generates human skin tactile data that can be shared by multiple people, human skin information can be collected without touching the skin or using a special tactile sensor, minimizing errors due to the characteristics of the person touching it. [Brief description of the drawings]

[0015] [Figure 1] 1 is a schematic flow diagram showing a method for generating human skin tactile data according to the present invention. [Diagram 2] FIG. 1 is a functional block diagram of a human skin tactile data generating device according to a first embodiment of the present invention. [Diagram 3] 4 is a detailed flow of a data generating process according to the first embodiment. [Figure 4] 11 shows an example of a range of human skin gel photography, a three-dimensional image of human skin gel simulating a person in their twenties, and a displacement curve calculated in S21 and S22 of the first embodiment. [Diagram 5] 3A to 3D images of human skin gel simulating a person in his 40s and examples of displacement curves calculated in S21 and S22 of the first embodiment. [Figure 6]3A to 3D images of human skin gel simulating a person in his 70s and examples of displacement curves calculated in S21 and S22 of the first embodiment. [Figure 7] 13 is a graph showing an example of the frequency spectrum of a displacement curve obtained by converting the horizontal axis of the displacement curve from distance to time in S23 in the first embodiment. [Figure 8] 10 is a graph showing an example of an acceleration spectrum obtained by converting the spectrum of the displacement curve into acceleration in S24 in the first embodiment. [Figure 9] 10 is a graph showing an example of corrected acceleration information obtained by converting the acceleration spectrum by applying a low-pass filter in S25 in the first embodiment. [Figure 10] 1 is a schematic flow diagram showing a method for presenting a human skin tactile sensation using human skin tactile sensation data generated by the human skin tactile sensation data generating method of the present invention. [Figure 11] FIG. 1 is an external view showing an example of a human skin tactile sensation providing device for realizing the tactile sensation providing process of the present invention. [Figure 12] FIG. 12 is a hardware block diagram of the human skin tactile sensation presentation device shown in FIG. 11 . [Figure 13] A diagram showing different types of human skin gels being directly touched with a finger to distinguish them. [Figure 14] 13 is a diagram showing the skin type discrimination results when the human skin gel is directly touched with a finger. [Figure 15] FIG. 13 is a diagram showing a state in which vibration data is created by the human skin tactile sensation data generation method including image acquisition of the present invention, and the vibration data is presented as a tactile sensation by a human skin tactile sensation presentation device. [Figure 16] FIG. 13 is a diagram showing the results of skin type discrimination when a tactile sensation is presented by a human skin tactile sensation presentation device using vibration data generated by the method of the present invention. [Figure 17] As a comparative example, this figure shows a state in which vibration data is created by touching an artificial finger AF equipped with an accelerometer α to a skin sample in a human skin tactile sensation presentation device, and the vibration data is presented as a tactile sensation in the human skin tactile sensation presentation device. [Figure 18] FIG. 13 is a diagram showing the results of skin type discrimination when a tactile sensation is reproduced by a human skin tactile sensation presentation device using vibration data generated by a method of a comparative example. [Figure 19]FIG. 11 is a functional block diagram of a human skin tactile data generating device according to a second embodiment of the present invention. [Figure 20] 13 is a detailed flow of a data generating process according to the second embodiment. [Figure 21] 13 shows an example of a luminance image of human skin and an example of a displacement curve using the luminance calculated in S201, S202, and S203 of the second embodiment. [Figure 22] 1 is a schematic flowchart showing a human skin tactile sensation evaluation method including a human skin tactile sensation data generation method of the present invention. [Diagram 23] FIG. 1 is a block diagram of a human skin texture evaluation device for implementing the human skin texture evaluation method of the present invention. [Figure 24] FIG. 1 is a schematic diagram showing a beauty advisor presenting skin quality to a customer during counseling using the human skin texture evaluation device of the present invention. [Diagram 25] 1 is a schematic diagram of a human skin texture evaluation and presentation system using the human skin texture evaluation method of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following drawings, the same components are denoted by the same reference numerals, and duplicated explanations may be omitted.

[0017] The present invention relates to a method for generating data for human skin tactile sensation, i.e., a method for generating data for rendering (representing, reproducing, reproducing, presenting) a human skin tactile sensation, and also to a device for generating data for human skin tactile sensation, a method for evaluating a human skin tactile sensation, a device for evaluating a human skin tactile sensation, and a method for presenting a human skin tactile sensation, all of which use the method for generating data for human skin tactile sensation.

[0018] In this specification, the "touch of human skin" refers to the sensation felt by the fingers or palm when touching human skin by stroking or rubbing with the fingers or palm.

[0019] <How to generate data for human skin texture> FIG. 1 is a schematic flow diagram showing a method for generating human skin tactile data according to the present invention.

[0020] As shown in Fig. 1, in the human skin texture data generation method of the present invention, first, in step S1, an image of skin is acquired (image acquisition step). The "skin" from which an image is acquired includes not only real human skin, but also skin models such as skin replicas (hard artificial skin that reproduces the unevenness of the surface of human skin) and human skin gel (soft artificial skin that expresses the surface and softness of human skin). The real human skin may be bare skin, or it may be skin after a specific cosmetic product has been applied.

[0021] In addition, the means for acquiring an image is a three-dimensional imaging means capable of acquiring a three-dimensional image in the first embodiment described below, and is a two-dimensional imaging means capable of acquiring a two-dimensional image in the second embodiment described below. In addition, as a modified example, it may be a two-dimensional imaging means capable of acquiring a two-dimensional UV image, a two-dimensional imaging means capable of acquiring a two-dimensional optical interference image, or a two-dimensional imaging means with an adjusted focus.

[0022] Then, in step S2, the acquired skin image is used to generate human skin texture data (data generation step).

[0023] Here, the human skin tactile data is tactile data felt on the fingers or palm, estimated as vibrations caused when touching skin (human skin or skin model). The human skin tactile data is vibration data (presentation data (also called reproduction data, expression data, or rendered data)) that is the source data of the virtual texture when presenting the tactile sensation, and is also comparison data used for comparison with evaluation standards, etc., during evaluation.

[0024] In the present invention, the details of the data generation process in step S2 will be described in a first embodiment, a second embodiment, and a modified example. An example of an apparatus capable of realizing the human skin tactile data generation method in FIG.

[0025] <First embodiment of the human skin tactile data generating device> FIG. 2 is a functional block diagram of a human skin tactile data generating device according to a first embodiment of the present invention.

[0026] As shown in FIG. 2, the human skin tactile data generating device 100 according to the first embodiment includes a three-dimensional imaging means 1, which is an image acquiring section, and an information processing device 2, which is a data generating section.

[0027] The three-dimensional imaging means 1 acquires an image of human skin or skin that is a skin model. In this embodiment, the three-dimensional imaging means 1 is a means capable of capturing a three-dimensional image of a living body's skin or skin that is an artificial skin model, such as a three-dimensional microscope (e.g., a laser biomicroscope), an ultrasonic cross-sectional imaging device, or a 3D camera.

[0028] The information processing device 2 uses the acquired skin image to generate human skin tactile data estimated as vibrations generated when touching human skin or a skin model. The information processing device 2 is, for example, a PC (personal computer) with a specific software installed, or a dedicated three-dimensional skin image analysis device.

[0029] The information processing device 2 according to this embodiment executablely includes a surface roughness information acquisition section 21, an acceleration information conversion section 22, and a human skin feel data output section 23 (hereinafter simply referred to as an output section).

[0030] The surface roughness information acquiring unit 21 acquires skin surface roughness information from the skin image. In detail, the surface roughness information acquiring unit 21 has a surface roughness extracting unit 211 that extracts the skin surface roughness (height on the skin cross section) at each position from the three-dimensional skin image, and a displacement curve acquiring unit 212 that acquires a displacement curve connecting information on the surface roughness (height on the skin cross section) at each position on a predetermined straight line.

[0031] The acceleration information conversion unit 22 converts the skin surface roughness information acquired by the surface roughness information acquisition unit 21 into acceleration information. The acceleration information conversion unit 22 has a displacement spectrum conversion unit 221, an acceleration spectrum conversion unit 222, and an acceleration information correction unit 223. Details of these functions will be described later with reference to FIG.

[0032] The output unit 23 sets the acceleration information calculated by the acceleration information conversion unit 22 as human skin feel data (human skin expression data) and outputs it.

[0033] In addition, in FIG. 2, a configuration is shown in which the functions of the surface roughness information acquisition unit 21, the acceleration information conversion unit 22, and the output unit 23 are realized by one information processing device 2, but these calculation functions may be executed using two or more information processing devices.

[0034] (Data generation process of the first embodiment) Details of the data generating process executable by the information processing device 2 shown in Fig. 2 will be described with reference to Fig. 3 to Fig. 9. Fig. 3 is a detailed flow of the data generating process according to the first embodiment.

[0035] The information processing device 2 of FIG. 1 can execute, as data generation processes, processes of acquiring skin surface roughness information from a skin image (S21, S22), processes of converting the skin surface roughness information into acceleration information (S23, S24, S25), and a process of setting the acceleration information to human skin tactile data (S26).

[0036] Here, the inventors of the present application generated data for human skin texture using sample A of human skin gel simulating a person in their 20s, sample B of human skin gel simulating a person in their 40s, and sample C of human skin gel simulating a person in their 70s. Figures 4 to 9 show the data generated at each stage of the process.

[0037] The human skin gels used as samples A, B, and C are examples of skin models (artificial skin) that mimic a specific skin condition, and are made of ultra-soft urethane resin that has the same softness as human skin. The human skin gel used in the following calculations was Bioskin (registered trademark) (manufactured by Bealux).

[0038] More specifically, in the process of acquiring the skin image shown in FIG. 1 in this embodiment, a 3D image of the surface of human skin or a skin model is acquired as the skin image using a laser microscope, which is an example of a 3D imaging means 1 shown in FIG. 2.

[0039] As an example of a laser microscope for acquiring images, a three-dimensional measuring microscope (VR-3100, Keyence) that acquires three-dimensional images of specific areas on the sample surface by laser scanning was used to acquire three-dimensional images of an area of ​​24 mm wide and 18 mm long on the human skin gel. Figure 4(a) shows the photographed area and horizontal center line on human skin gel sample A. The horizontal center line is an example of a straight line at a specified position that can be extracted from the cross-sectional height information included in the three-dimensional image.

[0040] In the three-dimensional image obtained here, the height information of each position on the skin cross section can be known according to the color. For example, Fig. 4(b) is a three-dimensional image of human skin gel A simulating a person in their 20s, Fig. 5(a) is a three-dimensional image of human skin gel B simulating a person in their 40s, and Fig. 6(a) is a three-dimensional image of human skin gel C simulating a person in their 70s. Here, in the figures, white parts indicate that the skin surface position (height on the skin cross section) is high, gray parts indicate that the skin surface position is medium, and black parts indicate that the skin surface position is low, i.e., concave.

[0041] By extracting the color of the three-dimensional image for each position in this manner, in step S21, the height on the skin cross section at each position is extracted from the three-dimensional skin image, that is, the surface roughness of the skin at each position.

[0042] Referring to the scale bars on the left of Figures 4(b), 5(a), and 6(a), in human skin gel A simulating a person in their 20s shown in Figure 4(b), the skin surface position is in the range of -0.037mm to +0.045mm, in human skin gel B simulating a person in their 40s shown in Figure 5(a), the skin surface position is in the range of -0.048mm to +0.049mm, and in human skin gel C simulating a person in their 70s shown in Figure 6(a), the skin surface position is in the range of -0.124mm to +0.11mm. This shows that the unevenness of the skin surface increases in the following order: human skin gel C (70s) > human skin gel B (40s) > human skin gel A (20s).

[0043] The displacement curve is the horizontal connection of the skin surface positions (height on the skin cross section) on a straight line at a specific position (for example, the horizontal center line indicated by the arrow in Figure 4(a)), which can be extracted from the cross-sectional height information contained in such a 3D image.

[0044] That is, in step S22, based on the surface roughness at each position on the skin (height on the skin cross section at each position), assuming that a predetermined distance is scanned along a straight line in a three-dimensional image of the surface of human skin or a skin model, a displacement curve on a straight line corresponding to the cross-sectional height position of the three-dimensional skin, with distance on the horizontal axis, is obtained as surface roughness information.

[0045] Specifically, from the 3D image of human skin gel A in Fig. 4(b), assuming that the horizontal center line indicated by the arrow in Fig. 4(a) is scanned over a distance of 24 mm, which is the horizontal width of the shooting range, which is the length of the arrow, each skin surface position (cross-sectional height) in the horizontal direction becomes the displacement curve. Here, the displacement curve is the height at each position on the cross section based on the horizontal center line, that is, the roughness of the line obtained on the horizontal center line, and therefore the displacement curve is said to show the surface roughness information of the skin on a straight line of a predetermined length.

[0046] In addition, extraction of the height on the skin cross section at each position from the three-dimensional image and acquisition of the displacement curve in S21 and S22 were performed by analysis and processing in a software environment (VR-3000 G2 APPLICATION) set in the information processing device 2.

[0047] The displacement curve in Fig. 5(b) shows the respective skin surface positions (cross-sectional heights) in the lateral direction when the horizontal center line indicated by the arrow in the 3D image of human skin gel B in Fig. 5(a) is scanned for a distance of 24 mm in width. The displacement curve in Fig. 6(b) shows the respective skin surface positions (cross-sectional heights) in the lateral direction when the horizontal center line indicated by the arrow in the 3D image of human skin gel C in Fig. 6(a) is scanned for a distance of 24 mm in width.

[0048] In Figures 4(c), 5(b), and 6(b), the horizontal axis indicates the horizontal position on a straight line in the image, and the vertical axis indicates the height on the skin cross section, i.e., the cross-sectional position of each position relative to the average cross-sectional position of the skin surface.

[0049] Comparing Figures 4, 5, and 6, it can be seen that as the color unevenness in the 3D image becomes larger and the surface roughness increases (human skin gel A < human skin gel B < human skin gel C), the displacement curve, which is the line roughness profile, shows a large vibration amplitude.

[0050] Here, since the vibration for presenting (reproducing) the tactile sensation of the skin is based on acceleration, it is necessary to convert the obtained displacement curve (line roughness curve) into an acceleration curve.

[0051] Therefore, in step S23, the moving speed when touching human skin or a skin model is assumed, and the horizontal axis of the linear displacement curve, which is the surface roughness information, is converted from distance to time to create a spectrum of the displacement curve.

[0052] As described above, the displacement curve indicates the height of the skin surface (height of the skin cross section) at each position on the selected straight line. Therefore, for example, if this curve is placed on a time axis with a length of about 1 second, it can be considered as a displacement curve of the surface height when the probe horizontally scans a distance of 24 mm in 1 second, and can be converted into a spectrum of the displacement curve.

[0053] Figure 7(d) shows the spectrum of the displacement curve converted based on the displacement curves of human skin gels A, B, and C of each age shown in Figures 7(a), (b), and (c). In Figure 7(d), the horizontal axis shows frequency (Hz) with 1 second set to frequencies from 0 to 500 Hz, and the vertical axis shows amplification (dB). In other words, the spectrum of the displacement curve shows the amplitude characteristics at each frequency.

[0054] The conversion from this displacement curve to a displacement curve spectrum was performed by audio software (for example, Audacity (registered trademark)) installed in the information processing device 2.

[0055] Then, in step S24, the spectrum of the displacement curve is converted into a spectrum of acceleration.

[0056] In more detail, when a displacement curve is Fourier transformed, the original curve is decomposed into sine waves of each frequency, and the equation for the sine wave with frequency i becomes equation (1) below. Since the Fourier transform serves to decompose a time series waveform into sine waves of each frequency, the resulting equation (1) contains the equation for a sine wave with frequency i (i = 1 to 500 Hz) within the sine wave. By calculating the second-order derivative of this equation (1), equation (2) for acceleration is obtained.

[0057] And the negative sign in equation (2) can be removed by adding a phase difference of π (180 degrees) to the sine curve. In the case of haptic rendering, this phase change has a negligible effect on perception, so we can remove it by subtracting ω from equation (1). 2 We obtain the formula (3) by multiplying the elements. In other words, for each frequency component, ω 2 It can be converted into an acceleration waveform by multiplying it by . When performing this processing using software (e.g. Audacity) equalizer, since the amplitude is expressed in logarithm, it is necessary to consider processing on a logarithmic graph, but this can be done by taking the logarithm of equation (3), and when expressed in dB, it becomes as shown in equation (4).

[0058]

number

[0059]

number

[0060]

number

[0061]

number

[0062] The conversion from the spectrum of the displacement curve to the acceleration spectrum was performed by an equalizer function of audio software (e.g., Audacity) set in the information processing device 2. When the above formulas (1) to (4) were calculated by the equalizer and applied to the displacement curve shown in the lower part of Fig. 8, an acceleration curve (acceleration spectrum) as shown in the upper part of Fig. 8 was obtained.

[0063] Here, it can be seen that in the acceleration spectra shown in the upper part of FIG. 8 and in FIG. 9(a), the amplitude increases significantly at 150 Hz or higher.

[0064] Therefore, in step S25, the acceleration spectrum is corrected by applying a low-pass filter. Fig. 9(b) is a graph showing an example of corrected acceleration information obtained by converting the acceleration spectrum in S25 by applying a low-pass filter in the first embodiment.

[0065] Here, the area of ​​the image of the gel taken to obtain the above displacement curve is larger than the area of ​​the actual contact area of ​​the finger with the gel, and the contact area between the finger and the gel acts as a high-frequency blocking (low-pass) filter, causing a deviation in vibrations of 150 Hz or higher. Therefore, a low-pass filter was applied to bring the acceleration spectrum closer to the actually measured acceleration spectrum.

[0066] For example, in S25, the amplitude of vibration in the range of 150 Hz or higher in the acceleration spectrum shown in FIG. 9(a) is reduced by −12 dB using a low-pass filter function of audio software (e.g., Audacity) set in the information processing device 2, and finally a spectrum as shown in FIG. 9(b) is obtained as corrected acceleration information.

[0067] The corrected acceleration information calculated in this manner (vibration data in FIG. 9(b)) is set as the human skin tactile sensation data as a result of haptic rendering for reproducing the actual human skin tactile sensation (step S26).

[0068] The human skin tactile data thus generated is used to present and evaluate the human skin tactile sensation described below.

[0069] <Method of presenting human skin sensation> FIG. 10 is a schematic flow diagram showing a method for presenting a human skin tactile sensation using the human skin tactile sensation data generated by the human skin tactile sensation data generating method of the present invention.

[0070] The human skin tactile sensation presentation method shown in FIG. 10 includes a step of presenting, in step S3, vibrations on a human skin model using human skin tactile sensation data (acceleration information) generated by the human skin tactile sensation data generation method S1 and S2 shown in FIG.

[0071] In the tactile sensation presentation step of step S3, the human skin tactile sensation data (acceleration information) generated in the data generation step is used to present the tactile sensation. When presenting the tactile sensation, an evaluation standard may be established in advance, and the tactile sensation may be presented based on the standard. The establishment of the evaluation standard will be described later together with the tactile sensation evaluation method (see FIG. 22).

[0072] In FIG. 10, in the human skin tactile sensation presentation method, a step of executing a human skin tactile sensation data generation method is provided prior to the presentation step. However, if pre-stored human skin tactile sensation data (e.g., human skin tactile sensation data for reference skin) is called up to present the sensation, S1 and S2 can be omitted and only the tactile sensation presentation step S3 is required.

[0073] <Human skin sensation presentation device> Here, an example of a human skin tactile sensation presentation device that realizes the tactile sensation presentation process of Fig. 10 will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is an external view showing an example of a human skin tactile sensation presentation device that realizes the tactile sensation presentation process of the present invention.

[0074] As shown in FIG. 11, the human skin tactile sensation presentation device 300 includes a human skin gel 31, a slide volume 32, a PC 33, an amplifier 34, a speaker 35, a transmission plate 36, and a device base 37.

[0075] The human skin gel 31 is an object that the user's finger or artificial finger touches, and is a skin model (artificial skin) that imitates a specific state of human skin, and is made of super-soft urethane resin (polyurethane skin plate) that expresses the same softness as human skin. For example, Bioskin (registered trademark) (manufactured by Viewlux) is used as the human skin gel 31.

[0076] The skin-care gel 31 is attached and fixed to a transmission plate (transmission member) 36. In Fig. 10, an example in which the skin-care gel 31 is circular is shown, but the skin-care gel 31 may be elongated in the left-right direction.

[0077] The slide volume 32 is provided above the human skin gel 31, and serves as a finger moving unit that moves the user's finger or artificial finger at a predetermined moving speed on the human skin gel 31. Alternatively, when the user's finger moves independently, the slide volume 32 also functions as a moving speed acquiring unit that acquires the moving speed of the finger.

[0078] The sliding volume 32 has a slide rail 321 , left and right support columns 322 that support the slide rail, a variable resistor 323 (see FIG. 12), a finger slider 324 , and a motor 325 .

[0079] The finger slider 324 is a handle part composed of a pair of clamping support parts, and when presenting a tactile sensation, the user's finger or artificial finger can be inserted between the clamping support parts, and the finger slider 324 slides left and right to move the finger or artificial finger on the human skin gel 31. Alternatively, the finger slider 324 may move together with the movement of the person's finger.

[0080] The resistance value of variable resistor 323 changes depending on the position of finger slider 324 relative to slide rail 321. In other words, when no force is applied from the finger, for example when a person who holds finger slider 324 has relaxed their finger or when finger slider 324 holds an artificial finger, the resistance value of variable resistor 323 can be changed to move finger slider 324 to the left or right.

[0081] Alternatively, when a person's finger actively moves, the position and moving speed of the finger can be calculated by reading the change in resistance value caused by the finger slider 324 moving left and right along with the finger movement by the downstream PC 33. The slide rail 321, the finger slider 324, and the variable resistor 323 form a moving speed acquisition unit that acquires the moving speed of the finger when the user's finger moves independently.

[0082] When no force is applied from the finger, motor 325 controls the driving so as to adjust the resistance value of variable resistor 323 in order to move finger slider 324 at a predetermined speed along slide rail 321. Motor 325, variable resistor 323, and finger slider 324 form a finger moving unit when moving an artificial finger or the user's finger on the device side.

[0083] The PC 33 is an example of an information processing device that is connected to the slide volume 32 and the amplifier 34 by wire or wirelessly. The PC 33 controls the speed at which the motor 325 of the slide volume 32 is driven, sets the moving speed of the finger on the human skin gel 31 in the left-right direction, and outputs the strength of vibration according to the moving speed. Alternatively, when the user actively moves his / her finger, the PC 33 calculates and obtains the moving speed of the finger based on the position of the finger acquired by the slide volume 32, and outputs the strength of vibration.

[0084] Furthermore, the PC 33 has a function of an original signal storage unit that stores an original signal of vibration (human skin texture data) associated with the skin texture (texture) of human skin. In the following, the original signal is a signal that is the source of vibration for generating vibrations by the speakers 35A and 35B, and is a signal in the form of an audio signal associated with the skin texture.

[0085] Therefore, PC 33 outputs the original signal (audio signal) selected according to the skin type to be expressed and information on the strength of the signal associated with the finger movement speed to amplifier 34. That is, PC 33 is also a vibration adjustment unit that adjusts the strength (amplitude value) of vibration when outputting vibration according to the original signal according to the finger movement speed. The hardware configuration of PC 33 will be described in detail with reference to FIG. 12.

[0086] An amplifier (audio amplifier) ​​34 amplifies the original signal (human skin tactile sensation data) output from the PC 33 in accordance with the intensity information.

[0087] The speaker 35 is connected to the amplifier 34 and is configured as a pair of left and right speakers 35A and 35B. The speakers 35A and 35B are vibration output units that convert the adjusted audio signal into physical vibrations and output the physical vibrations, and apply the vibrations to the human skin gel 31 via the transmission plate 36.

[0088] The speakers 35A, 35B are connected to the amplifier 34 by wire or wirelessly. The speakers 35A, 35B are in contact with the transmission plate 36 provided with the human skin gel 31, and support the transmission plate 36 by sandwiching the transmission plate 36 so that vibrations are transmitted.

[0089] In more detail, speakers 35A and 35B have a circular diaphragm 351 that generates physical vibrations, a frame 352 that surrounds diaphragm 351, and a support 353 to which frame 352 is attached.

[0090] In addition, a slit 354 is formed in the center of the vertical direction of the opposing faces of the speakers 35A and 35B, from the surface of the diaphragm 351 where vibrations are generated toward the back side. When the propagation plate 36 with the human skin gel 31 attached thereto is inserted into the left and right slits 354, the speakers 35A and 35B sandwich the diaphragm 351 from the left and right, so that the vibrations generated by the diaphragm 351 are propagated to the propagation plate 36.

[0091] The transmission plate 36 is a plate to which the skin-care gel 31 is attached on the upper surface thereof, thereby supporting the skin-care gel 31 and transmitting vibrations. The transmission plate 36 is made of, for example, an acrylic plate.

[0092] The device base 37 supporting the slide volume 32 and the speakers 35A, 35B includes a fixing plate 371, which is an acrylic plate below the propagation plate 36, a base 372, and a cushioning material 373.

[0093] The lower ends of the left and right frames 352 of the speakers 35A and 35B are fixed to the upper surface of the fixing plate 371. The lower surface of the fixing plate 371 is attached to the upper surface of a base 372 (apparatus pedestal, device pedestal) via a cushioning material 373. Because the cushioning material 373 prevents vibrations from the speakers 35A and 35B from propagating to the base 372, even if the human skin tactile sensation presentation device 300 is placed on a desk 9 (see FIG. 23), vibrations are not propagated to the surface of the desk 9, making it less likely to be damaged.

[0094] In the above, an example was described in which the human skin gel 31 is the same as room temperature, but in order to make the feel of the human skin gel closer to real human skin, a heating section that heats the human skin gel 31 to 15°C to 37°C may be provided on the back side of the transmission plate 36.

[0095] FIG. 12 is a hardware block diagram of the human skin tactile sensation presentation device 300 shown in FIG.

[0096] The PC 33 includes a USB interface 301, a memory 302, an operation system 303, a sound controller 304, an audio device connection unit 305, a system bus 306, an input unit 307, and a display unit 308. The input unit 307 is a mouse, a keyboard, a touch pad, a touch panel input surface, or the like, and the display unit 308 is an LCD (liquid crystal display), an organic EL display, or the like.

[0097] The USB interface 301, memory 302, operation system 303, sound controller 304, audio device connection unit 305, input unit 307, and display unit 308 are connected via a system bus 306. Alternatively, if the PC 33 is a desktop personal computer, the input unit 307 and display unit 308 may be provided as separate units so as to be connected to the main body via the USB interface 301.

[0098] The USB interface 301 receives the finger position information obtained by the variable resistor 323 of the slide volume 32 .

[0099] The memory 302, as an original signal storage section, stores in advance an original signal (human skin tactile data), which is an audio signal that is the source of vibrations corresponding to human skin. The human skin tactile data is acquired by a communication section (not shown) of the PC 33 transmitting and receiving data to and from the human skin tactile data generation device 100. Alternatively, data may be transferred from the human skin tactile data generation device 100 to the PC 33 via a storage medium such as a USB memory or a memory card.

[0100] The operation system (OS) 303 includes, for example, a central processing unit (CPU) 331, a field-programmable gate array (FPGA) 332, a random access memory (RAM) 333, a read only memory (ROM) 333, and the like.

[0101] The CPU 331 and FPGA 332 use the RAM 333 as a working area, execute various control programs stored in the ROM 334 , and output control commands for controlling various operations in the PC 33 .

[0102] In the OS 303 , for example, the CPU 331 generates a signal for controlling the driving of the motor 325 , thereby changing the resistance value of the variable resistor 323 of the slide volume 32 .

[0103] In the slide volume 32, the motor 325 is set to move the finger slider 324 along a certain trajectory at a predetermined moving speed by changing the resistance value of the variable resistor 323, for example, by PMW (Pulse Width Modulation) and PID (Proportional Integral Differential) control. This sets the moving speed of the finger or artificial finger on the human skin gel 31 during tactile sensation presentation.

[0104] Alternatively, when the user actively moves his / her finger, the OS 303, for example the CPU 331, reads the change in the resistance value of the variable resistor 323 of the slide volume 32 and calculates the moving speed of the finger on the human skin gel 31 during the tactile sensation presentation from the time derivative of the position.

[0105] In addition, in the OS 303 , the FPGA 332 calls the original signal stored in the memory 302 .

[0106] The sound controller 304 adjusts the volume of the called original signal (human skin tactile data) according to the movement of the finger. For example, it adjusts the volume so that the finger does not vibrate when the finger is stationary, but vibrates when the finger starts to move, and adjusts the volume so that the faster the finger moves, the larger the amplitude of the sound signal is output.

[0107] The audio device connection unit 305 is a terminal that connects to the amplifier 34 , and outputs the audio signal adjusted by the sound controller 304 to the amplifier 34 .

[0108] The sound controller 304 and audio device connection unit 305 function as a vibration adjustment unit that outputs an original signal (human skin tactile data) that is a finger movement speed and an audio signal, and an adjustment amount for the amplitude of the waveform of the original signal.

[0109] In this way, in the human skin tactile sensation presentation device 300 of the present invention, the PC 33 outputs a waveform (audio signal) obtained by adjusting the human skin tactile sensation data according to the finger movement speed to the amplifier 34, and the adjusted audio signal is converted into physical vibrations and output by the speakers 35A, 35B. The vibrations are applied as a tactile sensation to the human skin gel 31 via the propagation plate 36, thereby reproducing a virtual texture.

[0110] <Experimental Example> Here, in order to confirm the validity of the reproduction of the human skin tactile data generated by the method of the present application, the inventors of the present application conducted an experiment to see whether the three types of skin types could be distinguished by reproducing the human skin tactile data with the human skin tactile sensation presentation device 300 and by other methods. The subjects were six right-handed people aged 24±1 year old with no functional problems in their sense of touch.

[0111] (Experiment 1) The artificial skin samples used in the experiment had similar textures, so subjects may have difficulty distinguishing between the samples. Therefore, in Experiment 1, we conducted an experiment to confirm whether subjects could distinguish between the three different samples by directly touching the actual skin samples, as shown in Figure 13. Figure 13 shows the state in which multiple types of human skin gels were directly touched with a finger in order to distinguish them.

[0112] First, the experimenter presented one sample in front of the subject, who was instructed to explore it for three seconds with his eyes closed. After that, three samples were presented, and the subject was asked to explore each sample within five seconds, and finally to choose the sample he explored first by answering A, B, or C. The three samples were A, a human skin gel that imitated the skin of a person in their 20s, B, a human skin gel that imitated the skin of a person in their 40s, and C, a human skin gel that imitated the skin of a person in their 70s.

[0113] The results are shown in Figure 14. Figure 14 shows the skin type discrimination results when the human skin gel was directly touched with a finger. The accuracy rate was over 80% for all samples A, B, and C, which means that the textures (feel to touch) of the three skin samples A, B, and C were different enough for the subjects to distinguish them.

[0114] (Experiment 2) In experiment 1, the subjects directly touched the skin samples to be discriminated, but in experiments 2 and 3, experiments were conducted to verify whether a virtual texture reproduced using vibration data generated in advance on the human skin gel 31 of the human skin tactile sensation presentation device 300 could be correctly identified. Note that the human skin gel 31 is a human skin gel with a different feel from samples A, B, and C.

[0115] In experiment 2, an experimenter acquires three-dimensional images of skin samples A, B, and C in advance in accordance with the human skin tactile data generation method of the present invention, and generates data based on the three-dimensional images to acquire three pieces of human skin tactile data (vibration data) that respectively mimic skin samples A, B, and C. The vibration data is stored in the human skin tactile sensation presentation device 300. Fig. 15(a) is a diagram explaining how vibration data is created by acquiring an image of a skin sample using the human skin tactile data generation method of the present invention.

[0116] The subject was then instructed to insert his finger F into the finger slider 324 of the human skin tactile sensation presentation device 300 and touch the smooth human skin gel 31 with a pressure of about 0.5 N. The subject then moved his finger at a predetermined speed on the human skin gel 31, which was reproducing one of three vibration data simulating skin samples A, B, and C generated by the human skin tactile sensation reproduction method of the present invention, and searched for a virtual texture on the human skin gel 31 within 10 seconds under the same search conditions. Figure 15(b) is a diagram showing a state in which the vibration data generated based on the image acquired as in Figure 15(a) is presented as a tactile sensation by the human skin tactile sensation presentation device 300.

[0117] After the exploration, the subject was instructed to explore three actual skin samples A, B, and C as shown in Figure 13 and select the sample that was closest to the virtual texture reproduced by the human skin tactile sensation presentation device 300 immediately before by answering either A, B, or C.

[0118] This experiment was carried out twice, with different samples presented to each subject, so that the number of samples presented to each subject was approximately equal.

[0119] The results are shown in Fig. 16. Fig. 16 is a diagram showing the skin type discrimination results when a tactile sensation was presented by a human skin tactile sensation presentation device using vibration data generated by the method of the present invention. As shown in Fig. 16, the accuracy rate of each sample exceeded 70% when a virtual texture was presented by acquiring a skin image and generating data for the human skin tactile sensation using the method of the present invention.

[0120] (Experiment 3) In experiment 3, as a method for generating data for human skin tactile sensation according to a comparative example, an artificial finger AF is clamped on the finger slider 324 in the human skin tactile sensation presentation device 300 in advance, and an accelerometer α is attached to the upper surface of the artificial finger AF, and skin samples A, B, and C are set on the transmission plate 36 as human skin gel. Then, acceleration data is measured when the artificial finger AF is moved at a predetermined moving speed by the slide volume 32 while the artificial finger AF is touching the skin samples A, B, and C. Then, data is generated based on the acceleration data to obtain three pieces of data for human skin tactile sensation (vibration data) that imitate each of the skin samples A, B, and C, and the vibration data is stored in the human skin tactile sensation presentation device 300. FIG. 17(a) is a diagram explaining a state in which vibration data is created by touching the artificial finger AF, to which the accelerometer α is attached, to the skin sample in the human skin tactile sensation presentation device 300 as a comparative example.

[0121] Then, the subject was instructed to insert the finger F into the finger slider 324 of the human skin tactile sensation presentation device 300 and touch the smooth human skin gel 31 with a pressure of about 0.5 N, as in Experiment 2. Then, the subject moved his / her finger at a predetermined moving speed on the human skin gel 31, which was reproducing one of three vibration data simulating skin samples A, B, and C, generated by the human skin tactile sensation reproduction method of the comparative example, in the human skin tactile sensation presentation device 300, and searched for a virtual texture on the human skin gel 31 within 10 seconds under the same search conditions. Fig. 17(b) is a diagram showing a state in which the vibration data generated based on the acceleration data acquired as in Fig. 17(a) is presented as a tactile sensation by the human skin tactile sensation presentation device 300.

[0122] After the exploration, as in Experiment 2, the subjects were instructed to explore three actual skin samples A, B, and C as shown in Figure 13 and to select the sample that was closest to the virtual texture reproduced by the human skin tactile sensation presentation device 300 immediately before by answering either A, B, or C.

[0123] In this experiment, too, the samples presented to each subject were changed so that the number of samples presented was approximately equal, and the experiment was carried out twice.

[0124] The results are shown in Fig. 18. Fig. 18 is a diagram showing the skin type discrimination results when the tactile sensation was reproduced by a human skin tactile sensation presentation device using vibration data generated by the method of the comparative example. As shown in Fig. 18, in the virtual texture presented by acquiring a skin image and generating data for human skin tactile sensation by the method of the comparative example, the correct answer rate for each sample was about 50%. The correct answer rate decreased as the skin sample became rougher. In particular, the correct answer rate for sample C decreased to 45%, and the correct answer rates for samples B and C showed that subjects tended to perceive the virtual texture as smoother than the sample corresponding to the presented vibration data.

[0125] Comparing the experimental results of Fig. 16 and Fig. 18, the correct answer rate for each sample of the presented virtual texture is higher in Fig. 16. In Fig. 16, the difference in correct answer rate between sample A, which had the highest correct answer rate, and sample B, which had the lowest correct answer rate, is 10%, and the incorrect answer rate of sample B is higher than that of sample C, which had the roughest skin. Therefore, it can be said that the method of the present invention has not only raised the overall reproduction rate of virtual textures, but also reduced the tendency to mistake rough skin for smooth skin.

[0126] Based on these experimental results, it can be said that the human skin tactile data generation method of the present invention makes it possible to more accurately measure differences in tactile sensation by acquiring skin images, minimize errors due to the characteristics of the person touching the skin, and generate human skin tactile data that can be shared by multiple people, is highly reproducible, and has high readability of differences in tactile sensation.

[0127] Furthermore, comparing Experiments 2 and 3, the human skin tactile data generation method of the present invention can collect human skin information by acquiring an image of the skin, without touching the skin or using a special tactile sensor. Due to measures against the new coronavirus since 2020, it has become difficult to touch other people's skin with your fingers to acquire samples or to touch other people's skin with your fingers to evaluate the texture of the skin, but the present invention acquires an image and generates data based on that image, making it possible to acquire human skin tactile data with high reproducibility without actually touching the skin.

[0128] <Second embodiment of device and method for generating data for human skin texture> FIG. 19 is a functional block diagram of a human skin tactile data generating device according to a second embodiment of the present invention.

[0129] As shown in FIG. 19, the human skin tactile data generating device 400 according to the second embodiment includes a two-dimensional photographing means 4, which is an image acquiring section, and an information processing device 5, which is a data generating section.

[0130] The two-dimensional photographing means 4 captures an image of human skin or skin model. In this embodiment, the two-dimensional photographing means 4 is a means capable of photographing two-dimensional images such as skin color or UV photographs, and is, for example, a photographing terminal dedicated to photographing skin with a configurable light amount, a microscope (a photographing device for analyzing skin surface texture, a photographing device for analyzing pore shape), or a camera mounted on a smartphone or the like.

[0131] The information processing device 5 uses the acquired skin image to generate human skin texture data estimated as vibrations that occur when touching human skin or a skin model. The information processing device is, for example, a PC capable of running a specific software or a dedicated two-dimensional skin image analysis device.

[0132] In this embodiment, the information processing device 5 is operable to include a luminance information calculation unit 51, a surface roughness information acquisition unit 52, an acceleration information conversion unit 53, and a human skin tactile data output unit (output unit) 54.

[0133] The luminance information calculation unit 51 calculates the luminance of each position on the skin surface in a two-dimensional image of human skin or a skin model using luminance extraction software or the like.

[0134] The surface roughness information acquisition unit 52 includes a brightness / cross-sectional height conversion unit 521 and a displacement curve acquisition unit 522 .

[0135] The luminance / cross-section height conversion unit 521 calculates the skin surface height (height of the skin cross section) at each position in the two-dimensional image based on the luminance.

[0136] The displacement curve acquisition unit 522 assumes that a straight line in a two-dimensional image is scanned a predetermined distance, and acquires a displacement curve on the straight line as surface roughness information by converting the brightness of the skin surface on that straight line in the two-dimensional image into the height of the cross section of the skin at each position on the line, with the horizontal axis being distance.

[0137] 2, and converts the skin surface roughness information acquired by the surface roughness information acquisition unit 52 into acceleration information. The acceleration information conversion unit 53 has a displacement spectrum conversion unit 531, an acceleration spectrum conversion unit 532, and an acceleration information correction unit 533.

[0138] The output unit 54 sets the acceleration information converted by the acceleration information conversion unit 53 as human skin tactile data (human skin expression data) and outputs it.

[0139] In addition, in FIG. 19, a configuration is shown in which the functions of the luminance information calculation unit 51, the surface roughness information acquisition unit 52, the acceleration information conversion unit 53, and the output unit 54 are realized by one information processing device 5, but these calculation functions may be executed using two or more information processing devices.

[0140] (Data generation process of the second embodiment) Details of the data generation process executable by the information processing device shown in Fig. 19 will be described with reference to Fig. 20 and Fig. 21. Fig. 20 is a detailed flow of the data generation process according to the second embodiment. Only the differences from the flow in Fig. 3 will be described.

[0141] In S201, brightness information of the skin surface is calculated from the acquired two-dimensional skin image.

[0142] In S202, surface roughness information is estimated based on the skin brightness information.

[0143] In S203, a displacement curve is obtained from the skin surface roughness information calculated based on the luminance information, assuming scanning in a direction on a straight line. More specifically, assuming scanning a predetermined distance on a straight line in a two-dimensional image, the luminance of the skin surface on the straight line in the two-dimensional image is converted into the height of the cross section of the skin at each position on the straight line with the horizontal axis being the distance, and a displacement curve on the straight line is obtained as surface roughness information.

[0144] Here, in FIG. 21, (a) is an example of a luminance image of human skin, and (b) and (c) are examples of displacement curves using luminance calculated in the second embodiment.

[0145] In S202, the luminance information of each position on the line of the two-dimensional image is replaced with the cross-sectional height of the skin surface. For example, the acquired two-dimensional image data provides luminance information for each pixel as position information, as shown in Fig. 21(b), so the luminance for each pixel is replaced with the cross-sectional height of the skin surface. In the graph of Fig. 21(b), the horizontal axis indicates pixels and the vertical axis indicates luminance.

[0146] In this example, the conversion from the brightness of each pixel to the cross-sectional height of the skin surface was carried out by subtracting the average brightness value of the analysis area from the brightness value of each pixel to obtain the difference, and then multiplying each brightness value by 4, assuming that a brightness of 1 corresponds to a cross-sectional height of 4 μm.

[0147] In the conversion from Fig. 21(b) to Fig. 21(c), the change in brightness from the average value is converted as 4μm for brightness 1, but the relationship between the actual brightness value and roughness is thought to change depending on how the lighting is applied and the distance from the camera. Therefore, when using the same conversion formula, it is preferable to set the lighting method and the distance and angle between the subject and the camera to be constant.

[0148] In this example, the average brightness value is set to the cross-sectional height of the skin surface of 0. However, in the case of an image in which there is a deviation in the spread of brightness, the median or most frequent value of the spread of brightness may be set to the cross-sectional height of the skin surface of 0.

[0149] In this way, any suitable conversion formula may be used for converting the luminance information into the height of the skin surface as long as the conditions are consistent between the skins to be compared. Furthermore, the conversion formula may be changed whenever the photographing conditions are changed.

[0150] And, Fig. 21(c) is the graph of brightness for each pixel in Fig. 21(b) with the horizontal axis replaced with length. In this example, Fig. 21(b) was converted to Fig. 21(c) based on the relationship that the horizontal axis is 640 pixels = 6.7 mm.

[0151] In this way, using the displacement curve calculated via brightness based on the two-dimensional image, in S203 to S206, similar to S23 to S26 in Fig. 3, it is converted into a spectrum of the displacement curve, and then converted into an acceleration spectrum, and after that, it is corrected with a low-pass filter, and the human skin tactile data can be obtained.

[0152] In this embodiment, the two-dimensional image is converted into the height of the skin surface via the brightness to obtain the data for human skin tactile sensation. Therefore, if the data generating unit of the device for generating data for human skin tactile sensation has software that can obtain the brightness of the image, it is possible to obtain the image using a more general-purpose camera such as a smartphone camera without using an expensive three-dimensional imaging means, and generate the data for human skin tactile sensation based on the image.

[0153] Therefore, since human skin tactile data can be generated even when a skin information provider such as a subject is not present and a high-performance camera is not available, this embodiment can realize a human skin tactile data generating device that reduces the burden on the subject, is easier to put into practical use, and is more familiar to users.

[0154] (Variation 1) In addition, in Figs. 20 and 21, an example of calculating skin roughness information from a two-dimensional image in the human skin tactile data generation method of the present invention has been described, but data may be generated without using luminance by acquiring a special two-dimensional image, for example.

[0155] For example, a two-dimensional UV camera (e.g., Visio scan (registered trademark): two-dimensional skin surface texture analysis device) is used to acquire images, and the state of "texture and fine wrinkles" is extracted based on clear images of the skin surface morphology captured using UV light on a processing terminal attached to the UV camera. Then, based on the extracted state of texture and fine wrinkles, a displacement curve may be calculated on an information processing terminal other than the dedicated computing terminal.

[0156] (Variation 2) Furthermore, as other methods for obtaining skin roughness information from a two-dimensional image in the human skin tactile data generation method of the present invention, skin unevenness information may be obtained by interfering light when capturing a two-dimensional image, by moving the focus when capturing a two-dimensional image, or by taking a confocal lens when capturing a two-dimensional image.

[0157] <Human skin texture evaluation method> Next, a method for evaluating human skin feel including the method for generating data for human skin feel of the present invention will be described with reference to Figures 22 and 23. Figure 22 is a schematic flow chart showing a method for evaluating human skin feel including the method for generating data for human skin feel of the present invention.

[0158] In step S101, a skin image is acquired. Note that, during evaluation, the skin that is the subject of the image is the actual skin of the subject, customer, or the like.

[0159] In step S102, similar to S2 in FIG. 1, acceleration information is obtained as human skin tactile data based on the surface roughness information of the skin.

[0160] In step S103, a preset evaluation criterion is called (evaluation criterion reference step).

[0161] Then, in S104, the generated human skin tactile data is compared with the evaluation criteria to evaluate / estimate the tactile feel of the acquired image (evaluation step).

[0162] As shown in Fig. 22, in the evaluation step, the data (acceleration information) generated in the data generation step is used to evaluate the feel of the skin. Therefore, it is preferable to establish an evaluation standard in advance as a prerequisite for the evaluation, and to perform the evaluation based on the standard.

[0163] (Example of creating evaluation criteria for tactile evaluation) Here, the evaluation criterion is a relationship between the frequency spectrum of acceleration (vibration information) calculated from the surface roughness information of the skin image or skin model image and the sensory evaluation of the tactile feel of the skin or skin model, which has been constructed and stored in advance.

[0164] Specifically, for images of skin (or human skin models) for reference setting with different surface conditions, or images of multiple human skin gels with different characteristics, acceleration information calculated from the respective surface roughness information is obtained. The skin for reference setting refers to skin whose characteristics have already been clarified and is not unknown.

[0165] On the other hand, the results of a sensory evaluation of the tactile sensation perceived by a human when running a finger sideways across the "reference skin" (or human skin gel) from which surface roughness information has been obtained can be recorded, and the relationship between this data and the sensory evaluation can be compiled into a database.

[0166] When evaluating the feel of unknown skin, the sensory evaluation value of the feel of the unknown skin can be estimated by comparing the acceleration information calculated from the surface roughness information detected by the same method for the unknown skin with the above-mentioned relationship (database) that has been constructed in advance.

[0167] The relationship (evaluation criterion) between the values ​​relating to the vibration magnitude and characteristics of the human skin tactile data and the sensory evaluation can be established, for example, as follows.

[0168] In this case, for a known skin (skin model) with multiple understood characteristics, human skin tactile data (acceleration spectrum when scanning a certain distance) is obtained according to the human skin tactile data generation method of the present invention, and the relationship with the sensory evaluation is examined.

[0169] Specifically, images (three-dimensional or two-dimensional images) are acquired for multiple known skin models for standard setting according to the human skin tactile data generation method of the present invention, and data is generated based on the images to acquire multiple human skin tactile data (acceleration information) that imitate each of the known skin models. Then, a fast Fourier transform (FFT) is performed on the acquired acceleration information to calculate an amplitude spectrum from 0 Hz to 500 Hz. The calculated amplitude spectrum is then averaged for every 50 Hz frequency, and a statistical model is estimated using the average value of this spectrum as an independent variable (vibration feature).

[0170] The number of skins or skin samples used to set multiple standards for constructing an evaluation standard is preferably at least 10 or more, and the greater the number of samples, the more preferable.

[0171] Then, a statistical model is calculated from the vibration feature obtained above to estimate the moistness and smoothness of the skin. "Moist" skin means that the skin is neither sticky (excessive oil) nor dry, and has an appropriate amount of moisture (a moisturized state), and "smooth" skin means that the skin has less open pores and less noticeable acne to the touch, and has a smooth surface texture. Note that the sensory evaluation of the skin, such as "moist" and "smooth," is just one example, and as another example, a statistical model may be calculated that numerically evaluates the skin condition based on a skin moisture meter or a skin oil meter.

[0172] On the other hand, for each of the above-mentioned multiple reference skins, a human (expert panel) may perform a sensory evaluation on the evaluation items for the virtual texture presented by the above-mentioned human skin tactile sensation presentation device 300. The sensory evaluation by the expert panel may be an evaluation of the tactile sensation of the skin felt when experiencing the virtual texture presented by the human skin tactile sensation presentation device 300. The sensory evaluation value by this panel is preferably the calculated average value of 10 or more people, and the more people who evaluate in order to take the average of the calculated average values, the more preferable.

[0173] The statistical model and sensory evaluation obtained as described above become the evaluation standard for evaluating the reproduced tactile sensation of skin. In other words, if an evaluation standard is constructed by detecting and analyzing the human skin tactile data (acceleration information) for each of a number of known skins (reference skins) and conducting a sensory evaluation by a specialist panel, data based on the detected human skin tactile data (acceleration information) for an unknown skin can be obtained and applied to the constructed evaluation standard to estimate the sensory evaluation score of the human skin tactile data for the unknown skin. In this case, the human skin tactile data for the unknown skin is generated under the same conditions as the human skin tactile data generation of the present invention performed when obtaining the evaluation standard.

[0174] (Human skin texture evaluation device) FIG. 23 is a block diagram of a human skin texture evaluation device for implementing the human skin texture evaluation method of the present invention.

[0175] As shown in FIG. 23, the human skin texture evaluation device 600 comprises a three-dimensional imaging means 1 which is an image acquisition section, and an information processing device 6 which is a data generation and evaluation section.

[0176] Note that, while FIG. 23 shows an example in which the human skin tactile evaluation device 600 realizes processes from image acquisition to human skin tactile data generation by the method of the first embodiment, the human skin tactile evaluation device of the present invention may also realize processes from image acquisition to human skin tactile data generation by the method of the second embodiment or the method of the comparative example.

[0177] The three-dimensional imaging means 1 captures a three-dimensional image of human skin or skin that is a skin model.

[0178] The information processing device 6 uses the acquired skin image to generate and evaluate human skin texture data estimated as vibrations generated when touching human skin or a skin model. The information processing device 6 is, for example, a PC capable of running a specific software or a dedicated three-dimensional skin image analysis device.

[0179] The information processing device 6 executablely includes a surface roughness information acquisition unit 61, an acceleration information conversion unit 62, a reference data storage unit 63, an evaluation unit 64, and an evaluation result output unit 65.

[0180] 2, the surface roughness information acquisition unit 61 acquires skin surface roughness information from an image of the skin. In detail, the surface roughness information acquisition unit 61 has a surface roughness extraction unit 611 that extracts the skin surface roughness from the three-dimensional image of the skin, and a displacement curve acquisition unit 612 that acquires a displacement curve.

[0181] The acceleration information conversion section 62 converts the skin surface roughness information into acceleration information. The acceleration information conversion section 62 has a displacement spectrum conversion section 621, an acceleration spectrum conversion section 622, and an acceleration information correction section 623.

[0182] The reference data storage unit 63, which is a storage unit, pre-stores acceleration information of multiple types of skin associated with multiple types of human skin or skin models having set characteristics as a standard for skin feel, as a sensory evaluation standard.

[0183] The evaluation unit 64 evaluates the current skin feel based on the relationship between the acceleration information calculated from the current skin image calculated by the acceleration information conversion unit 62 and the sensory evaluation that serves as the standard for the skin feel stored in the reference data storage unit 63.

[0184] The evaluation result output unit (evaluation output unit) 65 is a display unit or a notification unit, and outputs the current skin evaluation result by displaying or transmitting it.

[0185] 23 shows a configuration in which the functions of the surface roughness information acquisition unit 61, the acceleration information conversion unit 62, the reference data storage unit 63, the evaluation unit 64, and the evaluation result output unit 65 are realized by one information processing device 6, but these calculation functions may be executed by two or more information processing devices. Also, the reference data storage unit 63 may be realized by a recording medium separate from the information processing device that performs the calculations.

[0186] In the human skin tactile sensation device of the present invention, by using the human skin tactile sensation data generated as described above for evaluation, it is possible to evaluate the skin tactile sensation that can be shared by multiple people with little error and high readability, without actually touching the skin.

[0187] <Application Example 1> In the above, a method and configuration for applying the human skin tactile data generated using the skin image of the present invention to both presentation and evaluation has been described, but the human skin tactile data may also be used in a human skin tactile evaluation and presentation device for both presentation and evaluation.

[0188] The human skin tactile sensation evaluation and presentation device will be described with reference to FIG. 24. FIG. 24 is a schematic diagram showing a beauty advisor presenting skin quality to a customer during counseling using the human skin tactile sensation evaluation and presentation device 700 of the present invention. Note that while FIG. 24 shows an example in which each component of the human skin tactile sensation evaluation and presentation device 700 is connected by wire, each component may be connected wirelessly by short-range communication. Note that in this example, either a three-dimensional image or a two-dimensional image may be acquired as the skin image, and the human skin tactile sensation data may be generated by any of the methods of the first embodiment, the second embodiment, and the modified example.

[0189] For example, in the human skin tactile sensation evaluation and presentation device 700, in order to reproduce normal skin conditions, desired skin conditions, etc., multiple pieces of human skin tactile sensation presentation data (e.g., waveforms as shown in FIG. 9(b)) corresponding to original vibration signals corresponding to multiple skin types are acquired in advance and stored in the PC 33.

[0190] Here, the beauty advisor BA inputs the selection of the skin type to be presented to the subject S (customer) so as to reproduce the feel of bare skin or the feel of skin after application of cosmetics, beauty products, oils, etc. Then, in the presentation mechanism 30 of the human skin feel evaluation and presentation device 700, human skin feel presentation data that serves as the original signal for vibration according to the selected skin type is set, and vibration is presented to the subject's (customer's) finger via the transmission plate 36 and human skin gel 31.

[0191] As an example, the beauty advisor BA acquires an image of the skin of the customer subject, and reproduces the current skin condition of the subject using the presentation mechanism 30 of the human skin feel evaluation and presentation device 700 and presents it to the subject's finger.

[0192] Furthermore, the beauty advisor BA acquires an image of the skin of the customer subject using the image acquisition unit 1, and evaluates the current skin condition of the subject using the evaluation mechanism 60 of the human skin texture evaluation and presentation device 700, and displays the evaluation result to the subject. Alternatively, when the beauty advisor's touch judgment level is immature or when the beauty advisor cannot directly touch the customer's skin due to COVID-19 countermeasures, it is effective to display a touch score using the evaluation criteria along with the touch presentation as a form of support.

[0193] Furthermore, ideal skin or a desired skin condition may be reproduced by the presentation mechanism 30 and presented to the subject's finger. Specifically, a state that can be achieved by using a specific cosmetic is reproduced by the presentation mechanism 30 by, for example, making the reference liquid R into drops of human skin gel and presenting it to the subject's finger. If the surface of the cosmetic changes over time when used, for example, by dissolving over time due to body temperature, the skin after 10 minutes, 20 minutes, or 30 minutes can be reproduced and presented by the human skin tactile sensation presentation device even if the time has not yet arrived. Alternatively, if it is desired to explain the effects of skin that appear when the cosmetic is used continuously, the skin after 1 day, 3 days, 1 week, 2 weeks, etc., can be reproduced and presented by the presentation mechanism 30 by continuously using the cosmetic, for example, twice a day, even if the time has not yet arrived.

[0194] In this way, it becomes possible to suggest the future skin condition of the subject when applying the cosmetic product by using the human skin tactile sensation presentation device, and by stimulating the sense of touch in addition to the visual sensation of a pamphlet, it is possible to appeal to the sensibilities of the subject customer in a multifaceted way. This allows the customer to gain a deeper understanding of their skin condition. Meanwhile, the beauty advisor can give advice that corresponds to the reproduced skin quality, which can actively promote purchases.

[0195] 24 shows an example in which a beauty advisor proposes to a customer the human skin tactile evaluation and presentation device 700 of the present invention, but the human skin tactile evaluation and presentation device 700 may also be used for training beauty advisors. For example, in the human skin tactile evaluation and presentation device 700, the beauty advisor can practice taking pictures using the image acquisition unit 1, and the tactile sensation of a reference skin quality can be presented, and the beauty advisor can answer the tactile sensation and also display an evaluation score showing the degree of agreement, thereby training the beauty advisor on the tactile sensation of the fingertips.

[0196] <Application Example 2> In FIG. 24 above, the human skin tactile sensation evaluation and presentation device 700 is realized in one place by a wired connection or a short distance connection, but the functions included in the human skin tactile sensation evaluation and presentation device 700 may be realized remotely as a system via a network.

[0197] FIG. 25 is a schematic diagram of a human skin texture evaluation and presentation system using the human skin texture evaluation method of the present invention.

[0198] In one example of a human skin texture evaluation and presentation system 800, a subject takes pictures using a camera mounted on a general smartphone, tablet, mobile phone, etc. as an image acquisition and communication terminal 81, transmits the acquired images via a network N, and is connected to an advice PC 82 and / or an analysis PC 83. In this example, since the subject has a general camera in hand, human skin texture data is generated using the method of the second embodiment.

[0199] The advising PC 82 calculates human skin texture data based on the skin image information sent from the subject's image acquisition / communication terminal 81, and sends comments such as advice to the image acquisition / communication terminal 81 of the subject S by referring to the correlation between the stored evaluation criteria and the vibration texture. The advising PC 82 may be operated manually by the beauty advisor BA, or it may be programmed to automatically respond by incorporating advice software inside the advising PC 82.

[0200] When the beauty advisor BA performs remote counseling, the tactile sensation can be understood by outputting vibration information of unknown skin from the presentation mechanism 30B, so there is no need to refer to the evaluation criteria.

[0201] The analytical PC 83 has a compilation software for collecting and analyzing the answers from the subjects built in. The advising PC 82 and the analytical PC 83 may be a single PC.

[0202] In this way, by subject S sending an image taken using a smartphone or the like, it becomes possible to share the tactile sensation of the skin image with multiple people in different spaces.

[0203] Alternatively, the input terminal 84 of the human skin texture evaluation and presentation system 800 of the present invention may be installed in public spaces such as shopping centers, rest areas, etc., so that it can be used without a beauty advisor. When a dedicated input terminal 84 is installed, either a three-dimensional image or a two-dimensional image may be acquired by the image acquisition unit 1, and the human skin texture data may be generated by any of the methods of the first embodiment, the second embodiment, and the modified example.

[0204] The input terminal 84, which is placed in a public space and connected to the image acquisition unit 1 and presentation mechanism 30A, has built-in functions of the information processing device 6 of the human skin tactile sensation evaluation device 600 and the PC 33 of the human skin tactile sensation presentation device 300, and can immediately calculate vibrations (human skin tactile sensation data) based on the currently captured image. This makes it possible to present one's own skin condition at this time using the presentation mechanism 30A, present reference human skin tactile sensation reproduction data as an evaluation standard using the presentation mechanism 30A, and output evaluation results that have been compared with the evaluation standard using the current human skin tactile sensation data.

[0205] Furthermore, it is preferable that the input terminal 84 has a built-in guidance program and / or advice program so that the input terminal 84 can be used comfortably without a beauty advisor.

[0206] In addition, when an input terminal 84 is provided, the accumulated data on the skin conditions of multiple subjects stored in the input terminal 84 can be transferred to the analytical PC 83 via the network N at predetermined intervals and used for analysis by the cosmetics developer CR.

[0207] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0208] This international application claims priority to Japanese Patent Application No. 2020-200573, filed on December 2, 2020, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0209] 1. 3D imaging means (image acquisition unit) 2. Information processing device (data generation unit) 4. 2D imaging methods 5. Information processing device (data generation unit) 6. Information processing device (data generation and evaluation section) 21,52 Surface roughness information acquisition unit 22,543 Acceleration information conversion section 23,54 Data output unit for human skin touch (output unit) 51 Luminance information calculation unit 30 Presentation Mechanism 31 Human Skin Gel 32 Slide volume (finger movement section, movement speed acquisition section) 33 PC (original signal storage section, vibration adjustment section) 34 Amplifier (vibration output section) 35 (35A, 35B) Speaker (vibration output section) 36 Transmission plate (transmission component) 61 Surface roughness information acquisition unit 62 Acceleration information conversion unit 63 Reference data storage unit (storage unit) 64 Evaluation Section 65 Evaluation result output unit (evaluation output unit) 100 Data generation device for human skin touch 300 Human skin touch display device 400 Data generation device for human skin touch 600 Human skin texture evaluation device 700 Human skin texture evaluation and presentation device 800 Human skin texture evaluation and presentation system

Claims

1. acquiring an image of skin, the image being human skin or a skin model; and generating human skin tactile data estimated as vibrations that occur when the human skin or skin model is touched using the acquired skin image. A method for generating data for human skin touch.

2. The step of generating the human skin tactile data includes: obtaining skin surface roughness information from the skin image; converting the skin surface roughness information into acceleration information; and setting the acceleration information as the human skin tactile data. The human skin tactile data generating method according to claim 1 .

3. In the step of acquiring the skin image, a three-dimensional image of a surface of human skin or a skin model is acquired as the skin image by a laser microscope; In the step of acquiring skin surface roughness information from the skin image, Assuming that a straight line of the three-dimensional image of the surface of the human skin or skin model is scanned for a predetermined distance, a displacement curve on the straight line, which corresponds to the cross-sectional height position of the three-dimensional skin with the horizontal axis being distance, is obtained as the surface roughness information. The human skin tactile data generating method according to claim 2.

4. In the step of acquiring the skin image, a color or black and white two-dimensional image of a surface of human skin or a skin model is acquired as the skin image by photography; In the step of acquiring skin surface roughness information from the skin image, calculating a luminance of a skin surface in the two-dimensional image of the human skin or skin model; and acquiring, as the surface roughness information, a displacement curve on the line, which is obtained by converting the brightness of the skin surface on the line in the two-dimensional image into the height of a cross section of the skin at each position on the line, with the horizontal axis being the distance, assuming that the line in the two-dimensional image is scanned for a predetermined distance. The human skin tactile data generating method according to claim 2.

5. In the step of converting the skin surface roughness information into acceleration information, A moving speed when touching the human skin or skin model is assumed, and the horizontal axis of the displacement curve on the straight line, which is the surface roughness information, is converted from distance to time to obtain a spectrum of the displacement curve; The spectrum of the displacement curve is converted into a spectrum of acceleration.

5. The method for generating human skin tactile data according to claim 3 or 4.

6. In the step of converting the skin surface roughness information into acceleration information, The method further includes a step of correcting the acceleration spectrum by applying a low-pass filter to obtain the acceleration information; In the step of setting the acceleration information as human skin tactile data, The acceleration spectrum is applied with a low-pass filter to correct the acceleration information, which is used as the human skin tactile data. The human skin tactile data generating method according to claim 5.

7. The human skin is bare skin or skin to which a cosmetic agent has been applied. The method for generating human skin tactile data according to any one of claims 1 to 6.

8. an image acquisition unit for acquiring an image of human skin or skin that is a skin model; a roughness information acquisition unit that acquires surface roughness information from the skin image; an acceleration information conversion unit that converts the skin surface roughness information into acceleration information; and an output unit that outputs the acceleration information as human skin tactile data. Data generation device for human skin touch.

9. A step of generating human skin tactile data, which is acceleration information, by the human skin tactile data generating method according to any one of claims 1 to 7; and evaluating the tactile sensation of the human skin or skin model using the acceleration information, which is the human skin tactile sensation data. A method for evaluating skin texture.

10. In the step of evaluating the feel of the human skin or the skin model, the feel of the skin is evaluated based on a relationship between the acceleration information and a sensory evaluation that is a reference for the feel of the skin, which is previously obtained. The method for evaluating the feel of skin according to claim 9.

11. The human skin tactile data generating device according to claim 8 ; an evaluation unit that evaluates the tactile sensation of the human skin or a skin model using the acceleration information, which is the human skin tactile data generated by the human skin tactile data generation device; and an evaluation output unit that outputs the evaluation of the tactile sensation. Human skin texture evaluation device.

12. the evaluation unit evaluates the tactile sensation of the skin based on a relationship between the acceleration information and a sensory evaluation that is a reference for the tactile sensation of the skin, the relationship being determined in advance; The human skin texture evaluation device comprises: The device further includes a storage unit for storing in advance acceleration information of a plurality of types of skin associated with a plurality of types of human skin or skin models having set characteristics as a reference for the skin feel. The human skin feel evaluation device according to claim 11.

13. a presentation mechanism for acquiring the acceleration information, which is the human skin tactile data generated by the human skin tactile data generating device according to claim 8, and presenting the acceleration information as vibration on a human skin model. A human skin tactile sensation presentation device.

14. The presentation mechanism includes: A propagation member through which vibration propagates; a human skin model that is provided on the propagation member and that comes into contact with a user's finger or an artificial finger; a finger moving unit that moves the finger or artificial finger on the human skin model at a predetermined moving speed; a movement speed acquisition unit that acquires a movement speed of the user's finger on the human skin model; an original signal storage unit that stores the acceleration information associated with the acquired skin or skin model as an original signal; A vibration adjustment unit that adjusts the amplitude of the original signal according to the moving speed of the finger or artificial finger; A vibration output unit that converts the adjusted original signal into a physical vibration and outputs the physical vibration. The human skin sensation providing device according to claim 13.

15. A step of generating human skin tactile data, which is acceleration information, by the human skin tactile data generating method according to any one of claims 1 to 7; and using the acceleration information, which is the human skin tactile data, to present the acceleration information as vibration on a human skin model. A method for presenting the sensation of human skin.

Images