Tactile sample presentation system and tactile sample presentation method

Abstract

To provide a tactile sample presentation system capable of presenting various kinds of impression to an evaluator.SOLUTION: A tactile sample presentation system 1 includes: a tactile sample 10 including a plurality of material pieces 11 having arbitrary physical properties; a camera 23 configured to image the tactile sample 10; an image database 42 configured to hold a plurality of texture images D1; an impression word acceptance part 44 configured to accept an impression word expressing a desired impression; a recommended combination selection part 45 configured to select a combination to be recommended from among the material pieces 11 and the texture images D1 according to the impression word; an image processing part 41 configured to generate a texture sample image D3 corresponding to a material piece 11 from a texture image D1 selected by the recommended combination selection part 45; and an image presentation part 43 configured to present a cross-reality tactile sample 50 with the texture sample image D3 imparted to the material piece 11 selected by the recommendation combination selection part 45.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a tactile sample presentation system and a tactile sample presentation method. [Background technology]

[0002] In the housing equipment, interior design, and fashion industries, material samples (touch samples), which are actual pieces of materials directly attached to a mount, are used during business negotiations, etc. They are highly valued because both the provider and the purchaser can see and touch the material sample to directly check the appearance and feel of the material. However, in order to create such material samples, it is necessary to procure the actual materials, which increases the time and cost required for production. Therefore, a material sample made of a printed material that imitates the texture of the actual material has been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-67468 A [Patent Document 2] JP 2003-223108 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, not only have the materials used in real life become more diverse, but the tastes of evaluators (purchasers) have also become more diverse. For this reason, the demand for material samples with a wide variety of material pieces is increasing year by year. Even with the same material, different colors can change the tactile sensation and impression. There are an infinite number of combinations of material appearances and textures. In order to respond to the preferences of evaluators, it is necessary to present a wide variety of material pieces. However, in conventional material samples, there is a limit to how many material pieces can be increased, and it is difficult to present material pieces that correspond to the evaluator's preferences, etc. In particular, it is difficult to present material pieces that express the evaluator's desired sensation with a high degree of accuracy. Therefore, an object of the present invention is to provide a tactile sample presentation system and a tactile sample presentation method that can present a wide variety of sensations to an evaluator. [Means for solving the problem]

[0005] The present invention encompasses the following aspects. [1] A tactile sample presentation system comprising: a tactile sample consisting of a plurality of material pieces having arbitrary physical properties; a camera for capturing images of the tactile sample; an image database for holding a plurality of texture images having visual impression elements; an impression term receiving unit for receiving impression terms expressing a desired impression; a recommended set selection unit for selecting a recommended combination from among the material pieces and the texture images based on the impression terms received by the impression term receiving unit; an image processing unit for generating a texture sample image corresponding to the material pieces from the recommended texture images selected by the recommended set selection unit; and an image presentation unit for presenting a cross-reality tactile sample in which the texture sample image is applied to the recommended material pieces selected by the recommended set selection unit. [2] The tactile sample presentation system according to [1], wherein the recommended set selection unit determines the combination based on material impression information indicating the degree of conformity of the material piece with the impression term and image impression information indicating the degree of conformity of the texture image with the impression term. [3] The tactile sample presentation system according to [1] or [2], further comprising a camera for capturing an image of the tactile sample, the image presentation unit being a display for displaying the tactile sample image acquired from the camera, and the display presenting an AR tactile sample in which the texture sample image is superimposed on the tactile sample image. [4] The tactile sample presentation system according to [1] or [2], wherein the image presentation unit is a video-transmitting or optically-transmitting head-mounted display, and the head-mounted display presents an MR tactile sample in which the texture sample image is superimposed on a transmitted image of the tactile sample. [5] The tactile sample presentation system according to any one of [1] to [4], wherein the image processing unit generates a composite texture image by alpha-blending the two texture images. [6] The tactile sample presentation system according to [5], wherein one of the texture images is a color image and the other is a pattern image. [7] The tactile sample presentation system according to any one of [1] to [6], wherein the image processing unit performs a trimming process on the area of ​​the texture sample image that is hidden by the evaluator's finger when the evaluator touches the tactile sample. [8] The tactile sample presentation system according to any one of [1] to [7], wherein the tactile sample is at least partially made of resin, the outer periphery of the sample is shape-stably maintained, and the front and back surfaces are released to form a tactile confirmation area. [9] A tactile sample presentation method comprising: a tactile sample presentation step of presenting a tactile sample consisting of a plurality of material pieces having arbitrary physical properties; an imaging step of imaging the tactile sample with a camera; an impression term receiving step of receiving an impression term expressing a desired impression; a recommended set selection step of selecting a recommended combination from among the material pieces and texture images having a plurality of visual impression elements held in an image database based on the impression term received by the impression term receiving unit; an image processing step of generating a texture sample image corresponding to the material pieces from the recommended texture image selected in the recommended set selection step; and an image presentation step of presenting a cross-reality tactile sample in which the texture sample image is applied to the recommended material pieces selected in the recommended set selection step. Effect of the Invention

[0006] According to the present invention, a tactile sample presentation system and a tactile sample presentation method capable of presenting a wide variety of sensations to an evaluator can be provided. In particular, a tactile sample (material piece) that expresses a sensation desired by the evaluator can be provided. [Brief description of the drawings]

[0007] [Figure 1]1 is a schematic diagram showing a schematic configuration of a tactile sample presentation system 1. FIG. [Diagram 2] FIG. 1 shows a tactile sample 10. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer 20. [Figure 4] FIG. 2 is a diagram illustrating an example of the software configuration of a computer 20. [Diagram 5] FIG. 11 is a flow chart showing a tactile sample presentation method. [Figure 6] FIG. 13 is a diagram illustrating the impression term receiving step S3. [Figure 7] This is a diagram showing impression term W, which is an example of a term used for residential building materials. [Figure 8] 13A is a diagram showing material impression information P2, (B) image impression information P3, and (C) a matrix multiplication calculation result. FIG. [Figure 9] FIG. 13 is a diagram showing (A) an image processing step S5 and (B) an image presentation step S6. [Figure 10] FIG. 13 is a diagram showing image processing when the texture sample image D3 and the evaluator's finger overlap. [Figure 11] FIG. 13 is a diagram showing tactile sample 13. [Figure 12] 1A and 1B are diagrams showing a tactile sample 15, in which (A) is a plan view and (B) is a cross-sectional view. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The present invention will be described in detail below. Note that the following description of the constituent elements is merely an example for explaining the present invention, and the present invention is not limited to the contents thereof.

[0009] [Tactile sample presentation system] FIG. 1 is a schematic diagram showing the general configuration of a tactile sample presentation system 1. As shown in FIG. The tactile sample presentation system 1 of this embodiment uses AR technology, and includes a tactile sample 10 and a computer 20.

[0010] <Texture sample 10> FIG. 2 is a diagram showing the tactile sample 10. As shown in FIG. The tactile sample 10 is a material sample whose surface is directly touched by an evaluator with his / her finger to feel (sense) the tactile sensation. The texture sample 10 has a plurality of material pieces 11 formed, for example, in the shape of a quadrilateral sheet or plate. The material pieces 11 are attached to a mount such as cardboard and placed on a flat surface such as a table. For example, three material pieces 11 (material pieces 11A to 11C) are arranged in a row. The evaluator may hold the tactile sample 10 in his / her hand or change the position and orientation of the tactile sample 10 as long as it is within the range (angle of view) that can be captured by a camera 23, which will be described later. It should be noted that the tactile sample 10 is not limited to a plurality of material pieces 11 arranged in a line.

[0011] The material piece 11 is, for example, a building material for a house, and the material (raw material) thereof is arbitrary. The material of the material pieces 11 is, for example, polyurethane. The material pieces 11 may also be polyethylene, polypropylene, ethylene-propylene copolymer, polybutadiene, polyvinyl acetate, polyester, polyacrylic, polyamide, polystyrene, or the like. The material pieces 11 may also be viscoelastic polymers such as natural rubber and latex, gel-like substances, or the like. The material pieces 11 may also be paper, wood, metal, plaster, diatomaceous earth, mortar, rocks, or the like. The physical properties of the portion (surface) of the material piece 11 that the evaluator touches with his / her finger are arbitrary. The physical properties are physical properties of the material piece 11, such as mass, density, volume, surface properties, elastic modulus (Young's modulus), thermal conductivity, refractive index, etc. Electrical and magnetic properties are also included. Depending on the physical properties of the material piece 11, an evaluator who touches the surface with his / her finger can feel various tactile sensations (memorize them). When the material piece 11 is made of polyurethane, the Young's modulus can be changed by changing the degree of cross-linking of the urethane, and the softness of the material when touched can be changed.

[0012] The size (area) of material piece 11 is also arbitrary. For example, when checking the tactile sensation by rubbing the surface of material piece 11 with a fingertip, the size may be slightly larger than the index finger of an evaluator (length 30 mm, width 50 mm). For example, when checking the tactile sensation by pinching the material piece 11 with the fingertips, the material piece needs to be large enough to be pinched between the evaluator's thumb and index finger (50 mm long, 50 mm wide). The thickness of the material piece 11 is preferably 50 μm or more and 20 mm or less. If the material piece 11 has a thickness in this range, the evaluator can touch or pinch it with his / her fingers.

[0013] AR markers 12 are attached to the four corners of the mount on which the material piece 11 is attached, and serve as marks for detecting the position and orientation of the material piece 11. The AR marker 12 is preferably a designated marker that reads a dedicated marker image, but may also be a free marker that uses a free design. When a designated marker is used as the AR marker 12, for example, a marker called a ChArUco board can be used. Also, a marker consisting of a square black frame and a predetermined pattern drawn inside the black frame can be used.

[0014] <Computer 20 (hardware configuration)> FIG. 3 is a diagram illustrating an example of a hardware configuration of the computer 20. As shown in FIG. The computer 20 is, for example, a desktop personal computer, and is placed near the tactile sample 10 . The computer 20 includes a control unit 21, an operation unit 22, a camera 23, a display 24, and the like.

[0015] The control unit 21 includes a CPU 31 , a display I / F 37 , and the like, and controls the computer 20 in an integrated manner. The CPU 31 reads out a control program stored in the ROM 32, and executes and controls various functions of the computer 20, such as image capture, image processing, image synthesis, image projection, and the like. The RAM 33 is used as a temporary storage area such as a main memory and a work area for the CPU 31 .

[0016] The HDD 34 is a large-capacity storage unit that holds (stores) a plurality of image data and various programs. The HDD 34 stores a wide variety of texture images D1. The texture image D1 is an image having visual impression elements, and includes, for example, a color image D1a and a pattern image D1b. The visual impression elements are color, pattern, gloss, texture, etc., and give an impression when touched by its appearance (visual sensation). For example, red is an element that gives the observer a sense of warmth. The color image D1a is an image having a large number of colors represented by the three attributes of color: hue, lightness, and saturation. The color image D1a includes not only chromatic images but also achromatic images. The pattern image D1b is an image that imitates the appearance (surface texture) of various materials. For example, it is an image having irregular patterns such as unevenness, wood grain, ripples (water surface), marble, etc., and geometric patterns such as checks, stripes, and dots. In addition, it may be an image that includes not only one pattern, but also multiple patterns. If the visual impression elements of the texture image D1 deviate too far from the physical properties (such as unevenness) of the sample piece 11, the evaluator will feel uncomfortable when touching the sample piece 11. For this reason, it is desirable that the visual impression elements of the texture image D1 are similar to the physical properties of the sample piece 11. The HDD 34 also holds (stores) the sensory parameters P1, material impression information P2, and image impression information P3. These parameters and information will be described later.

[0017] The operation unit I / F 35 is an interface that connects the operation unit 22 and the control unit 21. The operation unit 22 is equipped with a mouse, keyboard, etc., and accepts operations / inputs / instructions from the evaluator.

[0018] The camera I / F 36 is an interface that connects the camera 23 and the control unit 21. The camera 23 captures an image of the imaging target (the tactile sample 10) and inputs image data of the imaging target (tactile sample image Dc) to the control unit 21 via the camera I / F 36. The camera 23 is fixedly disposed above the tactile sample 10.

[0019] The display I / F 37 is an interface that connects the display 24 and the control unit 21 . The display 24 displays an image (tactile sample image Dc) of the tactile sample 10 captured by the camera 23. The display 24 is fixedly disposed between the evaluator and the tactile sample 10. The evaluator is allowed to touch the tactile sample 10 (material piece 11) with his / her fingertips while viewing the tactile sample image Dc displayed on the display 24.

[0020] <Computer 20 (Software Configuration)> FIG. 4 is a diagram showing an example of the software configuration of the computer 20. As shown in FIG. The computer 20 includes an image processing unit 41, a data management unit 42, a display control unit 43, an impression term reception unit 44, a recommended set selection unit 45, and the like.

[0021] The image processing unit 41 appropriately instructs various processes to the data management unit 42 and the display control unit 43. In particular, the image processing unit 41 processes and modifies (renders) the image (texture image D1) stored in the HDD 34 and the image (touch sample image Dc) captured by the camera 23. The image processing section 41 functions as the image processing section of the present invention.

[0022] The image processing unit 41 is a so-called AR engine, which is software for expressing AR (Augmented reality). It is sometimes expressed as an AR framework, and commercially available software is also available. For example, OpenCV, an open source library developed and published by Intel Corporation in the United States, can be used. The image processing unit 41 generates an image expressing AR (texture sample image D3). Specifically, the image processing unit 41 selects (selects) a predetermined texture image D1 from the texture images D1 stored in the HDD 34, and processes (or otherwise processes) the texture image D1 to generate a texture sample image D3 that corresponds (superimposes) to the captured image of the tactile sample 10 (tactile sample image Dc). When generating the texture sample image D3, a composite texture image D2 obtained by alpha-blending two texture images D1a and D1b can also be used. In other words, the texture sample image includes not only one texture image D1 that has been processed, but also two texture images D1 that have been processed and synthesized. One of the two texture images is a color image D1a, and the other is a pattern image D1b.

[0023] Furthermore, when the evaluator touches the surface of the tactile sample 10, the image processing section 41 processes the texture sample image D3. When the evaluator's fingertip touches the surface of the material piece 11, the fingertip and the texture sample image D3 overlap, that is, the texture sample image D3 is projected onto the upper surface of the fingertip. Therefore, the area of ​​the texture sample image D3 that overlaps with the evaluator's finger (where the surface of the material piece 11 is hidden by the finger) is trimmed, so that the texture sample image D3 is not projected onto the upper surface of the fingertip.

[0024] The data management section 42 stores and manages information relating to the tactile sample 10, information relating to images stored in the HDD 34, and various settings and parameter information (sensitivity parameters P1, material impression information P2, image impression information P3), and the like. The data management unit 42 cooperates with the HDD 34 to function as the image database of the present invention.

[0025] The display control unit 43 displays various images on the display 24 based on instructions from the image processing unit 41. The display control unit 43 displays images generated by the image processing unit 41 and images captured by the camera 23. The display control unit 43 cooperates with the display 24 to function as an image presentation unit of the present invention.

[0026] The impression term receiving unit 44 receives impression terms W from the evaluator. The impression term receiving unit 44 displays a plurality of impression terms W on the display 24 and allows the evaluator to select a term (word) expressing a desired impression from among them. The evaluator operates the operation unit 22 to input the impression terms W to the impression term receiving unit 44. The impression term receiving unit 44 cooperates with the operation unit 22 to function as the impression term receiving unit of the present invention. The impression term W is a word that expresses the desired impression that the evaluator has of the tactile sample 10 (material piece 11). The impression term W will be described later.

[0027] The recommended set selection unit 45 selects a recommended combination from among the material pieces 11 and texture images D1 based on the impression term W received by the impression term reception unit 44. One is selected from the plurality of material pieces 11, and one is selected from the plurality of texture images D1. As the texture image D1, one each of the color image D1a and the pattern image D1b may be selected. The recommended combination selection unit 45 determines a recommended combination based on the material impression information P2 regarding each material piece 11 and the image impression information P3 regarding each texture image D1. The recommended set selection unit 45 functions as a recommended set selection unit of the present invention.

[0028] [Method of presenting tactile samples] The tactile sample provision method of this embodiment is carried out using the tactile sample provision system 1 described above. FIG. 5 is a flow chart showing a tactile sample presentation method. FIG. 6 is a diagram for explaining the impression term receiving step S3. FIG. 7 is a diagram showing impression terms W, examples of which are used for residential building materials. FIG. 8(A) is a diagram showing material impression information P2, (B) is image impression information P3, and (C) is a diagram showing the matrix multiplication calculation result. FIG. 9(A) is a diagram illustrating the image processing step S5, and FIG. 9(B) is a diagram illustrating the image presentation step S6.

[0029] The tactile sample presentation method includes a tactile sample presentation step S1, an imaging step S2, an impression term reception step S3, a recommended set selection step S4, an image processing step S5, and an image presentation step S6. The recommended group selection step S4 includes a matrix product calculation S41 and a ranking and selection S42. The image processing step S5 includes a texture image synthesis step S51 and an AR image generation step S52. Furthermore, the tactile sample presentation method includes an image trimming process step S7 and an image modification determination step S8.

[0030] <Tactile sample presentation process S1> In the tactile sample presentation step S1, a tactile sample 10 made of a material having predetermined physical properties is presented. The tactile sample 10 is placed within reach of the evaluator and within a range where the camera 23 can capture an image.

[0031] <Imaging process S2> In the imaging step S2, the tactile sample 10 is imaged by the camera 23. The camera 23 outputs the image of the tactile sample 10 (tactile sample image Dc) to the data management section 42 of the computer 20 (see FIG. 6).

[0032] <Impression term reception process S3> In the impression term receiving step S3, impression terms W are received from the evaluator (see FIG. 6). The display control unit 43 displays a list of impression terms W on the display 24 based on a command from the impression term receiving unit 44. The evaluator uses the operation unit 22 to select one term (impression term Wa) that expresses a desired impression from the list.

[0033] Impression terms W are also called sensory vocabulary and are used in sensory evaluation (see Figure 7). Sensory evaluation is carried out in various fields such as architecture, food, textiles, automobiles, and cosmetics, and various impression terms W are collected and used in each field. Impression terms W are used to evaluate developed products and to control their quality. For example, "supple," "soft," "luxurious," "soft," "cool," "comfortable," etc. are displayed as impression terms W for residential building materials. The evaluator selects, for example, "supple" as the desired impression term Wa.

[0034] <Recommended group selection process S4> In the recommended combination selection step S4, a combination of the material piece 11 and the texture image D1 is selected based on the impression term Wa "supple" received by the impression term receiving unit 44. This combination is made up of the recommended material piece 11s and the recommended texture image D1s, and is a combination that is highly likely to produce the tactile sensation desired by the evaluator. In other words, it is a combination that is evaluated as one that should be recommended to the evaluator.

[0035] The recommended material piece 11s is the one, among the multiple material pieces 11, that has a high degree of suitability (weighting) with the impression term Wa "supple" received in the previous process. The recommended texture image D1s is the one, among the multiple texture images D1, that has a high degree of suitability for the impression term Wa "supple" received in the previous process. The recommended combination selection unit 45 determines a recommended combination (recommended material pieces 11s, recommended texture images D1s) based on the material impression information P2 regarding each material piece 11 and the image impression information P3 regarding each texture image D1.

[0036] The material impression information P2 is a group of information (a list of numerical values) in which the sensory parameters P1 are assigned to each material piece 11 (see FIG. 8(A)). This affective parameter P1 is a parameter indicating the degree of conformity of each material piece 11 with respect to the impression term W. It is a numerical representation of the strength of the feeling of the impression term W (the degree of tactile impression) that a person feels when touching the material piece 11. 8A, the affective parameter P1 of the impression term Wa "supple" for the material piece 11A is "1.20." The affective parameter P1 of the impression term Wa "supple" for the material piece 11B is "1.30." The larger the value of the sensory parameter P1, the more likely it is to be felt as "supple" when touched. The value of the sensory parameter P1 for each material piece 11 is determined in advance by sensory evaluation or the like. Although the value range of the affective parameter P1 is set to 0 to 2, the range is not limited to this and any value range can be adopted.

[0037] The image impression information P3 is a group of information (a list of numerical values) in which the affective parameters P1 are assigned to each texture image D1 (see FIG. 8(B)). This affective parameter P1 is a parameter indicating the degree of conformity of each texture image D1 with respect to the impression term W. It is a numerical representation of the strength of the feeling (degree of tactile impression) that a person has with respect to the impression term W when viewing the texture image D1. 8B, the affective parameter P1 of the impression term Wa “supple” for white (white color image D1a) is “1.26.” The affective parameter P1 of the impression term Wa “supple” for gray (gray color image D1a) is “1.17.” The larger the value of the sensory parameter P1, the more likely it is that the texture image D1 feels "supple" when viewed. The value of the sensory parameter P1 for each texture image D1 is determined in advance by sensory evaluation or the like.

[0038] (Matrix multiplication calculation S41) To obtain a recommended combination, first, the matrix product of the material impression information P2 and the image impression information P3 is calculated (see FIG. 8(C)). For example, in the case of the impression term Wa "supple," the combination of m material pieces 11 and n color images D1a is calculated by the matrix product of the n-row, 1-column matrix of the color image D1a (supple value in FIG. 8(A)) and the m-row, 1-column matrix of the material piece 11 (supple value in FIG. 8(B)). The m-row, n-column matrix obtained by the calculation is the impression value of "supple" consisting of the combination of the material pieces 11 and the color image D1a, as shown in FIG. 8(C). That is, the combination of the material piece 11A and the white color image D1a is calculated as "1.2×1.26=1.51." The combination of the material piece 11B and the gray color image D1a is calculated as "1.30×1.17=1.52."

[0039] (Ranking / Selection S42) Next, the combinations are ranked according to the magnitude of the calculated matrix product, and the combinations with the highest ranking are selected as the combinations to be recommended. A plurality of pairs can be selected as the recommended combination "recommended material piece 11s, recommended texture image D1s." For example, the first place is selected as "material piece 11B, light yellow" with a matrix product of 1.69 points, the second place is selected as "material piece 11B, white" with a matrix product of 1.64 points, the third place is selected as "material piece 11A, light yellow" with a matrix product of 1.56 points, the fourth place is selected as "material piece 11A, gray" with a matrix product of 1.52 points, etc. (see FIG. 6). In this example, the first, second, and fourth places cannot be presented to the evaluator at the same time because the material pieces 11A overlap. Therefore, the second and fourth places are excluded (reserved) as the recommended combination, and only the first and third places are selected.

[0040] In the recommended set selection step S4, one each of the color image D1a and the pattern image D1b may be selected as the recommended texture image D1s. In this case, the multiplication (matrix product) of the material impression information P2 for each material piece 11, the image impression information P3 for each color image D1a, and the image impression information P3 for each pattern image D1b is calculated.

[0041] <Image processing step S5> In the image processing step S5, a texture sample image D3 corresponding to the surface of the material piece 11 of the texture sample is generated based on the recommended texture images D1s selected in the recommended set selection step S4. The texture sample image D3 is generated according to the number of recommended texture images D1s selected. The texture sample image D3 is an image that expresses AR, and is generated by processing and synthesizing the texture image D1. The image processing step S5 includes texture image synthesis S51 and AR image generation S52.

[0042] (Texture image synthesis S51) The texture image synthesis S51 is executed when one color image D1a and one pattern image D1b are selected in the recommended pair selection step S4. The image processing unit 41 generates a composite texture image D2 by alpha-blending two texture images D1 (a color image D1a and a pattern image D1b) (see FIG. 9(A)). Alpha blending is the process of overlaying two images and synthesizing them based on the transparency (alpha value) set for each pixel. It allows you to see the image behind the transparent image overlapping through the transparent image.

[0043] The alpha value used when combining the color image D1a and the pattern image D1b can be set arbitrarily. The alpha value ranges from 0 to 255, with 0 representing a completely transparent image and 255 representing a completely opaque image. The alpha value may be always set to a predetermined value (for example, 128) or may be set randomly each time. In addition, when the evaluator selects the texture image D1 in the recommended set selection step S4, the alpha value may also be set. In addition, if only one of the color image D1a and the pattern image D1b is selected in the recommended pair selection step S4, the alpha value is set to, for example, 255 (completely opaque). When two texture images D1 are alpha blended, a wide variety of composite texture images D2 (and therefore texture sample images D3) can be generated by arbitrarily setting the alpha value.

[0044] (AR image generation S52) In the AR image generation S52, the texture image D1 or the composite texture image D2 is processed to generate a texture sample image D3. As described above, the texture sample image D3 is an image that expresses AR, and is an image that is superimposed on the tactile sample image Dc. The image processing unit 41 processes the texture image D1 or the synthetic texture image D2 to generate image data (texture sample image D3) having the same shape as the surface of the material piece 11 displayed in the tactile sample image Dc (see FIG. 9(A)). Next, the image processing unit 41 superimposes (combines) this texture sample image D3 onto the surfaces of the material pieces 11A to 11C in the tactile sample image Dc.

[0045] Position information and the like of the surfaces of the material pieces 11A-11C in the tactile sample image Dc is determined using the AR marker 12. To realize AR, it is essential to acquire position information of the tactile sample 10 (material piece 11). Highly accurate position recognition of the material piece 11 is required to avoid giving an uncomfortable feeling to the evaluator. Therefore, the AR marker 12 is imaged by the camera 23, and image recognition technology is used to obtain position information of the material piece 11. This type of position recognition is called a vision-based type or an image recognition type. When the AR marker 12 has a square black frame and a predetermined pattern drawn inside the black frame, the AR marker 12 is detected using the black frame, and the AR marker 12 is identified by the pattern inside the black frame. The pattern inside the black frame is then compared with the pattern file stored in the HDD 24, and the matching rate is calculated, so that the AR marker 12 is correctly detected. Furthermore, accurate position information of the material piece 11 is obtained by comparing the marker coordinate system calculated by extracting the outline of the AR marker 12 with the relative coordinates of the tactile sample 10 and the AR marker 12 stored in the HDD 24.

[0046] Based on position information acquired using the AR marker 12, the texture sample image D3 is superimposed on the surfaces of the material pieces 11A to 11C of the tactile sample image Dc (see FIG. 9(B)). The texture sample image D3 is placed in the same position as the surface of the material piece 11. In an image (AR image Dr) in which the texture sample image D3 is synthesized with the tactile sample image Dc, the surface of the material piece 11 and the texture sample image D3 completely match. In this manner, by superimposing the texture sample image D3 on the tactile sample image Dc, an AR image Dr capable of expressing AR is generated.

[0047] <Image presentation step S6> In the image presenting step S6, the AR image generated by the image processing unit 41 is presented to the evaluator. That is, the image processing unit 41 outputs an AR image Dr in which the texture sample image D3 is superimposed (applied) on the tactile sample image Dc to the display control unit 43. Then, the display control unit 43 displays this AR image Dr on the display 24 (see FIG. 9(B)).

[0048] As a result, the tactile sample 10 is presented within reach of the evaluator, and the AR image Dr displayed on the display 24 is also presented in front of the evaluator. In other words, an AR tactile sample 52 (cross-reality tactile sample 50) consisting of the tactile sample 10 and the AR image Dr is presented to the evaluator. The evaluator can touch the surface of the material piece 11 of the tactile sample 10 while looking at the AR image Dr displayed on the display 24. The evaluator can feel (observe) the physical properties of material piece 11 by touching the surface of material piece 11 with his / her fingertips, utilizing his / her sense of touch. The physical properties of the material piece 11 include the surface properties, Young's modulus, thermal conductivity, etc. of the material piece 11. The surface properties include roughness, unevenness, friction, etc., the Young's modulus gives the evaluator a sense of hardness or softness, and the thermal conductivity gives the evaluator a sense of hotness or coldness.

[0049] When the evaluator touches the tactile sample 10 (piece of material 11) while looking at the AR image Dr displayed on the display 24 (without looking at the tactile sample 10), a cross-modal interaction (interaction between senses) occurs. Cross-modal interaction is an interaction between different senses, and in this embodiment occurs through touch and vision. That is, when the evaluator recognizes the real piece of material 11 through touch while recognizing the virtual piece of material 11 (AR image Dr) through vision, the integration of touch and vision produces a sensation that cannot be obtained through touch or vision alone. That is, the tactile sensations recognized by the evaluator, such as roughness, unevenness, friction, hardness / softness, and hot / coldness of the piece of material 11, change.

[0050] Specifically, when a warm color image is superimposed on the material piece 11, the evaluator feels a warmer touch than when the material piece 11 is left as is without a color image. Also, when a cool color image is superimposed on the material piece 11, the evaluator feels a cooler touch than when the material piece 11 is left as is without a color image. Also, when a wood grain pattern image is superimposed on the material piece 11, the evaluator feels a natural warmth and calmness more than when the material piece 11 is left as is without a pattern image. In this way, by using the AR tactile sample 52, it is possible to realize a material sample that presents a plurality of tactile sensations to the evaluator even with a single material piece 11. In other words, it is possible to realize a material sample that presents a wide variety of sensations to the evaluator with little effort, even without preparing a plurality of different material pieces 11.

[0051] The AR tactile sample 52 has a plurality of material pieces 11, and an image (AR image Dr) is provided by synthesizing these images (tactile sample images Dc) with the texture sample image D3. That is, a plurality of AR images Dr are generated. In particular, impression terms expressing the impression desired by the evaluator are received, and the AR tactile sample 52 corresponding to these impression terms is generated. Therefore, the AR tactile sample 52 can accurately express the sensation desired by the evaluator.

[0052] <Image trimming process S7> 10 is a diagram showing image processing when the texture sample image D3 and the evaluator's finger overlap. For convenience of explanation, only one material piece 11 is shown in the figure. When the evaluator touches the actual material piece 11 while looking at the image (AR image Dr) of the material piece 11, the evaluator's finger is reflected in the tactile sample image Dc. The texture sample image D3 is then superimposed on the image of the evaluator's finger. This may give the evaluator who viewed the AR image Dr a strange feeling, which may result in a change in the tactile sensation of the material piece 11. Therefore, the image processing unit 41 performs a trimming process (cropping process) on the area of ​​the texture sample image D3 that is hidden by the evaluator's finger when the evaluator touches the material piece 11 of the texture sample 10. The image trimming process is performed using an existing method. As a result, the AR image Dr is presented only to the area excluding the evaluator's finger on the surface of the material piece 11. Therefore, it is possible to provide the AR tactile sample 52 that does not give the evaluator a sense of incongruity when he or she sees the AR image Dr.

[0053] <Image change determination step S8> Finally, it is determined (selected) whether to continue the tactile sample presentation method, in other words, whether to present again the AR tactile sample 52 that gives a different impression. In the image change determination step S8, first, it is determined whether or not the presentation of the AR tactile sample 52 needs to be continued (determination step S81). When the evaluator issues an instruction to end the tactile sample presentation from the operation unit 22, the tactile sample presentation method ends. On the other hand, in the judgment step S81, if the evaluator issues an instruction to continue the presentation of the tactile sample through the operation unit 22, it is judged whether or not to further change the impression term W (judgment step S82).

[0054] In the judgment step S82, if the evaluator issues an instruction to reject the change of the impression term W from the operation unit 22, the process returns to the ranking and selection step S42 of the recommended set selection step S4. Then, the combination (material piece 11, texture image D1) that was the runner-up or reserved in the previous step is selected. Then, the image processing step S5 to the image trimming processing step S7 are performed again. On the other hand, in the judgment step S82, if the evaluator issues an instruction to change the impression term W from the operation unit 22, the process returns to the impression term receiving step S3. Then, the recommended set selection step S4 to the image trimming process step S7 are performed again.

[0055] In this way, the tactile sample presentation system and tactile sample presentation method of this embodiment can present a wide variety of sensations to the evaluator using the AR tactile sample 52 (cross-reality tactile sample 50) that utilizes cross-modal interaction. In other words, a plurality of tactile textures and impressions (virtual tactile sensations, pseudo-tactile sensations) can be presented to the evaluator who touches the tactile sample 10 (material piece 11) of the AR tactile sample 52. In particular, it is possible to generate and present the AR tactile sample 52 that accurately expresses the sensation desired by the evaluator. Even if the tactile sample 10 is made up of a small number of material pieces 11, it is possible to present a large number of different sensations to the evaluator. This makes it possible to significantly reduce the cost and time required to produce the tactile sample 10. The size and weight of the tactile sample 10 can also be reduced. Furthermore, the cost of disposing of the tactile sample 10 can also be reduced.

[0056] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0057] The present invention can present a cross-reality tactile sample 50 using XR (Cross Reality / Extended reality) technology, which is a general term for technologies that combine the real world and the virtual world. That is, although the case of using AR (Augmented reality) technology has been described, the present invention is not limited to this. When a head mounted display is used as the display 24, an MR tactile sample using MR (Mixed Reality) technology can be presented as the cross reality tactile sample 50. The head mounted display presents an AR image Dr in which a texture sample image D34 is superimposed on a transparent image of the tactile sample 10. Furthermore, as the cross-reality haptic sample 50, a VR haptic sample using VR (Virtual Reality) technology may be presented.

[0058] FIG. 11 is a diagram showing the tactile sample 13. As shown in FIG. The tactile sample 13 is a set of multiple material pieces 11 arranged in both the vertical and horizontal directions. In the tactile sample 13, the multiple material pieces 11 are arranged horizontally (from left to right) in order of surface roughness, for example, from rough to smooth. In the vertical direction (from back to front), they are arranged in order of thermal conductivity, for example, from high to low. The AR markers 12 are placed at the four corners of the mount. This makes it possible to provide the evaluator with a greater number of AR tactile samples 52 (cross-reality tactile samples 50). Also, the AR tactile samples 52 (tactile samples 13) can express the sensation desired by the evaluator with a high probability.

[0059] 12A and 12B are diagrams showing a tactile sample 15, in which (A) is a plan view and (B) is a cross-sectional view. For convenience of explanation, only one material piece 16 is shown. The tactile sample 10 is not limited to one in which the tactile confirmation area is only on the front side (material piece 11), but may also be one in which the back side (rear side) is also included (tactile sample 15). The tactile sample 15 has a material piece 16 at least partly made of resin, and the outer periphery of the material piece 16 is held in a shape-stably manner. A front surface 16a and a back surface 16b are exposed (released) to form a tactile confirmation area. The evaluator can touch not only the surface of the material piece 16 but also both sides (the front and back) of the material piece 16. This allows the evaluator to pinch or pick up the material piece 16 between his / her index finger and thumb, and to grasp the tactile sensation such as hardness and thickness.

[0060] The shape of the material pieces 11, 16 is not limited to a rectangle (quadrangle), and may be a circle, an ellipse, or even a sphere (semisphere).

[0061] Although a desktop personal computer has been described as the computer 20, the present invention is not limited to this. A notebook PC, a tablet PC, a smartphone, or other portable computer may also be used. When a portable computer is used, the evaluator may use a camera mounted on the portable computer to capture an image of the tactile sample 10. Then, the position information of the camera 23 and the display 24 is obtained using a GPS mounted on the portable computer.

[0062] A mobile display or a head-mounted display may be used as the display 24. The head-mounted display may be of a so-called sealed type, an optical transmission type, or a video transmission type. When a video-transmitting head-mounted display is used, the camera 23 mounted on the video-transmitting head-mounted display captures an image of the tactile sample 10. In addition, the position information of the camera 23 and the display 24 is obtained using a GPS mounted on the video-transmitting head-mounted display.

[0063] The position and orientation of the material piece 11 may be detected by a markerless method without using the AR marker 12. For example, the four corners or feature points of the mount on which the material piece 11 is attached or the desk on which the texture sample 10 is placed are used instead of the AR marker 12. [Explanation of symbols]

[0064] 1. Tactile sample presentation system 10 Texture Sample 11 Material pieces 11s Recommended material pieces 12 AR Marker 13 Texture sample 15 Texture Sample 16 material pieces 16a surface 16b Back 20 Computers 21 Control section 22 Operation section (impression term reception section) 23 Camera 24 Display (image display unit) 31 CPU 32 ROM 33 RAM 34 HDD (image database) 41 Image processing section 42 Data Management Department (Image Database) 43 Display control unit (image presentation unit) 44 Impression Terms Reception Department 45 Recommended group selection section 50 Cross reality texture samples 51 AR tactile sample Dc Texture Sample Image D1 Texture Image D1a color image D1b pattern image D1s recommended texture images D2 Synthetic Texture Image D3 Texture Sample Image Dr AR Images W,Wa Impression term P1 Sensitivity parameters P2 Material impression information P3 Image impression information

Claims

1. A tactile sample consisting of a plurality of material pieces having arbitrary physical properties; A camera for capturing an image of the tactile sample; an image database that stores a plurality of texture images having visual impression elements; an impression term receiving unit that receives an impression term expressing a desired impression; a recommended combination selection unit that selects a recommended combination from among the material pieces and the texture image based on the impression term accepted by the impression term acceptance unit; an image processing unit that generates a texture sample image corresponding to the material piece from the recommended texture images selected by the recommended set selection unit; an image presenting unit that presents a cross-reality tactile sample in which the texture sample image is added to the recommended material piece selected by the recommended group selecting unit; A tactile sample presentation system comprising:

2. 2. The tactile sample presentation system according to claim 1, wherein the recommended set selection unit determines the combination based on material impression information indicating a degree of conformance of the material piece with the impression term and image impression information indicating a degree of conformance of the texture image with the impression term.

3. the image presentation unit is a display that displays a tactile sample image acquired by the camera, The tactile sample presentation system according to claim 1 , wherein the display presents an AR tactile sample in which the texture sample image is superimposed on the tactile sample image.

4. the image display unit is a video-transmitting or optical-transmitting head-mounted display, 2. The tactile sample presentation system according to claim 1, wherein the head mounted display presents an MR tactile sample in which the texture sample image is superimposed on a transparent image of the tactile sample.

5. The tactile sample presentation system according to claim 1 , wherein the image processing unit generates a composite texture image by alpha-blending the two texture images.

6. 6. The tactile sample presentation system according to claim 5, wherein one of the texture images is a color image and the other is a pattern image.

7. 2. The tactile sample presentation system according to claim 1, wherein said image processing section performs a trimming process on said texture sample image to remove an area hidden by a finger of said evaluator when said evaluator touches said tactile sample.

8. 2. The tactile sample presentation system according to claim 1, wherein the tactile sample is at least partially made of resin, the outer periphery of the sample is held in a shape-stably manner, and the front and back surfaces are opened to form a tactile confirmation area.

9. a tactile sample presenting step of presenting a tactile sample made of a plurality of material pieces having any physical property; an imaging step of imaging the tactile sample with a camera; an impression term receiving step of receiving an impression term expressing a desired impression; a recommended combination selection step of selecting a recommended combination from among the material pieces and texture images having visual impression elements held in a plurality of image databases based on the impression terms received by an impression term receiving unit; an image processing step of generating a texture sample image corresponding to the material piece from the recommended texture images selected in the recommended set selection step; an image presenting step of presenting a cross-reality tactile sample in which the texture sample image is added to the recommended material piece selected in the recommended group selection step; The method for providing a tactile sample is characterized by comprising the steps of:

Images