Tactile sensation generation system, tactile sensation generation method, and training method of generation model

Abstract

The invention provides a tactile sensation generation system, a tactile sensation generation method and a training method of a generation model. The touch generation system comprises a coding unit, a generation model, a generation result verification unit and at least one control unit. The coding unit is used for translating an object texture image. The generative model is connected to the coding unit. And the generation model is used for deducing texture touch feature information according to object texture image coding. And the generation result verification unit is connected to the generation model. The generation result verification unit is used for verifying whether the generated texture touch feature information meets a predetermined condition or not. And the control unit is connected to the generated result verification unit. If the generated texture touch feature information meets a predetermined condition, the control unit outputs a control signal to at least one actuator according to the generated texture touch feature information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a generation system, a generation method and a training method, and particularly relates to a tactile sensation generation system, a tactile sensation generation method and a training method of a generation model. BACKGROUND

[0002] In virtual reality applications, if the wearable device is to have tactile feedback, an object texture extraction device needs to be matched. When processing various objects, corresponding texture tactile sensations need to be established respectively. Therefore, a technology capable of generating simulated tactile sensations for various objects and capable of real-time feedback is needed. SUMMARY

[0003] The present disclosure relates to a generation system, a generation method and a training method, and particularly relates to a tactile sensation generation system, a tactile sensation generation method and a training method of a generation model.

[0004] According to an aspect of the present disclosure, a tactile sensation generation system is provided. The tactile sensation generation system comprises an encoding unit, a generation model, a generation result verification unit and at least one control unit. The encoding unit is configured to translate an object texture image. The generation model is connected to the encoding unit. The generation model is configured to infer texture tactile sensation feature information according to the encoding of the object texture image. The generation result verification unit is connected to the generation model. The generation result verification unit is configured to verify whether the generated texture tactile sensation feature information meets a predetermined condition. The at least one control unit is connected to the generation result verification unit. If the generated texture tactile sensation feature information meets the predetermined condition, the control unit outputs a control signal to at least one actuator according to the generated texture tactile sensation feature information.

[0005] According to another aspect of the present disclosure, a tactile sensation generation method is provided. The tactile sensation generation method comprises the following steps. An object texture image is translated. Texture tactile sensation feature information is inferred according to the object texture image. It is verified whether the texture tactile sensation feature information meets a predetermined condition. If the texture tactile sensation feature information meets the predetermined condition, a control signal is output to at least one actuator according to the texture tactile sensation feature information.

[0006] According to still another aspect of the present disclosure, a training method of a generation model is provided. The training method of the generation model comprises the following steps. A plurality of object texture image samples are provided. Under the limitation of parameter description information, training of the generation model is performed according to the object texture image samples until the texture tactile sensation feature information inferred by the generation model meets a predetermined condition.

[0007] In order to better understand the above and other aspects of the present disclosure, the following embodiments are described in detail below with reference to the accompanying drawings: BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 An embodiment of a tactile sensation generation technology according to the present disclosure is illustrated.

[0009] Figure 2 An embodiment of the haptic generation technology according to another embodiment of the present disclosure is illustrated.

[0010] Figure 3 An example illustrates a tactile sensation generation system and method according to an embodiment of the present disclosure.

[0011] Figure 4 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0012] Figure 5 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0013] Figure 6 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0014] Figure 7 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0015] Figure 8 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0016] Figure 9 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0017] Figure 10 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0018] Figure 11 Examples illustrate a tactile generation system and tactile generation method according to another embodiment of the present disclosure.

[0019] [Symbol Explanation]

[0020] 1000(1), 1000(2), 1000(3), 1000(4), 1000(5), 1000(6), 1000(7), 1000(8), 1000(9): Haptic Generation System

[0021] 100(1), 100(3), 100(4), 100(5), 100(6), 100(7), 100(8), 100(9): Local terminal device

[0022] 110: Encoding Unit

[0023] 120: Generative Model

[0024] 130: Result Verification Unit

[0025] 140: Preset information storage unit

[0026] 150: Comparison Unit

[0027] 160: Central Computing Unit

[0028] 200(1), 200(2), 200(3), 200(4), 200(5), 200(6), 200(7), 200(8), 200(9): Remote computing device

[0029] 210: Database Unit

[0030] 220: Encoding Unit

[0031] 230: Training Unit

[0032] 240: Generative Model

[0033] 250: Result Verification Unit

[0034] 260: Model Processing Unit

[0035] 300(1), 300(2), 300(3), 300(4), 300(5), 300(6), 300(7), 300(8), 300(9): Haptic actuators

[0036] 310: Control Unit

[0037] 320: Parameter Description Unit

[0038] 330: Feedback Evaluation Unit

[0039] 340: Physiological parameter measurement unit

[0040] 350: Inertial Parameter Measurement Unit

[0041] 410, 440: Actuators

[0042] 420: Monitor

[0043] 430: Loudspeaker

[0044] 500: hand

[0045] 600: Virtual Objects

[0046] 700: Human haptic feedback device

[0047] 800: Solid objects

[0048] 900: Machine structure actuation device

[0049] CS: Control signal

[0050] CT: Reservation Conditions

[0051] DS: Object text description information

[0052] FT: Texture and tactile characteristics

[0053] FT0: Pre-stored texture tactile feature information

[0054] FB: Feedback Information

[0055] LT: Text description limitation

[0056] LS: Special Effects Limitations

[0057] IM, IM6, IM8: Object texture images

[0058] IMi: Texture image sample

[0059] IM', IMi': Encoded information

[0060] IM0: Pre-stored texture image

[0061] IN: Inertial parameter signal

[0062] pH: Physiological parameter measurement signal

[0063] PM, PM': Parameter description information

[0064] PI: Panoramic Image

[0065] VS: Object sound effect characteristics

[0066] S100, S100', S110, S110', S111, S112, S120, S120', S122, S123, S130, S130', S131, S131', S132, S133, S134, S150, S160, S210, S220, S230, S240, S250, S260, S310, S311, S330, S331, S340, S341, S350, S351, S420, S430: Steps Detailed Implementation

[0067] The technical terms used in this specification are based on common terminology in the field. Where this specification provides explanations or definitions for certain terms, the interpretation of those terms shall be based on the explanations or definitions provided in this specification. Each embodiment of this disclosure has one or more technical features. Where feasible, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

[0068] Please refer to Figure 1 This illustration depicts an implementation of a haptic generation technique according to an embodiment of the present disclosure. In one embodiment, the user's hand 500 is wearing a human haptic feedback device 700 (e.g., a glove, but not limited to gloves). In other embodiments, the human haptic feedback device 700 may also be a joystick, handle, haptic feedback screen, sportswear, chair, watch, phone case, backpack, hat, helmet, etc. After the object texture image IM6 of the virtual object 600 is extracted, when the user wants to feel the texture of the virtual object 600, the actuator 410 in the human haptic feedback device 700 will generate haptic feedback such as vibration and friction on the user's hand 500.

[0069] Please refer to Figure 2 The illustration depicts an implementation of a tactile generation technique according to another embodiment of the present disclosure. In another embodiment, the system first captures an object texture image IM8 of the physical object 800 to be grasped, and then infers the texture tactile feature information FT (labeled in...) through texture image recognition and translation. Figure 3 After that, it provides tactile control signals such as grip strength and friction to control the machine structure actuator 900 (e.g., a robotic arm, but not limited to). Figure 2 The illustrated robotic arm is not limited to a single robotic arm. In this way, when the machine structure actuation device 900 grasps a solid object 800, the actuator 440 within the machine structure actuation device 900 can adjust the grip and friction appropriately to grasp the solid object 800. For example, when the solid object 800 is an egg, the actuator 440 within the machine structure actuation device 900 will reduce the grip and provide greater friction to prevent the egg from breaking or falling. The above... Figure 1 and Figure 2 These are merely some embodiments of this disclosure and are not intended to limit the application of the technology disclosed herein.

[0070] Please refer to Figure 3 Examples illustrate a tactile generation system 1000 (1) and a tactile generation method according to an embodiment of this disclosure. Figure 3In one embodiment, the haptic generation system 1000(1) includes a local device 100(1), a remote computing device 200(1), a haptic actuator 300(1), and an actuator 410. The actuator 410 is disposed within a human haptic feedback device 700 (e.g., a glove).

[0071] The local terminal device 100(1) includes an encoding unit 110, a generation model 120, and a generation result verification unit 130. The functions of each component are summarized below. The encoding unit 110 is used for image encoding. The generation model 120 is connected to the encoding unit and is used for feature information generation. The generation result verification unit 130 is connected to the generation model and is used for verifying the generation result.

[0072] The remote computing device 200(1) includes a database unit 210, an encoding unit 220, a training unit 230, a generative model 240, a generative result verification unit 250, and a model processing unit 260. The functions of each component are summarized below. The database unit 210 stores various types of data. The encoding unit 220 is connected to the database unit 210 and is used for image encoding. The training unit 230 is connected to the encoding unit 220 and is used for training the model. The generative model 240 is connected to the training unit 230 and generates various types of information using artificial intelligence technology. The generative result verification unit 250 is connected to the generative model 240 and is used to verify the generated results. The model processing unit 260 is connected to the generative model 240 and is used for performing model processing procedures (e.g., compression or distillation procedures).

[0073] Database unit 210 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar element or combination thereof, for storing multiple modules or various applications that can be executed by a processor.

[0074] The haptic actuator 300(1) includes a control unit 310 and a parameter description unit 320. The control unit 310 is connected to the generation result verification unit 130 and the actuator 410 for controlling the actuator 410. The parameter description unit 320 is connected to the control unit 310 and the generation result verification unit 250 for providing limits on various parameters.

[0075] The encoding unit 110, the generation model 120, the generation result verification unit 130, the encoding unit 220, the training unit 230, the generation model 240, the generation result verification unit 250, the model processing unit 260, the control unit 310, and the parameter description unit 320 are, for example, a circuit, a circuit board, a storage device for storing program code, or a chip. The chip is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), neural-network processing unit (NPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field-programmable gate array (FPGA), or other similar elements or combinations thereof.

[0076] In one embodiment, the local terminal device 100 (1), the haptic actuator 300 (1), and the actuator 410 are disposed, for example, within the human haptic feedback device 700. Alternatively, in another embodiment, only the haptic actuator 300 (1) and the actuator 410 are disposed within the human haptic feedback device 700. Alternatively, in another embodiment, only the actuator 410 is disposed within the human haptic feedback device 700.

[0077] exist Figure 3 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 3In the tactile generation system 1000(1), after the remote computing device 200(1) trains the generation model 240, it can process the generation model 240 to become the generation model 120. The processed generation model 120 can be loaded into the local terminal device 100(1). After the object texture image IM is input to the local terminal device 100(1), the object texture tactile feature information FT can be generated in real time through the generation model 120, and handed over to the tactile actuation device 300(1) for the control of the actuator 410 to generate diverse tactile sensations in real time. The operation of each component is described in detail below through the tactile generation method.

[0078] like Figure 3 As shown, the tactile sensation generation method includes steps S100, S110, S120, S130, S131, and S310. In step S100, the encoding unit 110 of the local terminal device 100 (1) translates the object texture image IM. The object texture image IM is, for example, a color photograph or a black and white photograph.

[0079] Then, in step S110, the encoding unit 110 of the local terminal device 100(1) converts the object texture image IM into encoded information IM' that can be read by the generation model 120. The encoded information IM' is, for example, a vector information.

[0080] Next, in step S120, the generation model 120 of the local terminal device 100(1) infers the texture tactile feature information FT based on the encoding information IM' of the object texture image IM. The texture tactile feature information FT may be, for example, a surface protrusion height, a spatial variation gradient, a fluctuation variation period, a fluctuation variation height, a friction direction, a friction coefficient, a surface variation curve, a principal component value, or an intrinsic mode function value.

[0081] Then, in step S130, the generation result verification unit 130 of the local terminal device 100(1) verifies whether the texture tactile feature information FT meets a predetermined condition CT. If the texture tactile feature information FT does not meet the predetermined condition CT, a prompt indicating that the touch cannot be recognized is output; if the texture tactile feature information FT meets the predetermined condition CT, then step S131 is entered. The predetermined condition CT is, for example, whether the Contrastive Language-Image Pretraining (CLIP) index is greater than 0.3.

[0082] In step S131, the result verification unit 130 transmits the texture tactile feature information FT to the control unit 310 of the tactile actuator 300 (1).

[0083] Next, in step S310, the control unit 310 of the tactile actuator 300(1) outputs a control signal CS to the actuator 410 based on the texture tactile feature information FT. The control signal CS is used, for example, to control a vibration frequency, a vibration intensity, a waveform, a pulse width, a pulse frequency, a protrusion height, a depression depth, a coefficient of friction, a normal force intensity, or a lateral force intensity.

[0084] Through the above steps, after the object texture image IM is input to the local terminal device 100 (1), the texture tactile feature information FT can be generated in real time by the generation model 120, and then handed over to the tactile actuation device 300 (1) for the control of the actuator 410 to generate diverse tactile sensations in real time. The generation model 120 is the result of processing the generation model 240 trained by the remote computing device 200 (1). The training method of the generation model 240 is further explained below. The training method of the generation model 240 includes steps S210 to S260.

[0085] In step S210, the database unit 210 provides several object texture image samples IMi to the encoding unit 220. The object texture image samples IMi are, for example, photographs of various object textures.

[0086] Next, in step S220, the encoding unit 220 of the remote computing device 200(1) converts the object texture image sample IMi into encoded information IMi' that can be read by the training unit 230. The encoded information IMi' is, for example, a vector information.

[0087] Then, in step S230, under the constraints of parameter description information PM, training unit 230 trains generative model 240 based on the encoding information IMi' of these object texture image samples IMi. Parameter description information PM is used to set, control, or describe various variables or set values ​​of the haptic actuator and / or actuator operating characteristics. Parameter description information PM includes single or composite haptic feedback device sensations, such as composite sensations like deformation feedback undulating hapticity / vibration feedback rough hapticity / friction feedback sticky hapticity, just-noticeable difference (JND), or other similar parameters or combinations of the above parameters.

[0088] Next, in step S250, the result verification unit 250 verifies whether the texture tactile feature information FT inferred by the generation model 240 meets the predetermined condition CT. If it does not meet the predetermined condition CT, the process returns to step S230 and retrains until the texture tactile feature information FT inferred by the generation model 240 meets the predetermined condition CT. The predetermined condition CT is, for example, CLIP less than 0.3.

[0089] Alternatively, in another embodiment, the parameter description information PM may not be provided to the training unit 230, but a predetermined condition CT may be directly added, which may also allow the trained generative model 240 to meet the limitations of the parameter description information PM.

[0090] Then, in step S260, the model processing unit 260 processes the generated model 240 after training, for example, through model compression or distillation, to obtain the generated model 120.

[0091] Next, in step S240, the processed generated model 120 is loaded into the local terminal device 100 (1). When the local terminal device 100 (1) receives the object texture image IM, it can infer the corresponding object texture tactile feature information FT and provide the texture tactile feature information FT to the tactile actuator 300 (1) so that the actuator 410 can provide tactile feedback to the user.

[0092] Please refer to Figure 4 Examples illustrate a tactile generation system 1000(2) and a tactile generation method according to another embodiment of this disclosure. Figure 4 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 4 In the embodiments, the haptic generation system 1000(2) includes a remote computing device 200(2), a haptic actuator 300(2), and an actuator 410. The remote computing device 200(2) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, and a generation result verification unit 250. The haptic actuator 300(2) includes a control unit 310 and a parameter description unit 320.

[0093] In this embodiment, steps S100', S110', S120', S130', and S131' of the tactile generation method are executed within the remote computing device 200(2). For example, in step S100', the encoding unit 220 of the remote computing device 200(2) translates the object texture image IM. In step S110', the encoding unit 220 of the remote computing device 200(2) converts the object texture image IM into encoded information IM' that can be read by the generation model 240. In step S120', the generation model 240 of the remote computing device 200(2) infers the texture tactile feature information FT based on the encoded information IM' of the object texture image IM. In step S130', the generation result verification unit 250 of the remote computing device 200(2) verifies whether the texture tactile feature information FT meets the predetermined condition CT. If the texture tactile feature information FT does not meet the predetermined condition CT, the process ends; if the texture tactile feature information FT meets the predetermined condition CT, the process proceeds to step S131'. In step S131', the result verification unit 250 transmits the texture tactile feature information FT to the control unit 310 of the tactile actuation device 300 (1).

[0094] In this embodiment, the training method for generating model 240 includes the above steps S210, S220, S230, and S250. These steps are also run on the remote computing device 200 (2), and will not be described again here.

[0095] Figure 4 The tactile generation method and training method of the embodiment are both performed on the remote computing device 200 (2), so the generation model 240 does not need to go through the compression process after training.

[0096] Please refer to Figure 5 Examples illustrate a tactile generation system 1000 (3) and a tactile generation method according to another embodiment of this disclosure. Figure 5 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 5In the embodiments, the haptic generation system 1000 (3) includes a local terminal device 100 (3), a remote computing device 200 (3), a haptic actuator 300 (3), and an actuator 410. The local terminal device 100 (3) includes an encoding unit 110, a generation model 120, and a generation result verification unit 130. The remote computing device 200 (3) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (3) includes a control unit 310, a parameter description unit 320, and a feedback evaluation unit 330. The feedback evaluation unit 330 is, for example, a circuit, a circuit board, a storage device storing program code, or a chip.

[0097] exist Figure 5 In this embodiment, the tactile sensation generation method further includes steps S330 and S331. In step S330, the feedback evaluation unit 330 provides feedback information FB to a user. The feedback evaluation unit 330 is connected to the actuator 410 and the generation model 120. When the actuator 410 generates tactile sensation, the user can manually input the tactile sensation into the generation model 120, for example, whether the vibration amplitude is too small or the inflation fluctuation amplitude is too large.

[0098] Then, in step S331, the generation model 120 optimizes the texture tactile feature information FT based on the feedback information FB. For example, the texture tactile feature information FT output by the generation model 120 can be adjusted in proportion or weight according to the feedback information FB, so that the control unit 310 can output an appropriate control signal CS (e.g., enhance or reduce the vibration mode or inflation amplitude of the actuator 410).

[0099] In this embodiment, the user can provide feedback on their tactile sensations, and the intensity of the actuator 410 can be adjusted in real time to provide the user with a better experience during use.

[0100] Please refer to Figure 6 Examples illustrate a tactile generation system 1000 (4) and a tactile generation method according to another embodiment of this disclosure. Figure 6 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 6In the embodiments, the haptic generation system 1000 (4) includes a local terminal device 100 (4), a remote computing device 200 (4), a haptic actuator 300 (4), and an actuator 410. The local terminal device 100 (4) includes an encoding unit 110, a generation model 120, and a generation result verification unit 130. The remote computing device 200 (4) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (4) includes a control unit 310, a parameter description unit 320, and a physiological parameter measurement unit 340. The physiological parameter measurement unit 340 is, for example, a smartwatch, a smart bracelet, a headset, an in-ear headphone, or a smart garment.

[0101] exist Figure 6 In this embodiment, the tactile sensation generation method further includes steps S340 and S341. In step S340, the physiological parameter measurement unit 340 performs measurements to obtain a physiological parameter measurement signal PH for one of the users. When the user uses the actuator 410, the physiological parameter measurement unit 340 simultaneously measures the user's physiological values, such as blood pressure, heart rate, body temperature, skin conductance response, or electromyography signal, and transmits the physiological parameter measurement signal PH to the parameter description unit 320.

[0102] Then, in step S341, the parameter description unit 320 optimizes the texture tactile feature information FT based on the physiological parameter measurement signal PH.

[0103] For example, the physiological parameter measurement unit 340 is connected to the parameter description unit 320, which sets the parameter description information PM' based on the physiological parameter measurement signal PH. Therefore, when the adjusted parameter description information PM' is transmitted to the training unit 230 of the remote computing device 200 (4) for training, the generated model 240 can better match the user's personalized mode.

[0104] Alternatively, the adjusted parameter description information PM' can also be transmitted to the generation result verification unit 250, and by directly adding the predetermined condition CT, the trained generation model 240 can also meet the constraints of the parameter description information PM'.

[0105] Alternatively, the physiological parameter measurement signal PH can also be transmitted to the generation model 120 of the local terminal device 100 (4) to directly adjust the generated texture tactile feature information FT, for example, by adjusting the scale or weight, so that the control unit 310 can output an appropriate control signal CS (e.g., to enhance or reduce the vibration mode or inflation amplitude of the actuator 410).

[0106] exist Figure 6In this embodiment, the measurement of the user's physiological values ​​is added. When the user has an excessive emotional reaction, high blood pressure, or a fast heart rate, the parameters can be adjusted appropriately according to the user's physical condition to train a more suitable generative model 240, 120 for the user's physiological condition and generate more suitable texture tactile feature information FT, reducing stimulation to the user and allowing the user to have a better tactile experience during the experience.

[0107] Please refer to Figure 7 The example illustrates a tactile generation system 1000 (5) and a tactile generation method according to another embodiment. Figure 7 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 7 In the embodiment, the haptic generation system 1000 (5) includes a local terminal device 100 (5), a remote computing device 200 (5), a haptic actuator 300 (5), and an actuator 410. The local terminal device 100 (5) includes an encoding unit 110, a generation model 120, and a generation result verification unit 130. The remote computing device 200 (5) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (5) includes a control unit 310, a parameter description unit 320, and an inertial parameter measurement unit 350. The inertial parameter measurement unit 350 is connected to the control unit 310 and the actuator 410. The inertial parameter measurement unit 350 is, for example, an accelerometer, a gyroscope, or a magnetometer, used to measure acceleration and angular velocity.

[0108] exist Figure 7 In this embodiment, the tactile sensation generation method further includes steps S350 and S351. In step S350, the inertial parameter measurement unit 350 measures an inertial parameter signal IN on the actuator 410.

[0109] Then, in step S351, the control unit 310 optimizes the control signal CS based on the inertial parameter signal IN. For example, when the actuator 410 tilts to the right to perform a touch action on the right side of a virtual object, the actuator 410 will provide appropriate tactile perception according to the touch.

[0110] Please refer to Figure 8 Examples illustrate a tactile generation system 1000 (6) and a tactile generation method according to another embodiment of this disclosure. Figure 8 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 8In the embodiments, the haptic generation system 1000 (6) includes a local terminal device 100 (6), a remote computing device 200 (6), a haptic actuator 300 (6), an actuator 410, a display 420, and a speaker 430. The local terminal device 100 (6) includes an encoding unit 110, a generation model 120, and a generation result verification unit 130. The remote computing device 200 (6) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (6) includes a control unit 310 and a parameter description unit 320.

[0111] exist Figure 8 In some embodiments, the tactile sensation generation method further includes steps S122, S123, S132, S133, S420, and S430.

[0112] In step S122, the generation model 120 infers the text description information DS of an object based on the object texture image IM.

[0113] In step S123, the generation model 120 infers the sound effect feature information VS of an object based on the object texture image IM. The generation model 120 is, for example, a multimodal generation model, and steps S122 and S123 can be executed simultaneously or separately.

[0114] Then, in step S132, the result verification unit 130 transmits the object text description information DS to the control unit 310 of the haptic actuator 300 (1).

[0115] In step S133, the result verification unit 130 transmits the object sound effect feature information VS to the control unit 310 of the haptic actuator 300 (1).

[0116] Next, in step S420, the control unit 310 outputs the object text description information DS to a display 420.

[0117] In step S430, the control unit 310 outputs the object sound effect characteristic information VS to a speaker 430.

[0118] For example, when the virtual object 600 touched by the user is a dinosaur, the display 420 will show the text "dinosaur" or "giant carnivore" and play the dinosaur's roar through the speaker 430, making the user feel more realistic during the touching process.

[0119] Please refer to Figure 9 Examples illustrate a tactile generation system 1000 (7) and a tactile generation method according to another embodiment of this disclosure. Figure 9In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 9 In the embodiments, the haptic generation system 1000 (7) includes a local terminal device 100 (7), a remote computing device 200 (7), a haptic actuator 300 (7), and an actuator 410. The local terminal device 100 (7) includes an encoding unit 110, a generation model 120, a preset information storage unit 140, a comparison unit 150, and a generation result verification unit 130. The remote computing device 200 (7) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (7) includes a control unit 310 and a parameter description unit 320. The comparison unit 150 is, for example, a circuit, a circuit board, a storage device storing program code, or a chip. The preset information storage unit 140 is used to store at least one preset texture image IM0 and at least one corresponding preset texture tactile feature information FT0. The preset information storage unit 140 is, for example, any type of fixed or removable random access memory or hard disk.

[0120] exist Figure 9 In some embodiments, the tactile sensation generation method further includes steps S150 and S311.

[0121] In step S150, the comparison unit 150 compares whether the received object texture image IM is similar to the pre-stored texture image IM0 stored in the preset information storage unit 140. If there is no similar pre-stored texture image IM0, the tactile generation method flow proceeds to steps S120, S310, and S131; if there is a similar pre-stored texture image IM0, the process proceeds to step S311.

[0122] In step S311, the control unit 310 outputs a control signal CS to the actuator 410 based on the pre-stored texture tactile feature information FT0. That is, when the comparison unit 150 finds that the object texture image IM is similar to the pre-stored texture image IM0, it is not necessary to infer from the generation model 120.

[0123] Therefore, the control unit 310 can directly output the control signal CS, which makes the generation of haptic feedback faster and more in line with the application requirements of haptic feedback, reduces the amount of haptic feedback variation, and can be used in applications with preset scene images, such as game scenes or factory robot operation.

[0124] Please refer to Figure 10 Examples illustrate a tactile generation system 1000 (8) and a tactile generation method according to another embodiment of this disclosure. Figure 10 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation.Figure 10 In the embodiments, the haptic generation system 1000 (8) includes a local terminal device 100 (8), a remote computing device 200 (8), a haptic actuator 300 (8), and an actuator 410. The local terminal device 100 (8) includes an encoding unit 110, a generation model 120, a preset information storage unit 140, a comparison unit 150, and a generation result verification unit 130. The remote computing device 200 (8) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuator 300 (8) includes a control unit 310 and a parameter description unit 320.

[0125] exist Figure 10 In one embodiment, the tactile sensation generation method further includes step S134.

[0126] In step S134, the generation result verification unit 130 uses the texture tactile feature information FT inferred by the generation model 120 and the corresponding object texture image IM as at least one pre-stored texture tactile feature information FT0 and the corresponding at least one pre-stored texture image IM0.

[0127] In this embodiment, when the user inputs a similar object texture image IM again, the pre-stored texture tactile feature information FTO and the corresponding pre-stored texture image IM0 can be read from the preset information storage unit 140.

[0128] Please refer to Figure 11 The example illustrates a tactile generation system 1000 (9) and a tactile generation method according to one embodiment. Figure 11 In the diagram, dashed lines represent actions during model training, while solid lines represent actions related to inference and haptic feedback generation. Figure 11 In the embodiments described, the haptic generation system 1000 (9) includes a local terminal device 100 (9), a remote computing device 200 (9), a haptic actuation device 300 (9), and several actuators 440. The local terminal device 100 (9) includes an encoding unit 110, a generation model 120, a generation result verification unit 130, and a central computing unit 160. The remote computing device 200 (9) includes a database unit 210, an encoding unit 220, a training unit 230, a generation model 240, a generation result verification unit 250, and a model processing unit 260. The haptic actuation device 300 (9) includes several control units 310 and a parameter description unit 320. The central computing unit 160 is connected to the encoding unit 110 and the control unit 310. The actuators 440 are disposed within the machine structure actuation device. The central computing unit 160 is, for example, a circuit, a circuit board, a storage device for storing program code, or a chip.

[0129] exist Figure 11 In this embodiment, the tactile sensation generation method further includes step S160. In step S160, the central computing unit 160 allocates texture tactile feature information FT to several control units 310 based on a panoramic image PI. The panoramic image PI displays the spatial relationship between the several control units 310 and their corresponding actuators 440. Each of the several control units 310 transmits a control signal CS to its corresponding actuator 440. In this way, multiple actuators 440 can be controlled individually or uniformly, which is suitable for scenarios such as factories.

[0130] According to the above embodiments, by synchronously processing the image and the tactile feedback of the tactile generation system, combined with the tactile generation and calculation mechanism, the tactile simulation can be improved more effectively. It is not limited to individuals, but can also be applied to factory manufacturing processes or used by multiple users at the same time.

[0131] In summary, although this disclosure has been presented above with reference to embodiments, it is not intended to limit the scope of this disclosure. Those skilled in the art to which this disclosure pertains can make various modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the appended claims.

Claims

1. A tactile sensation generating system, comprising: an encoding unit configured to translate an object texture image; a generating model connected to the encoding unit, the generating model configured to infer texture tactile sensation feature information from the object texture image; a generating result verification unit connected to the generating model, the generating result verification unit configured to verify whether the texture tactile sensation feature information meets a predetermined condition; and at least one control unit connected to the generating result verification unit, the control unit configured to output a control signal to an actuator according to the texture tactile sensation feature information if the texture tactile sensation feature information meets the predetermined condition.

2. The tactile sensation generating system of claim 1, wherein the texture tactile sensation feature information is a surface asperity height, a spatial variation gradient, an undulation variation period, an undulation variation height, a friction force direction, a friction coefficient, a surface variation curve, a principal component value, or an essential modal function value, and the control signal is configured to control a vibration frequency, a vibration intensity, a wave form, a pulse width, a pulse frequency, a protrusion height, a depression depth, a friction coefficient, a normal force intensity, or a lateral force intensity.

3. The tactile sensation generating system of claim 1, wherein the generating model and the generating result verification unit are disposed in a local device.

4. The tactile sensation generating system of claim 1, wherein the generating model and the generating result verification unit are disposed in a remote computing device.

5. The tactile sensation generating system of claim 1, further comprising: a feedback evaluation unit connected to the actuator and the generating model, the feedback evaluation unit configured to provide feedback information to the generating model for optimizing the texture tactile sensation feature information.

6. The tactile sensation generating system of claim 1, further comprising: a physiological parameter measurement unit connected to the generating model, the physiological parameter measurement unit configured to measure to obtain a physiological parameter measurement signal of a user, the physiological parameter measurement signal configured to optimize the texture tactile sensation feature information.

7. The tactile sensation generating system of claim 1, further comprising: an inertial parameter measurement unit connected to the control unit and the actuator, the inertial parameter measurement unit configured to measure an inertial parameter signal of the actuator, the inertial parameter signal configured to optimize the control signal.

8. The tactile sensation generating system of claim 1, wherein the generating model is further configured to infer object text description information from the object texture image, and the generating model is further configured to infer object sound effect feature information from the object texture image, the control unit is configured to output the object text description information to a display, and the control unit is configured to output the object sound effect feature information to a speaker.

9. The tactile sensation generating system of claim 1, further comprising: a preset information storage unit configured to store at least one pre-stored texture image and corresponding at least one pre-stored texture tactile sensation feature information; a comparison unit configured to compare whether a received object texture image is similar to the pre-stored texture image, and the control unit is configured to output the control signal to the actuator according to the pre-stored texture tactile sensation feature information if the received object texture image is similar to the pre-stored texture image.

10. The tactile sensation generating system of claim 9, wherein the texture tactile sensation feature information inferred by the generating model is inputted into the pre-stored information storage unit as the at least one pre-stored texture tactile sensation feature information and corresponding at least one pre-stored texture image.

11. The tactile sensation generating system of claim 1, wherein the number of the at least one control unit is plural, the tactile sensation generating system further comprising: a central computing unit connected to the encoding unit and the control units, the central computing unit distributing the texture tactile sensation feature information to the control units according to the panoramic image.

12. A tactile sensation generating method, comprising: translating a texture image of an object; inferring texture tactile sensation feature information according to the texture image of the object; verifying whether the texture tactile sensation feature information meets a predetermined condition; and if the texture tactile sensation feature information meets the predetermined condition, outputting a control signal to at least one actuator according to the texture tactile sensation feature information.

13. The tactile sensation generating method of claim 12, wherein the texture tactile sensation feature information is surface bump height, spatial variation gradient, bump variation period, bump variation height, friction direction, friction coefficient, surface variation curve, principal component value, or proper mode value, and the control signal is used to control vibration frequency, vibration intensity, wave form, pulse width, pulse frequency, protrusion height, depression depth, friction coefficient, normal force intensity, or lateral force intensity.

14. The tactile sensation generating method of claim 12, further comprising: providing feedback information for a user; and optimizing the texture tactile sensation feature information according to the feedback information.

15. The tactile sensation generating method of claim 12, further comprising: measuring to obtain a physiological parameter measurement signal of a user; and optimizing the texture tactile sensation feature information according to the physiological parameter measurement signal.

16. The tactile sensation generating method of claim 12, further comprising: measuring an inertia parameter signal of the actuator; and optimizing the control signal according to the inertia parameter signal.

17. The tactile sensation generating method of claim 12, further comprising: inferring object text description information according to the texture image of the object; inferring object sound effect feature information according to the texture image of the object; outputting the object text description information to a display; and outputting the object sound effect feature information to a speaker.

18. The tactile sensation generating method of claim 12, further comprising: comparing whether the received texture image of the object is similar to at least one pre-stored texture image; and if the received texture image of the object is similar to the pre-stored texture image, outputting the control signal to the actuator according to pre-stored texture tactile sensation feature information.

19. The tactile sensation generating method of claim 18, further comprising: inputting the inferred texture tactile sensation feature information and corresponding texture image of the object as the pre-stored texture tactile sensation feature information and corresponding pre-stored texture image.

20. A training method of a generating model, comprising: providing a plurality of texture image samples; and ​ ​ ​ ​ ​ ​ ​ Under the limitation of the parameter description information, training of the generation model is performed according to the texture image sample until the texture tactile feature information inferred by the generation model meets a predetermined condition.

Images