Methods and devices for controlling haptic devices

Abstract

The present invention relates to a method and apparatus for controlling a haptic device. More specifically, the method for controlling a haptic device according to the present invention may include the steps of: monitoring a part of a user's body to detect contact of said part of the body with a virtual object in a virtual space; controlling the height of said pin array based on the texture of said virtual object to control the degree to which a pin array provided in a haptic device linked to said virtual space protrudes from a reference plane; and controlling the rotation of said pin array while it is protruding to a specific height in conjunction with the movement of said part of the body to provide a tactile sensation corresponding to said texture to said part of the body.

Description

Technology Field

[0001] The present invention relates to a control method and apparatus for a haptic device that provides a tactile sensation corresponding to a virtual texture. Background Technology

[0002] Recently, interest in Extended Reality, which encompasses Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), has been increasing. Accordingly, various content and devices are being developed to enhance user immersion in Extended Reality environments.

[0003] In particular, these augmented reality contents are configured to detect the movement of the user's hands, arms, or other body parts and reflect it in interactions within the virtual space, and to provide haptic stimulation in addition to visual and auditory stimuli.

[0004] For example, augmented reality content recognizes the user's movements through camera-based vision sensors or wearable tracking devices and implements the recognized user's actions in a virtual space.

[0005] In addition, in another way, augmented reality content recognizes the user's movements through a haptic device and provides physical stimuli such as vibration, pressure, and texture according to the recognized movements, thereby enabling more immersive interactions.

[0006] However, conventionally, it was common to provide only simple vibration stimulation using vibration motor-based haptic devices, and this method has limitations in reproducing the realistic tactile sensation felt when exploring surfaces with various textures, such as real silk, leather, and sandpaper, with a finger.

[0007] In particular, there was a structural limitation in that it was difficult to distinguish even fine tactile differences, such as differences in physical surface roughness or the directionality of texture, using only the same vibration stimulus.

[0008] Accordingly, haptic technology capable of implementing more diverse and sophisticated tactile sensations, such as a method of expressing texture by controlling the height, arrangement, or rotation of pins using a pin array structure, is attracting attention.

[0009] This haptic technology enables the pin array to be adjusted in real time in conjunction with the user's scanning motion, thereby making realistic texture rendering possible compared to existing simple vibration-based devices.

[0010] Therefore, there is a need for a control method and device for a haptic device that can control the height and rotation of a pin array according to contact between a part of the user's body and a virtual object, and thereby provide tactile sensations of various textures. The problem to be solved

[0011] The present invention is intended to provide a control method and apparatus for a haptic device capable of tactilely conveying the surface texture of a virtual object to a user.

[0012] In addition, the present invention is intended to provide a control method and apparatus for a haptic device capable of changing the operation of a pin array according to texture information corresponding to a virtual object.

[0013] Furthermore, the present invention aims to provide a control method and apparatus for a haptic device in which a pin array rotates according to the movement of a user's finger to control tactile sensation.

[0014] Furthermore, the present invention aims to provide a control method and apparatus for a haptic device capable of providing texture stimulation based on the height and rotation of a pin array, rather than simple vibration.

[0015] Furthermore, the present invention is intended to provide a control method and apparatus for a haptic device capable of providing different tactile sensations to a plurality of objects having different textures.

[0016] Furthermore, the present invention aims to provide a control method and apparatus for a haptic device that can implement a tactile sensation similar to roughness or softness felt in reality, even in a virtual space.

[0017] Furthermore, the present invention aims to provide a control method and device for a haptic device in which the operation of a pin array can be automatically controlled according to texture information connected to a virtual object. means of solving the problem

[0018] To solve the problem described above, a control method for a haptic device according to the present invention may include the steps of: monitoring a part of a user's body to detect contact of said part of the body with a virtual object in a virtual space; controlling the height of said pin array based on the texture of said virtual object so as to adjust the degree to which a pin array provided in a haptic device linked to said virtual space protrudes from a reference plane; and controlling the rotation of said pin array while it is protruding to a specific height in conjunction with the movement of said part of the body so as to provide a tactile sensation corresponding to said texture to said part of the body.

[0019] Furthermore, the method may further include the step of setting multiple textures based on roughness and mapping multiple virtual objects displayed in the virtual space to one of the multiple textures.

[0020] Furthermore, the step of detecting contact of the body part may detect a specific virtual object among the plurality of virtual objects that contacts the body part, and the step of controlling the height of the pin array may set the height of the pin array based on a texture corresponding to the specific virtual object.

[0021] Furthermore, the height of the pin array can be controlled to sequentially increase or decrease using input from an external input device.

[0022] Furthermore, the step of controlling the rotation of the pin array may include the step of detecting the movement of the body part while in contact with the virtual object and calculating the speed of the movement, and the step of controlling the rotation speed of the pin array in conjunction with the speed of the movement.

[0023] Furthermore, the pin array may comprise a body, a plurality of pins protruding from the body, a pinion gear connected to a motor, and a rack gear that meshes with the pinion gear and is connected to the body.

[0024] Furthermore, the rotation of the motor can be controlled based on the texture of the virtual object to adjust the height of the pin array.

[0025] Furthermore, the body is formed in a cylindrical shape, and the rack gear and the body are rotatably connected, and the haptic device may further include a rotation module that transmits the rotational force of the pin array.

[0026] Meanwhile, the haptic device according to the present invention comprises a pin array in which a plurality of pins protrude, a first driving unit that provides a driving force in the height direction of the pin array so that the pin array protrudes from a reference plane, a second driving unit that provides a rotational force to the pin array so that the pin array rotates, and a control unit that controls the first driving unit and the second driving unit. The control unit monitors a part of a user's body to detect contact of the part of the body with a virtual object in a virtual space, controls the height of the pin array based on the texture of the virtual object to adjust the degree to which the pin array protrudes from the reference plane, and controls the rotation of the pin array in a state where the pin array protrudes to a specific height in conjunction with the movement of the part of the body to provide a tactile sensation corresponding to the texture to the part of the body.

[0027] Meanwhile, the program according to the present invention is a program that is executed by one or more processes in an electronic device and is stored on a computer-readable recording medium, and the program may include instructions that perform the steps of: monitoring a part of a user's body to detect contact of said part of the body with a virtual object in a virtual space; controlling the height of said pin array based on the texture of said virtual object to control the degree to which a pin array provided in a haptic device linked to said virtual space protrudes from a reference plane; and controlling the rotation of said pin array in a state where the pin array protrudes to a specific height in conjunction with the movement of said part of the body to provide a tactile sensation corresponding to said texture to said part of the body. Effects of the invention

[0028] As described above, the control method and device of the haptic device according to the present invention can control the rotation speed of the pin array according to the user's movement. Through this, it is possible to express tactile sensations of various textures that reflect actual hand movements, such as rubbing a virtual object quickly or touching it slowly, thereby providing a more realistic haptic experience.

[0029] In addition, the control method and device of a haptic device according to the present invention can automatically adjust the height of a pin array according to the texture information of a virtual object. Through this, the difference in texture between various virtual textures such as silk, leather, and sandpaper can be clearly distinguished, so that an object can be identified tactilely without visual information.

[0030] Furthermore, the control method and device of the haptic device according to the present invention can apply a protrusion height adjustment function and a rotation module together to the pin array. Through this, delicate texture changes can be expressed as if exploring a real surface rather than simple vibration, and can be applied to professional content such as education and medical applications that require advanced tactile expression.

[0031] Furthermore, the control method and device of the haptic device according to the present invention can systematically implement a mapping structure between a virtual object and a pin array. Through this, a user can experience texture stimulation corresponding to an object touched in a virtual space without separate input, and can improve immersion and naturalness of interaction.

[0032] Furthermore, the control method and device of the haptic device according to the present invention can manually adjust the height of the pin array through an external input device. Through this, convenience settings can be manually corrected in an experimental or user test environment, or the haptic feedback intensity can be directly controlled to suit the user's preference.

[0033] Furthermore, the control method and device of a haptic device according to the present invention can detect and control contact between a part of a user's body and a virtual object. Through this, haptic feedback is activated only when the user actually touches a specific object, thereby preventing unnecessary energy consumption and ensuring a degree of consistency with actual responsiveness. Brief explanation of the drawing

[0034] FIG. 1a is a conceptual diagram showing the configuration of a haptic device according to one embodiment of the present invention. FIG. 1b is a conceptual diagram showing the configuration of a pin array according to one embodiment of the present invention. FIG. 1c is a conceptual diagram showing the height control structure and output of a pin array according to one embodiment of the present invention. FIG. 1d is a conceptual diagram showing a structure in which a pin array performs a rotational operation according to one embodiment of the present invention. FIG. 2 is a conceptual diagram for explaining a haptic device according to the present invention. FIG. 3 is a flowchart illustrating a method for controlling a haptic device according to the present invention. FIGS. 4, FIGS. 5, FIGS. 6, FIGS. 7 and FIGS. 8 are conceptual diagrams for explaining a control method of a haptic device according to the present invention. Specific details for implementing the invention

[0035] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0036] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0037] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0038] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0039] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] FIG. 1a is a conceptual diagram showing the configuration of a haptic device according to one embodiment of the present invention.

[0041] FIG. 1b is a conceptual diagram showing the configuration of a pin array according to one embodiment of the present invention.

[0042] FIG. 1c is a conceptual diagram showing the height control structure and output of a pin array according to one embodiment of the present invention.

[0043] FIG. 1d is a conceptual diagram showing a structure in which a pin array performs a rotational operation according to one embodiment of the present invention.

[0044] Referring to FIG. 1a, the haptic device (10) includes a pin array (100) that contacts a part of the user's body, and the pin array (100) may include a plurality of pins (110) and a first driving unit (140) and a second driving unit (150) for controlling the protrusion height and rotational movement of the plurality of pins (110).

[0045] The pin array (100) can be formed, for example, as a cylindrical body, and can be formed in a shape in which a plurality of pins (110) are arranged in a radial or grid structure on the upper surface of the body.

[0046] A plurality of pins (110) directly contact a part of the user's body (or skin) to transmit tactile sensations, and each protrusion height and rotation state can be controlled individually or collectively according to the texture information of the virtual object.

[0047] The first driving unit (140) is configured to control the protrusion height relative to a reference plane of a plurality of pins (110) included in the pin array (100), and may include, for example, a pinion gear (120) connected to a motor and a rack gear (130) connected to the body of the pin array (100).

[0048] When the pinion gear (120) rotates, the rack gear (130) moves in a straight line, and this movement is transmitted through the entire pin array or the individual pin drive shafts to change the up and down position of the pins.

[0049] The second drive unit (150) is a rotational drive means that applies rotational force to cause the pin array (100) to perform rotational movement around a cylindrical body axis, and may include, for example, a belt, a pulley, a gear train, a motor, etc.

[0050] Rotation is performed in conjunction with the user's navigation direction, gestures, scanning speed, etc., and through rotational movements, haptic feedback such as the directionality of texture or frictional characteristics can be provided.

[0051] The haptic device (10) may further include a fixing part (not shown) to support the configuration of the first driving part (140) and the second driving part (150) and to stably combine the entire part. For example, the fixing part may be made of a structure such as a frame, a housing, or a guide rail.

[0052] The haptic device (10) configured in this way can independently control the protrusion height and rotational movement of a plurality of pins (110) of a pin array (100) according to the texture information of the virtual object when a part of the user's body comes into contact with a virtual object in a virtual space.

[0053] More specifically, referring to FIG. 1b, the pin array (100) may include at least one of a plurality of pins (110), a cylindrical body, a pinion gear (120), and a rack gear (130), which may be a hardware configuration for transmitting tactile sensations to a part of a user's body (or skin) based on texture information of a virtual object.

[0054] The external shape of the pin array (100) is formed, for example, as a cylindrical body, and a plurality of pins (110) arranged at regular intervals can be arranged to protrude from the upper surface of the body. The plurality of pins (110) can move up and down in the vertical direction and can be configured to provide tactile stimulation by making direct contact with the user's finger. Here, the protruding state of the plurality of pins (110) can be adjusted by reflecting the texture information of a specific virtual object.

[0055] A mechanical height control device corresponding to a first drive unit (140) may be provided inside the pin array (100), and may include a pinion gear (120) and a rack gear (130). The pinion gear (120) is rotated by an electric motor, and this rotational motion can be converted into linear motion by engaging with the rack gear (130). The rack gear (130) is coupled with the body of the pin array (100), and as the body rises or falls overall, the protrusion height relative to the reference plane of the plurality of pins (110) can also change.

[0056] Multiple pins (110) may be controlled individually, or specific pins may be controlled collectively, depending on the implementation method of the virtual texture that the pin array intends to render.

[0057] Referring to FIG. 1c, FIG. 1c (a) shows the structure of a first driving unit (140) for height control, and FIG. 1c (b) and (c) show examples in which a plurality of pins (110) included in a pin array according to the operation of the first driving unit have different protrusion heights.

[0058] Figure 1c (a) shows the structure of a first drive unit (140) as a driving means for adjusting the height of a pin array (100), and may include a pinion gear (120) and a rack gear (130). The pinion gear (120) is driven by rotation by a motor, and the rotational motion is converted into linear motion by engaging with the rack gear (130), and the vertical movement of the rack gear (130) is transmitted to the body of the pin array (100), thereby enabling the entire protruding height of a plurality of pins (110) to be controlled collectively.

[0059] Figure 1c (b) shows a state in which a plurality of pins (110) included in the pin array (100) by the first driving unit (140) have a high protrusion height relative to the reference plane, which may correspond to a case of implementing haptic feedback corresponding to a virtual object such as a texture with a high roughness value, i.e., sandpaper, wood, stone, etc., with a rough surface.

[0060] (c) of FIG. 1c represents a state in which a plurality of pins (110) have a low protrusion height close to a reference plane according to the operation of the first driven unit (140), which may correspond to a case of implementing haptic feedback corresponding to a virtual object such as a smooth texture with a low roughness value, i.e., silk, glass, cloth.

[0061] Referring to FIG. 1d, the pin array (100) includes a plurality of pins (110) and can be formed in the shape of a cylindrical body. At this time, the pin array (100) is made rotatable about a cylindrical axis, and the plurality of pins (110) are arranged at regular intervals on the top of the body to provide tactile stimulation by contacting the user's skin.

[0062] The rotation of the pin array (100) can be performed by a second drive unit (150). The second drive unit (150) is a mechanical means for rotational driving and may include, for example, a rotary motor, a belt, a pulley, a gear set, etc., and may be performed in a manner that transmits rotational force to the outer or central axis of the pin array.

[0063] The control unit can control the second drive unit (150) based on movement direction and speed information while in contact with a virtual object among the movements detected by the user. When the user moves as if exploring the surface with a finger, the corresponding scan movement, direction, and speed can be reflected in the rotation direction and rotation speed of the second drive unit. For example, the faster the user's movement speed, the greater the rotation speed of the pin array, and the slower the user's movement speed, the slower the user rotates, thereby providing tactile changes according to the scan speed.

[0064] FIG. 2 is a conceptual diagram for explaining a haptic device according to the present invention.

[0065] Referring to FIG. 2, the haptic device (200) according to the present invention may include at least one of a receiving unit (210), an output unit (220), a sensing unit (230), a storage unit (240), and a control unit (250).

[0066] The receiving unit (210) can receive data by being connected to an external virtual space system or content server, etc., via a wireless or wired network.

[0067] For example, at least one of a control signal for providing haptic feedback, virtual object information, and texture data can be received.

[0068] As another example, the receiver (210) can receive a control signal containing contact information between a specific virtual object in a virtual space and a part of the user's body.

[0069] The output unit (220) may include a first driving unit and a second driving unit that are involved in driving the pin array according to the control of the control unit (250), and may provide direct physical stimulation to the user's tactile sense.

[0070] For example, the output unit (220) can adjust the protrusion height of a pin array composed of multiple pins or rotate the entire pin array to provide a texture stimulus corresponding to the user's search direction and speed.

[0071] The detection unit (230) may include sensors for recognizing the movement of a user's finger, hand, or part of the body, and may determine whether a part of the user's body has come into contact with a virtual object. For example, the detection unit (230) may include a camera-based sensor, a proximity sensor, an IMU sensor, etc., and through this, information such as whether contact has occurred, the location of contact, the direction of movement, and speed can be collected in real time.

[0072] The storage unit (240) may be configured to store data and commands necessary for the operation of the haptic device (200) according to the present invention.

[0073] For example, at least one of texture data, virtual object mapping information, driving control commands, and reference setting values ​​required for the operation of a haptic device may be stored in the storage unit (240).

[0074] As another example, the storage unit (240) may store roughness coefficient values ​​for multiple texture types, and may also store a mapping table containing information that the textures are mapped to specific virtual objects.

[0075] The control unit (250) can perform a series of processes related to the present invention and can perform the role of controlling the overall operation of the haptic device (200).

[0076] For example, the control unit (250) can receive virtual object information and texture data through the receiving unit (210) and control the operation of the pin array (100) by referring to user input information obtained from the detection unit (230) and reference information stored in the storage unit (240).

[0077] In another example, when the control unit (250) detects that a part of the user's body has come into contact with a specific virtual object through the detection unit (230), it can utilize the texture information of the corresponding virtual object received through the receiving unit (210) and refer to the mapping information (e.g., texture and roughness) stored in the storage unit (240) to control the height of a plurality of pins (110) through the first driving unit (140).

[0078] As another example, when the direction and speed of movement of the user's hand or finger are detected, the control unit (250) can adjust the rotation speed of the pin array (100) through the second driving unit (150).

[0079] Below, based on the configuration of the haptic device (200) described above, a method for controlling a haptic device according to the present invention will be explained in more detail.

[0080] FIG. 3 is a flowchart for explaining a method of controlling a haptic device according to the present invention, and FIG. 4, FIG. 5, FIG. 6, FIG. 7 and FIG. 8 are conceptual diagrams for explaining a method of controlling a haptic device according to the present invention.

[0081] In the present invention, a process of detecting contact of a part of the body with a virtual object in a virtual space by monitoring a part of the user's body may be performed (S310, see FIG. 3).

[0082] The control unit (250) can monitor the movement and position of a part of the user's body (e.g., fingers, palm, wrist, etc.) when the user accesses and explores virtual objects in a virtual space (S410, see FIG. 4).

[0083] In one embodiment, as illustrated in FIG. 5, a user can interact with a plurality of virtual objects in a virtual space while wearing a head-mounted display (HMD) that is linked to a virtual reality (VR) system. For example, the plurality of virtual objects may have an appearance similar to real-world objects such as cups, boxes, and bricks.

[0084] Here, virtual space is a digital simulation environment separated from the real world, which can be implemented through technologies such as computer graphics (CG), virtual reality (VR), augmented reality (AR), and mixed reality (MR), and can be configured so that users perceive it as a real space through senses such as sight, hearing, and touch.

[0085] In addition, virtual objects refer to objects or things existing within a virtual space; although they do not actually exist, they can be visually represented so that users perceive them as real.

[0086] Here, a part of the user's body may be a body part that is the actual detection target that generates contact events with a virtual object in a virtual space and receives physical tactile stimulation from a haptic device. Although specific body parts such as fingers, palms, or the entire hand are given as examples in this specification, the invention is not limited thereto and may be applied to various body parts such as soles of the feet, elbows, and faces.

[0087] The sensing unit (230) may include a sensor for monitoring the movement or position of a part of the user's body, for example, at least one of a camera-based sensor, an optical position tracking sensor, an infrared (IR) sensor, a depth sensor, and a wearable motion capture sensor that the user can wear.

[0088] The detection unit (230) can collect data in real time to determine whether a part of the user's body has come into contact with a virtual object in the virtual space through these sensors. For example, the position of a fingertip can be captured as an image through a camera linked to an HMD, and the position of the body part can be tracked through this.

[0089] The control unit (250) can determine whether a part of the user's body has come into contact with a specific virtual object in the virtual space based on data collected through the detection unit (230). To this end, the control unit (250) receives texture information and spatial location information for a plurality of virtual objects through the receiving unit (210), and can correlate the information of the virtual objects stored in the storage unit (240) with the location data of the detected part of the body.

[0090] The control unit (250) can detect contact by determining whether a part of the user's body has approached a specific virtual object within the virtual space or whether the contact event condition with the object has been satisfied.

[0091] Here, the contact event condition is a criterion that can be determined when a part of the user's body is located within a certain distance of the surface of a virtual object or enters a predefined collision area, and can be set based on a 3D coordinate-based collision detection algorithm or a distance threshold.

[0092] In addition, when simultaneous access to multiple virtual objects occurs, the control unit (250) can analyze the relative distance between the object and a part of the body, the direction of access, the intention of the movement, etc., and determine the specific object that needs to be provided with actual haptic feedback to the user.

[0093] Haptic feedback refers to a method of conveying sensory information by providing physical stimulation to a part of a user's body. For example, haptic feedback may include at least one of tactile feedback, force feedback, texture feedback, mechanical feedback, and multiple feedback.

[0094] In this way, the control unit (250) can detect in real time the location of a part of the user's body in a virtual space and whether it is in contact with a specific virtual object by using data collected from various sensors of the detection unit (230).

[0095] Meanwhile, as illustrated in FIG. 6, the control unit (250) can set a plurality of textures based on roughness to provide a realistic tactile sensation to the user in a virtual space, and map each of the set plurality of textures to a plurality of virtual objects.

[0096] Here, roughness is an indicator that numerically expresses the texture characteristics of a surface perceived tactilely by a user, and can be defined, for example, as a value between 0.0 and 1.0. This roughness value can be used as data to control the protrusion height and rotational speed of a plurality of pins (110) of a pin array (100).

[0097] The control unit (250) can store a plurality of distinguishable roughness values ​​in advance in the storage unit (240) to configure a texture corresponding to each value. For example, Texture A is defined with a roughness of 0.1 and can simulate a smooth surface such as silk or glass, Texture B is defined with a roughness of 0.3 and can express an intermediate texture such as a box or leather, and Texture C is defined with a roughness of 0.9 and can reproduce a rough surface such as brick or sandpaper.

[0098] The control unit (250) can store such multiple texture information in the storage unit (240) and then map an appropriate texture to each of the multiple virtual objects displayed in the virtual space. This mapping information can be used as a standard to provide realistic haptic feedback by immediately controlling the multiple pins (110) of the pin array (100) when a user comes into contact with a virtual object (S420, see FIG. 4).

[0099] Mapped information can be stored in a storage unit (240) as a mapping table. Here, the mapping table is a table representing the relationship between a virtual object and the texture information of the virtual object. For example, a virtual object composed of a glass cup or silk cloth can be mapped to texture A, a box or leather wallet can be mapped to texture B, and a brick wall or sandpaper board can be mapped to texture C.

[0100] Texture information is information that quantifies or datafies the surface texture characteristics of a virtual object, and is reference data for providing a tactile or force sensation to a user through the control of a pin array of a haptic device. Texture information may include, for example, at least one of roughness value, friction value, elasticity, pattern shape, surface orientation, and temperature response characteristics.

[0101] For convenience of explanation, three textures are described as examples in this specification, but the present invention is not limited thereto and multiple textures including various roughness values ​​and other information (e.g., elasticity, friction, surface pattern, etc.) may be set.

[0102] In the present invention, a process of controlling the height of a pin array based on the texture of a virtual object can be performed to adjust the degree to which a pin array provided in a haptic device linked with a virtual space protrudes from a reference plane (S320, see FIG. 3).

[0103] The control unit (250) can control the height of the pin array (100) included in the haptic device (10) based on the texture information of the virtual object after contact between a part of the user's body and a specific virtual object in the virtual space is detected.

[0104] A plurality of texture information pre-set based on roughness can be stored in the storage unit (240) of the haptic device (10), and each texture can be used as a standard for adjusting the protrusion height of a plurality of pins (110).

[0105] For example, if the roughness value of texture A is set to 0.1, it may display a smooth texture, and if the roughness value of texture C is set to 0.9, it may display a rough texture.

[0106] The control unit (250) can extract texture information corresponding to the contacted virtual object using a mapping table stored in the storage unit (240).

[0107] The control unit (250) can determine the protrusion height of a plurality of pins (110) included in the pin array (100) based on the roughness value included in the extracted texture information.

[0108] Here, the protrusion height may refer to the degree to which a plurality of pins (110) protrude upward from the reference plane of the haptic device (10), i.e., the protrusion distance. For example, the protrusion height may be directly proportional to the roughness value of the corresponding texture, or may be determined according to a correspondence relationship according to a pre-set mapping function or mapping table.

[0109] In one embodiment, referring to FIG. 7(a), when a specific virtual object in contact with a part of a user's body in virtual space corresponds to texture A (roughness: 0.1), a plurality of pins (110) included in the pin array (100) can be controlled to have a protrusion height lower than the reference plane.

[0110] In another embodiment, referring to FIG. 7(b), when a specific virtual object in contact with a part of a user's body in virtual space corresponds to texture C (roughness: 0.9), a plurality of pins (110) included in the pin array (100) can be controlled to have a protrusion height higher than the reference plane.

[0111] The control unit (250) can control the overall height of the pin array (100) by driving the pinion gear (120) and the rack gear (130) included in the first drive unit (140).

[0112] At this time, the rotation of the pinion gear (120) is converted into linear motion of the rack gear (130), raising or lowering the body of the pin array (100), and the protrusion height of the plurality of pins (110) relative to the reference plane can also be adjusted accordingly (see FIGS. 1a to 1c).

[0113] Additionally, the control unit (250) can adjust the height of the pin array (100) to suit the user by using the user's individual settings or pressure values ​​input through an external input device.

[0114] For example, the external input device may be in the form of a pressure sensor, a rotary knob, a slider, a button, etc., and when a user inputs through the external input device, the control unit (250) can control the first driving unit (140) according to the input value to sequentially increase or decrease the protrusion height of the plurality of pins (110) (S430, see FIG. 4).

[0115] In this way, the control unit (250) can precisely control the protrusion height of the pin array (100) of the haptic device (10) by taking into account not only the texture information of the virtual object but also user input and sensitivity settings.

[0116] In the present invention, a process of controlling the rotation of a pin array while it protrudes to a specific height in conjunction with the movement of a body part can be performed to provide a tactile sensation corresponding to a texture to a body part (S320, see FIG. 3).

[0117] The control unit (250) can control the rotation of the pin array (100) in conjunction with the movement that occurs when a part of a user's body (e.g., fingers) navigates over the surface of a specific virtual object while a plurality of pins (110) included in the pin array (100) are protruding at a specific height.

[0118] Here, navigation refers to a movement to perceive the texture of a surface while the user moves in contact with the surface of a virtual object within a virtual space using a finger, palm, or other part of the body. Navigation movements may be performed by actions such as rubbing, swiping, and sliding.

[0119] Here, the state of being protruded to a specific height may be a state in which a plurality of pins (110) are protruded from the reference plane using the first driving unit (140) based on the texture information of the virtual object as described above.

[0120] In this state, part of the user's body comes into contact with the top surface of the pin array (100) and can explore the surface in various directions.

[0121] The control unit (250) can detect the movement of a part of the body in real time and calculate the direction and speed of movement. For example, when a user rubs the surface of the pin array (100) with a finger in the left-right or front-back direction, the movement information of the user's part of the body can be collected and analyzed in real time through the detection unit (230).

[0122] The sensing unit (230) is composed of, for example, a pressure sensor, a touch sensor, an optical sensor, a motion sensor, etc., and can accurately measure the speed and direction information of the user's movement.

[0123] The control unit (250) can calculate the user's search speed and direction using movement information collected through the detection unit (230), and determine the rotation speed and rotation direction of the pin array (100) based on the calculated search speed and direction.

[0124] According to the rotational speed and direction calculated here, the control unit (250) can control the rotation of the pin array (100) using the second driving unit (150) (S440 see FIG. 4).

[0125] The second drive unit (150) may include, for example, a motor, a belt, and a pulley structure, and the control unit (250) can drive the motor of the second drive unit (150) to rotate the pin array (100) at a desired speed and direction (see FIG. 1a and FIG. 1d).

[0126] When the user moves their finger faster while navigating a specific virtual object on the surface of the pin array (100), the control unit (250) can increase the rotation speed of the pin array (100) in conjunction with the user's movement speed, and when the movement speed slows down, it can decrease the rotation speed of the pin array (100).

[0127] In this way, the control unit (250) can provide friction and skin-slip phenomena to the user by controlling the user's search speed and rotation speed in conjunction.

[0128] In one embodiment, referring to FIG. 8(a), when a user slowly explores the top of the pin array (100) with a finger, the control unit (250) can accordingly reduce the rotation speed of the pin array and rotate it in a corresponding direction, providing a slip phenomenon in which the skin is slightly pushed.

[0129] In another embodiment, referring to FIG. 8(b), when a user quickly navigates the top of the pin array (100) with a finger, the control unit (250) can increase the rotation speed of the pin array accordingly to provide a stronger frictional sensation and haptic feedback.

[0130] With this rotational control, the user can experience a skin-slip sensation as if swiping or rubbing a real surface.

[0131] Additionally, when the user moves their finger in a specific direction, the rotation direction of the pin array (100) can be set to correspond to the user's direction of search. For example, if the user's finger moves to the left, the pin array (100) can rotate to the right, opposite to the user's skin, or rotate in the same direction in response.

[0132] Here, the rotation direction setting can be varied depending on the characteristics of the virtual object, user preferences, and the level of immersion required.

[0133] The storage unit (240) may store various optimal rotation speed or rotation pattern information for each texture in advance, and the control unit (250) may perform rotation control of the pin array (100) for a specific virtual object based on the rotation speed or rotation pattern information for each texture stored in the storage unit (240).

[0134] For example, in the case of a very rough texture, a strong tactile sensation can be provided through a fast rotation speed and strong rotational force, and in the case of a smooth texture, a soft tactile sensation can be provided through a slow rotation speed and weak rotational force.

[0135] As described above, the control method and device of the haptic device according to the present invention can control the rotation speed of the pin array according to the user's movement. Through this, it is possible to express tactile sensations of various textures that reflect actual hand movements, such as rubbing a virtual object quickly or touching it slowly, thereby providing a more realistic haptic experience.

[0136] In addition, the control method and device of a haptic device according to the present invention can automatically adjust the height of a pin array according to the texture information of a virtual object. Through this, the difference in texture between various virtual textures such as silk, leather, and sandpaper can be clearly distinguished, so that the object can be identified tactilely without visual information.

[0137] Furthermore, the control method and device of the haptic device according to the present invention can apply a protrusion height adjustment function and a rotation module together to the pin array. Through this, delicate texture changes can be expressed as if exploring a real surface rather than simple vibration, and can be applied to professional content such as education and medical applications that require advanced tactile expression.

[0138] Furthermore, the control method and device of the haptic device according to the present invention can systematically implement a mapping structure between a virtual object and a pin array. Through this, a user can experience texture stimulation corresponding to an object touched in a virtual space without separate input, and can improve immersion and naturalness of interaction.

[0139] Furthermore, the control method and device of the haptic device according to the present invention can manually adjust the height of the pin array through an external input device. Through this, convenience settings can be manually corrected in an experimental or user test environment, or the haptic feedback intensity can be directly controlled to suit the user's preference.

[0140] Furthermore, the control method and device of a haptic device according to the present invention can detect and control contact between a part of a user's body and a virtual object. Through this, haptic feedback is activated only when the user actually touches a specific object, thereby preventing unnecessary energy consumption and ensuring a degree of consistency with actual responsiveness.

[0141] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).

[0142] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.

[0143] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0144] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.

[0145] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.

[0146] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

Claim 1 A method for controlling a haptic device, characterized by comprising: a step of monitoring a part of a user's body to detect contact of said part of the body with a virtual object in a virtual space; a step of controlling the rotation of a pinion gear connected to a motor that constitutes the pin array based on the texture of said virtual object to control the degree to which a pin array provided in a haptic device linked to said virtual space protrudes from a reference plane, and a step of controlling the protrusion height of a plurality of pins protruding from said body through a rack gear connected to a body and meshing with said pinion gear; and a step of controlling the rotation of said pin array while the plurality of pins protrude to a specific height in conjunction with the movement of said part of the body to provide a tactile sensation corresponding to said texture to said part of the body. Claim 2 A method for controlling a haptic device according to claim 1, further comprising the step of setting a plurality of textures based on roughness and mapping a plurality of virtual objects displayed in the virtual space to one of the plurality of textures. Claim 3 A method for controlling a haptic device according to claim 2, wherein the step of detecting contact of a part of the body detects a specific virtual object among a plurality of virtual objects that contacts the part of the body, and the step of controlling the protrusion height of the plurality of pins sets the height of the pin array based on a texture corresponding to the specific virtual object. Claim 4 A method for controlling a haptic device according to claim 3, characterized in that the height of the pin array is controlled to sequentially increase or decrease using input from an external input device. Claim 5 A method for controlling a haptic device according to claim 1, wherein the step of controlling the rotation of the pin array comprises: a step of detecting the movement of a part of the body while in contact with the virtual object and calculating the speed of the movement; and a step of controlling the rotation speed of the pin array in conjunction with the speed of the movement. Claim 6 delete Claim 7 A method for controlling a haptic device according to claim 1, characterized by controlling the rotation of the motor based on the texture of the virtual object to adjust the height of the pin array. Claim 8 A method for controlling a haptic device according to claim 1, wherein the body is formed in a cylindrical shape, the rack gear and the body are rotatably connected, and the haptic device further comprises a rotation module that transmits the rotational force of the pin array. Claim 9 A haptic device comprising: a pin array in which a plurality of pins protrude; a first driving unit that provides a driving force in the height direction of the pin array so that the pin array protrudes from a reference plane; a second driving unit that provides a rotational force to the pin array so that the pin array rotates; and a control unit that controls the first driving unit and the second driving unit, wherein the control unit monitors a part of a user's body to detect contact of the part of the body with a virtual object in a virtual space, controls the rotation of a pinion gear connected to a motor that constitutes the pin array based on the texture of the virtual object to adjust the degree to which the pin array protrudes from the reference plane, controls the protrusion height of a plurality of pins protruding from the body through a rack gear connected to the body that meshes with the pinion gear, and controls the rotation of the pin array in a state where the plurality of pins protrude to a specific height in conjunction with the movement of the part of the body to provide a tactile sensation corresponding to the texture to the part of the body. Claim 10 A program stored on a computer-readable recording medium that is executed by one or more processes in an electronic device, the program comprising: a step of monitoring a part of a user's body to detect contact of said part of the body with a virtual object in a virtual space; a step of controlling the rotation of a pinion gear connected to a motor that constitutes the pin array based on the texture of said virtual object, and controlling the protrusion height of a plurality of pins protruding from the body through a rack gear connected to the body that meshes with said pinion gear, so as to control the degree to which a pin array provided in a haptic device linked to said virtual space protrudes from a reference plane; and a step of controlling the rotation of said pin array while the plurality of pins protrude to a specific height in conjunction with the movement of said part of the body to provide a tactile sensation corresponding to said texture to said part of the body.

Images