Bidirectional force feedback control system using haptic glove device and method thereof

Abstract

Disclosed are a bidirectional force feedback control system using a haptic glove device and a method thereof. The bidirectional force feedback control system using a haptic glove device, according to one embodiment of the present invention, comprises: a contact pressure detection unit for collecting contact pressure data inputted from at least one elastic member provided at one end of a prespecified haptic glove device; a bending detection unit for measuring movement in a finger joint region through at least one rotary member provided to the haptic glove device, and generating rotary member bending angle data; a data communication unit for transmitting the contact pressure data and the bending angle data to a prespecified user terminal; and a force feedback control unit for controlling the haptic glove device on the basis of a force feedback control signal received from the user terminal.

Description

Bidirectional force control system using a haptic glove device and method thereof

[0001] The present invention relates to a two-way force feedback control system using a haptic glove device, and more specifically, to a force feedback control system and method thereof that implements interaction by measuring the degree of bending of a finger and contact pressure information and providing a grip feeling corresponding to the appearance and texture of a virtual object.

[0002] The technology of haptic control, which transmits physical forces or phenomena occurring in virtual objects in a virtual environment to specific parts of the body to control them so that they feel as if they are actually touching the virtual object and can be interacted with, is being developed as devices that provide a more immersive experience to virtual reality users due to the diversification of virtual reality hardware, haptic feedback, mixed reality, content, popularization of virtual reality, and improvement of development tools.

[0003] Among them, haptic gloves are devices that transmit tactile sensations such as texture, vibration, temperature, force, and movement according to interaction with virtual objects during a virtual content experience, and can particularly affect the hands among human body parts.

[0004] Korean Patent Publication No. 10-2020-0036261, a prior art, discloses a method for providing realistic feedback when contacting a virtual object and a device therefor, and Korean Patent Registration No. 10-1726388 discloses a device for providing a realistic feeling to a user interacting with a virtual object in a virtual space.

[0005] However, in the case of conventional technology, there is a problem in that the physical force calculated in response to the virtual object is implemented in a one-way manner to unilaterally transmit it to the user, making interaction with virtual content unnatural.

[0006] Therefore, research is required on a two-way sensory control system and method that provides various motion sensations in response to real-time movements and objects of a user experiencing virtual reality by equipping the user with equipment that provides virtual reality and extended reality.

[0007] The purpose of the present invention is to provide a two-way haptic control system and method using a haptic glove device capable of two-way haptic control that transmits necessary information between a virtual object and a real body by first generating a haptic control signal corresponding to virtual content set through a user terminal and secondarily generating a corrected haptic control signal corresponding to press pressure information and finger joint bending information collected through a haptic glove device.

[0008] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems to be solved by the present invention that are not mentioned herein will be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from the description below.

[0009] A two-way force control system using a haptic glove device according to an embodiment of the present invention includes a contact pressure detection unit that collects contact pressure data input from at least one elastic member provided on one end of a predetermined haptic glove device, a bending detection unit that measures movement of a finger joint area through at least one rotating member provided on the haptic glove device and generates rotating member bending angle data, a data communication unit that transmits the contact pressure data and the bending angle data to a predetermined user terminal, and a force control unit that controls the haptic glove device based on a force control signal received from the user terminal.

[0010] In addition, the contact pressure detection unit includes an object recognition unit that determines whether an object coming into contact with the haptic glove device is a predetermined object, and a pressure sensing unit that converts a resistance value that changes based on a pressing pressure of an elastic member into contact pressure data when the object recognition unit recognizes the object coming into contact with the haptic glove device as an object, and the bending detection unit is characterized in that it calculates bending angle data based on a resistance value of a variable resistor provided in a rotating member, and converts the resistance value of the variable resistor that changes based on a movement of a finger joint area into the bending angle data.

[0011] In addition, the object recognition unit is characterized in that it recognizes a communication tag provided on an object through a tag recognition member provided on a haptic glove device, and extracts tag information of the communication tag recognized through the tag recognition member to determine whether the object is included in virtual content set through a user terminal.

[0012] In addition, the haptic control unit is characterized by including a first haptic control unit that generates a first haptic control signal corresponding to virtual content set through a user terminal and controls a haptic glove device based on the first haptic control signal, and a second haptic control unit that collects contact pressure data and bending angle data of the haptic glove device to which the first haptic control signal is applied, generates a second haptic control signal that corrects the contact pressure data and bending angle data corresponding to the virtual content, and controls the haptic glove device based on the second haptic control signal.

[0013] In addition, the invention further comprises a fault diagnosis unit that monitors a resistance value measured by at least one of a contact pressure detection unit and a bend detection unit, and diagnoses whether a fault has occurred in the contact pressure detection unit and the bend detection unit.

[0014] According to the present invention, a haptic control signal is first generated in response to virtual content set through a user terminal, and a corrected haptic control signal is secondarily generated in response to press pressure information and finger joint bending information collected through a haptic glove device, thereby enabling a two-way haptic control that mutually transmits necessary information between a virtual object and a real body.

[0015] FIG. 1 is a configuration diagram of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention.

[0016] FIG. 2 is a drawing for explaining a haptic glove device of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention.

[0017] FIG. 3 is a drawing for explaining a contact pressure detection unit of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention.

[0018] FIG. 4 is a drawing for explaining a haptic control unit of a two-way haptic control system using a haptic glove device according to an embodiment of the present invention.

[0019] Specific details, including the problems to be solved, means of solving them, and effects of the invention, are included in the embodiments and drawings described below. The advantages and features of the present invention, and methods for achieving them, will become clearer with reference to the embodiments described below in detail, along with the accompanying drawings.

[0020] The scope of the present invention is not limited to the embodiments described below, and various modifications may be made by a person having ordinary knowledge in the relevant technical field without departing from the technical spirit of the present invention.

[0021]

[0022] Hereinafter, the title of the invention of the present invention is described in detail with reference to the attached drawing 1.

[0023] FIG. 1 is a configuration diagram of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention, FIG. 2 is a diagram for explaining a haptic glove device of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention, FIG. 3 is a diagram for explaining a contact pressure detection unit of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention, and FIG. 4 is a diagram for explaining a feedback control unit of a two-way feedback control system using a haptic glove device according to an embodiment of the present invention.

[0024]

[0025] Example 1

[0026] Referring to FIGS. 1 and 2, a two-way pressure control system (100) using a haptic glove device according to an embodiment of the present invention may include a contact pressure detection unit (110), a bending detection unit (120), a data communication unit (130), and a pressure control unit (140).

[0027]

[0028] More specifically, the contact pressure detection unit (110) collects contact pressure data input from at least one elastic member (11) provided on one end of a predetermined haptic glove device (10), the bending detection unit (120) is provided on the haptic glove device (10) to measure movement of a finger joint area through at least one rotating member (12) and generate rotating member bending angle data, the data communication unit (130) transmits the contact pressure data and the bending angle data to a predetermined user terminal (20), and the force control unit (140) can control the haptic glove device (10) based on a force control signal received from the user terminal (20).

[0029] At this time, the data communication unit (130) may receive at least one piece of virtual content information from the user terminal (20) and link the haptic glove device (10) with a preset head mounted display.

[0030] Additionally, the haptic glove device (10) and the head mounted display can be combined with the user terminal (10).

[0031] Accordingly, the user (wearer) can experience the touch, texture, etc. corresponding to the object included in the virtual content through the haptic glove device (10) while watching the content in which the virtual content is played through the head mounted display device.

[0032]

[0033] For example, the data communication unit (130) may monitor the linkage status of the haptic glove device (10), the head mounted display, and the user terminal (20), and may also monitor whether the virtual content is played normally.

[0034]

[0035] Meanwhile, as illustrated in FIG. 3, the contact pressure detection unit (110) may include an object recognition unit (111) that determines whether an object coming into contact with the haptic glove device (10) is a predetermined object, and a pressure sensing unit (112) that converts a resistance value that changes based on the pressing pressure of the elastic member (11) into the contact pressure data when the object coming into contact with the haptic glove device (10) is recognized as the object by the object recognition unit (111).

[0036] That is, when an arbitrary object is contacted through the haptic glove device (10), it is determined whether the object contacted by the haptic glove device (10) is a designated object, and when the contact object is recognized as the object, the degree of pressing pressure between the object and the haptic glove device (10) can be calculated.

[0037] For example, the object recognition unit (111) can recognize a communication tag provided on the object through a tag recognition member provided on the haptic glove device (10).

[0038] More specifically, when the distance between the tag recognition member and the communication tag is less than a preset interval, the communication tag can be recognized through the tag recognition member, so the object recognition unit (111) can recognize that the object has come into contact with the haptic glove device (10) when the communication tag is recognized through the tag recognition member.

[0039] At this time, the recognition interval between the communication tag and the tag recognition member can be set through the user terminal.

[0040]

[0041] In addition, the object recognition unit (111) can extract tag information of the communication tag recognized through the tag recognition member and determine whether an object corresponding to the tag information is included in the virtual content set through the user terminal (20).

[0042] At this time, if an object corresponding to the tag information is included in the virtual content, the pressure sensing unit (112) can measure the pressing pressure information between the object contacting the haptic glove device (10) and the haptic glove device (10).

[0043] More specifically, the elastic member (11) is formed of a conductive material (e.g., conductive silicone, etc.) having elastic properties whose resistance value linearly changes depending on the pressing pressure, and the resistance value increases in response to the pressure applied by the user wearing the haptic glove device (10) to grip the object.

[0044] Accordingly, the pressure sensing unit (112) can calculate the contact pressure data based on the resistance value measured through the elastic member (11).

[0045]

[0046] Meanwhile, the bending detection unit (120) calculates the bending angle data based on the resistance value of the variable resistor provided in the rotating member (12), and can convert the resistance value of the variable resistor, which changes based on the movement of the finger joint area, into the bending angle data.

[0047] More specifically, the haptic glove device (10) includes a rotating member (12) provided for each finger joint area, and the rotating member (12) may include a rotor that rotates in response to the movement of the finger joint and a variable resistor that receives the rotational movement of the rotor.

[0048] Accordingly, the resistance value of the variable resistor changes in response to the rotational movement of the finger joint, and the bending detection unit (120) can calculate bending angle data of the finger joint area based on the resistance value of the variable resistor.

[0049]

[0050] Meanwhile, as illustrated in FIG. 4, the haptic control unit (140) may include a first haptic control unit (141) that generates a first haptic control signal corresponding to virtual content set through the user terminal (20) and controls the haptic glove device (10) based on the first haptic control signal, and a second haptic control unit (142) that collects the contact pressure data and the bending angle data of the haptic glove device (10) to which the first haptic control signal is applied, generates a second haptic control signal that corrects the contact pressure data and the bending angle data corresponding to the virtual content, and controls the haptic glove device (10) based on the second haptic control signal.

[0051] That is, when the object is grasped through the haptic glove device (10), the first haptic control unit (141) generates a first haptic control signal that implements the appearance and texture of the object in response to the virtual content, and controls at least one driving member (13) provided in the haptic glove device (10) based on the first haptic control signal.

[0052] At this time, the driving member (13) can restrict the movement of the finger joint of a user wearing the haptic glove device (10) by fixing or releasing a spring plate provided in the haptic glove device (10) and providing a path for the finger to pass through.

[0053] For example, if the virtual object is a solid object, a first haptic control signal may be generated in response to the appearance information and texture information of the virtual object. At this time, the first haptic control signal controls the fixing member (13) so that the spring plate is fixed at a preset pressure, thereby inducing the spring plate to move only when an external force greater than the pressure set by the haptic control signal is applied, thereby providing a realistic sensation of holding a solid object to a user wearing the haptic glove device (10).

[0054] Additionally, if the virtual object is a soft object, the first pressure control signal can control the fixed member (13) so that the spring plate moves even with very small pressure.

[0055]

[0056] Meanwhile, after the haptic glove device (10) is applied based on the first tactile control signal generated in response to the appearance information and texture information of the virtual object, the second tactile control unit (142) can collect the contact pressure data and the bending angle data.

[0057] Accordingly, the second tactile control unit (142) can generate the second tactile control signal by correcting the first tactile control signal by comparing the first tactile control signal initially generated by the first tactile control unit (141) in response to the virtual object with the data (the contact pressure data and the bending angle data) actually measured through the haptic glove device (10).

[0058] For example, the first pressure control signal may control the driving member (13) to move the finger joint area of ​​the haptic glove device (10) only when a pressure of '4' or more is applied between the object and the haptic glove device (10).

[0059] At this time, when the drive member (13) of the haptic glove device (10) is controlled by the first pressure control signal, and the contact pressure data collected by the second pressure control unit (142) is measured as '5', but the finger joint area does not move naturally, the second pressure control unit (142) can generate a second pressure control signal that controls the drive member (13) to move the finger joint area of ​​the haptic glove device (10) when a pressure of '3' or more is applied between the object and the haptic glove device (10).

[0060] That is, in response to the virtual object, the first haptic control signal is provided as an initial setting value, but the first haptic control signal can be corrected by the second haptic control unit (142) to provide a more realistic sensation depending on the wearer wearing the haptic glove device (10), and the second haptic control signal can be generated.

[0061] In addition, even after the second inverse control signal is generated, correction for the second inverse control signal may be repeatedly performed by the second inverse control unit (142) based on the virtual content.

[0062]

[0063] Meanwhile, the haptic glove device-using tactile control system (100) may further include a user learning unit (not shown) that extracts user information from the user terminal and learns the usage history of the haptic glove device (10) for each user.

[0064] Accordingly, the second haptic control unit (142) can correct the second haptic control signal more quickly and precisely based on the result of learning the usage history of the haptic glove device (10) for each user.

[0065]

[0066] Meanwhile, the haptic glove device-using control system (100) may further include a fault diagnosis unit (not shown) that monitors the resistance value measured by at least one of the contact pressure detection unit (110) and the bending detection unit (120) and diagnoses whether a fault has occurred in the contact pressure detection unit and the bending detection unit.

[0067] The above fault diagnosis unit can receive and analyze data collected through the contact pressure detection unit (110) and the bend detection unit (120) to determine whether noise is generated due to foreign matter being attached to the surface of at least one of the elastic member (11) and the rotation member (12) provided in the haptic glove device (20).

[0068] That is, when a foreign substance is attached to the elastic member (11) or the rotating member (12), the resistance value measured in response to the pressing pressure and the rotation angle will show an irregular or uneven pattern, and considering this characteristic, it is possible to determine whether a foreign substance is attached to the surface of the elastic member (11) or the rotating member (12), and further, to diagnose a failure of the haptic glove device (10).

[0069] To this end, the fault diagnosis unit calculates the average and standard deviation of a plurality of resistance values ​​measured for a preset period of time, and if there are a preset number (e.g., 5) or more resistance values ​​(hereinafter referred to as “noise data”) that deviate from the average by a standard deviation among the plurality of resistance values, it can be estimated that a foreign substance is attached to the surface of the elastic member (12) or the rotating member (12).

[0070] At the same time, the measured time interval of each of the above noise data is calculated, and if the time interval is not constant, it can be determined that a foreign substance is attached to the surface of the elastic member (12) or the rotating member (12).

[0071] Here, whether the time interval is not constant is determined by the maximum difference value (M) between the time intervals between noise data calculated by [Mathematical Formula 1] below. d ) is the mean time interval between noise data calculated according to [Mathematical Formula 2] (T av ) exceeds a constant multiple (e.g. 2), the time interval can be judged to be irregular.

[0072]

[0073] [Mathematical Formula 1]

[0074]

[0075] (Here, M d is the maximum difference in time interval between noise data, N max is the maximum value of the noise data time interval, N min ) represents the minimum value of the noise data time interval, respectively.

[0076]

[0077] [Equation 2]

[0078]

[0079] (Here, T av is the mean time interval between noise data, T (i-1) to i is the time interval between the (i-1)th noise data and the (i)th noise data, T 0 to 1 is the time interval between the time of the first resistance measurement and the time when the first noise data appears, and n represents the number of noise data.

[0080]

[0081] For example, from T0 to T as shown in below 20If the data are 9, 10, 11, 12, 14, 18, 9, 9, 6, 13, 2, 8, 9, 9, 10, 17, 9, 10, 3, 12, the average is 10 and the standard deviation is 3.42, so the noise data is 14, 18, 6, 2, 17, 3, which is 6. Since the number of noises is more than the preset number of 5, it can be assumed that foreign substances are attached to the surface of the elastic member (12) or the rotating member (12).

[0082]

[0083] Table 1

[0084]

[0085] After that, if the noise is calculated according to [Mathematical Formula 1] and [Mathematical Formula 2], the maximum difference in time interval between noise data (M) is calculated as shown in below. d ) becomes 8-1=7, and the average value of the time interval between noise data (T av ) becomes 3.5.

[0086]

[0087] Table 2

[0088]

[0089] Therefore, the maximum difference in time interval between noise data (M d ) 7 is the average value of the time interval between noise data (T av ) is more than twice that of 3.5, so in this case, it can be confirmed that a foreign substance is attached to the surface of the elastic member (12) or the rotating member (12).

[0090]

[0091] As described above, in one embodiment of the present invention, an error occurring due to foreign matter adhering to the surface of the elastic member (12) or the rotating member (12) can be monitored and diagnosed through a fault diagnosis unit, thereby providing an effect of more accurately determining whether the haptic glove device (10) is in a normal state.

[0092]

[0093] According to the present invention as described above, a two-way haptic control system and method using a haptic glove device capable of two-way haptic control that mutually transmits necessary information between a virtual object and a real body can be provided by first generating a haptic control signal corresponding to virtual content set through a user terminal and secondarily generating a corrected haptic control signal corresponding to press pressure information and finger joint bending information collected through a haptic glove device.

[0094]

[0095] In addition, a method for controlling a two-way force control system using a haptic glove device according to an embodiment of the present invention may be recorded on a computer-readable medium including program commands for performing various computer-implemented operations. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands may be specially designed and configured for the present invention or may be known and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0096]

[0097] Although the embodiments of the present invention have been described with limited examples and drawings, the embodiments of the present invention are not limited to the embodiments described above, and various modifications and variations are possible based on this description by those skilled in the art to which the present invention pertains. Therefore, the embodiments of the present invention should be understood solely by the scope of the claims set forth below, and all equivalent or equivalent modifications thereof are deemed to fall within the scope of the present invention.

[0098]

[0099] 10: Haptic glove device 11: Elastic member

[0100] 12: Rotating member

[0101] 13: Drive member

[0102] 20: User terminal

[0103] 110: Contact pressure detection unit 111: Object recognition unit

[0104] 112: Pressure sensing unit

[0105] 120: Bending detection unit

[0106] 130: Data Communication Department

[0107] 140: Reverse sensor control unit 141: First reverse sensor control unit

[0108] 142: Second Inverse Control Unit

Claims

1. A contact pressure detection unit that collects contact pressure data input from at least one elastic member provided on a predetermined haptic glove device; A bending detection unit that measures movement of a finger joint area through at least one rotating member provided in the haptic glove device and generates rotating member bending angle data; A data communication unit that transmits the above contact pressure data and the above bending angle data to a designated user terminal; and A haptic control unit that controls the haptic glove device based on a haptic control signal received from the user terminal; A two-way feedback control system using a haptic glove device including a .

2. In paragraph 1, The above contact pressure detection unit is, An object recognition unit that determines whether an object coming into contact with the haptic glove device is a designated object; and In the above object recognition unit, when an object coming into contact with the haptic glove device is recognized as the object, a pressure sensing unit converts a resistance value that changes based on the pressing pressure of the elastic member into the contact pressure data; The above bend detection unit, The bending angle data is calculated based on the resistance value of the variable resistor provided in the above rotating member, A bidirectional feedback control system using a haptic glove device characterized in that the resistance value of the variable resistor, which changes based on the movement of the finger joint area, is converted into the bending angle data.

3. In paragraph 2, The above object recognition unit, Recognize the communication tag provided in the above object through the tag recognition member provided in the haptic glove device, A two-way tactile control system using a haptic glove device, characterized in that the tag information of the communication tag recognized through the tag recognition member is extracted to determine whether the tag is an object included in the virtual content set through the user terminal.

4. In paragraph 1, The above-mentioned reverse sensing control unit, A first haptic control unit that generates a first haptic control signal corresponding to virtual content set through the user terminal and controls the haptic glove device based on the first haptic control signal; and A two-way force control system using a haptic glove device, characterized in that it includes a second force control unit that collects the contact pressure data and the bending angle data of the haptic glove device to which the first force control signal is applied, generates a second force control signal that corrects the contact pressure data and the bending angle data in response to the virtual content, and controls the haptic glove device based on the second force control signal.

5. In paragraph 1, A haptic glove device-based pressure control system, characterized in that it further includes a fault diagnosis unit that monitors a resistance value measured by at least one of the contact pressure detection unit and the bend detection unit, and diagnoses whether a fault has occurred in the contact pressure detection unit and the bend detection unit.

Images