Fixation state determination system, glove, information processing device, fixation state determination method and fixation state determination program

Abstract

A fixation state determination system (500) comprises a glove (200) worn by an operator (10) to grasp a target object (300). A photographic device (201) is positioned on the glove (200) corresponding to a segment between a pair of fingers grasping the target object (300) and photographs a target image (25). A position and orientation determination unit (110) determines the position and orientation of the target object (300). A range calculation unit (120) calculates a fingertip movement range, which is the range of movement of a fingertip of the operator (100). A contact position calculation unit (130) calculates a contact position of the fingertip on the target object (300) based on the position and orientation of the target object (300) and the fingertip movement range.A determination unit (140) determines a fixation state of the target object (300) on the basis of a displacement of the target object (300) and excitation force information (26) via an excitation force at the contact position of the fingertip for the target object (300).

Description

Technical area

[0001] The present disclosure relates to a fixation state determination system, a glove, an information processing device, a fixation state determination method, and a fixation state determination program. State of the art

[0002] There is a glove that an operator wears on-site to remotely check the fixing status of a device or piece of equipment.

[0003] With an existing glove, the change in a force applied to a contact surface and the vibration of a fingertip can be measured. However, information such as the position and orientation of a target object relative to the glove and the position of the fingertip on the target object cannot be obtained. Therefore, it is not possible to obtain information such as the direction or orientation of the force acting on the target object and the exact displacement or displacement of the target object.

[0004] Patent Literature 1 discloses a robot hand finger in which a CCD camera is mounted on the inner side of a part of the fingertip covered with an elastic body to detect contact with a target object, thereby measuring the deformation of the elastic body. Reference listPatent literature

[0005] Patent Literature 1: JP 2010-221359 A Summary of the inventionTechnical problem

[0006] To check a fixing condition of a device, such as rattling of the device, an on-site operator wears a glove and performs an operation such as grasping a target object with the glove and applying vibration to it.

[0007] By accurately detecting the position of a fingertip on the target object, the fixation state at that time can be determined with greater precision.

[0008] Even if the technique of Patent Literature 1 is applied to the glove, the contact position of a fingertip on the target object cannot be accurately obtained, making it impossible to obtain information such as the direction of a force acting on the target object and the accurate displacement of the target object. This raises the problem that the fixation state of the target object cannot be determined with high precision.

[0009] The object of the present disclosure is to determine a fixation state of a target object with higher precision by accurately obtaining the contact position of a fingertip on the target object. Solution to the problem

[0010] A fixation state determination system according to the present disclosure, which includes a glove worn by an operator to grasp a target object and which determines a fixation state of the target object, comprises: a photographing device placed at a position of the glove corresponding to a portion between a pair of fingers gripping the target object, for photographing the target object gripped by the pair of fingers as a target image; a position and attitude determining unit for determining a position and attitude of the target object using the target image; a range calculation unit for analyzing a movement of a fingertip of the operator using the target image, and calculating a fingertip movement range that is a movement range of a position of a fingertip of each finger of the operator; a contact position calculation unit for calculating a contact position of the fingertip on the target object based on the position and attitude of the target object and the fingertip movement range; and a determination unit for determining the fixation state of the target object based on a displacement of the target object and excitation force information, which is information about an excitation force at the contact position of the fingertip for the target object. Advantageous effects of the invention

[0011] An object of the present disclosure is to determine a fixation state of a target object with higher precision by accurately obtaining the contact position of a fingertip on the target object. Brief description of the drawings Fig. 1 is a diagram illustrating an overall configuration example of a fixation state determination system according to Embodiment 1. Fig. 2 is a diagram illustrating a configuration example of an information processing device according to Embodiment 1. Fig. 3 is a diagram illustrating a configuration example of a glove according to Embodiment 1. Fig. 4 is a diagram illustrating a situation in which the glove according to Embodiment 1 grasps a target object. Fig. 5 is a flowchart illustrating an operation example of the fixation state determination system according to Embodiment 1. Fig. 6 is a diagram illustrating an example of a target image according to Embodiment 1. Fig. 7 is a diagram for illustrating a concept of an angle of a proximal phalanx of each finger in an area calculation process according to Embodiment 1. Fig. 8 is a diagram for illustrating a concept of a fingertip movement area in the area calculation process according to Embodiment 1. Fig. 9 is a diagram for illustrating a concept of a contact position in a contact position calculation process according to Embodiment 1. Fig. 10 is a diagram illustrating a configuration example of a glove according to Modification 1 of Embodiment 1. Fig. 11 is a diagram illustrating a configuration example of a glove according to Modification 2 of Embodiment 1. Fig. 12 is a diagram illustrating a configuration example of a glove according to Modification 3 of Embodiment 1. Fig. 13 is a diagram illustrating a configuration example of a glove according to Modification 4 of Embodiment 1. Fig. 14 is a diagram illustrating an example of an information processing device according to Modification 5 of Embodiment 1. Description of embodiments

[0012] The present embodiment will be described below with reference to the drawings. In the drawings, the same or equivalent parts are designated by the same reference numerals. In the description of the embodiments, an explanation of the same or equivalent part is omitted or simplified where appropriate. Arrows in the drawings mainly indicate data flows or process flows. Furthermore, in the following drawings, the dimensional relationships between the components may appear different from the actual ones. In the description of the embodiment, a direction or orientation or positions such as "top", "bottom", "left", "right", "front", and "back" may be indicated. These representations are for descriptive purposes only and are not intended to limit the positioning, direction, or orientation of devices / equipment, tools, components, etc. Embodiment 1.*** Description of configurations ***

[0013] Fig. 1 is a diagram illustrating a configuration example of a fixing state determination system 500 according to the present embodiment.

[0014] The fixation state determination system 500 is a system in which an operator 10 wears a glove, grasps a target object 300, and applies an excitation force to the target object 300, thereby determining a fixation state of the target object 300.

[0015] The fixation state determination system 500 is equipped with a glove 200 worn by the operator 10 to grasp the target object 300, and an information processing device 100 that communicates with the glove 200. Communication between the glove 200 and the information processing device 100 can be either prioritized or wireless.

[0016] Detailed configurations of the information processing device 100 and the glove 200 are described below.

[0017] Fig. 2 is a diagram showing a configuration example of the information processing device 100 according to the present embodiment.

[0018] The information processing device 100 is a computer. The information processing device 100 is equipped with a processor 910 as well as other hardware devices, such as a main memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware devices via wired communication or wireless communication and controls the other hardware devices.

[0019] The information processing device 100 is equipped with a position and attitude determination unit 110, a range calculation unit 120, a contact position calculation unit 130, a determination unit 140, and a storage unit 160 as functional elements. A target object database 161 is stored in the storage unit 160.

[0020] The functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 are implemented by software. The storage unit 160 is provided in the main memory 921. The storage unit 160 may be provided in the auxiliary storage device 922, or may be provided in the main memory 921 and the auxiliary storage device 922 by distribution.

[0021] The processor 910 is a device running a fixation state determination program. The fixation state determination program is a program that implements the functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140. Note that the fixation state determination program includes a program that implements a function of the glove, which will be described later.

[0022] The processor 910 is an integrated circuit (IC) that performs computational processing. Specific examples of the processor 910 include a CPU, a DSP, and a GPU. Note that IC stands for integrated circuit; DSP stands for digital signal processor; and GPU stands for graphics processing unit.

[0023] The RAM 921 is a storage device in which data is temporarily stored. A specific example of the RAM 921 is an SRAM or a DRAM. Note that SRAM stands for Static Random-Access Memory, and DRAM stands for Dynamic Random-Access Memory.

[0024] The auxiliary storage device 922 is a storage device in which data is stored. A concrete example of the auxiliary storage device 922 is an HDD. The auxiliary storage device 922 may be a portable storage medium, such as an SD (registered trademark), a memory card, a CF, a NAND Flash, a flexible disk, an optical disk, a compact disk, a Blu-ray disk (registered trademark), and a DVD. Note that HDD stands for Hard Disk Drive, SD (registered trademark) for Secure Digital, CF for CompactFlash (registered trademark), and DVD for Digital Versatile Disc.

[0025] The input interface 930 is a connector for connecting to an input device, such as a mouse, a keyboard, and a touch panel. Specifically, the input interface 930 is a USB connector. The input interface 930 may be a connector for connecting to a LAN. Note that USB stands for Universal Serial Bus, and LAN stands for Local Area Network.

[0026] The output interface 940 is a connector to which a cable of an output device, such as a display, is to be connected. Specifically, the output interface 940 is a USB port or an HDMI (registered trademark) port. Specifically, the display is an LCD. The output interface 940 is also referred to as a display unit interface. Note that HDMI (registered trademark) stands for High Definition Multimedia Interface, and LCD stands for Liquid Crystal Display.

[0027] The communication device 950 includes a receiver and a transmitter. The communication device 950 is connected to a communication network, such as a LAN, the Internet, Wi-Fi (registered trademark), or a telephone line. Specifically, the communication device 950 is a communication chip or NIC. Note that NIC stands for Network Interface Card.

[0028] The fixed state determination program runs in the information processing device 100. The fixed state determination program is read by the processor 910 and executed by the processor 910. Not only the fixed state determination program, but also an OS is stored in the main memory 921. Note that OS stands for Operating System. The processor 910 executes the fixed state determination program while the OS is running. The fixed state determination program and the OS may be stored in the auxiliary storage device 922. The fixed state determination program and the OS stored in the auxiliary storage device 922 are loaded into the main memory 921 and executed by the processor 910. The fixed state determination program may be partially or completely integrated into the OS.

[0029] Information processing device 100 may be equipped with a plurality of processors that replace processor 910. The plurality of processors jointly execute the fixation state determination program. Each processor is a device that executes the fixation state determination program in the same way as processor 910.

[0030] Data, information, signal values, and variable values ​​used, processed, or output by the fixation state determination program are stored in the working memory 921, the auxiliary storage device 922, or a register or cache memory in the processor 910.

[0031] The term "unit" in each of the terms: position and attitude determination unit 110; area calculation unit 120; contact position calculation unit 130; and determination unit 140 may be replaced with "circuit," "stage," "method," "process," or "circuit." The fixation state determination program causes the computer to perform a position and attitude determination process, an area calculation process, a contact position calculation process, a determination process, and an image adjustment process. The term "process" in each of the terms: position and attitude determination process; area calculation process; contact position calculation process; and determination process may be replaced with "program," "program product," "program-stored computer-readable medium," or "program-recorded computer-readable medium."The fixing state determination process is a process performed by the information processing device 100 executing the fixing state determination program.

[0032] The fixation state determination program may be provided as stored on a computer-readable data medium. Alternatively, the fixation state determination program may be provided as a program product.

[0033] Fig. 3 is a diagram showing a configuration example of the glove 200 according to the present embodiment.

[0034] Fig. 4 is a diagram illustrating a situation in which the glove 200 according to the present embodiment grasps the target object 300.

[0035] When the operator 10 is to grasp the target object 300 and exert an excitation force thereon, the glove 200 is pulled over a hand of the operator 10, which is to grasp the target object 300.

[0036] Fig. 3 shows the glove 200 worn on the right hand of the operator 10. The left part of Fig. 3 shows the glove 200 as seen from the palm. The right part of Fig. 3 shows the glove 200 seen from the back of the hand.

[0037] Fig. 4 shows how the operator 10 wears the glove 200 to grasp the target object 300.

[0038] The glove 200 is equipped with a photography device 201, contact force sensors 202, a sensor data acquisition module 203 and markers 204.

[0039] The photographing device 201 is placed at a position of the glove 200 corresponding to a portion between a pair of fingers grasping the target object 300, and photographs the target object 300 grasped by a pair of fingers as a target image 25.

[0040] The photographing device 201 is, for example, a device formed by placing a module with an optical image sensor on a substrate. The photographing device 201 is, for example, a CCD camera.

[0041] People usually grasp an object with their thumb and a finger other than the thumb. Fig. 3, the operator 10 grasps the object with their fingers, including the thumb and index finger. Thus, the photographing device 201 is placed at a position on the glove 200 that corresponds to a palm-side portion of the thumb and index finger. Any other position is sufficient, as long as it is located at the location where the grasped target object 300 is to be photographed. The photographing device 201 can be placed at the base of a finger. Alternatively, the photographing device 201 can also be placed near a center of the palm.

[0042] The marker 204 is printed at a position on the glove 200 corresponding to the inside of a proximal phalanx of each finger of the operator 10. Specifically, the marker 204 is printed at a position on the glove 200 corresponding to the inside of a proximal phalanx of each finger of the operator 10.

[0043] The photographing device 201 photographs the target image 25 including the position of the marker when the glove 200 grasps the target object. That is, the target image 25 includes the marker 204 printed on each finger of the pair of fingers.

[0044] Each contact force sensor 202 is placed at a position of the glove 200 corresponding to the inside of the fingertip of the operator 10. The contact force sensor 202 acquires information about an excitation force of the target object 300 as excitation force information 26. The excitation force information 26 expresses, for example, a magnitude of the excitation force.

[0045] The sensor data acquisition module 203 acquires various types of sensor data and transmits the data to the information processing device 100. Specifically, the sensor data acquisition module 203 acquires the target image 25 from the photographing device 201 and the excitation force information 26 from the contact force sensor 202. Then, the sensor data acquisition module 203 transmits the target image 25 and the excitation force information 26 to the information processing device 100 via wired communication or wireless communication.

[0046] The sensor data acquisition module 203 is a computer that acquires sensor data such as the target image 25 and the excitation force information 26 and transmits the sensor data to the information processing device 100.

[0047] The information processing device 100 is equipped, for example, with hardware devices such as an input interface for inputting the sensor data, a processor, and a communication device for communicating with the information processing device 100. The explanation of the individual hardware corresponds to the explanation of the information processing device 100. *** Description of functions ***

[0048] The operations of the fixation state determination system 500 according to the present embodiment are described below. An operation flow of the fixation state determination system 500 corresponds to the fixation state determination method. A program that implements operations of the fixation state determination system 500 corresponds to the fixation state determination program.

[0049] Fig. 5 is a flowchart illustrating an operation example of the fixation state determination system 500 according to the present embodiment. < Photography process >

[0050] In step S101, the photographing device 201 photographs the target object 300 grasped by a pair of fingers as the target image 25.

[0051] Specifically, the photographing device 201 performs photographing to include the target object 300 grasped by the pair of fingers and the markers 204 printed on each of the pair of fingers.

[0052] The sensor data acquisition module 203 transmits the target image 25 to the information processing device 100.

[0053] Fig. 6 is a diagram illustrating an example of the target image 25 by the photographing device 201 according to the present embodiment.

[0054] As in Fig. 6, depending on the shape of the target object 300, contact portions of the target object 300 with the fingertips may not be photographed. Although not shown, the target image 25 includes the markers 204 printed on each of the pair of fingers. < Position and attitude determination process >

[0055] In step S102, the position and attitude determination unit 110 determines the position and attitude of the target object 300 using the target image 25. The position and attitude determination unit 110 compares the target image 25 with image candidates 61 accumulated in the target object database 161, thus determining the position and attitude of the target object 300.

[0056] In detail this is as follows.

[0057] Individual images of a plurality of combinations of position and location of the target object 300 are accumulated as image candidates 61 in the target object database 161.

[0058] The image candidates 61 are a variety of three-dimensional data obtained by photographing the target object 300 in combinations of different angles and different positions.

[0059] The position and attitude determination unit 110 compares the target image 25 with image candidates 61, and selects an image candidate 61 having the highest similarity, thereby determining the position and attitude of the target object 300 in the target image 25. < Area calculation process >

[0060] In step 103, the range calculation unit 120 analyzes a fingertip movement of the operator 10 using the target image 25, and calculates a fingertip movement range R, which is a movement range of a fingertip position of each finger of the operator. Step S103 includes steps S31 and S32.

[0061] In detail this is as follows.

[0062] Fig. 7 is a diagram for illustrating a concept of a proximal phalanx of each finger in the area calculation process according to the present embodiment.

[0063] In step S31, the area calculation unit 120 calculates the angle of the proximal phalanx of each finger of the operator 10 using the position of the marker 204 on the proximal phalanx in the target image 25. The example in Fig. Figure 7 shows how an angle θ1 of the proximal phalanx of the thumb and an angle θ2 of the proximal phalanx of the index finger are calculated.

[0064] Fig. 8 is a diagram for illustrating a concept of a moving range R in the range calculation process according to the present embodiment.

[0065] In step S32, the range calculation unit 120 calculates the fingertip movement range R based on the angle of the proximal phalanx of each finger of the operator 10. The range calculation unit 120 calculates the fingertip movement range R of each finger based on the angle of the proximal phalanx of the finger using information about the joint structure of the finger in a normal state. < Contact position calculation process >

[0066] In step S104, the contact position calculation unit 130 calculates a contact position P of the fingertip on the target object 300 based on the position and posture of the target object 300 and the fingertip movement range R.

[0067] Fig. 9 is a diagram for illustrating a concept of the contact position P in the contact position calculation process according to the present embodiment.

[0068] The contact position calculation unit 130 calculates a surface position of the target object 300 from the position and attitude of the target object 300. Then, the contact position calculation unit 130 calculates the contact position P between the target object 300 and the fingertip from an intersection between the surface position of the target object 300 and the fingertip movement range R. < Determination process >

[0069] In step S105, the determination unit 140 determines the fixation state of the target object 300 based on the displacement of the target object 300 and the excitation force information 26, which is information about the excitation force at the fingertip contact position P for the target object 300. Step S105 includes steps S51 and S52.

[0070] In detail this is as follows.

[0071] In step S51, the operator 10 puts on the glove 200, grasps the target object 300, and applies the excitation force to the target object 300. Then, the sensor data acquisition module 203 transmits a magnitude of the excitation force acquired by the contact force sensors to the information processing device 100 as the excitation force information 26. The excitation force information 26 is acquired via the contact force sensors 202 and transmitted from the glove 200 to the information processing device 100 via the sensor data acquisition module 203.

[0072] In step S51, the determination unit 140 determines the fixation state of the target object 300 based on the contact position P, the excitation force information 26, and the displacement of the target object 300.

[0073] The displacement of the target object 300 is acquired by the determination unit 140 from the target image 25.

[0074] The determination unit 140 calculates the magnitude of the excitation force and the direction of the excitation force at the contact position P, acquires the displacement which is a response of the target object 300, and integrates these calculation results, thus determining the fixation state or fixation situation of the target object 300.

[0075] As described above, the fixation state can be evaluated as an indicator using a relationship between the displacement of the target object 300 and the excitation force. *** Other configurations ***< Modification 1 >

[0076] Fig. 10 is a diagram illustrating an example of a configuration of a glove 200 according to Modification 1 of the present embodiment.

[0077] The glove 200 may be equipped with a motion sensor on the outside of a fingertip of each finger of an operator 10 to detect the movement of the fingertip.

[0078] A range calculation unit 120 calculates a fingertip movement range R based on an angle of the proximal phalanx of each finger of the operator 10 and the movement of the fingertip detected by a motion sensor 205.

[0079] The motion sensor 205 is, for example, an acceleration sensor that can measure acceleration. The range calculation unit 120 determines a fingertip position based on the amount of displacement from the initial position. The number of motion sensors to be used can be changed, taking into account variations in an assumed gripping method.

[0080] With the glove 200 according to Modification 1, the position detection accuracy of the fingertip is improved, and an excitation position can be determined accurately and stably. < Modification 2 >

[0081] Fig. 11 is a diagram illustrating an example of a configuration of a glove 200 according to Modification 2 of the present embodiment.

[0082] The glove 200 may be provided with a fixing tool 206 attached to each of a pair of fingers of the glove 200 to fix a photographing device 201. The fixing tool 206 is provided, for example, with a ring 261 attached to each of the pair of fingers of the glove 200, and with a rigid body 262 connecting the ring 261 to the photographing device 201.

[0083] With the glove 200 according to Modification 2, since the ring 261 fixed to the finger and the photographing device 201 are connected to each other with the rigid body 262, the position detection accuracy of the finger and the photographing device 201 improves, and the detection accuracy of the fingertip position also improves. < Modification 3 >

[0084] Fig. 12 is a diagram illustrating an example of a configuration of a glove 200 according to Modification 3 according to the present embodiment.

[0085] Each marker 204 may have a protruding shape.

[0086] As markers corresponding to the proximal phalanges, three-dimensional objects of various shapes can be formed between the fingers or fixed on a surface of the glove. Using a protrusion with a distinctive protruding shape as a position detection marker, which is attached to the palm side of the proximal phalange, expands the angular range of the proximal phalange for which a marker can be detected by a camera, allowing the fingertip position to be detected more reliably. < Modification 4 >

[0087] Fig. 13 is a diagram illustrating an example of a configuration of a glove 200 according to Modification 4 of the present embodiment.

[0088] The glove can be equipped with a plurality of photography devices 201. That is, a plurality of cameras can be mounted on the glove. If two or more camera modules are mounted on the glove as optical image sensors, the angular range in which the cameras can detect the markers is expanded, allowing the position of the fingertip to be determined more reliably. < Modification 5 >

[0089] In the present embodiment, the functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 are implemented by software. According to a modification, the functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 may be implemented by hardware.

[0090] Specifically, a fixation state determination system 500 is equipped with an electronic circuit 909 instead of a processor 910.

[0091] Fig. 14 is a diagram illustrating an example of a configuration of an information processing device 100 according to Modification 5 of the present embodiment.

[0092] The electronic circuit 909 is a dedicated electronic circuit that implements the functions of the position and attitude detection unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, a logic IC, a GA, an ASIC, or an FPGA. Note that GA stands for Gate Array; ASIC stands for Application-Specific Integrated Circuit; and FPGA stands for Field-Programmable Gate Array.

[0093] The functions of the position and attitude detecting unit 110, the area calculating unit 120, the contact position calculating unit 130, and the determining unit 140 may be implemented by one electronic circuit, or may be implemented by a plurality of electronic circuits through distribution.

[0094] According to a further modification, some of the functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 may be implemented by an electronic circuit, and the remaining functions may be implemented by software. Furthermore, some or all of the functions of the position and attitude determination unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 may be implemented by firmware.

[0095] The processor and electronic circuit are also referred to as a processing circuit. That is, the functions of the position and attitude detection unit 110, the area calculation unit 120, the contact position calculation unit 130, and the determination unit 140 are implemented by one processing circuit.

[0096] The description for modification 5 also applies to the computer mounted in the glove 200. *** Description of the effect of the embodiment ***

[0097] In the fixation state determination system according to the present embodiment, the on-site operator puts on a glove equipped with a contact force sensor, grasps a target object, and applies an excitation force to the target object. The haptic information is presented to an inspection person at a remote location based on the sensor information thus acquired.

[0098] For example, when grasping an object with a robot hand, if a hand-eye system is used that detects the position of the object with a camera attached to the robot hand unit, the fingertips enter a blind spot formed by the target object during grasping, making it impossible to determine the fingertip positions. Therefore, the application position and direction of an excitation force applied by the fingers to the target object cannot be obtained, which may accordingly affect the determination accuracy.

[0099] On the other hand, with a fixation state determination system according to the present embodiment, the contact positions of the fingertips on the target object can be accurately obtained. Thus, information such as the direction of a force acting on the target object and an exact displacement of the target object can be accurately obtained. As a result, a fixation state of the target object can be determined with high accuracy.

[0100] The fixation state determination system according to the present embodiment uses a glove in which markers different from finger to finger are printed on portions of the palm side corresponding to the bones at the base of the fingers (proximal phalanges). A compact camera is installed in the glove in a space between the thumb and index finger. The following processes are then performed. (1) The position of the target object is calculated by the camera using the shape data of a test object. (2) Based on the marker displayed on the camera, an angle of the proximal phalanx of each finger is calculated. (3) A candidate area for a fingertip position is calculated based on the angle information of the proximal phalanx. (4) A contact position is calculated from a relationship between position and shape information of the target object and the candidate fingertip position. (4) A change in a three-dimensional excitation force is calculated from a change in the fingertip contact position and a change in the contact force information obtained during an excitation process, and the calculated change is integrated with the information on the displacement of the target object to calculate the fixed state of the target object.

[0101] As described above, with the fixation state determination system according to the present embodiment, the fingertip position detection accuracy is improved, so that the excitation position can be detected accurately and stably.

[0102] In Embodiment 1 described above, the individual units of the fixation state determination system are described as independent functional blocks. However, the configuration of the fixation state determination system is not necessarily limited to the configuration described in the above-described embodiments. The functional blocks of the fixation state determination system can have any configuration as long as they can implement the functions described in the above embodiment. The fixation state determination system may be constituted by a plurality of devices instead of a single device.

[0103] Multiple parts of Embodiment 1 may be implemented in combination. Alternatively, only a part of the embodiment may be implemented. The embodiment may be implemented in whole or in part in any combination.

[0104] That is, in Embodiment 1, the individual embodiments can be freely combined; any component of each embodiment can be changed; and any component of each embodiment can be omitted.

[0105] The above-described embodiment is essentially a preferred example and is not intended to limit the scope of the present disclosure, the scope of an applied product of the present disclosure, or a range of application of the present disclosure. Various modifications may be made to the above-described embodiment as needed. The processes explained by means of flowcharts or sequence diagrams may be changed as needed. List of reference symbols

[0106] 10: Operator; 25: Target image; 26: Excitation force information; 61: Image candidate; 100: Information processing device; 110: Position and attitude determination unit; 120: Area calculation unit; 130: Contact position calculation unit; 140: Determination unit; 160: Storage unit; 161: Target object database; 200: Glove; 201: Photographing device; 202: Contact force sensor; 203: Sensor data acquisition module; 204: Marker; 205: Motion sensor; 206: Fixation tool; 261: Ring; 262: Rigid body; 300: Target object; 500: Fixation state determination system; 909: Electronic circuit; 910: Processor; 921: Memory; 922: Auxiliary storage device; 930: Input interface; 940: Output interface; 950: Communication device. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2010-221359 A

[0005]

Claims

[1] A fixation state determination system comprising a glove worn by an operator to grasp a target object and determining a fixation state of the target object, the fixation state determination system comprising: a photographing device placed at a position of the glove corresponding to a portion between a pair of fingers gripping the target object, for photographing the target object gripped by the pair of fingers as a target image; a position and attitude determining unit for determining a position and attitude of the target object using the target image; a range calculation unit for analyzing a movement of a fingertip of the operator using the target image, and calculating a fingertip movement range that is a movement range of a position of a fingertip of each finger of the operator; a contact position calculation unit for calculating a contact position of the fingertip on the target object based on the position and attitude of the target object and the fingertip movement range; and a determination unit for determining the fixation state of the target object based on a displacement of the target object and excitation force information, which is information about an excitation force at the contact position of the fingertip for the target object. [2] Fixation state determination system according to claim 1, wherein the glove has a marker at a position corresponding to an inner side of a proximal phalanx of each finger of the operator, wherein the photographing device photographs the target image containing the position of the marker at the time when the glove grasps the target object, and wherein the range calculation unit calculates an angle of the proximal phalanx of each finger of the operator using the position of the marker in the target image, and calculates the fingertip movement range based on the angle of the proximal phalanx of each finger of the operator. [3] Fixation state determination system according to claim 2, wherein the glove comprises a motion sensor on an outer side of the fingertip of each finger of the operator to detect the movement of the fingertip, and wherein the range calculation unit calculates the fingertip movement range based on the angle of the proximal phalanx of each finger of the operator and the movement of the fingertip detected by the motion sensor. [4] Fixation state determination system according to one of claims 1 to 3, comprising: a target object database in which individual images of a large number of combinations of the position and location of the target object are accumulated as image candidates, wherein the position and attitude determination unit compares the target image with the image candidates accumulated in the target object database and thereby determines the position and attitude of the target object. [5] Fixation state determination system according to one of claims 1 to 4, wherein the glove comprises a contact force sensor at a position corresponding to an inner side of the fingertip of the operator, and wherein the determination unit acquires the excitation force at the contact position of the fingertip for the target object from the contact force sensor, and acquires a displacement of the target object from the target image. [6] A glove intended for a fixation state determination system comprising the glove worn by an operator to grasp a target object and an information processing device for communicating with the glove, the fixation state determination system determining a fixation state of the target object, the glove comprising: a photographing device placed at a position corresponding to a portion between a pair of fingers gripping the target object, for photographing the target object gripped by the pair of fingers as a target image; and a communication device for transmitting the target image to the information processing device, wherein the information processing device determines a position and posture of the target object using the target image; calculating a fingertip movement range, which is a movement range of a position of a fingertip of each finger of the operator, using the target image; and calculating a contact position of the fingertip on the target object based on the position and posture of the target object and the fingertip movement range. [7] Glove according to claim 6, comprising: a marker at a position corresponding to an inner side of a proximal phalanx of each finger of the operator, wherein the photographing device photographs the target image containing the position of the marker at the time when the glove grasps the target object, and wherein the image processing device calculates an angle of the proximal phalanx of each finger of the operator using the position of the marker in the target image, and calculates the fingertip movement range based on the angle of the proximal phalanx of each finger of the operator. [8] Glove according to claim 7, comprising: a motion sensor on an outer side of the fingertip of each finger of the operator to detect movement of the fingertip, wherein the information processing device calculates the fingertip movement range based on the angle of the proximal phalanx of each finger of the operator and the movement of the fingertip detected by the movement sensor. [9] Glove according to claim 7 or 8, comprising: a fixing tool attached to each of the pair of fingers of the glove to fix the photographing device. [10] A glove according to claim 9, wherein the fixing tool comprises a ring attached to each of the pair of fingers of the glove and a rigid body connecting the ring to the photographing device. [11] A glove according to any one of claims 7 to 10, wherein the marker has a protruding shape. [12] A glove according to any one of claims 6 to 11, wherein the photographing device comprises a plurality of photographing devices. [13] Glove according to one of claims 6 to 12, comprising: a contact force sensor at a position corresponding to an inner surface of the operator’s fingertip, wherein the information processing means acquires information about an excitation force at a contact position of the fingertip for the target object from the contact force sensor as excitation force information and acquires a displacement of the target object from the target image. [14] An information processing device provided for a fixation state determination system comprising a glove worn by an operator to grasp a target object and an information processing device for communicating with the glove, the fixation state determination system determining a fixation state of the target object, the information processing device comprising: a position and attitude detecting unit for detecting a position and attitude of the target object using a target image photographed by a photographing device placed at a position of the glove corresponding to a portion between a pair of fingers gripping the target object to photograph the target object gripped by the pair of fingers as the target image; a range calculation unit for analyzing a movement of a fingertip of the operator using the target image, and calculating a fingertip movement range that is a movement range of a position of a fingertip of each finger of the operator; a contact position calculation unit for calculating a contact position of the fingertip on the target object based on the position and attitude of the target object and the fingertip movement range; and a determination unit for determining the fixation state of the target object based on a displacement of the target object and an excitation force of excitation at the contact position of the fingertip for the target object. [15] A fixation state determination method applied by a fixation state determination system comprising a glove worn by an operator to grasp a target object and determining a fixation state of the target object, the fixation state determination method comprising: a photographing device placed at a position of the glove corresponding to a portion between a pair of fingers gripping the target object, for photographing the target object gripped by the pair of fingers as a target image; by a computer, determining a position and attitude of the target object using the target image; by the computer, analyzing a movement of a fingertip of the operator using the target image, and calculating a fingertip movement range which is a movement range of a position of a fingertip of each finger of the operator; by the computer, calculating a contact position of the fingertip on the target object based on the position and location of the target object and the fingertip movement range; and by the computer, determining the fixation state of the target object based on a displacement of the target object and an excitation force of excitation at the contact position of the fingertip for the target object. [16] A fixation state determination program applied by a fixation state determination system comprising a glove worn by an operator to grasp a target object and determining a fixation state of the target object, the fixation state determination program causing a computer to execute: a position and attitude detection process of detecting a position and attitude of the target object using a target image photographed by a photographing device placed at a position of the glove corresponding to a portion between a pair of fingers gripping the target object, to photograph the target object gripped by the pair of fingers as the target image; a range calculation process of analyzing a movement of a fingertip of the operator using the target image, and calculating a fingertip movement range that is a movement range of a position of a fingertip of each finger of the operator; a contact position calculation process of calculating a contact position of the fingertip on the target object based on the position and attitude of the target object and the fingertip movement range; and a determination process of determining the fixation state of the target object based on a displacement of the target object and an excitation force of excitation at the contact position of the fingertip for the target object.

Images