Camera device and camera system

Abstract

To provide a camera device and a camera system that can easily detect the direction of a camera attached to the finger.SOLUTION: A fingertip camera device 20 of the present embodiment comprises: a camera unit 21 that is attached to a finger; a myoelectric information acquisition unit 42 that acquires myoelectric information on a palm; a camera direction detection unit 46 that, based on the acquired myoelectric information, detects a relative direction of the camera unit 21 with respect to a predetermined reference direction; and an output unit 52 that adds direction information of the camera unit 21 obtained from the camera direction detection unit 46 to imaging information picked up by the camera unit 21 and outputs the resultant information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a user interface for controlling a small camera, and more particularly to a technique for controlling a camera mounted on a fingertip using biometric information of a palm.

Background Art

[0002] Conventionally, a camera-equipped terminal including a stereo camera unit having a pair of left and right lenses and imaging elements, a stereo viewfinder and viewer unit having a pair of left and right eyepieces and display elements, and a portable information terminal unit wirelessly connected to a public communication line is known (see, for example, Patent Document 1). In this type of camera-equipped terminal, it is possible to complete shooting and viewing of stereo images with a single unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in recent years, cameras have been mounted on fingers and images captured by these cameras have been displayed. In this case, for example, a technique of mounting cameras on two (a plurality of) fingers and generating a stereo (three-dimensional) image by processing images captured by each camera has been explored. However, in this type of technology, for example, when a gyro sensor or the like is provided in the camera, the camera becomes large, and it has been difficult to easily detect the direction (imaging direction) of the camera mounted on the finger.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a camera device and a camera system capable of easily detecting the direction of a camera mounted on a finger.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, a camera device according to the present invention includes a camera unit attached to a finger, a biological information acquisition unit that acquires biological information of a palm, and a camera direction detection unit that detects a relative direction of the camera unit with respect to a predetermined reference direction based on the acquired biological information, and an output unit that adds and outputs the direction information of the camera unit obtained from the camera direction detection unit to imaging information captured by the camera unit. Further, a camera system according to the present invention includes the above-described camera device, and a plurality of images captured by the camera unit Image information and a stereoscopic image generation processing unit that generates stereoscopic image information representing a three-dimensional structure based on direction information indicating a relative direction of the corresponding camera unit.

Advantages of the Invention

[0007] According to the present invention, since the relative direction of the camera unit with respect to a predetermined reference direction is detected based on the acquired biological information, the direction of the camera attached to the finger can be easily detected.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited by this embodiment. In addition, the constituent elements in the following embodiments include those that can be replaced by those skilled in the art and are easy, or those that are substantially the same.

[0010] [First Embodiment] FIG. 1 is a schematic diagram showing an example of a fingertip camera system including a fingertip camera device according to the first embodiment. FIG. 2 is a schematic diagram showing an example of a state in which a fingertip camera device is attached to a hand. In the present embodiment, a fingertip camera system (camera system) 10 processes an image captured by a camera unit 21 attached to the fingertip of a user's hand and displays it on a predetermined display unit 34.

[0011] As shown in FIG. 1, the fingertip camera system 10 includes a fingertip camera device (camera device) 20 and an image processing unit 30. These fingertip camera device 20 and image processing unit 30 are connected via, for example, a LAN (Local Area Network). The LAN relays the fingertip camera device 20 and the image processing unit 30, and for example, a wireless LAN such as WI-FI (registered trademark) or a wireless communication path such as LTE or 5G is used.

[0012] The fingertip camera device 20 includes a camera unit 21, an electromyographic sensor (biological information detection unit) 22, and a control unit 23. As shown in FIG. 2, the camera unit 21 includes a camera body 210 attached to the fingertip of the user's hand (for example, the fingertip of the index finger (the fingertip of the second finger)) and a lens 211 provided on the camera body 210. The camera body 210 is formed in a bottomed cylindrical shape like a finger sack, and the fingertip is inserted from the opening side and attached to the fingertip. Further, an imaging unit (not shown) for imaging the periphery is built in the camera body 210 via the lens 211. The image captured by the imaging unit may be, for example, a moving image or a still image. The captured imaging information is transmitted to the control unit 23 at a predetermined frequency. The lens 211 has a function of condensing light and is provided at the tip (bottomed part) of the camera body 210. That is, in the example of FIG. 2, the optical axis (imaging direction) of the camera unit 21 extends in the direction of the fingertip of the attached finger.

[0013] The myoelectric sensor 22 is formed in a ring shape and is attached to the base portion of the finger on which the camera unit 21 is mounted. The myoelectric sensor 22 has a plurality of surface electrodes arranged on the inner peripheral surface of the ring-shaped main body, and these surface electrodes detect myoelectric information (biological information) generated according to the movement of the muscles of the user's finger. Here, the myoelectric information is a bioelectric signal generated by the contraction of muscles (muscle fibers) when the finger is moved. Also, the inner peripheral surface indicates the side that comes into contact with the outer peripheral surface of the user's finger. The myoelectric sensor 22 detects, for example, the myoelectric information generated by the extension of the left and right tendons of the finger and the extension of muscles (such as the transverse head of the adductor pollicis) when the finger is bent or when the finger is moved horizontally in the palm direction so as to change the distance between adjacent fingers. Thereby, the myoelectric sensor 22 is configured to be able to detect the movement of a plurality of muscles and tendons by detecting the myoelectric information. The detected myoelectric information is transmitted to the control unit 23 at any time at a predetermined frequency. In this embodiment, the myoelectric sensor 22 is attached to the base portion of the finger on which the camera unit 21 is mounted. However, depending on the position of the muscle corresponding to the myoelectric information to be detected, the myoelectric sensor 22 may be separately attached at the base portion of the finger and the wrist, or may be attached together to the wrist. When the myoelectric sensor is attached to the wrist, it can be provided in the control unit 23.

[0014] The control unit 23 is connected to the camera unit 21 and the electromyographic sensor 22 respectively, and controls the operations of these camera unit 21 and electromyographic sensor 22. As shown in FIG. 2, the control unit 23 is, for example, wound around the user's wrist and worn on the wrist. In the present embodiment, the control unit 23 is wired-connected to the camera unit 21 and the electromyographic sensor 22, supplies power to the camera unit 21 and the electromyographic sensor 22, and acquires various information from the camera unit 21 and the electromyographic sensor 22. Note that, in the present embodiment, although a wired connection is used, it is not limited thereto, and a wireless connection may be used. In this case, it is preferable that the camera unit 21 and the electromyographic sensor 22 are each provided with a power source (battery) for driving itself. As shown in FIG. 1, the control unit 23 includes an imaging information acquisition unit 40, an electromyographic information acquisition unit (biological information acquisition unit) 42, a position sensor (arm direction detection unit) 44, a camera direction detection unit 46, a storage unit 48, a power supply unit 50, an output unit 52, an operation unit 54, and a control unit 56.

[0015] The imaging information acquisition unit 40 is an interface that acquires the imaging information transmitted from the camera unit 21. The electromyographic information acquisition unit 42 acquires the electromyographic information transmitted from the electromyographic sensor 22. The acquired imaging information and electromyographic information are stored in the storage unit 48 in association with the camera unit 21, for example.

[0016] The position sensor 44 is a sensor that detects the position and orientation of the control unit 23, and is configured to include, for example, a gyro sensor, a three-axis acceleration sensor, and a geomagnetic sensor. With this position sensor 44, the position of the user's wrist (arm) where the control unit 23 is worn and the direction in which the user's arm extends can be detected. A specific example of calculating the direction in which the arm extends will be described. For example, two or more position sensors 44 are installed in the control unit 23. The two or more position sensors 44 are installed in the control unit 23 such that when the user attaches the control unit 23, the two or more position sensors 44 are located on the line in the extending direction of the wrist. By connecting the positions of the respective position sensors 44, it becomes possible to detect the extending direction. Also, there may be one position sensor 44, and the direction in which the arm extends can be calculated from the change in the position detected by the position sensor 44.

[0017] The camera direction detection unit 46 detects the relative direction (imaging direction) of the camera unit 21 with respect to a predetermined reference direction based on the acquired myoelectric information. In this configuration, the reference direction is the direction in which the user's arm extends as detected by the position sensor 44, and the relative direction of the camera unit 21 with respect to the direction in which this arm extends is detected. Here, the bending direction of the fingers, that is, the direction of the camera unit 21 with respect to the palm, is calculated as, for example, a three-dimensional direction vector based on the myoelectric information. In this case, data associating the myoelectric information of the muscles at the base of the fingers (for example, the transverse head of the adductor pollicis, the oblique adductor pollicis, the abductor pollicis brevis, the flexor pollicis brevis, etc.) with the respective direction information (direction vectors) in which the fingers moved at that time is called from the storage unit 48, and by comparing the acquired myoelectric information with this data, the respective direction information (direction vectors) in which the fingers moved is calculated, and the direction of the camera unit 21 with respect to the palm can be calculated. Also, using the myoelectric information measured for the muscles at the base of the fingers (for example, the transverse head of the adductor pollicis, the oblique adductor pollicis, the abductor pollicis brevis, the flexor pollicis brevis, etc.) and the respective direction information (direction vectors) in which the fingers moved at that time, a learning model obtained by machine learning these information as a teacher data set is generated, and by inputting the detected myoelectric information into this learning model, the direction of the camera unit 21 with respect to the palm can be calculated. Also, regarding the direction in which the arm extends, it is calculated as a three-dimensional direction vector based on the detection by the position sensor. Therefore, by setting the direction in which the arm extends as the reference direction, the relative direction of the camera unit 21 with respect to this reference direction can be detected (calculated).

[0018] The storage unit 48 is configured to include, for example, a RAM and a flash memory, and stores the acquired imaging information and myoelectric information. Also, it stores the relative direction (imaging direction) of the camera unit 21 with respect to a predetermined reference direction. Further, the above-described learning model is stored in the storage unit 48. This learning model is generated, for example, by machine learning the myoelectric information generated when the fingers are moved and the respective direction information (direction vectors) in which the fingers moved at that time as a teacher data set.

[0019] The power supply unit 50 is a power supply for driving the fingertip camera device 20. This power supply unit 50 is, for example, a rechargeable battery, and supplies power to the camera unit 21 and the myoelectric sensor 22. The output unit 52 outputs information from the control unit 23 to the image processing unit 30. In the present embodiment, the output unit 52 is an interface that adds the relative direction information of the camera unit 21 detected by the camera direction detection unit 46 to the imaging information captured by the camera unit 21 and outputs the result. In this case, it is sufficient to output the direction information and the imaging information in association (linkage).

[0020] The operation unit 54 operates the fingertip camera device 20. For example, the operation unit 54 includes a switch formed on the surface of the control unit 23 mounted on the wrist, and by operating this switch, the start and stop operations of the imaging operation of the camera unit 21 are performed. The control unit 56 controls the operations of the respective components of the control unit 23.

[0021] Note that the myoelectric information acquisition unit 42, the camera direction detection unit 46, and the control unit 56 are each constituted by a CPU, a ROM, a RAM, and the like.

[0022] As shown in FIG. 1, the image processing unit 30 includes a communication unit 31, a storage unit 32, a stereoscopic image generation processing unit 33, a display unit 34, and a control unit 35. Specifically, the image processing unit 30 is an information processing device such as a computer device or a smartphone. Further, the stereoscopic image generation processing unit 33 and the control unit 35 of the image processing unit 30 are constituted by, for example, a CPU, a ROM, a RAM, and the like. Specific examples of the communication unit 31, the storage unit 32, and the display unit 34 will be described later.

[0023] The communication unit 31 is an interface that receives the relative direction information and imaging information of the camera unit 21 output from the control unit 23 of the fingertip camera device 20. The storage unit 32 stores the received various types of information and control programs. The storage unit 32 can use, for example, a semiconductor memory element such as a flash memory, but may also be a storage device such as an HDD.

[0024] The stereoscopic image generation processing unit 33 generates stereoscopic image information representing a three-dimensional structure from a plurality of captured images. The stereoscopic image generation processing unit 33 can generate stereoscopic image information using a so-called photogrammetry (photo surveying) technique. The stereoscopic image generation processing unit 33 extracts an imaging object as feature points from a plurality of imaging information captured by the fingertip camera device 20, and performs association of the feature points extracted from the plurality of imaging information based on the direction information of a plurality of cameras corresponding to the imaging information, thereby generating stereoscopic image information. The generated stereoscopic image information is stored in the storage unit 32. Also, the stereoscopic image may be, for example, a three-dimensional model composed of a three-dimensional point cloud in a predetermined coordinate system.

[0025] The display unit 34 is, for example, a monitor unit composed of a liquid crystal display (LCD) or the like, and displays the generated stereoscopic image information. In the present embodiment, the display unit 34 is provided integrally with the image processing unit 30, but the display unit 34 may be separate, for example, a glasses-type display such as a head-mounted display. The control unit 35 controls the operation of the entire image processing unit 30.

[0026] Next, the operation of the fingertip camera system will be described. FIG. 3 is a flowchart showing the operation procedure of the fingertip camera system. Prior to the operation, the above-described fingertip camera device 20 is attached to the user's hand. In the present embodiment, as shown in FIG. 2, the camera unit 21 is attached to the fingertip of the index finger, and the myoelectric sensor 22 is attached to the base of the index finger. Also, the control unit 23 is attached to the wrist. In this case, the basic state is that the index finger is straight extended in the direction of the arm.

[0027] First, the operation unit 54 of the control unit 23 is operated to capture an image with the camera unit 21 attached to the fingertip (step S1). In this case, the captured imaging information is sent from the camera unit 21 to the control unit 23, and the imaging information acquisition unit 40 acquires it. Next, the myoelectric sensor 22 detects myoelectric information accompanying the movement of the finger during imaging and sends it to the control unit 23, and the myoelectric information acquisition unit 42 acquires the myoelectric information (step S2). These imaging information and myoelectric information are stored in the storage unit 48 in association with each other.

[0028] Next, the camera direction detection unit 46 first calculates the bending direction of the finger, that is, the angle (direction) of the camera unit 21 with respect to the palm, based on the myoelectric information (step S3). In this case, a learning model is generated by machine learning using the myoelectric information measured for the muscles at the base of the finger (for example, the transverse head of the adductor pollicis, the oblique adductor pollicis, the abductor pollicis brevis, the flexor pollicis brevis, etc.) and the respective direction information (direction vector) in which the finger moved at that time as a teacher data set. The camera direction detection unit 46 calculates the angle (direction) of the camera unit 21 with respect to the palm by inputting the detected myoelectric information into this learning model. This calculated angle information of the camera unit 21 is, for example, a three-dimensional direction vector.

[0029] Next, the camera direction detection unit 46 detects the relative direction (imaging direction) of the camera unit 21 with respect to the reference direction of the arm orientation (step S4). The camera direction detection unit 46 acquires the direction DS in which the user's arm extends, which is detected by the position sensor 44. This direction DS in which the arm extends is calculated as a three-dimensional direction vector. Then, by using this direction in which the arm extends as the reference direction, the camera direction detection unit 46 detects (calculates) the relative direction D1 of the camera unit 21 with respect to this reference direction.

[0030] Next, the output unit 52 adds the relative direction information of the camera unit 21 obtained from the camera direction detection unit 46 to the imaging information captured by the camera unit 21 and outputs it to the image processing unit 30 (step S5). In this case, a method of multiplexing the direction information with the imaging information (for example, data in a video compression format such as MPEG) can be adopted.

[0031] Next, the stereoscopic image generation processing unit 33 of the image processing unit 30 generates stereoscopic image information representing a three-dimensional structure based on a plurality of captured images captured by the camera unit 21 and direction information indicating the relative directions of the camera units corresponding to the captured information (step S6). In this case, the stereoscopic image generation processing unit 33 can generate stereoscopic image information using a so-called photogrammetry (photo surveying) technique. That is, the stereoscopic image generation processing unit 33 extracts imaging objects as feature points from a plurality of imaging information captured by the fingertip camera device 20 at predetermined time intervals, and based on the direction information of a plurality of cameras corresponding to the imaging information, performs association of the feature points extracted from the plurality of imaging information to generate stereoscopic image information.

[0032] Next, the control unit 35 of the image processing unit 30 displays the generated stereoscopic image information on the display unit 34 (step S7) and ends the process.

[0033] In the present embodiment, since the camera unit 21 is attached to the fingertip, it is possible to easily capture an image simply by pointing at a distant view or a nearby object. For example, it is also possible to capture a magnified video (image) close to a small object like a magnifying glass. Furthermore, since the myoelectric sensor 22 can simultaneously acquire information regarding the direction of the finger to which the camera unit 21 is attached, the imaging direction of the camera unit 21 attached to the fingertip can be easily detected.

[0034] [Second Embodiment] FIG. 4 is a schematic diagram showing an example of a fingertip camera system including a fingertip camera device according to the second embodiment. FIG. 5 is a schematic diagram showing an example of a state in which the fingertip camera device is attached to a hand. In the first embodiment, the fingertip camera device 20 is configured to include one camera unit 21 and one myoelectric sensor 22, but in the second embodiment, the configuration is different in that a plurality of camera units 21 and myoelectric sensors 22 are provided. The same components as those in the above-described embodiment are denoted by the same reference numerals and the description thereof is omitted.

[0035] As shown in FIG. 4, the fingertip camera system 10A includes a fingertip camera device 20A and an image processing unit 30. The fingertip camera device 20A includes two camera units 21A and 21B, two electromyographic sensors (biological information detection units) 22A and 22B, and a control unit 23A. As shown in FIG. 5, one camera unit 21A is attached to the fingertip of the user's hand (for example, the fingertip of the index finger (the fingertip of the second finger)), and the other camera unit 21B is attached to the adjacent fingertip (for example, the fingertip of the middle finger (the fingertip of the third finger)). In this embodiment, the two camera units 21A and 21B cooperate as a stereo camera. Specifically, a stereo image can be captured from the images captured by the respective camera units 21A and 21B by using the parallax between the camera units 21A and 21B attached to adjacent fingertips. The configurations of the respective camera units 21A and 21B and the electromyographic sensors 22A and 22B are the same as those in the above embodiment.

[0036] The control unit 23A is different in configuration from the control unit 23 in the above embodiment in that it includes an angle calculation unit 45 between adjacent cameras. The angle calculation unit 45 between adjacent cameras calculates the angles of the respective adjacent camera units 21A and 21B (mainly the angles in the horizontal direction with respect to the palm, the angles when spreading or narrowing the fingers) based on the electromyographic information of each finger detected by the two electromyographic sensors 22A and 22B. As described above, the respective directions (angles) of the camera units 21A and 21B with respect to the palm can be calculated by inputting the electromyographic information detected by a predetermined learning model. Therefore, the angles of the adjacent camera units 21A and 21B can be calculated from the directions (angles) of these camera units 21A and 21B. According to this configuration, the parallax between the camera units 21A and 21B functioning as a stereo camera can be calculated, so that a stereo image can be captured with high accuracy.

[0037] Next, the operation of the fingertip camera system 10A will be described. FIG. 6 is a flowchart showing the procedure of the operation of the fingertip camera system. In the present embodiment, as shown in FIG. 5, the camera unit 21A is attached to the fingertip of the index finger, and the electromyographic sensor 22A is attached to the base portion of the index finger. Also, the camera unit 21B is attached to the fingertip of the middle finger, and the electromyographic sensor 22B is attached to the base portion of the middle finger. Further, the control unit 23A is attached to the wrist.

[0038] First, the operation unit 54 of the control unit 23A is operated to capture a stereo image with the two camera units 21A and 21B attached to the two fingertips (step S11). In this case, the captured imaging information is sent from the camera units 21A and 21B to the control unit 23A, and is acquired by the imaging information acquisition unit 40. Next, the electromyographic sensors 22A and 22B detect the electromyographic information associated with the movement of each finger during imaging and send it to the control unit 23A, and the electromyographic information acquisition unit 42 acquires the electromyographic information of each finger (step S12). These imaging information and electromyographic information are stored in the storage unit 48 in association with each other.

[0039] Next, the adjacent camera angle calculation unit 45 calculates the angles of the adjacent camera units 21A and 21B based on the electromyographic information of each finger detected by the two electromyographic sensors 22A and 22B (step S13). Next, the camera direction detection unit 46 detects the relative direction (imaging direction) of each camera unit 21A and 21B with respect to the reference direction of the arm direction (step S14). The camera direction detection unit 46 acquires the extending direction DS of the user's arm detected by the position sensor 44. This extending direction DS of the arm is calculated as a three-dimensional direction vector. Then, by using this extending direction of the arm as the reference direction, the camera direction detection unit 46 detects (calculates) the relative direction D1 of the camera unit 21A with respect to this reference direction and the relative direction D2 of the camera unit 21B.

[0040] Next, the output unit 52 adds the relative direction information of the camera units 21A and 21B obtained from the camera direction detection unit 46 to the imaging information of the stereo images captured by the camera units 21A and 21B, respectively, and outputs the result to the image processing unit 30 (step S15). In this case, a method of multiplexing the direction information into the imaging information (for example, data in a video compression format such as MPEG) can be adopted.

[0041] Next, the stereo image generation processing unit 33 of the image processing unit 30 generates stereo image information representing a three-dimensional structure based on the captured images of the plurality of stereo images captured by the camera units 21A and 21B, and the direction information indicating the relative directions of the respective camera units 21A and 21B corresponding to the imaging information (step S16). In this case, the stereo image generation processing unit 33 can generate stereo image information using a so-called photogrammetry technique. That is, the stereo image generation processing unit 33 extracts the imaging target as feature points from the imaging information of the plurality of stereo images captured by the fingertip camera device 20A at predetermined time intervals, and generates stereo image information by associating the feature points extracted from the plurality of imaging information based on the direction information of the plurality of cameras corresponding to the imaging information.

[0042] Next, the control unit 35 of the image processing unit 30 displays the generated stereo image information on the display unit 34 (step S17) and ends the process.

[0043] In the present embodiment, since the camera unit 21 is attached to the fingertip, it is possible to easily capture an image simply by pointing at a distant view or a nearby object. For example, it is also possible to capture an enlarged video (image) in proximity to a small object like a magnifying glass. Further, since the electromyographic sensor 22 can also simultaneously acquire information regarding the direction of the finger to which the camera unit 21 is attached, the imaging direction of the camera unit 21 attached to the fingertip can be easily detected. In addition, in the present embodiment, since two camera units 21A and 21B are provided, a stereo image (parallax image) can be acquired, and a stereo image using parallax can be generated. Also, the three-dimensional space information can be reconstructed using the imaging information from the viewpoints of the two camera units 21A and 21B.

[0044] [Third Embodiment] FIG. 7 is a schematic diagram showing an example of a fingertip camera system including a fingertip camera device according to the third embodiment. FIG. 8 is a schematic diagram showing an example of a state in which a fingertip camera device is attached to a hand. In the second embodiment, the fingertip camera device 20A is configured to include two camera units 21A and 21B and two myoelectric sensors 22A and 22B. However, in the third embodiment, the configuration is different in that it includes five camera units 21A to 21E and myoelectric sensors 22A to 22E. The same components as those in the above-described embodiments are denoted by the same reference numerals, and the description thereof is omitted.

[0045] As shown in FIG. 7, the fingertip camera system 10B includes a fingertip camera device 20B and an image processing unit 30. The fingertip camera device 20B includes five camera units 21A to 21E, five myoelectric sensors (biological information detection units) 22A to 22E, and a control unit 23B. As shown in FIG. 8, the first camera unit 21A is attached to the fingertip of the user's hand (for example, the fingertip of the index finger (the fingertip of the second finger)), and the second camera unit 21B is attached to the adjacent fingertip (for example, the fingertip of the middle finger (the fingertip of the third finger)). Further, the third camera unit 21C is attached to the adjacent fingertip (for example, the fingertip of the ring finger (the fingertip of the fourth finger)), and the fourth camera unit 21D is attached to the adjacent fingertip (for example, the fingertip of the little finger (the fingertip of the fifth finger)). Further, the fifth camera unit 21E is attached to another fingertip (for example, the fingertip of the thumb (the fingertip of the first finger)). Here, the reference numerals of the respective camera units and the fingers to which they are attached are assigned for convenience of explanation and can be changed as appropriate. Similarly, myoelectric sensors 22A to 22E are attached to the base portions of the respective fingers.

[0046] The control unit 23B is different in configuration from the control unit 23 of the above embodiment in that it includes an adjacent camera angle calculation unit 45. The adjacent camera angle calculation unit 45 calculates the angles between two adjacent camera units (mainly the angles in the horizontal direction with respect to the palm and the angles when spreading or narrowing the fingers) based on the electromyogram information of each finger detected by the five electromyogram sensors 22A to 22E. As described above, the direction (angle) of each camera unit 21A to 21E with respect to the palm can be calculated by inputting the detected electromyogram information into a predetermined learning model. Therefore, the angle between two adjacent camera units can be calculated from the directions (angles) of these camera units 21A to 21E.

[0047] Next, the operation of the fingertip camera system 10B will be described. FIG. 9 is a flowchart showing the procedure of the operation of the fingertip camera system. In the present embodiment, as shown in FIG. 8, the camera unit 21A is attached to the fingertip of the index finger, and the electromyogram sensor 22A is attached to the base of the index finger. Also, the camera unit 21B is attached to the fingertip of the middle finger, and the electromyogram sensor 22B is attached to the base of the middle finger. Also, the camera unit 21C is attached to the fingertip of the ring finger, and the electromyogram sensor 22C is attached to the base of the ring finger. Also, the camera unit 21D is attached to the fingertip of the little finger, and the electromyogram sensor 22D is attached to the base of the little finger. Also, the camera unit 21E is attached to the fingertip of the thumb, and the electromyogram sensor 22E is attached to the base of the thumb. Also, the control unit 23B is attached to the wrist.

[0048] First, the operation unit 54 of the control unit 23B is operated to capture images with the five camera units 21A to 21E attached to the five fingertips respectively (step S21). In this case, the captured imaging information is sent from each camera unit 21A to 21E to the control unit 23B respectively, and the imaging information acquisition unit 40 acquires it. Next, the electromyogram sensors 22A to 22E detect the electromyogram information accompanying the movement of each finger during imaging and send it to the control unit 23B, and the electromyogram information acquisition unit 42 acquires the electromyogram information of each finger (step S22). These imaging information and electromyogram information are stored in the storage unit 48 in association with each other.

[0049] Next, the adjacent camera angle calculation unit 45 calculates the angles of two adjacent camera units based on the myoelectric information of each finger detected by the five myoelectric sensors 22A to 22E (step S23). Next, the camera direction detection unit 46 detects the relative directions (imaging directions) of the camera units 21A to 21E with respect to the reference direction of the arm orientation (step S24). The camera direction detection unit 46 obtains the direction DS in which the user's arm extends, which is detected by the position sensor 44. This direction DS in which the arm extends is calculated as a three-dimensional direction vector. By using this direction in which the arm extends as the reference direction, the camera direction detection unit 46 detects (calculates) the relative direction D1 of the camera unit 21A with respect to this reference direction, the relative direction D2 of the camera unit 21B, the relative direction D3 of the camera unit 21C, the relative direction D4 of the camera unit 21D, and the relative direction D5 of the camera unit 21E.

[0050] Next, the output unit 52 adds the relative direction information of the camera units 21A to 21E obtained from the camera direction detection unit 46 to the imaging information captured by the camera units 21A to 21E, respectively, and outputs the result to the image processing unit 30 (step S25). In this case, a method of multiplexing the direction information into the imaging information (for example, data in a video compression format such as MPEG) can be adopted.

[0051] Next, the stereoscopic image generation processing unit 33 of the image processing unit 30 generates stereoscopic image information representing a three-dimensional structure based on the plurality of captured images captured by the camera units 21A to 21E and the direction information indicating the relative directions of the camera units 21A to 21E corresponding to the imaging information (step S26). In this case, the stereoscopic image generation processing unit 33 can generate stereoscopic image information by using a so-called photogrammetry technique. That is, the stereoscopic image generation processing unit 33 extracts the imaging target as feature points from the plurality of imaging information captured by the camera units 21A to 21E of the fingertip camera device 20B, and generates stereoscopic image information by associating the feature points extracted from the plurality of imaging information based on the direction information of the plurality of cameras corresponding to the imaging information.

[0052] Next, the control unit 35 of the image processing unit 30 displays the generated stereoscopic image information on the display unit 34 (step S27) and ends the process.

[0053] In this embodiment, since the camera unit 21 is attached to the fingertip, it is possible to easily capture an image just by pointing at a distant view or a nearby object. For example, it is also possible to capture an enlarged video (image) close to a small object like a magnifying glass. Furthermore, since the myoelectric sensor 22 can simultaneously acquire information regarding the direction of the finger to which the camera unit 21 is attached, it is possible to easily detect the imaging direction of the camera unit 21 attached to the fingertip. Also, in this embodiment, since five camera units 21A to 21E are provided, for example, the spaces extending from each finger around the palm when grasping a ball can be imaged by the five camera units with respective parallaxes. Therefore, based on the five acquired imaging information and the relative direction information of each camera unit with respect to the arm, it is possible to perform an operation as a space with depth and horizontal and vertical parallaxes from the five image information, and generate a stereoscopic image.

[0054] Next, a modified example will be described. FIGS. 10 to 12 are diagrams showing a schematic configuration of a camera unit according to the modified example. In the example of FIG. 10, the camera unit 121 includes a camera body 122 disposed at the fingertip, a lens 123 provided on the camera body 122, and a ring 124 for attaching the camera body 122 to the fingertip. In this configuration, compared with the finger sack type shown in each of the above-described embodiments, since the ventral side of the finger is open, there is no hindrance to fine work at the fingertip or fingerprint authentication.

[0055] Also, in the example of FIG. 11, the camera unit 221 includes a camera body 222 disposed at the fingertip and a lens 223 provided on the camera body 222. The camera body 222 is attached to the fingertip nail by, for example, an adhesive or a suction cup. In this configuration, since no ring is used, the ventral side of the finger is more open, so that fine work at the fingertip and fingerprint authentication can be easily performed.

[0056] Also, in the example of FIG. 12, the camera unit 321 is of the finger-sac type, similar to the above-described embodiment, and includes a bottomed cylindrical camera body 322 into which the fingertip is inserted, and a lens 323 provided on the outer peripheral surface of the camera body 322. This lens 323 is disposed, for example, on the ventral side of the finger. According to this configuration, for example, when the camera units 321 are attached to five fingers and each finger is bent, each camera unit 321 is disposed on the arc formed by each fingertip, so that imaging can be performed in a state where the imaging target is surrounded 360°, and a stereoscopic image can be easily formed.

[0057] As described above, according to the present embodiment, the fingertip camera device 20 includes a camera unit 21 attached to the fingertip of a hand, an electromyographic information acquisition unit 42 that acquires electromyographic information of the palm, a camera direction detection unit 46 that detects the relative direction of the camera unit 21 with respect to a predetermined reference direction based on the acquired electromyographic information, and an output unit 52 that adds the direction information of the camera unit 21 obtained from the camera direction detection unit 46 to the imaging information captured by the camera unit 21 and outputs the result. Therefore, the direction of the camera unit 21 attached to the fingertip can be easily detected.

[0058] In addition, since the electromyographic information acquisition unit 42 acquires electromyographic information accompanying the movement of the finger to which the camera unit 21 is attached, the direction of the camera unit 21 accompanying the movement of the finger can be easily detected.

[0059] In addition, since the camera units 21 are respectively attached to a plurality of fingertips, a stereoscopic image can be generated, for example, using the captured images of the plurality of camera units 21.

[0060] Further, a position sensor 44 that is attached to the wrist and detects the direction of the arm is provided, and the camera direction detection unit 46 detects the relative direction of the camera unit 21 with respect to the detected direction of the arm as a reference direction. Therefore, the direction of the camera unit can be easily detected following the direction of the arm moved by the user.

[0061] In addition, since the electromyographic sensor 22 for detecting electromyographic information is attached to the base portion of the finger where the camera unit 21 is mounted or to the wrist, it is possible to accurately detect the electromyographic information associated with the movement of the finger, and it is possible to accurately detect the direction of the camera unit 21 associated with the movement of the finger.

[0062] In addition, the fingertip camera system 10 includes the fingertip camera device 20, a plurality of captured images captured by the camera unit 21, and a stereoscopic image generation processing unit 33 that generates stereoscopic image information representing a three-dimensional structure based on the direction information indicating the relative direction of the camera unit 21 corresponding to these captured images. Therefore, a stereoscopic image can be easily generated.

[0063] As described above, the embodiments of the present invention have been described, but the present invention is not limited to the above embodiments. For example, in the above-described embodiment, the electromyographic sensor 22 is provided at the base of the finger, but it may be provided integrally or separately from the control unit 23 attached to the wrist, for example.

Description of Reference Numerals

[0064] 10, 10A, 10B Fingertip camera system (camera system) 20, 20A, 20B Fingertip camera device (camera device) 21, 21A, 21B, 21C, 21D, 21E Camera unit 22, 22A, 22B, 22C, 22D, 22E Electromyographic sensor (biological information detection unit) 23, 23A, 23B Control unit 30 Image processing unit 33 Stereoscopic image generation processing unit 34 Display unit 40 Captured image information acquisition unit 42 Electromyographic information acquisition unit (biological information acquisition unit) 44 Position sensor (arm direction detection unit) 45 Adjacent camera angle calculation unit 46 Camera direction detection unit 52 Output unit

Claims

1. A camera unit attached to a finger, a biological information acquisition unit that acquires biological information of the palm, a camera direction detection unit that detects a relative direction of the camera unit with respect to a predetermined reference direction based on the acquired biological information, an output unit that adds and outputs the direction information of the camera unit obtained from the camera direction detection unit to imaging information captured by the camera unit, A camera device comprising the above.

2. The camera device according to claim 1, wherein the biological information is electromyographic information accompanying the movement of the finger to which the camera unit is attached.

3. Comprising a wrist direction detection unit attached to the wrist for detecting the direction of the arm, The camera device according to claim 1 or 2, wherein the camera direction detection unit detects a relative direction of the camera unit with respect to the reference direction using the detected direction of the arm as the reference direction.

4. The camera device according to any one of claims 1 to 3, wherein the biological information detection unit for detecting the biological information is attached to the base of the finger to which the camera unit is attached or to the wrist.

5. A camera device according to any one of claims 1 to 4, and a stereoscopic image generation processing unit that generates stereoscopic image information representing a three-dimensional structure based on direction information indicating the relative direction of the camera unit corresponding to a plurality of pieces of imaging information captured by the camera unit, A camera system comprising the above.

Images