Detection device and input device

Abstract

To provide a detection device capable of distinguishing and detecting various modes of contact, and an input device capable of inputting various types of information or a portion of the types of information into a target device and making it easier to input information.SOLUTION: An input device 1 including a detection device includes a contact part 30 that can elastically deform, a marker part 34 provided on the contact part 30 and displaced in response to the deformation of the contact part 30, an imaging unit 14 that acquires a reference image and a latest image related to the marker part 34, and a control unit 18 that stores and processes the reference image and the latest image. The marker part 34 includes a general marker and a reference marker. The control unit 18 generates contact detection information from the arrangement of the general marker and the reference marker in the latest image relative to the arrangement of the general marker and the reference marker in the reference image. The control unit 18 generates input command information based on the contact detection information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a detection device that detects contact and an input device for performing an operation to input various types of information to a target device. [Background technology]

[0002] 2. Description of the Related Art Input devices such as mice are known for inputting pointer movement information, which is information relating to the movement of a pointer (cursor), into various devices such as electronic devices and machine tools. 2. Description of the Related Art As a mouse for inputting pointer movement information to a computer or the like, an optical mouse is known, which is a mouse in which the amount of pointer movement is read by an image sensor. The optical mouse has a light emitting unit, an image sensor, a memory unit, a control unit, and a wheel. The light emitting unit shines light onto the location where the optical mouse is placed. The image sensor sequentially reads the state of the location as image data. The memory unit stores multiple image data. The control unit calculates the amount of pointer movement from the movement among the multiple stored image data, and generates pointer movement information. The wheel handles information different from the pointer movement information, such as scroll amount information. The control unit transmits the pointer movement information and wheel information to be input to the computer.

[0003] On the other hand, an optical contact sensor disclosed as a second embodiment in Japanese Patent No. 5660531 (Patent Document 1) is known. In this optical contact sensor, a dot pattern 20 is arranged on a hemispherical portion of a membrane 6 that can deform according to the shape of the target object, and when the hemispherical portion comes into contact with an object W1, the behavior in three-dimensional space related to the image of the dot pattern captured by the CCD camera 4 is extracted by the CPU 12, and tactile information regarding the contact with the object W1 is obtained. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5660531 Summary of the Invention [Problem to be solved by the invention]

[0005] With the optical mouse, only pointer movement information and wheel information can be input to a computer, etc. Furthermore, with the optical mouse, in order to input pointer movement information, it is necessary to move the entire optical mouse on a desk, etc. Furthermore, since the optical contact sensor obtains only contact information related to the contact of the object W1, it cannot be used as is in an input device that handles pointer movement information and the like.

[0006] One of the main objectives of the present disclosure is to provide a detection device capable of distinguishing and detecting various types of contact. Also, one of the main objects of the present disclosure is to provide an input device capable of inputting various types of information or a part of the types of information to a target device. Furthermore, another main object of the present disclosure is to provide an input device that allows information to be input to a target device more easily. [Means for solving the problem]

[0007] This specification discloses a detection device. The detection device may include a contact portion capable of elastic deformation. The detection device may include a marker portion provided on the contact portion and displaced in response to deformation of the contact portion. The detection device may include an imaging portion that acquires a first image and a second image related to the marker portion. The detection device may include a control portion that stores and processes the first image and the second image. The marker portion may include a marker. The control portion may generate contact detection information from an arrangement of the markers in the second image relative to an arrangement of the markers in the first image. The present specification also discloses an input device, which generates input command information based on the contact detection information generated by the detection device. Effect of the Invention

[0008] One of the main advantages of the present disclosure is that it provides a detection device that can distinguish and detect various types of contact. Moreover, one of the main effects of the present disclosure is that an input device capable of inputting various types of information or a part of the types of information to a target device is provided. Furthermore, another major effect of the present disclosure is to provide an input device that allows information to be input to a target device more easily. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a perspective view of an input device according to a first embodiment. [Diagram 2] 2 is a schematic central vertical cross-sectional view of the input device of FIG. 1; [Diagram 3] 2 is a bottom view of the touch pad in the input device of FIG. 1. [Figure 4] 2 is a flowchart showing an example of the operation of the input device of FIG. 1; [Diagram 5] Figure 5A is a schematic diagram of the process of identifying general markers in the input device of Figure 1, which involves the process of grasping the first vector and the first general marker; Figure 5B is a schematic diagram of the process of identifying general markers, which involves the process of grasping the second vector to the fourth vector; and Figure 5C is a schematic diagram of the process of identifying general markers, which involves the process of grasping the first general marker. [Figure 6] FIG. 2 is a schematic diagram showing the identification of a general marker in the input device of FIG. 1. [Figure 7] FIG. 5 is a schematic diagram relating to the parallel movement process of FIG. 4. [Figure 8] 5 is an image relating to the translation process of FIG. 4. [Figure 9] 5 is a schematic diagram relating to the enlargement process in the enlargement / reduction process of FIG. 4. [Figure 10] This is an image relating to the enlargement process of the enlargement / reduction process of FIG. [Figure 11]5 is a schematic diagram relating to a reduction process in the enlargement / reduction process of FIG. 4. [Figure 12] This is an image related to the reduction process of the enlargement / reduction process of FIG. [Figure 13] FIG. 5 is a schematic diagram of the twisting process in FIG. 4. [Figure 14] This is an image related to the twisting process in Figure 4. [Figure 15] FIG. 13 is a schematic diagram of a marker section in the input device of the second embodiment. [Figure 16] FIG. 16A is a schematic diagram of a case where general markers overlap when processing an image related to the marker section of the second form, and FIG. 16B is a schematic diagram of a case where general markers overlap in a manner other than that of FIG. 16A when processing an image related to the marker section of the second form. [Figure 17] FIG. 13 is a schematic diagram relating to identification of a general marker in image processing related to the marker section of the second form. [Figure 18] FIG. 13 is a schematic diagram of a marker section in the input device of the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure and modified examples thereof will be described with reference to the accompanying drawings. The present disclosure is not limited to the following embodiments and modified examples.

[0011] Fig. 1 is a perspective view of an input device 1 according to a first embodiment of the present disclosure, and Fig. 2 is a schematic central vertical cross-sectional view of the input device 1 of Fig. 1. The input device 1 includes a main body 2, a touch pad 4, and a cord 6. The input device 1 detects contact with the touch pad 4, and therefore may also be considered as a detection device. The input device 1 includes a detection device. The input device 1 generates input command information, which is information commanding an input to an input-receiving device, in response to contact detection information, which is information relating to the detected contact. The contact detection information and the input command information may be the same or different from each other. Furthermore, the detection device according to the present disclosure may be used for devices other than the input device 1, and may not need to generate input command information. For convenience, various directions of the input device 1 are set as shown in each drawing, but the input device 1 may be used in other orientations.

[0012] The main body 2 has a housing 10, an imaging area case 12, an imaging section 14, an illumination section 16, and a control section 18.

[0013] The housing 10 directly or indirectly supports various members. The outer surface of the housing 10 is colored black. However, the color of the outer surface of the housing 10 may be other than black. The outer surface of the housing 10 does not have to be colored. The housing 10 has a box body 20 and a lid 22. The housing 10 may be formed from something other than the box body 20 and the lid 22, such as being formed from three or more panels. The box body 20 is in the shape of a box with an open top. The front surface of the box body 20 is in the shape of a semi-cylindrical cylinder. However, the shape of the box body 20 may be other shapes. The lid 22 is plate-shaped and covers the upper opening of the box 20. A hole 24 for the touch pad 4 is formed in the front of the lid 22.

[0014] The touch pad 4 is elastic and deformable so as to return to its normal shape, i.e., the shape it has when no external force is applied, and has a contact portion 30, a base portion 32, and a marker portion 34. The contact portion 30 is usually a curved plate having upper and lower semi-spherical surfaces. The contact portion 30 is elastically deformable, and receives an operation by a user (a person inputting data) on its upper surface by the action of the user's finger or the like. The base portion 32 is usually in the form of a plate with holes extending outward from the lower side of the contact portion 30 in all directions. The touch pad 4 is fixed to the lid 22 with the contact portion 30 protruding upward and the contact portion 30 and the base portion 32 closing the hole 24.

[0015] FIG. 3 is a bottom view of the touch pad 4. As shown in FIG. A marker portion 34 is provided on the contact portion 30 of the touch pad 4. The marker portion 34 is displaced in response to the deformation of the contact portion 30. The marker section 34 includes a plurality of general markers 36 and a reference marker 38 . Each of the general markers 36 is dot-like. Each of the general markers 36 is provided on the lower surface of the contact portion 30. Each of the general markers 36 is regularly arranged in a line up in the front-rear and left-right directions. The reference marker 38 has a different shape from the general markers 36 and is "+" shaped. The reference marker 38 is provided in the center of the lower surface of the contact portion 30 and is disposed at the center of the arrangement of the general markers 36. Instead of the shape of the reference marker 38, or in addition to the shape of the reference marker 38, at least one of the color, pattern, and size may be different from that of the general markers 36. The touchpad 4 is semi-transparent, and the upper surface of the touchpad 4 is colored black. The general markers 36 and the reference markers 38 are white. In the drawings other than the photographs in this application, the touchpad 4 is shown in a black and white inverted state for convenience. Each of the general markers 36 and the fiducial markers 38 are applied to the contact portion 30 by printing. The shape of each of the general markers 36 may be other than a dot. The shape of the reference marker 38 may be other than a “+” shape. The shape of a part of each of the general markers 36 may be different from the shape of the other general markers 36. At least one of the general markers 36 and the reference marker 38 may be different in color instead of or in addition to the shape. The difference in color of the general markers 36 is not used in the first form, but is used in the second and third forms described below. The touch pad 4 may not be translucent, but may be transparent or opaque. The upper surface of the touch pad 4 may be colored other than black, or may not be colored. At least one of the general markers 36 and the reference marker 38 may be formed by a method other than printing, such as pasting a sticker. In addition, the arrangement of at least one of the general markers 36 and the reference markers 38 may be arranged in a regular manner other than the vertical and horizontal directions, such as arranged on a plurality of virtual concentric circles. Furthermore, the arrangement of the general markers 36 and / or the reference markers 38 may be irregular.

[0016] The imaging area case 12 is in the shape of a rectangular parallelepiped box. The imaging area case 12 is disposed within the housing 10 and supported by the housing 10 . The imaging area case 12 is disposed below the touch pad 4. The imaging area case 12 has a lower hole 40 which is a hole for the imaging unit 14, and an upper hole 42 which is a hole corresponding to the contact unit 30.

[0017] The imaging unit 14 is an imaging device capable of acquiring an electronic image, and is, for example, a CCD camera or a CMOS camera. Here, the imaging unit 14 can receive at least visible light and generate a captured image. The imaging unit 14 has an imaging element in a light receiving area. The imaging element has an R element that generates red shading information of the incident light, a G element that generates green shading information of the incident light, and a B element that generates blue shading information of the incident light. The imaging unit 14 is a color camera that can obtain a colored image. The imaging unit 14 may obtain a colored image using shading information other than RGB, or may be a monochrome camera that handles only shading information of the luminance of the incident light. The imaging unit 14 is inserted into a lower hole 40 arranged on the lower surface of the imaging area case 12, and can acquire an image of the upper side. The imaging unit 14 is fixed to the imaging area case 12. The imaging unit 14 is arranged below the contact unit 30, and can acquire an image of the marker unit 34 of the contact unit 30. The angle of view of the imaging unit 14 is set to cover the entire marker unit 34 at the fixed position of the imaging unit 14, as shown by the dashed dotted line L1 in FIG. 2. Here, since the marker unit 34 is arranged over the entire area of ​​the lower surface of the contact unit 30, the image acquired by the imaging unit 14 shows the contact unit 30 together with the marker unit 34.

[0018] The illumination unit 16 illuminates the marker unit 34 which is the imaging target. The illumination unit 16 includes a light emitting unit 50 and a half mirror 52 . The light emitting unit 50 is capable of emitting light in a planar manner, and in this embodiment is a plurality of LEDs fixed to the rear inner surface of the imaging area case 12 in a state of being aligned vertically and horizontally. The half mirror 52 is disposed in front of the light emitting unit 50 and above the imaging unit 14. The half mirror 52 is plate-shaped and fixed in the imaging area case 12 in a front-up (rear-down) position. As shown by the dashed-dotted arrow L2 in Fig. 2, the half mirror 52 reflects the illumination light traveling forward from the light emitting unit 50 upward so that the illumination light travels toward the marker unit 34 on the underside of the contact unit 30. The half mirror 52 also transmits light from the marker unit 34 on the underside of the contact unit 30 to the imaging unit 14 so that the imaging unit 14 can capture an image of the marker unit 34 even if the half mirror 52 is present. In addition, in the illumination unit 16, the half mirror 52 may be omitted, and the light emitting unit 50 may be disposed on the upper surface of the imaging area case 12, i.e., on the touch pad 4 side, so that the marker unit 34 is indirectly illuminated by light reflected on the inner surface of the imaging area case 12. Also, the illumination unit 16 may illuminate the marker unit 34 by another structure.

[0019] The control unit 18 controls the input device 1 . The control unit 18 is, for example, a bare circuit board within the housing 10, or an electronic component disposed within a control unit case. The control unit 18 is disposed within the housing 10 and supported by the housing 10. The control unit 18 has a ROM 60 as a part of a storage unit, an I / O 62, a RAM 64 as a part of the storage unit, and a CPU 66 as a control execution unit. The ROM 60 is a read-only memory that stores the control program, etc. The ROM 60 is a non-transitory computer-readable storage medium. The control program contains instructions for implementing the functions described below. The I / O 62 is an interface, and is communicatively connected to the RAM 64, the CPU 66, the imaging unit 14, the light emitting unit 50, and the cord 6. The RAM 64 is a readable and writable memory, and stores various types of information such as images. The CPU 66 is an information processing device, and is connected to the ROM 60. The CPU 66 is also communicatively connected to the RAM 64, the imaging unit 14, the light-emitting unit 50, and the cord 6 via the I / O 62. The CPU 66 executes a control program in the ROM 60 to control the input device 1. The CPU 66 reads and writes information from and to the RAM 64. The CPU 66 controls the imaging unit 14. The CPU 66 controls the light-emitting unit 50. All information flows are bidirectional, except for the one-way flow from the ROM 60 to the CPU 66 and the one-way flow from the I / O 62 to the light-emitting unit 50. The configuration of the control unit 18 can be changed in various ways. For example, the ROM 60 may be connected to the I / O 62, or the RAM 64 may be directly connected to the CPU 66. The ROM 60 and the RAM 64 may be integrated into one memory. In addition, the flow of information may be unidirectional from the imaging unit 14 to the I / O 62, or may be bidirectional between the I / O 62 and the illumination unit 16.

[0020] The cord 6 includes a signal line and a power line. The cord 6 is connected to a target device for input, that is, an input-receiving device, such as a computer or a game machine. The cord 6 is detachable from the input device 1 and the input-receiving device. Code 6 transmits and receives signals to the input device. The signal to the input device may include input command information. The input command information includes at least one of input type information and input command value. The input type information is information indicating the type of input, for example, at least one of cursor movement, zooming in / out of an object, and rotation of an object. The input command value is information indicating the degree of input, for example, at least one of cursor movement amount, zooming in / out of an object, and rotation amount. Moreover, the cord 6 transmits power from the input device to the main body 2. The power is used for emitting light from the light-emitting unit 50 and for driving the control unit 18. The cord 6 does not have to have a power line. Power may be supplied from a built-in battery or an external power source other than the input device. A wireless communication unit may be provided instead of the signal line of the cord 6, in which case wireless communication is performed between the input device 1 and the input device. The cord 6 may be non-detachable from at least one of the input device 1 and the input device. The communication in the cord 6 may be one-way communication in which only transmission to the input device is performed.

[0021] Next, an example of the operation of the input device 1 will be described. FIG. 4 is a flowchart according to an operation example. In the operational example, various processes may be replaced with equivalent processes, the order of the processes may be changed as appropriate, and at least some of the processes may be performed in parallel.

[0022] When the cord 6 is connected to the input device and power is supplied to the main body 2, the CPU 66 performs initial settings (step S1). In the initial setting, the CPU 66 confirms communication with the input device. Moreover, the CPU 66 issues a command to emit light to the light emitting unit 50. The light emitting unit 50 emits light based on this command. In addition, in the initial setting, other processing may be executed. Furthermore, the CPU 66 may command the light emitting unit 50 to emit light when, for example, the start of movement of the non-illuminated contact unit 30 is detected.

[0023] Next, the CPU 66 controls the imaging unit 14 to command the imaging unit 14 to acquire an image at a predetermined timing, and stores the image of the marker unit 34 from the imaging unit 14 in the RAM 64 (step S2). The predetermined timing is a timing having an interval that allows detection of the movement of the contact unit 30 with sufficient accuracy by analyzing a plurality of images. The CPU 66 stores the image in the RAM 64 each time the imaging unit 14 acquires a successive image. The RAM 64 stores a plurality of images within a predetermined period, from the latest image to an image going back a predetermined time. The predetermined period is, for example, one second. The CPU 66 erases past images that are outside the predetermined period from the RAM 64. The predetermined timing may be equal intervals every predetermined time, or may be unequal intervals. Since each image is acquired by the imaging unit 14 arranged below the marker unit 34 and on the opposite side of the contact unit 30 from the side receiving the user's operation, the front-to-back direction when each image is oriented along the horizontal plane does not match the front-to-back direction of the input device 1 shown in Fig. 1 etc. at the time of image acquisition, and the front of the input device becomes the rear of each image. In other words, when the imaging unit 14 is set to acquire an image in a state where the actual left part of the marker unit 34 becomes the left part of the image and the actual right part of the marker unit 34 becomes the right part of the image, that is, in a state where the actual left and right of the marker unit 34 match with the left and right of the image, the image is acquired in a state where the actual front and back of the marker unit 34 become the rear part of the image and the actual rear part of the marker unit 34 becomes the front part of the image, that is, in a state where the actual front and back of the marker unit 34 and the front and back of the image are reversed. Therefore, in consideration of further facilitating understanding, the CPU 66 performs a front-to-back inversion process on the image from the imaging unit 14 in order to match the front-to-back direction of the image with the front-to-back direction of the input device 1, and stores the image in the RAM 64. In other words, the CPU 66 converts an image corresponding to the marker unit 34 as seen from below the contact unit 30, i.e., the imaging unit 14 side, into an image as seen from above the contact unit 30, i.e., the user side, by inversion process. The images and their schematic diagrams shown in the drawings are both inverted. The CPU 66 may convert the input command information generated after image processing to match the actual direction without flipping the image. Also, unlike the above, when the imaging unit 14 is set in a state where the front-to-back direction of the image acquired by the imaging unit 14 matches the actual front-to-back direction, the actual left-to-right direction and the left-to-right direction of the image are reversed, so left-to-right flipping processing may be performed on the image. Furthermore, even in this case, the image may not be flipped left-to-right, and the input command information generated after image processing may be converted to match the actual direction.

[0024] Then, the CPU 66 identifies the general marker 36 (step S3). Fig. 5A is a schematic diagram of the process of identifying the general marker 36, which involves grasping the first vector V1 and the first general marker 36. Fig. 5B is a schematic diagram of the process of identifying the general marker 36, which involves grasping the second vector V2 to the fourth vector V4. Fig. 5C is a schematic diagram of the process of identifying the first general marker 36, which involves grasping the first general marker 36. Fig. 6 is a schematic diagram of the process of identifying the general marker 36. In the following, in order to simplify the configuration and make it easier to understand, the explanation will be given mainly on the case where the general markers 36 are arranged in 5 columns and 5 rows (excluding the center). Whether the general markers 36 are arranged as shown in FIG. 3 or in another arrangement, the processing after the identification of the general markers 36 can be performed in the same manner as described below.

[0025] The CPU 66 first identifies the reference marker 38 for the image to be processed in the RAM 64, i.e., the processing target image. The CPU 66 identifies the reference marker 38 based on the shape of the reference marker 38 which differs from that of the general marker 36. The processing target image here is the latest image. Note that the processing target image may be an image other than the latest image in the RAM 64.

[0026] Next, as shown in FIG. 5A, the CPU 66 identifies the general marker 36 adjacent to and behind the reference marker 38 in the processing target image, and sets the identification information for identifying the general marker 36 to number 1. In the marker portion 34, at least one of rotation, expansion / contraction, and translation is performed in accordance with the elastic deformation of the contact portion 30. In Fig. 5A, the marker portion 34 is rotated counterclockwise as viewed from above with respect to a reference position where the general markers 36 are aligned straight in the front, back, left and right directions. The reference position is the position of the marker portion 34 in the contact portion 30 in a natural state without elastic deformation, and can be considered as the initial position. Here, the identification information of the general marker 36 is a natural number. However, the identification information of the general marker 36 may be something other than a natural number.

[0027] Next, the CPU 66 grasps a first vector V1, which is a vector from the reference marker 38 to the first general marker 36, as shown in FIG. 5B. Moreover, the CPU 66 grasps a second vector V2, which is a vector obtained by rotating the first vector V1 by 90 degrees counterclockwise when viewed from above. Furthermore, the CPU 66 grasps a third vector V3, which is a vector obtained by rotating the first vector V1 by 180 degrees counterclockwise when viewed from above. The third vector V3 can be said to be a vector obtained by reversing the direction of the first vector V1 while maintaining the magnitude. In addition, the CPU 66 grasps a fourth vector V4, which is a vector obtained by rotating the first vector V1 by 270° counterclockwise when viewed from above. The fourth vector V4 can be said to be a vector obtained by reversing the direction of the second vector V2 while maintaining the magnitude. Alternatively, the fourth vector V4 can be said to be a vector obtained by rotating the first vector V1 by 90° clockwise when viewed from above.

[0028] 5C, the CPU 66 sets the identification information of the general marker 36 closest to the end point of the second vector V2 arranged with the No. 1 general marker 36 as the base point to No. 2. In other words, the CPU 66 sets the identification information of the general marker 36 closest to a position away from the position of the No. 1 general marker 36 by the distance of the second vector V2 to No. 2. Further, the CPU 66 sets the identification information of the general marker 36 that is closest to a position that is away from the position of the general marker 36 No. 2 by the distance of the third vector V3 as No. 3. Furthermore, the CPU 66 sets the identification information of the general marker 36 that is closest to a position away from the position of the general marker 36 No. 3 by the distance of the third vector V3 as No. 4. Furthermore, the CPU 66 sets the identification information of the general marker 36 that is closest to a position that is away from the position of the fourth general marker 36 by the distance of the fourth vector V4 as number 5. In addition, the CPU 66 sets the identification information of the general marker 36 that is closest to the position away from the position of the general marker 36 No. 5 by the distance of the fourth vector V4 as No. 6. Further, the CPU 66 sets the identification information of the general marker 36 that is closest to the position away from the position of the sixth general marker 36 by the distance of the first vector V1 as the seventh. 6, the CPU 66 repeats the above-described assignment of identification information to the general markers 36, assigning identification information to a total of 24 general markers 36, and identifying the general markers 36. The CPU 66 sequentially assigns the identification information to the general markers 36 in a spiral shape with the reference marker 38 as the base point and center. The CPU 66 may sequentially assign the identification information of the general marker 36 in a spiral pattern in the opposite direction to that shown in Fig. 6, or in a pattern other than a spiral pattern. The CPU 66 may also determine a forward vector as the first vector V1, and the second vector V2 and subsequent vectors.

[0029] Furthermore, except for the first processing target image, the CPU 66 refers to the immediately preceding image that is consecutive in time when identifying the general marker 36 related to the processing target image. By referring to the state in which the identification information was added in the immediately preceding image, the general marker 36 can be identified more accurately. The CPU 66 may also refer to other past processing target images, such as referring to the image immediately preceding the immediately preceding image. Furthermore, the CPU 66 does not need to refer to past processing target images when identifying the general marker 36. If the number of images acquired sequentially is small and the interval between acquisition of temporally consecutive images is long relative to the speed of movement of the marker portion 34, which corresponds to the speed of elastic deformation of the contact portion 30, then even if general marker number 36 in the temporally consecutive images has actually rotated by approximately 90°, general marker number 36, which has also rotated by approximately 90°, may be mistakenly recognized as general marker number 36 number 1, which may result in a malfunction of the input device 1 not detecting the rotational movement related to the actual input. In contrast, in the input device 1, if the image acquisition interval is sufficiently short compared to the maximum speed at which the marker unit 34 can move, i.e., if the image acquisition interval is sufficiently short compared to the changes in the marker unit 34, the CPU 66 can correctly track the progress of the first vector V1 as it actually is. Incidentally, by adjusting the degree of elasticity of the contact portion 30, the maximum angle related to the rotational movement of the marker portion 34 may be set to any angle within the angle range from 30° to 60°.

[0030] Next, the CPU 66 performs processes according to various movements of the marker portion 34 (steps S4 to S6). The various types of movement processing are performed on two images. The two images are a first image and a second image. Here, the first image is an image of the marker section 34 at the reference position, i.e., a reference image, and the second image is the latest image in the RAM 64, i.e., the latest image. In both of these images, identification information (numbers) is given to the general marker 36. The storage mode of at least one of the first image and the second image can be changed in various ways, such as storing the reference image in the ROM 60 and storing only one latest image in the RAM 64. The assignment of the identification number to the reference image, which is the first image, may be performed in advance by the CPU 66 or a device other than the CPU 66, and may be stored in advance as part of the information of the reference image or in a state associated with the reference image. Furthermore, the various types of movement processing may be performed on combinations other than the reference image and the latest image, such as two images that are consecutive in time. Furthermore, the various types of movement processing may be performed on three or more images. The various movement processes are performed based on the arrangement of the general markers 36 and the reference markers 38 in the latest image relative to the arrangement of the general markers 36 and the reference markers 38 in the reference image. Various input command information is calculated by the various movement processes. The various types of movement processing include a parallel movement (tilt) processing (step S4), a scaling (push & pull) processing (step S5), and a twist processing (step S6). Here, the CPU 66 processes steps S4 to S6 in this order. The CPU 66 may perform processes related to the movement of the marker unit 34 other than steps S4 to S6. The CPU 66 may also change the order of the processes of steps S4 to S6, or may perform at least any two of the processes in parallel.

[0031] Fig. 7 is a schematic diagram relating to the translation processing, and Fig. 8 is an image relating to the translation processing. In step S4, the CPU 66 grasps the amount of movement of the general markers 36 with the same number from the reference image to the latest image, and calculates the average amount of movement of all the general markers 36. More specifically, when the code of the general marker 36 in the reference image is the general marker 36S and the code of the general marker 36 in the latest image is the general marker 36N, the CPU 66 grasps the horizontal movement amount, which is the movement amount in the left-right direction (horizontal direction) from the first general marker 36S in the reference image to the first general marker 36N in the latest image, and the vertical movement amount, which is the movement amount in the front-back direction (vertical direction). When the vector from the first general marker 36S in the reference image to the first general marker 36N in the latest image is set to the first displacement vector indicating a change in position, the horizontal movement amount of the first general marker 36 can be said to be the horizontal component of the first displacement vector, and the vertical movement amount of the first general marker 36 can be said to be the vertical component of the first displacement vector. Similarly, the CPU 66 grasps the horizontal movement amount and the vertical movement amount of the general markers 36S and 36N with the same number from the reference image to the latest image. That is, including the first general marker 36, where n is a natural number between 1 and 24, the horizontal movement amount of the nth general marker 36 is the horizontal component of the nth displacement vector, and the vertical movement amount of the nth general marker 36 is the vertical component of the nth displacement vector. Then, the CPU 66 calculates the average value of the horizontal movement amount and the average value of the vertical movement amount for all of the general markers 36S, 36N. In reality, as shown in Figure 8, the magnitude of horizontal and vertical movement often differs for each number of the general marker 36, so the overall parallel movement amount of the general marker 36 can be grasped by taking the average value of each. In the input device 1, it can be considered that the detection of the parallel movement of the contact portion 30 is performed by generating an average value of the horizontal movement amount that is not zero and an average value of the vertical movement amount that is not zero. The average value of the horizontal movement amount and the average value of the vertical movement amount correspond to contact detection information that is information related to the detected contact with the contact portion 30. The contact detection information here relates to the parallel movement of the contact portion 30. In addition, the CPU 66 may take into account the amount of movement from the reference marker 38 in the reference image to the reference marker 38A in the latest image when calculating the average amount of horizontal movement and the average amount of vertical movement. The CPU 66 may also calculate the average amount of horizontal movement and the average amount of vertical movement based on the amount of movement of some general markers 36, such as excluding the amount of movement of the general marker 36S adjacent to the reference marker 38S in the reference image and the general marker 36N adjacent to the reference marker 38N in the latest image. In addition, the statistically processed value of the horizontal movement and the statistically processed value of the vertical movement may be grasped by a statistical method other than a simple average, such as calculating an average value weighted with respect to the amount of movement. On the other hand, in the input device 1 as a detection device, the parallel movement of the contact portion 30 may be detected when at least one of the value related to the horizontal movement and the value related to the vertical movement is equal to or greater than a predetermined threshold. The threshold value may be different between the value related to the horizontal movement and the value related to the vertical movement.

[0032] The CPU 66 causes the average value of the horizontal movement amount to correspond to a horizontal command value which is an input command value for the amount of movement in the horizontal direction for the input device, and causes the average value of the vertical movement amount to correspond to a vertical command value which is an input command value for the amount of movement in the vertical direction for the input device. The CPU 66 transmits the horizontal command value and the vertical command value together with input type information indicating that the input type information is related to cursor movement to the input device via the cord 6. The CPU 66 may temporarily store a part or all of the input type information in the RAM 64, and the same applies to other types of input command information. The input-receiving device, for example, moves a cursor on its display device in the horizontal direction based on a horizontal command value of the received input type information, and moves the cursor in the vertical direction based on a vertical command value of the received input type information. The input-receiving device, for example, adjusts the horizontal movement amount of the cursor in accordance with the magnitude of the horizontal command value, and adjusts the vertical movement amount of the cursor in accordance with the magnitude of the vertical command value. In this case, when the user deforms the upper part of the contact part 30 slightly to the left from its natural state by operating with a finger or the like, the cursor moves slightly to the left on the input device, when the user deforms the upper part of the contact part 30 significantly forward, the cursor moves significantly up, and when the user deforms the upper part of the contact part 30 to the lower right, the cursor moves to the lower right. When the user removes his / her finger from the contact part 30 and stops acting on the contact part 30, the contact part 30 returns to its natural state due to elasticity, the marker part 34 returns to its initial position, the reference image is continuously acquired as the latest image, and the cursor stops. In addition, at least one of the horizontal movement amount and the vertical movement amount may be a movement amount related to another direction, and at least one of the horizontal command value and the vertical command value may be an input command value related to another direction. In addition, the input-receiving device may perform horizontal scrolling of an active operation target on the display device in response to the received horizontal command value, or may perform vertical scrolling of an active operation target in response to the received vertical command value.

[0033] Fig. 9 is a schematic diagram relating to the enlargement process in the enlargement / reduction process. Fig. 10 is an image relating to the enlargement process. Fig. 11 is a schematic diagram relating to the reduction process in the enlargement / reduction process. Fig. 12 is an image relating to the reduction process. In the enlargement / reduction process of step S5, the CPU 66 performs enlargement (push) processing and reduction (pull) processing.

[0034] In the enlargement process, the CPU 66 targets the four outermost general markers 36, i.e., general markers 36 no. 12, 16, 20, and 24. The CPU 66 then calculates various distances D1 to D4 relating to these general markers 36 in the latest image, compares them with the distances DA to DD in the reference image, and generates an enlargement command value according to the increase in the sum of the distances D1 to D4 in the latest image relative to the sum of the distances DA to DD in the reference image. More specifically, the CPU 66 calculates a distance D1 from the 12th general marker 36N located at the rear right to the 16th general marker 36N located at the front right in the latest image. The CPU 66 also calculates a distance D2 from the 16th general marker 36N to the 20th general marker 36N located at the front left in the latest image. Furthermore, the CPU 66 calculates a distance D3 from the 20th general marker 36N to the 24th general marker 36N located at the rear left. In addition, the CPU 66 calculates a distance D4 from the 24th general marker 36N to the 12th general marker 36N.

[0035] Then, the CPU 66 sums up these distances D1, D2, D3, and D4 to calculate an increase with respect to the sum of the similar distances DA, DB, DC, and DD in the reference image, and generates an enlargement command value according to this increase. The enlargement command value is a type of input command value. The sum of the distances D1 to D4 corresponds to the first distance, and the sum of the distances DA to DD corresponds to the second distance. The first distance may be the distance between one set of two points, the sum of the distances between two or three sets of two points, or the sum of the distances between five or more sets of two points. In this respect, the second distance can also be changed in a similar manner. The first distance may be preset and stored as information of the reference image. In reality, as shown in Fig. 10, more than four general markers 36 that can be regarded as the outermost edge can be arranged, and the same process is performed by selecting four of them. Also, depending on how the user indents the contact portion 30, there is often a bias in the magnitude of the increase depending on the type of distances D1 to D4, such as the increase in distances D2 and D4 for distances DB and DD being larger than the increase in distances D1 and D3 for distances DA and DC, and a global enlargement command value is generated by summing up the distances D1 to D4. In the input device 1, the generation of an enlargement command value other than 0 can be regarded as detecting the pressing of the contact portion 30. The enlargement command value corresponds to contact detection information, which is information related to the detected contact with the contact portion 30, and can also be regarded as a pressing detection value. The contact detection information here relates to the pressing of the contact portion 30. The CPU 66 may perform the enlargement process on the general markers 36 other than the outermost edge, or may perform the enlargement process based on the distances related to three or less or five or more general markers 36. The CPU 66 may generate the enlargement command value based on the outward movement amount from the general marker 36S to the general marker 36N instead of the increase in the sum of the distances D1 to D4 between the general markers 36N in the latest image relative to the sum of the distances DA to DD between the general markers 36S in the reference image, or together with the increase. Furthermore, the enlargement command value may be generated by a statistical method other than the above, such as calculating a weighted sum according to the type of the distances D1 to D4, DA to DD, or calculating a value other than the sum of the distances D1 to D4, DA to DD. On the other hand, in the input device 1 as a detection device, the pressing down of the contact portion 30 may be detected when the pressing down detection value becomes equal to or greater than a predetermined threshold value.

[0036] The CPU 66 transmits the enlargement command value to the input-receiving device via the cord 6 together with input type information indicating that it is an enlargement command. The input-receiving device performs an enlarged display of the active operation target based on, for example, an enlargement command value of the received input command information The input-receiving device adjusts the amount of enlargement according to, for example, the magnitude of the enlargement command value. In this case, when the user presses down the upper part of the contact part 30 from the natural state by operating with a finger or the like to make the upper part of the contact part 30 concave downward, the lower part of the contact part 30 deforms so as to spread outward in the front-rear and left-right directions in accordance with the deformation of the upper part, and the marker part 34 spreads outward. Thus, the active operation target of the input device is expanded. If the user's depression is strong, the spread of the contact part 30 also becomes larger, and the total distance of the latest image becomes larger than the total distance of the reference image, and the enlargement amount of the active operation target becomes large. When the user removes his / her finger from the contact part 30 and stops acting on the contact part 30, the contact part 30 returns to its natural state due to elasticity, the total distance does not change between the reference image and the latest image, the increase in the total distance becomes 0, and the enlarged display of the active operation target stops. The input device may continue to enlarge the active operation target while receiving an enlargement command value whose magnitude is not 0, and may treat the magnitude of the enlargement command value as the magnitude of the increase in the enlargement amount. In this case, when the user depresses the contact unit 30 to a large extent, the enlargement amount increases correspondingly quickly, and the enlargement magnification of the enlarged display increases more quickly. In this case, after the user stops operating the contact unit 30, the enlarged display of the active operation target may be maintained with the last enlargement magnification maintained, or a process may be performed to return to the display state before the enlarged display.

[0037] On the other hand, in the reduction process, the CPU 66 performs processing similar to that of the enlargement process, except that in the reduction process, a reduction command value is generated based on the decrease in the total distance of the latest image relative to the total distance of the reference image, instead of the increase in the total distance of the latest image relative to the total distance of the reference image. In reality, as shown in FIG. 12, depending on how the user pinches up the contact portion 30, there is often a bias in the magnitude of the decrease depending on the type of distance D1 to D4, such as the decrease in distances D2 and D4 for distances DB and DD being greater than the decrease in distances D1 and D3 for distances DA and DC, and a global reduction command value is generated by summing up the distances D1 to D4. In the input device 1, the generation of a non-zero reduction command value can be regarded as detecting the pinching and lifting of the contact portion 30. The reduction command value corresponds to contact detection information, which is information related to the detected contact with the contact portion 30, and can also be regarded as a pinching and lifting detection value. The contact detection information here is related to the pinching and lifting of the contact portion 30. The reduction process may have the same modified examples as the enlargement process as appropriate. The enlargement process may calculate the increase in the total distance, while the reduction process may calculate the inward movement amount of the outermost general marker 36. The enlargement process may have different processing contents from the reduction process. On the other hand, in the input device 1 as a detection device, when the reduction command value becomes equal to or greater than a predetermined threshold, the pinch-up of the contact portion 30 may be detected. The threshold value may be different between the pressing-down detection value and the pinch-up detection value. Furthermore, at least one of the enlargement process and the reduction process may be omitted. At least one of the pressing-down and the pinch-up of the contact portion 30 may not be detected.

[0038] The CPU 66 transmits the reduction command value to the input-receiving device via the cord 6 together with input type information indicating that it is a reduction command. The input-receiving device reduces the display of the active operation target based on, for example, a reduction command value of the received input command information The input-receiving device adjusts the amount of reduction according to, for example, the magnitude of the reduction command value. In this case, when the user pinches up the contact portion 30 by operation with a finger or the like to make the contact portion 30 thinner, the lower portion of the contact portion 30 deforms so as to narrow inward in the front-rear and left-right directions in accordance with the deformation of the upper portion, and the marker portion 34 shrinks inward. Thus, the active operation target of the input device shrinks. If the user shrinks it strongly, the shrinkage of the contact portion 30 also increases accordingly, and the total distance of the latest image becomes smaller than the total distance of the reference image, so that the shrinkage amount of the active operation target increases. When the user removes his / her finger from the contact portion 30 and stops acting on the contact portion 30, the contact portion 30 returns to its natural state due to elasticity, the total distance does not change between the reference image and the latest image, the reduction in the total distance becomes 0, and the shrinkage display of the active operation target stops. The processing of the reduction command value in the inputted device may have the same modified examples as the processing of the enlargement command value as appropriate. Furthermore, in the inputted device, the processing contents of the enlargement command value and the processing of the reduction command value may be different. Furthermore, the inputted device may exchange the processing contents of the enlargement command value and the processing contents of the reduction command value described above. Furthermore, the inputted device may perform a first click (e.g., left click) operation in response to receiving an enlargement command value of a predetermined size or more, and may perform a second click (e.g., right click) or double click operation in response to receiving a reduction command value of a predetermined size or more.

[0039] Fig. 13 is a schematic diagram of the twisting process, and Fig. 14 is an image of the twisting process. In step S6, the CPU 66 grasps the amount of rotational movement of the general marker 36 having the same number from the reference image to the latest image, and calculates the average amount of rotational movement of the general marker 36. Here, the CPU 66 targets the four general markers 36 adjacent to the reference marker 38 in all directions, namely, general markers 36 no. 1, 3, 5, and 7. The CPU 66 then grasps the rotation angles of these general markers 36 from the positions of the reference image to the positions of the latest image as the amount of rotational movement, and calculates the average value of the four rotation angles. More specifically, the CPU 66 uses a trigonometric function to determine the rotation angle of the first general marker 36, indicated by an arrow M1 in FIG. 13, from the first-1 reference vector, which is a vector from the reference marker 38S of the reference image to the first general marker 36S, indicated by an arrow PA in FIG. 13, and the second-1 reference vector, which is a vector from the reference marker 38N of the latest image to the first general marker 36N, indicated by an arrow P1 in FIG. 13. Here, for the rotation angle, the sign of the counterclockwise rotation angle in FIG. 13 is positive, and the sign of the clockwise rotation angle is negative. Similarly, the CPU 66 uses a trigonometric function to determine the rotation angle (arrow M2) of the third general marker 36, from the first-2 reference vector (arrow PB) from the reference marker 38S of the reference image to the third general marker 36S, and the second-2 reference vector (arrow P2) from the reference marker 38N of the latest image to the third general marker 36N. Also, the CPU 66 uses a trigonometric function to determine the rotation angle (arrow M3) of the 5th general marker 36 from the 1st-3rd reference vector (arrow PC) from the reference marker 38S of the reference image to the 5th general marker 36S and the 2nd-3rd reference vector (arrow P3) from the reference marker 38N of the latest image to the 5th general marker 36N. Furthermore, the CPU 66 uses a trigonometric function to determine the rotation angle (arrow M4) of the 7th general marker 36 from the 1st-4th reference vector (arrow PD) from the reference marker 38S of the reference image to the 7th general marker 36S and the 2nd-4th reference vector (arrow P4) from the reference marker 38N of the latest image to the 7th general marker 36N. The 1-1 reference vector, the 1-2 reference vector, the 1-3 reference vector, and the 1-4 reference vector are each a type of first reference vector. The 2-1 reference vector, the 2-2 reference vector, the 2-3 reference vector, and the 2-4 reference vector are each a type of second reference vector.

[0040] Then, the CPU 66 calculates the average value of these rotation angles and generates a twist command value according to the magnitude of the average value. The twist command value is a type of input command value, and can also be considered as a rotation amount command value. In reality, as shown in FIG. 14, the rotation angles of the arrows M1 to M4 are often not uniform depending on how the user twists the contact portion 30, so a global twist command value is generated by averaging the rotation angles of the arrows M1 to M4. In the input device 1, it can be considered that the detection of the twist of the contact portion 30 is performed by generating a twist command value that is not 0. The twist command value corresponds to contact detection information that is information related to the detected contact with the contact portion 30. The contact detection information here relates to the twist of the contact portion 30. The CPU 66 may perform the twisting process on general markers 36 other than those adjacent to the reference marker 38 in the vertical and horizontal directions, or may perform the twisting process based on the rotation angles of three or less, or five or more general markers 36. The CPU 66 may generate a twist command value based on the rotation angle from the reference marker 38S to the reference marker 38N instead of the average value of the rotation angles of the arrows M1 to M4, or together with the average value. Furthermore, the twist command value may be generated by a statistical method other than the above, such as calculating a value other than the average value of the rotation angles of the arrows M1 to M4. On the other hand, in the input device 1 as a detection device, the twisting of the contact unit 30 may be detected when the twist command value becomes equal to or greater than a predetermined threshold value.

[0041] The CPU 66 transmits the twist command value together with input type information indicating that it is a rotation command to the input-receiving device via the cord 6. The CPU 66 may temporarily store the twist command value in the RAM 64. The inputted device rotates and displays the active operation target on the display device based on, for example, a twist command value of the received input command information. The inputted device adjusts the amount of rotation related to the rotation display according to, for example, the magnitude of the twist command value. In this case, when the user deforms the contact unit 30 from its natural state by slightly twisting it counterclockwise by operation with a finger or the like, the active operation target rotates slightly counterclockwise in the input device, and when the user twists the contact unit 30 largely clockwise, the active operation target rotates largely clockwise. When the user stops twisting the contact unit 30, the contact unit 30 elastically returns to an untwisted state, and the rotation display returns to the original state. The amount of rotation may be an amount of rotation related to a direction other than clockwise and counterclockwise. In addition, in processing the twist command value, the inputted device may continue to perform rotation display while receiving a twist command value whose magnitude is not 0, similar to processing the enlargement command value and the reduction command value, and may treat the positive / negative sign and magnitude of the twist command value as the magnitude of the increase in the amount of rotation in the direction indicated by the sign. Furthermore, the inputted device may scroll the active operation target forward or backward according to the received twist command value.

[0042] Then, the CPU 66 repeatedly performs the processes of steps S2 to S6. Based on such processing, the user can perform three types of input, translation, enlargement / reduction, and rotation, on the inputted device by operating the contact unit 30 of the input device 1. Furthermore, the input device 1 as a detection device can detect contact with the contact unit 30 in a state in which the type of contact, translation, pressing down, pinching up, and twisting, is distinguished, and further, the degree of each is grasped. The processing of steps S2 to S6 is repeated at a speed sufficiently fast compared to the user's input speed and the anticipated maximum movement speed of the marker unit 34. Therefore, even if the user makes two or more types of input to the contact unit 30, the CPU 66 can appropriately perform each type of processing and appropriately transmit each input signal to the input device. Even if at least any two of the three types of inputs are consecutively mixed in an extremely short time as if they were simultaneously mixed by the input device 1, the inputted device can perform display processing corresponding to the mixed inputs by processing each input. For example, when a user performs a sliding operation while twisting the contact unit 30, the inputted device moves the cursor while rotating the active operation object. Also, when a user performs a pressing operation while twisting the contact unit 30, the inputted device rotates and enlarges the active operation object. Also, the input device 1 as a detection device can detect at least any of parallel movement, pressing down or pinching up, and twisting when the contact with the contact unit 30 is mixed. At least one of the input device 1 and the input-received device may be capable of parallel processing for each type of input.

[0043] Next, a second embodiment, which mainly relates to imparting color to the general marker 136, will be described. The input device according to the second embodiment is similar to the first embodiment, including the modified examples, except for the parts specifically described below. In the description of the second embodiment, the same reference numerals are used for the members or parts similar to the input device 1 described above, and the description will be omitted as appropriate. In the input device according to the second embodiment, the marker unit 134 and the processing of the image thereof are changed from those in the input device 1 described above.

[0044] Fig. 15 is a schematic diagram of the marker section 134 in the second form. Fig. 16A is a schematic diagram of a case where a general marker 136 overlaps in image processing related to the marker section 134 in the second form. Fig. 16B is a schematic diagram of a case where a general marker 136 overlaps in a manner different from that of Fig. 16A in image processing related to the marker section 134 in the second form. Fig. 17 is a schematic diagram of identification of a general marker 136 in image processing related to the marker section 134 in the second form. The second type marker portion 134 includes a plurality of general markers 136 and a reference marker 138 .

[0045] Each general marker 136 is similar to the general marker 36 described above, except that each general marker 136 has a color. The general markers 136 come in four colors: red (R), green (G), blue (B), and yellow (Y). In Figs. 15 to 17, the type of line (solid line, dashed line, dashed line, and dashed double-dot line) that depicts the outline of the general marker 136 corresponds to the color of the general marker 136. The outline of all the general markers 136 is the same small circle. However, the outline of the general markers 136 does not have to be the same, and does not have to be a small circle. The reference marker 138 is disposed in the center of the marker section 134, and has the same shape as the general marker 136. The color of the reference marker 138 is red here. In the second form, identification information is also given to the reference marker 138. Its identification number is 0. With respect to color arrangement, the color of any general marker 136 or reference marker 138 is different from the colors of the eight adjacent general markers 136 or reference markers 138 adjacent to it in the front, back, left, right, front right, and rear left. In any general marker 136 or reference marker 138, the colors of the front and rear general markers 136 or reference markers 138, the colors of the left and right general markers 136 or reference markers 138, and the colors of the front right, rear right, front left, and rear left general markers 136 or reference markers 138 are different from each other. For example, for the red reference marker number 138, the front and rear general markers 136 are green, the left and right general markers 136 are yellow, and the right front, right rear, left front, and left rear general markers 136 are blue. Color information, which is information indicating the color of the general marker 136 or the reference marker 138, is stored in the ROM 60 in association with the identification information. The color information may be stored in the RAM 64. In the second form, the reference marker 138 may not be used. At least one of red, green, blue, and yellow in the general marker 136 or the reference marker 138 may be changed to another color. The colors of the general marker 136 or the reference marker 138 may be three or less, or five or more. Furthermore, the color arrangement of the general marker 136 or the reference marker 138 may be changed in various ways, such as making the colors different only between the adjacent front, back, left, and right. From the viewpoint of ease of identification, it is preferable that the colors of the general marker 136 or the reference marker 138 are different from each other between adjacent general markers 136 or reference markers 138.

[0046] The image processing of the marker unit 134 in the second embodiment is basically the same as that of the input device 1 in the first embodiment. Then, in processing the image of the marker portion 134, the accuracy of the processing is further improved, as shown in FIGS. 16A, 16B, and 17.

[0047] More specifically, first, as shown in Figures 16A and 16B, in the image of the marker section 134, it is possible to more clearly distinguish whether the general markers 136 or reference markers 138 numbered 2, 3, and 4 are closer to the imaging section 14 (i.e., lower) than the general markers 136 numbered 0, 1, and 5 (Figure 16A), or whether the general markers 136 or reference markers 138 numbered 2, 3, and 4 are closer to the imaging section 14 (i.e., upper) than the general markers 136 numbered 0, 1, and 5 (Figure 16B). The CPU 66 can determine whether a general marker 136 or a reference marker 138 that overlap each other in the image is on the upper side or the lower side based on the color of the overlapping portion. For example, in the case of the overlap between the reference marker 138 number 0 and the general marker 136 number 3, since the color of the overlapping portion is yellow in the case of Fig. 16A, it can be determined that the general marker 136 number 3 is on the lower side, and in the case of Fig. 16B, since the color of the overlapping portion is red, it can be determined that the reference marker 138 number 0 is on the lower side. The CPU 66 can also more accurately distinguish between the top and bottom based on the color when there is overlap of other general markers 136 or reference markers 138. That is, the CPU 66 can more accurately process the marker section 134 based on the color even when there is overlap (hiding) of the general markers 136 or reference markers 138. For example, the CPU 66 is prevented from mistaking the overlapping general markers 136 or reference markers 138 as a single marker and performing inaccurate processing. Furthermore, the CPU 66 may determine whether the general marker 136 or the reference marker 138 is above or below based on other factors, such as the size of the area of ​​each color of the overlapping general marker 136 or reference marker 138, instead of or in addition to the color of the overlapping portion.

[0048] Next, as shown in Fig. 17, in the process of identifying the general marker 136, even under extremely severe conditions where it is difficult to refer to the progress of the first vector V1, such as when the user twists the contact portion 30 by more than 45° in an instant, causing the marker portion 134 to rotate by more than 45° in an instant, the CPU 66 can grasp the first vector V1 as the movement from the 0th reference marker 138 to the green 1st general marker 136. Furthermore, when the control unit 18 attempts to process the 7th general marker 136 located behind the 0th reference marker 138 as the 1st general marker 136, since the 7th general marker 136 is yellow and not green, which the 1st general marker 136 should have, by checking the color of the general marker 136, it is possible to avoid an error of misidentifying the 7th general marker 136 as the 1st general marker 136. This prevents the CPU 66 from mistakingly identifying the first vector V1 as pointing to the general marker 136 (yellow) No. 7 in Figure 17 based solely on the criterion that the vector is from the reference marker 138 No. 0 to the adjacent vector to the rear. Here, in the second form, the determination of the reference marker 138 numbered 0, which has the same form as the red general marker 136, is performed by searching for the red reference marker 138 surrounded by the green general markers 136 numbered 1 and 5, and the yellow general markers 136 numbered 3 and 7. If there are more general markers 136 and multiple such red general markers 136 exist, the CPU 66 extracts the one located at the most center as the reference marker 138, rather than these red general markers 136. The determination of the reference marker 138 may also be modified in various ways, such as by taking into account at least any of the numbers 2, 4, 6, and 8. Moreover, the color of the No. 5 general marker 136 is the same green as the color of the No. 1 general marker 136 disposed adjacent to the No. 0 reference marker 138 on the other side (opposite side) of the No. 0 reference marker 138. Here, from the viewpoint of preventing malfunction, it is difficult to imagine settings such as the image acquisition interval and the elasticity of the contact portion 30 that would cause the marker portion 134 to instantly rotate by about 180°. Therefore, even if the No. 1 and No. 5 general markers 136 are the same color, it is unlikely to lead to malfunction.

[0049] Additionally, a third form similar to the second form is described. The input device according to the third embodiment is similar to the second embodiment, including modifications, except for the parts specifically described below. In the description of the third embodiment, the same reference numerals are used for the members or parts similar to those of the first and second embodiments, and the description thereof will be omitted as appropriate. In the input device according to the third embodiment, the marker unit 234 and the processing of its image are changed from those in the second embodiment.

[0050] FIG. 18 is a schematic diagram of the marker portion 234 in the third form. The second form of marker portion 234 includes a plurality of general markers 236 and a reference marker 238 .

[0051] Each of the general markers 236 and the reference markers 238 is similar to the general markers 136 and the reference markers 138 described above, except for the type of color. The general markers 236 and the reference markers 238 are regularly arranged in 5 rows and 5 columns. The color of each of the general markers 236 and the reference markers 238 is purple (a mixture of red and blue) having red brightness (R brightness) and blue brightness (B brightness). Note that some of the colors of the general markers 236 and the reference markers 238 may not have R or B brightness. At least one of the R and B brightness may be changed to the brightness of another color. Furthermore, three or more types of color brightness may be used. In addition, the outline of the general markers 136 and the outline of the reference markers 238 may not be the same, and may not be small circles.

[0052] The R brightness of each general marker 236 and reference marker 238 depends on the position in the left-right direction. Of the general markers 236, the five arranged in the leftmost column (first column) have the smallest R luminance ("R1" in FIG. 18). The R luminance of each of the general markers 236 and the reference marker 238 is set to be larger as it moves from the five general markers 236 to the right. More specifically, the R luminance of each of the general markers 236 in the second column, which is adjacent to the first column on the right, is the second smallest R luminance ("R2" in FIG. 18). Similarly, the R luminance of each of the general markers 236 and the reference marker 238 in the third column is the third smallest R luminance ("R3" in FIG. 18), the R luminance of each of the general markers 236 in the fourth column is the second largest R luminance ("R4" in FIG. 18), and the R luminance of each of the general markers 236 in the fifth column is the largest R luminance ("R5" in FIG. 18). The R luminance may be set in various ways. For example, the R luminance may be set to gradually decrease from the first column to the fifth column.

[0053] The B intensity of each general marker 236 and reference marker 238 depends on the position in the front-to-back direction. Of the general markers 236, the five arranged in the frontmost row (first row) have the smallest B luminance ("B1" in FIG. 18). The B luminance of each of the general markers 236 and the reference marker 238 is set to be larger as it goes further back from each of the five general markers 236. More specifically, the B luminance of each of the general markers 236 in the second row, which is adjacent to the first row at the rear, has the second smallest B luminance ("B2" in FIG. 18). Similarly, the B luminance of each of the general markers 236 and the reference marker 238 in the third row has the third smallest B luminance ("B3" in FIG. 18), the B luminance of each of the general markers 236 in the fourth row has the second largest B luminance ("B4" in FIG. 18), and the B luminance of each of the general markers 236 in the fifth row has the largest B luminance ("B5" in FIG. 18). The setting of the B luminance may be changed in various ways. For example, the B luminance may be set to gradually decrease from the first row to the fifth row. Also, the colors of the general markers 236 and the reference markers 238 may be set so that the R luminance gradually changes in the front-back direction and the B luminance gradually changes in the left-right direction.

[0054] By setting the R and B intensities in this way, the colors of the general markers 236 and the reference markers 238 are all different from each other, although they are all generally purple. The general marker 236 with an R intensity of R1 and a B intensity of B1 is the darkest purple. On the other hand, the general marker 236 with R5 and B5 is the brightest purple.

[0055] The processing of the image of the marker portion 234 in the third embodiment is similar to that in the second embodiment. As an outline of the process of assigning identification information in this process, first, the control unit 18 determines a marker having a luminance of R3B3 in the image as the reference marker 238. Next, the control unit 18 determines a marker having a luminance of R3B4 in the image as the first general marker 136. Next, the control unit 18 determines a marker having a luminance of R4B4 in the image as the second general marker 136. The control unit 18 then identifies the subsequent general markers 236 in the same manner. Note that in the third embodiment, the control unit 18 does not need to calculate the first vector V1 to the fourth vector V4.

[0056] In the third embodiment, as in the second embodiment, the control unit 18 can more accurately detect contact with the contact unit 30 and generate input command information by processing the marker unit 234 based on color. Furthermore, if the marker identification process using the first vector V1 to the fourth vector V4 in the first form and the marker identification process based on color in the second or third form are used in combination, the marker portion 234 can be processed more accurately.

[0057] The input device 1 of the first form and the input devices of the second and third forms as the above detection device include a contact section 30 capable of elastic deformation, a marker section 34, 134 provided on the contact section 30 and displaced in response to deformation of the contact section 30, an imaging section 14 for acquiring a reference image and a latest image related to the marker section 34, 134, and a control section 18 for storing and processing the reference image and the latest image. The marker section 34 includes general markers 36, 136 and reference markers 38, 138. The control section 18 generates contact detection information from the arrangement of the general markers 36, 136 and the reference markers 38, 138 in the latest image relative to the arrangement of the general markers 36, 136 and the reference markers 38, 138 in the reference image. Therefore, contact with the contact portion 30 is accurately detected based on the contact detection information. Furthermore, by generating input command information based on the contact detection information, the first type of input device 1 and the second type of input device are provided, which allow information to be easily input to an input-receiving device by operating one elastic contact portion 30.

[0058] Moreover, the marker unit 34 includes a plurality of general markers 36, 136, 236 and reference markers 38, 138, 238. The control unit 18 generates contact detection information from the arrangement of some or all of the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image relative to the arrangement of some or all of the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the reference image. Thus, contact with the contact section 30 is detected in a state in which the type of contact can be distinguished. Also, by generating input command information based on the contact detection information, a first type of input device 1 and a second type of input device are provided that can input various types of information or a part of the types of information to an inputted device, which is a device that is the target of input. Furthermore, a first type of input device 1 and a second type of input device are provided that can more easily input information to an inputted device by various operations on one elastic contact section 30.

[0059] Moreover, the reference images are images of the marker portions 34, 134, and 234 in the undeformed contact portion 30. This provides sufficient accuracy and reduces the amount of calculation. Furthermore, the latest images are successively acquired, and the control unit 18 successively generates contact detection information. Therefore, when the detection of the contact with the contact unit 30 is appropriately and continuously performed, and the detection is used for the input device, the input is appropriately and continuously accepted.

[0060] Moreover, in the input devices of the second and third forms, the colors of the general markers 136, 236 and the reference markers 138, 238 are different from each other. Numbers are assigned as identification information to the general markers 136, 236 and the reference markers 138, 238 in the reference image. The control unit 18 executes a process of assigning numbers to the general markers 136, 236 and the reference markers 138, 238 in the latest image based on the difference in color. This further improves the accuracy of identifying the general markers 136, 236 and the reference markers 138, 238, and improves the accuracy of the detection device or the input device. Furthermore, the general markers 136, 236 and the reference markers 138, 238 are arranged in a regular arrangement, and the color of any general marker 136, 236 or reference marker 138, 238 is different from the color of the general marker 136, 236 or reference marker 138, 238 adjacent in the regular arrangement. Thus, a detection device or input device with good accuracy is provided with a relatively small number of types of colors.

[0061] In the input device 1 of the first form and the input devices of the second and third forms, the marker section 34, 134, 234 includes the general marker 36, 136, 236 and the reference marker 38, 138, 238. Therefore, by setting the reference marker 38, 138, 238, the general marker 36, 136, 236 can be easily identified, improving accuracy, and a wider variety of types of detection or input can be processed.

[0062] Furthermore, the control unit 18 generates contact detection information based on a displacement vector, which is a vector from the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the reference image to the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image (step S4). Thus, detection of contact related to translation to the contact unit 30 is performed with sufficient accuracy. Moreover, the marker section 34, 134, 234 includes a plurality of general markers 36, 136, 236 and reference markers 38, 138, 238. Numbers are assigned to the plurality of general markers 36, 136, 236 and reference markers 38, 138, 238 in the reference image. The control unit 18 executes a process of assigning numbers to the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image (step S3), and calculates a plurality (24) of displacement vectors from the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the reference image to the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image having numbers corresponding to the numbers of the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the reference image, and generates contact detection information based on the plurality of displacement vectors (step S4). Thus, detection of contact related to translation to the contact unit 30 is performed with higher accuracy by referring to the plurality of general markers 36, 136, 236 and the reference markers 38, 138, 238. Furthermore, input command information regarding the movement of the cursor is generated based on contact detection information regarding the translation to the contact portion 30. Therefore, in the input device 1 of the first form and the input device of the second form, the process of moving the cursor (step S4) is performed with sufficient accuracy.

[0063] Furthermore, the marker section 34, 134, 234 includes a plurality of general markers 36, 136, 236 and reference markers 38, 138, 238, and the plurality of general markers 36, 136, 236 and reference markers 38, 138, 238 in the reference image are assigned numbers (step S2), and a first distance DA to DD, which is the distance between at least two of the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the first image, is obtained in advance or calculated by the control unit 18. The control unit 18 executes a process of assigning numbers to the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image (step S3), and calculates second distances D1-D4 between at least two general markers 36, 136, 236 and the reference markers 38, 138, 238 in the latest image having numbers corresponding to the numbers of the general markers 36, 136, 236 and the reference markers 38, 138, 238 in the reference image, and generates contact detection information based on the first distances DA-DD and the second distances D1-D4 (step S5). Thus, the detection of contact related to pressing down and picking up on the contact unit 30 is performed with sufficient accuracy. Furthermore, input command information relating to enlargement and reduction is generated based on contact detection information relating to pressing down and picking up on the contact portion 30. Therefore, the enlargement and reduction process (step S5) is performed with sufficient accuracy.

[0064] Also, the marker unit 34, 134, 234 includes a general marker 36, 136, 236 and a reference marker 38, 138, 238, and a first reference vector (arrows PA to PD) which is a vector from the reference marker 38, 138, 238 to the general marker 36, 136, 236 in the reference image is obtained in advance or calculated by the control unit 18. The control unit 18 calculates a second reference vector (arrows P1 to P4) which is a vector from the reference marker 38, 138, 238 to the general marker 36, 136, 236 in the latest image, and generates contact detection information based on the first reference vector and the second reference vector (step S6). Therefore, the detection of contact related to twisting to the contact unit 30 is performed with sufficient accuracy. Furthermore, the marker unit 34, 134, 234 includes a plurality of general markers 36, 136, 236. The control unit 18 calculates a plurality of first reference vectors (arrows PA to PD) and a plurality of second reference vectors (arrows P1 to P4) (step S6). Thus, detection of a twist-related contact with the contact unit 30 is performed with higher accuracy by referring to the plurality of general markers 36, 136, 236. Furthermore, input command information regarding rotation is generated based on contact detection information regarding the twisting of the contact portion 30. Therefore, the rotation process (step S6) is performed with sufficient accuracy.

[0065] The above-described embodiment and its modified examples can be further modified as follows. At least one of the general markers 136, 236 and the reference markers 138, 238 of the marker section 134, 234 may be changed in shape, size, pattern, or the like in addition to or in addition to the color.

[0066] The marker unit 34, 134, 234 may have only one marker. Even in this case, the control unit 18 can grasp the arrangement of the markers in the second image relative to the arrangement of the markers in the first image and generate input command information related to the movement of the cursor on the input device. The control unit 18 may generate input command information based on the density of markers in a specific area of ​​the first image (first density) and the density of markers in a corresponding area of ​​the second image (second density). For example, input command information instructing enlargement may be generated according to an increase in the second density relative to the first density, and input command information instructing reduction may be generated according to a decrease in the second density relative to the first density. [Explanation of symbols]

[0067] 1··input device (detection device), 14··imaging unit, 18··control unit, 30··contact unit, 34, 134, 234··marker unit, 36, 136, 236··general marker (marker), 38, 138, 238··reference marker (marker), DA to DD··first distance, D1 to D4··second distance.

Claims

1. a contact portion capable of elastic deformation; a marker portion provided on the contact portion and displaced in response to deformation of the contact portion; an imaging unit for acquiring a first image and a second image related to the marker unit; A control unit that stores and processes the first image and the second image; Equipped with The marker portion includes a marker, The control unit generates contact detection information from an arrangement of the markers in the second image relative to an arrangement of the markers in the first image. A detection device comprising:

2. The marker portion includes a plurality of the markers, The control unit generates contact detection information from an arrangement of some or all of the markers in the second image relative to an arrangement of some or all of the markers in the first image.

2. The detection device according to claim 1 .

3. The first image is an image of the marker portion at the contact portion without deformation.

2. The detection device according to claim 1 .

4. The second images are acquired sequentially; The control unit sequentially generates contact detection information.

2. The detection device according to claim 1 .

5. At least one of the color, shape, pattern, and size of some or all of the markers is different from each other; Identification information is assigned to the plurality of markers in the first image, The control unit is A process of assigning the identification information to the plurality of markers in the second image based on at least any one of the differences in the color, the shape, the pattern, and the size is performed.

3. The detection device according to claim 2.

6. The markers are arranged in a regular arrangement, At least one of the color, shape, pattern, and size of any one of the markers is different from at least one of the color, shape, pattern, and size of any one of the markers adjacent to the markers in the regular arrangement.

8. The detection device according to claim 7.

7. The marker portion includes a general marker and a reference marker.

3. The detection device according to claim 2.

8. The control unit generates the contact detection information based on a displacement vector that is a vector from the marker in the first image to the marker in the second image.

2. The detection device according to claim 1 .

9. The marker portion includes a plurality of the markers, Identification information is assigned to the plurality of markers in the first image, The control unit is A process of assigning the identification information to the plurality of markers in the second image is performed, calculating a plurality of displacement vectors from the marker in the first image to the marker in the second image having the identification information corresponding to the identification information of the marker in the first image; The contact detection information is generated based on a plurality of the displacement vectors.

9. The detection device according to claim 8.

10. The contact detection information is related to a translational movement of the contact portion.

10. The detection device according to claim 8 or 9.

11. An input device using the detection device according to claim 10, The control unit generates input command information relating to cursor movement based on the contact detection information.

1. An input device comprising:

12. The marker portion includes a plurality of the markers, Identification information is assigned to the plurality of markers in the first image, A first distance between at least two of the markers in the first image is obtained in advance or calculated by the control unit; The control unit is A process of assigning the identification information to the plurality of markers in the second image is performed, calculating a second distance in the second image between at least two markers having identities that correspond to the identities of the markers in the first image; generating the contact detection information based on the first distance and the second distance; 2. The detection device according to claim 1 .

13. The contact detection information is related to pressing down and picking up of the contact portion. Detector device according to claim 12.

14. An input device using the detection device according to claim 13, The control unit generates input command information regarding enlargement and reduction based on the contact detection information.

1. An input device comprising:

15. The marker portion includes a general marker and a reference marker, A first reference vector, which is a vector from the reference marker to the general marker in the first image, is obtained in advance or calculated by the control unit; The control unit is Calculating a second reference vector, which is a vector from the reference marker to the general marker in the second image; generating the contact detection information based on the first reference vector and the second reference vector; 2. The detection device according to claim 1 .

16. The marker portion includes a plurality of the general markers, The control unit calculates a plurality of the first reference vectors and a plurality of the second reference vectors.

16. The detection device according to claim 15.

17. The contact detection information is related to a twist of the contact portion. Detecting device according to claim 15 or 16.

18. An input device using the detection device according to claim 17, The control unit generates input command information regarding rotation based on the contact detection information.

1. An input device comprising:

Images