Information input device, information input program, and storage medium

Abstract

To provide an information input device which allows accurate information input through simple operation.SOLUTION: An information input device includes an imaging unit 2 for imaging motion of a user; a display unit 3 which displays an image captured by the imaging unit 2; a storage unit 7 for storing specific information in association with selection images; and a control unit 9. The control unit 9 includes: selection image display means which displays selection images 11 on the display unit 3; specific part recognition means which recognizes specific parts 6a, 6b of the user in the captured image; tracking image display means which displays tracking images 12, 13 on coordinates of the specific parts 6a, 6b recognized by the specific part recognition means; overlap detection means which detects overlapping between the tracking images 12, 13 and the selection image 11; definitive display means which definitively displays the selection image 11 when the overlap detection means detects overlapping between the tracking images 12, 13 and the selection image 11; and output means which outputs information associated with the selection image 11 definitively displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information input device, an information input program, and a storage medium for inputting information to a personal computer, an information terminal, or the like in a non-contact manner. [Background technology]

[0002] Conventionally, when inputting information into a personal computer, information terminal, etc., input devices such as a keyboard or mouse are used, or specific parts of a touch panel are touched with a finger. To prevent infection by viruses or bacteria, it is desirable to avoid direct contact with the input devices or touch panel.

[0003] For example, in dental clinics, when treating a patient, tools such as mirrors, vacuums, tweezers, probes, excavators, and fillers are arranged on trays and are held in hands wearing disposable gloves before being used in the patient's oral cavity. Because the patient's saliva and cleaning solution adhere to the hands, when inputting information into a computerized electronic medical record, etc., direct contact with the input device or touch panel is not possible, so the input information is usually communicated by voice to an assistant or dental hygienist who then inputs it.

[0004] Patent Document 1 describes an information input device that captures an image of a user's hand grasping a menu space in a virtual space, recognizes the hand's movement and position, and determines which menu space the hand has grasped. Patent Document 2 describes a contactless information input device that captures an image of a user's hand, recognizes the shape of the hand, and determines a specific operation indicated by the user's hand shape.

[0005] The information input devices of Patent Documents 1 and 2 require the user to grasp the object with their hand or to form a specific shape, and have the problem that they cannot recognize the object depending on the position of the hand or the speed of the movement. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-75991 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-50177 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made in consideration of the above-mentioned problems of the prior art, and an object of the present invention is to provide an information input device, an information input program, and a storage medium that allow information to be input reliably with simple operations. [Means for solving the problem]

[0008] As a means for solving the above problems, the information input device according to the present invention comprises: an imaging unit that captures the user's actions; a display unit that displays the image captured by the imaging unit; a storage unit that stores specific information in association with a plurality of selected images; a control unit; The control unit a selected image display means for displaying the plurality of selected images on the display unit; a specific part recognition means for recognizing a specific part of the user in the captured image; a tracking image display means for displaying a tracking image at the coordinates of the specific part recognized by the specific part recognition means; an overlap detection means for detecting an overlap between the tracking image and the selected image; a confirmation display means for confirming and displaying the selected image when the overlap detection means detects an overlap between the tracking image and the selected image; and an output unit for outputting information associated with the selected image displayed in the finalized state.

[0009] The selected image preferably includes an indication corresponding to particular information associated with the selected image.

[0010] The specific part of the user is preferably an object held by the user.

[0011] It is preferable that the tracking image has a shape that is the same as or similar to the outer shape of the specific part.

[0012] a color information detection means for detecting color information of the specific part recognized by the specific part recognition means; It is preferable that the image forming apparatus further comprises a coordinate determining means for determining the coordinates of the specific portion by comparing the color information detected by the color information detecting means with a predetermined threshold value.

[0013] The control unit preferably includes a threshold setting unit that sets color information of the specific portion recognized by the specific portion recognition unit to the predetermined threshold before the information input process.

[0014] the specific portion of the user includes a first specific portion and a second specific portion; It is preferable that the confirmation display means displays the selection state of the selected image when it detects an overlap between the tracking image and the selected image of the first specific portion, and displays the confirmation state of the selected image when it detects an overlap between the tracking image and the selected image of the second specific portion.

[0015] The information input program according to the present invention comprises: a selected image display means for displaying a plurality of selected images on a display unit; a specific part recognition means for recognizing a specific part of a user in a photographed image; a tracking image display means for displaying a tracking image at the coordinates of the specific part recognized by the specific part recognition means; an overlap detection means for detecting an overlap between the tracking image and the selected image; a confirmation display means for confirming and displaying the selected image when the overlap detection means detects an overlap between the tracking image and the selected image; and an information input program for causing the display device to function as an output unit that outputs information associated with the selected image that has been confirmed and displayed.

[0016] A computer-readable recording medium having an information input program according to the present invention recorded thereon comprises: a selected image display means for displaying a plurality of selected images on a display unit; a specific part recognition means for recognizing a specific part of a user in a photographed image; a tracking image display means for displaying a tracking image at the coordinates of the specific part recognized by the specific part recognition means; an overlap detection means for detecting an overlap between the tracking image and the selected image; a confirmation display means for confirming and displaying the selected image when the overlap detection means detects an overlap between the tracking image and the selected image; A computer-readable recording medium having recorded thereon an information input program for causing the recording medium to function as an output means for outputting information associated with the selected image displayed in the finalized state. [Effects of the Invention]

[0017] According to the present invention, specific information associated with the selected image is output simply by superimposing a tracking image of a specific part of the user in the captured image on the selected image displayed on the display unit, thereby providing the effect of enabling information to be input reliably with a simple operation. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a block diagram of an information input device according to an embodiment of the present invention. [Figure 2] FIG. 4 is a diagram showing an example of an image on a display unit. [Figure 3] 10 is a flowchart of an information input process. [Figure 4] 4 is a flowchart of the information input process following FIG. 3. [Figure 5] 10 is a flowchart of a calibration process. [Figure 6] FIG. 10 is a diagram showing the areas near both sides of the linear component of the mirror handle. [Figure 7] FIG. 10 is a diagram showing the procedure of information input processing. [Figure 8] FIG. 10 is a diagram showing the procedure of a calibration process for a mirror portion. [Figure 9] 10A to 10C are diagrams showing the procedure of a calibration process for a mirror handle portion. [Figure 10] 10A to 10C are diagrams showing the procedure of another embodiment of the calibration process of the mirror handle. [Figure 11] FIG. 10 is an explanatory diagram of a calibration area of ​​the mirror handle. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0020] 1 shows the configuration of an information input device 1 according to an embodiment of the present invention. The information input device 1 is installed on a side table of a dental treatment unit in a dental clinic, and is suitable for users such as dentists and dental hygienists to input necessary information into a patient's electronic medical record, such as a medical record number, measurements of periodontal pockets, tooth mobility, and caries progression, findings, treatment details, and prescription details. The information input device 1 includes an imaging unit 2, a display unit 3, and a control unit 4.

[0021] The imaging unit 2 is a camera such as a video camera, digital camera, or single-lens reflex camera. The imaging unit 2 can be fixed to a side table of the dental treatment unit with a tripod, attached to an arm supporting the side table, or fixed to the top of a monitor installed on the side table, so as to capture images of the movements of the hands of a user such as a dentist or dental hygienist, specifically, dental instruments held by the user. The HDMI (registered trademark) terminal of the imaging unit 2 is connected to the USB terminal of the control unit 4 via a cable via the capture device 5.

[0022] The dental instrument carried by the user is preferably one of a mirror, tweezers, excavator, filler, or probe, which is placed in a tray on a side table, and in particular a mirror 6 having a circular mirror portion 6a and a linear mirror handle portion 6b.

[0023] The display unit 3 is a liquid crystal display or an organic EL display, and is provided above the side table so that the user can see it during treatment or procedure. The display unit 3 is connected to the control unit 4 by a cable.

[0024] The control unit 4 is a desktop or notebook personal computer, and includes a storage unit 7, a memory 8, and a central processing unit (CPU) 9.

[0025] The storage unit 7 is a hard disk (HDD), SSD, etc., and includes a main storage unit 7a that stores a predetermined program, and a selected image storage unit 7b that stores specific information in association with multiple selected images. Programs for performing information input processing, etc., are copied from a recording medium or downloaded from the Internet or a cloud, and stored in the main storage unit 7a.

[0026] The selected image may have any shape that can be distinguished from the specific part in the captured image, and if the specific part is a circle or a straight line, a rectangle is preferable. If the specific information is a number from 0 to 9, ten display coordinates of the selected image are stored corresponding to each number.

[0027] The memory 8 is a RAM and has sections 8a, 8b, 8c, 8d, and 8e for storing camera image data, mirror circle component coordinates, mirror straight line component coordinates, determination data, and calibration values, respectively.

[0028] The central processing unit 9 performs predetermined processing in accordance with the program read from the main memory unit 7 a. The central processing unit 9 constitutes the selected image display means, specific portion recognition means, tracking image display means, overlap detection means, confirmation display means, and output means of the present invention.

[0029] Next, the operation of the information input device 1 will be described with reference to the flowchart shown in FIG. 3-5.

[0030] Figure 3-5 shows the operation of a flow in which a user such as a dentist or dental hygienist holds a mirror 6 and inputs a medical record number into an electronic medical record using the information input device 1. This flow includes a main loop 1, a loop 2 that performs processing to detect the circular components of the mirror portion 6a, and a loop 3 that performs processing to detect the linear components of the mirror handle portion 6b.

[0031] (Information input processing) In the information input process, the patient's chart number information, which is made up of the numbers 0, 1-9, is inputted in a non-contact manner using the mirror 6 .

[0032] <Loop 1> Loop 1 selects and confirms the selected image associated with the numbers 0, 1-9. In step 1, a camera image is input from the imaging unit 2, and in step 2, the camera image is displayed on the display unit 3. Figure 2 shows the user's hand holding the mirror 6. In step 3, it is determined whether or not the calibration mode is active. The calibration mode is a mode for setting thresholds for the color values ​​of the mirror portion 6a and the mirror handle portion 6b when detecting the circular component of the mirror portion 6a and the linear component of the mirror handle portion 6b. The calibration mode only needs to be performed once at the beginning of loop 1. This calibration mode will be described in detail later.

[0033] If the calibration mode is not selected, the selection rectangles 11 are displayed on the display unit 3 as the selection images in step 4. Figure 2 shows images of 10 selection rectangles 11, with the numbers 0 to 9 displayed within the frames of these selection rectangles 11.

[0034] In step 5, the camera image data is output to the first memory 8a. The camera image data output to the first memory 8a is used in loop 2.

[0035] In step 6, the mirror circle component coordinates are input from the second memory 8b, and in step 7, the mirror tracking circle 12 is rendered and displayed in the mirror circle component coordinates of the camera image. In FIG. 2, the circular tracking circle 12 is displayed around the image of the mirror portion 6a. When the user moves the mirror 6, the tracking circle 12 moves in accordance with this.

[0036] In step 8, it is determined whether the tracking circle 12 overlaps with the selection rectangle 11. This is done based on whether the center coordinates of the tracking circle 12 are within the range of the selection rectangle 11.

[0037] If it is determined in step 8 that the tracking circle 12 does not overlap the selection rectangle 11 and if it is determined in step 9-1 that the selection rectangle 11 is not in a selected state, the process returns to step 1 and is repeated until the tracking circle 12 overlaps the selection rectangle 11 in step 8.

[0038] In step 8, it is determined that the tracking circle 12 overlaps with the selection rectangle 11, and in step 9, it is determined that the selection rectangle 11 is not in a selected state. In step 10, it is determined whether or not other selection rectangles 11 other than the overlapping selection rectangle 11 are in a selected state.

[0039] If the other selection rectangles 11 are not selected in step 10, the overlapping selection rectangles 11 are marked to indicate that they are selected in step 12, and the process moves to step 13. In FIG. 2, the selected state of the selection rectangle 11 is indicated by the thick border of the selection rectangle 11.

[0040] A possible reason why another selection rectangle 11 is selected in step 10 is that the user has placed the tracking circle 12 over an unselected selection rectangle 11, even though the other selection rectangle 11 is selected. In this case, the selection status marking of the other selection rectangle 11 is reset in step 11, and the selection status is marked on the overlapping selection rectangle 11 in step 12, and the process proceeds to step 13.

[0041] If it is determined in step 8 that the tracking circle 12 does not overlap the selection rectangle 11 and it is determined in step 9-1 that the selection rectangle 11 is in a selected state, or if it is determined in step 8 that the tracking circle 12 overlaps the selection rectangle 11 and it is determined in step 9 that the selection rectangle 11 is in a selected state, the selection of the selection rectangle 11 has already been completed, so the process proceeds to step 13.

[0042] 4, the camera image data is output to the first memory 8a. The camera image data output to the first memory 8a is used in loop 3.

[0043] In step 14, the mirror straight line component coordinates are input from the third memory 8c, and in step 15, the mirror tracking line 13 is rendered and displayed at the mirror straight line component coordinates of the camera image. In FIG. 2, the tracking line 13, which consists of two straight lines, is displayed along the image of the mirror handle 6b. When the user moves the mirror 6, the tracking line 13 moves in accordance with this.

[0044] In step 16, it is determined whether the tracing line 13 overlaps with the selection rectangle 11. This is done based on whether the center coordinates of the tracing line 13 are within the range of the selection rectangle 11.

[0045] If it is determined in step 16 that the tracing line 13 does not overlap the selection rectangle 11, the process returns to step 1 and is repeated until the tracing line 13 overlaps the selection rectangle 11 in step 16.

[0046] If it is determined in step 16 that the tracking line 13 overlaps the selection rectangle 11, the overlapping selection rectangle 11 is marked to indicate that it is confirmed in step 17, and the process proceeds to step 18. In FIG. 2, the confirmation state of the selection rectangle 11 is indicated by hatching within the selection rectangle 11.

[0047] In step 18, the number corresponding to the selection rectangle 11 in the confirmed state is output to the fourth memory 8d as confirmed data. The confirmed data output to the fourth memory 8d is displayed in the confirmed data display section 14 of the display unit 3 in step 19. In FIG. 2, 791852 is displayed in the confirmed data display section 14 as information that has already been input.

[0048] When the display of the confirmed data is completed, the marking of the confirmed state is reset in step 20, and the process returns to step 1 at the beginning of loop 1, and the same steps are repeated.

[0049] <Loop 2> 3 performs a circular component detection process for the mirror portion 6a. In step 21, the calibration value is input from the fifth memory 8e, and in step 22, the camera image data is input from the first memory 8a.

[0050] In step 23, a circular component recognition process is performed based on the camera image data to determine whether or not a circular component has been detected. If a circular component has not been detected, the process returns to step 21 and the recognition process is repeated until a circular component is detected.

[0051] The circular component recognition process executes an image recognition program downloaded or installed in the main memory unit 7a. First, a grayscale converter converts the camera image data into black and white pixel data, and each pixel of the camera image data is replaced with integer data between 0 and 255. Next, the Canny edge detection algorithm is applied to the black and white pixel data to binarize it and detect contours (edges). Next, a Hough transform is used to detect circular components.

[0052] If a circular component is detected in step 23, the average values ​​of the RGB values ​​(three integer data of 0 to 255 for Red, Green, and Blue) of the circular component detection unit are detected in step 24 and compared with the calibration values ​​to perform color matching. If color matching is not possible, it is not a circular component of the mirror unit 6a, so the process returns to step 21 and repeats the color matching process until a circular component of the mirror unit 6a is detected.

[0053] After color matching in step 24, the coordinate values ​​of the mirror circle components are output to memory in step 25, and the process returns to step 21 of loop 2 to repeat the same steps. As a result, the mirror circle component coordinates are updated in time series in accordance with the movement of the mirror 6 held by the user.

[0054] <Loop 3> Loop 3 performs a process of detecting the linear component of the mirror handle 6b. In step 31, the calibration value is input from the fourth memory 8d, and in step 32, the camera image data is input from the first memory 8a.

[0055] In step 33, a recognition process for a straight line component is performed based on the camera image data, and it is determined whether or not a straight line component has been detected. If a straight line component has not been detected, the process returns to step 31 and the recognition process is repeated until a straight line component is detected.

[0056] Similar to the process for recognizing circular components, the process for recognizing straight line components involves first converting the camera image data into black and white pixel data using a grayscale converter, and then replacing each pixel in the camera image data with integer data between 0 and 255. Next, the Canny edge detection algorithm is applied to the black and white pixel data to binarize it and detect contours (edges). Next, the straight line components are detected using a Hough transform.

[0057] Once the linear component is detected in step 33, the average values ​​of the RGB values ​​(three integer data items ranging from 0 to 255 for Red, Green, and Blue) of the nearby areas on both sides of the linear component are detected in step 34, and the RGB values ​​are compared with the calibration values ​​to perform color matching. Figure 6 shows the nearby areas Sa and Sb on both sides of the linear components S1 and S2 of the mirror pattern portion 6b. If neither of the nearby areas on either side can be color matched, it is not a linear component of the mirror pattern portion 6b, so the process returns to step 21 and repeats the color matching process until the linear component of the mirror pattern portion 6b is detected.

[0058] If at least one of the neighboring areas on both sides is color-matched in step 34, the coordinate values ​​of the mirror linear component are output to the third memory 8c in step 35, and the process returns to step 31 of loop 3 to repeat the same steps. As a result, the mirror linear component coordinates are updated in chronological order to follow the movement of the mirror 6 held by the user.

[0059] The procedure for the information input process will be described with reference to FIG. 7. FIG. 7(a) shows a camera image of the mirror 6 displayed on the display unit 3. In FIG. 7(b), a selection rectangle 11 is displayed on the display unit 3, a tracking circle 12 is displayed on the mirror portion 6a of the camera image of the mirror 6, and a tracking line 13 is displayed on the mirror handle portion 6b. When the user moves the mirror 6, the tracking circle 12 follows the mirror portion 6a, and the tracking line 13 follows the mirror handle portion 6b. The user moves the mirror 6 to bring the tracking circle 12 closer to the desired selection rectangle 11. As shown in FIG. 7(c), when the tracking circle 12 overlaps the selection rectangle 11, the selection rectangle 11 changes to a selected state. Next, when the user moves the mirror 6 and overlaps the tracking line 13 with the selected selection rectangle 11 as shown in FIG. 7(d), the selection rectangle 11 changes to a confirmed state.

[0060] As described above, in the information input process, to input a medical record number using the numbers 0 and 1 to 9, the user uses the mirror 6 to select the selection rectangle 11 associated with the numbers 0, 1 to 9 by overlapping the tracking circle 12 of the mirror portion 6a with the selection rectangle 11, and then confirms it by overlapping the tracking straight line 13 of the mirror handle portion 6b with the selection rectangle 11, thereby allowing the medical record number to be input using the numbers 0, 1 to 9.

[0061] (Calibration process) Figure 5 shows the calibration process. The calibration process involves setting the allowable range (threshold) of the RGB color values ​​of the mirror portion 6a and mirror pattern portion 6b of the mirror 6 that appear in the camera image as calibration values ​​before performing the information input process in loop 1, the circular component detection process in loop 2, and the straight component detection process in loop 3.

[0062] First, the calibration process for the mirror unit 6a will be described. In step 41, the calibration selection rectangle 21 is rendered and displayed on the display unit 3. In Fig. 8, the calibration selection rectangle 21 for the mirror unit 6a is displayed in the lower right corner of the display unit 3.

[0063] In step 42, the calibration guide circle 22 is rendered and displayed on the display unit 3. In Fig. 8, the calibration guide circle 22 and a rectangular detection portion 23 are displayed in the center of the display unit 3.

[0064] In step 43, the camera image is output to the first memory 8a, and in step 44, the mirror circle component coordinates are input from the second memory 8b. In step 45, the mirror tracking circle 12 is rendered and displayed. In Fig. 8(a), the mirror tracking circle 12 is displayed in bold.

[0065] In step 46, it is determined whether the mirror tracking circle 12 overlaps the calibration guide circle 22. This is done based on whether the center coordinates of the mirror tracking circle 12 are within the range of the calibration guide circle 22. FIG. 8(b) shows the state in which the mirror tracking circle 12 overlaps the calibration guide circle 22.

[0066] If it is determined in step 46 that the mirror tracking circle 12 does not overlap the calibration guide circle 22, the process returns to step 41 via steps 1 to 3, 3-1, and 3-2 of loop 1, and the same steps are repeated.

[0067] If it is determined in step 46 that the mirror tracking circle 12 overlaps with the calibration guide circle 22, a marking display indicating the start of calibration is performed in step 47. In FIG. 8(b), the color of the calibration guide circle 22 has changed.

[0068] In step 48, the size of the detection circle is measured, and in step 49, the color value of the image of the mirror section 6a in the detection unit 23 is detected. Specifically, the pixels of the image of the mirror section 6a are scanned, and the average value of each of the RGB values ​​(three integer data of 0 to 255 for Red, Green, and Blue) of all pixels is calculated. In step 50, the calibration value is output to the fifth memory 8e. The calibration value is a threshold value within a predetermined range centered on the average value of each of the RGB values ​​(three integer data of 0 to 255 for Red, Green, and Blue). For example, if the average value of R among RGB is 100, the calibration value for R is set to 100±50. Calibration values ​​are determined similarly for G and B.

[0069] A marking indicating the end of calibration is displayed in step 51. In FIG. 8(c), the calibration guide circle 22 has returned to its original color.

[0070] In step 52, the process waits for a predetermined time, and in step 53, the marking display of the calibration guide circle 22 is reset to end the calibration process, and the process returns to step 1 of loop 1.

[0071] Next, the calibration process for the mirror handle 6b will be described. In step 61, a calibration selection rectangle 31 is rendered and displayed on the display unit 3. In Fig. 9, the calibration selection rectangle 31 for the mirror handle 6b is displayed in the lower right corner of the display unit 3.

[0072] In step 62, calibration guide rectangles 32a and 32b are rendered and displayed on the display unit 3. In Fig. 9, two calibration guide rectangles 32a and 32b are displayed in the center of the display unit 3, with a rectangular detection section 33 between them.

[0073] In step 63, the camera image is output to the first memory 8a, and in step 64, the linear component coordinates of the mirror handle 6b are input from the third memory 8c. In step 65, the mirror tracking lines 13a and 13b are rendered and displayed. In Figure 9(a), the two mirror tracking lines 13a and 13b are displayed in bold.

[0074] In step 66, it is determined whether one of the two mirror tracking lines 13a, 13b, the tracking line 13a, overlaps the calibration guide rectangle 32a, and in step 67, it is determined whether the other mirror tracking line 13b overlaps the calibration guide rectangle 32b. This is done based on whether the center coordinates of the mirror tracking lines 13a, 13b are within the ranges of the guide rectangles 32a, 32b. Figure 9(b) shows the state in which the mirror tracking lines 13a, 13b overlap the calibration guide rectangles 32a, 32b.

[0075] If it is determined in steps 66 and 67 that the mirror tracking lines 13a and 13b do not overlap the calibration guide rectangles 32a and 32b, the process returns to step 61 via steps 1 to 3 and 3-1 of loop 1, and the same steps are repeated.

[0076] If it is determined in steps 66 and 67 that the mirror tracking lines 13a and 13b overlap the calibration guide rectangles 32a and 32b, a marking indicating the start of calibration is displayed in step 68. In Fig. 9(b), the colors of the calibration guide rectangles 32a and 32b have changed.

[0077] In step 69, the color values ​​of the image of the mirror pattern portion 6b in the detection unit 33 are detected. Specifically, the pixels of the image of the mirror pattern portion 6b are scanned, and the average value of each RGB value (three integer data of 0 to 255 for Red, Green, and Blue) of all pixels is calculated. In step 70, the calibration value is output to the fifth memory 8e. The calibration value is a threshold value within a predetermined range centered on the average value of each RGB value (three integer data of 0 to 255 for Red, Green, and Blue). For example, if the average value of R among RGB is 100, the calibration value for R is set to 100±50. Calibration values ​​are determined similarly for G and B.

[0078] A marking indicating the end of calibration is displayed in step 71. In Fig. 9(c), the calibration guide rectangles 32a and 32b have returned to their original colors.

[0079] In step 72, the process waits for a predetermined time, and in step 73, the marking display of the calibration guide rectangles 32a and 32b is reset to end the calibration process, and the process returns to step 1 of loop 1.

[0080] As described above, in the calibration process, the color values ​​of the mirror portion 6a and mirror pattern portion 6b of the mirror 6 are measured and stored as calibration values ​​in the fifth memory 8e, and the calibration values ​​are used as thresholds in the circular component detection process of loop 2 and the linear component detection process of loop 3, so that the circular component and linear component of the mirror 6 can be detected with high accuracy.

[0081] 10 shows another example of the calibration process for the mirror handle 6b. In this calibration process, one calibration guide rectangle 32 is provided, and the rectangular area between the two mirror tracking lines 13a and 13b is used as the calibration area for the mirror handle 6b and the dynamic detection area 6c.

[0082] As shown in FIG. 11( a), when the end points and midpoint of mirror tracing line 13a are A1, B1, and C1, respectively, the end points and midpoint of mirror tracing line 13b are A2, B2, and C2, respectively, the points where a perpendicular line passing through end points A1 and B1 of mirror tracing line 13a intersects with mirror tracing line 13b are D and E, respectively, the point where a perpendicular line passing through midpoint C1 of mirror tracing line 13a intersects with mirror tracing line 13b is F, the lengths of line segments A1D, B1E, C1F, A1C1, and FC2 are l, m, n, α, and β, respectively, and the thresholds for the width of mirror handle portion 6b are a minimum threshold Tmin and a maximum threshold Tmax, (A) The mirror tracking lines 13a and 13b are within the calibration guide rectangle 32. (B) Tmin < l < Tmax (C) Tmin < m < Tmax (D) Tmin < n < Tmax (E)0 ≦ β< α Calibration is performed when both of the above conditions are met simultaneously.

[0083] To derive the calibration area 6c, as shown in FIG. 11(b-1), when point D is not on the line segment A2B2, the point where the perpendicular line passing through point A2 of the line segment A2B2 intersects with the line segment A1B1 is defined as G, and as shown in FIG. 11(b-2), when point E is not on the line segment A2B2, the point where the perpendicular line passing through point B2 of the line segment A2B2 intersects with the line segment A1B1 is defined as H. (1) As shown in FIG. 11(c-1), when only point G exists on line segment A1B1, the inside of a quadrangle having vertices G, B1, E, and A2 is defined as calibration area 6c. (2) As shown in FIG. 11(c-2), when only point H exists on line segment A1B1, the inside of a rectangle with vertices A1, H, B2, and D is defined as calibration area 6c. (3) As shown in FIG. 11(c-3), when points G and H exist on the line segment A1B1, the inside of a rectangle with vertices G, H, B2, and A2 is defined as the calibration area 6c. (4) As shown in FIG. 11(c-4), when points G and H do not exist on the line segment A1B1, the inside of a quadrangle having vertices A1, B1, E, and D is set as the calibration area 6c. The calibration area 6c is clearly indicated by color.

[0084] If it is determined that the above AE conditions are simultaneously met, the color of the calibration guide rectangle 32 is changed to indicate that calibration has started, as shown in Figure 10(b). All pixels in the calibration area and dynamic detection unit 6c are then scanned to calculate the average value of each RGB value (three integer data values ​​ranging from 0 to 255 for Red, Green, and Blue), and output the respective calibration values. Next, as shown in Figure 10(c), the calibration guide rectangle 32 is returned to its original color, indicating that calibration has ended.

[0085] The present invention is not limited to the above-described embodiments, and can be appropriately modified and changed without departing from the gist of the invention as defined in the claims.

[0086] For example, in the above embodiment, the mirror portion 6a and the mirror handle portion 6b are specified as specific parts of the mirror 6 held by the user, and the selected image 11 is in a selected state using the mirror portion 6a, and the selected image 11 is in a confirmed state using the mirror handle portion 6b. However, the confirmed state may be achieved using only one of the mirror portion 6a and the mirror handle portion 6b. Alternatively, the selected state may be achieved when the mirror portion 6a is displayed as a circle, and the confirmed state may be achieved when the mirror 6 is twisted and the mirror portion 6a is displayed as an oval or a straight line.

[0087] In addition, in the above embodiment, the specific parts of the user are the mirror part 6a and the mirror handle part 6b of the mirror 6 held by the user, but it is not limited to the mirror 6, and it may be other equipment, or the user's fingertips, marks on gloves worn by the user, etc.

[0088] Furthermore, in the above embodiment, the information input by the information input device is the dental electronic medical record number, which is made up of numbers. However, information including symbols and letters may also be used. In this case, it is necessary to display a selection rectangle corresponding to the symbols or letters on the display unit. [Explanation of symbols]

[0089] 1. Information input device 2. Filming Department 3 Display section 4. Control section 5. Capture Device 6. Mirror 6a Mirror part 6b Mirror handle 7 Memory section 8. Memory 9 Central Processing Unit 11 Selection Rectangle 12 Tracking Circles 13 Tracking Line 14 Confirmed data display section 21 Calibration selection rectangle 22 Calibration guide circle 23 Detector 31 Calibration Selection Rectangle 32a, 32b Calibration guide rectangle 33 Detection unit

Claims

1. an image capturing unit configured to capture an image of a motion of a specific part of an object held by a user, the specific part including a first specific part and a second specific part; a display unit that displays the image captured by the imaging unit; a storage unit that stores specific information in association with a plurality of selected images; a control unit; The control unit a selected image display means for displaying the plurality of selected images on the display unit; a specific part recognition means for recognizing a specific part of an object held by a user in the captured image; a tracking image display means for displaying a tracking image at the coordinates of the specific part recognized by the specific part recognition means; an overlap detection means for detecting an overlap between the tracking image and the selected image; a confirmation display means for displaying a selection state of the selected image when an overlap between the tracking image of the first specified portion and the selected image is detected based on whether or not the tracking image of the first specified portion is within the range of the selected image, and for displaying a confirmation state of the selected image when an overlap between the tracking image of the second specified portion and the selected image is detected based on whether or not the tracking image of the second specified portion is within the range of the selected image; An information input device comprising: an output unit that outputs information associated with the selected image that has been confirmed and displayed.

2. The information input device according to claim 1 , wherein the selected image includes an indication corresponding to specific information associated with the selected image.

3. 3. The information input device according to claim 1, wherein the tracking image has a shape that is the same as or similar to the outer shape of the specific part.

4. a color information detection means for detecting color information of the specific part recognized by the specific part recognition means; 4. The information input device according to claim 1, further comprising: a coordinate determination unit that compares the color information detected by the color information detection unit with a predetermined threshold value to determine the coordinates of the specific portion.

5. 5. The information input device according to claim 4, wherein the control unit includes threshold setting means for setting color information of the specific portion recognized by the specific portion recognition means as the predetermined threshold before the information input process.

6. a selected image display means for displaying a plurality of selected images on a display unit; a specific part recognition means for recognizing a specific part of an object held by a user in a captured image; a tracking image display means for displaying a tracking image at the coordinates of the specific part recognized by the specific part recognition means; an overlap detection means for detecting an overlap between the tracking image and the selected image; a confirmation display means for displaying a selection state of the selected image when an overlap between the tracking image of a first specific portion and the selected image is detected based on whether or not the tracking image of the first specific portion of the object held by the user is within the range of the selected image, and for displaying a confirmation state of the selected image when an overlap between the tracking image of the second specific portion and the selected image is detected based on whether or not the tracking image of the second specific portion of the object held by the user is within the range of the selected image; an information input program for causing the program to function as an output means for outputting information associated with the selected image displayed in the finalized state;

7. 7. A computer-readable storage medium storing the information input program according to claim 6.

Images