Information processing system and information processing method

Abstract

To solve the problem that a touch panel and a keyboard require learning and skill for input and there are many errors. To provide a method for allowing anyone to easily and surely determine specific selection information.SOLUTION: All pieces of selection information are acquired from identification images including specific information by using an image input part 3 and an information processor 1, and the specific selection information is determined according to which identification image is hidden by covering a part of it with a finger or an article.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing system and an information processing method for processing specific information that is information linked to a two-dimensional barcode or the like. [Background technology]

[0002] Conventionally, when multiple two-dimensional codes are printed on a printed material, in order to process the desired code, it is necessary to read the code with a barcode reader or to import the printed material and display it on a screen, select the part with a touch panel, cut out the area around the selected part, and have the barcode reader detect it. In computer-based systems and applications, the problem of complicated operations when users give specific instructions often arises.

[0003] Multi-function applications tend to have complex user interfaces, making it difficult to find the operations you need. This requires users to take multiple steps to perform the desired operation, which takes time and effort.

[0004] When a user interface is not intuitive, it is difficult to understand the operations required to provide specific instructions, increasing the likelihood of mistakes. For example, if an important function is hidden deep in a menu, users will have difficulty finding it.

[0005] When users have to manage many settings and options at once, it can be difficult to provide proper guidance, especially when the dependencies between settings are complex, and incorrect configurations can affect the behavior of the entire system. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2008-299781 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0007] Keyboards, mice, voice inputs, and touchpads are used as devices to transmit specific selected information to information processing systems. While these are convenient in many situations, they can be complex and difficult to operate for the elderly, children, and people with special needs.

[0008] In addition, keyboard inexperienced users are prone to typing errors because they do not remember the exact positions of the keys and make frequent mistakes.

[0009] When multiple codes are printed in advance and scanned to determine specific selection information, there is a chance that an adjacent QR Code (registered trademark) may be read by mistake.

[0010] While leaving sufficient space between adjacent QR codes can reduce the risk of misreading, the operation of scanning the code the user wants to select takes some getting used to and can be particularly difficult for the elderly and children.

[0011] As the number of functions in an information processing system increases, the menu hierarchy becomes more complex, making it difficult for users to find the function they are looking for. This reduces the intuitiveness of operation and poses a major barrier, especially for users with memory or concentration challenges.

[0012] Touchpads have problems such as small, hard-to-see screens and poor response due to humidity or hand oils. Elderly people, in particular, are prone to poor response due to dry fingers, leading to frequent misoperation. Furthermore, voice input requires clear speech, making it difficult for seniors to use. An object of the present invention is to provide an information processing system and an information processing method that are easy to operate and allow even elderly people and children to input information with almost no operational errors. [Means for solving the problem]

[0013] The information processing system of the present invention comprises image information acquisition means for acquiring image information, image information processing means for processing the acquired image information acquired by the image information acquisition means, and information output means for outputting specific selection information determined by the image information processing means, wherein the image information processing means includes first selection information acquisition means for acquiring all selection information selected from first captured image information, second selection information acquisition means for acquiring remaining display selection information excluding hidden selection information from second captured image information, selection information comparison means for comparing all selection information acquired by the first selection information means with display selection information acquired by the second selection information acquisition means, and specific selection information determination means for determining specific selection information based on the all selection information and the display selection information. All selection information is acquired from image information acquired by the image information acquisition means. Hereinafter, a device having such functions will be referred to as first selection information means.

[0014] In such an information processing system, specific information is hidden using a finger, an auxiliary body part, or an object, and then selected information is acquired from image information acquired by image information acquisition means. Hereinafter, a means having such a function will be referred to as second selected information acquisition means.

[0015] The all selection information acquired by the first selection information acquisition means is compared with the display selection information acquired by the second selection information acquisition means to determine the selected specific selection information. Hereinafter, a means having such a function will be referred to as a specific selection information determination means.

[0016] It is preferable that the position of all the selected information acquired by the first selected information acquisition means is acquired as a coordinate value. For example, the distance to the information based on the positional relationship from the selection operator can be acquired as coordinate information.

[0017] The selection information comparison means compares the coordinate information of all the selection information with the coordinate information of the display selection information and determines the specific selection information based on the display selection information acquired by the second selection information acquisition means. In this case, the selection information furthest back as seen by the selector can be determined as the specific selection information.

[0018] By publishing the details of what can be done, it is possible to get an overview of all the possible operations. By adding new selection information to the book, you can add or change functions. Furthermore, the information processing method of the present invention is characterized by including a first imaging information acquisition step of acquiring all selection information by an image information acquisition means, a second imaging information acquisition step of acquiring display selection information excluding hidden selection information by the image information acquisition means, a selection information saving step of comparing the all selection information with the display selection information, and a specific selection information determination step of determining specific selection information based on the all selection information and the display selection information. [Effects of the Invention]

[0019] According to the information processing system and information processing method of the present invention, specific selection information is determined by comparing the full selection information acquired by the first selection information acquisition means with the display selection information acquired by the second selection information acquisition means, so input operations can be performed without using input means such as a keyboard or mouse, making it easy for even elderly people and children to input information. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a diagram showing a schematic diagram of the device configuration of this embodiment; [Figure 2] FIG. 1 is a block diagram showing the functions of an information processing device 1 according to an embodiment of the present invention. [Figure 3] An example of an information card photographed by the image input device 3 in this embodiment [Figure 4] Example of traditional menu structure 43 [Figure 5] FIG. 1 is a diagram showing the positional relationship when this embodiment is implemented. [Figure 6] Main part of the detailed flow chart for the processing of this embodiment [Figure 7] Label A section of the detailed flow chart for the processing of this embodiment [Figure 8] Example of menu used in Example 3 DETAILED DESCRIPTION OF THE INVENTION [Example]

[0021] FIG. 1 shows the configuration of the information processing system of this embodiment, which includes an information processing device 1, a display device 2, an image input device 3 (constituting an image information acquisition means), an internet connection device 4, an operation button device 5, an imaging stand 13, a lighting device 14, and an audio output device 16.

[0022] The operation button device 5 (constituting the input operation means) is recognized by the information processing device 1 as, for example, a keyboard. The key to be defined can be changed by setting the operation button device 5.

[0023] The following is an example of the definition of the operation button device 5. Rotating the rotary operation button 6 to the right is defined as pressing the right arrow key on the keyboard, rotating it to the left is defined as pressing the left arrow key on the keyboard, pressing it is defined as pressing the space key, the "No" button 7 is defined as pressing the ESC key on the keyboard, and the "Yes" button 8 is defined as pressing the Enter key on the keyboard.

[0024] The imaging stand 13 is used to place an information sheet such as that shown in FIG.

[0025] The lighting device 14 is used to brighten the photographing stand 13 when acquiring image information with the image input device 3 (image information acquisition means). When photographing with a camera serving as the image input device 3, the brighter it is, the narrower the aperture becomes, and the deeper the depth of field becomes. The deeper the depth of field, the more accurate the focus becomes, and even finer image information can be acquired.

[0026] The audio output device 16 is configured by, for example, a speaker. A speaker built into the information processing device 1 or the display device 2 can also be used.

[0027] FIG. 2 is a functional block diagram showing an outline of information processing performed by the information processing device 1 of this embodiment shown in FIG.

[0028] The display processing unit 19 processes the guidance display to be displayed on the screen determined by the control processing unit 18, the image captured from the image input device 3, and the like to be displayed on the display device 2.

[0029] The operation processing unit 20 processes information from the operation button device 5 .

[0030] The image input processing unit 21 transfers an image captured from an image input device 3 (for example, a camera) to a storage processing unit 22 as image information.

[0031] The image analysis processing unit 23 (constituting the image information processing means) extracts selection information from the image information in the storage processing unit 22 and transfers it to the storage processing unit 22 as a list of selection information.

[0032] The network connection processing unit uses the Internet connection device 4 to receive and send emails from the Internet.

[0033] The voice output control processing unit uses a voice synthesis function to provide voice guidance. The lighting control processing unit controls the lighting used when the image input device 3 captures an image, but if sufficient illuminance is already obtained depending on the environment in which it is used, the lighting device 14 may be omitted.

[0034] In this embodiment, the image analysis processing unit 23 (image information processing means) includes a first selection information acquisition means 101, a second selection information acquisition means 102, a selection information saving means 103, a specific selection information determination means 104, and an information output means 105. Note that the information output means 105 may be configured separately from the image analysis processing unit 23.

[0035] As will be explained in detail later, the first selection information acquisition means acquires all selection information selected from the first captured image information, the second selection information acquisition means acquires the remaining display selection information excluding hidden selection information from the second captured image information, the selection information comparison means compares the all selection information acquired by the first selection information means with the display selection information acquired by the second selection information acquisition means, the specific selection information determination means determines the specific selection information based on the all selection information and the display selection information, and the information output means 105 outputs the determined specific information.

[0036] Figure 3 shows an example of an information card containing specific information. The specific information includes commands such as LOGIN, MSG, SEND, and RECV, as well as additional information for those commands.

[0037] The authentication card 28 is a card on which the command LOGIN is written, and is used, for example, to inform the information processing device 1 of the identity of the person operating it.

[0038] The mark 29 on the authentication card clearly indicates what kind of card it is so that the user 44 can recognize it. The mark 29 can be easily understood by the user 44 by printing or writing letters, illustrations, symbols, or attaching a sticker or photograph.

[0039] The specific information 30 of the authentication card is information that the information processing device 1 reads. For example, if a two-dimensional barcode Version 5 with 37x37 cells and error correction rate level M is used, it is possible to store information of 52 kanji characters or 122 alphanumeric characters. The authentication card includes, for example, the following information: “LOGIN,Big Brother,username001@denshobako.com,password”

[0040] The above information LOGIN is called a command, and describes an instruction to the information processing device 1. Big Brother is used as a message to be displayed on the display device 2 or addressed by the voice output control processing unit 26. In this case, for example, the voice guidance would be something like, "Hey big brother, welcome back. What would you like to do?"

[0041] The above information username001@denshobako.com and password are used to access the mail server.

[0042] The command card 31 is used to tell the information processing device 1 what to do.

[0043] Message A "I'm home" 33 includes, for example, the following information: "MSG, Mom's home, usermama001@denshobako.com" When this card is used, a message saying "Mom's home" will be sent to usermama001@denshobako.com.

[0044] Message B "Let's go swimming" 34 contains the following information: "MSG, Mommy's going swimming, usermama001@denshobako.com" When you use this card, a message saying "Mom, I'm going swimming" will be sent to usermama001@denshobako.com.

[0045] The mail destination card 37 is a card on which information about the recipient of the mail is written.

[0046] The destination A38 includes, for example, the following information: "SEND,To Dad,userpapa001@denshobako.com"

[0047] The handwritten card 42 is a handwritten card with an image written on it to be attached when sending by e-mail. This card is taken as a photograph by the image input device 3, so it can be anything you want to send as a photograph other than a handwritten card.

[0048] FIG. 4 shows a conventional menu structure 43, and when selecting a message to send and a destination, a menu like the one shown in the figure must be created in advance.

[0049] FIG. 5 is a diagram showing the positional relationship of this embodiment. Between the image input device 3 and the user 44 there is a photographing stand 13, on which are placed the authentication card 28, command card 31, mail destination card 37, and handwritten card 42 as shown in Fig. 3. In other words, when viewed from the user (in other words, the person selecting the specific information), the photographing stand 13 is placed in front of the user, and the image input device 3 (for example, a camera) is placed in front of that.

[0050] FIG. 6 is a detailed flow diagram of this embodiment.

[0051] In step S002, the authentication card 28 is read.

[0052] In step S003, the specific information 30 of the authentication card is stored in the storage processing unit 22.

[0053] In step S004, if no operation is performed for a certain period of time after authentication, a timeout measurement counter for ending the authentication state is cleared to 0. The image analysis processing unit 23 clears to 0 a list stabilization number counter that determines whether all the selection information has been acquired.

[0054] In step S005, the image input processing unit 21 stores in the storage processing unit 22 the image information obtained by the image input device 3 (camera).

[0055] In step S006, the image analysis processing unit 23 (image information processing means) acquires the specific information, acquires the command and its additional information, and stores them in the storage processing unit 22.

[0056] For example, when specific information is obtained from command card 31, four items of specific information are obtained: message A "I'm home" 33, message B "I'm going swimming" 34, message C "I'm going to the piano" 35, and message D "I'm going to play" 36.

[0057] In step S007, the image analysis processing unit 23 stores the coordinate position of the specific position information in the storage processing unit 22. From the perspective of the image input device 3 (camera), the user 44 is recognized as being far away. The distance is obtained when the specific selection information is acquired, specifically, when the QR code is analyzed, and the position of the QR code is obtained in camera pixel coordinates. In this embodiment, the first selection information acquisition means 101 includes a first coordinate information creation means (not shown) for creating coordinate information for all selection information, and this first coordinate information creation means creates coordinate information for the position of the QR code based on the camera pixel coordinates.

[0058] In step S008, the time elapsed since the user authentication is performed is counted up.

[0059] In step S009, if the device is left unused for a long period of time, the device transitions to a standby state. After the device transitions to the standby state, authentication is required again.

[0060] In step S012, it is confirmed whether specific information has been acquired, and if the information is available, in step S013, the process proceeds to label A of step S013 in Fig. 7. If the information is not available, the process proceeds to step S005.

[0061] 7, it is determined whether the number of times the list of specific information has been stabilized exceeds a specified number N. If it does, the process proceeds to step S016. If it does not, the process proceeds to step S024.

[0062] In step S024, a list of specific information stored in the storage processing unit 022 is created.

[0063] In step S025, it is determined whether the list of specific information created in step S024 is the same as the previous list of specific information. If they are the same, the process proceeds to step S026, and if they are different, the process proceeds to step S029.

[0064] In step S026, the number of times the list of specific information is stable is incremented by one.

[0065] In step S029, the number of times the list of specific information is stable is cleared to zero.

[0066] In step S027, it is determined whether the number of times the list of specific information has become stable is equal to or greater than a specified number N, and if the list of specific information is stable, the process proceeds to step S028. If it is not stable, the process proceeds to step S030.

[0067] In step S028, the list of the stabilized specific information is stored in the storage processing unit 22 as all selected information acquired by the first selected information acquisition means 101.

[0068] In step S028, a specific code selection start guide is displayed on the display device 2, and an operation to hide the code to be selected is prompted.

[0069] In step S030, a message is displayed on the display device 2 indicating that a list of specific information is being created.

[0070] In step S016, all the selection information acquired by the first selection information acquisition means 101 is compared with the selection information acquired by the second selection information acquisition means 102 in step S014 to acquire the hidden specific selection information.

[0071] In step S017, it is determined whether there is one piece of selected information. If the specific information is hidden by something such as the user's arm 46, multiple pieces of selected information are acquired. If there are multiple pieces of selection information, the process proceeds to step S018.

[0072] In step S018, the coordinate information recorded in the storage processing unit 022 is referenced, and it is determined that the specific information farthest from the user 44 has been selected. In this embodiment, the second selection information acquisition means 102 also includes a second coordinate information creation means (not shown) for creating coordinate information for the remaining display selection information excluding the hidden selection information, and this second coordinate information creation means creates coordinate information for the position of the QR code based on camera pixel information, as described above.

[0073] As shown in FIG. 5, a photographing stand 13 is placed between the image input device 3 and the user 44, and a mail address card 37, for example, is placed on the photographing stand 13.

[0074] In the case of Figure 5, when the list of specific information is stabilized, the first selection information acquisition means 101 acquires the specific information of destination A38, destination B39, destination C40, and destination D41, and the first coordinate information creation means (not shown) creates coordinate information for these destinations 38 to 41, and the created coordinate information is stored in the storage processing unit 22 (see Figure 2).

[0075] In contrast, when the destination 38A is covered by the selected portion 45 (for example, a finger), the destination D41 is also covered by the user's arm or the like 46.

[0076] In step S016, a stable list of destinations A38, B39, C40, and D41 (i.e., all selected information) and the selected information of destinations B39 and C40 (i.e., the remaining displayed selected information excluding the hidden selected information) is obtained.

[0077] By comparing the all-selection information with the display selection information, it can be determined that destinations A38 and D41 have been hidden. In step S017, it is determined that there are two pieces of selection information. Specifically, the selection information saving means 103 compares the full selection information acquired by the first selection information acquisition means 101 (coordinate information of destinations A38 to D41 generated by the first coordinate information creation means) with the display selection information acquired by the second selection information acquisition means 102 (coordinate information of destinations B39 and C40 generated by the second coordinate information creation means). Then, the specific selected information determination means 104 determines the hidden selected information as the specific selected information. In this case, there are two hidden destinations, destination A38 and destination D41, and the specific selected information determination means 104 determines the selected information (hidden selected information) that is farthest from the user 44 as the specific selected information. In other words, the hidden selected information is the selected information farthest from destination A38 (its coordinate information) and destination D41 (its coordinate information), i.e., destination A38, as the specific selected information. The information output means 105 outputs the determined specific selection information, and the next control is performed based on this specific selection information.

[0078] In step S019, it is determined whether the selected information has been identified.

[0079] In step S020, a message is displayed on the display device 2 to confirm whether the command for the specific information is correct.

[0080] In step S021, it is determined whether the Yes button 8 has been pressed, and if it has been pressed, the command is executed.

[0081] In step S023, it is determined whether the No button 7 has been pressed, and if it has been pressed, the specification of the selected information is cancelled, and the process waits for different selected information to be selected. In this embodiment, the information output means 105 is included as part of the function of the image analysis processing unit 23 (image information processing means), but it can also be configured as an independent unit separate from the image analysis processing unit 23. In this embodiment, the first selection information acquisition means 101 for acquiring all selection information from the first captured image information and the second selection information acquisition means 102 for acquiring display selection information from the second captured image information are configured separately, but these two selection information acquisition means 101, 102 can also be configured as a common unit. In this case, the common selection information acquisition means will perform a first selection information acquisition function for acquiring all selection information (including a first coordinate information creation function for generating coordinate information thereof) and a second selection information acquisition function for acquiring display selection information (including a second coordinate information creation function for generating coordinate information thereof). [Example]

[0082] The command card 31 and mail destination card 37 of FIG. 5 can be a single page of paper or a 100 page book. [Example]

[0083] When ordering food at a restaurant using a menu card 47, the order can be completed simply by pointing with the finger at the food A48 on the printed menu. [Example]

[0084] By preparing command cards for inputting data into computers at production plants, it becomes possible to input data even while wearing protective gloves. Even long, complex commands can be entered without error, and there is no need to worry about accidentally entering commands that are not printed on the card. [Example]

[0085] The auxiliary body parts can be used to operate the computer. People with limb disabilities can operate it with their arms, legs, or even a stick held in their mouth. The invention is based on an adaptive design that is inclusive, allowing access to a wide variety of users 44 with equal ease, while still providing customization for each individual user 44. [Explanation of symbols]

[0086] 1. Information processing equipment 2 Display device 3 Image input device 4. Internet connection device 5 Operation button device 6 Rotary Control Button 7 "No" button 8 "Yes" button 9 Image input device connection cable 10 Internet connection device connection cable 11 Operation button device connection cable 12 Display device connection cable 13 Photo stand 14 Lighting equipment 15 Lighting device connection cable 16 Audio output device 17 Audio output device connection cable 18 Control processing section 19 Display processing section 20 Operation processing section 21 Image input processing unit 22 Memory Processing Unit 23 Image analysis processing unit 25 Network connection processing unit 26 Audio output control processing unit 27 Lighting control processing section 28 Authentication Card 29 Authentication Card Mark 30 Authentication Card Specific Information 31 Command Cards 32 Message A Marker 33 Message A "I'm home" 34 Message B "Let's go swimming" 35 Message C "I'm going to the piano" 36 Message D "Let's go play" 37 Mail Address Card 38 Destination A 39 Destination B 40 Destination C 41 Destination D 42 Handwritten Cards 43 Traditional Menu Structure 44 User 45 Selection 46 User's arms, etc. 47 Menu Card 48 Cooking A 49 Cooking B 50 Cooking C 51 Cooking D 101 first selection information acquisition means 102 second selection information acquisition means 103 Selection information evacuation means 104 Specific selection information determination means 105 Information output means

Claims

1. an information processing system comprising: an image information acquisition means for acquiring image information; an image information processing means for processing the acquired image information acquired by the image information acquisition means; and an information output means for outputting specific selection information determined by the image information processing means, wherein the image information processing means comprises: a first selection information acquisition means for acquiring all selection information selected from first captured image information; a second selection information acquisition means for acquiring remaining display selection information excluding hidden selection information from second captured image information; a selection information comparison means for comparing the all selection information acquired by the first selection information acquisition means with the display selection information acquired by the second selection information acquisition means; and a specific selection information determination means for determining specific selection information based on the all selection information and the display selection information.

2. 2. The information processing system according to claim 1, wherein the first selection information acquisition means includes a first coordinate information creation means for creating coordinate information for all selection information, and the second selection information acquisition means includes a second coordinate information creation means for creating coordinate information for display selection information.

3. The information processing system according to claim 2, characterized in that the selection information comparison means compares the coordinate information of all the selected information with the coordinate information of the displayed selected information, and the specific selection information determination means determines the selection information that is furthest back from the perspective of the selector as the specific selected information based on the coordinate information of all the selected information and the coordinate information of the displayed selected information.

4. An information processing method comprising: a first imaging information acquisition step for acquiring all selection information by an image information acquisition means; a second imaging information acquisition step for acquiring display selection information excluding hidden selection information by the image information acquisition means; a selection information saving step for comparing the all selection information with the display selection information; and a specific selection information determination step for determining specific selection information based on the all selection information and the display selection information.

Images