Information processing system and information processing program

Abstract

To continue a detection state of a physical object, compared with a case in which, when a physical object is detected in a detection area included in a display area of an object operated in a non-contact manner, the detection area remains inside the display area.SOLUTION: An information processing system comprises a processor, and in performing a non-contact operation on an object in a screen by using a physical object, when the physical object is detected in a detection area included in a display area in which the object is displayed in the screen, the processor extends the detection area to the outside of the display area.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system and an information processing program. [Background technology]

[0002] Patent Document 1 discloses a touch panel input device that includes a touch panel that displays a keyboard for input, a display control unit that displays keys on the touch panel, a contact detection unit that detects the position where a finger touches the touch panel, an input determination unit that determines the key that is input from the contact position, and a three-dimensional sensor that detects the position of the finger that is touching or leaving the touch panel and the distance from the touch panel, and is characterized in that when the three-dimensional sensor detects an approaching finger, the display control unit enlarges and displays the keys with the position of the finger as the center, and when the input determination unit confirms the key input, the display control unit returns the keys to their standard size.

[0003] Patent document 2 discloses an information processing device that includes a processor that displays an image containing one or more items on a screen, detects non-contact input from a user to the items, and selects the item corresponding to the detection area when the non-contact input is received for a period of time equal to or longer than a threshold in a detection area whose area is smaller than the area of ​​the items. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-149049 [Patent Document 2] Japanese Patent Publication No. 2023-40955 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure aims to maintain the detection state of an object when the object is detected in a detection area included in the display area of ​​an object operated non-contact, compared to a configuration in which the detection area remains within the display area. [Means for solving the problem]

[0006] A first aspect of the present disclosure is an information processing system that includes a processor, and when a non-contact operation is performed by an object on a screen with the object, the processor expands the detection area outside the display area when the object is detected in a detection area included in a display area in which the object is displayed on the screen.

[0007] A second aspect of the present disclosure is the information processing system of the first aspect, wherein the processor expands the detection area depending on the detection time of the object.

[0008] A third aspect of the present disclosure is the information processing system of the first or second aspect, wherein the processor expands the detection area downward relative to the display area.

[0009] A fourth aspect of the present disclosure is the information processing system of the third aspect, wherein the processor expands the detection area so that it widens toward the end.

[0010] A fifth aspect of the present disclosure is the information processing system according to the first or second aspect, wherein the processor expands the entire detection area to outside the display area.

[0011] A sixth aspect of the present disclosure is the information processing system of the first or second aspect, wherein the processor changes the shape of the detection area depending on the distance of the object from the screen.

[0012] A seventh aspect of the present disclosure is an information processing system according to the first aspect, wherein the processor confirms an operation on the object when the target object is detected on the detection area for a predetermined time.

[0013] An eighth aspect of the present disclosure is an information processing system in which, in the information processing system of the seventh aspect, the processor stops accepting operations on objects other than the object being operated when multiple objects are displayed on the screen.

[0014] A ninth aspect of the present disclosure is an information processing system according to the seventh or eighth aspect, wherein the processor stops the operation when the target object moves outside the detection area of ​​the object being operated.

[0015] A tenth aspect of the present disclosure is an information processing system according to the seventh or eighth aspect, wherein the processor changes at least one of the color and shape of the object when the object is detected on the detection area for a predetermined period of time.

[0016] An eleventh aspect of the present disclosure is an information processing system in which, in the information processing system of the first aspect, the processor switches between selecting the object and confirming the operation assigned to the selected object depending on the distance of the object from the screen.

[0017] A twelfth aspect of the present disclosure is an information processing program for causing a computer to execute the following: when a non-contact operation is performed on an object on a screen by an object, if the object is detected in a detection area included in a display area in which the object is displayed on the screen, the detection area is expanded to outside the display area. [Effects of the Invention]

[0018] In the information processing system of the first aspect, when an object is detected in a detection area included in the display area of ​​an object operated non-contact, the detection state of the object can be continued compared to when the detection area remains within the display area.

[0019] In the information processing system of the second aspect, once the size of the detection region is expanded, it is possible to continue detecting the object, compared to when the size remains constant.

[0020] In the information processing system of the third aspect, compared to when the detection area is expanded upward relative to the display area, it is possible to continue detecting the object even if the position of the object shifts downward due to its own weight.

[0021] In the information processing system of the fourth aspect, compared to when the detection area is expanded downward by a fixed width relative to the display area, the detection state of the object can be continued even if the position of the object shifts diagonally downward due to its own weight.

[0022] In the information processing system of the fifth aspect, compared to when only a part of the detection area is expanded outside the display area, it is possible to continue detecting the object even if the position of the object shifts outside the display area.

[0023] In the information processing system of the sixth aspect, it is easier to continue detecting an object compared to when the shape of the detection area remains constant regardless of the distance of the object from the screen.

[0024] In the information processing system of the seventh aspect, it is possible to prevent erroneous operations by the user compared to when an operation on an object is confirmed when a target object is detected in the detection area.

[0025] In the information processing system of the eighth aspect, it is possible to prevent erroneous operations by the user compared to when operations on objects other than the object currently being operated are accepted.

[0026] In the information processing system of the ninth aspect, it is possible to reduce erroneous operations by the user compared to when the operation is continued even if the target object moves out of the detection area of ​​the object being operated.

[0027] In the information processing system of the tenth aspect, the user can more easily recognize the detected object than in a case where the display mode of the object is maintained even if the target object is detected in the detection area for a predetermined time.

[0028] In the information processing system of the eleventh aspect, it is possible to reduce erroneous operations by the user compared to when switching between selection and confirmation of an operation depending on the time it takes to detect an object.

[0029] In the information processing program of the 12th aspect, when an object is detected in a detection area included in the display area of ​​an object operated non-contact, the detection state of the object can be continued compared to when the detection area remains within the display area. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system equipped with a contactless user interface that allows a user to perform operations without contact. [Figure 2] 1 is a perspective view showing a main part of an image processing apparatus according to an embodiment; [Figure 3] 1A is a cross-sectional view of the operation panel, and FIG. 1B is a plan view of the operation panel as viewed from a position opposite the display surface of the operation panel. [Figure 4] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing apparatus according to an embodiment. [Figure 5] 10A to 10C are diagrams illustrating an example of screen transitions, showing how screens displayed on an operation panel change in response to user operations on the operation panel. [Figure 6] FIG. 10 is a diagram showing an example of a selection in which a user selects a copy button, which is an example of an object, on the home screen. [Figure 7]FIG. 2 is a block diagram showing an example of the configuration of a main part of an electrical system of the image processing apparatus according to the embodiment. [Figure 8] FIG. 2 is a front view of a copy screen when the operation panel of the image processing apparatus is viewed from the front. [Figure 9] FIG. 10 is a front view of the copy screen, showing a state in which the user's finger has been moved over an object. [Figure 10] The copy screen in FIG. 9 shows a state in which the object detection area begins to expand. [Figure 11] The copy screen in FIG. 10 shows a state in which the object detection area has been further expanded. [Figure 12] The copy screen in FIG. 11 shows a state in which the user's finger has moved onto another object within the detection area of ​​the object. [Figure 13] The copy screen in FIG. 12 shows a state in which the user's finger has moved outside the object detection area. [Figure 14] 4 is a flowchart of an information processing program according to the present embodiment. [Figure 15] FIG. 10 is a side view showing the space between the operation panel and the user's finger in another embodiment. [Figure 16] FIG. 10 is a plan view showing another embodiment in which the object detection area is further expanded on the copy screen. DETAILED DESCRIPTION OF THE INVENTION

[0031] An example of an embodiment of the technology of the present disclosure will be described in detail below with reference to the drawings. Note that components and processes that perform the same operations, actions, and functions are given the same reference numerals throughout the drawings, and duplicated descriptions may be omitted as appropriate. Each drawing is merely a schematic illustration to allow a sufficient understanding of the technology of the present disclosure. Therefore, the technology of the present disclosure is not limited to the illustrated examples. Furthermore, in this embodiment, descriptions of configurations that are not directly related to the present disclosure or well-known configurations may be omitted.

[0032] FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 including an information processing device equipped with a non-contact user interface that allows a user to perform operations without contact.

[0033] The information processing device in the information processing system 1 may be a device applicable to any field as long as it has a contactless user interface. Examples of the information processing device include an image processing device, an ATM (Automatic Teller Machine), a vending machine, a ticket issuing machine, etc. The information processing device may be a device for personal use or a device used by an unspecified number of users.

[0034] 1 and 2, an image processing device 10 installed in a workplace or the like will be described as an example of an information processing device.

[0035] FIG. 2 is a perspective view showing the main parts of the image processing device 10 according to this embodiment.

[0036] As will be explained later, the image processing device 10 is a device that executes image-related functions in accordance with user instructions. The image processing device 10 is connected to a plurality of terminals 4 used by each user via a communication line 2, for example.

[0037] The user transmits image data generated by the terminal 4 to the image processing device 10 via the communication line 2, thereby causing the image processing device 10 to perform the desired image processing. Alternatively, the user may store image data in a portable storage medium such as a USB (Universal Serial Bus) memory or a memory card, carry the image data to the image processing device 10, and connect the portable storage medium to the image processing device 10 to cause the image processing device 10 to perform the desired image processing. Furthermore, the user may carry an original 11 containing at least one of text and images to the image processing device 10, and have the image processing device 10 read the original 11 to cause the image processing device 10 to perform the desired image processing.

[0038] There are no restrictions on the connection form of the communication line 2 connecting the image processing device 10 and the terminal 4, and it may be wired, wireless, or a combination of wired and wireless. There is also no restriction on the number of terminals 4 connected to the image processing device 10, and for example, no terminal 4 may be connected to the image processing device 10.

[0039] The terminal 4 refers to an information device used by a user. The terminal 4 may be any type of information device as long as it has a data storage function and a data communication function. The terminal 4 includes, for example, a computer that is not intended to be used while being carried around, as well as a mobile terminal that is intended to be used while being carried around, such as a smartphone or a wearable device.

[0040] 2, the image processing device 10 has, as an example, a scanning function for reading an image recorded on a recording medium such as paper as image data, a printing function for forming an image represented by the image data on a recording medium, and a copying function for forming an image identical to the image formed on the recording medium on another recording medium. The copying function, printing function, and scanning function are examples of image processing in the image processing device 10.

[0041] An original reading unit 12, for example, is provided at the top of the image processing device 10 shown in FIG. 2, and an image forming unit 14 is disposed below the original reading unit 12.

[0042] The document reading unit 12 is configured to include an optical reading device (not shown) and a document transport device 18 inside the document cover 16. The document transport device 18 sequentially pulls in the documents 11 placed on a document table 16A provided in the document cover 16 and transports them onto a document transport reading glass (not shown). The document reading unit 12 reads the contents of the documents 11 transported onto the document transport reading glass as image data using the optical reading device. After that, the document transport device 18 ejects the documents 11, whose contents have been read, onto an ejection table 16B provided in the document cover 16.

[0043] On the other hand, the image forming unit 14 forms an image represented by image data on a recording medium stored in a storage shelf 19, which is classified by type and size of the recording medium. There is no limitation on the color of the image formed on the recording medium by the image forming unit 14, and the image may be a color image or a monochrome image.

[0044] An operation display unit 13 is provided on the front surface of the image processing device 10 to receive operations from the user for executing various functions such as a copy function, a print function, and a scan function.

[0045] Specifically, the operation display unit 13 includes a reader device 17 that acquires information about the user who performs the operation, and an operation panel 15 that accepts the user's operation.

[0046] The reader device 17 is a device that, when an employee ID card held by a user is brought close to the reader device 17, reads, in a non-contact manner, identification information (referred to as a "user ID") that uniquely identifies the user from an IC chip embedded in the employee ID card.

[0047] The operation panel 15 is a display on which a touch panel is superimposed, and objects to be operated by the user to execute a desired function are displayed as icon images on the operation panel 15. There are no restrictions on the type of object as long as it is an object that can be operated by the user, and examples of objects include buttons, scroll bars, check boxes, and radio buttons. When the user operates an object, a process previously associated with the operation content is executed on the image processing device 10, and a response to the operation is displayed on the operation panel 15.

[0048] 3A and 3B are diagrams showing an example of an operation panel 15 that detects a user's operation position 6 in a non-contact manner. Fig. 3A is a cross-sectional view of the operation panel 15, and Fig. 3B is a plan view of the operation panel 15 as viewed from a position opposite the display surface of the operation panel 15.

[0049] Operation panel 15 detects the position of the user's finger, i.e., operation position 6, without contact. "Detecting operation position 6 without contact" means that the position of the user's finger can be detected by simply holding a finger over a location away from the display surface of operation panel 15 within the space above the display surface of operation panel 15 within the range of the display surface of operation panel 15, without the user having to press the finger against the display surface of operation panel 15. Hereinafter, the space above the display surface of operation panel 15 within the range of the display surface of operation panel 15 will be referred to as "above operation panel 15." Furthermore, when a user "holds a finger over" operation panel 15, it means that the user points at operation panel 15 without touching the display surface of operation panel 15.

[0050] Operation panel 15 is equipped with a so-called capacitive touch panel that detects operation position 6 from a change in capacitance that occurs when a user places a finger over operation panel 15. In operation panel 15 equipped with such a touch panel, the change in capacitance at the point closest to the user's finger is larger than the change in capacitance at other points. Therefore, operation panel 15 outputs the point within the range of operation panel 15 with the largest change in capacitance as the user's operation position 6.

[0051] In order to identify the user's operation position 6 on the operation panel 15, an operation coordinate system is defined on the operation panel 15 to define a detection area for detecting the position of the user's finger without contact. The operation coordinate system is expressed as a three-dimensional coordinate system with an origin P at a certain point on the operation panel 15. In the example of the operation panel 15 shown in FIGS. 3(A) and 3(B), the origin P is set at one of the vertices on the outline of the rectangular operation panel 15. Furthermore, in the example of the operation panel 15 shown in FIGS. 3(A) and 3(B), with respect to the origin P, an X-axis is set along the horizontal direction of the operation panel 15, a Y-axis is set along the vertical direction of the operation panel 15, and a Z-axis is set so as to be perpendicular to the X-axis and Y-axis. The Z-axis direction is referred to as the height direction of the operation panel 15.

[0052] The user's operation position 6 on the operation panel 15 is represented by a coordinate point (x, y) which is a combination of the coordinate value x of the X coordinate and the coordinate value y of the Y coordinate at the point within the range of the operation panel 15 where the change in capacitance is greatest.

[0053] When objects are displayed on operation panel 15, the object displayed so as to include user operation position 6 is recognized as the object being operated by the user. In the example of operation panel 15 shown in FIG. 3(B), the user operation position 6 is included within the area of ​​button 8 arranged on screen 30 displayed on operation panel 15, so it is recognized that the user is operating button 8. Hereinafter, the object displayed so as to include user operation position 6 may be referred to as the "object corresponding to operation position 6." Furthermore, operation position 6 is an example of the "detection position where a user operation is detected" according to this embodiment.

[0054] As shown in FIG. 3A, the length of a perpendicular line drawn from a user's finger 3 held over the operation panel 15 to the display surface of the operation panel 15, i.e., the distance from the user's finger 3 to the operation panel 15 in the height direction of the operation panel 15, is defined as the "operation distance D." The user's finger 3 is an example of an object held by the user. This object may be a part of the user's hand or finger, or may be a touch pen or the like owned by the user. On the operation panel 15, as the operation distance D becomes shorter, the change in capacitance at the user's operation position 6 becomes larger. Conversely, on the operation panel 15, as the operation distance D becomes longer, the change in capacitance at the user's operation position 6 becomes smaller. Therefore, by previously associating the relationship between the operation distance D and the amount of change in capacitance, the operation distance D can be obtained from the amount of change in capacitance on the operation panel 15.

[0055] By using the correspondence relationship between the operation distance D and the amount of change in capacitance, the operation panel 15 can recognize the user's operation position 6 not only as a two-dimensional operation position 6 along the display surface of the operation panel 15, but also as a three-dimensional operation position 6 taking the operation distance D into consideration. That is, when the user's operation position 6 is expressed three-dimensionally, the user's operation position 6 is expressed by a coordinate point (x, y, z) obtained by combining a coordinate value z representing the operation position 6 in the height direction of the operation panel 15 with the coordinate point (x, y). The coordinate value z is a coordinate value on the Z axis at a position that is the operation distance D away from the origin P along the Z axis.

[0056] Note that when the coordinate value z=0, it means that the user is operating by touching the display surface of the operation panel 15 with his / her finger. Therefore, the image processing device 10 can also recognize differences in the user's operation style, such as whether the user is operating the operation panel 15 in a non-touch manner or by touching the operation panel 15 with his / her finger. In this way, the operation panel 15 supports both a touch operation in which the user operates by touching the display surface of the operation panel 15 with his / her finger, and a touchless operation in which the user operates by holding his / her finger over the operation panel 15.

[0057] As described above, in operation panel 15, the change in capacitance at the user's operation position 6 decreases as the operation distance D increases, and therefore there is an upper limit to operation distance D. Even if the user holds his / her finger over operation panel 15 at a position beyond the upper limit of operation distance D, the capacitance at the user's operation position 6 does not change, and operation panel 15 does not respond to the user's operation.

[0058] The detection area corresponding to the object is an area in space, for example, about 3 cm away from the operation panel 15. In other words, when the user's finger 3 is brought within about 3 cm of the operation panel 15, the capacitance on the object side changes, and a non-contact input is detected. At this time, the XYZ coordinates of the user's finger 3 in the detection area are acquired as the operation position 6. Then, when the user's finger 3 is brought closer than 3 cm, the XYZ coordinates at that position are acquired.

[0059] 4 is a diagram showing an example of the functional configuration of the image processing device 10 according to this embodiment. The image processing device 10 includes functional units, namely, a control unit 20, a reception unit 21, a display unit 22, a document reading unit 23, and an image forming unit 24.

[0060] The reception unit 21 receives the user ID of the user operating the image processing device 10 from the reader device 17 of the operation display unit 13, and also receives the user's operation position 6 on the operation panel 15 from the operation panel 15 of the operation display unit 13. The reception unit 21 also receives image data from the terminal 4 or a portable storage medium connected to the image processing device 10. The reception unit 21 notifies the control unit 20 of the received user ID, user's operation position 6, and image data.

[0061] When the control unit 20 is notified of a user ID by the reception unit 21, the control unit 20 performs authentication processing to determine whether the user represented by the user ID is a user (referred to as a "registered user") who is authorized to use the image processing device 10. When the control unit 20 is notified of the user's operation position 6 on the operation panel 15 by the reception unit 21, the control unit 20 determines whether an object displayed at the user's operation position 6 on the screen 30 displayed on the operation panel 15 has been selected, and executes processing previously associated with the selected object. For example, if the object is a button 8 that starts a print function, the control unit 20 starts the print function to form an image represented by the image data received by the reception unit 21 on a recording medium.

[0062] Since the image processing device 10 has a copy function, a print function, and a scan function, the control unit 20 includes a scan control unit 20A that controls the scan function, a print control unit 20B that controls the print function, and a copy control unit 20C that controls the copy function. Depending on the content of the process associated with the object operated by the user, control is performed by one of the scan control unit 20A, the print control unit 20B, and the copy control unit 20C. Although not shown, the image processing device 10 may also have a facsimile function, in which case the control unit 20 includes a facsimile control unit that controls the facsimile function.

[0063] If the user's operation via the object is an operation related to the scan function, the scan control unit 20A executes the scan function by controlling the document reading unit 23. If the user's operation via the object is an operation related to the print function, the print control unit 20B executes the print function by controlling the image forming unit 24. If the user's operation via the object is an operation related to the copy function, the copy control unit 20C generates image data of the document 11 by controlling the document reading unit 23. Thereafter, the copy control unit 20C forms an image represented by the generated image data on a recording medium by controlling the image forming unit 24.

[0064] The document reading section 23 drives the document reading unit 12 under the control of the scan control section 20A and the copy control section 20C, and, for example, transports the document 11 placed on the document table 16A and generates image data of the transported document 11.

[0065] The image forming section 24 drives the image forming unit 14 in accordance with the control of the print control section 20B and the copy control section 20C, and performs, for example, transporting recording media stored in the storage shelf 19 and forming an image represented by image data on the transported recording media.

[0066] In accordance with instructions from the control unit 20, the display unit 22 displays, for example, the results of authentication processing for the user and the results of processing performed by the control unit 20 in response to user operations via objects on the operation panel 15 in the operation display unit 13.

[0067] FIG. 5 is a diagram showing an example of transition of screen 30, showing how screen 30 displayed on operation panel 15 changes in response to an operation on operation panel 15 by the user.

[0068] Note that display of screen 30 on operation panel 15 is performed by display unit 22, but since display unit 22 displays screen 30 in accordance with instructions from control unit 20, it can also be said that control unit 20 displays screen 30 on operation panel 15. Furthermore, the space along the Z axis with the display range of screen 30 displayed on operation panel 15 as the bottom is referred to as "on screen 30," and the space along the Z axis with the display range of an object displayed on screen 30 as the bottom is referred to as "on the object." Like the notation "on operation panel 15," the terms "on screen 30" and "on the object" do not mean the upper side based on the up, down, left, and right directions in real space, but rather refer to the space in the direction relative to screen 30 and the space in the direction relative to the object, respectively.

[0069] For ease of explanation, when different types of screens 30 are described separately, an alphabetical symbol corresponding to each type of screen 30 is added after "screen 30." When it is not necessary to describe different types of screens 30 separately, each type of screen 30 is collectively referred to as "screen 30." Furthermore, when different types of buttons 8, which are an example of objects, are described separately, an alphabetical symbol corresponding to each type of button 8 is added after "button 8." When it is not necessary to describe different types of button 8 separately, each type of button 8 is collectively referred to as "button 8."

[0070] If the control unit 20 determines through authentication processing that the user performing the operation is a registered user, it displays a start screen 30A on the operation panel 15. The start screen 30A displays instructions to the user, such as "Hold your hand over the screen to start Touch Less!"

[0071] When the user holds his / her finger over the start screen 30A, a cursor is displayed at the user's operation position 6 on the start screen 30A. In the example of the start screen 30A shown in FIG. 5, a hand-shaped cursor is displayed, but the shape of the cursor is merely an example, and a circular cursor may be displayed, for example. When the user holds his / her finger over the start screen 30A, the home screen 30B is displayed. Note that the instructions to the user on the start screen 30A also serve to teach the user how to operate the operation panel 15.

[0072] The home screen 30B displays, for example, buttons 8 for selecting various functions of the image processing device 10 and a navigation bar 9 that displays information useful for the user when performing operations. Because the image processing device 10 has a copy function, a print function, and a scan function, the home screen 30B displays a copy button 8A for selecting the copy function, a print button 8B for selecting the print function, and a scan button 8C for selecting the scan function. The navigation bar 9 displays, for example, the name of the authenticated user such as "User A," the name of the screen displayed on the operation panel 15 such as "Home," and information such as "Touch Less" that notifies the user that the operation panel 15 is in a non-touch operation mode.

[0073] When the user places his / her finger over the copy button 8A and performs a predetermined selection operation, the copy button 8A is selected. When the copy button 8A is selected, a copy screen 30D is displayed on the operation panel 15. On the copy screen 30D, buttons 8D to 8G for setting copy conditions and a copy start button 8H for starting copying according to the set copy conditions are displayed. The selection operation performed by the user to select an object will be described in detail later.

[0074] The copy screen 30D shown in FIG. 5 displays examples of buttons 8 for setting copy conditions, such as a color mode button 8D for selecting the color of the copy, a double-sided / single-sided selection button 8E for selecting the copy side, an N-up button 8F for selecting the method of allocating images to the recording medium, and a number of copies button 8G for selecting the number of copies to make.

[0075] When the user places his / her finger over any of buttons 8D to 8G for setting copy conditions and performs a selection operation, the button 8 corresponding to the user's operation position 6 is selected, and a screen 30 for setting the copy conditions corresponding to the selected button 8 is displayed. When double-sided / single-sided selection button 8E is selected on copy screen 30D, double-sided / single-sided selection screen 30G for selecting the copy side is displayed on operation panel 15 superimposed on copy screen 30D.

[0076] 5 displays, for example, a double-sided → double-sided selection button 8S for sequentially copying the contents of both sides of the document 11 onto both sides of a recording medium, a single-sided → double-sided selection button 8T for sequentially copying the contents of the document 11, which has text or the like written on only one side, onto both sides of a recording medium, and a single-sided → single-sided selection button 8U for sequentially copying the contents of the document 11, which has text or the like written on only one side, onto one side of a recording medium. When a user places their finger over any of the buttons 8S to 8U on the double-sided / single-sided selection screen 30G to perform a selection operation, the button 8 corresponding to the user's operation position 6 is selected, and the copy side corresponding to the selected button 8 is set. The example of the double-sided / single-sided selection screen 30G shown in FIG. 5 illustrates a state in which the double-sided → double-sided selection button 8S has been selected by the user.

[0077] When the copy side is set on double-sided / single-sided selection screen 30G, copy screen 30D is displayed on operation panel 15. After the copy side is set, the copy side selected on double-sided / single-sided selection screen 30G is displayed within double-sided / single-sided selection button 8E on copy screen 30D.

[0078] The above describes an example in which the user selects the double-sided / single-sided selection button 8E on the copy screen 30D. Even if the user selects any of the color mode button 8D, N-up button 8F, and number of copies button 8G on the copy screen 30D, a selection screen for selecting the copy conditions corresponding to each button 8, such as the double-sided / single-sided selection screen 30G, is displayed on the operation panel 15.

[0079] When the user places his / her finger over the copy start button 8H on the copy screen 30D and performs a selection operation, the copy start button 8H is selected. When the copy start button 8H is selected, a copy process is executed in which the contents of the document 11 are copied to a recording medium in accordance with the set copy conditions. Note that before the copy conditions are set, the buttons 8D to 8G on the copy screen 30D display the initial settings of the copy conditions that have been set in advance.

[0080] On the other hand, when the user places his / her finger over the print button 8B on the home screen 30B and performs a selection operation, the print button 8B is selected. When the print button 8B is selected, the print screen 30E is displayed on the operation panel 15.

[0081] Print screen 30E displays print information buttons 8J that display information about the image data to be printed, and an all print start button 8M that starts printing all of the image data corresponding to each print information button 8J. The example of print screen 30E shown in Figure 5 represents print screen 30E when two sets of image data to be printed are accepted. In other words, print screen 30E displays print information buttons 8J corresponding to each set of image data, the number of which is equal to the number of image data accepted from the user to be printed.

[0082] If there is more image data than can fit on the print screen 30E at one time, when the user makes a gesture of moving their finger up or down on the print information button 8J, the operation panel 15 detects the movement of the operation position 6 and scrolls the print information buttons 8J, so that the print information buttons 8J that cannot be displayed on the print screen 30E are also displayed on the print screen 30E.

[0083] Each print information button 8J displays the file name of the image data to be printed and the print conditions that the user has set in advance for the image data. For example, when the user transmits image data from the terminal 4 to the image processing device 10, the print conditions that the user has set from the terminal 4 are displayed on the print information button 8J.

[0084] When the user places his / her finger over the all print start button 8M and performs a selection operation, the all print start button 8M is selected. When the all print start button 8M is selected, a print process is executed in which the images represented by the image data are printed on a recording medium according to the set print conditions.

[0085] Furthermore, when the user places his / her finger over one of the print information buttons 8J and performs a selection operation, the print information button 8J is selected. When the print information button 8J is selected, a print edit screen 30H is displayed on the operation panel 15. The print edit screen 30H shown in Fig. 5 represents an example of the display when the user selects the print information button 8J corresponding to the image data of "Document B.pdf."

[0086] The print edit screen 30H displays, for example, a delete button 8V for deleting image data corresponding to a selected print information button 8J, a change button 8W for changing the print conditions of the image data corresponding to the selected print information button 8J, and an individual print start button 8X for printing only the image data corresponding to the selected print information button 8J. The print edit screen 30H shown in Figure 5 displays, as an example of the change button 8W, a change button 8W for changing the number of copies to be printed. Note that change buttons 8W for changing other print conditions, such as the color of the image to be printed, are also displayed on the print edit screen 30H, but are not shown in the figure.

[0087] On the other hand, when the user places his / her finger over the scan button 8C on the home screen 30B and performs a selection operation, the scan button 8C is selected. When the scan button 8C is selected, a scan screen 30F is displayed on the operation panel 15.

[0088] The scan screen 30F displays a scan setting button 8N for setting scan conditions and a scan start button 8R for starting reading of the document 11 under the set scan conditions.

[0089] When the user places their finger over the scan setting button 8N and performs a selection operation, the scan setting button 8N corresponding to the user's operation position 6 is selected, and a selection screen (not shown) for selecting the scan conditions corresponding to the selected scan setting button 8N is displayed. That is, the user sets the scan conditions associated with the scan setting button 8N in the same manner as setting the copy conditions from the copy screen 30D. When the user places their finger over the scan start button 8R and performs a selection operation, the scan start button 8R is selected. When the scan start button 8R is selected, a scan process is executed to convert the written contents of the document 11 into image data according to the set scan conditions.

[0090] On the other hand, on home screen 30B, when the user holds his / her finger over navigation bar 9 and performs a selection operation, navigation bar 9 is selected. When navigation bar 9 is selected, a logout process for the authenticated user is performed, and a display indicating that logout has been completed is displayed on navigation bar 9, as shown on screen 30C.

[0091] FIG. 6 is a diagram showing an example of a selection in which a user selects a copy button 8A, which is an example of an operation object, on the home screen 30B. When the user holds his / her finger over the copy button 8A, an operation position 6 is detected within the display area of ​​the copy button 8A. When the user continues to hold his / her finger over the copy button 8A and the detected operation position 6 remains within the display area of ​​the copy button 8A for a predetermined time, the copy button 8A is selected as shown in FIG. 6, and a copy screen 30D is displayed on the operation panel 15. Each operation object on the screen 30 is associated in advance with a process to be executed when the operation object is selected, so that a copy process is executed when the copy start button 8H is selected. The operation objects are displayed on the screen 30 in association with items representing the content of the process. Therefore, each operation object is an example of an "item displayed on the screen" according to this embodiment.

[0092] Next, the main configuration of the electrical system of the image processing device 10 will be described with reference to Fig. 7. The image processing device 10 is configured using a computer 40, for example.

[0093] In the computer 40, a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, a ROM (Read Only Memory) 43, a non-volatile memory 44, and an input / output interface (I / O) 45 are connected to each other via a bus 46.

[0094] The CPU 41 is an example of a processor that performs processing for each functional unit of the image processing device 10 shown in FIG. 4. The RAM 42 is an example of a storage medium used as a temporary work area for the CPU 41. The ROM 43 is an example of a storage medium that stores an information processing program executed by the CPU 41. The nonvolatile memory 44 is an example of a storage medium that maintains stored information even when power supplied to the nonvolatile memory 44 is cut off, and may be, for example, a semiconductor memory or a hard disk. The nonvolatile memory 44 does not necessarily need to be built into the computer 40, but may be a storage medium that is detachable from the computer 40, such as a memory card. The I / O 45 is connected to, for example, the document reading unit 12, the image forming unit 14, the input unit 31, the display unit 32, and the communication unit 33.

[0095] The document reading unit 12 and the image forming unit 14 are devices that perform the operations already described. The input unit 31 is a device that receives user instructions and a user ID and notifies the CPU 41, and the touch panel and reader device 17 that constitute the operation panel 15 are examples of the input unit 31. The display unit 32 is a device that visually displays information processed by the CPU 41, and the display that constitutes the operation panel 15 is an example of the display unit 32. The communication unit 33 is connected to the communication line 2 and has a communication protocol for communicating with the terminal 4. Note that the units connected to the I / O 45 are not limited to the units exemplified in FIG. 7. Units required to realize functions of the image processing device 10 are connected to the I / O 45 depending on the functions the image processing device 10 has.

[0096] Next, a description will be given of a predetermined selection operation for selecting an object in the image processing device 10 according to this embodiment. Examples of the object selection operation in the image processing device 10 include an operation in which a user selects an object by holding a finger over the object for a certain period of time, and an operation in which a user selects an object by moving a finger toward or away from the object. In this embodiment, an operation in which a user selects an object by holding a finger over the object for a certain period of time is adopted as an example.

[0097] The object selection operation will be described below using the copy screen 50 shown in FIG. 8 as an example.

[0098] 8 displays, as examples of objects, a color mode button 8D, a double-sided / single-sided selection button 8E, a paper selection button 52, an N-up button 8F, a number of copies button 8G, a magnification button 54, and a copy start button 8H. Note that the copy screen 50 is an example in which the magnification button 54 and the paper selection button 52 are added to the copy screen 30D shown in FIG. 5, and the other configurations are the same as those of the copy screen 30D. Therefore, the paper selection button 52 and the magnification button 54 will be described here.

[0099] The paper selection button 52 is a button that has the function of selecting the paper to print on. When the paper selection button 52 is selected, the screen transitions to a screen (not shown) for setting the paper to print on (paper size).

[0100] The magnification button 54 is a button that has the function of selecting the print magnification of the print target. When the magnification button 54 is selected, the screen transitions to a screen for setting the print magnification (see FIG. 9).

[0101] Here, when the magnification button 54 is selected on the copy screen 50, a print magnification selection screen 50A for selecting a magnification is displayed on the operation panel 15 in a form superimposed on the copy screen 30D.

[0102] 9 displays a window 54A that displays the print magnification, an up button 54B for increasing the print magnification, and a down button 54C for decreasing the print magnification. The up button 54B and the down button 54C are examples of objects.

[0103] When the user places their finger over either the Up button 54B or the Down button 54C to perform a selection operation, the button corresponding to the user's operation position 6 is selected, and the process corresponding to the selected button is executed. Here, when the Up button 54B is selected, the print magnification increases. On the other hand, when the Down button 54C is selected, the print magnification decreases. The increase or decrease in print magnification is displayed in the magnification display window 54A.

[0104] Next, an example of contactless operation of the image processing device 10 according to this embodiment will be described with reference to FIGS.

[0105] In this embodiment, as shown in FIG. 9 , when a user performs a non-contact operation with their finger 3 on an object in the print magnification selection screen 50A, if the CPU 41 detects the user's finger 3 in a detection area included in the display area where the object is displayed in the print magnification selection screen 50A, the CPU 41 expands the detection area beyond the display area (see FIG. 10 ). Specifically, when the user's finger 3 is on an object in the print magnification selection screen 50A, in other words, when the object is at operation position 6, the CPU 41 expands the detection area DR of the object in the print magnification selection screen 50A beyond the display area of ​​the object. More specifically, when the user's finger 3 is on the Up button 54B in the print magnification selection screen 50A, in other words, when the Up button 54B is at operation position 6, the CPU 41 expands the detection area DR of the Up button 54B in the copy screen 50 beyond the display area of ​​the Up button 54B.

[0106] In this embodiment, the display area of ​​the object includes the detection area DR, but the display area of ​​the object and the detection area DR may be the same.

[0107] Here, the CPU 41 may expand the detection region DR downward relative to the display region of the object, as shown in Fig. 10. Specifically, the CPU 41 expands the detection region DR downward relative to the display region of the up button 54B. More specifically, the CPU 41 may expand the detection region DR so that it widens toward the end relative to the display region of the up button 54B. That is, the CPU 41 may expand the width of the detection region DR downward. In this embodiment, as an example, the detection region DR is expanded into a trapezoidal shape.

[0108] Furthermore, as shown in FIG. 9 , the CPU 41 may change at least one of the color and shape of an object when the user's finger 3 is detected on the detection area DR for a predetermined time. Specifically, the CPU 41 may change the color of the up button 54B when the user's finger 3 is detected on the detection area DR of the up button 54B for a predetermined time. In the present embodiment, as an example, the color of the up button 54B is changed. Note that the present disclosure is not limited to this configuration. The CPU 41 may change the shape of an object when the user's finger 3 is detected on the detection area DR for a predetermined time. Specifically, the outer shape of the object may be changed. Furthermore, the frame indicating the outer shape of the object may be a multiple frame. Furthermore, the CPU 41 may change the shape of the object while changing the color of the object.

[0109] Furthermore, the CPU 41 may confirm an operation on an object when the user's finger 3 is detected on the detection area DR for a predetermined time. Specifically, when the CPU 41 detects the user's finger 3 on the detection area DR of the up button 54B for a predetermined time, the CPU 41 may select the up button 54B and confirm the operation of the selected up button 54B. When the operation of the up button 54B is confirmed, a process of increasing the print magnification is executed.

[0110] 11, the CPU 41 may expand the detection area DR according to the detection time of the user's finger 3. In other words, the CPU 41 may gradually expand the detection area DR according to the detection time of the user's finger 3. In the present embodiment, as an example, the detection area DR gradually expands downward. Note that the detection area DR may expand downward in a similar shape, or in a shape other than the similar shape. The detection time may be set to, for example, 1 second.

[0111] 12, when multiple objects are displayed on the copy screen 50, the CPU 41 stops accepting operations on objects other than the object being operated. Specifically, when the up button 54B is selected, the CPU 41 does not accept a selection operation on the down button 54C even if the user's finger 3 is on the down button 54C.

[0112] 13, when the user's finger 3 moves out of the detection area DR of the object being operated, the CPU 41 stops operating the object. Specifically, when the user's finger 3 moves out of the detection area DR of the up button 54B being operated, the CPU 41 stops the process of increasing the print magnification. Similarly, when the user's finger 3 moves too far away from the operation panel 15, the user's finger 3 is no longer detected, and the process of increasing the print magnification is stopped.

[0113] Next, the operation of the image processing device 10 of this embodiment will be described with reference to FIG.

[0114] FIG. 14 is a flowchart showing an example of the flow of processing by the information processing program according to this embodiment.

[0115] First, when a command to execute non-contact input is given on the operation panel 15, the CPU 41 starts up an information processing program and executes the following steps. The following describes the flow of processing for selecting an object on the copy screen 50 and executing the assigned process.

[0116] In step S200, the CPU 41 determines whether or not the current finger coordinates (air coordinates) of the user's finger 3, which is the object for performing the non-touch operation, can be detected. If the user's finger 3 cannot be detected, this process ends. If the finger coordinates of the user's finger 3 are detected, the process proceeds to step S202.

[0117] In step S202, the CPU 41 determines whether or not the user's finger 3 is on the object. If the user's finger 3 is on the object, the process proceeds to step S204. On the other hand, if the user's finger 3 is not on the object, the CPU 41 determines that the object is being selected, and returns to step S200.

[0118] In step S204, the CPU 41 changes the color of the object on which the user's finger 3 is located (see, for example, FIG. 9). Note that the shape of the object on which the user's finger 3 is located may be changed, or both the color and the shape of the object may be changed.

[0119] In step S206, the CPU 41 expands the detection region DR of the selected object (see, for example, FIGS. 10 and 11). Note that the detection region DR of the object may be expanded gradually or in one go.

[0120] In step S208, the CPU 41 determines whether or not the user's finger 3 is within the detection region DR of the selected object. If the user's finger 3 is within the detection region DR, the process proceeds to step S210. On the other hand, if the user's finger 3 is not within the detection region DR, that is, if the user's finger 3 is outside the detection region, the process ends.

[0121] In step S210, the CPU 41 confirms the operation of the object. Specifically, as shown in FIGS. 10 and 11, when the operation of the up button 54B is confirmed, the print magnification display increases. After the print magnification has been increased, this process ends. Note that the print magnification can be increased or decreased by repeating this process.

[0122] In this embodiment, when a non-contact operation is performed with the user's finger 3 on an object on a screen (for example, copy screen 50), if the CPU 41 detects the user's finger 3 on the detection area DR, the detection area DR is expanded to outside the display area of ​​the object. Therefore, the detection state (selection state) of the user's finger 3 can be maintained compared to when the detection area DR remains within the display area when the user's finger 3 is detected on the detection area DR of the object operated non-contact. That is, as shown in FIG. 11 , by expanding the detection area DR, the detection state of the user's finger 3 can be maintained even if the user's finger 3 moves out of the initial detection area.

[0123] In this embodiment, when the CPU 41 expands the detection area DR in accordance with the detection time of the user's finger 3, the detection state of the user's finger 3 can be continued compared to when the size of the detection area DR remains constant after it has been expanded once. In other words, expanding the detection area DR makes it less likely that the user's finger 3 will fall out of the detection area DR.

[0124] In this embodiment, when the CPU 41 expands the detection area DR downward relative to the display area of ​​the object, the detection state of the user's finger 3 can be continued even if the position of the user's finger 3 shifts downward due to its own weight, compared to when the detection area DR is expanded upward relative to the display area.

[0125] In this embodiment, when the CPU 41 expands the detection area DR so that it widens at the ends, the detection state of the user's finger 3 can be continued even if the position of the user's finger 3 shifts diagonally downward due to its own weight, compared to when the detection area DR is expanded downward by a constant width.

[0126] In this embodiment, when the CPU 41 detects the user's finger 3 on the detection area DR for a predetermined time and confirms an operation on an object, erroneous operations by the user can be reduced compared to when the operation on an object is confirmed when the user's finger 3 is detected on the detection area DR.

[0127] In this embodiment, when the CPU 41 stops accepting operations on objects other than the object being operated when multiple objects are displayed on the screen, it is possible to reduce user erroneous operations compared to when operations on objects other than the object being operated are accepted.

[0128] In this embodiment, when the CPU 41 stops the operation when the user's finger 3 moves outside the detection area DR of the object being operated, it is possible to reduce erroneous operations by the user compared to when the operation is continued even if the user's finger 3 moves outside the detection area DR of the object being operated.

[0129] In this embodiment, when the CPU 41 changes at least one of the color and shape of the object when the user's finger 3 is detected on the detection area DR for a predetermined time, the user can more easily recognize the object detected by the CPU 41 compared to when the display mode of the object is maintained even when the user's finger 3 is detected on the detection area DR for a predetermined time.

[0130] (Other embodiments) In the above-described embodiment, when the user's finger 3 is detected in the object detection area DR for a predetermined time, the CPU 41 selects the target object and executes the process set for the object. However, the present disclosure is not limited to this configuration. For example, the CPU 41 may switch between selecting an object and confirming (executing the process) the operation assigned to the selected object depending on the distance of the user's finger 3 from the operation panel 15. Specifically, as shown in FIG. 15 , the distance from the user's finger 3 to the operation panel 15 (i.e., the operation distance D) is divided into two areas. Of the two areas, the area located closer to the user's finger 3 is designated as a first area R1, and the area located closer to the operation panel 15 than the first area R1 is designated as a second area R2. The first area R1 and the second area R2 are separated by a threshold Th. The threshold Th is a threshold value for the distance from the operation panel 15 to the user's finger 3, and may be set to an appropriate value within a range that does not exceed the distance at which the user's finger 3 can be detected. By associating the first region R1 with an object selection operation and the second region R2 with a confirmation operation assigned to the selected object, the operation can be switched between selecting an object and confirming the operation assigned to the selected object depending on the distance of the user's finger 3 from the operation panel 15. Taking the print magnification as an example, when the user's finger 3 is in the first region R1, the user can select either the Up button 54B or the Down button 54C. By pressing the user's finger 3 on the Up button 54B or the Down button 54C, i.e., toward the operation panel 15, the operation of the selected button is confirmed and the print magnification increases or decreases. In this way, when the CPU 41 switches between selecting an object and confirming the operation assigned to the selected object depending on the distance of the user's finger 3 from the operation panel 15, the user's erroneous operation can be reduced compared to when the CPU 41 switches between selecting an object and confirming the operation depending on the detection time of the user's finger 3.

[0131] In the above-described embodiment, the CPU 41 extends the detection region DR downward relative to the display area of ​​the object. However, the present disclosure is not limited to this configuration. For example, the CPU 41 may extend the detection region DR upward, leftward, or rightward relative to the display area of ​​the object. Furthermore, the CPU 41 may extend the detection region DR upward, downward, leftward, or rightward relative to the display area of ​​the object in a combination of two or more directions. The CPU 41 may extend the entire detection region DR relative to the display area of ​​the object so that it extends beyond the display area of ​​the object. Taking the above-described print magnification as an example, for example, as shown in FIG. 16 , the CPU 41 may extend the entire detection region DR relative to the display area of ​​the up button 54B. When the CPU 41 extends the entire detection region DR relative to the display area of ​​the up button 54B in this manner, the detection state of the user's finger 3 can be maintained even if the position of the user's finger 3 shifts outside the display area, compared to when the CPU 41 extends only a portion of the detection region DR beyond the display area.

[0132] In the above-described embodiment, the CPU 41 expands the detection area DR depending on the detection time of the user's finger 3, but the present disclosure is not limited to this configuration. For example, the CPU 41 may change the shape of the detection area DR depending on the distance from the screen. Specifically, the detection area DR may gradually expand as the user's finger 3 approaches the operation panel 15. Alternatively, as shown in the example of FIG. 15 , the detection area DR may expand when the user's finger 3 moves from the first area R1 to the second area R2. In this case, it is easier to maintain the detection state of the user's finger 3 compared to when the shape of the detection area DR remains constant regardless of the distance of the user's finger 3 from the operation panel 15.

[0133] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0134] Furthermore, the operations of the processor in each of the above embodiments may not only be performed by a single processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

[0135] The above describes an example of an image processing device as an information processing device according to an embodiment. The embodiment may be in the form of a program for causing a computer to execute the functions of the information processing device. The embodiment may be in the form of a non-transitory storage medium that stores the program and is readable by a computer.

[0136] Furthermore, the configuration of the information processing device described in the above embodiment is merely an example, and may be changed depending on the situation without departing from the spirit of the invention.

[0137] Furthermore, the processing flow of the program described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged within the scope of the main idea.

[0138] In the above embodiment, the processing according to the embodiment is realized by a software configuration using a computer by executing a program, but the present invention is not limited to this. The embodiment may be realized by, for example, a hardware configuration or a combination of a hardware configuration and a software configuration.

[0139] The following is further disclosed regarding the above embodiment.

[0140] (((1))) a processor; The processor: When a non-contact operation is performed on an object on a screen by an object, if the object is detected in a detection area included in a display area in which the object is displayed on the screen, the detection area is expanded to outside the display area. Information processing system.

[0141] (((2))) The processor expands the detection area according to the detection time of the object. The information processing system according to (((1))).

[0142] (((3))) The processor extends the sensing area downward relative to the display area. The information processing system according to (((1))) or (((2))).

[0143] (((4))) The processor expands the detection area so that it diverges. The information processing system according to (((3))).

[0144] (((5))) The processor extends the entire detection area to outside the display area. The information processing system according to any one of (((1))) to (((4))).

[0145] (((6))) The processor changes the shape of the detection area according to the distance of the object from the screen. The information processing system according to any one of (((1))) to (((5))).

[0146] (((7))) The processor determines an operation on the object when the object is detected on the detection area for a predetermined time. The information processing system according to any one of (((1))) to (((6))).

[0147] (((8))) When a plurality of objects are displayed on the screen, the processor stops accepting operations on objects other than the object being operated. The information processing system according to (((7))).

[0148] (((9))) The processor stops the operation when the object moves out of the detection area of ​​the object being operated. The information processing system according to (((7))) or (((8))).

[0149] (((10))) the processor changes at least one of a color and a shape of the object when the object is detected on the detection area for a predetermined time. The information processing system according to (((7))) or (((8))).

[0150] (((11))) The processor switches between selecting the object and confirming the operation assigned to the selected object depending on the distance of the object from the screen. The information processing system according to any one of (((1))) to (((6))).

[0151] (((12))) When a non-contact operation is performed on an object on a screen by an object, if the object is detected in a detection area included in a display area in which the object is displayed on the screen, the detection area is expanded to outside the display area. An information processing program to be executed by a computer.

[0152] According to the information processing system (((1))), when an object is detected in a detection area included in the display area of ​​an object operated non-contact, the detection state of the object can be continued compared to when the detection area remains within the display area.

[0153] According to the information processing system (((2))), once the size of the detection area is expanded, it is possible to continue detecting the object, compared to when the size remains constant.

[0154] According to the information processing system of (((3))), the detection state of the object can be continued even if the position of the object shifts downward due to its own weight, compared to when the detection area is expanded upward relative to the display area.

[0155] According to the information processing system (((4))), the detection state of the object can be continued even if the position of the object shifts diagonally downward due to its own weight, compared to when the detection area is expanded downward by a fixed width relative to the display area.

[0156] According to the information processing system of (((5))), the detection state of the object can be continued even if the position of the object shifts outside the display area, compared to when only a part of the detection area is expanded outside the display area.

[0157] According to the information processing system (((6))), it is easier to continue detecting an object compared to when the shape of the detection area remains constant regardless of the distance of the object from the screen.

[0158] According to the information processing system (((7))), it is possible to reduce erroneous operations by the user compared to when an operation on an object is confirmed when a target object is detected in the detection area.

[0159] According to the information processing system (((8))), it is possible to reduce erroneous operations by the user compared to when operations on objects other than the object currently being operated are accepted.

[0160] According to the information processing system of (((9))), it is possible to reduce erroneous operations by the user compared to when the operation is continued even if the target object moves out of the detection area of ​​the object being operated.

[0161] According to the information processing system (((10))), it is easier for the user to recognize the detected object compared to when the display mode of the object is maintained even if the target object is detected in the detection area for a predetermined time.

[0162] According to the information processing system of (((11))), it is possible to reduce user operation errors compared to when switching between selection and operation confirmation depending on the time it takes to detect an object.

[0163] According to the information processing program (((12))), when an object is detected in a detection area included in a display area of ​​an object operated non-contact, the detection state of the object can be continued compared to when the detection area remains within the display area. [Explanation of symbols]

[0164] 3. User's finger (example of target object) 10 Image processing device 50A Print magnification selection screen (example of the screen) 54B Up button (an example of an object) 54C Down button (an example of an object) DR detection area

Claims

1. a processor; The processor: When a non-contact operation is performed on an object on a screen by an object, if the object is detected in a detection area included in a display area in which the object is displayed on the screen, the detection area is expanded to outside the display area. Information processing system.

2. The processor expands the detection area according to the detection time of the object. The information processing system according to claim 1 .

3. The processor extends the sensing area downward relative to the display area.

3. The information processing system according to claim 1.

4. The processor expands the detection area so that it diverges. The information processing system according to claim 3 .

5. The processor extends the entire detection area to outside the display area.

3. The information processing system according to claim 1.

6. The processor changes the shape of the detection area according to the distance of the object from the screen.

3. The information processing system according to claim 1.

7. The processor determines an operation on the object when the object is detected on the detection area for a predetermined time. The information processing system according to claim 1 .

8. When a plurality of objects are displayed on the screen, the processor stops accepting operations on objects other than the object being operated. The information processing system according to claim 7 .

9. The processor stops the operation when the object moves out of the detection area of ​​the object being operated.

9. The information processing system according to claim 7 or 8.

10. the processor changes at least one of a color and a shape of the object when the object is detected on the detection area for a predetermined time.

9. The information processing system according to claim 7 or 8.

11. The processor switches between selecting the object and confirming the operation assigned to the selected object depending on the distance of the object from the screen. The information processing system according to claim 1 .

12. When a non-contact operation is performed on an object on a screen by an object, if the object is detected in a detection area included in a display area in which the object is displayed on the screen, the detection area is expanded to outside the display area. An information processing program to be executed by a computer.

Images