Information processing system and information processing method

Abstract

To provide an information processing system and an information processing method capable of increasing the display number of images stored in an image forming apparatus and improving the operability of user operations on the same images.SOLUTION: In an information processing system 10, an image acquisition processing unit 211 acquires an image of an image forming apparatus 1 that has an AR marker indicating an IP address of the image forming apparatus 1. An identification information acquisition processing unit 212 acquires the IP address of the image forming apparatus 1 from the AR marker displayed in the image acquired by the image acquisition processing unit 211. An image data acquisition processing unit 213 acquires the image data stored in the image forming apparatus 1 corresponding to the IP address acquired by the identification information acquisition processing unit 212. A display processing unit 214 displays an image corresponding to the image data acquired by the image data acquisition processing unit 213.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for displaying an image stored in an image forming device. [Background technology]

[0002] Image forming devices such as printers and multifunction devices are equipped with an operation panel that accepts user operations and displays various information (see, for example, Patent Document 1). For example, a user can use the operation panel to check images stored in the image forming device and give instructions to print the images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-155857 A Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional image forming devices, the operation panel has been made smaller. This results in a smaller display area on the operation panel, which reduces the amount of information that can be displayed. For example, when multiple images stored in the image forming device are displayed on the display panel as a list, not only does the number of images that can be displayed at one time decrease, but the user must repeatedly scroll the screen to search for a desired image, which reduces operability.

[0005] An object of the present invention is to provide an information processing system and an information processing method that can increase the number of images stored in an image forming device that can be displayed and improve the ease of user operations on the images. [Means for solving the problem]

[0006] An information processing system according to one aspect of the present invention includes a captured image acquisition processing unit, an identification information acquisition processing unit, an image data acquisition processing unit, and a display processing unit. The captured image acquisition processing unit acquires a captured image of an image forming device on which an identification mark indicating identification information of the image forming device is displayed. The identification information acquisition processing unit acquires the identification information of the image forming device from the identification mark displayed in the captured image acquired by the captured image acquisition processing unit. The image data acquisition processing unit acquires image data stored in the image forming device corresponding to the identification information acquired by the identification information acquisition processing unit. The display processing unit displays an image corresponding to the image data acquired by the image data acquisition processing unit.

[0007] An information processing method according to another aspect of the present invention includes a captured image acquisition step, an identification information acquisition step, an image data acquisition step, and a display step. In the captured image acquisition step, a captured image of an image forming device on which an identification mark indicating the identification information of the image forming device is displayed is acquired. In the identification information acquisition step, the identification information of the image forming device is acquired from the identification mark displayed in the captured image acquired in the captured image acquisition step. In the image data acquisition step, image data stored in the image forming device corresponding to the identification information acquired in the identification information acquisition step is acquired. In the display step, an image corresponding to the image data acquired in the image data acquisition step is displayed. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing system and an information processing method that can increase the number of images stored in an image forming device that can be displayed and improve the operability of user operations on the images. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing an example of image information according to the embodiment of the present invention. [Figure 4] FIG. 4 is a schematic diagram showing an example of capturing an image of an AR marker in the image forming apparatus according to the embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing an example of a captured image displayed on the operation terminal according to the embodiment of the present invention. [Figure 11] FIG. 11 is a flowchart showing an example of a procedure of a display control process executed in the operation terminal according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings to facilitate understanding of the present invention. Note that the following embodiment is an example of the present invention and does not limit the technical scope of the present invention.

[0011] [Information Processing System 10] As shown in FIG. 1, an information processing system 10 according to an embodiment of the present invention includes an image forming apparatus 1 and an operation terminal 2. The information processing system 10 is an example of the information processing system of the present invention. The image forming apparatus 1 and the operation terminal 2 are capable of communicating with each other via a network N1 such as a telephone line, the Internet, or a LAN. There may be one image forming apparatus 1 and one operation terminal 2, or there may be multiple image forming apparatuses. The operation terminal 2 is an information processing device such as a smartphone, a mobile terminal, or a wearable terminal. The operation terminal 2 is an example of the operation terminal of the present invention.

[0012] In the information processing system 10, for example, when a user photographs the image forming apparatus 1 with the camera 25 of the operation terminal 2 (see FIG. 4), the operation terminal 2 identifies the image forming apparatus 1 based on the captured image, obtains image data from the identified image forming apparatus 1, and displays the image (see FIG. 5). The operation terminal 2 also accepts a user operation (e.g., a print operation) on the displayed image (see FIG. 6), and outputs an instruction (print instruction) to the image forming apparatus 1 according to the user operation.

[0013] Incidentally, in conventional image forming devices, the operation panel (operation display unit 12) has been made smaller. This results in a problem that the display area of ​​the operation panel is narrowed, resulting in a reduced amount of information that can be displayed. For example, when multiple images stored in the image forming device are displayed in a list on the display panel, not only does the number of images that can be displayed at one time decrease, but the user must repeatedly scroll the screen when searching for a desired image, and operability also decreases. In contrast, in the information processing system 10 according to this embodiment, as shown below, it is possible to increase the number of images displayed that are stored in the image forming device 1 and improve the operability of user operations on the images.

[0014] [Image forming device 1] 2, the image forming apparatus 1 includes a control unit 11, an operation display unit 12, an ADF (Auto Document Feeder) 13, an image reading unit 14, an image forming unit 15, a communication I / F 16, a storage unit 17, an AR marker 18, etc. Specifically, the image forming apparatus 1 is a multifunction machine having a printer function, a scanner function, a copy function, a facsimile function, etc. Note that the image forming apparatus of the present invention is not limited to a multifunction machine, and can be applied to any image forming apparatus such as a copy machine, a printer, or a facsimile machine.

[0015] The operation display unit 12 includes a display unit such as a liquid crystal display that displays information, and an operation unit such as a touch panel and operation buttons that accept user operations.

[0016] The ADF 13 is an automatic document feeder that includes a document setting section, a transport roller, a document holder, and a paper discharge section, and transports documents to be read by the image reading section 14 .

[0017] The image reading unit 14 includes a document table, a light source, a mirror, an optical lens, and a CCD (Charge Coupled Device), and is capable of reading an image of a document and outputting the image as image data.

[0018] The image forming unit 15 is capable of performing a print process based on image data by an electrophotographic method or an inkjet method, and forms an image on a sheet based on the image data. For example, when the image forming unit 15 is an electrophotographic image forming unit, the image forming unit 15 includes a photoconductor drum, a charger, an exposure device, a developing device, a transfer device, and a fixing device.

[0019] The communication I / F 16 is a communication interface capable of performing wired or wireless data communication with an external device. For example, the communication I / F 16 performs data communication with an external device via a network. The external device includes the operation terminal 2 operated by a user, a storage device for storing data, and the like.

[0020] The storage unit 17 is a non-volatile storage unit such as a flash memory, a hard disk, or an EEPROM (registered trademark), etc. The storage unit 17 stores various control programs executed by the control unit 11, various data, and the like.

[0021] Further, the storage unit 17 stores image information D1 relating to an image. FIG. 3 shows an example of the image information D1. The image information D1 includes an image ID, image data, attribute information, and the like. The image ID is identification information of an image, and is an arbitrary identification number that is assigned according to the order of acquisition (input order) of the image. The attribute information is information relating to the attribute of the image. The attribute information includes, for example, information such as the creation date and time or update date and time assigned when an image is generated, resolution, and operation information (operated or not operated) relating to the operation (printing, scanning, transmission, etc.) performed on the image in the image forming apparatus 1. The image data stored in the storage unit 17 includes image data (print data, FAX reception data, etc.) input from the outside via the network N1, image data (copy data, scan data, etc.) read in the image forming apparatus 1, and the like. The image data may be sorted by folder and stored in the storage unit 17. The image data may also be stored in a storage area different from the image information D1. In this case, the image data is stored in association with each piece of information in the image information D1 via a common image ID.

[0022] In addition, the image data may be given access authority of the user. For example, when the access authority of user A is given to image data P1 (also called image P1) and the access authority of user B is not given, user A is permitted to obtain image P1 and to perform a predetermined operation (e.g., print instruction) on image P1, and user B is prohibited from obtaining image P1 and performing a predetermined operation on image P1.

[0023] Each image forming apparatus 1 stores individual image information D1.

[0024] The AR marker 18 is provided on the exterior of the image forming device 1 (see FIG. 4). The AR marker 18 is an example of an identification sign of the present invention. The AR marker 18 has a predetermined shape and size, and a two-dimensional code image is drawn inside. The operation terminal 2 can determine the position and orientation of the image forming device 1 based on the position, shape, and size of the AR marker 18 in the image (captured image) captured by the camera 25. The operation terminal 2 can also read out the IP address (identification information) of the image forming device 1 by decoding the two-dimensional code image represented on the AR marker 18. Each image forming device 1 is provided with an individual AR marker 18.

[0025] The control unit 11 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile or non-volatile storage unit used as a temporary storage memory (work area) for various processes executed by the CPU.

[0026] Specifically, the control unit 11 includes a reception processing unit 111, a job execution processing unit 112, a transmission processing unit 113, etc. The control unit 11 functions as each of these processing units by executing various processes according to the control program. The control unit 11 may also include an electronic circuit that realizes a part or a plurality of processing functions of each of these processing units.

[0027] The reception processing unit 111 receives various operations from a user. Specifically, when a user performs an operation such as a print instruction, a copy instruction, or a scan instruction on the operation screen displayed on the operation display unit 12 or the operation terminal 2, the reception processing unit 111 receives the operation.

[0028] The job execution processing unit 112 executes various jobs. For example, when the reception processing unit 111 receives an operation such as a print instruction, a copy instruction, or a scan instruction from a user, the job execution processing unit 112 executes a job according to the instruction.

[0029] The transmission processing unit 113 transmits image data stored in the image forming apparatus 1 to the operation terminal 2. Specifically, when the transmission processing unit 113 receives a transmission request for image data from the operation terminal 2, it transmits the image data to the operation terminal 2. The transmission processing unit 113 may also transmit attribute information (see FIG. 3) together with the image data to the operation terminal 2. The transmission request will be described later.

[0030] In addition, the control unit 11 controls each unit such as the ADF 13, the image reading unit 14, the image forming unit 15, and the like, thereby performing overall control of the image forming apparatus 1.

[0031] [Operation terminal 2] 2, the operation terminal 2 includes a control unit 21, a storage unit 22, an operation display unit 23, a communication I / F 24, and a camera 25. The operation terminal 2 is an information processing device having a communication function, such as a smartphone, a mobile terminal, or a wearable terminal. The wearable terminal is, for example, an HMD (Head Mounted Display).

[0032] Camera 25 is a digital camera that captures an image of image forming apparatus 1, which is a subject, and outputs it as digital image data. For example, as shown in Fig. 4, a user activates camera 25 and captures an image of a predetermined range including AR marker 18 provided on image forming apparatus 1. The captured image captured by camera 25 is output to control unit 21. Control unit 21 acquires the captured image for each capture operation by the user.

[0033] The operation display unit 23 includes a display unit such as a liquid crystal display that displays information, and an operation unit such as a touch panel and operation buttons that accept user operations. When the operation terminal 2 is configured as an HMD, the operation display unit 23 is set in a virtual space and accepts gesture operations from the user.

[0034] The communication I / F 24 is a communication interface capable of performing wired or wireless data communication with an external device. For example, the communication I / F 24 performs data communication with the image forming apparatus 1 via the network N1.

[0035] The storage unit 22 is a non-volatile storage unit such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory that stores various types of information. The storage unit 22 stores various control programs executed by the control unit 21, various types of data, and the like. The control programs executed by the control unit 21 include a display control program for causing the control unit 21 to execute a display control process (see FIG. 11 ) described below. The display control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) included in the operation terminal 2 and stored in the storage unit 22. The display control program may also be downloaded from a server (cloud server) and stored in the storage unit 22.

[0036] The control unit 21 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile or non-volatile storage unit used as a temporary storage memory (work area) for various processes executed by the CPU.

[0037] Specifically, the control unit 21 includes a captured image acquisition processing unit 211, an identification information acquisition processing unit 212, an image data acquisition processing unit 213, a display processing unit 214, and a reception processing unit 215. The control unit 21 functions as each of these processing units by executing various processes according to the display control program. The control unit 21 may also include an electronic circuit that realizes a part or a plurality of processing functions of each of these processing units.

[0038] The captured image acquisition processing unit 211 acquires a captured image capturing a predetermined range including the AR marker 18 provided on the image forming apparatus 1. For example, as shown in FIG. 4, when a user captures an image of the image forming apparatus 1 with the camera 25 of the operation terminal 2, the captured image acquisition processing unit 211 acquires the captured image (image data) from the camera 25.

[0039] The identification information acquisition processing unit 212 acquires the identification information of the image forming device 1 from the AR marker 18 displayed in the captured image acquired by the captured image acquisition processing unit 211. Specifically, the identification information acquisition processing unit 212 reads out the IP address (identification information) of the image forming device 1 by decoding the two-dimensional code image displayed in the AR marker 18. Since the AR marker 18 of each image forming device 1 includes a two-dimensional code image in which an individual IP address is written, the identification information acquisition processing unit 212 can acquire the IP address corresponding to the image forming device 1 displayed in the captured image.

[0040] Image data acquisition processing unit 213 acquires image data stored in image forming apparatus 1 corresponding to the IP address (identification information) acquired by identification information acquisition processing unit 212. Specifically, when identification information acquisition processing unit 212 acquires the IP address of a specific image forming apparatus 1, image data acquisition processing unit 213 outputs a transmission request for image data (image) to image forming apparatus 1 corresponding to the IP address. When image forming apparatus 1 receives the transmission request from operation terminal 2, it transmits image data stored in storage unit 17 to operation terminal 2. As a result, image data acquisition processing unit 213 acquires image data from image forming apparatus 1 corresponding to the IP address.

[0041] Moreover, the image data acquisition processing unit 213 acquires attribute information (see FIG. 3) corresponding to the image data from the image forming apparatus 1 corresponding to the IP address.

[0042] The display processing unit 214 displays an image corresponding to the image data acquired by the image data acquisition processing unit 213. Specifically, the display processing unit 214 displays an image corresponding to the image data acquired by the image data acquisition processing unit 213 on the operation display unit 23 by combining the image with the captured image. In Fig. 5, reference symbol A1 indicates a captured image acquired by the captured image acquisition processing unit 211. An image of the image forming device 1 is displayed in the captured image A1.

[0043] The display processing unit 214 causes the captured image A1 captured by the camera 25 to be displayed on the operation display unit 23. Furthermore, the display processing unit 214 causes images (e.g., images P1 to P10) corresponding to the image data acquired by the image data acquisition processing unit 213 from the image forming apparatus 1 to be composited with the captured image A1 and displayed. For example, the display processing unit 214 determines the display positions of the images P1 to P10 based on the position of the AR marker 18 of the image forming apparatus 1, and causes the images P1 to P10 to be displayed as thumbnail images (thumbnail images Sp) in a list. This allows the user to check multiple images stored in the image forming apparatus 1 collectively on his / her own operation terminal 2.

[0044] The reception processing unit 215 receives a user operation on a display screen that displays an image corresponding to the image data acquired by the image data acquisition processing unit 213. Specifically, the reception processing unit 215 receives a user operation on a captured image A1 on which images P1 to P10 (thumbnail images Sp) are combined.

[0045] 6, when a user selects (touches) a desired image (here, "image P4") in captured image A1, the reception processing unit 215 receives the selection operation of image P4 and displays a print button K1 for receiving a print instruction and a property button K2 for receiving an instruction to display property information (attribute information of the image) on captured image A1. The user presses the print button K1 if he / she wishes to print image P4, and presses the property button K2 if he / she wishes to check the property information of image P4.

[0046] When the user presses the print button K1, the reception processor 215 transmits a print instruction for image P4 to the target image forming apparatus 1 (the image forming apparatus 1 corresponding to the IP address). When the reception processor 111 of the image forming apparatus 1 receives the print instruction for image P4 from the operation terminal 2, the job execution processor 112 executes the print process for image P4. This allows the user to issue a print instruction for an image stored in the image forming apparatus 1 from his / her own operation terminal 2.

[0047] Furthermore, when the user presses the property button K2, the display processing unit 214 displays attribute information (property information F1) corresponding to the image P4, as shown in Fig. 7. The display processing unit 214 may display the property information F1 by combining it with the captured image A1 (see Fig. 7), or may display it on a screen different from the display screen of the captured image A1. This allows the user to check the property information of the image stored in the image forming apparatus 1 on his / her own operation terminal 2.

[0048] In this way, the reception processing unit 215 receives an operation from the user to select whether to execute a print process or display property information of the image displayed on the operation terminal 2. The user operation may be a touch operation of touching the display screen, or a gesture operation of making a gesture toward the display screen (virtual space).

[0049] As another embodiment, for example, when the image data is associated with a user's access authority in the storage unit 17 of the image forming apparatus 1, the image data acquisition processing unit 213 may acquire, from the image forming apparatus 1 corresponding to the IP address, image data for which the user of the operation terminal 2 has access authority, among the image data stored in the image forming apparatus 1. For example, the control unit 11 of the image forming apparatus 1 may acquire a user ID when acquiring the transmission request from the operation terminal 2, and transmit only the image data associated with the user ID to the operation terminal 2. This makes it possible to prevent unauthorized acquisition of images by a user without access authority.

[0050] As another embodiment, the display processing unit 214 may display the multiple images in a display mode according to the property information (attributes). For example, as shown in FIG. 8, the display processing unit 214 may display multiple images (image group Pa) that are printed or unprinted in an overlapping manner. As shown in FIG. 9, the display processing unit 214 may display the image size of the printed image in a small size and the image size of the unprinted image in a large size. The display processing unit 214 may display the outer frame of the printed image and the outer frame of the unprinted image in different modes (color, thickness, line type, etc.). The information of "printed" and "unprinted" is an example of attribute information (property information).

[0051] As another embodiment, the display processing unit 214 may display an image corresponding to the position and direction of the operation terminal 2 with respect to the image forming apparatus 1, among a plurality of images corresponding to the image data acquired by the image data acquisition processing unit 213. For example, as shown in FIG. 10, the display processing unit 214 displays a list of a plurality of images (thumbnail images) in a virtual space, and when the operation terminal 2 is moved in the direction of the arrow, displays an image corresponding to the position after the movement. As another embodiment, when the user performs an operation (gesture operation) to slide the image list in the virtual space while fixing the operation terminal 2 in a predetermined position, the image list may be moved (by the amount of sliding) in response to the operation. This allows the user to easily check a large number of images on the operation terminal 2 by intuitive operation.

[0052] As another embodiment, the display processing unit 214 may display a list of a plurality of images corresponding to the image data acquired by the image data acquisition processing unit 213 on the operation terminal 2, and may preferentially display images showing an object designated by the user among the plurality of images. For example, when a user designates an object (structure, object, scenery, landscape, etc.) on the operation terminal 2, the display processing unit 214 extracts images showing the object from among the plurality of images, and displays the extracted image or images at the top (first) position (for example, upper left) of the captured image A1. Note that the method of designating the object may be a method in which the user designates the object by inputting text into a predetermined input field, or a method in which the user selects and designates a predetermined image.

[0053] [Display control processing] Next, an example of the procedure of the display control process executed by the control unit 21 of the operation terminal 2 will be described with reference to FIG. 11. The control unit 21 starts execution of the display control program in response to a user's shooting operation (see FIG. 4), thereby starting execution of the display control process. Here, steps S1, S2, ... indicate the numbers of the processing procedures (steps) executed by the control unit 21. Furthermore, although an example will be described here in which the control unit 21 of the operation terminal 2 executes each step in the display control process, in other embodiments, a plurality of processors may execute each step in the display control process in a distributed manner. Also, a computer (operation terminal 2, server, etc.) may execute each step in the display control process.

[0054] The present invention can be understood as an invention of an information processing method for executing one or more steps included in the display control process.

[0055] <Step S1> First, in step S1, the control unit 21 determines whether or not image data captured by the camera 25 (captured image A1) has been acquired. For example, when a user captures an image of the image forming apparatus 1 with the camera 25 of the operation terminal 2 (see FIG. 4), the control unit 21 acquires the captured image A1 from the camera 25. When the control unit 21 acquires the captured image A1 (S1: Yes), the control unit 21 shifts the process to step S2. The control unit 21 waits until the captured image A1 is acquired (S1: No). Step S1 is an example of a captured image acquisition step of the present invention.

[0056] <Step S2> In step S2, the control unit 21 determines whether or not the AR marker 18 is recognized in the captured image A1. If the control unit 21 recognizes the AR marker 18 in the captured image A1 (S2: Yes), the control unit 21 shifts the process to step S3. If the control unit 21 does not recognize the AR marker 18 in the captured image A1 (S2: No), the control unit 21 returns the process to step S1.

[0057] <Step S3> In step S3, the control unit 21 acquires an IP address (identification information) based on the AR marker 18. Specifically, the control unit 21 acquires the IP address of the image forming device 1 by decoding the two-dimensional code image displayed in the AR marker 18, and identifies the image forming device 1. Step S3 is an example of an identification information acquisition step of the present invention.

[0058] <Step S4> Next, in step S4, control unit 21 acquires image data (image) from image forming apparatus 1. Specifically, when control unit 21 acquires the IP address of image forming apparatus 1, it outputs a request to transmit image data to image forming apparatus 1 corresponding to the IP address. When image forming apparatus 1 receives the transmission request from operation terminal 2, it transmits image data and attribute information (see FIG. 3) stored in storage unit 17 to operation terminal 2. As a result, control unit 21 acquires image data and attribute information from image forming apparatus 1 corresponding to the IP address. Step S4 is an example of an image data acquisition step of the present invention.

[0059] <Step S5> Next, in step S5, the control unit 21 causes the operation display unit 23 to display the images acquired from the image forming apparatus 1 in thumbnail format. For example, as shown in FIG. 5, the control unit 21 causes the operation display unit 23 to display the images acquired from the image forming apparatus 1 (images P1 to P10) by combining them with the captured image A1. The control unit 21 determines the display positions of the images P1 to P10 based on the position of the AR marker 18 of the image forming apparatus 1, and displays the images P1 to P10 as thumbnail format images (thumbnail images Sp) in a list. Note that the control unit 21 may display a plurality of printed or unprinted images (image group Pa) in an overlapping manner as shown in FIG. 8, or may display printed images and unprinted images in different display modes as shown in FIG. 9. The control unit 21 may switch the displayed image according to the position and direction of the operation terminal 2 relative to the image forming apparatus 1, or a gesture operation, as shown in FIG. 10. The control unit 21 may preferentially display an image in which an object designated by the user is included. Step S5 is an example of a display step of the present invention.

[0060] <Step S6> Next, in step S6, the control unit 21 determines whether or not a user operation has been received for the captured image A1 on which the thumbnail image Sp has been composited. If the control unit 21 receives the user operation (S6: Yes), the control unit 21 transitions the process to step S7. On the other hand, if the control unit 21 does not receive the user operation (S6: No), the control unit 21 ends the display control process. The user operation includes, for example, an instruction to print an image selected by the user, an instruction to display property information of an image selected by the user, and the like, as shown in FIG. 6.

[0061] <Step S7> In step S7, control unit 21 executes a process corresponding to the user operation accepted in step S6.

[0062] For example, when the user selects image P4 and presses print button K1 (see FIG. 6), control unit 21 transmits a print instruction for image P4 to the target image forming device 1 (image forming device 1 corresponding to the IP address). When image forming device 1 receives the print instruction for image P4 from operation terminal 2, it executes print processing for image P4.

[0063] Furthermore, for example, when the user selects the image P4 and presses the property button K2, the control unit 21 causes the attribute information (property information F1) corresponding to the image P4 to be displayed, as shown in FIG.

[0064] As described above, the control unit 21 acquires an image from the image forming apparatus 1 in response to a user's shooting operation of the image forming apparatus 1, displays the image, and accepts operations on the image. The control unit 21 executes the display control process every time the user performs a shooting operation.

[0065] As described above, the information processing system 10 according to this embodiment acquires a captured image of the image forming apparatus 1 on which an identification mark (AR marker 18) indicating the identification information (IP address) of the image forming apparatus 1 is displayed, and acquires the identification information of the image forming apparatus 1 from the identification mark displayed in the captured image. The information processing system 10 also acquires image data stored in the image forming apparatus 1 corresponding to the identification information, and displays an image corresponding to the image data.

[0066] For example, the information processing system 10 acquires a captured image captured by a user's operation terminal 2 (such as a smartphone or an HMD), and causes the operation terminal 2 to display a composite image corresponding to the image data on the captured image (see FIG. 5).

[0067] According to the above configuration, images stored in the image forming apparatus 1 can be displayed on an operation terminal 2 different from the operation display unit 12 provided on the image forming apparatus 1, so that many images can be displayed together. Also, the displayed images can be switched in response to user operations on the operation terminal 2 (such as a movement operation of the operation terminal 2 (see FIG. 10), a gesture operation by the user, etc.), so that the user can easily search for and select a desired image. Thus, according to the above configuration, it is possible to increase the number of images displayed that are stored in the image forming apparatus 1 and improve the operability of user operations on the images.

[0068] The information processing system of the present invention may be configured with the image forming apparatus 1 and the operation terminal 2, or may be configured with the operation terminal 2 alone, or may be configured with the image forming apparatus 1 alone. The information processing system of the present invention may be configured with a server alone. For example, the server may acquire a captured image of the image forming apparatus 1 on which an identification sign (AR marker 18) indicating the identification information (IP address) of the image forming apparatus 1 is displayed, acquire the identification information of the image forming apparatus 1 from the identification sign displayed in the captured image, acquire image data stored in the image forming apparatus 1 corresponding to the identification information, and display an image corresponding to the image data on the operation terminal 2. That is, the server may be configured with the functions of each processing unit of the operation terminal 2 (captured image acquisition processing unit 211, identification information acquisition processing unit 212, image data acquisition processing unit 213, display processing unit 214, and reception processing unit 215). The server may also be provided with an image database that associates and stores the identification information (IP addresses) of each of the image forming apparatuses 1 with a plurality of images stored in each image forming apparatus 1. The image database may also be stored in a data server (cloud server) different from the server. When the information processing system 10 includes the server, the operation terminal 2 may execute a browser process to communicate with the server and execute a display process of the captured image A1 (see FIGS. 5 to 10).

[0069] [Disclosure Note] The following will provide an outline of the disclosure extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined in any desired manner.

[0070] <Appendix 1> a captured image acquisition processing unit that acquires a captured image of an image forming apparatus on which an identification mark indicating identification information of the image forming apparatus is displayed; an identification information acquisition processing unit that acquires identification information of the image forming apparatus from the identification mark shown in the captured image acquired by the captured image acquisition processing unit; an image data acquisition processing unit that acquires image data stored in the image forming apparatus corresponding to the identification information acquired by the identification information acquisition processing unit; a display processing unit that displays an image corresponding to the image data acquired by the image data acquisition processing unit; An information processing system comprising:

[0071] <Appendix 2> the captured image acquisition processing unit acquires the captured image captured by a user's operation terminal, The display processing unit causes the operation terminal to display an image corresponding to the image data acquired by the image data acquisition processing unit by combining the captured image with the image. 2. An information processing system according to claim 1.

[0072] <Appendix 3> A user's access authority is associated with each of the image data; the image data acquisition processing unit acquires, from the image data associated with the identification information acquired by the identification information acquisition processing unit, the image data to which the user of the operation terminal has access authority; 3. An information processing system according to claim 2.

[0073] <Appendix 4> The display processing unit displays an image corresponding to the image data acquired by the image data acquisition processing unit in a display mode according to attributes assigned to the image. 4. An information processing system according to claim 2 or 3.

[0074] <Appendix 5> The attribute includes at least one of information indicating that the image has been printed, information indicating that the image has not been printed, and information indicating the update date and time of the image. 5. An information processing system according to claim 4.

[0075] <Appendix 6> the display processing unit causes the operation terminal to display a list of a plurality of images corresponding to the image data acquired by the image data acquisition processing unit, and causes an image in which an object designated by the user is captured to be preferentially displayed among the plurality of images. An information processing system according to any one of appendix 2 to 5.

[0076] <Appendix 7> a reception processing unit that receives, from the user, an operation for selecting whether to execute a print process for the image displayed on the operation terminal or to display property information of the image, An information processing system according to any one of appendixes 2 to 6.

[0077] <Appendix 8> a reception processing unit that receives a user operation on a display screen that displays an image corresponding to the image data acquired by the image data acquisition processing unit, The user operation includes at least one of a touch operation of touching the display screen and a gesture operation of making a gesture toward the display screen. An information processing system according to any one of appendixes 2 to 7.

[0078] <Appendix 9> the display processing unit displays an image corresponding to a position and a direction of the operation terminal with respect to the image forming apparatus, among a plurality of images corresponding to the image data acquired by the image data acquisition processing unit; An information processing system according to any one of appendixes 2 to 8.

[0079] <Appendix 10> acquiring a captured image of an image forming apparatus on which an identification mark indicating identification information of the image forming apparatus is displayed; acquiring identification information of the image forming apparatus from the identification mark shown in the acquired captured image; acquiring image data stored in the image forming device corresponding to the acquired identification information; displaying an image corresponding to the acquired image data; An information processing method executed by one or more processors.

[0080] <Appendix 11> acquiring a captured image of an image forming apparatus on which an identification mark indicating identification information of the image forming apparatus is displayed; acquiring identification information of the image forming apparatus from the identification mark shown in the acquired captured image; acquiring image data stored in the image forming device corresponding to the acquired identification information; displaying an image corresponding to the acquired image data; An information processing program for causing one or more processors to execute the above. [Explanation of symbols]

[0081] 1: Image forming device 2: Operation terminal 10: Information processing systems 11: Control section 18: AR Marker 21: Control section 23: Operation display section 25: Camera 111: Reception department 112: Job execution processing unit 113: Transmission processing unit 211: Image capture acquisition processing unit 212: Identification information acquisition processing unit 213: Image data acquisition processing unit 214: Display processing unit 215: Reception department A1: Captured image D1: Image information F1 :Property Information K1: Print button K2: Properties button P1: Image Sp : Thumbnail image

Claims

1. a captured image acquisition processing unit that acquires a captured image of an image forming apparatus on which an identification mark indicating identification information of the image forming apparatus is displayed; an identification information acquisition processing unit that acquires identification information of the image forming apparatus from the identification mark shown in the captured image acquired by the captured image acquisition processing unit; an image data acquisition processing unit that acquires image data stored in the image forming apparatus corresponding to the identification information acquired by the identification information acquisition processing unit; a display processing unit that displays an image corresponding to the image data acquired by the image data acquisition processing unit; An information processing system comprising:

2. the captured image acquisition processing unit acquires the captured image captured by a user's operation terminal, the display processing unit causes the operation terminal to display an image corresponding to the image data acquired by the image data acquisition processing unit, by combining the captured image with the image data. The information processing system according to claim 1 .

3. A user's access authority is associated with each of the image data; the image data acquisition processing unit acquires, from the image data associated with the identification information acquired by the identification information acquisition processing unit, the image data to which the user of the operation terminal has access authority; The information processing system according to claim 2 .

4. The display processing unit displays an image corresponding to the image data acquired by the image data acquisition processing unit in a display mode according to attributes assigned to the image. The information processing system according to claim 2 .

5. The attribute includes at least one of information indicating that the image has been printed, information indicating that the image has not been printed, and information indicating the update date and time of the image.

5. The information processing system according to claim 4.

6. the display processing unit causes the operation terminal to display a list of a plurality of images corresponding to the image data acquired by the image data acquisition processing unit, and causes an image in which an object designated by the user is captured to be preferentially displayed among the plurality of images. The information processing system according to claim 2 .

7. a reception processing unit that receives, from the user, an operation for selecting whether to execute a print process for the image displayed on the operation terminal or to display property information of the image, The information processing system according to claim 2 .

8. a reception processing unit that receives a user operation on a display screen that displays an image corresponding to the image data acquired by the image data acquisition processing unit, The user operation includes at least one of a touch operation of touching the display screen and a gesture operation of making a gesture toward the display screen. The information processing system according to claim 2 .

9. the display processing unit displays an image corresponding to a position and a direction of the operation terminal with respect to the image forming apparatus, among a plurality of images corresponding to the image data acquired by the image data acquisition processing unit; The information processing system according to claim 2 .

10. a captured image acquiring step of acquiring a captured image of the image forming apparatus on which an identification mark indicating identification information of the image forming apparatus is displayed; an identification information acquiring step of acquiring identification information of the image forming apparatus from the identification mark shown in the captured image acquired in the captured image acquiring step; an image data acquisition step of acquiring image data stored in the image forming apparatus corresponding to the identification information acquired in the identification information acquisition step; a display step of displaying an image corresponding to the image data acquired in the image data acquisition step; An information processing method comprising:

Images