Information processing device and information processing program

Abstract

To obtain an information processing device and an information processing program which can virtually superimpose and display related information with respect to a printed matter, without requiring special print.SOLUTION: A server 10 comprises: an acquisition unit 11A which acquires superposition information which is virtually superimposed and displayed with respect to a printed matter, when print indication of the printed matter is performed from a terminal device 30 with respect to an image forming device 50; and a control unit 11C which associates the superposition information with the printed matter to perform control to store the information in a storage unit 13, when a detection unit 11B detects that the printed matter has been printed by the image forming device 50.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing program. [Background technology]

[0002] Patent Document 1 discloses an information processing device that aims to appropriately determine an object that a user is viewing from among a plurality of objects and to present extended information.

[0003] This information processing device detects multiple objects from a captured image, determines a target object from the multiple objects, obtains the positions of the multiple objects and their angles relative to a reference, and for a reference object among the multiple objects whose position and angle relative to the target object satisfy predetermined conditions, displays an image of extended information related to the reference object superimposed on the image of the reference object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-49847 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology disclosed in Patent Document 1 has a problem in that the target objects are specific markers or character strings, and require special printing.

[0006] An object of the present disclosure is to provide an information processing device and an information processing program that can virtually superimpose and display related information on a printed matter without requiring special printing. [Means for solving the problem]

[0007] In order to achieve the above object, an information processing device according to a first aspect includes a processor, and when an instruction to print a printed matter is given to an image forming device from an instruction device, the processor acquires superimposition information to be virtually superimposed and displayed on the printed matter, and when it is detected that the printed matter has been printed by the image forming device, the processor displays the superimposition information on the printed matter. and position information indicating the position of the printed matter in three-dimensional space. and stores them in the memory. An information processing apparatus, wherein the processor detects that the printed matter has been printed by detecting that the printed matter has been separated from the image forming apparatus. .

[0008] In addition, an information processing device according to a second aspect is the information processing device according to the first aspect, in which the processor obtains the overlay information by extracting it from print job information used to print the printed material.

[0009] An information processing device according to a third aspect is the information processing device according to the second aspect, wherein the superimposition information includes at least one of information about the title, author, and creation date of the printed matter.

[0011] Also, 4 The information processing device according to the aspect includes: Any one of the first to third aspects In the information processing device according to the present invention, the processor detects that the printed material has been moved away from the image forming device by detecting that the gaze of the person who printed the printed material has been directed toward the printed material.

[0012] Also, 5 The information processing device according to the embodiment is the first embodiment to the third embodiment. 4 In an information processing device according to any one of the aspects, when multiple copies of the printed material are printed and the multiple printed materials are stacked, the processor controls the creation of integrated information that integrates the overlay information that corresponds individually to each of the multiple printed materials, and stores the integrated information in the memory unit in association with the multiple printed materials.

[0013] Also, 6 The information processing device according to the embodiment is the first embodiment to the third embodiment. 5In the information processing device according to any one of the aspects, the processor performs control to virtually display the printed matter in addition to the superimposition information.

[0014] Also, 7 The information processing device according to the embodiment is the first embodiment to the third embodiment. 6 In an information processing device relating to any one of the aspects, when the processor detects that the printed matter has been printed by the image forming device, it obtains printer information indicating the printer who performed the printing from the image forming device, and uses the obtained printer information to identify the superimposition information to be superimposed on the printed matter.

[0015] Furthermore, in order to achieve the above objectives, 8 The information processing program according to the aspect includes, when an instruction to print a printed matter is given to an image forming apparatus from an instruction device, acquiring superimposition information to be virtually superimposed and displayed on the printed matter, and when it is detected that the printed matter has been printed by the image forming apparatus, displaying the superimposition information on the printed matter. and position information indicating the position of the printed matter in three-dimensional space. and stores the data in a storage unit in association with the data. a process for detecting that the printed matter has been printed by detecting that the printed matter has been separated from the image forming device; The processing is executed by a computer. [Effects of the Invention]

[0016] First aspect and 8 According to this aspect, related information can be virtually superimposed and displayed on a printed matter without requiring special printing. Furthermore, according to the first and eighth aspects, it is possible to detect that a printed matter has been printed by utilizing the natural behavior of the person who printed it.

[0017] According to the second aspect, it is possible to generate superimposition information more easily compared to creating superimposition information anew.

[0018] According to the third aspect, the applied information can be virtually superimposed on the printed matter and displayed.

[0020] No. 4According to this aspect, it is possible to further improve convenience for the printer compared to when it is detected that the printed material has been separated without using the line of sight.

[0021] No. 5 According to this aspect, it is possible to more easily control the superimposition of the superimposition information on the printed matter compared to when the superimposition information is not integrated.

[0022] No. 6 According to the aspect, the effects of the present disclosure can be enjoyed not only in an AR (Augmented Reality) environment but also in a VR (Virtual Reality) environment.

[0023] No. 7 According to this aspect, it is possible to more easily identify the target superimposition information compared to when the printer information is not acquired from the image forming apparatus. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a block diagram showing an example of the configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to the embodiment. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a terminal device according to an embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of a control unit in the image forming apparatus according to the embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of the hardware configuration of a control unit in the AR glasses according to the embodiment. [Figure 6] FIG. 2 is a block diagram illustrating an example of a functional configuration of a server according to the embodiment. [Figure 7] FIG. 2 is a schematic diagram showing an example of a configuration of a management information database according to the embodiment. [Figure 8] FIG. 2 is a schematic diagram illustrating an example of a configuration of an association information database according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of a print instruction process according to the embodiment. [Figure 10] FIG. 10 is a front view showing an example of a print instruction screen according to the embodiment. [Figure 11] 10 is a flowchart illustrating an example of information processing according to the embodiment. [Figure 12] 10 is a flowchart illustrating an example of a printing process according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of a tracking process according to the embodiment. [Figure 14] FIG. 2 is a perspective view showing an example of an AR image according to the embodiment. [Figure 15] FIG. 10 is a perspective view showing another example of an AR image according to the embodiment. [Figure 16] FIG. 10 is a block diagram showing an example of the configuration of an information processing system according to a second embodiment. [Figure 17] FIG. 10 is a block diagram showing an example of the functional configuration of a server according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the present embodiment, an example in which the technology of the present disclosure is applied to an image forming apparatus installed in an office will be described. However, the technology of the present disclosure is not limited to offices, and can be applied to any place where an image forming apparatus can be installed, such as a school, a home, etc.

[0026] [First embodiment] First, the configuration of an information processing system 1 according to this embodiment will be described with reference to Fig. 1. In this embodiment, a case will be described in which the technology of the present disclosure is applied to an information processing system that includes a server as an information processing device of the technology of the present disclosure, a terminal device that instructs execution of printing, an image forming device that performs the printing, and AR glasses, and that realizes an AR environment. Fig. 1 is a block diagram showing an example of the configuration of the information processing system 1 according to this embodiment.

[0027] 1, an information processing system 1 according to this embodiment includes a server 10, a plurality of terminal devices 30A, 30B, etc., a plurality of image forming devices 50A, 50B, etc., and AR glasses 70. In the following, when the terminal devices 30A, 30B, etc. are described without distinction, they will be collectively referred to simply as "terminal device 30." In the following, when the image forming devices 50A, 50B, etc. are described without distinction, they will be collectively referred to simply as "image forming device 50."

[0028] Examples of the server 10 and the terminal device 30 include information processing devices such as a personal computer and a server computer. In this embodiment, a digital multifunction peripheral having an image printing function, an image reading function, an image transmission function, etc. is applied as the image forming device 50. However, this is not limited to this, and other image forming devices such as an image forming device having only an image printing function or an image forming device having only an image printing function and an image reading function may also be applied as the image forming device 50.

[0029] It goes without saying that the image forming devices 50A, 50B, etc. are not necessarily all of the same specifications, and may differ in the services that can be executed, the configurable items for those services, and the optional items that are equipped.

[0030] The server 10, the terminal device 30, the image forming device 50, and the AR glasses 70 are connected via a network N, and the server 10 is capable of communicating with the terminal device 30, the image forming device 50, and the AR glasses 70 via the network N.

[0031] In this embodiment, the network N is a combination of public communication lines such as the Internet and a telephone network, and internal company communication lines such as a LAN (Local Area Network) and a WAN (Wide Area Network), but is not limited to this. For example, the network N may be configured to use only either the public communication lines or the internal company communication lines. In addition, in this embodiment, the network N is configured to use both wired and wireless communication lines, but is not limited to this. The network N may be configured to use only either a wireless communication line or a wired communication line.

[0032] Next, the configuration of the server 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the server 10 according to this embodiment.

[0033] 2, the server 10 according to this embodiment includes a CPU (Central Processing Unit) 11 as a processor, a memory 12 as a temporary storage area, a non-volatile storage unit 13, an input unit 14 such as a keyboard and a mouse, a display unit 15 such as a liquid crystal display, a medium read / write device (R / W) 16, and a communication interface (I / F) unit 18. The CPU 11, memory 12, storage unit 13, input unit 14, display unit 15, medium read / write device 16, and communication I / F unit 18 are connected to one another via a bus B1. The medium read / write device 16 reads information written in a recording medium 17 and writes information to the recording medium 17.

[0034] The storage unit 13 according to this embodiment is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. An information processing program 13A is stored in the storage unit 13 as a storage medium. The information processing program 13A is stored (installed) in the storage unit 13 when a recording medium 17 on which the information processing program 13A is written is connected to the medium reading and writing device 16 and the medium reading and writing device 16 reads the information processing program 13A from the recording medium 17. The CPU 11 reads the information processing program 13A from the storage unit 13, expands it in the memory 12, and sequentially executes the processes of the information processing program 13A.

[0035] Furthermore, a management information database 13B and an association information database 13C are stored in the storage unit 13. The management information database 13B and the association information database 13C will be described in detail later.

[0036] Next, the configuration of the terminal device 30 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the hardware configuration of the terminal device 30 according to this embodiment.

[0037] 3, the terminal device 30 according to this embodiment includes a CPU 31 as a processor, a memory 32 as a temporary storage area, a non-volatile storage unit 33, an input unit 34 such as a keyboard and a mouse, a display unit 35 such as a liquid crystal display, a medium read / write device (R / W) 36, and a communication I / F unit 38. The CPU 31, memory 32, storage unit 33, input unit 34, display unit 35, medium read / write device 36, and communication I / F unit 38 are connected to one another via a bus B2. The medium read / write device 36 reads information written in a recording medium 37 and writes information to the recording medium 37.

[0038] The storage unit 33 according to this embodiment is realized by an HDD, SSD, flash memory, or the like. A print instruction processing program 33A is stored in the storage unit 33 as a storage medium. The print instruction processing program 33A is stored (installed) in the storage unit 33 when a recording medium 37 on which the print instruction processing program 33A is written is connected to the medium reading and writing device 36 and the medium reading and writing device 36 reads the print instruction processing program 33A from the recording medium 37. The CPU 31 reads the print instruction processing program 33A from the storage unit 33, expands it in the memory 32, and sequentially executes the processes of the print instruction processing program 33A.

[0039] Next, the configuration of the control unit in the image forming apparatus 50 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the hardware configuration of the control unit in the image forming apparatus 50 according to this embodiment.

[0040] 4, an image forming apparatus 50 according to this embodiment includes a CPU 51 as a processor, a memory 52 as a temporary storage area, a non-volatile storage unit 53, an input unit 54 such as a touch panel, a display unit 55 such as a liquid crystal display, a medium reading and writing device (R / W) 56, and a communication I / F unit 58. The CPU 51, memory 52, storage unit 53, input unit 54, display unit 55, medium reading and writing device 56, and communication I / F unit 58 are connected to one another via a bus B3. The medium reading and writing device 56 reads information written in a recording medium 57 and writes information to the recording medium 57.

[0041] The storage unit 53 according to this embodiment is realized by an HDD, an SSD, a flash memory, or the like. A print processing program 53A is stored in the storage unit 53 as a storage medium. The print processing program 53A is stored (installed) in the storage unit 53 when a recording medium 57 on which the print processing program 53A is written is connected to the medium reading and writing device 56 and the medium reading and writing device 56 reads the print processing program 53A from the recording medium 57. The CPU 51 reads the print processing program 53A from the storage unit 53, expands it in the memory 52, and sequentially executes the processes of the print processing program 53A.

[0042] It goes without saying that the image forming apparatus 50 is provided with various image-related processing units such as an image forming engine, an image reading unit, an image transmitting unit, and the like, in addition to the configuration of the control unit described above.

[0043] Next, the configuration of the control unit in the AR glasses 70 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the hardware configuration of the control unit in the AR glasses 70 according to this embodiment.

[0044] 5, the AR glasses 70 according to this embodiment include a CPU 71 as a processor, a memory 72 as a temporary storage area, a non-volatile storage unit 73, an input unit 74 such as a microswitch, a projection unit 75 that projects various types of information, and a wireless communication unit 78. The AR glasses 70 according to this embodiment also include an imaging unit 77 and a position detection unit 79. The CPU 71, memory 72, storage unit 73, input unit 74, projection unit 75, imaging unit 77, wireless communication unit 78, and position detection unit 79 are connected to one another via a bus B4.

[0045] The storage unit 73 according to this embodiment is realized by an HDD, an SSD, a flash memory, or the like. A tracking processing program 73A is stored in the storage unit 73 as a storage medium. During the manufacturing process of the AR glasses 70, the latest version of the tracking processing program 73A is stored in advance in the storage unit 73, and the AR glasses 70 are shipped in that state. When the tracking processing program 73A is updated, the AR glasses 70 download the latest version via the wireless communication unit 78 and update the program stored in the storage unit 73. The CPU 71 reads the tracking processing program 73A from the storage unit 73, loads it into the memory 72, and sequentially executes the processes of the tracking processing program 73A.

[0046] Furthermore, the wireless communication unit 78 according to this embodiment can communicate wirelessly with the server 10 via the network N. In the wireless communication unit 78 according to this embodiment, a mobile communication standard such as 4G or 5G is applied as a communication standard for communicating with the server 10, but it goes without saying that the communication standard is not limited to this.

[0047] Furthermore, the wireless communication unit 78 according to the present embodiment can also communicate directly and wirelessly with the image forming apparatus 50. Therefore, although not shown in the drawings, the image forming apparatus 50 has a built-in wireless communication unit that is capable of wireless communication with the wireless communication unit 78 of the AR glasses 70. In the wireless communication unit 78 according to the present embodiment, the BLE (Bluetooth Low Energy (registered trademark)) standard is applied as the communication standard for communication with the image forming apparatus 50, but this is not limiting. For example, in addition to the BLE standard, any communication standard that enables short-range wireless communication with the image forming apparatus 50, such as the Wi-Fi (registered trademark) standard, can be applied as the standard for communication with the image forming apparatus 50.

[0048] Furthermore, the position detection unit 79 according to this embodiment detects its own position and outputs position information, and although a GPS (Global Positioning Systems) is used in this embodiment, the present invention is not limited to this. For example, the position detection unit 79 may use position information acquired from a Wi-Fi (registered trademark) router, detect the position using a beacon, or detect the position by image analysis using a captured image.

[0049] Furthermore, the photographing section 77 according to this embodiment photographs moving images and outputs image information obtained by the photographing.

[0050] Although not shown, it goes without saying that the AR glasses 70 are equipped with other components for realizing the function of glasses, such as binocular lenses and eyeglass frames. The projection unit 75 according to this embodiment projects various information directly onto the lenses. In this manner, the AR glasses 70 according to this embodiment display various information by projection using the projection unit 75, but this is not limiting. For example, instead of the projection unit 75, a dedicated display that displays various information in a state visible to the wearer may be provided, and various information may be displayed using this display.

[0051] Although not shown in the drawings to avoid confusion, in the information processing system 1 according to this embodiment, a pair of AR glasses 70 is provided for each user of the terminal device 30.

[0052] Next, the functional configuration of the server 10 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the functional configuration of the server 10 according to this embodiment.

[0053] 6, the server 10 includes an acquisition unit 11A, a detection unit 11B, and a control unit 11C. The CPU 11 of the server 10 executes an information processing program 13A, thereby functioning as the acquisition unit 11A, the detection unit 11B, and the control unit 11C.

[0054] When an instruction to print a printed material is given to the image forming apparatus 50 from an instruction device (terminal device 30 in this embodiment), the acquisition unit 11A according to this embodiment acquires superimposition information to be virtually superimposed and displayed on the printed material from the terminal device 30 and the AR glasses 70. Note that in this embodiment, five types of superimposition information are applied: the title, author, creation date, content, and thumbnail of the printed material; however, this is not limited to these. Any one type of these pieces of information, or a combination of two or more to four types, may be applied as the superimposition information. Alternatively, in addition to the five types of information, other information such as the user of the printed material, the storage location of the printed material, and the storage period of the printed material may be applied as the superimposition information.

[0055] Furthermore, the detection unit 11B according to the present embodiment detects that a printed material has been printed by detecting that the printed material has been removed from the image forming device 50. Note that in the present embodiment, the detection unit 11B detects that a printed material has been removed from the image forming device 50 by detecting that the gaze of the person who printed the printed material is directed toward the printed material. That is, the AR glasses 70 according to the present embodiment are equipped with a gaze detection function that detects the direction of the wearer's gaze, and the detection unit 11B according to the present embodiment uses the gaze detection function of the AR glasses 70 to detect that a printed material has been removed from the image forming device 50. However, the method for detecting that a printed material has been removed from the image forming device 50 is not limited to this. In the image forming device 50, when printed materials are discharged into a chamber and the discharge unit is illuminated until the printed material discharged into the chamber is removed, the detection unit 11B may detect that the printed material has been removed by detecting that the light from the illumination has been turned off. Furthermore, a paper sensor may be provided at the printout discharge section of the image forming apparatus 50, and the paper sensor may be used to detect that the printout has been released.

[0056] Then, when the detection unit 11B detects that a printed matter has been printed by the image forming device 50, the control unit 11C in this embodiment controls the memory unit 13 to associate the superimposition information with the printed matter.

[0057] In this embodiment, the acquisition unit 11A acquires the title, author, and creation date of the printed matter in the superimposition information by extracting them from the print job information used to print the printed matter. Furthermore, in this embodiment, if the printed matter is text information, the acquisition unit 11A acquires the content of the printed matter in the superimposition information by extracting it from the text information. Furthermore, in this embodiment, the acquisition unit 11A acquires the thumbnail in the superimposition information by extracting it from the attribute information of the electronic file of the printed matter, assuming that the electronic file of the printed matter is in PDF format. However, the method of acquiring the superimposition information by the acquisition unit 11A is not limited to the above method. For example, the superimposition information may be acquired by having the person who instructed the printing of the printed matter input the superimposition information. Furthermore, if the electronic file of the printed matter is an image file, the superimposition information may be acquired using conventionally known OCR (Optical Character Recognition) technology.

[0058] Incidentally, when printing one printed sheet at a time using the image forming device 50, it is sufficient to associate the overlay information with only that one printed sheet. However, when multiple printed sheets are printed at a time and the multiple printed sheets are stacked, the convenience for users would be significantly improved if individual overlay information could be collectively associated with each printed sheet.

[0059] Therefore, when multiple printed sheets are printed and the multiple printed sheets are stacked, the control unit 11C in this embodiment controls the creation of integrated information that integrates the overlay information corresponding to each of the multiple printed sheets individually, and stores the integrated information in the memory unit 13 in association with the multiple printed sheets.

[0060] Furthermore, when the acquisition unit 11A according to this embodiment detects that a printed matter has been printed by the image forming device 50, it acquires printer information indicating the printer who performed the printing from the image forming device 50. Then, the control unit 11C according to this embodiment uses the printer information acquired by the acquisition unit 11A to identify the superimposition information to be superimposed on the printed matter.

[0061] In this embodiment, the image forming device 50 performs login authentication of the printer before printing the printed material, and the information used during the login authentication that can identify the printer (in this embodiment, the user ID (Identification) described later) is applied as the printer information, but it goes without saying that this is not limited to this.

[0062] When the printed material associated with the superimposition information by the control unit 11C according to the present embodiment is viewed by the user through the AR glasses 70, the superimposition information is displayed in a virtually superimposed state. In this case, a configuration may be adopted in which the superimposition information is selectively displayed either constantly or only when a predetermined condition is satisfied. In this configuration, examples of the predetermined condition include a condition that the user's gaze is directed toward the target printed material, a condition that a predetermined operation is performed on the input unit 74, etc.

[0063] Next, the management information database 13B according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a schematic diagram showing an example of the configuration of the management information database 13B according to this embodiment.

[0064] The management information database 13B according to this embodiment is a database in which information about printed matter printed by the image forming apparatus 50 and information about superimposition information that is virtually superimposed and displayed on the printed matter when the printed matter is one sheet is registered. As shown in Fig. 7 as an example, the management information database 13B according to this embodiment stores information such as an information ID, a user ID, a multifunction device ID, a document name, the number of sheets, superimposition information, and a reception date and time in association with each other.

[0065] The information ID is information assigned individually to each printed material to identify the corresponding printed material, the user ID is information assigned individually to each user to identify the user who instructed the printing of the corresponding printed material, the multifunction device ID is information assigned individually to each image forming device 50 to identify the image forming device 50 that will print the corresponding printed material, and the document name is information indicating the name of the document that will become the corresponding printed material. The number of sheets is information indicating the number of printed sheets of the corresponding printed material, the superimposition information is information indicating the superimposition information itself to be virtually superimposed and displayed on the corresponding printed material, and the reception date and time is information indicating the date and time when the corresponding superimposition information, etc. was received from the terminal device 30.

[0066] 7 shows that a printed matter assigned with an information ID of "J001" is printed by image forming device 50 assigned with a multifunction device ID of "M001" in response to an instruction from a user assigned with a user ID of "U001." The example shown in FIG. 7 also shows that the document title of the printed matter is "Patent Publication...," that only one page will be printed, and that superimposed information such as a company name will be virtually superimposed and displayed on the printed matter.

[0067] Next, the association information database 13C according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a schematic diagram showing an example of the configuration of the association information database 13C according to this embodiment.

[0068] The association information database 13C according to this embodiment is a database in which information for associating the position of a printed matter printed by the image forming apparatus 50 with superimposition information that is virtually superimposed on the printed matter is registered. As shown in FIG. 8 as an example, the association information database 13C according to this embodiment stores information IDs, position information, and superimposition information in association with each other.

[0069] The information ID is the same information (information indicating a printed matter) as the information ID in the management information database 13B, the position information is information indicating the coordinate position of the corresponding printed matter with a predetermined position in three-dimensional space as the origin, and the superimposition information is information to be virtually superimposed and displayed on each printed matter when there are multiple corresponding printed matters.

[0070] That is, in the information processing system 1 according to this embodiment, when a printed sheet is printed by the image forming device 50, the user who printed the printed sheet directs their gaze toward the printed sheet while wearing the AR glasses 70. In response, the AR glasses 70 identify the location of the printed sheet using the gaze detection function described above and a conventionally known image recognition technology. Specifically, the AR glasses 70 detect the direction of the user's gaze using the gaze detection function, and use image recognition technology to detect an object that exists in the detected gaze direction and matches the characteristics of the printed sheet from the image information obtained by the photographing unit 77. The AR glasses 70 then continue to track the detected printed sheet and sequentially transmit position information indicating the coordinate position of the printed sheet in the three-dimensional space, which is sequentially obtained by the tracking, to the server 10. The server 10 sequentially stores (updates) the position information received from the AR glasses 70 in the association information database 13C as the position information of the corresponding printed sheet.

[0071] Furthermore, in the information processing system 1 according to the present embodiment, when multiple printed materials are printed by the image forming device 50, the user who printed the printed materials is supposed to wear the AR glasses 70, pick up the multiple printed materials, and check the contents of each printed material one by one. In response to this, the AR glasses 70 detect the position of each printed material using the line-of-sight detection function and image recognition technology, as in the case where only one printed material is printed, and then continue to track each of the detected printed materials and sequentially transmit to the server 10 position information indicating the coordinate position in three-dimensional space of each printed material sequentially obtained by the tracking. The server 10 sequentially stores (updates) the position information received from the AR glasses 70 in the association information database 13C as the position information of the corresponding printed material.

[0072] Furthermore, when multiple printed materials are printed by the image forming device 50, the AR glasses 70 acquire the title and layout of each of the detected printed materials using conventionally known OCR technology or the like, and transmit this information to the server 10. The server 10 stores the title and layout information received from the AR glasses 70 as superimposition information in the association information database 13C, in association with the corresponding printed material.

[0073] 7 and 8, it is shown that a printed matter to which "J001" is assigned as the information ID is currently located at the coordinate position (X1, Y1, Z1), and the superimposed information to be virtually superimposed and displayed is a company name, etc. Also, in the example shown in Fig. 8, a printed matter to which "J003" is assigned as the information ID is composed of three sheets, the first of which is currently located at the coordinate position (X31, Y31, Z31), and the superimposed information to be virtually superimposed and displayed is "Device Design A" and "Text + Drawing."

[0074] Next, the operation of the information processing system 1 according to this embodiment will be described with reference to FIGS.

[0075] First, the operation of the terminal device 30 when executing the print instruction process will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the print instruction process according to this embodiment.

[0076] In the information processing system 1 according to this embodiment, a user of the terminal device 30 (hereinafter referred to as a "print instructor") issues an instruction to print a target printed matter (hereinafter referred to as a "target printed matter") using the terminal device 30. When the print instructor issues an instruction to print, the CPU 31 of the terminal device 30 executes the print instruction processing program 33A, thereby executing the print instruction processing shown in FIG.

[0077] In step 100 of Fig. 9, CPU 31 controls display unit 35 to display a print instruction screen having a predetermined configuration. In step 102, CPU 31 waits until predetermined information is input. Fig. 10 shows an example of the print instruction screen according to this embodiment.

[0078] 10, the print instruction screen according to this embodiment displays a message prompting the user to specify an image forming device or the like that will print the target printed material. The print instruction screen according to this embodiment also displays information indicating the names of image forming devices 50 that can be used by the terminal device 30. The print instruction screen according to this embodiment also displays a designation section 35A for designating the image forming device 50 to be used, and a designation section 35B that is designated when the AR glasses 70 are to present superimposition information, i.e., when the superimposition information is to be virtually superimposed and displayed on the target printed material.

[0079] 10 is displayed, the print instruction person uses input unit 34 to specify designation unit 35A corresponding to image forming apparatus 50 that will print the target printed matter. Furthermore, the print instruction person uses input unit 34 to specify designation unit 35B corresponding to the case where superimposition information is to be virtually superimposed on the target printed matter. Then, after completing the above designations, the print instruction person uses input unit 34 to specify end button 35C. When end button 35C is specified by the print instruction person, step 102 is judged as positive and the process proceeds to step 104.

[0080] In step 104, CPU 31 creates print job information for the target printed matter. In step 106, CPU 31 determines whether or not designation section 35B has been designated. If the determination is negative, CPU 31 proceeds to step 110, whereas if the determination is positive, CPU 31 proceeds to step 108.

[0081] In step 108, if there is only one target printed sheet, CPU 31 acquires superimposition information to be virtually superimposed on the target printed sheet as described above, and transmits the acquired superimposition information to server 10. CPU 31 also transmits print-related information to server 10, including a user ID previously assigned to the print instructor, information indicating the image forming device 50 that will print the target printed sheet, information indicating the document name of the target printed sheet, and information indicating the number of copies of the target printed sheet. Upon receiving the print-related information, server 10 assigns a new information ID to the received information and stores (registers) it in management information database 13B.

[0082] In step 110, CPU 31 transmits the created print job information to image forming apparatus 50 designated by designation unit 35A, and then ends this print instruction process.

[0083] When the image forming device 50 receives the print job information, it temporarily stores the received print job information, and after undergoing login authentication for the print instructor, prints the printed material using the received print job information in accordance with the instructions of the print instructor.

[0084] In the present embodiment, as described above, the selection and designation of the image forming device 50 to be used and the designation of whether or not to virtually superimpose and display the superimposition information are performed via the dedicated screen shown in Fig. 10 as an example, but the present invention is not limited to this. For example, these designations may be performed using a standard function provided in the browser.

[0085] Next, the operation of the server 10 when executing information processing will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of information processing according to this embodiment.

[0086] In the information processing system 1 according to this embodiment, when the target printed material is printed by the image forming device 50, information capable of identifying the print instructor (corresponding to the above-mentioned printer information, hereinafter referred to as "printer information") obtained during the login authentication described above is transmitted to the server 10. In the information processing according to this embodiment, when the printer information or the print-related information transmitted in the above-mentioned print instruction processing is received, the CPU 11 of the server 10 executes the information processing program 13A, thereby executing the information processing shown in FIG.

[0087] In step 200 of FIG. 11, the CPU 11 determines whether the received information is print-related information, and if the determination is affirmative, the process proceeds to step 202.

[0088] In step 202, CPU 11 stores (registers) the received print-related information together with the newly generated information ID in management information database 13B. Furthermore, if CPU 11 receives overlay information together with the print-related information, CPU 11 also stores (registers) the overlay information in management information database 13B in association with the newly generated information ID. Then, when CPU 11 has finished storing the above various information, it ends this information processing. Through the processing of step 202, the print-related information and overlay information received from each terminal device 30 are sequentially registered in management information database 13B together with the corresponding information ID, and management information database 13B is constructed.

[0089] On the other hand, if the determination in step 200 is negative, the received information is considered to be printer information, and the process proceeds to step 204 .

[0090] In step 204, CPU 11 assumes that the printing of the printed matter by the printing instructor indicated by the received printing instructor information will be performed soon, and reads all information stored in association with the user ID of the printing instructor (hereinafter referred to as "target candidate information") from management information database 13B.

[0091] The image forming device 50 (hereinafter referred to as the "target device") that sent the printer information sends the printer information to the server 10, and then sends document identification information (in this embodiment, an information ID) to the server 10, which is information that can identify the target printed matter specified for printing by the print instructor.

[0092] Therefore, in step 206, the CPU 11 waits until document identification information is received from the target device. In step 208, the CPU 11 determines whether or not superimposition information is associated with the target printed material identified by the document identification information, and if the determination is negative, the process proceeds to step 212, whereas if the determination is positive, the process proceeds to step 210. In step 210, the CPU 11 transmits the superimposition information associated with the target printed material to the AR glasses 70 used by the print instructor.

[0093] In this embodiment, although not shown in the drawings, information that can communicate with the AR glasses 70 used by each user of the information processing system 1 is registered in advance, and the information is used to transmit superimposition information to the AR glasses 70 used by the print instructor. However, this is not limited to this. For example, the installation positions of each image forming apparatus 50 may be registered in advance, and the AR glasses 70 closest to the target apparatus, as detected by a position detection unit 79 provided in the AR glasses 70, may be considered to be the AR glasses 70 of the print instructor, and the superimposition information may be transmitted to the AR glasses 70.

[0094] On the other hand, as described above, when the AR glasses 70 detect a printed matter, they sequentially transmit position information indicating the position of the printed matter in three-dimensional space, which is sequentially obtained by tracking the printed matter, to the server 10. Furthermore, when multiple printed matters are printed, the AR glasses 70 transmit information indicating the title and layout of each printed matter to the server 10.

[0095] Therefore, in step 212, the CPU 11 uses the information received from the AR glasses 70 to execute an association process for associating the printed matter with the corresponding superimposition information, as will be described below.

[0096] That is, when the CPU 11 receives information indicating the title and layout of each of a plurality of printed sheets, it regards the plurality of printed sheets as being overlapped. Then, the CPU 11 associates the information indicating the title and layout with the corresponding printed sheet and stores it as the above-mentioned integrated information in the corresponding storage area of ​​the association information database 13C.

[0097] Next, the CPU 11 associates the location information successively sent from the AR glasses 70 with the corresponding printed matter, and sequentially stores (updates) the information in the corresponding storage area of ​​the association information database 13C. At this time, the CPU 11 assumes that the corresponding printed matter is the printed matter indicated by the document identification information received by the processing of step 206.

[0098] Then, when a predetermined end timing arrives, the CPU 11 stops storing the position information in the association information database 13C, thereby ending the association process. When the association process in step 212 ends, this information process ends. Note that in this embodiment, the end timing is the timing when reception of position information from the AR glasses 70 has ceased for a predetermined period of time (for example, one minute), but this is not limited to this. For example, the end timing may be the timing when the person instructing the printing removes the AR glasses 70 or when the power switch of the AR glasses 70 is turned off.

[0099] Next, the operation of the image forming apparatus 50 when executing a printing process will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the printing process according to this embodiment.

[0100] As described above, in the information processing system 1 according to this embodiment, login authentication is performed on the print instructor before printing of a printed material is performed in the image forming apparatus 50. Prior to performing this login authentication, the print instructor inputs an instruction to perform login authentication via the input unit 54 provided in the image forming apparatus 50. When this instruction is input, the CPU 51 of the image forming apparatus 50 executes the print processing program 53A, thereby performing the print processing shown in FIG.

[0101] 12, the CPU 51 performs processing related to the login authentication of the print instructor described above. In step 302, the CPU 51 determines whether the login authentication has been successful, and if the determination is negative, the process returns to step 300, whereas if the determination is positive, the process proceeds to step 304. In step 304, the CPU 51 transmits the print instructor information described above to the server 10.

[0102] In step 306, the CPU 51 controls the display of a print target selection screen (not shown) on the display unit 55. The print target selection screen displays, in a list format, information indicating the printed matter to be printed based on the print job information received from the terminal device 30 used by the print instructor (information indicating the document name of the printed matter in this embodiment). In step 308, the CPU 51 waits until predetermined information is input.

[0103] When the print target selection screen is displayed, the person instructing printing specifies the document name of the document they wish to print from the displayed document names via input unit 54. When this document name is specified, the determination in step 308 is affirmative, and the process proceeds to step 310.

[0104] In step 310, the CPU 51 transmits the above-mentioned document specification information to the server 10, and in step 312, the CPU 51 executes printing of the printed matter corresponding to the document name specified by the print instruction user, and then terminates this printing process.

[0105] Next, the operation of the AR glasses 70 when performing tracking processing will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of tracking processing according to this embodiment.

[0106] When the printing instructor wears the AR glasses 70 and inputs an instruction to start execution via the input unit 74, the CPU 71 of the AR glasses 70 executes the tracking processing program 73A, thereby performing the tracking processing shown in Figure 13.

[0107] In step 400 of FIG. 13, the CPU 71 waits until a printed matter printed by the image forming device 50 is detected as described above.

[0108] In step 402, the CPU 71 determines whether multiple printed sheets have been printed, and if the determination is negative, the process proceeds to step 408, whereas if the determination is positive, the process proceeds to step 404. As described above, when multiple printed sheets have been printed by the image forming apparatus 50, the person instructing the printing picks up the multiple printed sheets and checks the contents of each printed sheet one by one. Therefore, the CPU 71 makes the determination in step 402 by determining whether this check has been performed using the image information obtained by the photographing unit 77.

[0109] In step 404 , the CPU 71 detects information indicating the title and layout from the plurality of printed sheets as described above, and in step 406 , the CPU 71 transmits the detected information indicating the title and layout to the server 10 .

[0110] In step 408, the CPU 71 tracks the detected printed matter as described above, and transmits to the server 10 location information indicating the location of the printed matter obtained through the tracking.

[0111] In step 410, the CPU 71 determines whether a predetermined end timing for ending tracking has arrived. If the determination is negative, the process returns to step 408, whereas if the determination is positive, the tracking process ends. Note that in this embodiment, the end timing is the timing when the print instructor removes the AR glasses 70, but this is not limiting. For example, the end timing may be the timing when the print instructor performs a predetermined operation on the AR glasses 70 or the timing when the printed material disappears from view.

[0112] The AR glasses 70 execute a process of virtually superimposing and displaying superimposition information corresponding to a printed matter visible through the lenses on the printed matter, using the information registered in the association information database 13C by each of the above processes, the received superimposition information, etc. Therefore, as with the technology disclosed in Patent Document 1, it is possible to virtually superimpose and display the related information on the printed matter without requiring special printing, such as printing additional information on the printed matter to add the virtual related information.

[0113] As described above, in the present embodiment, a case has been described in which the server 10 performs the process of associating the superimposition information with the actual printed matter, but the present invention is not limited to this. For example, the process of associating the superimposition information may be performed by the AR glasses 70. In this case, the CPU 71 of the AR glasses 70 associates the superimposition information received from the server 10 and the superimposition information (title and layout information) detected by the glasses 70 with the printed matter it detects, stores the information in the storage unit 73, and virtually displays the information. In this case, the information processing device of the present disclosure is included in the AR glasses 70.

[0114] 14 is a perspective view showing an example of an AR image according to this embodiment that is viewed through the AR glasses 70 when superimposition information is associated with only one printed material. Note that in FIG. 14, to avoid confusion, images of the background and the contents of the printed material are omitted, except for the printed material 90 and the hand 92 of the person instructing the printing to hold the printed material 90.

[0115] 14, in this case, in the AR image, superimposition information 98A corresponding to the printed matter 90 is displayed in a speech bubble 96A originating from an area of ​​the printed matter 90. Even if the printed matter 90 is moved by the person instructing printing, the speech bubble 96A is displayed in the same state, following the movement.

[0116] 15 is a perspective view showing an example of an AR image according to this embodiment that is viewed through the AR glasses 70 when superimposition information is associated with multiple printed materials (two printed materials in the example shown in FIG. 15). Note that in FIG. 15 as well, to avoid confusion, images of the background and content of the printed materials are omitted, except for printed materials 90A and 90B and the hand 92 of the person instructing the printing.

[0117] 15, in the AR image in this case, superimposition information 98B and 98C corresponding to printed materials 90A and 90B, respectively, are displayed in speech bubbles 96B and 96C, each originating from an area of ​​the corresponding printed materials 90A and 90B. Even in this case, even if the printed materials 90A and 90B are moved by the printing instructor, speech bubbles 96B and 96C are displayed in the same state, following the movement.

[0118] 14 and 15, the size of the speech bubbles may be changed depending on the amount of information to be displayed. In this case, simply increasing the size of the speech bubbles depending on the amount of information makes it difficult to see other objects such as printed materials. Therefore, the size of the speech bubbles may be fixed, and the information displayed in the speech bubbles may be changed by scrolling or switching the display range all at once. In this case, the change may be instructed by operating the input unit 74 provided on the AR glasses 70 or by a gesture by the person instructing the printing.

[0119] 15 illustrates an example in which individual pieces of superimposed information are displayed separately for each of a plurality of printed materials, but this is not limiting. For example, the superimposed information may be displayed together, or may be displayed together when the printed materials cannot be distinguished and then displayed separately when the printed materials can be distinguished. In this case, the printing instructor may instruct the user to switch the display of the superimposed information. In this case, the instruction to switch may also be given by operating the input unit 74 provided on the AR glasses 70 or by a gesture by the printing instructor himself.

[0120] [Second embodiment] In this embodiment, the technology disclosed herein is described as being applied to an information processing system that includes a server as an information processing device of the technology disclosed herein, a terminal device that instructs printing to be performed, an image forming device that performs the printing, and VR goggles, and that realizes a VR environment.

[0121] First, the configuration of the information processing system 1 according to this embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing an example of the configuration of the information processing system 1 according to this embodiment, and the same components as those in Fig. 1 are assigned the same reference numerals as in Fig. 1, and their description will be omitted.

[0122] As shown in FIG. 16, the information processing system 1 according to this embodiment differs from the information processing system 1 according to the first embodiment in that the AR glasses 70 are replaced with VR goggles 80.

[0123] Next, the functional configuration of the server 10 according to this embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the functional configuration of the server 10 according to this embodiment, and the same components as those in Fig. 6 are assigned the same reference numerals as those in Fig. 6, and the description thereof will be omitted.

[0124] As shown in FIG. 17, the server 10 according to this embodiment differs from the server 10 according to the first embodiment in that a control unit 11D is applied instead of the control unit 11C.

[0125] That is, the control unit 11D according to this embodiment differs from the control unit 11C according to the first embodiment in that it controls to virtually display printed matter in addition to superimposed information. This control makes it possible to display superimposed information in an AR environment, as achieved in the first embodiment, in a VR environment as well.

[0126] For example, the print instructor uses the terminal device 30 in the real space to instruct the image forming device 50 to print a printed material, and then causes the image forming device 50 to print the printed material, and performs the operations up to retrieving the printed material in the same manner as in the first embodiment described above.

[0127] On the other hand, the image forming device 50 is similar to the first embodiment in that when a print instruction is made by the print user, the image forming device 50 transmits document-specific information relating to the print to the server 10. However, the difference from the first embodiment is that the subsequent processing is performed on the VR goggles 80.

[0128] Overall, the print instruction processing, information processing, printing processing, and tracking processing described in the first embodiment are almost the same as those in the first embodiment, except that the controlled object changes from the AR glasses 70 to the VR goggles 80, so further explanation will be omitted.

[0129] In the above embodiments, the position of the target printed matter is defined as a coordinate position with a predetermined position in three-dimensional space as the origin, but the present invention is not limited to this. For example, information indicating a relative position based on the position of a stationary object in the three-dimensional space may be used as information indicating the position of the target printed matter.

[0130] Furthermore, the above embodiments did not mention the timing for dissociating the target printed material from the superimposition information. An example of this timing is when it is detected that the target printed material has been discarded using a shredder or the like. Other examples include timing set by the user, such as when a predetermined time has passed.

[0131] In addition, in the above embodiments, when multiple printed materials are printed at once, individual superimposition information is virtually superimposed on each printed material and displayed, but this is not limited to this. For example, when multiple printed materials are stacked, only the superimposition information corresponding to the first printed material may be displayed, and when the multiple printed materials are separated, individual superimposition information may be displayed for each printed material.

[0132] Furthermore, in the above-described embodiments, the case where each piece of superimposition information is applied to only a single user has been described, but this is not limiting. For example, each piece of superimposition information may be shared and used by a plurality of users.

[0133] Furthermore, in the above-described embodiments, the AR glasses 70 (VR goggles 80) communicate directly with another device, but the present invention is not limited to this. For example, the AR glasses 70 (VR goggles 80) may communicate with another device via a portable information processing device such as a smartphone or tablet terminal owned by the user. In this case, if login authentication is performed using FIDO (Fast IDentity Online) authentication or the like when using the portable information processing device, the login authentication may also serve as login authentication performed by the image forming device 50.

[0134] By performing login authentication on the image forming device 50 based on receiving information from a portable information processing device, such as FIDO authentication, a wireless connection is established between the image forming device 50 and the portable information processing device for the authentication. This allows subsequent transmission of information to the AR glasses 70 (VR goggles 80) via the image forming device 50 to continue using the connection.

[0135] In the above embodiments, when the server 10 receives title and layout information, it considers that multiple corresponding printed materials are superimposed, and stores the superimposition information as integrated information, collectively associated with the corresponding printed materials. In this configuration, if the corresponding printed materials are subsequently separated from each other, the superimposition information may be treated as separate pieces of information and individually associated with the corresponding printed materials.

[0136] Furthermore, in the above-described embodiments, the various databases are registered in the server 10, but the present invention is not limited to this. For example, the various databases may be registered in the terminal device 30 used by the administrator of the information processing system 1 or in the image forming device 50 that is primarily used.

[0137] Furthermore, in the above-described embodiments, the case where the gaze detection function provided in the AR glasses 70 (VR goggles 80) is used to detect that the printed matter has been separated from the image forming device 50 has been described, but the present invention is not limited to this. For example, it is also possible to detect that the printed matter has been separated from the image forming device 50 by detecting, using OCR technology or the like, that the title of the printed matter or a characteristic image is included in an image captured by the imaging unit 77 provided in the AR glasses 70 (VR goggles 80). In this case, if there is only one sheet of printed matter, the AR glasses 70 (VR goggles 80) obtains superimposition information from the server 10, and therefore, it is also possible to determine that the printed matter has been separated from the image forming device 50 by detecting, using the OCR technology or the like, the information included in the superimposition information.

[0138] Although the embodiments have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the gist of the invention, and such modifications and improvements are also included in the technical scope of the present invention.

[0139] Furthermore, the above-described embodiments do not limit the inventions described in the claims, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. The above-described embodiments include inventions at various stages, and various inventions can be extracted by combining the multiple disclosed constituent elements. Even if some constituent elements are deleted from all the constituent elements shown in the embodiments, as long as the effect is obtained, the configuration from which these constituent elements are deleted can be extracted as an invention.

[0140] In addition, in each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0141] Furthermore, in each of the above embodiments, various processes are described as being implemented by a software configuration using a computer by executing a program, but the present invention is not limited to this. For example, various processes may be implemented by a hardware configuration or a combination of a hardware configuration and a software configuration.

[0142] In addition, the configurations of the server 10, terminal device 30, image forming device 50, and AR glasses 70 described in each of the above embodiments are examples, and it goes without saying that unnecessary parts may be deleted or new parts may be added within the scope of the present invention.

[0143] Furthermore, the various processing flows described in the above embodiments are merely examples, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, within the scope of the spirit of the present invention. [Explanation of symbols]

[0144] 1. Information Processing Systems 10 Servers 11 CPU 11A Acquisition Department 11B Detector 11C Control unit 11D Control Unit 12 Memory 13 Storage section 13A Information Processing Program 13B Management Information Database 13C Association Information Database 14 Input section 15 Display section 16 Media reading and writing device 17 Recording Media 18 Communication I / F section 30 Terminal Equipment 31 CPU 32 memory 33 Storage section 33A Print instruction processing program 34 Input section 35 Display section 36 Media reading and writing device 37 Recording Media 38 Communication I / F section 50 Image forming device 51 CPU 52 memory 53 Storage section 53A Print Processing Program 54 Input section 55 Display section 56 Media reading and writing device 57 Recording Media 58 Communication I / F section 70 AR Glasses 71 CPU 72 memory 73 Memory section 73A Tracking Processing Program 74 Input section 75 Projection section 77 Photography Department 78 Radio Communication Department 79 Position detection unit 80 VR goggles 90, 90A, 90B printed matter 92 moves 96A, 96B, 96C speech bubbles 98A, 98B, 98C Overlay information

Claims

1. a processor; The processor: When an instruction to print a printed matter is given to an image forming apparatus from an instruction device, superimposition information is acquired to be virtually superimposed on the printed matter and displayed; When it is detected that the printed matter has been printed by the image forming device, control is performed to associate the superimposition information and position information indicating the position of the printed matter in three-dimensional space with the printed matter and store them in a storage unit. An information processing device, The processor: detecting that the printed matter has been printed by detecting that the printed matter has been separated from the image forming device; Information processing device.

2. The processor: The overlay information is obtained by extracting it from print job information used to print the printed matter. The information processing device according to claim 1 .

3. The superimposed information includes at least one piece of information about the title, author, and creation date of the printed matter. The information processing device according to claim 2 .

4. The processor: The detection of the separation of the printed matter from the image forming device is performed by detecting that the gaze of a person who printed the printed matter is directed toward the printed matter.

4. The information processing device according to claim 1.

5. The processor: When a plurality of sheets of the printed matter are printed and the printed sheets are superimposed, control is performed to create integrated information that integrates the superimposition information that individually corresponds to each of the plurality of printed matters, and to store the integrated information in the storage unit in association with the plurality of printed matters.

5. The information processing device according to claim 1.

6. The processor: and performing control to virtually display the printed matter in addition to the superimposed information.

6. The information processing device according to claim 1.

7. The processor: When it is detected that the printed matter has been printed by the image forming device, printer information indicating the printer who printed the printed matter is acquired from the image forming device; using the acquired printer information, identifying the superimposition information to be superimposed on the printed matter; 7. The information processing device according to claim 1.

8. When an instruction to print a printed matter is given to an image forming apparatus from an instruction device, superimposition information is acquired to be virtually superimposed on the printed matter and displayed; When it is detected that the printed matter has been printed by the image forming apparatus, control is performed to associate the superimposition information with the printed matter and position information indicating a position of the printed matter in a three-dimensional space, and store the superimposition information and position information in a storage unit, detecting that the printed matter has been printed by detecting that the printed matter has been separated from the image forming device; An information processing program that causes a computer to execute a process.

Images