Computer program, information processing apparatus, and system

Abstract

To provide a mechanism for suitably printing, with an image forming apparatus, outputs generated by, for example, generative AI.SOLUTION: The present invention relates to a computer program that extends the functionality of a generative AI service. The computer program causes a computer to function so as to specify a print range when a print request is received via the generative AI service. The computer program further causes the computer to generate a print job on the basis of the print request and the print range, and to register the generated print job with a print server. In addition, the computer program causes the computer to acquire information on a connection destination for executing the print job from the print server, and to present the print job and information about the connection destination to a user via the generative AI service.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing device, and a system for expanding the functionality of an artificial intelligence system. [Background technology]

[0002] Generative AI is an artificial intelligence system known as a system that can generate text, images, or other media in response to prompts. Generative AI, such as ChatGPT (registered trademark), allows for functionality expansion using plugins. For example, if a sentence containing the keyword "print" is entered as an instruction to the generative AI, a plugin related to printing is selected, and the plugin can perform processes that the generative AI cannot respond to, thereby enabling functionality expansion.

[0003] On the other hand, an image forming apparatus can input print data from a cloud server and print it. For example, Patent Document 1 proposes a technology that enables printing of documents managed by a cloud server in a chat conversation format. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-128843 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when printing output such as text generated by a generation AI on an image forming device, a user or the like must copy the output to a file or the like and then execute printing from a print driver, etc. In other words, the user must copy the output generated by the generation AI in order to execute printing, which is time-consuming.

[0006] The present invention has been made in consideration of at least one of the above problems, and provides a mechanism for allowing an image forming device to suitably print the output from a generation AI. [Means for solving the problem]

[0007] The present invention is, for example, a computer program that extends the functionality of a generation AI service, and is characterized in that it causes a computer to function as an identification means that identifies a print range when a print request is received via the generation AI service, a generation means that generates a print job based on the print request and the print range, a reservation means that registers the generated print job to a print server, an acquisition means that acquires information regarding a connection destination for executing the print job from the print server, and a presentation means that presents the print job and information regarding the connection destination to a user via the generation AI service.

[0008] The present invention also provides a printing system that includes, for example, an information processing device that provides a generation AI service to a user terminal, a print server, and an image forming device, wherein the information processing device, by executing a computer program that extends the functionality of the generation AI service, functions as an identification means that identifies a print range when a print request is received via the generation AI service, a generation means that generates a print job based on the print request and the print range, a reservation means that registers the generated print job to the print server, an acquisition means that acquires information regarding a connection destination for executing the print job from the print server, and a presentation means that presents the print job and information regarding the connection destination to the user via the generation AI service, and the print server is characterized in that it includes a provision means that provides information regarding the connection destination when the print job is registered, and an execution means that submits the print job to the image forming device when the user accesses the information regarding the connection destination. [Effects of the Invention]

[0009] According to the present invention, the output from the generation AI can be suitably printed on an image forming device, and the present invention also reduces the user's effort when printing text generated by the generation AI on an image forming device. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a system configuration according to an embodiment. [Figure 2] FIG. 1 is a diagram showing the arrangement of an MFP according to an embodiment. [Figure 3] FIG. 2 is a diagram showing the configuration of a user terminal according to an embodiment. [Figure 4] FIG. 1 is a diagram showing the configuration of a generation AI server according to an embodiment. [Figure 5] FIG. 1 is a diagram showing the configuration of an extended application server according to an embodiment. [Figure 6] FIG. 2 is a diagram showing the configuration of a cloud print server according to an embodiment. [Figure 7] An example screen showing an interaction with a generation AI server displayed on a user terminal according to one embodiment. [Figure 8] FIG. 4 is a sequence diagram of document printing according to an embodiment. [Figure 9] An example screen showing an interaction with a generation AI server displayed on a user terminal according to one embodiment. [Figure 10] 10 is an example of a screen displayed on a user terminal after a print log is specified according to an embodiment. [Figure 11] FIG. 4 is a sequence diagram of document printing according to an embodiment. [Figure 12] 10 is a flow chart for an extended application server according to one embodiment. [Figure 13] 10 is a flow chart for an extended application server according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment <System configuration> A first embodiment of the present invention will be described below. In this embodiment, an MFP (Multi Function Peripheral) having print, scan, and fax functions will be described as an example of an image processing device. FIG. 1 shows an example of the overall configuration of a printing system that provides a print service via a generation AI service according to the present invention. This printing system includes an image forming device 10 such as a printing device, a user terminal 20, a generation AI server 30, an extended application server 40, and a cloud print server 50. Each of these is connected to each other via a network 70 so that they can communicate with each other. The network 70 is a wireless or wired network configured as a WAN or LAN.

[0013] The image forming device 10 refers to a printing device that has the function of printing images notified by the cloud print server 50. The user terminal 20 refers to an information terminal such as a smartphone, tablet terminal, or personal computer that a user uses when using a generation AI service such as ChatGPT. The user operates the user terminal 20 to access the generation AI server 30 on the cloud and execute the generation AI application. The user terminal 20 and the generation AI server 30 are connected to a wired or wireless communication network, and are capable of sending and receiving data between them. In this embodiment, a public line on the Internet is assumed as an example of a communication network, but a dedicated line may also be used.

[0014] The generation AI server 30 is a cloud server (information processing device) located on the cloud 60 that provides generation AIs such as ChatGPT, a text generation AI. The generation AI server 30 provides services in cooperation with the extended application server 40. The generation AI server 30 interprets messages sent from the user terminal 20, generates appropriate answers, and displays them as responses on the screen of the user terminal 20. Furthermore, the generation AI server 30's functions can be expanded by communicating with the extended application server 40.

[0015] The extended application server 40 is a cloud server (information processing device) located on the cloud 60, and has the function of providing additional functions (plug-ins) to the generation AI server 30. The generation AI server 30 cooperates with the extended application server 40 and implements predetermined plug-ins, enabling it to perform processes that it could not perform on its own. The cloud print server 50 receives a print request from the user terminal 20 and notifies the image forming device 10 that a print job has arrived. In response to the notification from the cloud print server 50, the image forming device 10 receives the print job from the cloud print server 50 and executes printing. Note that while an example in which various servers are provided separately is described here, they may also be provided integrated with other servers. For example, the generation AI server 30 and the extended application server 40 may be provided integrated.

[0016] <Hardware configuration of image forming device> An example of the hardware configuration of image forming apparatus 10 according to this embodiment will be described with reference to Fig. 2. Image forming apparatus 10 includes a control unit 110, an operation unit 116, a reading unit 118, a printing unit 120, a wireless communication unit 122, a FAX communication unit 124, and a communication unit 126. Control unit 110 includes a CPU 111, a ROM 112, a RAM 113, an HDD 114, an operation unit I / F 115, a reading unit I / F 117, a printing unit I / F 119, a wireless communication unit I / F 121, a FAX unit I / F 123, and a communication unit I / F 125.

[0017] The control unit 110, including the CPU 111, controls the overall operation of the image forming apparatus 10. The CPU 111 loads a control program stored in the ROM 112 or the HDD 114 into the RAM 113 to perform various types of control, such as reading control and printing control. The ROM 112 stores control programs executable by the CPU 111. The ROM 112 also stores a boot program, font data, and the like. The RAM 113 is a main memory, and is used as a work area and a temporary storage area for expanding the various control programs stored in the ROM 112 and the HDD 114. The HDD 114 stores image data, print data, various programs, various addresses, and various setting information. The HDD 114 is a storage medium, and may be an SSD (Solid State Drive), eMMC (Embedded Multi Media Card), or the like instead of an HDD.

[0018] In the image forming apparatus 10 of this embodiment, one CPU 111 executes each process shown in the flowcharts described below using one memory (RAM 113), but this is not limited to this. For example, each process can be executed by multiple CPUs, RAMs, ROMs, and HDDs working together. Also, some processes can be executed using hardware circuits such as ASICs and FPGAs.

[0019] The operation unit I / F 115 connects the control unit 110 to an operation unit 116, which includes a display unit such as a touch panel and hard keys. The operation unit 116 displays information to the user and detects inputs from the user. The reading unit I / F 117 connects the control unit 110 to a reading unit 118, such as a scanner. The reading unit 118 reads an image from a document, and the CPU 111 converts the image into image data such as binary data. Image data generated based on the image read by the reading unit 118 is sent to an external device or printed on recording paper. The printing unit I / F 119 connects the control unit 110 to a printing unit 120, such as a printer. The CPU 111 transfers image data (print data) stored in the RAM 113 to the printing unit 120 via the printing unit I / F 119. The printing unit 120 prints an image based on the transferred image data on recording paper fed from a paper feed cassette.

[0020] The wireless communication unit I / F 121 is an I / F for controlling the wireless communication unit 122, and connects to the control unit 110 and external wireless devices via wireless communication. The FAX unit I / F 123 is connected to the public line network 80 by controlling the FAX communication unit 124. The FAX unit I / F 123 is an I / F for controlling the FAX communication unit 124, and by controlling the modem and NCU for facsimile communication, it is possible to connect to the public line network, control the facsimile communication protocol, and the like.

[0021] The communication unit I / F 125 connects the control unit 110 to the network 70. The communication unit I / F 125 allows the communication unit 126 to transmit image data and various internal device information to external devices on the network 70, and to receive print data and information on the network 70 from external devices on the network 70. Methods for transmitting and receiving data via the network 70 include electronic mail (E-mail) and file transmission using other protocols (e.g., FTP, SMB, WEBDAV, etc.). Image data and message data can also be transmitted and received over the network 70 by accessing the image forming device 10 via HTTP communication from the user terminal 20 and the generation AI server 30. Furthermore, it is possible to access the image forming device 10 via the network from a web browser on the user terminal 20 and configure various settings of the image forming device 10 (remote UI function).

[0022] <Hardware configuration of user terminal> An example of the hardware configuration of the user terminal 20 according to this embodiment will be described with reference to Fig. 3. Note that the user terminal 20 according to this embodiment is assumed to be a device such as a smartphone or tablet PC, but may be any other information processing device that can connect to the network 70 using Wi-Fi communication or the like. The user terminal 20 includes an operation panel 201, a camera 204, an NFC communication unit 205, a Bluetooth (registered trademark) communication unit 206, a CPU 207, a ROM 208, a RAM 209, an HDD 210, and a wireless LAN communication unit.

[0023] The CPU 207 reads out a control program stored in the ROM 208 and executes various processes for controlling the operation of the user terminal 20. The ROM 208 stores the control program. The RAM 209 is used as a temporary storage area such as the main memory and work area of the CPU 207. The HDD 210 stores various data such as photographs and electronic documents.

[0024] The operation panel 201 has a touch panel function that can detect touch operations by the user, and displays various screens provided by the OS and email sending application. The operation panel 201 is also used to check information stored in the generation AI server 30. The user can input desired operation instructions to the user terminal 20 by inputting touch operations on the operation panel 201. The user terminal 20 is also equipped with hardware keys (not shown), and the user can also input operation instructions to the user terminal 20 using these hardware keys.

[0025] Camera 204 takes an image in response to a user's instruction to take an image. The image taken by camera 204 is stored in a predetermined area of HDD 210. It is also possible to obtain information from a QR code read by camera 204 using a program capable of analyzing QR codes.

[0026] The user terminal 20 can exchange data with various peripheral devices via the NFC communication unit 205, the Bluetooth communication unit 206, and the wireless LAN communication unit 211. The Bluetooth communication unit 206 of the user terminal 20 may be compatible with Bluetooth Low Energy (BLE).

[0027] <Hardware configuration of the generative AI server> An example of the hardware configuration of the generation AI server 30 according to this embodiment will be described with reference to Fig. 4. The generation AI server 30 includes a CPU 301, a ROM 302, a RAM 303, a communication unit 304, and an HDD 305.

[0028] The CPU 301 executes processing to control the operation of generating appropriate responses using a control program stored in the ROM 302 and a learning model stored in the HDD 305. The ROM 302 stores the control program. The RAM 303 is used as a temporary storage area such as the main memory and work area of the CPU 301. The HDD 305 stores various data such as the learning model, the generation AI application, and logs of messages sent from the user terminal 20 and the generated responses.

[0029] Data can be exchanged with various devices such as the user terminal 20, the image forming apparatus 10, and the extended application server 40 via the communication unit 304. The communication unit 304 may perform wired communication using Ethernet (registered trademark) or wireless communication such as Wi-Fi.

[0030] <Hardware configuration of the extended application server> An example of the hardware configuration of the extended application server 40 will be described with reference to Fig. 5. The extended application server 40 includes a CPU 401, a ROM 402, a RAM 403, a communication unit 404, and an HDD 405.

[0031] The CPU 401 reads out the control program stored in the ROM 402 and executes processing according to messages received from the generation AI server 30. The ROM 402 stores the control program. The RAM 403 is used as a temporary storage area such as the main memory and work area of the CPU 401. The HDD 405 stores the contents of messages received from the generation AI server 30 or parts of them. Data can be sent and received with each model, such as the generation AI server 30, via the communication unit 404.

[0032] <Cloud Print Server Hardware Configuration> 6, an example of the hardware configuration of the cloud print server 50 will be described. The cloud print server 50 includes a CPU 501, a ROM 502, a RAM 503, a communication unit 504, and an HDD 505.

[0033] The CPU 501 reads out a control program stored in the ROM 502, enabling it to receive print jobs from the user terminal 20 and acquire saved print jobs. The ROM 502 stores the control program. The RAM 503 is used as a temporary storage area such as the main memory and work area of the CPU 501. The HDD 505 stores print jobs received from the user terminal 20, etc. Data can be sent and received with various models such as the generation AI server 30 via the communication unit 504.

[0034] <Print instructions> 7, the interaction between the user and the generation AI server 30 when instructing the generation AI server 30 according to this embodiment to print will be described. By accessing the generation AI server 30 from a browser running on the user terminal 20, a screen 700 is displayed on the operation panel 201. On the screen 700, the message text entered from the user terminal 20 and the response generated by the generation AI server 30 are displayed in chronological order (in an interactive format).

[0035] The prompt input field 701 is an input field for the user to input an instruction (prompt) to the generation AI server 30. The user inputs in natural language text format from the operation panel 201 of the user terminal 20. The user can input an instruction by operating the touch panel of the user terminal 20. The send button 702 is a button that triggers sending the prompt entered in the prompt input field 701 to the generation AI server 30.

[0036] Questions 703 and 705 are prompts sent from the user terminal 20 requesting a response from the generation AI server 30. Answers 704 and 706 are responses generated by the generation AI server, and are answers to the questions asked by the user in questions 703 and 705.

[0037] The print request prompt 707 is an example of a prompt for instructing the generation AI server 30 to print. It must include the range of logs to be printed by the generation AI server 30 and a command to print them. In this embodiment, the content instructs the printing of the two most recent responses, answer statements 704 and 706. Here, it is assumed that the answer statement instructing printing and the print request prompt are in the same conversation log, but past logs stored on the HDD 305 of the generation AI server 30 may also be printed.

[0038] The print request response 708 is a response to the print request prompt 707 generated by the generation AI server 30 in cooperation with the extended application server 40. The print request response 708 includes settings for the print to be performed, a reserved job URL 709 (described later), and a preview image 710.

[0039] The reserved job URL 709 is a URL that points to a print job stored in the cloud print server 50. A user can perform cloud printing (submitting a print job from the cloud print server 50 to the image forming device 10) by accessing this URL. In addition, when changing the print settings of a reserved job, an HTTP request is sent to the reserved job URL. The preview image 710 is a preview image that shows an image of the output when printed with the current print settings. This image allows the user to get a rough idea of how the output will look before printing.

[0040] <sequence> 8, we will explain the interactions between the system components from when a user in this embodiment instructs the generation AI server 30 to print until the print job is actually submitted to the image forming device 10. Note that in the sequence described below, if the generation AI server 30 and the extended application server 40 are integrated, the control of the two devices is integrated.

[0041] In S801, the user terminal 20 sends the contents entered into the print request prompt 707 to the generation AI server 30 according to user input via the browser. Next, in S802, the generation AI server 30 interprets the contents of the print request prompt 707 and determines that the logs to be printed are the answer statements 704 and 706, i.e., specifies the printing range. In the print request prompt 707, it interprets from "Answer" that only the response log is applicable, and from "These two" that the two logs counting backward from the print request prompt 707 should be printed.

[0042] Other examples of methods for specifying the logs to be printed (print range) include: For example, if the log contains words such as "question and answer" or "exchange," it can be interpreted as an instruction to print pairs of prompts and responses. The range of logs to be printed can also be interpreted by combining words indicating criteria, such as "from the top" or "most recent," with words indicating numbers or quantities. It is also possible to interpret specific logs as targets based on words indicating the content of the prompt or response, such as printing answer sentence 704 with "answer regarding XX." If there are not enough words to specify the print range, the user may be asked questions to fill in the missing words. Alternatively, the user may be provided with multiple options for the print range and allowed to select one.

[0043] In S803, the generation AI server 30 sends a print request to the extended application server 40. At this time, the text data of the answer statements 704 and 705 is also sent. In S804, the extended application server 40 generates print data from the received text data of the answer statements 704 and 705. The print data is in a data format that can be accepted by the cloud print server 50, such as a PDF file. Next, the default print settings are saved as the current print settings. In this embodiment, one piece of print data is generated from multiple pieces of received log data, but one piece of print data may also be generated for each piece of log data.

[0044] In S805, the extended application server 40 transmits a job reservation request to the cloud print server 50. At this time, the print data generated in S804 and the current print settings are transmitted together. In S806, the cloud print server 50 registers a reserved job based on the print data and the current print settings received from the extended application server 40. The cloud print server 50 also transmits a reserved job URL 709 pointing to the reserved job, A preview image 710, which is an output image of the reserved job, is generated and saved. In S807, the cloud print server 50 returns the reserved job URL 709 and preview image 710 to the extended application server 40 as a response to the job reservation request. In S808, the extended application server 40 saves the reserved job URL 709 in association with the JobID, and returns the reserved job URL 709 and preview image 710 to the generation AI server 30 as a response to the print request. In S809, the generation AI server 30 returns the reserved job URL 709 and preview image 710 to the user terminal 20 as a response to the print request prompt 707. The user terminal 20 displays the print request response 708 on the operation panel 201.

[0045] In S810, the user terminal 20 accesses the reserved job URL 709 in accordance with user input via the browser. In S811, the cloud print server 50 returns a login screen (not shown) for the cloud print service to the user terminal 20. In S812, the user terminal 20 accepts login information in accordance with user input via the browser and logs in to the cloud print service. If the user does not have an account for the cloud print service, an account creation operation (not shown) is required. In S813, the cloud print server 50 identifies the MFP used by the user from the logged-in user information. If the user has not registered an MFP, an MFP registration operation (not shown) is required. If multiple MFPs are registered, the user may be allowed to select an MFP through interactive communication (not shown) with the user.

[0046] In S814, the cloud print server 50 submits the reserved job to the identified image forming apparatus 10. In S815, the image forming apparatus 10 executes printing in accordance with the reserved job submitted from the cloud print server 50. In S816, the image forming apparatus 10 notifies the cloud print server 50 that printing of the reserved job has been completed. In S817, the cloud print server 50 returns a print completion screen (not shown) to the user terminal 20 to display it, and this sequence ends.

[0047] <Extended application server processing procedure> The processing procedure of the extended application server 40 according to this embodiment will be described with reference to FIG. 12. The processing described below is realized, for example, by the CPU 401 reading a program stored in the ROM 402 into the RAM 403 and executing it. The processing described below is also realized by executing a plug-in that extends the functionality of the generation AI server 30. In this embodiment, the plug-in (program) is described as being managed and executed by the extended application server 40, but this is not intended to limit the present invention. For example, a plug-in managed by the extended application server 40 may be installed in the generation AI server 30 and executed by the generation AI server 30. Alternatively, the generation AI server 30 and the extended application server 40 may be integrated, in which case the integrated server controls the management, installation, and execution of the plug-in.

[0048] In S1201, the CPU 401 accepts a print request from the generation AI server 30 and acquires text data to be printed. Here, the text data to be printed is, for example, text data of a log that will be the print range. Note that while an example of acquiring log data that will be the print range from the generation AI server 30 is described here, this is not intended to limit the scope of the present invention. For example, the extended application server 40 may identify the print range from the log data in response to a user interaction. In this case, it is desirable for the generation AI server 30 to transmit to the extended application server 40 the storage location of all log data that may be the target and the user's input regarding the print instruction. Based on this information, the extended application server 40 identifies the print range and acquires the necessary log data from the storage location. Alternatively, the extended application server 40 may request information indicating the necessary data from the generation AI server 30 and receive the log data in response.

[0049] In S1202, the CPU 401 generates and saves a unique JobID. In S1203, the CPU 401 converts the text data to be printed into a format acceptable to the cloud print server 50 to generate print data, and saves the print data in memory such as the RAM 403 or HDD 405 in association with the JobID. Note that various saving processes described below are also similarly performed in memory such as the RAM 403 or HDD 405. The memory may be externally connected to the extended application server 40. Furthermore, in S1204, the CPU 401 reads out the default print settings, and saves them as the current print settings in association with the JobID.

[0050] Next, in S1205, the CPU 401 sends a job reservation request to the cloud print server 50. At this time, the CPU 401 sends the converted print data together with the current print settings. If the communication is successful and the reserved job URL 709 and the like are returned from the cloud print server 50, the process proceeds to S1206. On the other hand, if an error notification is received from the cloud print server 50 or if a response is not received within a predetermined time, the process determines that the request has failed and proceeds to S1211.

[0051] In S1206, the CPU 401 saves the reserved job URL 709 in association with the JobID. In S1207, the CPU 401 returns the current print settings, the reserved job URL 709, and the preview image 710 as a response to the generation AI server 30. In S1208, the CPU 401 determines whether a print setting change request has been accepted from the generation AI server 30. If a print setting change request has been accepted, the process proceeds to S1209; if the request has not been accepted, the process of this flowchart ends.

[0052] In S1209, the CPU 401 reads the current print settings associated with the JobID, reflects the requested setting changes, and saves them overwritten. In S1210, the CPU 401 reads the reserved job URL 709 associated with the JobID, and sends a request to change the settings of the reserved job to the cloud print server 50. At this time, the CPU 401 sends both the converted print data and the print settings that reflect the changes. If the communication is successful and the reserved job URL 709, etc. is returned from the cloud print server 50, the process proceeds to S1207. On the other hand, if the communication fails as described above, the process proceeds to S1211. In S1211, the CPU 401 returns a failure message to the generation AI server 30, and ends the processing of this flowchart.

[0053] As described above, the computer program according to this embodiment is a computer program that extends the functionality of a generation AI service and causes a computer to identify a print range upon receiving a print request via the generation AI service. The computer program also causes the computer to generate a print job based on the print request and the print range and register the generated print job on a print server. The computer program also causes the computer to obtain information about the connection destination for executing the print job from the print server and present the print job and information about the connection destination to the user via the generation AI service. This reduces the user's effort when printing information generated by the generation AI on an image forming device. Thus, this embodiment provides a mechanism for optimally printing output from the generation AI on an image forming device.

[0054] <Second embodiment> The following describes a second embodiment of the present invention. In the first embodiment, an example was described in which a print range was acquired based on input content included in a print request from a user. In contrast, in this embodiment, a form will be described in which, after a print request is accepted, multiple options for the print range are provided to the user, and the print range is acquired based on the user's selection.

[0055] <Print instructions> 9 and 10, an explanation will be given of the interactive screen when the extended application server 40 according to this embodiment accepts the selection of the log to be printed from the user via the generation AI server 30. In the interactive screen 900, such a screen is displayed on the operation panel 201 by accessing the generation AI server 30 from a browser running on the user terminal 20. The message text (input content) input and sent from the user terminal 20 and the responses (replies) generated by the generation AI server 30 and the extended application server 40 are displayed in chronological order.

[0056] The prompt input field 901 is an input field for the user to input instructions (prompts) to the generation AI server 30. The user inputs in natural language text format from the operation panel 201 of the user terminal 20. The send button 902 is a button (object) that triggers sending the prompt entered in the prompt input field 901 to the generation AI server 30. Question sentences 903 and 905 are prompts sent from the user terminal 20 requesting a response from the generation AI server 30. Answer sentences 904 and 906 are responses generated by the generation AI server and are answers to the questions asked by the user in question sentences 903 and 905.

[0057] The print request prompt 907 is an example of a prompt for instructing the generation AI server 30 to print. When the generation AI server 30 receives a prompt instructing printing, it transmits multiple log data and a print request to the extended application server 40. In this embodiment, it is assumed that all logs on the same screen are transmitted as the log data to be transmitted. However, a portion of past logs stored in the HDD 305 of the generation AI server 30 may be transmitted as print candidates. Alternatively, the extended application server 40 may be notified of the storage location where the logs (interaction history) are stored, so that the extended application server 40 can access and acquire the necessary log data. Alternatively, a portion of the logs may be transmitted from the generation AI server 30 to the extended application server 40 along with a print request, and the extended application server 40 may access and acquire the missing log information as described above. In this embodiment, the generation AI server 30 is used as an example of the storage location for the logs, but the logs may also be stored in a storage unit of a device separate from the generation AI server 30.

[0058] The print log selection response 908 is a response (selection object) for accepting from the user the selection of the prompt and response log to be printed. The print log selection response 908 is sent from the extended application server to the generation AI server 30 and displayed in the browser on the user terminal 20. 909 is a list of logs, displaying a list of the serial number and opening text for each log received by the extended application server. 910 is a check box for each log, which switches between selected and unselected each time the user makes a selection. The send button 911 is a button for sending a print specification prompt based on the selection state of the check box 910. In this embodiment, an example is shown in which the check boxes are used to simplify log selection. As an alternative method, only the log data list 909 with serial numbers may be sent as a response, and the user may enter the serial number in the prompt input field 901 and send it to accept the selection.

[0059] Reference numeral 912 is a sidebar that manages log data by group of conversations and displays them in a selectable state. Reference numeral 913 is a new conversation button that resets the display on the user's device browser and allows a prompt to be sent as a new conversation. Reference numerals 914 and 915 are objects that show past conversation logs, and display part of the prompt at the beginning of the conversation log and the last update date and time. Operating the object displays the corresponding conversation log on the browser screen, and also makes it possible to send additional prompts. Reference numeral 916 is the currently displayed conversation, and is highlighted (bold framed).

[0060] FIG. 10 shows an interactive screen 1000 displayed on the browser after the Send button 911 in FIG. 9 is operated. The print text selection response 1001 is in the state after the print request has been sent, with the Send button hidden. 1002 is a print instruction prompt, which is automatically sent in response to the operation of the Send button 911 and displayed in the form of a prompt. In this case, the content specifies printing of logs 2 and 4, which were selected in the checkbox 910. The print request response 1003 is a response to the print instruction prompt 1002, which the generation AI server 30 generated in cooperation with the extended application server 40. The print request response 1003 includes settings for the upcoming print, a reserved job URL 1004, and a preview image 1005. The preview image 1005 indicates that only the contents of the log specified in the print instruction prompt 1002 will be included in the output.

[0061] <sequence> With reference to Fig. 11, we will explain the interaction sequence between the system components when a print log is specified using the procedures in Fig. 9 and Fig. 10. The same steps as in Fig. 8 are assigned the same step numbers, and their explanations will be omitted.

[0062] In S1101, the user terminal 20 transmits the content entered in the print request prompt 907 to the generation AI server 30 in accordance with user input via the browser. In S1102, the generation AI server 30 transmits a group of log data candidates for printing to the extended application server 40. The log data group to be transmitted may include all currently displayed log data, or only a portion of the log data may be transmitted, such as the currently displayed data or the dialogue history related to the currently displayed content, to reduce the effort required for specification. Alternatively, the number of log data may be transmitted so that it is below a threshold, or only logs recorded after a certain date and time may be transmitted. Alternatively, the range of logs to be transmitted when no range is specified in the print request prompt may be preset as a setting in the extended application. Furthermore, the content of the prompt instructing printing may be interpreted, and log data belonging to a different conversation may be transmitted as a print candidate. For example, in response to a print prompt such as "Print the conversation about □□□," log data included in the conversation log 914 may be transmitted. Alternatively, in response to a print prompt such as "Print the exchange on October 27," log data included in the conversation log 915 may be transmitted.

[0063] In S1103, the extended application server 40 generates a print log selection response 908 from the log data group and sends it to the generation AI server 30. In S1104, the generation AI server 30 sends the print log selection response 908 to the user terminal 20, and the user terminal 20 displays the print log selection response 908 on the browser.

[0064] In S1105, the user terminal 20 sends print specification information to the generation AI server 30 in response to the operation of the send button 911. In S1106, the generation AI server 30 sends the print specification information to the extended application server 40. In S1107, the extended application server 40 generates print data from the specified log data. The print data is in a data format that can be accepted by the cloud print server 50, such as a PDF file. Next, the default print settings are saved as the current print settings. In this embodiment, one piece of print data is generated from multiple pieces of received log data, but one piece of print data may also be generated for each piece of log data.

[0065] In S1108, the extended application server 40 sends a job reservation request to the cloud print server 50. At this time, the print data generated in S1107 and the current print settings are sent together. The subsequent sequence is the same as the sequence in Fig. 8, so a description thereof will be omitted.

[0066] <Extended application server processing procedure> The processing procedure of the extended application server 40 according to this embodiment will be described with reference to FIG. 13. FIG. 13 shows detailed processing of S1201 in the flowchart of FIG. 12. The processing described below is realized, for example, by the CPU 401 reading a program stored in the ROM 402 into the RAM 403 and executing it. The processing described below is also realized by executing a plug-in that extends the functionality of the generation AI server 30. In this embodiment, the plug-in (program) is described as being managed and executed by the extended application server 40, but this is not intended to limit the scope of the present invention. For example, a plug-in managed by the extended application server 40 may be installed in the generation AI server 30 and executed by the generation AI server 30. Alternatively, the generation AI server 30 and the extended application server 40 may be integrated, in which case the integrated server controls the management, installation, and execution of the plug-in.

[0067] In S1301, the CPU 401 acquires at least a portion of the log data transmitted from the generation AI server 30. As described above, at least a portion of the log data includes the currently displayed dialogue history, the dialogue history related to the currently displayed content, log data within a predetermined threshold, and log data within a predetermined period. In S1302, the CPU 401 determines whether the print range can be identified from the most recent dialogue history. As described above, the print range may be identified using any method, but here it is determined whether the most recent dialogue history contains the information necessary for identification. For example, this determination is made based on whether the dialogue history contains terms that can identify the print range. If the print range can be identified, the process proceeds to S1306; if not, the process proceeds to S1303.

[0068] In S1303, because the CPU 401 cannot identify the print range, it generates a log selection screen (print request response 1003) for the user to specify the print range, and in S1304 sends the information of the generated log selection screen to the generation AI server 30. The log selection screen is displayed on the operation panel 201 of the user terminal 20. In S1305, the CPU 401 determines whether or not a print range specification by the user has been accepted via the log selection screen displayed on the operation panel 201. If a specification has been accepted, the process proceeds to S1306, where the CPU 401 acquires print data corresponding to the print range and ends the processing of this flowchart. Note that in S1306, if the specified print range is within the range of the log data acquired in S1301, the process ends without acquiring new data.

[0069] As described above, the computer program according to this embodiment inquires of the computer of the information processing device, via the generation AI service, about multiple options for print range candidates based on the interaction history between the generation AI service and the user included in the print request. The computer program also instructs the computer to identify one or more print range candidates selected from the multiple options as the print range. This makes it easy to select and print only the necessary information generated by the generation AI.

[0070] The disclosure of this specification includes the following computer program, information processing device, and system. (Item 1) A computer program that enhances the functionality of a generative AI service, comprising: A specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting information about the print job and the connection destination to a user via the generation AI service; A computer program characterized by causing the computer to function as follows. (Item 2) The computer program according to item 1, wherein the specifying means specifies the print range from the interaction history between the generated AI service and the user included in the print request. (Item 3) The computer program described in item 2 is characterized in that the identification means inquires through the generation AI service about multiple options that are candidates for the printing range from the interaction history between the generation AI service and the user included in the printing request, and identifies one or more candidate printing ranges selected from the multiple options as the printing range. (Item 4) Item 3. The computer program of item 3, wherein the plurality of options include input content from a user and response content from the generation AI service. (Item 5) 5. The computer program according to any one of items 2 to 4, wherein the specifying means acquires data to be printed from a storage means that stores the dialogue history for a predetermined period of time. (Item 6) The computer program according to any one of items 1 to 5, wherein the presentation means further presents to the user the contents of the print settings and a preview image of the print generated by the print server as information about the print job. (Item 7) The computer program according to any one of items 1 to 6, characterized in that the generation means converts the text data included in the print request received via the generation AI service into a format acceptable to the print server to generate print data. (Item 8) 8. The computer program according to any one of items 1 to 7, wherein the generating means sets predetermined print settings for the generated print job. (Item 9) The computer program according to any one of items 1 to 8, characterized in that the computer program is implemented in the computer as a plug-in for the generation AI service. (Item 10) An information processing device that executes a computer program that expands the functionality of a generation AI service, A specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting information about the print job and the connection destination to a user via the generated AI service; An information processing device comprising: (Item 11) A printing system including an information processing device that provides a generation AI service to a user terminal, a print server, and an image forming device, The information processing device executes a computer program that expands the functionality of the generation AI service, A specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting information about the print job and the connection destination to a user via the generation AI service; It functions as The print server a providing means for providing information about the connection destination when the print job is registered; an execution means for inputting the print job to the image forming apparatus when a user accesses the information about the connection destination; A printing system comprising: (Item 12) Item 12. The printing system according to item 11, wherein the print server is a cloud print server.

[0071] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0072] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0073] 10: MFP, 20: User terminal, 30: Generation AI server, 40: Extended application server, 50: Cloud print server, 60: Cloud, 70: Network

Claims

1. A computer program for enhancing the functionality of a generation AI service, comprising: a specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting the print job and information about the connection destination to a user via the generation AI service; A computer program characterized by causing the computer to function as follows.

2. 2. The computer program according to claim 1, wherein the specifying unit specifies the print range from a dialogue history between the generated AI service and the user included in the print request.

3. The computer program described in claim 1, characterized in that the identification means inquires through the generation AI service about multiple options that can be candidates for the printing range from the interaction history between the generation AI service and the user included in the printing request, and identifies one or more candidate printing ranges selected from the multiple options as the printing range.

4. 4. The computer program of claim 3, wherein the plurality of options include input content from a user and response content from the generation AI service.

5. 5. The computer program according to claim 2, wherein the specifying unit acquires data to be printed from a storage unit that stores the dialogue history for a predetermined period of time.

6. 6. The computer program according to claim 5, wherein the presenting unit further presents to the user print setting contents and a print preview image generated by the print server as the print job information.

7. The computer program according to claim 6, characterized in that the generation means generates print data by converting text data included in the print request accepted via the generation AI service into a format acceptable to the print server.

8. 8. The computer program according to claim 7, wherein the generating means sets predetermined print settings for the generated print job.

9. The computer program according to claim 5 , wherein the computer program is implemented on the computer as a plug-in for the generation AI service.

10. An information processing device that executes a computer program that expands the functionality of a generation AI service, a specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting information about the print job and the connection destination to a user via the generation AI service; An information processing device comprising:

11. A printing system including an information processing device that provides a generation AI service to a user terminal, a print server, and an image forming device, The information processing device executes a computer program that expands the functionality of the generation AI service, a specifying means for specifying a print range when a print request is received via the generating AI service; a generating means for generating a print job based on the print request and the print range; a reservation means for registering the generated print job in a print server; an acquisition unit that acquires information about a connection destination for executing the print job from the print server; a presentation means for presenting the print job and information about the connection destination to a user via the generation AI service; It functions as The print server a providing means for providing information about the connection destination when the print job is registered; an execution means for inputting the print job to the image forming apparatus when a user accesses the information about the connection destination; A printing system comprising:

12. The printing system according to claim 11 , wherein the print server is a cloud print server.

Images