Storage medium, information processing apparatus, printing system, and computer program product

Abstract

The invention provides a storage medium, an information processing apparatus, a printing system, and a computer program product. A non-transitory computer-readable storage medium stores a computer program that extends a function of a generative AI service, the computer program causing a computer of an information processing apparatus to specify a print range upon receipt of a print request via the generative AI service, and to perform a function of extending the function of the generative AI service on the basis of the print request and the print range. The method includes generating a print job, registering the generated print job to a print server, obtaining information related to a connection destination for executing the print job from the print server, and presenting the print job and the information related to the connection destination to a user via the generative AI service.

Description

Technical Field

[0001] The present invention relates to a storage medium, an information processing apparatus, a printing system, and a computer program product for expanding the functions of an artificial intelligence system. Background Art

[0002] Artificial intelligence systems known as generative AI can generate text, images, or other media in response to a prompt. Regarding generative AI such as , plugins can be used to expand the functions. For example, when a statement including the keyword "print" is input to the generative AI as an instruction, a print-related plugin is selected, and the plugin performs processing that the generative AI cannot answer. As an alternative, in this way, the expansion of functions is achieved.

[0003] Meanwhile, regarding an image forming apparatus, printing can be performed by inputting print data from a cloud server. For example, Japanese Unexamined Patent Application Publication No. 2018-128843 proposes a technique that enables printing a document managed in a cloud server in a chat-related session form.

[0004] However, in the case of printing an output such as text generated by generative AI using an image forming apparatus, a user or the like needs to copy a file or the like and perform printing from, for example, a print driver. That is, in order to perform printing, the user needs to trouble to copy the output generated by generative AI. Summary of the Invention

[0005] The present invention can implement a mechanism that enables an image forming apparatus to appropriately print an output of generative AI.

[0006] One aspect of the present invention provides a non-transitory computer-readable storage medium that stores a computer program for expanding the functions of a generative AI service. The computer program causes a computer of an information processing apparatus to perform the following operations: when a print request is received via the generative AI service, specify a print range, generate a print job based on the print request and the print range, register the generated print job in a print server, obtain information related to a connection destination for executing the print job from the print server, and display the print job and the information related to the connection destination to a user via the generative AI service.

[0007] Another aspect of the present invention provides an information processing apparatus that executes a computer program for expanding the functions of a generative AI service. The information processing apparatus includes: one or more memory devices that store an instruction set; and one or more processors that execute the instruction set to perform the following operations: when a print request is received via the generative AI service, specify a print range, generate a print job based on the print request and the print range, register the generated print job in a print server, obtain information related to a connection destination for executing the print job from the print server, and display the print job and the information related to the connection destination to a user via the generative AI service.

[0008] Another aspect of the present invention provides a printing system that includes an information processing apparatus, a print server, and an image forming apparatus. The information processing apparatus provides a generative AI service to a user terminal and includes: one or more first memory devices that store an instruction set of a computer program for expanding the functions of the generative AI service; and one or more first processors that execute the instruction set to perform the following operations: when a print request is received via the generative AI service, specify a print range, generate a print job based on the print request and the print range, register the generated print job in the print server, obtain information related to a connection destination for executing the print job from the print server, and display the print job and the information related to the connection destination to a user via the generative AI service. The print server includes: one or more second memory devices that store an instruction set; and one or more second processors that execute the instruction set to perform the following operations: when the print job has been registered, provide information related to the connection destination, and when a user has accessed the information related to the connection destination, input the print job to the image forming apparatus.

[0009] Another aspect of the present invention provides a computer program product that stores a computer program for expanding the functions of a generative AI service. The computer program causes a computer of an information processing apparatus to perform the following operations: when a print request is received via the generative AI service, specify a print range, generate a print job based on the print request and the print range, register the generated print job in a print server, obtain information related to a connection destination for executing the print job from the print server, and display the print job and the information related to the connection destination to a user via the generative AI service.

[0010] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a diagram showing the system configuration according to an embodiment.

[0012] Figure 2 This is a diagram showing the configuration of an MFP according to an embodiment.

[0013] Figure 3 This is a diagram showing the configuration of a user terminal according to an embodiment.

[0014] Figure 4 This is a diagram showing the configuration of a generative AI server according to an embodiment.

[0015] Figure 5 This is a diagram showing the configuration of an extended application server according to an embodiment.

[0016] Figure 6 This is a diagram showing the configuration of a cloud printing server according to an embodiment.

[0017] Figure 7 This is an example of a screen that is displayed on a user terminal according to an embodiment and shows a conversation with a generative AI server.

[0018] Figure 8 This is a sequence diagram of document printing according to an embodiment.

[0019] Figure 9 This is an example of a screen that is displayed on a user terminal according to an embodiment and shows a conversation with a generative AI server.

[0020] Figure 10 This is an example of a screen that is displayed on a user terminal according to an embodiment after specifying a print log.

[0021] Figure 11 This is a sequence diagram of document printing according to an embodiment.

[0022] Figure 12 This is a flowchart related to an extended application server according to an embodiment.

[0023] Figure 13 This is a flowchart related to an extended application server according to an embodiment. Detailed Description of the Embodiment

[0024] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the following embodiments are not intended to limit the scope of the present invention. In the embodiments, multiple features are described, but the invention is not limited to inventions that require all of these features, and multiple such features can be appropriately combined. In addition, in the drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0025] <First Embodiment>

[0026] <System Configuration>

[0027] The first embodiment of the present invention will be described below. This embodiment will be described by taking a multi-functional peripheral device (MFP) having a printing function, a scanning function, and a fax function as an example of the image processing apparatus. Figure 1 An example of an overall configuration diagram of a printing system that provides a printing service via a generative AI service according to the present invention is shown. This printing system is configured to include an image forming apparatus 10 (i.e., a printing apparatus, etc.), a user terminal 20, a generative AI server 30, an extended application server 40, and a cloud printing server 50. They are interconnected via a network 70 in a manner that allows them to communicate with each other. The network 70 is a wireless or wired network composed of a WAN or a LAN.

[0028] The image forming apparatus 10 represents a printing apparatus having a function of printing an image notified by the cloud printing server 50. When a user uses a generative AI service such as ChatGPT, the user terminal 20 represents an information terminal such as a smartphone, a tablet terminal, and a personal computer. The user accesses the generative AI server 30 in the cloud by operating the user terminal 20 and executes a generative AI application. The user terminal 20 and the generative AI server 30 are connected to a wired or wireless communication network and can send / receive data to / from each other. Although a public line on the Internet is assumed as an example of the communication network in this embodiment, a dedicated line may also be used.

[0029] The generative AI server 30 is a cloud server (information processing apparatus) arranged in the cloud 60 and provides a generative AI such as ChatGPT (i.e., text generative AI). The generative AI server 30 provides services in cooperation with the extended application server 40. The generative AI server 30 interprets a message sent from the user terminal 20, generates an appropriate response, and displays the response on the screen of the user terminal 20 as a reply. In addition, the generative AI server 30 can expand its functions by communicating with the extended application server 40.

[0030] The extended application server 40 is a cloud server (information processing device) arranged in the cloud 60 and has a function of providing additional functions (plugins) to the generative AI server 30. The generative AI server 30 cooperates with the extended application server 40 to incorporate a predetermined plugin; in this way, processing that cannot be performed only by itself can be executed. When the cloud printing server 50 receives a print request from the user terminal 20, it notifies the image forming device 10 of the arrival of the print job. The image forming device 10 receives the print job from the cloud printing server 50 based on the notification from the cloud printing server 50 and executes printing. Note that although this description is provided using an example in which each type of server is provided separately, each type of server can be provided integrally with other servers. For example, the generative AI server 30 and the extended application server 40 can be provided integrally.

[0031] <Hardware Configuration of Image Forming Device>

[0032] Reference will be made to Figure 2 An example of the hardware configuration of the image forming device 10 according to the present embodiment will be described. The image forming device 10 includes a control unit 110, an operation unit 116, a reading unit 118, a printing unit 120, a wireless communication unit 122, a facsimile communication unit 124, and a communication unit 126. The control unit 110 includes a CPU 111, a ROM 112, a RAM 113, an HDD 114, an operation unit I / F 115, a reading unit I / F 117, a printing unit I / F 119, a wireless communication unit I / F 121, a facsimile unit I / F 123, and a communication unit I / F 125.

[0033] The control unit 110 including the CPU 111 controls the operation of the entire image forming device 10. The CPU 111 reads out the control program stored in the ROM 112 or the HDD 114 into the RAM 113 and performs various types of control such as reading control and printing control. The ROM 112 stores the control program that can be executed by the CPU 111. In addition, the ROM 112 also stores a startup program, font data, etc. The RAM 113 is a main storage memory and is used to expand various types of control programs stored in the ROM 112 and the HDD 114 in a temporary storage area such as a work area. The HDD 114 stores image data, print data, various types of programs, various types of addresses, and various types of setting information. The HDD 114 is a storage medium and can be a solid state drive (SSD), an embedded multimedia card (eMMC), etc. instead of the HDD.

[0034] Note that, although it is assumed that the image forming apparatus 10 according to the present embodiment performs various types of processing shown in the flowcharts described later when one CPU 111 uses one memory (RAM 113), this is not limitative. For example, various types of processing may be performed by causing a plurality of CPUs, RAMs, ROMs, and HDDs to cooperate with each other. Further, part of the processing may be performed using hardware circuits such as ASICs and FPGAs.

[0035] The operation unit I / F 115 is connected between the operation unit 116 and the control unit 110. The operation unit 116 includes, for example, a display unit such as a touch panel and hardware keys. The operation unit 116 displays information for the user and detects the user's input. The reading unit I / F 117 is connected between the reading unit 118 (e.g., similar to a scanner) and the control unit 110. The reading unit 118 reads an image of a document, and the CPU 111 converts the image into image data such as binary data. The image data generated based on the image read by the reading unit 118 is sent to an external device and printed on a recording sheet. The printing unit I / F 119 is connected between the printing unit 120 (e.g., similar to a printer) and the control unit 110. The CPU 111 transfers the image data (print data) stored in the RAM 113 to the printing unit 120 via the printing unit I / F 119. The printing unit 120 prints an image based on the transferred image data on a recording sheet fed from a paper feed cassette.

[0036] The wireless communication unit I / F 121 is an I / F for controlling the wireless communication unit 122 and wirelessly connects the control unit 110 to an external wireless device. The facsimile unit I / F 123 is connected to the public network 80 by controlling the facsimile communication unit 124. The facsimile unit I / F 123 is an I / F for controlling the facsimile communication unit 124 and can be connected to the public network, control a facsimile communication protocol, etc. by controlling a modem and an NCU (network control unit) for facsimile communication.

[0037] The communication unit I / F 125 is connected between the control unit 110 and the network 70. The communication unit I / F 125 enables the communication unit 126 to send image data and various types of information inside the device to an external device in the network 70, and receive print data and information in the network 70 from an external device in the network 70. As a method of sending / receiving via the network 70, sending / receiving using e-mail can be performed, as well as file sending using other protocols (such as FTP, SMB, WEBDAV, etc.). In addition, image data and message data can be sent / received in the network 70 by accessing from the user terminal 20 and the generative AI server 30 via HTTP communication. In addition, various types of settings (removing the UI function) of the image forming device 10 can be configured by accessing the image forming device 10 from a web browser of the user terminal 20 via the network.

[0038] <Hardware Configuration of User Terminal>

[0039] Reference will be made to Figure 3 Describe an example of the hardware configuration of the user terminal 20 according to the present embodiment. Note that although it is assumed that the user terminal 20 according to the present embodiment is a device such as a smartphone or a tablet PC, the user terminal 20 can be other devices as long as the user terminal is an information processing device that can be connected to the network 70 using, for example, Wi-Fi communication. The user terminal 20 includes an operation panel 201, a camera 204, an NFC communication unit 205, a communication unit 206, a CPU 207, a ROM 208, a RAM 209, an HDD 210, and a wireless LAN communication unit.

[0040] The CPU 207 reads out the control program stored in the ROM 208 and executes various types of processing for controlling the operation of the user terminal 20. The ROM 208 stores the above control program. The RAM 209 is used as a temporary storage area such as a main memory and a work area for the CPU 207. The HDD 210 stores various types of data such as photos and electronic documents.

[0041] The operation panel 201 has a touch panel function capable of detecting a user's touch operation and displays various types of screens provided by the OS and the e-mail sending application. In addition, the operation panel 201 is also used to confirm the information stored in the generative AI server 30. The user can input a desired operation instruction to the user terminal 20 by inputting a touch operation to the operation panel 201. Note that the user terminal 20 includes hardware keys (not shown), and the user can also use these hardware keys to input operation instructions to the user terminal 20.

[0042] The camera 204 captures an image according to a shooting instruction from the user. The photo captured by the camera 204 is stored in a predetermined area in the HDD 210. In addition, information can be obtained from the QR code read by the camera 204 using a program capable of analyzing the QR code.

[0043] The user terminal 20 can exchange data with various types of peripheral devices via the NFC communication unit 205, the Bluetooth communication unit 206, and the wireless LAN communication unit 211. The Bluetooth communication unit 206 of the user terminal 20 can support Bluetooth Low Energy (BLE).

[0044] <Hardware Configuration of the Generative AI Server>

[0045] Reference will be made to Figure 4 Describe an example of the hardware configuration of the generative AI server 30 according to this embodiment. The generative AI server 30 includes a CPU 301, a ROM 302, a RAM 303, a communication unit 304, and an HDD 305.

[0046] The CPU 301 uses the control program stored in the ROM 302 and the training model stored in the HDD 305 to execute processing for controlling operations to generate appropriate responses. The ROM 302 stores the above control program. The RAM 303 serves as a temporary storage area such as a main memory and a work area for the CPU 301. The HDD 305 stores various types of data (such as logs of messages sent from the training model, generative AI applications, and the user terminal 20) and generated responses.

[0047] Data can be exchanged with various types of devices (such as the user terminal 20, the image forming apparatus 10, and the extended application server 40) via the communication unit 304. Note that the communication unit 304 can use for wired communication, or can perform wireless communication such as Wi-Fi.

[0048] <Hardware Configuration of the Extended Application Server>

[0049] Reference will be made to Figure 5 Describe an example of the hardware configuration of the extended application server 40. The extended application server 40 includes a CPU 401, a ROM 402, a RAM 403, a communication unit 404, and an HDD 405.

[0050] The CPU 401 reads out the control program stored in the ROM 402 and executes processing according to the message received from the generative AI server 30. The ROM 402 stores the above control program. The RAM 403 is used as a temporary storage area such as a main memory and a work area for the CPU 401. The HDD 405 stores, for example, the content or a part of the message received from the generative AI server 30. Data can be sent / received to / from various types of devices such as the generative AI server 30 via the communication unit 404.

[0051] <Hardware Configuration of Cloud Print Server>

[0052] Reference will be made to Figure 6 Describe an example of the hardware configuration of the cloud print server 50. The cloud print server 50 includes a CPU 501, a ROM 502, a RAM 503, a communication unit 504, and an HDD 505.

[0053] The CPU 501 reads out the control program stored in the ROM 502 and enables the reception of print jobs from the user terminal 20 and the acquisition of stored print jobs. The ROM 502 stores the above control program. The RAM 503 is used as a temporary storage area such as a main memory and a work area for the CPU 501. The HDD 505 stores, for example, the print jobs received from the user terminal 20. Data can be sent / received to / from various types of devices such as the generative AI server 30 via the communication unit 504.

[0054] <Print Instruction>

[0055] Now refer to Figure 7 Describe the conversation between the user and the generative AI server 30 when instructing the generative AI server 30 according to this embodiment to perform printing. Accessing the generative AI server 30 from the browser operated on the user terminal 20 causes the operation panel 201 to display the screen 700. The screen 700 displays, in chronological order (in the form of a conversation), the message sentences input from the user terminal 20 and the responses generated by the generative AI server 30.

[0056] The prompt input field 701 is an input field for the user to input an instruction sentence (prompt) to the generative AI server 30. The user inputs in the form of natural language text from the operation panel 201 of the user terminal 20. The user can input the instruction sentence by operating the touch panel of the user terminal 20. The send button 702 is a button that serves as a trigger for sending the prompt input to the prompt input field 701 to the generative AI server 30.

[0057] The inquiry sentences 703 and 705 are prompts that have been sent from the user terminal 20 and request a response from the generative AI server 30. The response sentences 704 and 706 are responses generated by the generative AI server and are answers corresponding to the content of the inquiries made by the user in the inquiry sentences 703 and 705.

[0058] The print request prompt 707 is an example of a prompt that instructs the generative AI server 30 to perform printing. This prompt needs to include: the range of the log to be printed by the generative AI server 30 and an indication to perform printing. In the present embodiment, the content of this prompt represents an instruction to print the response sentences 704 and 706 (i.e., the two immediately preceding responses). Although it is assumed here that the response sentences to be printed by instruction and the print request prompt are in the same session log, past logs stored in the HDD 305 of the generative AI server 30 can be the objects to be printed.

[0059] The print request response 708 is a response to the print request prompt 707 and is generated by the generative AI server 30 in cooperation with the extended application server 40. The print request response 708 includes: settings for the printing to be performed from now on, and a reservation job URL 709 and a preview image 710 to be described later.

[0060] The reservation job URL 709 is a URL pointing to a print job stored in the cloud print server 50. The user can perform cloud printing (input the print job from the cloud print server 50 into the image forming apparatus 10) by accessing this URL. In addition, in the case where the print settings of the reservation job are also to be changed, the change is made by sending an HTTP request to the reservation job URL. The preview image 710 is a preview image showing an image of the output material in the case of printing under the current print settings. This image enables the user to understand the approximate result of the output material before printing.

[0061] <Sequence>

[0062] Now refer to Figure 8 Describe the conversation between the system components according to the present embodiment after the user instructs the generative AI server 30 to perform printing until the print job is actually input into the image forming apparatus 10. Note that in the sequence described below, in the case where the generative AI server 30 and the extended application server 40 are integrally provided, these two devices are controlled in an integrated manner.

[0063] In step S801, the user terminal 20 sends the content input to the print request prompt 707 according to the user input via the browser to the generative AI server 30. Subsequently, in step S802, the generative AI server 30 interprets the content of the print request prompt 707 and designates the answer sentences 704 and 706 as the logs to be printed, that is, the print range. In the case of the print request prompt 707, only the logs interpreted as responses based on "answer" are applicable, and the two logs to be printed in front of the print request prompt 707 based on "these two".

[0064] The following examples can be used as other methods for designating the logs (print range) to be printed. For example, in the case of including terms such as "question and answer" and "conversation", it can be interpreted that an instruction to print a pair of prompts and responses has been issued. In addition, the range of the logs to be printed can be interpreted by a combination of terms indicating standards (such as "from the top" and "latest") and terms indicating numbers and quantities. In addition, as in the case of printing the answer sentence 704 based on "answer related to ○○", specific logs can also be interpreted as objects based on the terms indicating the content of the prompt or response. Note that in the case where the terms for designating the print range are insufficient, the user can be asked to compensate for the insufficient terms. Alternatively, multiple options for the print range can be provided to the user and the user can be asked to select from them.

[0065] In step S803, the generative AI server 30 sends a print request to the extended application server 40. At this time, the text data of the answer sentences 704 and 705 are sent together. In step S804, the extended application server 40 generates print data based on the received text data of the answer sentences 704 and 705. The print data adopts a data format acceptable to the cloud print server 50 and can be, for example, a PDF file. Next, the default print settings are stored as the current print settings. Although in this embodiment, one print data is generated based on the received multiple log data, one print data can be generated for one log data.

[0066] In step S805, the extended application server 40 sends a job reservation request to the cloud printing server 50. At this time, the print data generated in step S804 and the current print settings are sent together. In step S806, the cloud printing server 50 registers the reservation job based on the print data and the current print settings received from the extended application server 40. In addition, the cloud printing server 50 generates and stores a reservation job URL 709 and a preview image 710 (i.e., an image of the output material of the reservation job) pointing to the reservation job. In step S807, as a response to the job reservation request, the cloud printing server 50 returns the reservation job URL 709 and the preview image 710 to the extended application server 40. In step S808, the extended application server 40 stores the reservation job URL 709 in association with the job ID, and returns the reservation job URL 709 and the preview image 710 to the generative AI server 30 as a response to the print request. In step S809, the generative AI server 30 returns the reservation job URL 709 and the preview image 710 to the user terminal 20 as a response to the print request prompt 707. The user terminal 20 displays the print request response 708 on the operation panel 201.

[0067] In step S810, the user terminal 20 accesses the reservation job URL 709 according to the user input via the browser. In step S811, the cloud printing server 50 returns a login screen (not shown) of the cloud printing service to the user terminal 20. In step S812, the user terminal 20 accepts the login information according to the user input via the browser and logs in to the cloud printing service. Note that in the case where the user does not have an account for the cloud printing service, a task of creating an account (not shown) is required. In step S813, the cloud printing server 50 specifies the MFP used by the user according to the information of the logged-in user. Note that in the case where the user has not registered the MFP, a task of registering the MFP (not shown) is required. In addition, in the case where multiple MFPs are registered, the MFP can be selected by interacting with the user (not shown).

[0068] In step S814, the cloud printing server 50 inputs the reservation job to the specified image forming apparatus 10. In step S815, the image forming apparatus 10 performs printing according to the reservation job input from the cloud printing server 50. In step S816, the image forming apparatus 10 notifies the cloud printing server 50 that the printing of the reservation job has been completed. In step S817, the cloud printing server 50 returns a print completion screen (not shown) to the user terminal 20, causes the print completion screen to be displayed, and ends this sequence.

[0069] <Processing procedure of the extended application server>

[0070] Will refer to Figure 12Describe the processing procedure of the extended application server 40 according to this embodiment. For example, the program stored in the ROM 402 is read out to the RAM 403 by the CPU 401 and the program is executed to implement the following processing. In addition, the following processing is implemented by executing a plug-in that implements the functions of the extended generative AI server 30. This embodiment will be described under the following assumption: the plug-in (program) is managed and executed by the extended application server 40; however, this is not intended to limit the present invention. For example, the plug-in managed by the extended application server 40 can be installed on the generative AI server 30 and executed on the generative AI server 30. Alternatively, the generative AI server 30 and the extended application server 40 can be integrally provided; in this case, the integrated server controls the management, installation, and execution of the plug-in.

[0071] In step S1201, the CPU 401 receives a print request from the generative AI server 30 and obtains the text data to be printed. Here, the text data to be printed is, for example, the text data representing the log of the print range. Note that an example of obtaining the log data representing the print range from the generative AI server 30 is described below; however, this is not intended to limit the present invention. For example, the extended application server 40 can specify the print range corresponding to the user dialogue based on the log data. In this case, it is desirable for the generative AI server 30 to send to the extended application server 40: the storage location of all data of the log that can be an object, and the content of the user input related to the print instruction. Based on this information, the extended application server 40 specifies the print range and obtains the necessary log data from the storage location. Alternatively, a request for information indicating the necessary data can be sent to the generative AI server 30, and the log data can be received as a reply thereto.

[0072] In step S1202, the CPU 401 generates and stores a unique job ID. In step S1203, the CPU 401 generates print data by converting the text data to be printed into a format acceptable to the cloud print server 50, and stores the print data in association with the job ID in a memory such as the RAM 403 and the HDD 405. Note that various types of storage processing described below are similarly performed on memories such as the RAM 403 and the HDD 405. The memory can be an external memory connected to the extended application server 40. In addition, in step S1204, the CPU 401 reads out the default print settings and stores them in association with the job ID as the current print settings.

[0073] Next, in step S1205, the CPU 401 sends a job reservation request to the cloud printing server 50. At this time, the CPU 401 sends the converted print data together with the current print settings. When the communication is successful and a reservation job URL 709, etc., is returned from the cloud printing server 50, the process proceeds to step S1206. On the other hand, when the cloud printing server 50 provides an error notification, or when a response is not received within a predetermined time period, it is determined that the communication has failed, and the process proceeds to step S1211.

[0074] In step S1206, the CPU 401 stores the reservation job URL 709 in association with the job ID. In step S1207, the CPU 401 returns the current print settings, the reservation job URL 709, and the preview image 710 as a response to the generative AI server 30. In step S1208, the CPU 401 determines whether a change request for the print settings has been received from the generative AI server 30. If a change request for the print settings has been received, the process proceeds to step S1209; if the request has not been received, the processing of this flowchart ends.

[0075] In step S1209, the CPU 401 reads out the current print settings associated with the job ID, reflects the requested setting changes therein, and stores the resulting print settings by overwriting. In step S1210, the CPU 401 reads out the reservation job URL 709 associated with the job ID and sends a change request for the reservation job settings to the cloud printing server 50. At this time, the CPU 401 sends the converted print data together with the print settings reflecting the changes. When the communication is successful and the reservation job URL 709, etc., is returned from the cloud printing server 50, the process proceeds to step S1207. On the other hand, when the communication fails in the above manner, the process proceeds to step S1211. In step S1211, the CPU 401 returns a failure message to the generative AI server 30 and ends the processing of this flowchart.

[0076] As described above, the computer program according to the present embodiment is a computer program that extends the functions of a generative AI service, and causes a computer to operate to specify a print range when a print request is received via the generative AI service. In addition, this computer program causes the computer to operate based on the print request and the print range to generate a print job, and registers the generated print job in a print server. Further, this computer program also causes the computer to operate to obtain information related to a connection destination for executing the print job from the print server, and to present the print job and the information related to the connection destination to the user via the generative AI service. This can reduce the trouble encountered by the user when printing information generated by generative AI on an image forming apparatus. In this way, the present embodiment can provide a mechanism for appropriately printing the output of generative AI by the image forming apparatus.

[0077] <Second Embodiment>

[0078] A second embodiment of the present invention will be described below. In the above-described first embodiment, an example of obtaining a print range based on input content included in a print request from a user was described. On the other hand, this embodiment will be described in relation to a mode in which, after a print request is accepted, a plurality of options are provided to the user as a print range, and the print range is obtained based on the user's selection.

[0079] <Print Instruction>

[0080] Now refer to Figure 9 and Figure 10 Describe a dialogue screen in the case where an extended application server 40 according to the present embodiment accepts a selection of a log to be printed from a user via a generative AI server 30. Accessing the generative AI server 30 from a browser operating on the user terminal 20 causes the operation panel 201 to display the dialogue screen 900. The interactive screen displays, in chronological order: message sentences (input content) input and sent by the user terminal 20, and responses (replies) generated by the generative AI server 30 and the extended application server 40.

[0081] The prompt input field 901 is an input field for the user to input an instruction sentence (prompt) to the generative AI server 30. The user inputs in the form of natural language text from the operation panel 201 of the user terminal 20. The send button 902 is a button (object) that serves as a trigger for sending the prompt input to the prompt input field 901 to the generative AI server 30. The inquiry sentences 903 and 905 are prompts that have been sent from the user terminal 20 and request a reply from the generative AI server 30. The answer sentences 904 and 906 are responses generated by the generative AI server, and are answers corresponding to the content of the inquiries made by the user in the inquiry sentences 903 and 905.

[0082] The print request prompt 907 is an example of a prompt instructing the generative AI server 30 to perform printing. When the generative AI server 30 receives a prompt for issuing a print instruction, it sends a print request and multiple log data to the extended application server 40. Although in this embodiment it is assumed that each log on the same screen is sent as the transmitted log data, a part of the past logs stored in the HDD 305 of the generative AI server 30 may be sent as print candidates. Alternatively, the storage location of the stored logs (conversation history) may be notified to the extended application server 40, and the extended application server 40 may obtain the necessary log data by accessing the storage location. In addition, the generative AI server 30 may send a part of the logs together with the print request to the extended application server 40, and the extended application server 40 may obtain the lacking log information by accessing the storage location of the above logs. Although in this embodiment the generative AI server 30 is an example of the log storage location, the logs may be stored in a storage unit of another device different from the generative AI server 30.

[0083] The print log selection response 908 is a response (selection object) for accepting from the user the selection of the logs of the prompts and responses to be printed. The print log selection response 908 is sent from the extended application server to the generative AI server 30 and is displayed on the browser of the user terminal 20. 909 is a list of the logs, and the serial number and the beginning of the text of each log received by the extended application server are displayed as the list. 910 represents a checkbox for each log, and each time the user makes a selection, the checkbox alternates between the selected state and the unselected state. The send button 911 is a button for sending the print-specified prompt based on the selection state of the checkbox 910. This embodiment gives an example of facilitating log selection through checkboxes. As another example, only the list 909 of the log data having serial numbers may be sent as a response, and user selection may be accepted by having the user input the serial number into the prompt input field 901 and sending the serial number.

[0084] 912 is a sidebar, and the log data is displayed in a state where the log data is managed in units of session groups and can be selected. 913 is a new session button, and it enables resetting the display on the browser of the user terminal and sending a prompt as a new session. 914 and 915 are objects indicating the past session logs, and a part of the prompt indicating the beginning of the session log, as well as the date and time of the last update, are displayed. Operating these objects enables the corresponding session log to be displayed on the browser screen and also enables sending additional prompts. 916 is the currently displayed session, and it is displayed with emphasis (bold frame).

[0085] Figure 10 Shown in Figure 9The dialogue screen 1000 displayed on the browser after operating the send button 911 in the middle. The print text selection response 1001 is in the state after sending a print request, and the send button is not displayed therein. 1002 is a print specification prompt that is automatically sent and displayed in the form of a prompt in response to the operation on the send button 911. Here, its content specifies printing the second log and the fourth log, and these two logs are placed in the selected state via the checkbox 910. The print request response 1003 is a reply to the print specification prompt 1002 and is generated by the generative AI server 30 in cooperation with the extended application server 40. The print request response 1003 includes the settings for the printing to be performed from now on, the reservation job URL 1004, and the preview image 1005. The preview image 1005 indicates that the content of the output material only includes the logs specified by the print specification prompt 1002.

[0086] <Sequence>

[0087] Now refer to Figure 11 Describe in Figure 9 and Figure 10 During the process of Figure 8 The sequence of the dialogue between the system components in the case of specifying the print log. The same step numbers are given to the processes similar to

[0088] In step S1101, the user terminal 20 sends the content input to the print request prompt 907 to the generative AI server 30 according to the user input via the browser. In step S1102, the generative AI server 30 sends the log data group used as the print candidate to the extended application server 40. Regarding the sent log data group, all the log data currently displayed can be sent, or considering the trouble of specification, only part of the log data can be sent, such as only sending the currently displayed data and only sending the conversation history related to the currently displayed content. Alternatively, the number of sent log data can be equal to or less than the threshold, or only the logs with the recording time and date on or after a specific date and time are sent. Alternatively, the range of the logs sent in the case where the print request prompt does not specify a range can be preset as the setting of the extended application. In addition, the content of the prompt for issuing the print instruction can be interpreted, and the log data belonging to different sessions can be sent as the print candidate. For example, in response to the print prompt indicating "print the session about □□□", the log data included in the session log 914 can be sent. Alternatively, in response to the print prompt indicating "print the communication on October 27th", the log data included in the session log 915 can be sent.

[0089] In step S1103, the extended application server 40 generates a print log selection response 908 based on the log data group and sends it to the generative AI server 30. In step S1104, the generative AI server 30 sends the print log selection response 908 to the user terminal 20, and the user terminal 20 displays the print log selection response 908 on the browser.

[0090] In step S1105, the user terminal 20 sends print specification information to the generative AI server 30 in response to an operation on the send button 911. In step S1106, the generative AI server 30 sends the print specification information to the extended application server 40. In step S1107, the extended application server 40 generates print data based on the specified log data. The print data is in a data format acceptable to the cloud printing server 50 and can be, for example, a PDF file. Next, the default print settings are stored as the current print settings. Although in this embodiment, one print data is generated based on multiple received log data, one print data can be generated for one log data.

[0091] In step S1108, the extended application server 40 sends a job reservation request to the cloud printing server 50. At this time, the print data generated in step S1107 and the current print settings are sent together. From here on, the process of this sequence is similar to Figure 8 the process of the sequence of

[0092] <Processing procedure of the extended application server>

[0093] Reference will be made to Figure 13 describe the processing procedure of the extended application server 40 according to this embodiment. Figure 13 is Figure 12 the detailed processing of step S1201 in the flowchart of

[0094] In step S1301, the CPU 401 obtains at least a part of the log data sent from the generative AI server 30. As described above, at least a part of the log data represents the currently displayed conversation history, the conversation history related to the currently displayed content, the log data falling within a predetermined threshold, the log data within a predetermined time period, and the like. In step S1302, the CPU 401 determines whether a print range can be specified from the latest conversation history. As described above, any method can be used to specify the print range; however, here it is determined whether the information required for the specification is included in the latest conversation history. For example, this determination is made based on whether the conversation history includes terms that can be used to specify the print range. In the case where the print range can be specified, the process proceeds to step S1306; otherwise, the process proceeds to step S1303.

[0095] In step S1303, since the print range cannot be specified, the CPU 401 generates a log selection screen (print request response 1003) for the user to specify the print range, and in step S1304, sends the information of the generated log selection screen to the generative AI server 30. The log selection screen is displayed on the operation panel 201 of the user terminal 20. In step S1305, the CPU 401 determines whether the user's specification of the print range has been accepted via the log selection screen displayed on the operation panel 201. If the specification has been accepted, the process proceeds to step S1306, and the CPU 401 obtains the print data corresponding to the print range, and ends the processing of this flowchart. Note that in step S1306, if the specified print range is within the range of the log data obtained in step S1301, the processing ends without re-obtaining the data.

[0096] As described above, the computer program according to the present embodiment causes the computer of the information processing apparatus to query, via the generative AI service, a plurality of options that are candidates for the print range in the conversation history between the generative AI service and the user including the print request. In addition, this computer program causes the computer to specify one or more candidates for the print range selected from the plurality of options as the print range. This makes it easy to select and print only the necessary information in the information generated by the generative AI.

[0097] Other embodiments

[0098] Embodiments of the present invention can also be implemented by a computer of a system or apparatus that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more of the functions of the above embodiments, and / or includes one or more circuits (e.g., an application specific integrated circuit (ASIC)) for performing one or more of the functions of the above embodiments. Moreover, embodiments of the present invention can be implemented by a method of, for example, reading and executing computer-executable instructions from the storage medium by the computer of the system or apparatus to perform one or more of the functions of the above embodiments, and / or controlling the one or more circuits to perform one or more of the functions of the above embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random access memory (RAM), a read only memory (ROM), a memory of a distributed computing system, an optical disc (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD) TM )), a flash device, and a memory card, etc.

[0099] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (a computer program product including computer programs / instructions) that performs the functions of the above embodiments to a system or apparatus through a network or various storage media, and the computer (central processing unit (CPU), microprocessing unit (MPU)) of the system or apparatus reads and executes the computer programs / instructions.

[0100] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation so as to cover all such variations and equivalent structures and functions.

Claims

1. A non-transitory computer-readable storage medium storing a computer program for expanding the functions of a generative AI service, the computer program causing a computer of an information processing apparatus to perform the following operations: When a print request is received via the generative AI service, specifying a print range; Generating a print job based on the print request and the print range; Registering the generated print job in a print server; Obtaining information related to a connection destination for executing the print job from the print server; and Displaying the print job and the information related to the connection destination to a user via the generative AI service.

2. The non-transitory computer-readable storage medium according to claim 1, wherein the computer program further causes the computer of the information processing apparatus to specify a print range based on a conversation history between the generative AI service and the user, the conversation history including the print request.

3. The non-transitory computer-readable storage medium according to claim 1, wherein the computer program further causes the computer of the information processing apparatus to query, via the generative AI service, a plurality of options serving as candidates for a print range in a conversation history between the generative AI service and the user, and to specify one or more candidates for the print range selected from among the plurality of options as the print range, the conversation history including the print request.

4. The non-transitory computer-readable storage medium according to claim 3, wherein the plurality of options include: content of an input from the user and content of a response from the generative AI service.

5. The non-transitory computer-readable storage medium according to claim 2, wherein the computer program further causes the computer of the information processing apparatus to obtain data representing a print range from a storage unit that stores a conversation history for a predetermined period.

6. The non-transitory computer-readable storage medium according to claim 5, wherein the computer program further causes the computer of the information processing apparatus to display to the user: content of print settings and a print preview image generated by the print server, as information about the print job.

7. The non-transitory computer-readable storage medium according to claim 6, wherein the computer program further causes the computer of the information processing apparatus to generate print data by converting text data included in a print request received via the generative AI service into a format acceptable to the print server.

8. The non-transitory computer-readable storage medium according to claim 7, wherein the computer program further causes the computer of the information processing apparatus to set predetermined print settings for the generated print job.

9. The non-transitory computer-readable storage medium according to claim 5, wherein the computer program is incorporated into the computer as a plug-in for the generative AI service.

10. An information processing apparatus that executes a computer program for expanding the functions of a generative AI service, the information processing apparatus including: One or more memory devices that store an instruction set; And One or more processors that execute the instruction set to perform the following operations: When a print request is received via the generative AI service, specify a print range, Generate a print job based on the print request and the print range, Register the generated print job with a print server, Obtain information related to a connection destination for executing the print job from the print server, and Display the print job and the information related to the connection destination to the user via the generative AI service.

11. A printing system, comprising an information processing device, a print server, and an image forming device, wherein the information processing device provides a generative AI service to a user terminal and includes: One or more first memory devices that store an instruction set of a computer program for expanding the function of the generative AI service; And One or more first processors that execute the instruction set to perform the following operations: When a print request is received via the generative AI service, specify a print range, Generate a print job based on the print request and the print range, Register the generated print job with a print server, Obtain information related to a connection destination for executing the print job from the print server, and Display the print job and the information related to the connection destination to the user via the generative AI service, The print server includes: One or more second memory devices that store an instruction set; And One or more second processors that execute the instruction set to perform the following operations: When the print job has been registered, provide information related to the connection destination, and When the user has accessed the information related to the connection destination, input the print job to the image forming device.

12. The printing system according to claim 11, wherein, The print server is a cloud print server.

13. A computer program product that stores a computer program for expanding the function of a generative AI service, the computer program causing a computer of an information processing device to perform the following operations: When a print request is received via the generative AI service, specify a print range, Generate a print job based on the print request and the print range, Register the generated print job with a print server, Obtain information related to a connection destination for executing the print job from the print server, and Display the print job and the information related to the connection destination to the user via the generative AI service.

Images