Computer program, information processing apparatus, and system

Abstract

To provide a mechanism for suitably using a scan function of an image processing apparatus via generative AI.SOLUTION: The present information processing apparatus, upon receiving a scan request for reading an image from a document via a generative AI service, acquires information on a driver of an image processing apparatus. The information processing apparatus asks a user to check a processing condition for the scan request via the generative AI service, on the basis of the acquired information on the driver and the scan request, and generates a job reflecting the processing condition that has been checked. The information processing apparatus submits the generated job to the image processing apparatus and causes the image processing apparatus to execute scanning.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing device, and a system for expanding the functionality of an artificial intelligence system. [Background technology]

[0002] Generative AI is an artificial intelligence system known as a system capable of generating text, images, or other media in response to prompts. Another known AI support function is Microsoft 365 Copilot, a system with generative AI capabilities that works in conjunction with applications such as Microsoft Excel and generates documents and charts using natural language input. Generative AI's functionality can be expanded using plug-ins. For example, when a command containing the keyword "print" is entered into the generative AI, a plug-in related to printing is selected, and the plug-in performs processes that the generative AI cannot handle, enabling expanded functionality.

[0003] Meanwhile, a technology is known in which an image forming device applies OCR (Optical Character Recognition) to a scanned image to extract characters written on the scanned document. Patent Document 1 proposes a technology in which characters extracted by an image forming device are inserted into Microsoft Excel (registered trademark). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-3321 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, if you want to insert an image scanned by an image forming device into a document, you first need to send the scanned image to a server or email, and then insert the image data into the document, which requires time and effort on the part of the user. If these tasks, especially the scanning function of an image processing device, could be requested by natural language input to the generation AI, user convenience could be improved.

[0006] The present invention has been made in consideration of at least one of the above-mentioned problems, and provides a mechanism for optimally utilizing the scanning function of an image processing device via a generating AI. [Means for solving the problem]

[0007] The present invention is, for example, a computer program that expands the functionality of a generation AI service, and is characterized by comprising: an acquisition means that acquires information related to the driver of the image processing device when an information processing device computer receives a scan request to read an image from a document via the generation AI service; a confirmation means that confirms with a user via the generation AI service the processing conditions for the scan request based on the acquired information related to the driver and the scan request; a generation means that generates a job that reflects the confirmed processing conditions; and an execution means that submits the generated job to the image processing device to perform scanning. [Effects of the Invention]

[0008] According to the present invention, the scanning function of the image processing device can be suitably utilized via the generation AI. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a system configuration according to an embodiment. [Figure 2] FIG. 1 is a diagram showing the arrangement of an MFP according to an embodiment. [Figure 3] FIG. 1 is a diagram showing the configuration of a generation AI server according to an embodiment. [Figure 4]FIG. 1 is a diagram showing the configuration of an extended application server according to an embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a screen displayed on a user terminal according to an embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a screen displayed on a user terminal when performing a scan according to an embodiment. [Figure 7] FIG. 4 is a sequence diagram of a scan according to an embodiment. [Figure 8] 10 is a flow chart for an extended application server according to one embodiment. [Figure 9] FIG. 1 is a diagram showing a system configuration according to an embodiment. [Figure 10] FIG. 4 is a sequence diagram of a scan according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] First Embodiment <System configuration> A first embodiment of the present invention will be described below. An example of the overall system configuration of a scan service according to this embodiment will be described with reference to FIG. 1. This system for providing a scan service includes an MFP 10, which is an image processing device, a user terminal 20, a generation AI server 30, and an extended application server 40. A document creation application 21 and an assist application 22 are installed on the user terminal 20. Each device is connected via a network 60 and can communicate with each other. The network 60 is a wireless or wired network configured as a WAN or LAN.

[0012] The MFP 10 is an image forming device that has a function of executing a scan in response to a scan instruction notified from the extended application server 40. The user terminal 20 is an information terminal such as a smartphone, tablet terminal, or personal computer that a user uses to create documents. The user can operate the user terminal 20 to create documents using a document creation application 21 and an assist application 22 installed on the user terminal 20.

[0013] The document creation application 21 is an application for creating documents, such as Microsoft Excel (registered trademark), installed on the user terminal 20. The assist application 22 is installed on the user terminal 20, operates in conjunction with the document creation application 21, and is an AI support function equipped with a generation AI function that creates documents and charts simply by instructing them via text. The assist application 22 accesses the generation AI server 30 on the cloud and executes the generation AI application.

[0014] The generation AI server 30 is a cloud server located on the cloud 50, and provides services in cooperation with the extended application server 40. The generation AI server 30 interprets messages sent from the user terminal 20, generates an appropriate answer, and displays it as a response on the screen of the user terminal 20. Furthermore, the generation AI server 30's functions can be expanded by communicating with the extended application server 40. The extended application server 40 is a cloud server located on the cloud 50, and provides additional functions to the generation AI server 30. By working with the extended application server 40, the generation AI server 30 becomes able to perform processing that was not possible on its own.

[0015] <Configuration of image processing device> An example of the hardware configuration of MFP 10 according to this embodiment will be described with reference to Fig. 2. MFP 10 includes a control unit 110, an operation unit 116, a reading unit 118, a printing unit 120, a wireless communication unit 122, a FAX communication unit 124, and a communication unit 126. Control unit 110 includes a CPU 111, a ROM 112, a RAM 113, a HDD 114, an operation unit I / F 115, a reading unit I / F 117, a printing unit I / F 119, a wireless communication unit I / F 121, a FAX unit I / F 123, and a communication unit I / F 125.

[0016] The control unit 110 including the CPU 111 controls the overall operation of the MFP 10. The CPU 111 loads a control program stored in the ROM 112 or the HDD 114 into the RAM 113 to perform various types of control, such as reading control and printing control. The ROM 112 stores control programs executable by the CPU 111. The ROM 112 also stores a boot program, font data, and the like. The RAM 113 is a main memory, and is used as a work area and a temporary storage area for expanding the various control programs stored in the ROM 112 and the HDD 114. The HDD 114 stores image data, print data, various programs, various addresses, and various setting information. The HDD 114 is a storage medium, and may be a solid state drive (SSD) or an embedded multi-media card (eMMC).

[0017] In the MFP 10 according to this embodiment, one CPU 111 executes each process shown in the flowcharts described below using one memory (RAM 113), but this is not limiting. For example, each process can be executed by multiple CPUs, RAMs, ROMs, and HDDs working together. Also, some processes can be executed using hardware circuits such as ASICs and FPGAs.

[0018] The operation unit I / F 115 connects the control unit 110 to an operation unit 116, which includes a display unit such as a touch panel and hardware keys. The operation unit 116 functions as a user interface that displays information to the user and detects inputs from the user. The reading unit I / F 117 connects the control unit 110 to a reading unit 118, such as a scanner. The reading unit 118 reads an image of a document, and the CPU 111 converts the image into image data such as binary data. The reading unit 118 performs ADF reading, which transports and reads a document, and pressure plate reading, which reads an image placed on a document table. Image data generated based on the image read by the reading unit 118 is sent to an external device or printed on recording paper. The printing unit I / F 119 connects the control unit 110 to a printing unit 120, such as a printer. The CPU 111 transfers image data (print data) stored in the RAM 113 to the printing unit 120 via the printing unit I / F 119. The printing unit 120 prints an image based on the transferred image data onto a sheet of recording paper or the like fed from a paper feed cassette.

[0019] The wireless communication unit I / F 121 is an I / F for controlling the wireless communication unit 122, and wirelessly connects the control unit 110 to external wireless devices such as the user terminal 20. The FAX unit I / F 123 controls the FAX communication unit 124 to connect to the public line network 70. The FAX unit I / F is an I / F for controlling the FAX communication unit 124, and controls a modem and an NCU for facsimile communication, thereby enabling connection to the public line network 70 and control of facsimile communication protocols. The communication unit I / F 125 connects the control unit 110 to the network 60. The communication unit I / F 125 allows the communication unit 126 to transmit image data and various internal device information to external devices on the network 60, and to receive print data and information on the network 60 from information processing devices on the network 60. Methods for transmitting and receiving data via the network 60 include transmission and reception using electronic mail (E-mail) and file transmission using other protocols (e.g., FTP, SMB, WEBDAV, etc.).

[0020] <Configuration of the generation AI server> An example of the hardware configuration of the generation AI server according to this embodiment will be described with reference to Fig. 3. The generation AI server 30 includes a CPU 301, a ROM 302, a RAM 303, a communication unit 304, and an HDD 305.

[0021] The CPU 301 executes processing to control the operation of generating an appropriate response using a control program stored in the ROM 302 and a learning model stored in the HDD 305. The ROM 302 stores the control program. The RAM 303 is used as a temporary storage area such as the main memory and work area of ​​the CPU 301. The HDD 305 stores various data such as the learning model and the generation AI application. The generation AI server 30 can exchange data with various devices such as the user terminal 20, the MFP 10, and the extended application server 40 via the communication unit 304. The communication unit 304 may perform wired communication using Ethernet (registered trademark) or wireless communication such as Wi-Fi.

[0022] <Extended application server configuration> An example of the hardware configuration of the extended application server 40 according to this embodiment will be described with reference to Fig. 4. The extended application server 40 includes a CPU 401, a ROM 402, a RAM 403, a communication unit 404, and an HDD 405.

[0023] The CPU 401 reads the control program stored in the ROM 402 and executes processing according to messages received from the generation AI server 30. The ROM 403 stores the control program. The RAM 402 is used as temporary storage area such as the main memory and work area of ​​the CPU 401. The HDD 405 stores the contents of messages received from the generation AI server 30 or portions thereof. The extended application server 40 can send and receive data to and from various models, including the generation AI server 30, via the communication unit 404.

[0024] <Example of screen displayed on user device> 5, an example of a screen of the document creation application 21 and the assist application 22 displayed on the user terminal 20 according to this embodiment will be described. A screen 500 is displayed on the display unit of the user terminal 20. The screen 500 includes a document area 501 and an assist application area 502.

[0025] The document area 501 is a display area for the document creation application 21, and is an area for creating documents. In the document area 501, an electronic document (hereinafter simply referred to as a "document") can be created that combines characters, figures, tables, and inserted images input by the user. The assist application area 502 is a display area for the assist application 22, and is an area for inputting messages to be sent to the generation AI server 30 and displaying responses from the generation AI server 30 that are displayed in the assist application 22. Messages are added downward in chronological order and displayed in an interactive format.

[0026] The prompt input field 503 is an example of a receiving means, and is an input field for the user to input an instruction (prompt) to be sent to the generation AI server 30 via the assist application 22. Here, an example will be described in which input is made in the text format of natural language on the user terminal 20. Of course, natural language input may also be received by voice input using a microphone (not shown) on the user terminal 20.

[0027] The send button 504 is a button that triggers sending the prompt entered in the prompt input field 503 from the assist application 22 to the generation AI server 30. A prompt 505 shows an example of the sending history of messages entered by the user. A response 506 shows a response generated by the generation AI server 30 and received by the assist application 22.

[0028] <Example of a screen when instructing a scan> 6, examples of screens displayed on the user terminal 20 according to this embodiment when instructing a scan will be described. Screens 600, 610, and 620 have the same configuration as screen 500, and show the display history of prompts and responses when the user instructs a scan by inputting natural language.

[0029] As shown in screen 600, prompt 601 is an example of a prompt used when instructing the generation AI server 30 to scan via the assist application 22. Here, the command is to scan a printed document and paste the scanned image into a document creation file. Response 602 is a response to prompt 601 generated by the generation AI server 30 in cooperation with the extended application server 40. This response indicates a response confirming where the scanned image should be inserted. Thereafter, the interactive exchange between the user and the assist application 22 is displayed in chronological order as a combination of prompts and responses.

[0030] The generation AI server 30 receives a message from the user terminal 20 via the assist application 22, analyzes the contents, and returns an appropriate response. Furthermore, if it determines from the contents of the generated message that the message should be processed by the extended application server 40, it executes a processing request to the extended application server 40. Furthermore, it creates a response to the user terminal 20 from the response contents from the extended application server 40.

[0031] The image 603 in the document area 501 is an image that the assist application 22 displays to indicate where in the document creation application 21 the scanned image should be inserted (processing condition). The response message 604 is a message for notifying the user that the size and position of the image 603 have been confirmed. Here, an example of a positive response is shown, but a negative response may also be used. As described above, according to this embodiment, the processing conditions can be confirmed interactively via the generation AI service. Note that in the case of a negative response, the insertion position may be specified by operating a predetermined position in the document area 501.

[0032] Screen 610 shows the display of response message 604 on screen 600 followed by response 605 notified via the generation AI server 30 and extended application server 40. Response 605 is an example of a display object that can set settings such as processing conditions, and is a response to response message 604 generated by the generation AI server 30 in cooperation with the extended application server 40. This response is for specifying the reading settings for the scan to be performed by MFP10.

[0033] In response 605, the color mode, single-sided or double-sided, document size, document type (text, photo, etc.), and data size for scanning can be set. A selectable pull-down button is displayed for each item, and the user can set each item by operating the pull-down button. Once the user has set the reading settings, the prompt input field 503 notifies the user that the settings have been completed. Response message 606 is a message that notifies the user that the reading settings in response 605 have been set and that the settings have been finalized.

[0034] Screen 620 shows the screen displayed following response message 606. Response 607 shows a response to response message 606 generated by the generation AI server 30 in cooperation with the extended application server 40. This response notifies that a scanned image has been inserted at the position specified by image 603. A scanned image 608 is inserted and displayed in the document area 501. The scanned image 608 is an image that was read by the MFP 10 and inserted by the document creation application 21.

[0035] <sequence> The sequence between the devices when inserting a scanned image into a document in accordance with natural language input by a user according to this embodiment will be described with reference to Fig. 7. The numbers following S below indicate the step numbers of each process.

[0036] In S701, the document creation application 21 of the user terminal 20 accepts a prompt 601 requesting scanning from the user as natural language input, and sends the accepted natural language input to the assist application 22. Here, the prompt 601 ("Insert the scanned image") is an example of a scan request made in natural language input, and is not intended to limit the present invention, and other natural language input may also be used. In S702, the assist application 22 sends the natural language input in the prompt 601 to the generation AI server 30.

[0037] In S703, the generation AI server 30 sends a launch request to the extended application server 40 to launch an extended application capable of executing a scan based on the natural language keyword "scan" received in S702. Furthermore, in S704, the generation AI server 30 sends a scan execution API to the extended application server 40.

[0038] In S705, the extension application server 40 sends an information acquisition request API for the scan driver to the generation AI server 30. In S706, the generation AI server 30 sends an information acquisition request for the scan driver to the assist application 22. In S707, the assist application 22 sends an information acquisition request for the scan driver to the document creation application 21.

[0039] In S708, the document creation application 21 sends scan driver information to the assist application 22. Here, the document creation application 21 obtains the scan driver information from the scan driver installed in the user terminal 20. If multiple scan drivers are installed, the document creation application 21 obtains information about the scan driver registered as the default setting, for example. The scan driver information includes configurable items, default values, image save paths, etc.

[0040] In S709, the assist application 22 transmits the scan driver information received in S708 to the generation AI server 30. In S710, the generation AI server 30 transmits a notification API for the scan driver information, including the information received in S709, to the extension application server 40.

[0041] In S711, the extended application server 40 sends an API requesting acquisition of image paste destination information to the generation AI server 30. In S712, the generation AI server 30 sends an acquisition request for image insertion destination information, including content displaying a response 602 specifying the insertion destination of the scanned image and an image 603 specifying the insertion of the scanned image, to the assist application 22. In S713, the assist application 22 sends the response 602 specifying the insertion destination of the scanned image and the image 603 specifying the insertion of the scanned image to the document creation application 21, and prompts the user to specify the image insertion destination.

[0042] In S714, the document creation application 21 accepts user input specifying the size and position via the image 603. Note that although an example of accepting user input by manipulating the image 603 has been described here, it can also be accepted by natural language input. In the case of natural language input, not only a positive input "yes" to the response 602 but also an input requesting correction may be used. Note that if correction is requested, the natural language input is notified to the generation AI server 30 and the extended application server 40, and the corrected insertion image, etc. are returned as a response, and it is desirable for the user to confirm again.

[0043] In S715, the document creation application 21 transmits the response message 604 entered by the user and the image insertion destination information to the assist application 22. Here, the image insertion destination information is information including the type of document creation application 21 (presentation application, spreadsheet application, document application, etc.), size, and insertion destination position information (for example, above the figure number, above the table number, etc.).

[0044] In S716, the assist application 22 sends the image insertion destination information received in S715 to the generation AI server 30. In S717, the generation AI server 30 sends a notification API for the image insertion destination information received in S716 to the extended application server 40. In S718, the extended application server 40 sends a scan setting notification API including configurable items and values ​​to the generation AI server 30. In S719, the generation AI server 30 creates a scan setting response 605 from the information received in S718 and sends a scan setting notification including the creation information to the assist application 22. In S720, the assist application 22 sends the scan setting response 605 to the document creation application 21, prompting the user to perform scan settings.

[0045] In S721, the document creation application 21 accepts a user operation and changes the settings in the scan setting response 605. The user operation is performed by directly operating various setting items displayed in the response 605. In S722, the document creation application 21 sends a response message (natural language input) 606 entered by the user to the assist application 22. In S723, the assist application 22 sends the setting contents (processing conditions including setting values, etc.) set by the user in S721 and the response message 606 to the generation AI server 30. In S724, the generation AI server 30 sends an API for instructing to execute a scan including the setting values ​​received in S723.

[0046] In S725, the extended application server 40 generates a scan job that reflects the scan driver information (including the image save path) received in S710 and the setting values ​​(processing conditions) received in S724, and submits the job by sending a scan execution request to the MFP 10. In S726, the extended application server 40 sends an image save path notification API, including the image save path used in S725, to the generation AI server 30. In S727, the generation AI server 30 sends an image save path notification to the assist application 22.

[0047] In S728, when the MFP 10 completes the scan, it sends the image data to the image storage path specified in S725. The assist application 22 acquires the scanned image from the image storage path received in S727. In S729, the assist application 22 inserts the scanned image 608 into the document creation application 21 and displays the scan completion response 607.

[0048] <Processing of the extended application server 40> The processing procedure of the extended application server 40 in this embodiment will be described with reference to Fig. 8. The processing described below is realized, for example, by the CPU 401 of the extended application server 40 reading a program stored in the ROM 402 or HDD 405 into the RAM 403 and executing it.

[0049] In S801, the CPU 401 determines whether or not an API for executing a scan has been received from the generation AI server 30. If so, the process proceeds to S802; if not, the determination in S801 is repeated. In S802, the CPU 401 sends an API requesting acquisition of scan driver information to the generation AI server 30. In S803, the CPU 401 determines whether or not an API for notifying scan driver information has been received from the generation AI server 30. If so, the process proceeds to S804; if not, the determination in S803 is repeated.

[0050] In S804, the CPU 401 sends an API for obtaining image insertion destination information to the generation AI server 30. In S805, the CPU 401 determines whether or not an API for notifying image insertion destination information has been received from the generation AI server 30. If it has been received, the process proceeds to S806; if not, the process repeats the determination of S805. In S806, the CPU 401 determines whether the image insertion destination is a presentation application from the image insertion destination information received in S804. If it is determined that it is a presentation application, the process proceeds to S807; if not, the process proceeds to S808.

[0051] In S807, the CPU 401 sets the text and photo settings to photo among the setting items of the scan driver information received in S803, and proceeds to S809. Meanwhile, in S808, the CPU 401 sets the text and photo settings to text among the setting items of the scan driver information received in S803, and proceeds to S809.

[0052] In S809, the CPU 401 determines whether the image insertion destination size is equal to or greater than a threshold value from the image insertion destination information received in S804. For example, if either the main scanning direction or the sub-scanning direction is equal to or greater than a preset threshold value, or if both the main scanning direction and the sub-scanning direction are equal to or greater than a threshold value, it determines that the size is equal to or greater than the threshold value. If the size is equal to or greater than the threshold value, the process proceeds to S810; if not, the process proceeds to S811. In S810, the CPU 401 sets the data size setting, among the setting items of the scan driver information received in S803, to image quality priority, and then proceeds to S812. On the other hand, in S811, the CPU 401 sets the data size setting, among the setting items of the scan driver information received in S803, to standard, and then proceeds to S812.

[0053] In S812, the CPU 401 creates scan settings that combine the setting items and default values ​​of the scan driver information received in S803 with the items set in S807, S808, S810, and S811. In S813, the CPU 401 sends an API scan setting notification including the scan settings created in S811 to the generation AI server 30. In S814, the CPU 401 determines whether or not it has received an API scan execution instruction including the setting values ​​set by the user in the document creation application 21 from the generation AI server 30. If it has been received, the process transitions to S815; if not, the determination of S814 is repeated.

[0054] In S815, the CPU 401 generates a scan job from the scan driver information (including the image save path) received in S803 and the setting values ​​received in S814, and transmits a scan execution request to the MFP 10. In S816, the CPU 401 transmits an image save path notification API including the image save path specified in S815 to the generation AI server 30, and ends the processing of this flowchart.

[0055] As described above, when an information processing device according to this embodiment receives a scan request to read an image from a document via a generation AI service, it acquires information related to the driver of the image processing device. Furthermore, based on the acquired driver-related information and the scan request, the information processing device confirms processing conditions for the scan request with a user via the generation AI service and generates a job that reflects the confirmed processing conditions. The information processing device submits the generated job to the image processing device to execute scanning. Furthermore, the processing conditions for the scan request may include settings for items that can be set in the driver of the image processing device, and / or the position and size of the inserted image when inserting the scanned image into an electronic document. Thus, according to this embodiment, a user can input natural language into an assistance application to enable the scanning function of the image processing device and the insertion of an image into a document creation application. Therefore, according to this embodiment, a mechanism can be provided that allows the scanning function of the image processing device to be optimally utilized via a generation AI.

[0056] The present invention is not limited to the above embodiment, and various other embodiments are also possible without departing from the spirit of the present invention. For example, in the above embodiment, the image storage path is obtained from the scan driver information, but the image storage path may be specified as the extended application server 40, and the image data may be inserted into the document creation application 21 by sending it to the generation AI server 30.

[0057] <Second embodiment> The following describes a second embodiment of the present invention. In the first embodiment, scan driver information is acquired from the document creation application 21. In this embodiment, scan driver information is acquired from a cloud scan server.

[0058] <System configuration> An example of the overall system configuration of the scan service according to this embodiment will be described with reference to Fig. 9. Here, only the configuration that differs from the first embodiment will be described.

[0059] In addition to the configuration shown in FIG. 1, the system according to this embodiment further includes a cloud scan server 80 on the cloud 50 as a cloud server. The cloud scan server 80 manages information about image processing devices in association with cloud scan IDs. Therefore, by notifying the cloud scan server 80 of a cloud scan ID, information about the corresponding image processing device, for example, information related to the driver of the image processing device, can be obtained. Note that while an example in which various servers are provided separately is described here, they may also be provided integrated with other servers. For example, the generation AI server 30, the extension application server 40, and the cloud scan server 80 may be provided integrated.

[0060] <sequence> The sequence between the devices when inserting a scanned image into a document in accordance with natural language input by a user according to this embodiment will be described with reference to Fig. 10. Note that the description of steps S901 to S904 (corresponding to steps S701 to S704) and steps S914 to S932 (corresponding to steps S711 to S729), which are the same as those in the first embodiment, will be omitted.

[0061] In S905, the extension application server 40 sends a cloud scan ID acquisition request API to the generation AI server 30. In S906, the generation AI server 30 generates a cloud scan ID acquisition response and sends a cloud scan ID acquisition request including the cloud scan ID acquisition response to the assist application 22. In S907, the assist application 22 sends the cloud scan ID acquisition response received in S906 to the document creation application 21, and prompts the user to input a cloud scan ID.

[0062] In S908, the document creation application 21 accepts user input of a cloud scan ID. Here, by entering the cloud scan ID associated with the image processing device to be used, the user can execute a job using the scan function on that image processing device. Note that it is also possible for the user not to specify a cloud scan ID, and in such cases, a default cloud scan ID is used. For example, a default cloud scan ID may be selected and set based on a group associated with the user ID. In S909, the document creation application 21 sends a response message including the accepted cloud scan ID to the assist application 22. In S910, the assist application 22 sends the response message received in S909 to the generation AI server 30. In S911, the generation AI server 30 sends a notification API of the cloud scan ID received in S910 to the extension application server 40.

[0063] In S912, the extended application server 40 transmits an MFP acquisition request to acquire the MFP information registered in the cloud scan ID received in S911 to the cloud scan server 80. In S913, the cloud scan server 80 transmits an MFP response (including information related to the driver of the image processing device) to the extended application server 40. The processing from S914 onwards is the same as the sequence shown in Fig. 7, and therefore description thereof will be omitted.

[0064] As described above, the information processing apparatus according to this embodiment acquires, from a cloud server, information related to the driver of the image processing apparatus that runs on the user terminal operated by the user. By acquiring scan driver information from the cloud scan server, it becomes possible to scan with the image forming apparatus and insert the image into a document creation application.

[0065] The disclosure of this specification includes the following computer program, information processing device, and system. (Item 1) A computer program that expands the functionality of a generation AI service, comprising: an acquisition means for acquiring information related to a driver of an image processing device when a scan request for reading an image from a document is received via the generation AI service; a confirmation means for confirming with a user, via the AI ​​service, processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; A computer program comprising: (Item 2) 2. The computer program according to item 1, wherein the processing conditions for the scan request include settings of items that can be set by a driver of the image processing device. (Item 3) 3. The computer program according to item 1 or 2, wherein the processing conditions for the scan request include at least one of a position at which the scanned image is to be inserted into the electronic document and a size of the image to be inserted. (Item 4) The computer program according to any one of items 1 to 3, characterized in that the confirmation means confirms with the user the processing conditions for the scan request in an interactive format via the generation AI service. (Item 5) The computer program according to any one of items 1 to 4, characterized in that the confirmation means acquires processing conditions for the scan request based on natural language input by a user entered via the generation AI service. (Item 6) The computer program according to any one of items 1 to 4, characterized in that the confirmation means provides a display object on the user's user terminal that allows the user to set processing conditions for the scan request, and acquires the setting contents set via the display object. (Item 7) The computer program according to item 3, characterized in that the confirmation means provides an image representing the electronic document to the user terminal of the user and acquires at least one of the position and the size specified via the image. (Item 8) The computer program described in any one of items 1 to 7, characterized in that the acquisition means acquires information related to the driver of the image processing device operating on the user's user terminal via a generation AI server that provides the generation AI service. (Item 9) 8. The computer program according to any one of items 1 to 7, wherein the acquisition means acquires information related to a driver of the image processing device operating on the user terminal of the user from a cloud server. (Item 10) An information processing device that executes a computer program that expands the functionality of a generation AI service, an acquisition means for acquiring information related to a driver of an image processing device when a scan request for reading an image from a document is received via the generation AI service; a confirmation means for confirming with a user, via the AI ​​service, processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; An information processing device comprising: (Item 11) A system including an information processing device that provides a generation AI service to a user terminal, and an image processing device, The user terminal A providing means for providing the generated AI service via a screen; a receiving means for receiving a scan request for reading an image from a document in accordance with a natural language input received via the screen; Equipped with The information processing device includes: an acquisition means for acquiring information related to a driver of an image processing device when the scan request is received via the generation AI service; a confirmation means for confirming with a user, via the AI ​​service, processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; A system comprising:

[0066] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0067] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0068] 10: MFP, 20: User terminal, 30: Generation AI server, 40: Extended application server, 50: Cloud, 60: Network, 80: Cloud scan server

Claims

1. A computer program for expanding the functionality of a generation AI service, comprising: an acquisition unit that acquires information related to a driver of an image processing device when a scan request for reading an image from a document is received via the generation AI service; a confirmation means for confirming with a user via the generating AI service processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; A computer program comprising:

2. 2. The computer program according to claim 1, wherein the processing conditions for the scan request include settings of items that can be set by a driver of the image processing apparatus.

3. 3. The computer program according to claim 2, wherein the processing conditions for the scan request include at least one of a position when the scanned image is inserted into the electronic document and a size of the image to be inserted.

4. 4. The computer program according to claim 3, wherein the confirmation means confirms with a user the processing conditions for the scan request in an interactive format via the generation AI service.

5. 5. The computer program according to claim 4, wherein the confirmation means acquires processing conditions for the scan request based on a natural language input by a user input via the generation AI service.

6. 5. The computer program according to claim 4, wherein the confirmation means provides a display object on the user's user terminal that allows the user to set processing conditions for the scan request, and acquires the setting contents set via the display object.

7. 5. The computer program according to claim 4, wherein the confirmation means provides an image representing the electronic document to the user terminal of the user, and acquires at least one of the position and the size specified via the image.

8. A computer program according to any one of claims 1 to 7, characterized in that the acquisition means acquires information related to the driver of the image processing device operating on the user's user terminal via a generation AI server that provides the generation AI service.

9. 8. The computer program according to claim 1, wherein the acquisition unit acquires information relating to a driver for the image processing device that runs on the user terminal of the user from a cloud server.

10. An information processing device that executes a computer program that expands the functionality of a generation AI service, an acquisition unit that acquires information related to a driver of an image processing device when a scan request for reading an image from a document is received via the generation AI service; a confirmation means for confirming with a user via the generating AI service processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; An information processing device comprising:

11. A system including an information processing device that provides a generation AI service to a user terminal and an image processing device, The user terminal A providing means for providing the generated AI service via a screen; a receiving means for receiving a scan request for reading an image from a document in accordance with a natural language input received via the screen; Equipped with The information processing device includes: an acquisition means for acquiring information related to a driver of an image processing device when the scan request is received via the generation AI service; a confirmation means for confirming with a user via the generating AI service processing conditions for the scan request based on the acquired driver-related information and the scan request; a generation means for generating a job that reflects the confirmed processing conditions; an execution means for inputting the generated job to the image processing device to execute scanning; A system comprising:

Images