Program, information processing device, and information processing method

Abstract

In AI assistants, if multiple exchanges in a chat format are necessary to perform the desired action from the user, the chat can get lost. [Solution] The program of the present invention is an object for setting the print settings of a first file, and has a display step of displaying the first object which includes an input field, characterized in that the display step displays a second object for setting the print settings of a second file in a state in which the first object is displayed when natural language indicating a print instruction for a second file different from the first file is entered into the input field.

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing apparatus and an information processing method. [Background Art]

[0002] Patent Document 1 discloses a technique in which a user instructs print settings and print execution by having a conversation with a chatbot in natural language. [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2020-52507 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] There are cases where a user wants to specify detailed print settings for printing a file using natural language. When performing print settings using the technique described in Patent Document 1, print settings can be performed by exchanging chats with a chatbot for each setting item to specify the settings of each setting item. The same settings can be made when a user performs print settings in natural language using AI (Artificial Intelligence).

[0005] When a user inputs a natural language indicating a print instruction for a certain file, it is assumed that an object (card) for performing setting confirmation and setting change is displayed on the chat screen. It is also conceivable that the object includes a dedicated text input field. It is assumed that natural language indicating setting changes or additional instructions is input into this text input field, but there is also a possibility that natural language indicating a print instruction for another file may be input. In such a case, updating the already displayed object means that the object is no longer for the original file, resulting in the problem that it is difficult to issue a print instruction for the original file.

[0006] This disclosure is made in view of the above-mentioned issues and aims to make it easier for users to specify files to be printed using natural language. [Means for solving the problem]

[0007] The program of the present invention is an object for setting the print settings of a first file, and includes a display step of displaying the first object which includes an input field, characterized in that the display step displays a second object for setting the print settings of a second file in a state in which the first object is displayed, when natural language indicating a print instruction for a second file different from the first file is entered into the input field. [Effects of the Invention]

[0008] This makes it easier for users to specify the files to be printed using natural language. [Brief explanation of the drawing]

[0009] [Figure 1] Network configuration diagram according to the present invention [Figure 2A] Client PC Hardware Configuration Diagram [Figure 2B] Plugin server hardware configuration diagram [Figure 3A] Application software configuration diagram [Figure 3B] AI Assistant Software Configuration Diagram [Figure 3C] Plugin software configuration diagram [Figure 3D] Example of a plugin manifest [Figure 3E] Plugin server software configuration diagram [Figure 4A] An example of a card post made by the plugin according to the present invention (prompt displayed directly) [Figure 4B] An example of a card post made by the plugin according to the present invention (parameter display) [Figure 4C] An example of a card post posted by the plug-in according to the present invention (abstract file specification) [Figure 4D] An example of a card post posted by the plug-in according to the present invention (attached file) [Figure 5A] An example of a card update updated by the plug-in according to the present invention (print setting change - before update) [Figure 5B] An example of a card update updated by the plug-in according to the present invention (print setting change - after update) [Figure 5C] An example of a card update updated by the plug-in according to the present invention (file change - before update) [Figure 5D] An example of a new card post posted by the plug-in according to the present invention (file change) [Figure 5E] An example of a card update updated by the plug-in according to the present invention (new card generation setting) [Figure 5F] An example of a new card post posted by the plug-in according to the present invention (new card generation setting) [Figure 5G] An example of a card update posted by the plug-in according to the present invention (print execution - after update) [Figure 5H] An example of a card update posted by the plug-in according to the present invention (print execution - after print execution) [Figure 5I] An example of a card update posted by the plug-in according to the present invention (file specification after print execution) [Figure 5J] An example of a card update posted by the plug-in according to the present invention (error card) [Figure 6A] Plug-in call sequence in an AI assistant [Figure 6B] An example of a card creation sequence in a plug-in execution control unit [Figure 6C] Plug-in call sequence when updating a card or creating a new card [Figure 7] An example of an instruction statement for generating print settings [Figure 8] An example of a card template according to the present invention [Figure 9]Card update sequence in the second embodiment [Figure 10A] An example of card update in the second embodiment (print settings change - before update) [Figure 10B] An example of card update in the second embodiment (print settings changed - after update) [Figure 10C] An example of card renewal in the second embodiment (after selecting a link) [Modes for carrying out the invention]

[0010] The embodiments for carrying out the present invention will be described below with reference to the drawings. Note that the following embodiments are not intended to limit the invention as defined in the claims, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0011] In this embodiment, Generative AI refers to a technology that automatically generates various types of content such as text, images, music, and videos, similar to those created by humans. Furthermore, Generative AI does not execute processes according to predetermined rules like conventional AI, but generates these based on knowledge learned using deep learning.

[0012] Simultaneously, in this embodiment, the AI ​​assistant refers to a system that accepts natural language input, for example, through the application's chat field (input field), and displays the response to that input, generated using generative AI, in the application's chat field.

[0013] In this embodiment, natural language input may be referred to as prompt 3100. The specifications of this prompt are not particularly limited and will conform to the specifications of each AI assistant, but for example, the user may attach image files or document files.

[0014] In this embodiment, natural language form refers to a form similar to everyday conversations between people, such as "Tell me the weather in Tokyo tomorrow." Natural language form has the advantage of being easier to use because it does not require the use of word-separated input, such as "Tokyo weather tomorrow" as used in web searches.

[0015] In this embodiment, a card is a form of post (display) that includes information such as text and images, and an object for the user to instruct further actions after viewing the generated content. The displayed content is updated according to the user's actions. This information and object are displayed, for example, within the same area.

[0016] In this embodiment, LLM is an abbreviation for Large Language Model.

[0017] [First Embodiment] Figure 1 shows an example of the system configuration in this embodiment. The network configuration diagram according to this embodiment includes a terminal device, a computer 1000, an AI assistant server 4000, a plug-in server 6000, and an external system server 7000. Furthermore, these are connected by the internet 8000 and can communicate with each other. The computer 1000 has an application 2000, and the application 2000 has an AI assistant 3000 and a plug-in 5000.

[0018] Computer 1000 is located, for example, within an office and is connected to the external internet 8000 via an internal network (not shown) and a router (not shown). Here, computer 1000, AI assistant server 4000, plug-in server 6000, and external system server 7000 are examples of information processing devices. Computer 1000 is also an example of a user terminal.

[0019] Application 2000 is an application that runs on Computer 1000 and includes AI Assistant 3000. Furthermore, the functionality of Application 2000 can be extended by adding Plugin 5000.

[0020] In this invention, the plugin 5000 refers to a function that communicates with the plugin server 6000 and performs arbitrary processing, and primarily uses a generating AI in its processing process, although it is not necessarily required that a generating AI be used in the processing process. Furthermore, the plugin 5000 can be called not only by user operation in the application 2000 but also by the AI ​​assistant 3000. In addition, the plugin 5000 can also provide the AI ​​assistant server 4000 with information acquired via the plugin server 6000 as an additional source of information for generating an answer.

[0021] The AI ​​assistant server 4000 has the function of generating a response to prompt 3100, which is input to the AI ​​assistant 3000 on application 2000, using generative AI models specialized in natural language processing, such as LLM.

[0022] The plug-in server 6000 has the function of performing arbitrary operations on the external system server 7000 according to the content of communication with the application 2000. Furthermore, the plug-in server 6000 has the function of generating products using arbitrary generating AI according to the content of the prompt 3100, and the function of posting information including at least the products of the generating AI as a card 3200 to the AI ​​assistant 3000.

[0023] In this embodiment, card 3200 is an example of an object, and card 3200 displays various information using objects such as display fields, input fields, and buttons. Furthermore, the user can instruct the execution of the process corresponding to the selected button by selecting a button displayed on card 3200.

[0024] The external system server 7000 is connected to the plug-in server 6000 via the Internet 8000 and has the function of receiving operations from the plug-in server 6000.

[0025] Furthermore, application 2000 may be a client application residing on computer 1000, or a web application accessible via the internet 8000. Similarly, plugin 5000 may be deployed on computer 1000, on a web application, or on the AI ​​assistant server 4000.

[0026] Furthermore, the function by which computer 1000 communicates with AI assistant server 4000 and plug-in server 6000 may be a function included in application 2000, or a function possessed by AI assistant 3000 and plug-in 5000. AI assistant 3000 and plug-in 5000 are called by application 2000.

[0027] <Hardware Configuration> Referring to Figure 2, an example of the hardware configuration of each device constituting the system according to this embodiment will be described.

[0028] Figure 2(A) shows an example of the hardware configuration of computer 1000. Figure 2(B) shows an example of the hardware configuration of plug-in server 6000.

[0029] The hardware configuration of the AI ​​Assistant Server 4000 is the same as that of the Plug-in Server 6000 shown in Figure 2(B). Furthermore, the hardware configuration of the External System Server 7000 is the same as that of Computer 1000 shown in Figure 2A. Therefore, a detailed explanation is omitted.

[0030] As shown in Figure 2(A), the computer 1000 has a display unit 110, an operation unit 120, a storage unit 130, a control unit 140, and a network communication unit 150, which are connected to each other so as to be able to communicate with one another. The computer 1000 is, for example, a desktop or notebook personal computer (PC), a tablet terminal, or a smartphone. The control unit 140 is composed of a CPU (Central Processing Unit) 141 and memory 142, and controls the entire computer 1000.

[0031] The display unit 110 is composed of a display such as an LCD panel and can display images and various screens. The operation unit 120 is composed of a mouse or keyboard and can accept input operations from the user. Alternatively, it may be composed of a touch panel or the like that has the functions of both the display unit 110 and the operation unit 120.

[0032] The storage unit 130 is composed of a storage medium such as a hard disk or SSD (Solid State Drive), and stores various programs (software) necessary for the operation of the computer 1000. The programs are loaded into memory 142 as needed and executed by the CPU 141. The programs include those that cause the CPU 141 to execute various parts and functions of application 2000.

[0033] This program is not limited to being stored on computer 1000. For example, the program may be stored on an external device that can communicate with computer 1000, which runs application 2000, such as the AI ​​assistant server 4000, the plug-in server 6000, or the external system server 7000. In that case, computer 1000 communicates with the external device when executing the program, accesses the program, and performs the processing.

[0034] The network communication unit 150 performs data input and output with external devices via an external network. This allows the computer 1000 to exchange information with external devices such as the AI ​​assistant server 4000 or the plug-in server 6000 via a network such as the Internet 8000.

[0035] As shown in Figure 2(B), the plug-in server 6000 has a display unit 610, an operation unit 620, a storage unit 630, a control unit 640, and a network communication unit 650, which are connected to each other so as to be able to communicate with one another.

[0036] The control unit 640 comprises a CPU 641, memory 642, and a GPU (Graphics Processing Unit) 643, and controls the entire plug-in server 6000.

[0037] Detailed explanations of the hardware configuration of Plugin Server 6000, which is the same as that of Computer 1000, will be omitted.

[0038] The GPU643 is a processing unit composed of image processing processors. For example, the GPU643 performs calculations to transform input image and text data using a Large-Scale Language Model (LLM) according to control commands given by the CPU641.

[0039] Furthermore, while it is preferable that the plug-in server 6000 and AI assistant server 4000, which perform processing using the generated AI, have a GPU 643, the effects of the present invention can still be achieved even in a configuration without a GPU 643.

[0040] <Software Configuration> Referring to Figure 3, the software configuration of each device according to this embodiment will be described. Figure 3(A) shows the modules related to the processing of the AI ​​Assistant 3000 and Plugin 5000 in Application 2000. Figure 3(B) shows the modules related to the response generation process, including the execution of Plugin 5000, in the AI ​​Assistant 3000. Figure 3(C) shows the software configuration of Plugin 5000. Figure 3(D) shows an example of Plugin Manifest 5010 in Plugin 5000. Figure 3(E) shows the software configuration of Plugin Server 6000.

[0041] <Software configuration of Application 2000> As shown in Figure 3(A), application 2000 includes at least an AI assistant 3000, a plugin 5000, a plugin index 2100, and a card processing unit 2200. Application 2000 is not particularly limited except for the inclusion of the AI ​​assistant 3000, and could include, for example, a communication tool, an email system, or document editing software. Here, the communication tool may be an application for exchanging messages via a chat service.

[0042] First, the AI ​​Assistant 3000 has the function of returning an answer to prompt 3100 when prompt 3100 is entered in the AI ​​Assistant 3000 within application 2000.

[0043] The plug-in 5000 is software added for the purpose of extending the functions of the application 2000. For example, it can provide functions for operating external systems such as various storage services and core business systems through the application 2000. Multiple plug-ins 5000 can be installed for the application 2000, and the upper limit of installable plug-ins 5000 is subject to the specifications of the application 2000. Furthermore, the plug-in 5000 can also be called from the AI assistant 3000, and is used by the AI assistant 3000 as reference information for generating a response to the prompt 3100. Note that a single plug-in 5000 may include multiple commands, and the AI assistant 3000 may determine and call an appropriate command. There is also no particular limitation on the method for calling the plug-in 5000. For example, the plug-in 5000 may be a plug-in that is called only from the application 2000. Any calling method may be adopted, including the plug-in 5000 called only from the AI assistant 3000, and the plug-in 5000 compatible with both the application 2000 and the AI assistant 3000.

[0044] The plug-in index 2100 is a list of plug-in information for each plug-in 5000 installed in the application 2000. For the plug-in information, metadata including the name, description (description in natural language format), parameters and the like of the plug-in 5000 described in the plug-in manifest 5010 of the plug-in 5000 can be registered. In addition, for managing the plug-in 5000, an ID allocated by the application 2000 side can also be registered. Based on this plug-in information, the AI assistant 3000 determines the plug-in 5000 to be executed.

[0045] <Software Configuration of AI Assistant 3000> As shown in Figure 3(B), the AI assistant 3000 includes a chat input / output control unit 3010, an external data acquisition control unit 3020, a log management unit 3030, and an answer generation unit 3040.

[0046] The chat input / output control unit 3010 has the function of receiving prompts 3100 from application 2000 and sending output generated via AI assistant server 4000 to application 2000. In this case, the format of the input and output prompts 3100 is not limited to natural language instructions only; for example, it is also possible to attach file content such as image files or document files.

[0047] The external data acquisition control unit 3020 has several functions for acquiring additional information for the AI ​​model 6020 to generate an answer in the answer generation unit 3040. The web search control unit 3021 provides the function of acquiring information that exists on the web via a search engine. The business data acquisition unit provides the function of acquiring documents and information created in daily operations from cloud storage and core business systems linked with application 2000. The plug-in data acquisition control unit 3023 provides the function of acquiring information on external systems, either linked with application 2000 or not linked with application 2000. Details of the processing of the plug-in data acquisition control unit 3023 will be described in the embodiment.

[0048] The log management unit 3030 has the function of saving and providing as an operation log the prompts 3100 previously entered by the user and the responses output by the AI ​​assistant 3000 in the AI ​​assistant 3000. The operation logs saved by the log management unit 3030 are used as information for determining parameters when calling the plug-in server 6000 and for the AI ​​assistant server 4000 to generate responses.

[0049] The response generation unit 3040 has the function of communicating with the AI ​​assistant server 4000 and generating a response to the prompt 3100 using LLM. In doing so, it is possible to use as additional information information information acquired by the external data acquisition control unit 3020, the conversation history conducted on the AI ​​assistant 3000, and other information that can be obtained from the application 2000. The information that can be obtained from the application 2000 is not particularly limited and may include, for example, posts by the user / others in a communication tool, information of the file creator and update history in presentation software, etc.

[0050] <Software configuration of Plugin 5000> As shown in Figure 3(C), plugin 5000 has at least a plugin manifest 5010. The plugin manifest 5010 is a file that describes information about plugin 5000, is registered in the plugin index 2100, and is used as information for determining which plugin 5000 the AI ​​assistant 3000 should execute. Here, referring to Figure 3(D), an example of plugin manifest 5010 is shown. The plugin manifest 5010 shown in Figure 3(D) is a file content in JSON (JavaScript Object Notation) format and contains information 5011 to 5015. 5011 describes a setting that indicates the name of the plugin. 5012 describes a description (natural language description) for the plugin and is mainly referred to when determining which plugin the AI ​​assistant 3000 should execute. For example, if prompt 3100 is entered, such as "Print Sample1.pdf in 5 copies / saving prints", keyword search or AI-based conceptual search may be performed between the input and the description in Figure 3(D) to determine whether or not to execute plugin 5000. 5013 is a component of the plugin and includes the commands in the plugin shown in 5014. 5014 is a command implemented in the plugin and includes the ID, description, and parameters of each command in Figure 3(D). 5015 is a parameter for each command and includes the parameter name, description, and input format in Figure 3(D). Descriptions exist for both 5014 and 5015, which are primarily referenced by the AI ​​assistant 3000 when determining the parameters for creating a request to the plugin server. Multiple commands can be set for a single plugin, but the maximum number of commands is subject to the specifications of application 2000.

[0051] The above is merely an example, and the contents of Plugin Manifest 5010 are not limited to this. For example, it may include developer information, sample prompts, the plugin's API definition, and authentication method. Furthermore, the software configuration of Plugin 5000 is not particularly limited; for example, it may include an icon image and an API definition file for performing operations on Plugin Server 6000.

[0052] <Software configuration of Plugin Server 6000> As shown in Figure 3(E), the plug-in server 6000 includes a card creation processing control unit 6010, a generation AI model 6020, a card template 6030, and a content management unit 6040.

[0053] The card creation processing control unit 6010 has the function of controlling the processing performed by the plug-in server 6000 when it is called from the AI ​​assistant 3000 using the plug-in 5000. Details of the processing performed by the AI ​​assistant server 4000 will be described later based on an embodiment. For example, these include operations on external systems other than the plug-in server 6000, operations on the generated AI model 6020, and operations on both the external systems and the generated AI model 6020. Furthermore, these operations include creating a card 3200 based on the card template 6030 using the results of these operations, and sending the created card 3200 to the application 2000 or the AI ​​assistant 3000.

[0054] The generative AI model 6020 is a type of machine learning model trained using a large dataset, and is a type of AI (artificial intelligence) capable of creating original content. Examples include LLMs (Large Language Models) suitable for natural language processing and GANs (Generative Adversarial Networks) suitable for image processing. The plug-in server 6000 may also contain other generative AI models 6020. At the same time, the generative AI model 6020 is not limited to existing within the plug-in server 6000, but may also be a component of an external system server 7000 located outside the plug-in server 6000.

[0055] The card template 6030 is a template for the card 3200 to be sent to the AI ​​assistant 3000. It is possible to create the card 3200 to be posted to the AI ​​assistant 3000 by applying the parameters generated during processing in the card creation processing control unit 6010. Furthermore, the card template 6030 not only displays the output results of the plugin server 6000, but can also accept user operations on the application 2000 or the AI ​​assistant 3000. For example, the card 3200 displayed on the AI ​​assistant 3000 can be used to perform operations on the plugin server 6000, and the card 3200 can be updated based on the results. In addition, the card template 6030 may be a single file, or multiple files may exist, and the appropriate card template 6030 may be used depending on the prompt 3100. Note that the card template 6030 may exist not only on the plugin server 6000, but also as part of the plugin 5000 installed in the application 2000. In the latter case, instead of the plug-in server 6000, the application 2000 or the AI ​​assistant 3000 itself needs to apply the parameters received from the plug-in server 6000 to the card template 6030 and create the card 3200.

[0056] The content management unit 6040 has the function of managing file content in the plug-in server 6000. The file content managed by the content management unit 6040 is not particularly limited. For example, this includes the attached file 3130 attached to the prompt 3100 in the AI ​​assistant 3000, and files obtained by the card creation processing control unit 6010 from the external system server 7000 using plug-in information. Alternatively, it also includes files generated by the generation AI model 6020.

[0057] <Example of card posting UI> Referring to Figure 4, an example of the UI for card posting according to this embodiment will be explained. The screen of the AI ​​assistant 3000 shown in Figure 4 displays a prompt 3100, a card 3200, and a chat sending field 3300. The prompt 3100 is posted by the user to the AI ​​assistant 3000, and the card 3200 is returned to the AI ​​assistant 3000 by the plug-in server 6000.

[0058] Furthermore, the chat message field 3300 is an example of an input field, and is used by the user to post a prompt 3100 to the AI ​​assistant 3000. The user enters natural language into the chat message field 3300 to set the settings to be used when the file is printed, and the AI ​​assistant 3000 accepts this.

[0059] Furthermore, users can post files in the chat message field 3300, and the AI ​​assistant 3000 will retrieve them. When specifying a file, users can post only the file, attach the file to a text message, or post a string indicating the file's location, such as a URL.

[0060] From here on, we will explain using a plugin 5000 as an example, which provides a function that allows the user to set print settings via the AI ​​assistant 3000 and print the file content specified by the user. However, the display of card 3200 and other elements necessary for this process are not particularly limited.

[0061] Plugin 5000 requests at least two parameters from Plugin Server 6000 as input: File name 3110 and print settings 3120. File name 3110 is the file name of the file content to be printed, and print settings 3120 are specific (or abstract) print settings such as the number of copies, paper type, and whether or not to print double-sided.

[0062] The plugin server 6000, which can communicate via plugin 5000, has the function of retrieving file content to be printed from an external system server 7000, such as cloud storage. It also has the function of generating print settings 3120, creating a preview image by applying the generated print settings 3120 to the retrieved file content, and returning a card 3200 containing the generated print settings 3120 and the preview image.

[0063] Plugin 5000 creates a card 3200 containing print settings and a preview image based on a prompt 3100 that includes a file name 3110 and print settings 3120, and posts it to the AI ​​assistant 3000. The user can then interact with the AI ​​assistant 3000 via the card 3200, using the information stored in the card to change print settings, change the print target, and issue print execution commands.

[0064] In this process, the results of each interaction and process are displayed not by posting a new card 3200, but by updating an already posted card 3200. This prevents an increase in the number of unnecessary chats posted through multiple interactions. As a result, the usability and visibility of the user's application 2000 are improved.

[0065] Furthermore, if the user wants to print a different item with different settings, they can enter prompt 3100 again into the chat message field 3300 and invoke plugin 5000, which will post a new card. The user can then engage in a new exchange through the newly posted card 3200. Hereafter, plugin 5000 may be referred to as the "print plugin," plugin server 6000 as the "print plugin server," and external system server 7000 as "cloud storage."

[0066] In this embodiment, the card 3200 includes an input content display 3210, a generation reason display 3220, a print settings display 3230, a preview image display 3240, and operation buttons 3250. The operation buttons 3250 also include a print settings change button 3251, a file change button 3252, and a print execution button 3253.

[0067] Card 3200 is an object that is displayed when a file is retrieved and when settings are accepted based on natural language entered in the chat message field 3300. As is clear from the diagram, card 3200 does not contain the chat message field 3300; each is displayed independently in a separate area.

[0068] This section describes prompt 3100, which was submitted by a user to AI Assistant 3000. In Figure 4(A), the user entered the natural language instruction "Print Sample1.pdf in 5 copies / save printing" as prompt 3100 (Figure 4(B) also shows the same prompt 3100).

[0069] The AI ​​assistant 3000 analyzes the prompt 3100 entered (posted) by the user and, if it determines that executing the print plugin is appropriate, it calls the print plugin server. At that time, the AI ​​assistant 3000 extracts the file name "Sample1.pdf" 3110 and the print setting "5 copies / economy printing" 3120 information from the prompt 3100 based on the parameter specifications described in the plugin manifest 5010. These two pieces of information are passed from the AI ​​assistant 3000 to the print plugin server and used in the processing of the card creation processing control unit 6010.

[0070] Note that the user's input prompt 3100 is not limited to the example above. For example, the user may call the print plugin by specifying the print target with an abstract file specification, such as "Print 5 copies of tomorrow's meeting materials (saving on printing costs)," as shown in Figure 4(C). In that case, the business data acquisition control unit 3022 in the AI ​​assistant 3000 searches for the document corresponding to "tomorrow's meeting materials" and passes it to the print plugin server when calling. Alternatively, the card creation processing control unit 6010 of the print plugin server may need to perform processing such as searching for the document corresponding to "tomorrow's meeting materials" from cloud storage.

[0071] Alternatively, as shown in Figure 4(D), the user may attach a file to the prompt 3100 and input a combination of natural language and file content, such as "Print this file in 5 copies / saving prints." Furthermore, if the application 2000 equipped with the AI ​​assistant 3000 provides file editing functions such as document editing software or presentation software, the following processing may be performed. The AI ​​assistant 3000 accepts the instruction "Print this file itself in 5 copies / saving prints" and may pass information held by the application 2000 to the print plug-in server to process it. The information used for identification includes the file name 3110 and the file ID.

[0072] Next, we will describe the card 3200 returned (posted) by the print plugin server. In the examples in Figures 4(A) to 4(D), the card 3200 displays the input content 3210, the reason for generation 3220, the print settings 3230, the preview image 3240, and the operation buttons 3250.

[0073] The input content display 3210 shows the prompt that the user entered and posted in the chat message field 3300. In other words, when the print plugin is called, the content sent from the AI ​​assistant 3000 to the print plugin server is displayed, allowing the user to confirm it.

[0074] In the example in Figure 4(A), the prompt 3100 posted by the user, "Input prompt: Print Sample1.pdf 5 times / save printing," is displayed directly in the input content display 3210. However, the content displayed in the input content display 3210 is not limited to the above.

[0075] For example, as shown in Figure 4(B), the parameters extracted from prompt 3100 and sent to the print plugin server may be displayed instead of the actual prompt 3100 entered, such as "File name: Sample1.pdf Print settings: 5 copies / economy printing". Alternatively, as shown in Figure 4(C), the file name 3110, which has been searched and estimated based on an abstract file specification, may be displayed, such as "Input prompt: Print tomorrow's meeting materials in 5 copies / economy printing File name: Sample1.pdf". Note that the display of input content 3210 is not mandatory, and the configuration may be such that the input content display 3210 is not displayed.

[0076] The generation reason display 3220 and the print settings display 3230 show the parameters generated by the print plugin server using the generated AI model 6020.

[0077] In the example in Figure 4(A), the reason for generation display 3220 shows "You can save on printing costs by selecting recycled paper as the paper type and performing double-sided printing," and the print settings display 3230 shows "Number of copies: 5 Paper type: Recycled paper Double-sided printing: Yes Do you want to print with these settings?"

[0078] The content displayed in the generation reason display 3220 and the print settings display 3230 is not limited to the above content. For example, as in the example in Figure 4(C), the following may be displayed in addition to the reason for generating the print settings 3120, such as, "The meeting scheduled for tomorrow is the XX meeting at 13:00. Sample1.pdf is attached as a document in the scheduler." This indicates the basis for the estimation from the abstract file specification.

[0079] The preview image display 3240 shows an image (preview image) in which the print settings 3120 generated by the generating AI model 6020 have been applied to the print target file specified by the file name 3110, etc., as shown in the example in Figure 4(A). This allows the user to confirm whether the print settings and other content instructed by the user via prompt 3100 have been reflected as intended by the user.

[0080] Thus, the card 3200 can not only display the products generated by the plug-in server 6000 using the generated AI model 6020, but can also display the results of further processing performed using those products.

[0081] The operation button 3250 displays buttons for the user to perform additional operations after reviewing the generated output. In the example in Figure 4(A), there is a print settings change button 3251, a file change button 3252, and a print execution button 3253.

[0082] The print settings change button 3251, when pressed, allows the CPU 141 to accept additional user prompts 3100 and regenerate the print settings 3120 (described later in Figure 5A). The file change button 3252, when pressed, allows the CPU 141 to accept user re-selection of a file and change the print target. The print execution button 3253, when pressed, allows the CPU 141 to submit a print job to the printer with the contents displayed on the card 3200.

[0083] Note that there may be only one operation button 3250 per card 3200, or multiple buttons may be displayed, and the upper limit of the number of buttons that can be displayed is in accordance with the specifications of application 2000.

[0084] The above shows examples of the screen when card 3200 is posted, referring to Figures 4(A) to 4(D). As shown above, even when printing the same file with the same settings, the elements and display methods on card 3200 may differ depending on how prompt 3100 is written. In the example shown, the same file "Sample1.pdf" is printed with the same settings: "Number of copies: 5 copies, Paper type: Recycled paper, Double-sided printing: Yes". On the other hand, it is also possible to have a configuration where the display is always performed in one pattern as shown above.

[0085] Furthermore, although Figure 4 was explained under the assumption that one prompt 3100 posts one card 3200 associated with one print job, the number of cards 3200 that the plug-in server 6000 can post at once is not particularly limited.

[0086] For example, a prompt 3100 such as "Suggest optimal print settings 3120 for Sample1.pdf / Sample2.pptx" may be configured to post individual cards 3200 for each file. In this case, the user can set print settings for each corresponding file in each card 3200.

[0087] <Example of card update UI> Referring to Figure 5, an example of the screen (UI) when updating the card 3200 according to this embodiment will be explained. For the sake of explanation, Figure 5 assumes that the prompt 3100 and card 3200 shown in Figure 4(A) have been posted. In the following explanation, the selection of the operation button 3250 associated with updating the card refers to a general user operation to perform an arbitrary operation, such as clicking the mouse or pressing the Enter key on the keyboard to select a button.

[0088] <Example of UI for updating cards when print settings are changed> If a user wants to change the print settings 3120 generated by the print plugin server, they can select the print settings change button 3251 on the card 3200 to generate different print settings.

[0089] Figure 5(A) shows an example of card 3200 where the prompt input field 3254 and the card update button 3255 are displayed because the print settings change button 3251 has been selected. The prompt input field 3254 and the card update button 3255 are elements that are not displayed when the card is first posted (Figure 4(A)), but are displayed when the print settings change button 3251 is selected.

[0090] The prompt input field 3254 is a field where the user can input prompts in natural language format. The user can input prompts using the operation unit 120 or software keys (not shown). The AI ​​assistant 3000 receives the natural language input from the user in the prompt input field 3254 and uses it to configure settings when the file is printed.

[0091] For example, Figure 5(A) shows a state where the user has entered the prompt "Print in 2-in-1". There are no particular limitations on the content of the prompt that can be entered in the prompt input field 3254, but the number of characters that can be entered and whether or not files can be attached are subject to the limitations of Plugin 5000 and Application 2000.

[0092] Once the user has finished entering prompts in the prompt input field 3254, they select the card update button 3255. When the card update button 3255 is selected, a request is sent to the print plugin server containing the current print settings and file name held by card 3200, along with the prompts entered in the prompt input field 3254. This causes the print settings to be regenerated using the generation AI model 6020, and card 3200 is updated based on the results.

[0093] In other words, at this time, a new card is not posted; rather, the display content and print settings of the already posted card 3200 are updated according to the user's prompt. Alternatively, the print settings corresponding to the prompt entered by the user may be displayed on the new card.

[0094] Figure 5(B) shows an example of a card 3200 that has been updated as a result of selecting the card update button 3255 in Figure 5(A). In Figure 5(B), the input content display 3210, the generation reason display 3220, the print settings display 3230, and the preview image display 3240 have been updated based on the prompt "Print in 2-in-1" entered in Figure 5(A). Furthermore, a highlight 3260 indicating the content of the operation performed, "Print settings have been changed!", has been added as an element of the new card 3200.

[0095] At this point, the preview image has been updated, and the preview image with the settings based on the initial user prompt is no longer displayed. However, to show the difference between before and after the update, the pre-update preview image may continue to be displayed.

[0096] If the user enters "Print in 4-in-1" in the prompt input field 3254, the input content display 3210, the generation reason display 3220, the print settings display 3230, and the preview image display 3240 are updated based on this prompt. In addition to the print settings, if the user specifies, for example, the number of copies to print, the item corresponding to the specified number of copies is updated.

[0097] For example, if the user enters "Print 6 copies" in the prompt input field 3254, the highlighting 3260 will display something like "Number of copies changed!". Additionally, the input content display 3210 will display something like "Input prompt: Print 6 copies", and the generation reason display 3220 will display something like "The number of copies was specified, so the setting value for the number of copies has been changed". Furthermore, regarding the print settings display 3230, as shown in Figure 5(A), the content "Number of copies: 5 copies" that was displayed before the user selected the card update button 3255 will be updated to "Number of copies: 6 copies" (not shown).

[0098] In this case, the string "6 copies" may be displayed instead of "5 copies." That is, if the user changes the print settings as described above, the print settings before the change may not be displayed, and the print settings after the change may be displayed. Alternatively, both the print settings before and after the change may be displayed. Furthermore, the display of print settings here may refer to all of the multiple setting items, or only the setting item that has been changed.

[0099] <Example of UI for generating a new card when print settings are changed> Next, we will explain an example where, if the user wants to apply the print settings generated by the print plugin server to printing another file, they can select the print settings change button 3251 on card 3200 and apply the print settings 3120 to another file.

[0100] Figure 5(C) shows an example where the input to prompt input field 3254 is different from that in Figure 5(A). Here, the input is "Print Sample2.pdf with the same settings". When the user executes the card update button 3255, a request is sent to the print plugin server containing the current print settings and file name held by card 3200, and the prompt entered in prompt input field 3254. Upon receiving the request, the print plugin server uses the generation AI model 6020 to decide whether to update card 3200 or generate a new card 3400. Here, since a different file (Sample2.pdf) is specified than the file currently being printed (Sample1.pdf), it decides to generate a new card. Subsequently, a new card 3400 is generated based on the prompt (especially the specified file information) and print settings, and card 3400 is posted based on the result. In other words, it is determined whether the prompt entered in the prompt input field 3254 of card 3200 specifies a different file from the file currently being printed by card 3200. If the determination is made that it is a different file, a new card, card 3400, which is a different card from card 3200, is created and posted to the chat field. The print settings associated with card 3400 (the print settings associated with Sample2.pdf) are then based on the prompt entered in the prompt input field 3254 of card 3200. In the example in Figures 5(C)(D), "with the same settings" is entered, so the same print settings as those set in card 3200 are set in card 3400.

[0101] Furthermore, in Figures 5(C) and 5(D), the second object, card 3400, is displayed when natural language input indicating a print command for the second file, Sample2.pdf, is entered. At this time, card 3400 is displayed with the first object, card 3200, and the chat message field 3300 (another input field) also displayed.

[0102] Furthermore, if natural language is entered into the prompt input field 3254 of card 3200 to instruct the user to change the print settings of the first file, Sample1.pdf (and not to instruct the user to print another file), a new object will not be displayed, as in card 3400. Instead, the updated card 3200 will be displayed as shown in Figure 5(B).

[0103] Figures 5(C) and 5(D) illustrate an example where natural language indicating a print instruction for another file is entered into the prompt input field 3254 of card 3200. Even if "Print Sample2.pdf double-sided" is entered into the chat message field 3300, another object can be posted to the chat room, as shown in card 3400. However, in this case, regardless of whether another file is specified or not, a new object (card) is always generated and posted (displayed) when entered into the chat message field 3300. In other words, there are multiple input fields on the screen where the user can enter natural language, and there is a possibility that a print instruction for another file may be mistakenly entered into the prompt input field 3254 of card 3200. In this invention, even in such cases, a new card can be generated, posted, and displayed.

[0104] Figure 5(D) shows an example of card 3400 that was posted in Figure 5(C) when a prompt was entered in the prompt input field 3254 and the card update button 3255 was pressed. In Figure 5(D), card 3400 is displayed separately from card 3200. Card 3400 has the same configuration as card 3200 in Figure 5(B), so only the differences will be explained. The input content display 3410 displays the prompt 3100 entered in the prompt input field 3254, but of course, it may also display the information in the input content display 3210. The preview image display 3440 displays a print preview image with the print settings 3430 applied to the print target file specified by the file name included in the prompt entered in the prompt input field 3254 in Figure 5(C). In addition, a highlight 3460 indicating the content of the operation performed, "A different file was specified, so a new card has been created!" is also displayed.

[0105] Unlike the examples shown in Figures 5(A) and (B), the card 3200 will not have any updates to print settings, etc. In this case, depending on whether the card update button 3255 is executed, the prompt input field 3254 and the card update button 3255 may be hidden again.

[0106] These configurations allow users to apply the same print settings to different files and print them accordingly. Furthermore, it enables comparison of print results between card 3200 and card 3400.

[0107] Figures 5(C) and 5(D) show examples of generating a new card when a different file from the original print target is specified, but of course, a new card can be generated under different conditions. Figure 5(E) shows an additional "Create New Card" setting button 3257, which allows the user to specify whether or not to generate a new card. In the prompt input field 3254 in Figure 5(E), "Print in 2-in-1" is entered, just like in Figure 5(A). At this time, if the "Create New Card" setting button 3257 is enabled and the "Update Card" button 3255 is executed, a new card 3500 is posted as shown in Figure 5(F), rather than updating card 3200. Since card 3500 has the same configuration as cards 3200 and 3400, a detailed explanation is omitted, but the input content display 3510 shows an example of displaying the two prompts entered so far together. In addition, a highlighting 3560 indicating the content of the executed operation, "A new card with different print settings has been created!", is also displayed. In Figure 5(F), card 3200 is omitted for convenience, but it may also be displayed without omission, as in Figure 5(E).

[0108] These configurations allow users to compare card 3200 and card 3500 to see and verify the print results when different print settings are applied to the same file.

[0109] Figures 5(C) through 5(F) illustrate an example of how new cards are generated, but new cards may be generated under different conditions. For example, this could occur if the user enters information in prompt input field 3254 such as "I want to compare this with a setting that saves even more paper," indicating their intention to compare and verify the print results. Similarly, if the user requests the presentation of multiple print results, such as "Please provide two options for print settings that save even more paper," the system may create multiple new cards.

[0110] <Example of card update UI during print execution> Using Figure 5(G), we will explain an example of the card update UI when a print job is submitted. Note that this example describes the case where the print execution button 3253 in Figure 5(B) is executed.

[0111] When a user confirms the print settings and print target displayed on card 3200 and wishes to print, they can execute the print execution button 3253 on card 3200, submitting the print job to the printer through the print plug-in server. Figure 5(G) shows card 3200 indicating that a print job has been submitted after the print execution button 3253 was executed. In Figure 5(G), in addition to the highlighting 3260 and preview image display 3240, a job details display 3270 showing the print job that was ultimately submitted to the printer is displayed. Note that in Figure 5(G), the operation button 3250 is not displayed because card 3200 has already submitted a print job and the processing has shifted to the print plug-in server and the connected printer. The display on card 3200 is not limited to this; for example, an operation button 3250 such as "Cancel Print" may be displayed to provide a function to cancel the submitted print job.

[0112] Furthermore, even without user interaction with card 3200, card 3200 may be updated by processing by the plug-in server 6000. Figure 5(H) shows card 3200 updated as a result of actual printing being performed by the printer. In Figure 5(H), the highlighting 3260 and job content display 3270 have been updated from Figure 5(G), and the operation buttons 3250 are also displayed. The operation buttons 3250 include the print settings change button 3251, the file specification button 3262 which accepts print instructions for a different file with the same settings, and the reprint execution button 3261 which has the same function as the print execution button 3253.

[0113] Figure 5(I) shows card 3200 with a scrollable file selection field 3256 and a card update button 3255 displayed after the file selection button 3262 is executed. The process for obtaining each file displayed in this file selection field 3256 is not particularly limited. For example, the print plugin server may access an external service and display the obtained information, or the card 3200 may display the information in conjunction with the business data acquisition function of application 2000 (or AI assistant 3000). When the card update button 3255 is executed, a request is sent to the print plugin server including the file specified in the file selection field 3256 and the print settings shown in the job details display 3270. Based on the new card information returned based on that request, a new card (not shown) different from card 3200 is posted.

[0114] These configurations allow users to apply the same print settings to other files and print them even after printing has been completed.

[0115] In addition, in Figures 5(H) and (I), if the print settings change button 3251 is executed, a prompt is entered in the displayed prompt input field 3254, and the card update button 3255 is executed, a new card may be posted instead of updating card 3200. With this configuration, even if you want to print the same file with different print settings, the printed card 3200 will remain displayed (without being updated), making it possible to refer to it as part of the print history or compare it with a new card.

[0116] <Example of card update UI when an error occurs> Furthermore, if the print settings 3120 specified by prompt 3100 are inappropriate, a card 3200 indicating that prompt 3100 is inappropriate may be displayed. Figure 5(J) shows an example of a card 3200 that has been updated when the inappropriate prompt 3100 "Increase the number of copies to 500" is entered in the prompt input field 3254 in Figure 5(A). This card 3200 contains highlighting 3260, input content display 3210, error content display 3280, prompt input field 3254, and a card update button 3255. In particular, the error content display 3280 shows an explanation of the cause of the error and how to deal with it, stating, "Error: The specified number of copies cannot be set. Please specify the number of copies as an integer in the range of 1 to 99." Another feature of Figure 5(J) is that, as in Figures 5(A) to 5(I), there is no preview image display 3240 on the card 3200. Therefore, in the following explanation, a card 3200 that has a preview image display 3240 will be referred to as a preview card, and a card 3200 that does not have a preview image display 3240 will be referred to as an error card. These preview cards and error cards may be created from different card templates 6030 or from the same card template 6030. Furthermore, even within preview cards and error cards, multiple card templates 6030 may be used depending on the purpose. Note that there are no particular limitations on the conditions under which an error card is created; for example, conditions such as the specified file content not existing or a prompt 3100 unrelated to printing being entered may occur.

[0117] The above describes examples of the UI for creating and updating cards, referring to Figures 4 and 5. However, the card UI is not limited to these examples and may take various forms in accordance with the specifications of Plugin 5000. For example, if the user enters "Print @Sample1.pptx" (without specifying print settings), the display may be changed according to the user's input / generation AI response, such as displaying frequently used print buttons like "Economical Printing," "Meeting Materials," and "Poster Printing." Alternatively, on mobile devices, the display may be changed depending on the device running Application 2000, such as adjusting the size of the preview image display 3240.

[0118] <Example Sequence> Referring to Figure 6, the processing flow from the CPU 141 of the computer 1000 to the submission of card 3200 will be explained in this embodiment. In the following, "S" indicates a processing step, and the following number indicates the step number. Note that application 2000 operates under the control of the CPU 141 of the computer 1000.

[0119] <Plugin call sequence when creating a card> Using Figure 6(A), the sequence from when the CPU 141 controls each part to when it calls the plug-in 5000 and posts the card 3200 will be explained. Note that the application 2000 and the AI ​​assistant 3000 execute their respective functions using the CPU 141. In addition, the computer 1000, the AI ​​assistant server 4000, and the plug-in server 6000 each have their own CPUs that control each device, and their respective network communication units enable communication between the devices.

[0120] In S101, the CPU 141, which controls application 2000, accepts user input at prompt 3100.

[0121] In S102, the AI ​​assistant 3000, controlled by the CPU 141, requests a list of the 5000 plugins registered in application 2000 from the plugin index 2100.

[0122] In S103, the plugin index 2100 controlled by the CPU 141 returns a list of plugins 5000 requested by the AI ​​assistant 3000, allowing the AI ​​assistant to receive (obtain) the list of plugins 5000. Alternatively, if the plugin index 2100 is a database, the AI ​​assistant 3000 may obtain the list of plugins 5000 by referencing the plugin index 2100.

[0123] In S104, the AI ​​assistant 3000, controlled by the CPU 141, determines which plugin 5000 to execute in response to the user's instructions entered as prompt 3100. The AI ​​assistant 3000 makes this determination, for example, by referring to the description of each plugin (a natural language description of each plugin) based on the prompt 3100 entered by the user, and performing keyword searches or AI-based conceptual searches.

[0124] Let's explain with another concrete example. The AI ​​assistant 3000, controlled by the CPU 141, first performs a conceptual search between the prompt 3100 entered by the user in S101 and the description of the print plugin, as exemplified in information 5012 in Figure 3(D). If there are multiple candidates, it searches and evaluates each plugin 5000, and if it meets certain conditions such as ranking or score, the AI ​​assistant 3000 can determine that it is appropriate to execute the print plugin.

[0125] Furthermore, if multiple suitable plugins 5000 exist, the AI ​​Assistant 3000 may process each plugin individually, or it may process only one of the selected plugins 5000. On the other hand, if no suitable plugin 5000 exists, it does not need to process any plugins 5000.

[0126] Furthermore, a single plugin 5000 may have multiple commands, and the AI ​​assistant 3000 may refer to the description set for each command to determine which plugin 5000 is which on a command-by-command basis. Once the AI ​​assistant 3000 controlled by the CPU 141 has determined which plugin and which commands of plugin 5000 will execute the processing, the process moves to S105.

[0127] In S105, the AI ​​assistant 3000, controlled by CPU 141, determines the parameters for executing Plugin 5000. The AI ​​assistant 3000 determines the parameters based on, for example, the prompt 3100, the operation log, and other information that the AI ​​assistant 3000 can obtain. At this time, the AI ​​assistant 3000 also refers to the description of each parameter and performs parameter extraction or parameter generation using the generated AI. The operation log is, for example, a log managed by the AI ​​assistant 3000.

[0128] For example, in this parameter determination, the AI ​​generates parameters from prompt 3100 based on prompt 3100 and the descriptions of the "fileName" and "printConfig" parameters, as exemplified in information 5015. This allows the AI ​​assistant 3000 to determine that the appropriate value for the "fileName" parameter is "Sample1.pdf" and the appropriate value for the "printConfig" parameter is "5 copies / economy printing". Specifically, in this case, the determination is based on the "filename of the file content to be printed" and the "specific (or abstract, such as economy settings) print settings, including the number of copies, paper type, and whether or not to print on both sides."

[0129] Alternatively, as shown in Figure 4(C), if the user specifies the printing target with an abstract file specification, such as "Print 5 copies of tomorrow's meeting materials using a cost-saving method," the AI ​​Assistant 3000 may retrieve the relevant file by referring to the schedule management function of Application 2000. Specifically, for example, the AI ​​Assistant 3000 retrieves the user's schedule for the next day and then retrieves the relevant file by searching for files or folders that contain the string "meeting appointment." If a file is attached to the schedule, the AI ​​Assistant 3000 can also retrieve that file.

[0130] Additionally, the AI ​​Assistant 3000 obtains the next date (as a string) based on the current date, and then searches for files and folders containing that string to retrieve the relevant files.

[0131] Furthermore, the process of referencing the schedule management function of this application 2000 may also be handled in the same way as the determination of plugins and parameters, by application 2000 referring to prompt 3100 and using a generating AI or the like to determine whether or not to execute the process.

[0132] In S106, the AI ​​assistant 3000 sends a request to the plug-in server 6000 to generate card 3200. Upon receiving the request, the plug-in server 6000 starts the card generation process for card 3200 in the card creation processing control unit 6010 controlled by the CPU 641.

[0133] In S107, the card creation processing control unit 6010, controlled by the CPU 641, instructs the generation AI model 6020 to generate the product. For example, the print plug-in server inputs the "printConfig" parameter specified in S105 from the information received from the AI ​​assistant 3000 into the generation AI model (generation AI model 6020) and instructs it to generate print settings in JSON format.

[0134] In S108, the card creation processing control unit 6010 receives the returned product from the generation AI model 6020 and generates the card 3200 based on the received information. Details of the card generation process will be explained using Figure 6B.

[0135] In S109, the card creation processing control unit 6010, controlled by the CPU 641, sends (returns) the card 3200, which was created in S108, to the AI ​​assistant 3000 of the computer 1000.

[0136] In S110, the AI ​​assistant 3000, having received the card 3200 generated by the plug-in server 6000, sends a request to the AI ​​assistant server 4000 to generate an answer, using the prompt 3100 and the card 3200 as input. Upon receiving this, the CPU 441 of the assistant server 4000 controls an LLM unit (not shown) to generate an answer corresponding to the received request. This answer will later be the answer that the AI ​​assistant 3000 provides to the prompt entered by the user in S101.

[0137] In S111, CPU441 sends (returns) the answer generated using the LLM unit to computer1000.

[0138] In S112, the AI ​​assistant 3000, controlled by the CPU 141, displays its response on the display unit 110 of the computer 1000. This displays a card 3200 as shown in Figure 4, allowing the user to perform operations on the plug-in server 6000 via the operation buttons 3250.

[0139] Note that the processing in S110 and S111 may be omitted. For example, if both the card 3200 returned by the plug-in server 6000 and the summary of the card 3200 generated by the LLM unit are displayed in response to the user's prompt 3100, content duplication may occur. On the other hand, the user may want to see many elements at once in the card 3200 returned by the plug-in server 6000. In that case, the user experience can be improved by not omitting these processes and generating and displaying a summary of the card 3200 generated by the LLM unit.

[0140] The above is an example of a basic Plugin 5000 call sequence, but the order of the Plugin 5000 call sequence is not limited to this, and other processes may be included in the sequence. Furthermore, some of the processes may be omitted as long as the effects of the present invention can be achieved.

[0141] <Card creation process in the plugin execution control unit> Next, using Figure 6(B), an example of the card creation process will be explained using the print plugin described in the Card Submission UI section. Note that the process shown in this flowchart is executed by the CPU 641 of the plugin server 6000 controlling the card creation processing control unit 6010 and the generation AI model 6020.

[0142] Furthermore, this process details the card generation process described before and after S108. This process begins when the AI ​​assistant 3000 sends a request to the plug-in server 6000 to generate card 3200 (S106). At this time, the card creation processing control unit 6010 starts the card 3200 generation process.

[0143] As mentioned above, the print plugin shall input at least two parameters, the file name 3110 and the print settings 3120, to the print plugin server. Furthermore, each device shall communicate via its own network communication unit.

[0144] In S201, the card creation processing control unit 6010, controlled by the CPU 641, performs authentication processing against the cloud storage used by the user in order to obtain the file content of the file the user wants to print. Here, for example, it is assumed that the user's authentication information is entered when the user adds Plugin 5000 to Application 2000, or when Plugin 5000 is executed for the first time, and this information is used when executing the authentication process.

[0145] In S202, the card creation processing control unit 6010, controlled by the CPU 641, instructs the cloud storage to search for the items to be printed. At this time, the CPU 641 can instruct the search by sending signals, such as commands, to the cloud service that manages the cloud storage. The cloud storage, upon receiving this, executes the search according to the received instructions. Once the search is complete, it sends (returns) the search results to the plug-in server 6000.

[0146] In S203, CPU641 determines whether or not there is a print target in the cloud storage. Specifically, for example, if the cloud storage successfully searches and identifies the print target, CPU641 receives information to that effect. On the other hand, if the cloud storage fails to search and does not identify the print target, CPU641 receives a signal to that effect. Based on the received information, CPU641 then determines whether or not there is a print target. Note that CPU641 may also be configured to determine that there is no print target if there is no response from the cloud storage after a predetermined time has elapsed.

[0147] Based on the signals received by CPU641, if it determines that there is a print target, the process proceeds to S204; otherwise, it proceeds to S212.

[0148] In S204, CPU641 receives the file to be printed following the signal from S203. This allows CPU641 to retrieve the file content. Alternatively, if the system determines that there is a file to be printed in the cloud storage, CPU641 may receive the file itself instead of a signal indicating this. In this case, in S203, CPU641 determines that there is a file to be printed based on the file it received.

[0149] In S212, the card creation processing control unit 6010, controlled by the CPU 641, creates an error card (card 3200), for example, as described in Figure 5F. Note that the design of the error card is not limited to that shown in Figure 5F; an appropriate design or text will be displayed depending on the nature of the error.

[0150] In S205, the card creation processing control unit 6010, controlled by the CPU 641, executes an instruction statement creation process to generate the print settings 3120. This creates the instruction statement.

[0151] The content of the instruction text created here is not particularly limited, and as illustrated in Figure 7, it may be a natural language string consisting of the instruction content, the user's print command, the output format, the print configurable items, an example response, and an example error output. The instruction text created in this way includes the natural language string entered by the user (5 copies / economy print), and also includes information indicating the possible print settings (number of copies 1-99, paper size A4, B5, Letter, etc.). It also includes a prompt instructing the user to configure the print settings based on the information indicating the print setting options. It also includes a prompt to specify the format of the output produced by the generating AI. Furthermore, it includes an error message to be output if it is determined that the print settings based on the information indicating the print setting options cannot be configured (i.e., the settings instructed by the user in the prompt are outside the range of print setting options).

[0152] In S206, the card creation processing control unit 6010, controlled by the CPU 641, sends the instruction statement created in S205 to the generation AI model 6020 for input. Upon receiving the instruction statement, the generation AI model 6020, controlled by the CPU 641, generates print settings according to the instruction statement. Once the generation of print settings is complete, the generation AI model 6020 returns the generated print settings to the card creation processing control unit 6010.

[0153] In S207, the card creation processing control unit 6010, controlled by the CPU 641, receives the print settings 3120 generated by the generation AI model 6020.

[0154] In S208, the card creation processing control unit 6010, controlled by the CPU 641, determines whether or not an error has occurred in the print settings. The CPU 641 checks the generated print settings 3120, and if it determines that an error has occurred, the process moves to S212. In S212, the card creation processing control unit 6010, controlled by the CPU 641, creates an error card.

[0155] The decision process in S208 is based on whether or not a JSON file containing only the key value "error" has been output, as shown in the example error output in Figure 7. If CPU 641 determines that there are no errors in the generated print settings 3120, the process proceeds to S209.

[0156] In S209, the card creation processing control unit 6010, controlled by the CPU 641, generates a preview image based on the file content acquired in S204 and the print settings 3120 generated in S207. Specifically, for example, the preview image generated here is an image with the acquired file content and the generated print settings applied. The preview image creation process may be performed using a rule-based method or by generating an image using a generation AI model.

[0157] In S210, the card creation processing control unit 6010, controlled by the CPU 641, creates a preview card using the card template 6030.

[0158] In addition, in S210 and S212, the card creation processing control unit 6010 may perform pre-processing to adjust the appearance of the card 3200. For example, instead of displaying the print settings as text, the print settings may be displayed as a bulleted list as shown in the print settings display 3230 in Figure 4A. To perform such a display, the CPU 641 may create a string including escape characters from the JSON format data.

[0159] Additionally, the CPU 641 may change the size of images and text displayed on the generated card depending on the execution terminal of application 2000 equipped with AI assistant 3000, that is, depending on the size of the display unit 110.

[0160] This process can be executed either by CPU641 adjusting the parameters, or by using different card templates 6030.

[0161] In S211, the card creation processing control unit 6010, controlled by the CPU 641, sends the card 3200 created in S210 to the computer 1000 and returns it to the AI ​​assistant 3000 of application 2000.

[0162] <Plugin call sequence when updating or creating a new card> Next, Figure 6(C) will be used to explain the plugin call sequence when updating a card or creating a new card. Unlike the sequence when creating a card, the sequence for updating a card or creating a new card calls the plugin server 6000 without going through the AI ​​assistant 3000. Therefore, in S301, the user first enters prompt 3100 for card 3200 in application 2000. Then, in S302, the card processing unit sends update information for card 3200 to the plugin server 6000. At this time, card 3200 may add information other than the entered prompt 3100. For example, in the case of a print plugin, it may send information such as the currently set print settings 3120, file content information, or operation logs that have been performed in the past via the AI ​​assistant 3000 or card 3200. Therefore, card 3200 can have this information as parameters that are not displayed on card 3200. Alternatively, this information may be managed on the plugin server 6000 side. At the same time, card 3200 may have information to identify the return destination of card 3200, such as the ID of card 3200 (for example, card number 3290, which will be described later in the second embodiment) and the chat ID in the AI ​​assistant 3000. Next, since card 3200 itself does not have the function to extract parameters from prompt 3100, the card creation processing control unit 6010 determines the parameters for executing plugin 5000 in S303. In this process, the method for determining the parameters is not particularly limited, as in S105. For example, based on the prompt entered by the user, the information sent from the card processing unit, and the information managed by the plugin server 6000, the plugin server 6000 refers to the description of each parameter it holds. Then, a method such as performing parameter extraction or parameter generation using AI is appropriate. These parameters include information for deciding whether to update card 3200 or create a new card.As mentioned above, the system will include information to create a new card if the file to be printed is changed (Figure 6(C)(D)) or if the new card creation setting button 3257 is enabled (Figure 6(E)(F)). The same applies when the card update button 3255 is executed on a card that has already been printed (Figure 6(H)(I)), etc. Of course, it is also possible to have a configuration in which information to create a new card instead of updating card 3200 is included based on other conditions.

[0163] From this point onward, S304 performs the same processing as S107, S305 as S108, and S308 as S112, so their explanations will be omitted. In S306, the card creation processing control unit 6010 returns card information that includes information on whether to update the aforementioned card 3200 or create a new card. The card processing unit of application 2000 then performs the processing of either updating card 3200 (S307) or submitting a new card (S308) according to that information.

[0164] The sequence of processes performed between the application 2000, the AI ​​assistant server 4000, and the plug-in server 6000 in this embodiment has been described above with reference to Figure 6. However, the above is merely an example, and the sequence of processes is not limited to this.

[0165] By performing the above process, if the natural language entered in the prompt input field 3254 within card 3200 is a print instruction for a file other than the one corresponding to card 3200, a new card 3400 can be generated and posted to the chat screen. The original card 3200 is not deleted and remains accessible and operable by the user. This makes it easy to issue print instructions for the new file while maintaining the ease of issuing print instructions for the original file. Furthermore, by inheriting the print settings of the original file, it is possible to omit the settings for the new file.

[0166] <Card Template> Referring to Figure 8, an example of a card template 6030 is shown using a preview card in the print plugin as an example. The card template 6030 shown in Figure 8 is a JSON file content containing information 6031 to 6038. 6031 describes the setting that indicates the name of the card. Data structure 6032 is a data structure enclosed under the key "body". Here, settings for displaying elements such as text, images, or input fields on the card UI are described, such as input content display 3210, generation reason display 3220, print settings display 3230, preview image display 3240, and highlighting 3260. In the example in Figure 8, there is a description enclosed in ${}, such as "${Input}", and by replacing the parameters in the card creation processing control unit 6010 of the plugin server 6000, a card according to the processing content can be created. Data structure 6033 is a data structure enclosed under the key "actions", and describes the settings related to the operation button 3250 that is provided as a button on the card UI. Setting 6034 describes how pressing the print settings change button 3251, as shown in Figure 5A, will display the prompt input field 3254 and the card update button 3255, which are initially hidden elements. Setting 6034 also describes how to display hidden elements by describing a data structure enclosed by the "body" key and "actions" key within the setting itself, similar to a nested structure. This description method conforms to the application 2000 specifications. Setting 6035 describes how the card creation processing control unit 6010 can identify buttons executed by the user on the card. Setting 6036 describes how to display the prompt input field 3254 on the card UI. Parameter 6037 describes the parameters that are passed to the plug-in server 6000 when the card update button 3255 is executed. In the example in Figure 8, settings for parameters related to file content, print settings, and operation logs are described. Setting 6038 describes the settings for displaying the file selection field 3256.Setting 6039 describes the settings for displaying the print execution button 3253.

[0167] The above is just one example for the print plugin; the content of the card template is not limited to this and is subject to the specifications of the application and plugin.

[0168] [Second Embodiment] The following describes a second embodiment of the present invention. The second embodiment describes a configuration in which the card 3200 is updated even when the user enters a second prompt (3600) instead of entering an instruction in the prompt input field 3254. The same configuration and processing as in the first embodiment will not be described.

[0169] Referring to Figures 9, 10(A), 10(B), and 10(C), the plugin 5000 call sequence during card update or creation in the AI ​​assistant 3000 in the second embodiment will be described. Here, we assume that the card 3200 described in Figure 4(A) has been generated and posted.

[0170] In S401, the user enters prompt 3600 into the chat message field 3300 and sends it to the AI ​​assistant 3000 in application 2000. The AI ​​assistant 3000 then first requests a list of plugins 5000 registered in application 2000 from the plugin index 2100 in S402. Next, in S403, when the plugin index 2100 returns the list of plugins 5000, in S404, the plugin 5000 determines which plugin 5000 to execute in response to the user's instruction entered as prompt 3600. The method for determining this plugin 5000 is not particularly limited. For example, based on the prompt 3600 entered by the user, a method such as keyword search or AI-based conceptual search is used, referring to the description (natural language description of plugin 5000) of each plugin 5000. Furthermore, if multiple suitable plugins 5000 exist, processing may be performed for each plugin 5000 individually. However, if no suitable plugin 5000 exists, processing may not be performed for any plugin 5000. In addition, a single plugin 5000 can contain multiple commands, and the plugin 5000 may be determined on a command-by-command basis by referring to the description set for each command. After determining the plugin to be executed and the commands of the plugin 5000, in S405, the AI ​​assistant 3000 determines the parameters for executing the plugin 5000. Note that the method for determining the parameters in S405 is not particularly limited. For example, the description of each parameter may be referenced based on the user-entered prompt 3600, the operation log managed by the AI ​​assistant 3000, and other information inside and outside the application 2000 that can be obtained from the AI ​​assistant 3000. Then, a method such as parameter extraction or parameter generation using AI may be used.

[0171] In S406, the AI ​​Assistant 3000 determines which card's print settings should be changed. Normally, it targets the most recently posted card, but depending on the natural language input, it may also target earlier cards or cards identified by a card ID, such as card number 3290.

[0172] If the AI ​​Assistant 3000 can identify the target card, it will perform a card update process (S408 to S413). If it determines that the target card does not exist, it will perform a new card creation process (similar to S106 to S112 mentioned above, so the explanation is omitted).

[0173] S408 to S412 perform the same processing as S302 to S307 as explained in Figure 6(C). In Figure 6(C), the plug-in server 6000 was called from card 3200, but here the difference is that the plug-in server 6000 is called from AI assistant 3000. In the following S413, AI assistant 3000 posts a message notifying that card 3200 has been updated. Message 3700 is an example of this, and it includes a link 3501 that allows the user to move to the updated card 3200.

[0174] If link 3701 is selected, the entire card may be highlighted, as shown in the shaded area of ​​card 3200 in Figure 10(C), to improve user visibility. Alternatively, even if the print setting change button 3251 is not executed, the prompt input field 3254 and the card update button 3255 may be displayed to prompt the user to enter information in the prompt input field 3254 if they wish to make further print changes.

[0175] In the first embodiment, an example was described in which the user enters instructions to change the print settings into the prompt input field 3454 that appears when the print setting change button 3251 in card 3200 is executed. However, the user may not know how to execute the print setting change button 3251 or how to use the prompt input field 3454, and may instead enter instructions to change the print settings by entering the prompt again. In the second embodiment, even in such cases, the card update (print setting change) can be performed in accordance with the user's intention. Furthermore, since a message 3700 containing a link 3701 is posted after the card update, it becomes easier for the user to know that the card has been updated.

[0176] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

Claims

1. An object for setting print settings for a first file, having a display step for displaying the first object including an input field, The program is characterized in that, in the display step, when natural language indicating a print instruction for a second file different from the first file is entered into the input field, a second object for setting the print settings for the second file is displayed in the same state as the first object.

2. The program according to claim 1, characterized in that the first object and the second object are displayed on the chat screen during the display step.

3. The first object includes print settings and a print preview of the first file. The program according to claim 1, characterized in that the second object includes print settings and a print preview of the second file.

4. The program according to claim 1, characterized in that the first object is displayed in the display step when natural language indicating a print instruction for the first file is entered in an input field other than the aforementioned input field.

5. The program according to claim 4, characterized in that, in the display step, natural language indicating a print instruction for a second file different from the first file is entered into the input field, and the second object is displayed with the first object and the other input field displayed.

6. The program according to claim 1, characterized in that, in the display step, if the natural language entered in the input field is natural language indicating an instruction to change the print settings, the second object is not displayed, and the first object with updated print settings is displayed.

7. An object for setting print settings for a first file, having a display step for displaying the first object including an input field, The information processing method is characterized in that, in the display step, when natural language indicating a print instruction for a second file different from the first file is entered into the input field, a second object for setting the print settings for the second file is displayed in the same state as the first object.

8. An object for configuring print settings for a first file, having a display means for displaying the first object including an input field, The display means is characterized in that, when natural language indicating a print instruction for a second file different from the first file is entered into the input field, a second object for setting the print settings for the second file is displayed in the same state as the first object.

9. The information processing apparatus according to claim 8, characterized in that the display means displays the first object and the second object on the chat screen.

10. The first object includes print settings and a print preview of the first file. The information processing device according to claim 8, characterized in that the second object includes print settings and a print preview of the second file.

11. The information processing apparatus according to claim 8, characterized in that when natural language indicating a print instruction for the first file is entered into an input field other than the aforementioned input field, the display means displays the first object.

12. The information processing apparatus according to claim 11, characterized in that the display means displays the second object in a state in which the first object and the other input fields are displayed when natural language indicating a print instruction for a second file different from the first file is entered into the input field.

13. The information processing apparatus according to claim 8, characterized in that, when the natural language entered in the input field is natural language indicating an instruction to change the print settings, the display means does not display the second object, but displays the first object with updated print settings.

Images