Information processing device, program, and information processing system

Abstract

An object is to provide a method for easily solving an error not recognized by a printer. 【Solution means】 To solve the above problems, the information processing apparatus of the present application Image processing device receives information generated trouble from the user in natural language by reception means, and trouble when the information is input into the generation AI, Output it is information for trouble solving the Image processing device and the reception means receives First information for executing the process by the ,before and the received First information is Including the second piece of information sent to the Image processing device by transmission means. Based on the transmitted information, the Image processing device executes the process, which is characterized in that.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, and a program.

Background Art

[0002] A method of searching a knowledge base and executing processing on a computer system based on the information obtained by the search is known.

[0003] In Patent Document 1, a trouble occurring in a software base and a software command set to be executed by a computer to solve the trouble are linked and registered in a knowledge base. When a trouble occurs, a method of automatically performing troubleshooting by causing a computer to execute a command set corresponding to the trouble is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, when a trouble (error) occurs in a printer, in order to solve the error by the method described in Patent Document 1, it is necessary for the printer itself to recognize the trouble. Therefore, when an error that the printer does not recognize occurs, such as the ink of a printed matter being faint, the printer does not acquire a command set and cannot solve the error.

[0006] Therefore, it is necessary for the user to search for information for error resolution by himself / herself, which is troublesome.

[0007] This invention has been made in view of the above-mentioned problems, and aims to provide a method for easily resolving errors that the printer does not recognize. [Means for solving the problem]

[0008] To solve the above problems, the information processing device of this application comprises: a receiving means for receiving error information from a user in natural language regarding an error that occurred in the printer; a receiving means for receiving information obtained when the error information is input to a generating AI, which is information that the printer will use to perform a process to resolve the error; and a transmitting means for transmitting the received information to the printer, wherein the printer will perform the process based on the transmitted information. [Effects of the Invention]

[0009] According to the present invention, errors that the printer does not recognize can be easily resolved. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram showing an example of a printing system network configuration. [Figure 2] Block diagram showing an example of the hardware configuration of the computer and each server. [Figure 3] Block diagram showing an example of a printer's hardware configuration. [Figure 4] A sequence diagram illustrating an example of the process from user input of natural language to completion of processing. [Figure 5] This diagram shows an example of a screen displayed during processing. [Figure 6] A flowchart showing an example of the execution process for the generated sequence. [Figure 7] A flowchart showing an example of sequence execution process. [Figure 8] This diagram shows an example of the processing performed by the CPU 7041 of the control mechanism execution server 3000. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will now be described in detail with reference to the drawings. The configurations described in the following embodiments are merely illustrative, and the scope of the present invention is not limited by the configurations described in the embodiments. For example, each part constituting the present invention can be replaced with any configuration that can perform a similar function. In addition, any additional components may be added.

[0012] [Example 1] <System Configuration> Figure 1 is a block diagram illustrating an example of the network configuration of a printing system for performing this troubleshooting. As shown in Figure 1(a), the printing system 100 comprises a computer 1000 which is a terminal device, a printer (printing device) 2000 configured to print, a control mechanism execution server 3000, and a network 4000 that connects these to each other so that they can communicate with one another. The printing system 101 shown in Figure 1(b) is a modified configuration of the printing system 100, and shows an example where a part of the control mechanism execution server 3000 of the printing system 101 exists as a separate server. Note that the printing system 100 and the printing system 101 are examples of information processing systems.

[0013] Computer 1000 is an example of an information processing device, such as a personal computer, smartphone, or tablet terminal. Printer 200 is an example of an image processing device or image forming device, such as an MFP (Multi-Function Peripheral), multifunction printer, or scanner. Control mechanism execution server 3000 is an example of an information processing device, such as a server composed of computers, such as a personal computer.

[0014] The control mechanism execution server 3000's interoperability interface 3010 is used to input and output data to and from external devices via the network. The control mechanism 3020 is equipped with an environment for executing troubleshooting generated using the generation AI 3040.

[0015] In addition, various types of data used for troubleshooting are stored in various databases (DBs) 3030. The various types of data include, for example, materials describing the operation method of the printer 2000, such as the manual of the printer 2000, API data, FAQ data, and troubleshooting documents. Here, API is an abbreviation for Application Programming Interface, and FAQ is an abbreviation for Frequently Asked Questions.

[0016] Note that as shown in Fig. 1(b), this system may be configured such that the various DBs 3030 are separated from the control mechanism execution server 3001 and arranged in the DB server 5000 for use. Similarly, the generation AI 3040 may also be separated from the control mechanism execution server 3001 and arranged in the generation AI execution server 6000 for use. Also, a form in which the program created by the control mechanism execution server 3001 is distributed to the computer 1000 or the like and the program is executed by the computer 1000 may be adopted.

[0017] <Hardware Configuration> Fig. 2 is a block diagram showing an example of the hardware configuration of a server 7000 such as the computer 1000 or the control mechanism execution server 3000.

[0018] Fig. 2(a) is a block diagram showing an example of the hardware configuration of the computer 1000. As shown in Fig. 2(a), the computer 1000 includes a display unit (an example of display means) 1010, an operation unit 1020, a storage unit 1030, a control unit 1040, and a network communication unit 1050, which are connected to each other so as to be communicable. The computer 1000 can use, for example, a desktop or notebook personal computer, a tablet terminal, a smartphone, or the like.

[0019] The control unit 1040 consists of a CPU (Central Processing Unit) 1041 and memory 1042, and controls the entire computer 1000. The display unit 1010 consists of a display such as an LCD panel, and can display images, for example. The operation unit 1020 consists of a mouse, keyboard, etc., and allows input operations by the user 8000. Alternatively, the computer 1000 may have a touch panel, and the touch panel may perform the roles of both the display unit 1010 and the operation unit 1020.

[0020] The storage unit 1030 is composed of a storage medium such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores various information such as various programs (software) necessary for the operation of the computer 1000. Programs are loaded into memory 1042 as needed and executed by the CPU 1041. The programs include programs that cause the CPU 1041 (computer) to execute various parts and means (control methods for the printing system) of the printing system 100.

[0021] The location where this program is stored is not limited to the storage unit 1030 of the computer 1000. The program may be stored, for example, in the printer 2000 or the control mechanism execution server 3000, or it may be stored in a distributed manner across the computer 1000, the printer 2000, and the control mechanism execution server 3001.

[0022] The network communication unit 1050 performs data input and output with external devices via an external network.

[0023] Figure 2(b) is a block diagram showing an example of the hardware configuration of server 7000, which includes a control mechanism execution server 3000, a DB server, and a generation AI execution server. As shown in Figure 2(b), server 7000 has a display unit 7010, an operation unit 7020, a storage unit 7030, a control unit 7040, and a network communication unit 7050, which are connected to each other so as to be able to communicate with one another.

[0024] The control unit 7040 consists of a CPU 7041 and memory 7042 and controls the entire server 7000. Preferably, the control unit has a GPU (Graphics Processing Unit) 7043. The display unit 7010 consists of a display such as an LCD panel and can display images, etc. The operation unit 7020 consists of a mouse, keyboard, etc., and allows input operations by an administrator. The computer 1000 may have a touch panel, and the touch panel may perform the roles of the display unit 1010 and the operation unit 1020. However, the server 7000 may be configured to be operated by connection from an external machine and may not have an operation unit 7020.

[0025] The storage unit 7030 is composed of a storage medium such as an HDD or SSD, and stores various information, such as various programs (software), necessary for the operation of the server 7000. The programs are loaded into memory 7042 as needed and executed by the GPU 7043. Note that the location where these programs are stored is not limited to the storage unit 7030 of the server 7000. Thus, the hardware configuration of the server 7000 is the same as that of the computer 1000.

[0026] Figure 3 is a block diagram showing an example of the hardware configuration of a printer. As shown in Figure 3, the printer 2000 has a display unit (display means) 2010, an operation unit 2020, a storage unit 2030, a control unit 2040, a network communication unit 2050, and a printing unit 2060, which are connected to each other in a way that allows them to communicate with one another. The printer 2000 only needs to have a printing function, and for example, an MFP (Multi-Function Printer / Peripheral) or an SFP (Single Function Printer) can be used.

[0027] The control unit 2040 consists of a CPU 2041 and a memory 2042, and controls the entire printer 2000. The display unit 2010 consists of a display such as an LCD panel, and can display images, for example. The operation unit 2020 consists of a touch panel and various buttons, and allows input operations by the user 8000. Alternatively, the computer 1000 may have a touch panel, and the touch panel may perform the roles of the display unit 1010 and the operation unit 1020.

[0028] The memory unit 2030 is composed of a storage medium such as an HDD or SSD, and stores various information, such as various programs (software), necessary for the printer 2000 to operate. The programs are loaded into memory 2042 as needed and executed by the CPU 2041.

[0029] The network communication unit 2050 performs data input and output with external devices via an external network. The printing unit 2060, in accordance with instructions from the control unit 2040, converts the digital data stored in the storage unit 2030 and memory 2042 into an image and prints it on a printing medium (paper). The printing method can be, for example, an inkjet method, a toner method, or other methods.

[0030] <Description of this embodiment> This embodiment will explain the series of processes performed using troubleshooting as an example. Note that the processes are not limited to troubleshooting, and various other processes may be performed. Furthermore, in this embodiment, errors include not only those caused by human error, but also bugs, malfunctions, and defects.

[0031] Figure 4 is a sequence diagram showing an example of the flow from natural language input by the user to the completion of processing, illustrating an example of the interaction of each element in the present invention. Figure 5 is a diagram showing an example of a screen displayed when the processing of the present invention is executed. Figure 5(a) is an embodiment of a troubleshooting service to which the present invention is applied, and shows the state of UI500 when troubleshooting is completed. UI is an abbreviation for User Interface. In this process, the interaction shown in UI500 proceeds in stages. Also, the response will differ depending on the content sent.

[0032] The input field 514 of UI500 is an area where the user 8000 can freely input natural language (prompts). When the send button 515 is pressed, the content entered in input field 514 is confirmed and sent to the control mechanism execution server 3001. The sent content and the response are also displayed on UI500 as shown in texts 500-513. In Figure 5(a), texts 502, 507, 510, and 513 show natural language text prompts entered and sent by the user, while texts 501, 503-506, 508-509, and 511-512 are response texts generated by the generation AI.

[0033] In this embodiment, UI500 is shown as the UI of an independent application displayed on the computer 1000, but it is not limited to this. For example, it may be displayed in the driver of the printer 2000, or it may be displayed in conjunction with other applications or services. The application that displays UI500 is executed by the CPU 1041. In this embodiment, such a UI is represented on the display unit 1010 of the computer 1000, but it is not limited to this. For example, it may be displayed on the display unit 2010 of the printer 2000, or it may be displayed on a touch panel.

[0034] This embodiment will be explained using Figures 4 and 5(a).

[0035] User 8000 opens UI500, for example, to resolve a problem. UI500 can be opened in any way; for example, it may be opened when any button is pressed, or when there has been no activity for a certain period of time. Furthermore, UI500 is displayed on the display unit 1010 by the CPU 1041 controlling the display unit 1010.

[0036] UI500 displays text 501 prompting user 8000 to enter their inquiry. Note that displaying this text 501 is not mandatory.

[0037] In S400, user 8000 inputs details of the current problem or inquiry in natural language into input field 514 of UI500 (text 502). Specifically, user 8000 expresses information about an error that occurred in printer 2000 as a string of natural language and inputs it. This corresponds to S400 in Figure 4, and the natural language (prompt) is received by CPU 1041 via operation unit 1020. In this embodiment, the accepted natural language is described as a string. The user can express and input the error status of printer 2000 in natural language.

[0038] In S401, the CPU 1041 of computer 1000, which received input from user 8000, sends the inquiry details and a start command for troubleshooting to the communication interface 3010 of the control mechanism execution server 3000. The CPU 7041, which received the start command via the communication interface 3010 of the control mechanism execution server 3000, inputs the inquiry details and the start command for troubleshooting to the control mechanism 3020. As a result, the CPU 7041 receives the error information about the printer that was entered by user 8000 in natural language.

[0039] In S401, the computer 1000 may send the user 8000's input directly to the control mechanism execution server 3000. Alternatively, the computer 1000 may interpret the content entered by the user 8000 and send the trouble details and troubleshooting instructions to the control mechanism execution server 3000.

[0040] In S402, the CPU 7041 of the control mechanism execution server 3000 notifies the computer 1000 that it will start troubleshooting. Specifically, the CPU 7041 of the control mechanism execution server 3000 sends a request to the display unit 1010 of the computer 1000 to display the UI 500.

[0041] In S403, the CPU 1041 of the computer 1000, which received the notification, displays UI 500 on the display unit 1010. As a result, the UI 500 displayed on the display unit 1010 shows the text 503, "Searching for and creating a solution procedure."

[0042] Note that S402 and S403 are performed because it takes time to collect main unit information and create a troubleshooting sequence, allowing user 8000 to understand the current status, such as whether processing has started. However, these are not mandatory processes, and it is possible to configure the system so that S402 and S403 are not performed, and S404 is performed immediately after S401. In that case, text 503 will not be displayed on UI 500.

[0043] In this embodiment, the query is initiated when user 8000 opens UI500, but the trigger could be something else, such as the CPU 2041 of printer 2000 detecting an error that occurred in the printer itself.

[0044] Next, in S404, the CPU 7041 of the control mechanism execution server 3000 requests the printer 2000's main unit information. Specifically, the CPU 7041 of the control mechanism execution server 3000 sends a request to the printer 2000 to respond with the printer 2000's main unit information. Alternatively, the printer 2000 being used by the user may be identified from a database (not shown) that stores user information, or if there are multiple candidates, the candidates may be displayed and the user may select the correct one.

[0045] Here, the printer 2000 to which CPU 7041 makes a request is, for example, a printer 2000 connected to computer 1000. The connection method can be either wired or wireless, and the request is sent to any communicable printer on the same network.

[0046] In S405, the CPU 2041 of printer 2000, having received the request in S404, sends the printer 2000's main unit information to the control mechanism execution server 3000.

[0047] In this embodiment, the control mechanism 3020 controlled by the CPU 7041 communicates with the printer 2000 via API to confirm that no errors have occurred in the printer 2000, that there is sufficient ink, and that there are no abnormalities in other sensor values. The main unit information collected here is not limited to this information, but may be any information that can be used for troubleshooting, such as operational information such as the number of pages printed so far.

[0048] The main unit information is an example of status information managed by the printer 2000. The control mechanism 3020 may obtain this information directly from the printer 2000 or indirectly. Indirect acquisition means, for example, obtaining information stored in a database 3030 used as a so-called cloud service.

[0049] In this embodiment, the control mechanism 3020 does not limit the information it acquires from the printer 2000. It is not necessary to acquire all data, and it may acquire only the necessary data based on the content entered by the user 8000.

[0050] In S406, the CPU 7041 of the control mechanism execution server 3000 searches for and identifies information necessary for troubleshooting (responding to the inquiry) from the inquiry content received in S401 and the main unit information obtained in S405. The search target is the various DB3000 stored in the control mechanism execution server 3000. However, the search target is not limited to this and may also be DBs stored in other devices. The search method may also be, for example, to search for the solution corresponding to the error number if the error number is known.

[0051] Furthermore, for inquiries without error numbers, for example, a search method that searches for documents similar to the inquiry text is also acceptable. Specifically, the text data of documents describing how to operate the printer 2000, such as manuals and FAQs, could be vectorized, and the cosine similarity between this vectorized version of the inquiry text and the vectorized version could be compared to search for documents. Alternatively, both of these search methods could be combined, or a search method that selects the most suitable one by ranking the results could also be used. Note that S406 is not a mandatory configuration and may not be performed.

[0052] In S407, the CPU 7041 of the control mechanism execution server 3000 generates a sequence. Specifically, the CPU 7041 of the control mechanism execution server 3000 inputs a sequence generation instruction for troubleshooting to the generation AI 3040. The information input to the generation AI 3040 includes the inquiry received in S101, the printer 2000's main unit information obtained in S405, and the information identified by the search in S406 (see reference information below).

[0053] The generation AI 3040 generates and outputs a sequence based on this input information. If S406 is not executed, the information obtained by the search in S406 will not be input to the generation AI 3040. Furthermore, it is not necessary for all of this information to be input to the generation AI 3040; some of it may be used.

[0054] Upon receiving a sequence generation instruction, the generation AI3040 generates a sequence containing commands executable by the printer 2000 based on the received query content, device information, and reference information. Here, a command is an instruction that tells the computer to execute a specific function, and a sequence is a set of one or more commands. In this embodiment, an example of executing processing using a sequence is described, but it is not mandatory to make commands into a sequence. Furthermore, in this embodiment, a command is specifically described as information for the printer 2000 to execute.

[0055] The sequence generated here is information that is produced when the error information entered by user 8000 in S400 is input to the generation AI, and it is the information (sequence) that printer 2000 uses to execute the process to resolve this error.

[0056] In the troubleshooting sequence, by using some or all of the search results from S406, such as manuals and troubleshooting documents, as reference information, the generating AI3040 can create a logical response in natural language. By also including API-linked information as reference information, the generating AI3040 can create a program executable on the printer 2000 for running the troubleshooting sequence.

[0057] Furthermore, the generating AI 3040 responds by transmitting the sequence it generated in accordance with the received sequence generation instruction to the control mechanism 3020.

[0058] Furthermore, when the generation AI 3040 generates a sequence in S407, if there is insufficient reference information, it may send an instruction to the control mechanism 3020 to perform the reference information search performed in S406 again, and the control mechanism 3020 may perform the search again and send the results to the generation AI 3040. Also, if sequence creation is not completed in a single exchange with the generation AI 3040, the control mechanism 3020 may exchange information with the various DBs 3030 and the generation AI 3040 multiple times.

[0059] In S408, the control mechanism 3020 prepares to execute the sequence created by the generation AI 3040. Once preparation is complete, this processing flow moves to the step of troubleshooting (S409). Note that S409 is executed repeatedly until the problem is resolved, as will be described later. UI 500 notifies the user 8000 of this, as shown in text 504. In addition, for the sequences generated by the control mechanism execution server 3000 that the printer 2000 itself is to perform, the printer 2000 executes them when commands are sent from the control mechanism execution server 3000 to the printer 2000 as appropriate.

[0060] In this embodiment, the control mechanism 3020 is configured to sequentially execute the troubleshooting sequence. However, it is also possible to configure the system to create a program by extracting and compiling the program from the sequence in order to execute the program created by the generating AI 3040.

[0061] The program created by the control mechanism 3020 in this manner may be executed on a device other than the control mechanism execution server 3000. In this case, the control mechanism 3020 may notify the computer 1000 that the creation of the trouble resolution sequence is complete, and this fact may be displayed on the UI 500 so that the user 8000 can understand it.

[0062] In this embodiment, a configuration in which the computer 1000, printer 2000, and control mechanism execution server 3000 are connected to the same network 4000 has been described. However, this does not apply if they are not connected to the same network 4000. For example, consider a case where the computer 1000 and printer 2000 are connected to network 4000, and the control mechanism execution server 3000 is not connected to network 4000 but is connected to the computer 1000. In this case, the CPU 7041 of the control mechanism execution server 3000 cannot directly send information to the printer 2000, which is not connected to the control mechanism execution server 3000 via network 4000.

[0063] In this case, the sequence (command) generated by the control mechanism execution server 3000 is transmitted from the control mechanism execution server 3000 to the computer 1000. Specifically, the CPU 7041 of the control mechanism execution server 3000 transmits information to the computer 1000 via the network communication unit 7050 (cooperation I / F 3010), and the computer 1000 receives it. Subsequently, the CPU 1041 of the computer 1000 transmits the information received from the control mechanism execution server 3000 to the printer 2000 via the network communication unit 1050, and the printer 2000 receives it. The printer 2000, having received this information, executes processing based on this information, thereby realizing the above function.

[0064] As shown in Figure 1(b), the same applies when a DB server 5000 exists independently in addition to the control mechanism execution server 3000. In this case, a configuration may be used in which communication is performed indirectly via an information processing device such as a computer 1000, in order to realize the above functions. Conversely, a configuration may be used in which the computer 1000 executes the processing of the control mechanism execution server 3000. In this case, communication between the computer 1000 and the control mechanism execution server 3000 is not required.

[0065] Next, since the troubleshooting sequence is a complex procedure, we will first explain the flow of the system's internal processing in S409 of Figure 4, and then specifically explain the processing performed in this embodiment using Figures 5 and 6.

[0066] In S410, the CPU 7041 of the control mechanism execution server 3000 controls the control mechanism 3020 to confirm the next step to be executed. In S411, the CPU 7041 of the control mechanism execution server 3000 sends an instruction to the computer 1000 via the linkage interface 3010 to display the contents of the process on the display unit 1010 of the computer 1000 so that the contents of the process can be understood by the user 8000.

[0067] In S412, the CPU 1041 of computer 1000 displays the UI 500 on the display unit 1010 in accordance with the received instructions, so that user 8000 can understand the processing details on the UI 500. Note that the display location of the UI 500 is not limited to the display unit 1010 of computer 1000, but may be, for example, 2010 of printer 2000.

[0068] The operation in S413 is a step performed when user 8000 requires action; specifically, S414 is the step when action is taken in software, and S418 is the step when action is taken in hardware.

[0069] In the step of using software in S414, in S415, user 8000 inputs what they want to do (setting change instruction) regarding the process displayed in S412. User 8000's input may be made by selecting from the displayed options, or it may be made by user 8000 entering natural language into input field 514 of UI500, for example.

[0070] In S416, the CPU 1041 of computer 1000 sends the setting change instruction entered in S415 by user 8000 to the control mechanism execution server 3000. Specifically, it sends it to the control mechanism 3020 via the control mechanism execution server 3000's interoperability interface 3010.

[0071] In S417, the CPU 7041, which controls the control mechanism 3020 of the control mechanism execution server 3000 that received the setting change instruction, controls the printer 2000 to change its main settings according to the received setting change instruction. In other words, the CPU 7041 sends information (commands) to the printer 2000 via the network communication unit 7050 so that the printer 2000 can perform error resolution processing.

[0072] This change to the printer's settings may be performed using an API provided by the manufacturer of the printer 2000. Alternatively, the computer 1000 may connect to the printer 2000 using a communication method such as Wi-Fi (Wireless Fidelity), and the settings of the printer may be directly changed by the computer 1000's control.

[0073] The CPU 2041 of printer 2000, upon receiving a command, executes processing based on the received command. In other words, the CPU 2041 of printer 2000 executes processing based on the information (command) sent by the control mechanism execution server 3000. As a result, in S417, the control mechanism execution server 3000 can send instructions to printer 2000 and control printer 2000.

[0074] On the other hand, in the step where the S418 hardware is used, in S419, user 8000 performs operations on the printer 2000. These operations include, for example, various settings of the printer 2000 via the control unit 2020, as well as replacing ink cartridges and replenishing paper.

[0075] If the operation performed by user 8000 on the printer 2000 can be detected by sensors installed on the printer 2000 (S420), in S421, the CPU 2041 of the printer 2000 notifies the control mechanism 3020 that a setting has been changed or an operation has been performed. On the other hand, if the operation performed on the printer 2000 cannot be detected (S422), user 8000 inputs that an undetectable operation has been performed via the operation unit 1020 of the computer 1000 or the operation unit 2020 of the printer 2000 (S423). The CPU 1041 or CPU 2041 similarly transmits this information to the control mechanism 3020 (S424).

[0076] In S425, the control mechanism 3020 controlled by the CPU 7041 of the control mechanism execution server 3000 reflects the results of the actions taken in S414 or S418 into the sequence. In S426, the control mechanism 3020 controlled by the CPU 7041 of the control mechanism execution server 3000 notifies the computer 1000 of the results of the actions taken in S113 or S114 via the cooperation interface 3010.

[0077] In S427, the CPU 2041 of computer 1000 notifies the user by displaying the response result notified in S426 on UI 500. However, the method of notifying user 8000 in S427 is not limited to this. In this embodiment, user 8000 is notified by the CPU 1041 displaying UI 500 on the display unit 1010 of computer 1000, but the user may also be notified by the UI 500 being displayed on the display unit 2010 of printer 2000 or the like.

[0078] If there are multiple troubleshooting steps, these steps (S409) are repeated until the problem is resolved.

[0079] The system processing is as described above, but because it involves complex processing including loops and conditional branching, it will be explained in detail below using Figures 5 and 6. Figure 6 is a flowchart showing an example of the execution process of a troubleshooting sequence generated to resolve the problem exemplified in Figure 5. The flowchart in Figure 6 shows an example of the processing to resolve the example problem in this embodiment, and the sequence generated will differ depending on the problem that occurred and the prompt (inquiry content) entered by the user. Note that even if the cause of the problem is different, if the problem can be resolved by the same processing, the same sequence may be generated in each case.

[0080] First, the control mechanism 3020 receives the inquiry content entered by user 8000 in the input field 514 of UI 500, as shown in S400 and S401. In the example in Figure 5, user 8000 makes the inquiry, "The ink is not adhering properly." (text 502). Processing continues, and in S407, the generation AI 3040 generates a sequence to answer this inquiry. In other words, the user has entered the error status in natural language because a printing defect error has occurred in the printed material produced by printer 2000.

[0081] In S601, the control mechanism 3020 checks the ink level of the printer 2000. The ink level is checked using the printer 2000's main unit information obtained in S405. In this embodiment, as described in S405, it has been confirmed that the ink level is sufficient. Therefore, in S602, the CPU 7041, which controls the control mechanism 3020, determines that the ink level is sufficient.

[0082] The CPU 1041 displays UI500 on the display unit 1010 (text 505). Also, the branch at S602 proceeds to YES (S603). On the other hand, if it is determined that there is insufficient ink, the process proceeds to S608, and a method to resolve the printing problem by replenishing the ink is presented to UI500. In this case, a message such as "Ink level is low. Please replace the ink" is displayed on UI500.

[0083] In this embodiment, an example was described in which the main unit information acquired in S405 includes information on the remaining ink level. However, if the information on the remaining ink level cannot be acquired, a method may be presented in which the user 8000 checks the remaining ink level by operating the printer 2000 themselves.

[0084] In S603, the CPU 7041, which controls the control mechanism 3020, checks the ink tank. In this embodiment, it specifically checks whether the protective film on the ink tank has been removed. Since this requires hardware operation, the user 8000 is informed of this, including the check procedure (text 506). This process is carried out as described in S411-S412.

[0085] User 8000 actually checks the status of printer 2000 and confirms that there are no problems with the ink tanks. User 8000 inputs the results of the check on UI 500 (text 507). In S603, the control mechanism 3020 checks the ink tanks based on this user input.

[0086] In accordance with the control mechanism 3020 correctly receiving the report from user 8000, the CPU 10410 of computer 1000 displays text 08 to the UI 500 to confirm this. In this embodiment, since user 8000 has confirmed that there is no defect in the ink tank, the process proceeds to the next S605 (branch YES).

[0087] On the other hand, let's consider the case where a protective film remains attached to the ink tank and an ink tank error occurs. In this case, when user 8000 confirms that the protective film is still attached, they enter something like "there was a film on it" into input field 514 and send it. Based on this input from user 8000, the control mechanism 3020 determines that there is an ink tank error and proceeds to S607. At this point, UI500 displays instructions to user 8000 on how to remove the protective film.

[0088] Furthermore, if user 8000 confirms that the ink tank has a protective film, removes the protective film, correctly installs it in printer 2000, and enters this information in input field 514, UI500 will display that the ink has been correctly filled and the next steps will be displayed.

[0089] In S605, the printer 2000 performs nozzle cleaning under the control of the control mechanism 3020. In S606, the control mechanism 3020 confirms that the printer 2000 prints a check pattern to verify the improvement in the printing status. Specifically, the CPU 1041 displays text 509 on the UI 500 to inform the user 8000 that the printer 2000 will perform the nozzle cleaning and check pattern printing procedure (text 509).

[0090] When user 8000 approves the nozzle cleaning and check pattern printing (text 510), the CPU 7041 of the control mechanism 3020 issues an execution command to printer 2000, and S605 and S606 are executed. At this time, text 511 is displayed on UI 500, but this is not mandatory.

[0091] When the printer 2000 finishes performing nozzle cleaning, the control mechanism 3020 displays text 512 requesting the user 8000 to check the print result of the check pattern. The user 8000 responds by entering the result in the input field 514 and submitting it. In this embodiment, the user 8000 checks the printout and confirms that the problem (the content of the inquiry) has been resolved (text 513).

[0092] In this way, by using the present invention, a sequence for resolving problems can be created from data in documents such as manuals and device information, making it possible to resolve problems that involve a complex interplay of software and hardware operations.

[0093] Next, we will show a case where the above Example 1 is actually provided as a service. The system configuration and other configurations are the same as in Example 1, and we will explain the improvements to the trouble resolution sequence flow using Figure 7. Figure 7 is a flowchart of an example of the sequence execution process.

[0094] When S406 in the sequence shown in Figure 4 is initiated, the loop of S701 begins. The loop of S701 is repeated a predetermined number of times (an integer N in Figure 7), or until the problem is resolved. This predetermined number of repetitions N may be changed by the service provider or by user 8000.

[0095] Normally, using a generation AI becomes more costly with each use, so repeating unnecessary interactions is undesirable as it increases costs. Therefore, setting an upper limit on the number of repetitions (uses of the generation AI) allows for cost adjustment.

[0096] S702 corresponds to S406 and S407 in Figure 4, and refers to the creation of a troubleshooting sequence, including the search for troubleshooting steps. The S701 loop is for creating a new troubleshooting sequence using the generation AI3040, incorporating the information if the first troubleshooting sequence created did not resolve the problem. This allows for multiple troubleshooting attempts, improving the troubleshooting success rate. Next, the program enters the S703 loop.

[0097] S703 is a process that loops through the sequence execution process (S704) of S409. If the problem is not resolved even after executing the sequence once, it is possible that a step was skipped or the step was not executed correctly. To prevent this, a loop can be set in S703. The loop in S703 can be set to loop a set number of times M times from the beginning if the sequence execution is completed once, or if it fails midway. This set number M can be determined by the service provider or by the user 8000.

[0098] In this way, loop 2 and loop 1 are completed at S705 and S706 respectively, and processing can be carried out to resolve the problem.

[0099] When performing Example 1, it is possible that new problems may arise during the execution of the troubleshooting sequence. In this case, the troubleshooting process may be performed recursively. The number of calls to the recursive process may be limited to a predetermined number of times determined by the service provider.

[0100] Figure 8 shows an example of processing performed by the CPU 7041 of the control mechanism execution server 3000. This processing is initiated when a query is entered by user 8000, as explained in S400. In this processing, the generation AI 3040 is implemented by the control of the CPU 7041 of the control mechanism execution server 3000. Also, as shown in Figure 1(b), when the generation AI 3040 operates on the generation AI execution server 6000, it is implemented by the control of the CPU 7041 of the generation AI execution server 6000.

[0101] In S801, the CPU 7041 receives a troubleshooting instruction from the computer 1000 via the communication interface 3010 (S401). Upon receiving this troubleshooting instruction, the CPU 7041 controls the control mechanism 3020 and starts processing to resolve the problem. At this time, as shown in text 503, processing to notify the computer 1000 that troubleshooting has started (S403) may also be performed.

[0102] In S802, CPU7041 obtains the printer 2000's main unit information via the communication interface 3010. Specifically, CPU7041 sends a request for main unit information to printer 2000 via the communication interface 3010 (S404), and obtains the printer 2000's main unit information sent from printer 2000 in response to the request (S405).

[0103] The printer 2000's main unit information can be obtained directly from the printer 2000, or from the computer 1000. Alternatively, it can be obtained from the printer 2000 via the computer 1000, or from various DB3030s in step S803 described later.

[0104] In S803, CPU7041 retrieves necessary reference information for troubleshooting (responding to the inquiry) from various DB3030s based on the inquiry received in S801 and the printer 2000's main unit information obtained in S802 (S406). Note that S803 is not a mandatory configuration and may be omitted. In that case, S804 will be executed after S802.

[0105] In S804, the CPU 7041 uses the printer 2000's main unit information obtained in S802 and the reference information obtained in S803 to input a troubleshooting generation instruction to the generation AI 3040. The generation AI 3040 generates a troubleshooting sequence based on the input information and returns the result to the control mechanism 3020. As a result, the CPU 7041 obtains the troubleshooting sequence generated by the generation AI 3040.

[0106] In S805, CPU7041 prepares to execute the troubleshooting sequence acquired in S804 (S408). This preparation may differ depending on the troubleshooting sequence generated by generation AI3040 in S804.

[0107] For example, if the output format of the generation AI3040 in S804 is a language that requires compilation, such as a compiled language, then the preparation for execution is to perform the compilation. Alternatively, if the output format of the generation AI3040 is a language that does not require compilation, such as a scripting language, then an environment capable of executing sequences (commands) in that language is prepared. In addition, it is also possible to generate only function call sequences or a custom language, in which case the preparation is to create a control mechanism 3020 that corresponds to that.

[0108] In S806, CPU7041 repeatedly executes the troubleshooting sequence prepared in S805 until the problem is resolved (S409). If the problem is resolved in a single process, there is no need to repeat the process unnecessarily. The specific repetition procedure is shown in Figure 7. However, if CPU7041 determines that the problem cannot be resolved by the user alone, the process may be terminated by suggesting steps to resolve the problem, such as directing (displaying) a link to a customer support contact page.

[0109] As described above in the above embodiment, according to this embodiment, based on natural language inquiries from user 8000, documents such as manuals are input as reference information to the generating AI, enabling optimal troubleshooting without operator intervention. However, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist. The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of that system or device read and execute the program.

[0110] Furthermore, in this embodiment, the UI 500 is displayed on the display unit 1010 of the computer 1000, and communication takes place between the computer 1000 and the control mechanism execution server 3000, while communication with the user 8000 is conducted via the UI 500; however, the embodiment is not limited to this.

[0111] For example, it could be a browser UI such as Microsoft Edge or Google Chrome. Alternatively, it could be a generation application UI such as ChatGPT, a printing application UI such as Microsoft Office, or a chat application UI such as Microsoft Teams or Copilot. In that case, the control mechanism 3020, etc., exists as a plugin for these UIs, and each process is executed.

[0112] Furthermore, while this embodiment describes troubleshooting ink defects in printing as an example, the problems are not limited to this. For example, inquiries such as "the operation is not working as expected" may also be included.

[0113] Furthermore, in the printing system 101 shown in Figure 1(b), for example, each server, such as the generation AI execution server 6000, may be located outside of Japan, while the terminal device, computer 1000 (hereinafter referred to as "terminal device"), may be located within Japan. Even in this case, each server can send each file and data to the terminal device, and the terminal device can receive each file and data.

[0114] Even if each server is located outside of Japan, the transmission and reception (sending and receiving) of files and data within this system are performed as a single integrated process. Furthermore, since the system functions by receiving these files and data on terminal devices located within Japan, it can be considered that such transmission and reception took place domestically.

[0115] In this system, even if, for example, the server is located outside of Japan and the terminal device is located within Japan, the terminal device can perform the main functions of the system, and the effects of those functions can be realized within Japan. For example, even if the server is located outside of Japan, if the terminal device constituting this system is located within Japan, it is possible to use the system within Japan using that terminal device. Furthermore, the use of such a system may affect the economic interests of, for example, the patent holder.

[0116] Furthermore, although this embodiment uses a printing system as an example, it is not limited to the printer 2000, and could be a system for resolving problems with, for example, a camera or other electronic device. Moreover, the methods used in this invention are not limited to resolving problems, but can be applied to various general processes, such as answering inquiries about how to operate the device.

[0117] [Example 2] This example describes additional operations of Example 1. The hardware and software configurations of each component are the same as in Example 1.

[0118] This embodiment describes the processing that occurs when an interrupt, such as an additional inquiry from user 8000, is received by the CPU 7041 of the control mechanism execution server 3000 during the execution of a troubleshooting sequence. Figure 5(b) shows an example of the UI 500 screen during the execution of the troubleshooting sequence. In this process, the interaction shown in UI 500 progresses in stages, and the response differs depending on the content sent.

[0119] Figure 5(b) shows that after text 509, the AI's response text, is displayed, user 8000 sends text 520 in response to text 509, which is intended to request permission to execute the process, asking about the content of the process. Specifically, user 8000, not knowing the meaning of the term "nozzle clean" contained in text 509, inquires what "nozzle clean" means, as in text 520.

[0120] In this embodiment, as shown in text 521, the system presents (displays) a response to the user 8000's inquiry, asks the user whether to execute the process, and requests permission to execute the process. Because this system utilizes a generation AI, it can interpret the prompt from user 8000 as an inquiry rather than an instruction to execute a process in the troubleshooting sequence, and respond to the inquiry. As a result, user 8000 can understand the content of the process and decide whether or not to instruct the execution of the process, and then execute the process, thus increasing the likelihood that the process will resolve the problem.

[0121] This interpretation by the generating AI is performed when natural language (prompt) is received from the user (not shown). Specifically, in Figure 4, the generating AI 3040 in the control mechanism execution server 3000 interprets the prompt. Note that this is not the only configuration; for example, a system that operates the UI 500 may perform the interpretation. With this configuration, the generating AI 3040 interprets the prompt entered by the user 8000, enabling the CPU 7041 to perform interrupt processing.

[0122] User 8000, having confirmed the response (text 521), sends text 522. In the example in Figure (b), User 8000, having understood the "nozzle clean" process from the response (text 521) and determined that this process needs to be performed, sends a message (text 522) indicating that the process will be executed. This completes the interrupt processing, and from this point onward, the same display as in Figure 5(a) is shown, and the sequence is executed in the same manner. As stated in text 521, nozzle clean is a process that cleans the printer nozzles, and is an example of a process performed by printer 2000.

[0123] Thus, by using this system, even when user 8000 makes an inquiry during sequence execution, the generating AI 3040 can respond to the inquiry, and the sequence can be executed to resolve the problem.

[0124] [Other examples] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of Symbols]

[0125] 100, 101 Printing Systems 1000 computers 2000 Printer 3000,3001 Control mechanism execution server 4000 Networks 5000 DB servers 6000 AI generation execution server

Claims

1. A means of receiving information about printer errors from the user in natural language, Information obtained by inputting the error information into the generating AI, which is information that the printer uses to perform a process to resolve the error, is received by a receiving means. The system includes a transmission means for transmitting the received information to the printer, Based on the transmitted information, the printer performs the process. An information processing device characterized by the following:

2. The system further includes an acquisition means for acquiring information about the printer from the printer, The receiving means receives information obtained when the error information and the acquired printer information are input to the generating AI, which is information that the printer will use to perform a process to resolve the error. The information processing apparatus according to feature 1.

3. It further has an input means, The input means inputs the received natural language and the acquired printer information into the generating AI. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, based on the input of the natural language received by the input means and the acquired printer information. The information processing apparatus according to feature 2.

4. The transmission means transmits the received natural language and the acquired printer information to the server that enables the generation AI to function. The receiving means receives from the server information generated by the generating AI, which is information that the printer uses to perform error resolution processing, when the natural language received by the transmitting means and the acquired printer information are transmitted to the server. The information processing apparatus according to feature 2.

5. The system further includes identification means for identifying information necessary to resolve the error from the database, The receiving means receives information generated by the generating AI based on the error information and the identified information, which is information that the printer uses to perform a process to resolve the error. The information processing apparatus according to feature 2.

6. It further has an input means, The input means inputs the received natural language and the acquired printer information into the generating AI. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, based on the input of the natural language received by the input means and the acquired printer information. The information processing apparatus according to feature 5.

7. The transmission means transmits the received natural language and the acquired printer information to the server that enables the generation AI to function. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, upon transmission of the natural language received by the transmitting means and the acquired printer information to the server. The information processing apparatus according to feature 5.

8. The information processing device according to any one of claims 5 to 7, characterized in that the information necessary to resolve the error is information contained in a document describing how to operate the printer.

9. The information processing device according to claim 1, characterized in that the error information is a representation of the error state occurring in the printer using a natural language string.

10. The information processing device according to claim 1, characterized in that the printer information is information used for troubleshooting the error that occurred in the printer, and is status information managed by the printer.

11. The information processing device according to claim 1, characterized in that the information for the printer to perform the aforementioned processing is a command.

12. The aforementioned error is a printing defect in the printed material produced by the printer. The process for resolving the aforementioned error is the process of cleaning the nozzles of the printer. The information processing apparatus according to feature 1.

13. A means of receiving information about printer errors from the user in natural language, Information obtained by inputting the error information into the generating AI, which is information that the printer uses to perform a process to resolve the error, is received by a receiving means. The system includes a transmission means for transmitting the received information to the printer, The printer executes the processing based on the transmitted information. An information processing system characterized by the following:

14. The system further includes an acquisition means for acquiring information about the printer from the printer, The receiving means receives information obtained when the error information and the acquired printer information are input to the generating AI, which is information that the printer will use to perform a process to resolve the error. The information processing system according to feature 13.

15. It further has an input means, The input means inputs the received natural language and the acquired printer information into the generating AI. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, based on the input of the natural language received by the input means and the acquired printer information. The information processing system according to feature 14.

16. The transmission means transmits the received natural language and the acquired printer information to the server that enables the generation AI to function. The receiving means receives from the server information generated by the generating AI, which is information that the printer uses to perform error resolution processing, when the natural language received by the transmitting means and the acquired printer information are transmitted to the server. The information processing system according to feature 14.

17. The system further includes identification means for identifying information necessary to resolve the error from the database, The receiving means receives information generated by the generating AI based on the error information and the identified information, which is information that the printer uses to perform a process to resolve the error. The information processing system according to feature 14.

18. It further has an input means, The input means inputs the received natural language and the acquired printer information into the generating AI. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, based on the input of the natural language received by the input means and the acquired printer information. The information processing apparatus according to feature 17.

19. The transmission means transmits the received natural language and the acquired printer information to the server that enables the generation AI to function. The receiving means receives information generated by the generating AI, which is information that the printer uses to perform error resolution processing, upon transmission of the natural language received by the transmitting means and the acquired printer information to the server. The information processing apparatus according to feature 17.

20. The information processing system according to any one of claims 17 to 19, characterized in that the information necessary to resolve the error is information described in a document describing how to operate the printer.

21. The information processing system according to claim 13, characterized in that the error information is a representation of the error state occurring in the printer using a natural language string.

22. The information processing system according to claim 13, characterized in that the printer information is information used for troubleshooting the error that occurred in the printer, and is status information managed by the printer.

23. The information processing system according to claim 13, characterized in that the information for the printer to perform the aforementioned processing is a command.

24. The aforementioned error is a printing defect in the printed material produced by the printer. The process for resolving the aforementioned error is the process of cleaning the nozzles of the printer. The information processing apparatus according to feature 13.

25. A receiving process that receives information about printer errors from the user in natural language, A receiving step which receives information obtained by inputting the error information into the generating AI, which is information that the printer uses to perform a process to resolve the error, The computer is made to perform a transmission step of sending the received information to the printer, Based on the transmitted information, the printer performs the process. A program characterized by the following features.

Images