A generation AI application for troubleshooting assistance, its client application, and an information processing device equipped with the same.

Abstract

In troubleshooting support using generative AI, we provide a generative AI application, a client application, and an information processing device that smoothly implement the necessary countermeasures. [Solution] In the equipment diagnostic system, the diagnostic application obtains a method for dealing with a problem that has occurred with the equipment in S404, and if the generating AI application determines that the first user involved in the problem cannot deal with it themselves, it answers YES in S413, and the client PC receives instructions from the first user to request assistance in S415. Then, based on the received instructions, the generating AI application requests assistance in dealing with the problem from a second user, who is different from the first user, in S417.

Description

Technical Field

[0001] This disclosure relates to equipment diagnosis technology using generative AI.

Background Art

[0002] In recent years, there has been an increasing number of cases where functions such as context understanding and information extraction by AI technology are utilized to assist in troubleshooting electronic devices such as MFPs. For example, Patent Document 1 discloses a technology that diagnoses and predicts troubles using the telemetry data of a device with AI and makes settings changes to the device based on the diagnosis and prediction results. Also, the generative AI tool "Copilot" provided by Microsoft realizes troubleshooting support by collaborating with general applications such as Word.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In troubleshooting support for equipment using generative AI, cooperation from someone other than the user involved in the trouble may be necessary to implement the actions required to resolve the trouble. For example, the user does not have the permissions necessary to change the settings or perform operations on the target equipment, operations by technical support with specialized knowledge are required, or the situation where it is difficult for the user himself / herself to directly access the target equipment, etc. And in the conventional troubleshooting support using generative AI, when facing the above situations, it may happen that the actions required to resolve the trouble cannot be realized.

Means for Solving the Problems

[0005] The generation AI application for assisting in troubleshooting equipment according to this disclosure is characterized by comprising: a first acquisition step of acquiring a method for dealing with a problem that has occurred with the equipment; a first reception step of receiving instructions from a first user who is involved in the problem and has received the suggested method for dealing with the problem, if the first user is unable to deal with the method themselves; and a request step of making a request to a second user, different from the first user, to deal with the problem based on the received instructions. [Effects of the Invention]

[0006] According to the technology disclosed herein, necessary actions can be taken smoothly in troubleshooting equipment using generated AI. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram illustrating the system configuration and hardware configuration of the equipment diagnostic system. [Figure 2] A block diagram illustrating the software configuration of the equipment diagnostic system according to Embodiment 1. [Figure 3] (a) to (e) are diagrams showing examples of tables that are under the management of a database module. [Figure 4] A sequence diagram showing the operation flow of the equipment diagnostic system according to Embodiment 1. [Figure 5] This diagram shows an example of a UI screen for a generated AI client application that provides troubleshooting assistance. [Figure 6] This diagram shows an example of a UI screen for a generated AI client application that provides troubleshooting assistance. [Figure 7] A block diagram illustrating the software configuration of the equipment diagnostic system according to Embodiment 2. [Figure 8] A sequence diagram showing the operation flow of the equipment diagnostic system according to Embodiment 2. [Figure 9]This diagram shows an example of a UI screen for a generated AI client application that provides troubleshooting assistance. [Figure 10] This diagram shows an example of a UI screen for a generated AI client application that provides troubleshooting assistance. [Modes for carrying out the invention]

[0008] The embodiments for carrying out the present invention will be described below with reference to the drawings. Note that the following embodiments are not limiting to the present invention, and not all combinations of features described in these embodiments are essential to the solution of the present invention. The same components will be denoted by the same reference numerals.

[0009] [Embodiment 1] In this embodiment, as an example of a device diagnostic system, we will describe a case in which a diagnostic service and a generation AI service operating on the cloud are used to diagnose a Multifunction Peripheral (MFP), commonly known as a multifunction printer. Note that the device to be diagnosed is not limited to an MFP; it may also be an electronic device specialized in a specific function, such as a printer, fax machine, or scanner.

[0010] <System Configuration / Hardware Configuration> Figure 1 is a block diagram illustrating the system configuration and hardware configuration of the equipment diagnostic system of this embodiment.

[0011] The equipment diagnostic system consists of a diagnostic cloud 100, a generation AI cloud 120, a client computer 140, and an MFP 160, all interconnected by a network 180.

[0012] The CPU 101 of the diagnostic cloud 100 is an arithmetic processing unit that comprehensively controls each device connected to the system bus 111 and executes application programs stored in the ROM 103 or external memory 110. The CPU 101 also opens various registered windows based on commands indicated by a mouse cursor (not shown) on the display 109 and performs various data processing. The RAM 102 functions as the main memory and work area of ​​the CPU 101. The ROM 103 is a read-only memory that functions as a storage area for basic I / O programs, etc. The operating system program (hereinafter referred to as "OS"), which is the control program for the CPU 101, is stored in this ROM 103 or external memory 110. Furthermore, the ROM 103 or external memory 110 stores files and other various data used when executing the above application programs, etc. The network I / F 104 is an interface that connects to the network 180 and communicates with external devices. The keyboard I / F 105 is an interface that controls input from the keyboard 108 and a pointing device (not shown). The display I / F 106 is an interface that controls the display of the display 109. The external memory I / F 107 is an interface that controls access to external memory 110, such as a hard disk drive (HDD) or solid state drive (SSD). External memory 110 consists of an HDD or SSD and stores boot programs, various applications, user files, edit files, etc. The diagnostic cloud 100 operates when the CPU 101 is executing the basic I / O program and OS written to ROM 103 or external memory 110. The basic I / O program is written to ROM 103, and the OS is written to either ROM 103 or external memory 110. When the computer is powered on, the initial program load function in the basic I / O program writes the OS from ROM 103 or external memory 110 to RAM 102, and the OS starts operating. The system bus 111 is a bus that connects each device.Furthermore, the hardware resources that make up the diagnostic cloud 100, such as the CPU 101, ROM 103, and external memory 110, are supplied on demand using virtualization technology. Because these hardware resources are supplied on demand using virtualization technology, the diagnostic cloud 100 is configured as a virtual server on a cloud computing environment.

[0013] The hardware configuration of the generation AI cloud 120 and client computers (hereinafter referred to as "client PCs") 140 is the same as that of the diagnostic cloud 100, so the explanation will be omitted.

[0014] In the MFP160, the CPU161 is a processing unit that comprehensively controls each device connected to the system bus 173 and executes control programs corresponding to each function. For example, when a user uses the printer function, it executes a printer control program and outputs an image signal as output information to the printer 168 via the printer I / F 167. The control program is stored in ROM 163 or external memory 172. The CPU161 also communicates with external devices via the network I / F 164. Furthermore, the CPU161 executes various processes based on application programs stored in ROM 163 or external memory 172. RAM 163 functions as the main memory and work area of ​​the CPU161, and its memory capacity can be expanded by optional RAM connected to an expansion port (not shown). RAM 162 is used for the output information expansion area, environment data storage area, NVRAM, etc. ROM 163 is a read-only memory that functions as a storage area for basic I / O programs, etc. The ROM 163 or external memory 172 stores the control program and application programs for the CPU 161, font data used when generating the above output information, and information used on the MFP 160. The operation unit I / F 165 is an interface with the operation unit 166, and is an interface that outputs image data to be displayed to the operation unit 166 and receives information input by the user via the operation unit 166. The operation unit 166 is composed of, for example, an operation panel equipped with switches and LEDs for user operation. The printer I / F 167 is an interface that outputs image signals as output information to the printer 168 (printer engine). The scanner I / F 169 is an interface that receives image signals as input information from the scanner 170 (scanner engine). The external memory I / F (memory controller) 171 is an interface that controls access to external memory 172 such as a hard disk drive (HDD) or IC card. Note that there is not limited to one external memory 172, and multiple external memory 172 may be connected.Furthermore, it may have an NVRAM (not shown) and be configured to store the printer mode setting information from the operation unit 166. The system bus 173 is a bus that connects each device.

[0015] <Software Configuration> FIG. 2 is a block diagram for explaining the software configuration of the device diagnosis system according to the present embodiment. Hereinafter, the software configuration of the device diagnosis system will be described with reference to FIG. 2.

[0016] ≪Software Configuration of Diagnosis Cloud≫ First, the software configuration of the diagnosis cloud 100 will be described. The diagnosis cloud 100 is composed of a network module 200, a web server service module 201, a device diagnosis application 202, and a database module 208. And these components exist as files stored in the external memory 110. That is, these are program modules that are loaded into the RAM 102 and executed by modules that use the OS and its modules at the time of execution. In addition, the device diagnosis application 202 can be added to the HDD or SSD of the external memory 110 supplied on demand by virtualization technology in the cloud computing environment.

[0017] The network module 200 performs network communication with the generation AI cloud 120 and the MFP 160 using an arbitrary communication protocol. The web server service module 201 provides a service that receives an HTTP request from the Plugin application 224 of the generation AI cloud 120 and returns an HTTP response. When returning an HTTP response, the web server service module 201 may request the device diagnosis application 202 to generate the HTTP response.

[0018] The device diagnostic application 202 is an application that diagnoses the MFP160 and takes action based on the diagnosis results via the network 180. The device diagnostic application 202 is implemented as a program that executes processing in response to requests to a Web API provided by, for example, the Web server service module 201. Together with the Web server service module 201, the device diagnostic application 202 realizes a cloud service that diagnoses the MFP160 and takes action based on the diagnosis results. The device diagnostic application 202 includes a Web API module 203, a device management module 204, a diagnostic module 205, an action module 206, and a user management module 207. The Web API module 203 generates an HTTP response by calling the necessary modules based on a request from the Web server service module 201. The device management module 204, diagnostic module 205, action module 206, and user management module 207 are examples of modules called by the Web API module 203. However, the Web API module 203 may also call other modules. The device management module 204 obtains device information, such as the device's ID and status, and log information, such as job history, from the target device (in this case, the MFP160) via the network module 200. The acquisition of device information and log information is performed using any communication protocol, either through push communication from the MFP160 or polling from the diagnostic cloud 100. Examples of communication protocols include HTTPS (Hypertext Transfer Protocol Secure). The device management module 204 stores the device information and log information obtained from the MFP160 in the device information table and log table managed by the database module 208 (described later). It also retrieves information from the device information table and log table as needed. The diagnostic module 205 obtains the necessary diagnostic information from multiple tables on the database module 208 (described later) to diagnose problems (defects / issues) occurring in the target device. The diagnostic results are returned as an HTTP response via the Web API module 203.The action module 206 performs actions to resolve problems occurring in the target device based on the diagnostic results of the diagnostic module 205. The results of the actions are sent as an HTTP response via the Web API module 203. Actions performed by the action module 206 include, for example, setting changes or restart instructions for the target device. The action module 206 will send instructions to the target device via the network module 200, using any communication protocol. Examples of communication protocols used by the action module 206 include HTTPS as mentioned above, as well as XMPP, MQTT, AMQP, etc. Here, XMPP stands for Extensible Messaging and Presence Protocol. MQTT stands for Message Queuing Telemetry Transport. AMQP stands for Advanced Message Queuing Protocol. The user management module 207 receives a check request regarding the permissions of the user making the trouble inquiry and returns information about the permissions held by that user as an HTTP response via the Web API module 203. In this process, the necessary information is retrieved from the user table 303 and authorization table 304 on the database module 208 (described later) and a response is provided. The database module 208 manages various types of data and stores and retrieves data according to requests from other modules. The database module 208 may be an external module accessible from the device diagnostic application 202, and may be implemented, for example, as a database service on another cloud computing environment.

[0019] ≪Software Configuration of the Generative AI Cloud≫ Next, the software configuration of the generation AI cloud 120 will be described. The generation AI cloud 120 consists of a network module 220, a web server service module 201, a generation AI application 222, and a plugin application 224. These components exist as files stored in external memory 110. In other words, these are program modules that are loaded into RAM 102 and executed at runtime by the OS or modules that utilize these modules. Furthermore, the generation AI application 222 and the plugin application 224 can be added to the HDD or SSD of the external memory 110, which is supplied on demand using virtualization technology in the cloud computing environment. The network module 220 performs network communication with the diagnostic cloud 100 and the client PC 140 using any communication protocol. The web server service module 221 receives HTTP requests from the generation AI client application 241 on the client PC 140 and provides a service to return HTTP responses. When returning an HTTP response, the web server service module 221 may request the generation AI application 222 to generate an HTTP response. The Generative AI Application 222 is an artificial intelligence system application that generates responses to arbitrary text information input by a user using a Large Language Model (LLM), etc. The Generative AI Application 222 is implemented, for example, as a program that executes processing in response to requests to a Web API provided by the Web Server Service Module 221. Together with the Web Server Service Module 221, the Generative AI Application 222 realizes a cloud service for Generative AI. The Generative AI Application 222 has a Web API Module 223. Based on requests from the Web Server Service Module 221, the Web API Module 223 calls the necessary modules to generate an HTTP response.Plugin application 224 is an application that adds and executes specific functions to generation AI application 222 through the plug-in mechanism of generation AI application 222. In this embodiment, the remote diagnostic function of the diagnostic cloud 100 is added to generation AI application 222 by Plugin application 224. The cooperation between Plugin application 224 and the device diagnostic application 202 of the diagnostic cloud 100 is achieved by sending and receiving HTTP request messages and HTTP response messages. Plugin application 224 sends an HTTP request message to the Web API module 203 of the diagnostic cloud 100 via the network module 220 and receives an HTTP response message returned from the diagnostic cloud 100.

[0020] ≪Client PC Software Configuration≫ Next, the software configuration of the client PC 140 will be described. The client PC 140 consists of a network module 240, a generation AI client application 241, and a printer driver 242. These components exist as files stored in external memory 110. In other words, these are program modules that are loaded into RAM 102 and executed at runtime by the OS or modules that utilize this module. The network module 240 performs network communication with the generation AI cloud 120 and MFP 160 using any communication protocol. The generation AI client application 241 sends an HTTP request message to the generation AI cloud 120 via the network module 240 and receives and displays an HTTP response message from the generation AI cloud 120. Access from the client PC 140 to the generation AI cloud 120 is performed through the generation AI client application 241. The printer driver 242 creates a print job, which is print instruction data, and sends it to the MFP 160 via the network module 240. The printer driver 242 also receives and displays the execution result of the print job from the MFP 160 via the network module 240.

[0021] ≪MFP Software Configuration≫ Next, the software configuration of the MFP160 will be described. The MFP160 consists of a network module 260, a print module 261, a scan transmission module 262, a fax module 263, a management application 264, and a UI module 268. These components exist as files stored in ROM 163 or external memory 172, and are loaded into RAM 162 and executed at runtime. The network module 260 communicates with the diagnostic cloud 100 and client PC 140 using any communication protocol. The print module 261 receives print jobs sent from the printer driver 242 of the client PC 140 via the network module 260 and executes the printing process according to the print job. The print module 261 also creates log information of completed print jobs and sends it to the log management module 265. The scan transmission module 262 generates and executes scan jobs and scan data transmission jobs based on scan instructions from the user received via the UI module 268, which will be described later. Here, protocols such as email or SMB (Server Message Block) are used to send scanned data. The scan transmission module 262 also creates log information for completed scan jobs and scan data transmission jobs and sends it to the log management module 265. The fax module 263 receives fax reception jobs from fax devices (not shown) via the network module 260. Received fax reception jobs are printed via the print module 261 or forwarded to other MFPs, etc., via the network module 260. In addition, the fax module 263 generates and executes fax transmission jobs based on fax transmission instructions received from users via the UI module 268. The fax module 263 also creates log information for completed fax reception jobs and fax transmission jobs and sends it to the log management module 265. The management application 264 consists of the log management module 265, the settings management module 266, and the power management module 267.The log management module 265 receives requests for device information / log information from the device management module 204 of the diagnostic cloud 100 via the network module 260 and returns the device information and log information. The configuration management module 266 is a module that manages the setting values ​​for various settings of the MFP 160. Based on setting confirmation and setting change instructions received from the user via the UI module 268, the configuration management module 266 processes the return or change of the setting values ​​related to the instructions. The configuration management module 266 also receives setting change instructions sent from the treatment module 204 of the diagnostic cloud 100 via the network module 260 and changes the setting values ​​of the settings related to the change instructions. The power management module 267 is a module that manages the power status of the MFP 160. Based on power off or restart instructions received from the user via the UI module 268, the power management module 267 powers off or restarts the MFP 160. Furthermore, the power management module 268 receives power-off or restart instructions transmitted from the diagnostic cloud 100's treatment module 204 via the network module 260 and performs power-off or restart of the MFP 160. The UI module 268 draws the UI screen displayed on the MFP 160's operation unit 166 and receives user instructions on the operation unit 166.

[0022] <An example of a table managed by the database module> Next, we will explain the various tables managed by the database module 208 of the diagnostic cloud 100. Figures 3(a) to (e) show examples of tables under the management of the database module 208.

[0023] Figure 3(a) is a device information table that stores information about the device being diagnosed (in this case, the MFP160). The device information table in Figure 3(a) consists of the following items: "Device ID", "Device Name", "Model Name", "IP Address", "Serial No.", "Status", and "Last Updated". Here, "Device ID" contains an identifier that uniquely identifies the device. "Device Name" and "Model Name" contain the name and model name of the device, respectively. "IP Address" contains the IP address assigned to the device, and "Serial No." contains the serial number of the device. "Status" contains information indicating the status of the device, and "Last Updated" contains information indicating the date and time when the record was last updated.

[0024] Figure 3(b) shows the log table in which the device management module 204 stores log information acquired from the target device. The log table in Figure 3(b) consists of the following items: "Device ID", "Job ID", "Job Type", "Start Time", "END Time", "User Name", "Result", and "Error code". Here, "Device ID" contains an identifier that uniquely identifies the target device, and "Job ID" contains an identifier that uniquely identifies the target job. "Job Type" contains information indicating the type of job. "Start Time" and "END Time" contain information on the job execution start date and time and the job execution end date and time. "User Name" contains the name of the user who instructed the job execution, "Result" contains information indicating whether the job execution was successful or unsuccessful, and "Error code" contains the error code if the job execution failed. Here, a job refers to an execution command generated when the MFP160 performs tasks such as printing, scanning, sending, or faxing, and the job identifier is an identifier that uniquely identifies the job. The error code is a code used to uniquely identify the cause of the job error.

[0025] Figure 3(c) is a table that stores the diagnostic logic used by the diagnostic module 205 when performing a diagnosis. The diagnostic logic table in Figure 3(c) consists of the following items: "Logic ID", "Model Name", "Job Type", "Error code", "Status", and "Remediation". Here, "Logic ID" contains an identifier that uniquely identifies each logic, and "Model Name" contains the model name of the target device. "Job Type" contains information indicating the type of job. "Error code" contains the error code when the job fails. "Status" contains information indicating the status of the target device, and "Remediation" contains the method for dealing with the trouble (i.e., the action to be taken for each malfunction / problem that matches the model name, job type, error code when the job failed, status, etc.).

[0026] Figure 3(d) shows the user table that stores information for each user of the diagnostic cloud. The user table in Figure 3(d) consists of the following items: "User ID", "User Name", and "Role". "User ID" contains an identifier that uniquely identifies each user, "User Name" contains the user's name, and "Role" contains information indicating whether the user is a regular user or an administrator.

[0027] Figure 3(e) is a permission table that stores information about the permissions of each user in the diagnostic cloud. The permission table in Figure 3(e) consists of two items: "Action" and "Permissions". "Action" contains information indicating the content of the function or operation, and "Permissions" contains information indicating the user attributes required for that function or operation. Note that the permission table in Figure 3(e) is a table that stores permission information on a per-user basis in the diagnostic cloud, but it could also be a table that stores permission information on a per-user basis in, for example, the MFP160. Furthermore, the diagnostic cloud 100 may acquire user information and permission information managed internally by the MFP160 and manage it as a table. In addition, the tables shown in Figures 3(a) to (e) are merely examples, and different information may be stored in each table, and there may be other tables besides those shown in Figures 3(a) to (e).

[0028] <Operation sequence of the equipment diagnostic system> Next, the operation of the equipment diagnostic system according to this embodiment will be explained using the sequence diagram in Figure 4. Here, we will explain using as an example a scenario in which a user who has failed to send scanned images of a document externally (scan transmission) using the MFP160 uses the generation AI to troubleshoot. In the following explanation, the symbol "S" represents a step.

[0029] In S401, the generation AI client application 241 on the client PC 140 receives an inquiry (prompt) about a problem occurring in the MFP 160 and sends it to the generation AI cloud 120. The user making the inquiry logs in to the generation AI cloud 120 from the client PC 140 and enters the inquiry details as text information via the UI screen displayed by the generation AI client application 241. Figure 5 is an example of a user interface screen (UI screen) for which the generation AI client application 241 provides troubleshooting support, and the username of the logged-in user, "User1", is displayed in the username area 500. At this step, the text information 5011 of the inquiry details entered by the user is displayed in the prompt area 501 of the UI screen in Figure 5. In the following explanation, the user who made the inquiry will be referred to as the "inquiry user".

[0030] In S402, the generation AI application 222 of the generation AI cloud 120 receives a query (prompt) from the client PC 140 and forwards the query to the Plugin application 224. Then, in S403, the Plugin application 224 sends a diagnostic request for the query received from the generation AI application 222 to the diagnostic cloud 100.

[0031] In S404, the device diagnostic application 202 of the diagnostic cloud 100, which received the request, performs a diagnosis regarding the query. Specifically, first, the device diagnostic application 202 identifies the job to be diagnosed by referring to the log table of the database module 208 via the device management module 204. In the example above, the job "Scan&Send(Email)" which is experiencing an execution error due to the querying user's username "User1" is identified. Once the job to be diagnosed is identified, the device diagnostic application 202 obtains the corrective action associated with the identified job by referring to the diagnostic logic management table 302 of the database module 208 via the diagnostic module 205. In the example above, the corrective action "Turn off communication setting 1" corresponding to error code "777" is obtained. Once the diagnostic results are obtained, in S405, the device diagnostic application 202 responds to the generating AI cloud 120, which is the source of the diagnostic request, with the diagnostic results (the error log of the job related to the query and its corrective action).

[0032] In S406, the Plugin application 224 of the generation AI cloud 120 receives the diagnostic results and forwards them to the generation AI application 222. Then, in S407, the generation AI application 222 responds to the inquiry user's client PC 140 with the diagnostic results received from the Plugin application 224.

[0033] In S408, the AI ​​client application 241 on the client PC 140 presents the diagnostic results received from the AI ​​cloud 120 to the user making the inquiry. Specifically, the error log of the job being diagnosed and its solution, which are included in the diagnostic results, are displayed on the UI screen viewed by the user making the inquiry. In the example above, the prompt area 501 in Figure 5 displays a message 5012 indicating that the problem is a scan transmission error with the error code "777," and that the suggested solution is to "turn off communication setting 1." If the user making the inquiry wishes to implement the suggested solution, they will enter text information 5013 to that effect. Upon receiving this user instruction, the AI ​​client application 241 sends a command to execute the solution to the AI ​​cloud 120.

[0034] In S409, the generation AI application 222 of the generation AI cloud 120 receives an action execution instruction from the inquiry user's client PC 140 and forwards it to the Plugin application 224. Then, in S410, the Plugin application 224 sends a check request to the diagnostic cloud 100 to check whether the execution of the action included in the diagnostic results received in S406 is possible with the inquiry user's authority.

[0035] In S411, the device diagnostic application 202 of the diagnostic cloud 100, which received the request for an execution permission check, checks the execution permission of the querying user via the user management module 207 and the database module 208. Specifically, it refers to the user table and permission table shown in Figures 3(d) and (e) above to confirm whether the querying user has the execution permission for the action related to the execution instruction. Currently, the querying user "User1" only has general user permission, and it is found that administrator permission is required for the action related to the execution instruction, "Turn off communication setting 1". In addition, a search for users with administrator permission (hereinafter referred to as "authorized users") is also performed, and for example, "User3" is identified as an authorized user. In the following S412, the device diagnostic application 202 responds to the generation AI cloud 120, the source of the check request, with the result of the execution permission check. Currently, the check result indicating that the querying user "User1" does not have the permission to perform the action "Turn off communication setting 1", and that "User3" and others have the execution permission, is sent to the generation AI cloud 120.

[0036] In S413, the Plugin application 224 of the generation AI cloud 120 receives the result of the execution permission check and determines whether cooperation from another user is necessary based on the result. Based on the determination, the next process to be executed is distributed to either S414 or S420. If the querying user does not have execution permission, a response (S414) indicating that action by an authorized user with execution permission is required is made next. On the other hand, if the querying user does have execution permission, a request to execute the solution (S420) is made next. In the example above, the querying user "User1" does not have the administrator privileges necessary for the solution "Turn off communication setting 1", so it is determined that action by an authorized user is required, and S414 is executed.

[0037] In S414, the Plugin application 224 responds to the inquiry user's client PC 140, indicating that action by an authorized user is required. In S415, the generation AI client application 241 on the inquiry user's client PC 140 displays information on the UI screen indicating that action by an authorized user is required, based on the response from the generation AI cloud 120, and accepts instructions for a proxy request. Specifically, the UI screen displays a list of one or more authorized user usernames included in the received response, allowing the inquiry user to select whether to request an authorized user to perform the action on their behalf. In the example above, the prompt area 501 in Figure 5 displays a message 5014 indicating that administrator privileges are required to perform the action, along with a list of selectable authorized user usernames 5015. Furthermore, a "Request" button 5016 is displayed to the right of the list display for requesting action from an authorized user. Through such a UI screen, the inquiry user can request action from another user who has the authority to perform the action (in this case, "User 3" is selected). When displaying a list of authorized users who are candidates for selection, authorized users who are not logged into the generation AI cloud 120 may be grayed out or excluded from the list. Furthermore, the environment of the selected authorized user may be verified as secure, and the "Request" button may be displayed only if it is confirmed to be secure. Once the inquiring user has provided instructions for a request for assistance, the generation AI client application 241 sends the request to the generation AI cloud 120.

[0038] In S416, the Plugin application 224 of the generation AI cloud 120 receives a request for assistance from the inquiry user's client PC 140. In the following S417, the Plugin application 224 sends information to the client PC 140 of the authorized user specified in the received request for assistance that another user has requested assistance with a solution. Furthermore, the Plugin application 224 sends information to the inquiry user's client PC 140 that it has requested assistance with a solution.

[0039] In S418, the AI ​​client application 241 on the authorized user's client PC 140 displays a message on the UI screen indicating that another user has requested assistance in resolving the issue, based on information received from the AI ​​cloud 120. The AI ​​client application then receives instructions from the authorized user to execute the resolving action. Figure 6 shows an example of the UI screen used by the AI ​​client application 241 to solicit the authorized user for assistance in resolving the issue, with the username "User3" of the logged-in user displayed in the username area 600. At this step, the prompt area 601 of the UI screen in Figure 6 displays a message 6011 indicating the content of the assistance request. If the authorized user agrees to assist in resolving the issue, they will enter text information 6012 as a prompt to that effect. Upon receiving this input, the AI ​​client application 241 sends instructions to execute the resolving action to the AI ​​cloud 120. Furthermore, an authorized user who receives a request for assistance may ask questions to confirm the situation before agreeing to perform the solution (for example, "Will changing the settings affect other users?") by entering text information in the prompt area 601. If a response message such as "It will affect other users" is displayed in response to this question, the authorized user may refuse to perform the solution. If the authorized user refuses to perform the solution, the generating AI client application 241 will send a response to the generating AI cloud 120 indicating that the request for assistance has been refused.

[0040] Furthermore, in S419, the generation AI client application 241 on the inquiry user's client PC 140 displays a message on the UI screen indicating that it has requested an authorized user to take over the handling of the issue based on the information received from the generation AI cloud 120. In the example shown in Figure 5 above, at this step, a message 5017 indicating that a request has been made to the authorized user to take over the handling of the issue will be displayed in the prompt area 501.

[0041] In S420, the Plugin application 224 of the generation AI cloud 120 sends a request to the diagnostic cloud 100 to execute a corrective action. Here, if the execution request is made via S413, the credentials of the querying user are used, and if the execution request is made via S418, the credentials of the authorized user are used. This allows the diagnostic cloud 100, upon receiving the execution request, to verify (user authentication) that the user who requested the execution of the corrective action has the authority to execute that corrective action by referring to the user table and authorization table. Then, in S421, the device diagnostic application 202 of the diagnostic cloud 100 executes the corrective action related to the request (such as changing the device settings or restarting) via the action module 206 based on the received execution request. In the specific example above, based on the instructions from the diagnostic cloud 120, the setting change to "Communication setting 1 OFF" is performed on the MFP 160. In the following S422, the device diagnostic application 202 responds to the generation AI cloud 120 with the result of executing the corrective action.

[0042] In S423, the Plugin application 224 of the generation AI cloud 120 receives the result of the countermeasure execution and forwards it to the generation AI application 222. Then, in S424, the generation AI application 222 responds to the client PCs 140 of the authorized user and the querying user with the result of the countermeasure execution received from the Plugin application 224.

[0043] In S425, the AI ​​client application 241 on the authorized user's client PC 140 displays the execution result of the corrective action received from the AI ​​cloud 120 on the UI screen. In the example of Figure 6 above, at this step, the prompt area 601 displays the execution result of the corrective action and a message 6013 informing the inquiring user of that fact. Meanwhile, in S426, the AI ​​client application 241 on the inquiring user's client PC 140 displays the execution result of the corrective action received from the AI ​​cloud 120 on the UI screen. In the example of Figure 5 above, at this step, the prompt area 501 displays the execution result of the corrective action and a message 5018 prompting the user to re-execute the job. The above describes the series of operations of the device diagnostic system according to this embodiment.

[0044] <Variation> In the above embodiment, it is assumed that a countermeasure is implemented as a permanent solution to a problem that occurred in the MFP160. However, if the countermeasure requested by proxy is, for example, a change in equipment settings, it may be implemented as a temporary measure (for example, automatically reverting to the previous settings after a certain period of time) considering the impact on other users, etc. It is also conceivable that the authorized user may not take any action even after a certain period of time has passed since the request for a countermeasure was made. In such cases, subsequent processing may be interrupted, or, for example, the Plugin application 224 may request the execution of a setting change without instructions from the authorized user and then, if necessary, have the authorized user roll back. Furthermore, in this embodiment, when a request for a proxy is made by the inquiry user, the request is made to the authorized user via the UI screen displayed by the generation AI client application, but this is not limited to this. For example, if the authorized user is not logged into the generation AI cloud 120, the request may be made to the authorized user via the chat function of a web conferencing application such as Teams or Zoom, or via email. Moreover, after obtaining consent from the authorized user to execute the countermeasure, the inquiry user may issue instructions to execute the countermeasure on behalf of the authorized user. In the execution request (S420) for the countermeasure based on the execution instruction at this time, the credentials of the authorized user will be used. Furthermore, in this embodiment, device troubleshooting support is provided by the generation AI application through functional extension by plug-in, but the generation AI cloud itself may have a device diagnostic function built in. Also, in this embodiment, it is assumed that the inquiry user and the authorized user access the same generation AI cloud, but they may each access a different generation AI cloud and cooperate with each other between the generation AI clouds.

[0045] As described above, according to this embodiment, when using generated AI to assist in troubleshooting devices such as MFPs, multiple users can seamlessly cooperate and smoothly implement the necessary measures to resolve the problem.

[0046] [Embodiment 2] In Embodiment 1, the generating AI cloud received inquiries from users directly regarding equipment problems, and the diagnostic cloud initiated equipment diagnosis. Next, Embodiment 2 describes an embodiment in which the diagnostic cloud automatically detects problems occurring in pre-registered monitored equipment, automatically initiates diagnosis, and notifies users presumed to be involved in the problem about the occurrence of the problem and how to deal with it via the generating AI cloud. Note that the system configuration / hardware configuration of the equipment diagnostic system, the various tables managed by the database module 208 of the diagnostic cloud 100, and applicable modifications are the same as in Embodiment 1, so their explanation will be omitted. Below, we will focus on explaining the differences from Embodiment 1, namely the software configuration and operation sequence.

[0047] <Software Configuration> Figure 7 is a block diagram illustrating the software configuration of the equipment diagnostic system according to this embodiment. The difference from the software configuration of the equipment diagnostic system according to Embodiment 1 is that the MFP160 has a generation AI client application 700. The generation AI client application 700 sends an HTTP request message to the generation AI cloud 120 via the network module 260 and receives and displays an HTTP response message from the generation AI cloud 120. Access from the MFP160 to the generation AI cloud 120 is performed through the generation AI client application 700.

[0048] <Operation sequence of the equipment diagnostic system> Next, the operation of the device diagnostic system according to this embodiment will be explained using the sequence diagram in Figure 8. In the following explanation, we will use as an example a scenario in which a user instructs an MFP160, which has been pre-registered as a device to be diagnosed in the device diagnostic cloud, to print a document from their client PC, but printing fails due to a paper jam. In the following explanation, the symbol "S" represents a step.

[0049] In S801, the MFP160 transmits device information and log information to the device diagnostic application 202 of the diagnostic cloud 100. This transmission is performed in response to periodic acquisition requests from the device diagnostic cloud 100, such as through polling, or via push communication caused by a job error in the MFP160. The received device information and log information are stored in the device information table and log table of the database module 208 via the device management module 204.

[0050] In S802, the device diagnostic application 202 of the diagnostic cloud 100 refers to the device information table and / or log table via the device management module 204 to detect whether a problem has occurred in the device being diagnosed. For example, when the device information table contains the contents shown in Figure 3(a) above, a "Paper Jam" error, meaning a paper jam error, will be automatically detected in MFP160 with device ID "5". When a problem is detected in this way, the device diagnostic application 202 refers to the log table to identify the user who submitted the job that caused the detected problem (hereinafter referred to as the "causing user"). Here, it is assumed that the user named "User1" who submitted a print job at the time the paper jam occurred in MFP160 with device ID "5" is identified as the causing user.

[0051] Then, in S803, the device diagnostic application 202 performs a diagnosis of the detected trouble. Specifically, it refers to the diagnostic logic table of the database module 208 via the device management module 204 to obtain a solution for the detected trouble. In the example of the diagnostic logic table in Figure 3(c) above, "Open paper feeds and remove any stuck pieces" is obtained as the solution for the status "Paper Jam". Once a diagnostic result corresponding to the trouble is obtained, in S804 the device diagnostic application 202 sends the details of the detected trouble, the solution, and information such as the username of the user responsible to the AI ​​cloud 120 as a diagnostic result.

[0052] In S805, the Plugin application 224 of the generation AI cloud 120 receives the diagnostic results and forwards them to the generation AI application 222. Then, in S806, the generation AI application 222 sends the diagnostic results received from Plugin application 224 to the client PC 140 of the user causing the problem. It is assumed that the user causing the problem is logged into the generation AI cloud 120 from their client PC 140.

[0053] In S807, the generating AI client application 241 on the client PC 140 presents the diagnostic results received from the generating AI cloud 120 to the user causing the problem. Specifically, a UI screen containing a message describing the nature of the problem and how to deal with it is displayed on the display of the user causing the problem's client PC 140. Figure 9 is an example of a UI screen for providing troubleshooting support by the generating AI client application 241, where the username of the user causing the problem, "User1," is displayed in the username area 900. At this step, the prompt area 901 of the UI screen in Figure 9 displays a message 9011 asking about the detected problem, how to deal with it, and whether the user can handle it themselves. If the user causing the problem is near the MFP 160 of "Device 5" and can easily handle it themselves, they will enter text information to that effect (for example, "Yes, I am near the MFP."). On the other hand, if the user causing the problem is not near the MFP 160 of "Device 5" and it is difficult for them to handle it themselves, they will enter text information 9012 to that effect. The generation AI client application 241, upon receiving this user input, sends a response to the generation AI cloud 120 indicating whether or not it can be addressed.

[0054] In S808, the generation AI application 222 of the generation AI cloud 120 receives a response from the inquiry user's client PC 140 indicating whether the issue can be resolved and forwards it to the Plugin application 224. Then, in S809, the Plugin application 224, based on the received response indicating whether the issue can be resolved, executes S810 if the user causing the issue cannot resolve it themselves. On the other hand, if the user causing the issue can resolve it themselves, the user takes the necessary action (in this case, removing jammed paper). Then, S817, described below, is executed on the MFP 160.

[0055] In S810, the Plugin application 224 responds to the cause user's client PC 140, requesting confirmation of whether they wish to ask another user to handle the issue on their behalf. In S811, the generation AI client application 241 on the cause user's client PC 140 displays a message on the UI screen asking whether they wish to ask another user to handle the issue on their behalf, based on the response from the generation AI cloud 120. It then accepts the cause user's instruction to request assistance. In the example of Figure 9 above, at this step, message 9013 is displayed in the prompt area 901. If the cause user wishes to ask another user to handle the issue on their behalf, they will enter text information 9014 to that effect. Through this UI screen, the cause user can request assistance from another user. Once the generation AI client application 241 receives the instruction to request assistance from the cause user, it sends the request to the generation AI cloud 120.

[0056] In S812, the Plugin application 224 of the generation AI cloud 120 receives a request for assistance from the client PC 140 of the user causing the problem. In the subsequent S813, the Plugin application 224 sends a request to the MFP 160 to execute a process to request an unspecified other user to take over the task of resolving the issue. Furthermore, the Plugin application 224 sends information to the client PC 140 of the user causing the problem that the process of requesting an unspecified other user to take over the task of resolving the issue has been completed.

[0057] In S814, the generation AI client application 700 of the MFP160 displays a UI screen on the operation unit 166 that includes a message indicating that a request for troubleshooting has been received from a certain user, based on information received from the generation AI cloud 120. Figure 10 is an example of a UI screen used by the generation AI client application 700 to solicit troubleshooting from an unspecified user. In the UI screen of Figure 10, the message area 1001 displays a message indicating the content of the request for assistance. In S815, the generation AI client application 241 of the client PC 140 of the user causing the problem displays a message on the UI screen indicating that it has requested other users to take over the troubleshooting steps, based on information received from the generation AI cloud 120. In the example of Figure 9 mentioned above, at this step, the prompt area 901 displays a message 9015 indicating that a request for assistance has been made to an unspecified user. Then, when another user sees such a UI screen displayed on the operation unit 166 of the MFP160 and takes action to resolve the problem, the MFP160 sends device information and log information to the device diagnostic application 202 of the diagnostic cloud 100 in S816. Furthermore, when the user responsible for the problem takes the necessary action (such as removing jammed paper), the MFP160 sends device information and log information to the device diagnostic application 202 of the diagnostic cloud 100 in S817. The transmissions in S816 and S817 are also performed in response to periodic acquisition requests from the device diagnostic cloud 100 via polling, etc., or by push communication triggered by the detection of problem resolution in the MFP160. The diagnostic cloud 100 then processes the received device information and log information to reflect it in the device information table and log table of the database module 208 via the device management module 204.

[0058] In S818, the device diagnostic application 202 of the diagnostic cloud 100 refers to the device information table and / or log table via the device management module 204 and performs a detection process to determine whether the problem has been resolved in the diagnostic target device where the problem occurred. If the resolution of the problem is detected, in S819 the device diagnostic application 202 sends a notification of the problem resolution to the generation AI cloud 120. If the necessary countermeasures to resolve the problem are not taken after a certain period of time, information to that effect will be notified as a result of a request for assistance.

[0059] In S820, the Plugin application 224 of the generation AI cloud 120 receives a notification that the problem has been resolved and forwards it to the generation AI application 222. Then, in S821, the generation AI application 222 sends information that the problem has been resolved to the client PC 140 of the user who caused the problem.

[0060] In S822, the generating AI client application 241 on the client PC 140 of the user causing the problem displays a message on the UI screen indicating that the problem has been resolved, based on the information received from the generating AI cloud 120. In the example shown in Figure 9 above, at this step, a message 9016 indicating that the problem has been resolved will be displayed in the prompt area 901.

[0061] The above describes the series of operations of the equipment diagnostic system according to this embodiment. In this embodiment, the troubleshooting method is requested from an unspecified user operating the MFP160 that is experiencing a problem, but this is not limited to this. For example, the Plugin application 224 of the generation AI cloud 120 may obtain information about the user executing a job on the MFP160 that is experiencing a problem and directly request that user to take on the troubleshooting method.

[0062] In this embodiment, as in Embodiment 1, when using the generated AI to assist in troubleshooting equipment, multiple users can seamlessly cooperate and smoothly implement the necessary actions to resolve the problem.

[0063] (Other examples) This disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0064] Furthermore, this disclosure includes the following configurations and methods.

[0065] [Configuration 1] A generative AI application that assists in troubleshooting equipment, The first acquisition step is to acquire methods for dealing with problems that occur with the aforementioned equipment, If the first user involved in the trouble who has been presented with the aforementioned solution is unable to implement the solution himself, the first user provides instructions to request that the solution be implemented on his behalf, and the first reception step is to receive instructions from the first user to implement the solution on his behalf. A request step in which a second user, different from the first user, is asked to perform the aforementioned countermeasure based on the instructions received, A generative AI application characterized by having the following features.

[0066] [Configuration 2] The system further includes a second reception step for receiving inquiries about the aforementioned trouble from the first user, The acquisition step acquires the countermeasure based on the inquiry received in the second reception step. The generation AI application according to configuration 1, characterized by the above.

[0067] [Configuration 3] The generation AI application according to configuration 1 or 2, characterized in that the second user is a user who has the authority to execute the countermeasure when the first user does not have the authority to execute the countermeasure.

[0068] [Structure 4] The system further includes a search step to find a user who has the aforementioned execution privileges, The second user is a user designated by the first user from among the users who have execution privileges that were found. The generation AI application according to configuration 3, characterized by the above.

[0069] [Composition 5] The generation AI application according to configuration 4, characterized in that the second user is a user designated by the first user from among the users who have the execution rights that have been searched and are in a state where they can perform the countermeasure on behalf of the first user.

[0070] [Composition 6] The generation AI application according to configuration 4, characterized in that the second user is a user designated by the first user from among the users who have execution privileges that have been searched, and who has been confirmed to be trustworthy for security reasons.

[0071] [Composition 7] A second acquisition step involves acquiring device information, including the status of the aforementioned device. A detection step that detects a problem occurring in the equipment based on the acquired equipment information, It further possesses, The first acquisition step involves acquiring the countermeasure based on the detection of a problem in the detection step. The generation AI application according to configuration 1, characterized by the above.

[0072] [Structure 8] The generation AI application according to configuration 7, characterized in that the second user is a user who can easily perform the countermeasure when it is difficult for the first user to perform the countermeasure.

[0073] [Composition 9] The generation AI application according to configuration 7, characterized in that the second user is a user who is operating the device or is near the device.

[0074] [Configuration 10] The process further includes the step of identifying the second user, In the aforementioned request step, the identified second user is directly requested to perform the aforementioned countermeasure on behalf of the user. The generation AI application according to configuration 9, characterized by the following:

[0075] [Composition 11] The generation AI application according to any one of configurations 1 to 10, characterized in that, in the request step, the second user is informed that the request is being made on behalf of the first user.

[0076] [Composition 12] The generation AI application according to any one of configurations 1 to 10, further comprising a response step of responding to the first user with the result of executing the countermeasure based on the response of the second user who received the proxy request.

[0077] [Composition 13] The generation AI application according to any one of configurations 1 to 12, characterized in that the countermeasure is one of changing the settings of the device, operating it, or restarting it.

[0078] [Composition 14] The generation AI application according to configuration 13, characterized in that, if the countermeasure is a change in the settings of the device, the settings are automatically reverted to the settings before the change after a certain period of time has elapsed.

[0079] [Composition 15] If the countermeasure is to change the settings of the equipment, and the second user does not take action even after a certain period of time has elapsed since the request for the countermeasure to be performed was made, Control the system so that the aforementioned setting change is executed. After the aforementioned configuration change is performed, the changed settings are rolled back to the second user. A generation AI application according to configuration 13, characterized by the features described above.

[0080] [Composition 16] The generation AI application according to any one of configurations 1 to 15, characterized in that the aforementioned device is a printer.

[0081] [Composition 17] A client application for utilizing troubleshooting support provided by the generating AI application described in any one of items 2 to 6 of the configuration, wherein the first user makes an inquiry about the trouble.

[0082] [Composition 18] A client application installed in the device for utilizing troubleshooting support provided by the generating AI application described in any one of items 7 to 10 of the configuration, the client application providing the device information.

[0083] [Composition 19] An information processing device equipped with the client application described in configuration 17.

[0084] [Configuration 20] The device equipped with the client application described in configuration 18.

Claims

1. A generation AI application that provides troubleshooting support for equipment, The first acquisition step involves obtaining a method for dealing with problems that occur with the aforementioned equipment, If the first user involved in the trouble who has been presented with the countermeasure is unable to implement the countermeasure himself, a first reception step is taken to receive instructions from the first user requesting that the countermeasure be implemented on his behalf. A request step in which a second user, different from the first user, is asked to perform the aforementioned countermeasure based on the instructions received, A generative AI application characterized by having [a certain feature].

2. The system further includes a second reception step for receiving inquiries about the aforementioned trouble from the first user, The acquisition step involves acquiring the countermeasure based on the inquiry received in the second reception step. The generation AI application according to feature 1.

3. The generation AI application according to claim 1, characterized in that the second user is a user who has the authority to execute the countermeasure when the first user does not have the authority to execute the countermeasure.

4. The system further includes a search step to find a user who has the aforementioned execution privileges, The second user is a user designated by the first user from among the users who have execution privileges that were found. The generation AI application according to feature 3.

5. The generation AI application according to claim 4, characterized in that the second user is a user designated by the first user from among the users who have execution privileges that have been searched and are in a state where they can perform the countermeasure on behalf of the first user.

6. The generation AI application according to claim 4, characterized in that the second user is a user designated by the first user from among the users who have execution privileges that have been searched, and who has been confirmed to be trustworthy for security reasons.

7. A second acquisition step involves acquiring device information, including the status of the aforementioned device. A detection step that detects a problem occurring in the equipment based on the acquired equipment information, It further possesses, The first acquisition step is to acquire the countermeasure based on the detection of a problem in the detection step. The generation AI application according to feature 1.

8. The generation AI application according to claim 7, characterized in that the second user is a user who can easily perform the countermeasure when it is difficult for the first user to perform the countermeasure.

9. The generation AI application according to claim 7, characterized in that the second user is a user who is operating the device or is near the device.

10. The process further includes the step of identifying the second user, In the aforementioned request step, the identified second user is directly requested to perform the aforementioned countermeasure on behalf of the user. The generation AI application according to feature 9.

11. The generation AI application according to claim 1, characterized in that, in the request step, the second user is informed that the request is being made on behalf of the first user.

12. The generation AI application according to claim 1, further comprising a response step of responding to the first user with the result of executing the countermeasure based on the response of the second user who received the proxy request.

13. The generation AI application according to claim 1, characterized in that the countermeasure is one of the following: changing the settings of the device, operating it, or restarting it.

14. The generation AI application according to claim 13, characterized in that, if the countermeasure is a change in the settings of the device, the settings are automatically reverted to the settings before the change after a certain period of time has elapsed.

15. If the countermeasure is to change the settings of the equipment, and the second user does not take action even after a certain period of time has elapsed since the request for the countermeasure to be performed was made, Control the system so that the aforementioned setting change is executed. After the aforementioned setting change is performed, the changed setting is rolled back to the second user. The generation AI application according to feature 13.

16. The generation AI application according to claim 1, characterized in that the aforementioned device is a printer.

17. A client application for utilizing troubleshooting support provided by a generating AI application according to any one of claims 2 to 6, wherein the first user makes an inquiry about the trouble.

18. A client application installed in the device for utilizing troubleshooting support provided by the generating AI application described in any one of claims 7 to 10, the client application providing the device information.

19. An information processing device equipped with the client application described in claim 17.

20. The device equipped with the client application described in claim 18.

Images