Information processing system and information processing method

Abstract

An information processing system includes: acquisition means to acquire device information, the device information including identification information of a device used by a user, an application executed on the device, and information on a screen displayed on the device; inference means to infer a problem that the user has with usage of the device, based on the acquired device information; and notification means to provide a notification to the user based on an inference result.

Description

[DESCRIPTION][Title of Invention]INFORMATION PROCESSING SYSTEM AND INFORMATION PROCESSINGMETHOD[Technical Field]

[0001] The present disclosure relates to an information processing system and an information processing method.[Background Art]

[0002] A user experiencing trouble with a device during use may inquire about a solution to the trouble. The inquiry about the solution is mainly answered by staff such as call center staff (human support) or answered automatically by, for example, a computer. Computer-based support is desirable in terms of reduced staff costs.

[0003] Patent Literature (PTL) 1 discloses an automatic response system including a response inference unit configured to provide a notification about at least some of Q&A groups inferred in a ranking form by a trained model. The automatic response system disclosed in PTL 1 can provide a notification about a plurality of types of responses to a single inquiry.[Citation List][Patent Literature]

[0004] [PTL 1]Japanese Unexamined Patent Application Publication No. 2021-56580[Summary of Invention][Technical Problem]

[0005] However, the automatic response system disclosed in PTL 1 prompts the user to explicitly enter an inquiry, and places a burden on the user to make the inquiry.[Solution to Problem]

[0006] The present disclosure described herein provides an information processing system including acquisition means, inference means, and notification means. The acquisition means acquires device information. The device information includes identification information of a device used by a user, an application executed on the device, and information on a screen displayed on the device. The inference means infers a problem that the user has with usage of the device, based on the acquired device information. The notification means provides a notification to the user based on an inference result.The present disclosure described herein provides an information processing method including inferring a problem that a user has with usage of a device used by the user, based on deviceinformation, the device information including identification information of the device, an application executed on the device, and information on a screen displayed on the device; and generating information for providing a notification based on an inference result obtained in the inferring.[Advantageous Effects of Invention]

[0007] According to one aspect of the present disclosure, the information processing system and the information processing method described above provide a response to a problem with a device. This saves the user from having to explicitly enter an inquiry.[Brief Description of Drawings]

[0008] A more complete appreciation of embodiments of the present disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings.FIG. l is a schematic diagram illustrating an example hardware configuration of an information processing system.FIG. 2A is a diagram illustrating an example hardware configuration of an image forming apparatus in the information processing system illustrated in FIG. 1.FIG. 2B is a diagram illustrating an example hardware configuration of a server apparatus in the information processing system illustrated in FIG. 1.FIG. 3 is a block diagram of software components included in the information processing system illustrated in FIG. 1.FIG. 4 is a diagram illustrating an example of a learning phase and a use phase.FIG. 5 is a diagram illustrating an example of a response record database.FIG. 6 is a sequence diagram illustrating an example of processing in a use phase of a trained model.FIG. 7 is a diagram illustrating an example of data processing in the use phase of the trained model.FIG. 8A is a diagram illustrating an example of the inference of a problem and the proposal of a solution to the problem.FIG. 8B is a diagram illustrating an example of the inference of a problem and the proposal of a solution to the problem.FIG. 8C is a diagram illustrating an example of the inference of a problem and the proposal of a solution to the problem.FIG. 9 is a flowchart illustrating an example of a process executed by the information processing system illustrated in FIG. 1.FIG. lOA is a diagram illustrating an example of a displayed screen.FIG. 10B is a diagram illustrating an example of a displayed screen.FIG. 10C is a diagram illustrating an example of a displayed screen.FIG. 11 is a flowchart illustrating another example of the process executed by the information processing system illustrated in FIG. 1.The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted. Also, identical or similar reference numerals designate identical or similar components throughout the several views. [Description of Embodiments]

[0009] In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.Referring now to the drawings, embodiments of the present disclosure are described below. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.The present disclosure is described with reference to an embodiment, but the present disclosure is not limited to the embodiment described herein. In the drawings to be referred to herein, common elements are designated by the same reference numerals, and the description thereof will be omitted as appropriate.

[0010] FIG. l is a schematic diagram illustrating the hardware configuration of an information processing system 100 according to the present embodiment. FIG. 1 illustrates, as an example, an environment in which a plurality of image forming apparatuses 110 are connected to a server apparatus 120 via a network 130 such as the Internet or a local area network (LAN). The information processing system 100 may include any number of image forming apparatuses 110, and the number of image forming apparatuses 110 is not limited to that illustrated in FIG. 1. The image forming apparatuses 110 and the server apparatus 120 may connect to the network 130 in a wired or wireless manner.

[0011] The image forming apparatuses 110 are apparatuses that form an image mainly on a sheet of paper, and are also referred to as multi -function peripherals (MFPs). Each of the image forming apparatuses 110 can execute a print job received from, for example, a personal computer (PC) terminal, copy or scan paper media, perform facsimile communication via the network 130, and perform other processing.

[0012] The server apparatus 120 is, for example, an information processing apparatus such as a server computer, and can provide a service according to the present embodiment. In one example, when a user experiences trouble, such as not knowing how to operate the image forming apparatus 110 during use, the server apparatus 120 proposes a solution to address thetrouble. In the embodiment described herein, trouble with the image forming apparatus 110 during use may be referred to as a "problem".

[0013] The following embodiment describes a configuration for providing a solution to a problem with an image forming apparatus 110 as illustrated in FIG. 1, by way of example, but the embodiment is not limited thereto. Accordingly, an embodiment may be configured for developing a solution to address a problem with a device other than the image forming apparatus 110 during use. Examples of the device other than the image forming apparatus 110 include industrial devices such as a medical device and a machine tool. In a preferred embodiment, the device includes a display device (display).

[0014] Next, the hardware configuration of each apparatus will be described. FIGs. 2A and 2B are diagrams illustrating hardware components included in the apparatuses of the information processing system 100. FIG. 2A illustrates the hardware components of each of the image forming apparatuses 110, and FIG. 2B illustrates the hardware components of the server apparatus 120.

[0015] First, the image forming apparatus 110 illustrated in FIG. 2A will be described. The image forming apparatus 110 includes a central processing unit (CPU) 210, a random access memory (RAM) 220, a read-only memory (ROM) 230, a storage device 240, a communication interface (I / F) 250, a display 260, an input device 270, a printer device 280, and a scanner device 290. Such hardware components are connected to each other via a bus.

[0016] The CPU 210 is a device that executes a program for controlling the operation of the image forming apparatus 110 and performs predetermined processing. The RAM 220 is a volatile memory functioning as an area for deploying a program executed by the CPU 210, and is used for storing or expanding programs and data. The ROM 230 is a non-volatile memory for storing, for example, programs and firmware to be executed by the CPU 210.

[0017] The storage device 240 is a readable and writable non-volatile memory that stores, for example, an operating system (OS) or various types of software for causing the image forming apparatus 110 to function, setting information, and various types of data. Examples of the storage device 240 include a hard disk drive (HDD) and a solid state drive (SSD).

[0018] The communication I / F 250 connects the image forming apparatus 110 to the network 130, and enables the image forming apparatus 110 to communicate with another apparatus via the network 130. Communication via the network 130 may be either wired or wireless, and various types of data can be transmitted and received using a predetermined communication protocol such as transmission control protocol / Internet protocol (TCP / IP).

[0019] The display 260 is a device that displays, for example, various types of data and the state of the image forming apparatus 110 to the user. Examples of the display 260 include a liquid crystal display (LCD). The input device 270 is a device used by the user to operate the image forming apparatus 110. Examples of the input device 270 include physical buttons. The display 260 and the input device 270 may be separate devices, or may be integrated into one device such as a touch panel display.

[0020] The printer device 280 is a device configured to form an image on a sheet of paper by using a printing method such as a laser method or an inkjet method. The scanner device 290 is a device configured to read an image on a printed material and convert the read image into data. For example, the image forming apparatus 110 can copy a printed material through cooperation of the scanner device 290 and the printer device 280. Further, the image forming apparatus 110 can implement a facsimile function through cooperation of the scanner device 290 and the communication I / F 250, for example.

[0021] Next, the server apparatus 120 illustrated in FIG. 2B will be described. The server apparatus 120 includes a CPU 210, a RAM 220, a ROM 230, a storage device 240, a communication I / F 250, a display 260, and an input device 270. Such hardware components are connected to each other via a bus. The hardware configuration of the server apparatus 120 is similar to that of the image forming apparatus 110 illustrated in FIG. 2 A, and the detailed description thereof will thus be omitted. In another example, the server apparatus 120 does not include the display 260 or the input device 270.

[0022] The hardware configurations included in the information processing system 100 have been described. Next, functional means executed by the hardware components according to the present embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram of software components included in the information processing system 100.

[0023] As illustrated in FIG. 3, the image forming apparatus 110 includes functional means, namely, a printing unit 311, a reading unit 312, an authentication processing unit 313, a device information acquisition unit 314, a communication unit 315, a display unit 316, an operation unit 317, and a history storage unit 318. The server apparatus 120 includes functional means, namely, a communication unit 321, a problem inference unit 322, a notification information generation unit 323, and a trained model 324. The functional means will be described in detail hereinafter.

[0024] First, the functional means included in the image forming apparatus 110 will be described. The printing unit 311 is a means for controlling the operation of the printer device 280 to form an image on a sheet of paper and perform printing. The printing unit 311 is configured as printing means according to the present embodiment.

[0025] The reading unit 312 is a means for controlling the operation of the scanner device 290 to scan a surface of paper. The reading unit 312 is configured as reading means according to the present embodiment. The reading unit 312 can store data of the scanned surface of paper or transfer the data to various information processing apparatuses.

[0026] The authentication processing unit 313 is a means for authenticating a user who uses the image forming apparatus 110. The authentication processing unit 313 is configured as authentication means according to the present embodiment. The authentication processing unit 313 can receive, for example, an input of an ID and a password from the user and authenticate whether the user is authorized to use the image forming apparatus 110. The authenticated user is allowed to use the image forming apparatus 110. Through the authentication process, a user having a specific attribute may or may not be allowed to use a specific function of the image forming apparatus 110.

[0027] The device information acquisition unit 314 is a means for acquiring various types of information related to the image forming apparatus 110. The device information acquisition unit 314 is configured as acquisition means according to the present embodiment. The device information acquisition unit 314 can acquire, for example, a model name, an application, a display screen, a job status, an error history, and an operation history of the image forming apparatus 110. The model name is an example of identification information of the image forming apparatus 110. The identification information of the image forming apparatus 110 is information used to identify at least the model of the image forming apparatus 110, and may be information other than the model name, such as the model number of the image forming apparatus 110. The model name is the name of the model of the image forming apparatus 110 used by the user at the time of occurrence of the problem. The application is the name of an application such as a facsimile, copying, printing, or scanning application of the image forming apparatus 110 used by the user at the time of occurrence of the problem. The display screen is the name of a screen displayed on the display 260 of the image forming apparatus 110 at the time of occurrence of the problem. The device information acquisition unit 314 can acquire information on a screen displayed on the display 260 through the display unit 316 at the point in time when, for example, a help button is pressed by the user to figure out how to operate the image forming apparatus 110. The job status is the status of the application used by the user at the time of occurrence of the problem. In the embodiment described herein, the device information acquisition unit 314 can acquire at least the model name, the application, and the display screen as the device information, and may also acquire the job status. The various types of information acquired by the device information acquisition unit 314 are input to the problem inference unit 322 and used to infer a problem. The device information acquisition unit 314 can also acquire information related to the user who is using the image forming apparatus 110 with which the problem has occurred.

[0028] The communication unit 315 is a means for controlling the operation of the communication I / F 250 to perform communication between the image forming apparatus 110 and another apparatus via the network 130. The communication unit 315 is configured as communication means according to the present embodiment. For example, when the user is unable to figure out how to operate the image forming apparatus 110, the communication unit 315 can transmit information used to acquire information for supporting the user in operating the image forming apparatus 110 to the server apparatus 120. In one example, the communication unit 315 can transmit screen information acquired by the device information acquisition unit 314 to the server apparatus 120. Further, the communication unit 315 can receive support information from the server apparatus 120.

[0029] The display unit 316 is a means for controlling the operation of the display 260 to display various types of information. The display unit 316 is configured as display means according to the present embodiment. The display unit 316 according to the present embodiment can display, for example, a screen for performing an operation such as a printing process or a reading process, and the support information received from the server apparatus 120. The display unit 316 may be configured as notification means for notifying the user of an inference result obtained by the problem inference unit 322.

[0030] The operation unit 317 is a means for receiving an operation performed by the user through the input device 270. The operation unit 317 is configured as operation means according to the present embodiment. The operation unit 317 can be used to, for example, press the help button to figure out how to operate the image forming apparatus 110, or select the support information received from the server apparatus 120.

[0031] The history storage unit 318 is a means for controlling the operation of the storage device 240 to store history information used to acquire the support information. The history storage unit 318 is configured as storage means according to the present embodiment. The history storage unit 318 can store, for example, a history of operations of the image forming apparatus 110 (first history information) and a history of error information of the image forming apparatus 110 (second history information).

[0032] Next, the functional means included in the server apparatus 120 will be described. The communication unit 321 is a means for controlling the operation of the communication I / F 250 to perform communication between the server apparatus 120 and another apparatus via the network 130. The communication unit 321 is configured as communication means according to the present embodiment. For example, the communication unit 321 can receive a help request from the image forming apparatus 110 and transmit the support information to the image forming apparatus 110.

[0033] The problem inference unit 322 is a means for inputting various types of information acquired by the device information acquisition unit 314 to the trained model 324 to infer a problem. The problem inference unit 322 is configured as inference means according to the present embodiment. The problem inference unit 322 can infer a problem, based on device-related information. The device-related information includes information on a display screen at the point in time when the problem occurs, the model of the image forming apparatus 110, and an application running on the image forming apparatus 110.

[0034] The problem inference unit 322 inputs information related to the display screen, the model, and the application to the trained model 324 and infers a problem. The problem inference unit 322 inputs information on a display screen at the time of occurrence of the problem (i.e., a screen displayed on the device in a situation where the user is having trouble), the model of the image forming apparatus 110, and the application running on the image forming apparatus 110 to the trained model 324, and infers a problem that the user is experiencing (i.e., the content to be inquired about) through the trained model 324. Accordingly, the problem inference unit 322 and the trained model 324 can cooperate with each other to form a neural network. The problem inference unit 322 can acquire various types of information from the image forming apparatus 110 via, for example, a web application programming interface (API).

[0035] The problem inference unit 322 can use the information on the model, the application, and the display screen acquired by the device information acquisition unit 314 as an input to the neural network. The neural network can weight the information in each layer and output a match rate indicating how similar the information is to the data of previous inquiries. Accordingly, the problem inference unit 322 can infer a problem by determining an inquiry having a high match rate. The problem inference unit 322 can also infer one or more problems having high match rates as problems that the user is experiencing. In one example, the problem inference unit 322 can quantitatively infer a problem by outputting a match rate. In another example, the problem inference unit 322 can appropriately infer a situation where the user is having trouble by inputting information related to the display screen to the trained model 324 and inferring a problem. Accordingly, the problem inference unit 322 can present an appropriate solution to the problem of the user.

[0036] The problem inference unit 322 may infer a problem in consideration of information such as a job status in addition to the information on the display screen, the model name, and the application. The problem inference unit 322 may be configured to perform weighting based on information unique to the user, such as the attribute of the user logging in, and infer a problem.

[0037] The notification information generation unit 323 is a means for generating information for notifying the user of an inference result. The notification information generation unit 323 is configured as notification information generation means according to the present embodiment. The notification information generation unit 323 can generate, for example, information on a display screen indicating the problem inferred by the problem inference unit 322 or a solution to the problem. The notification information generated by the notification information generation unit 323 is transmitted to the image forming apparatus 110 via the communication unit 321. In another embodiment, a notification means other than the display unit 316 is used. For example, a speaker is used to notify the user of an inference result. In this case, the notification information generation unit 323 can generate a voice via which the inference result is provided.

[0038] The trained model 324 is a program for inferring a problem in cooperation with the problem inference unit 322, and is stored in the storage device 240. The trained model 324 has learned various types of information on previous inquiries about the image forming apparatuses 110 (see FIG. 1). The previous inquiries may be, for example, inquiries made by the users of the image forming apparatuses 110 to a contact desk (hereinafter also referred to as a support desk) such as a call center. Accordingly, the trained model 324 has learned from a database of solutions that can address problems. Such solutions are developed by human support staff responding to the previous inquiries. The trained model 324 will be described in detail below.

[0039] The software blocks illustrated in FIG. 3 correspond to functional means implemented by causing the hardware components to function in response to the CPU 210 executing the program according to the present embodiment. The functional means presented in the embodiments described herein may entirely be implemented as software, or may entirely or partially be implemented as hardware that provides equivalent functions.

[0040] Further, not all of the functional means described above may be included in the manner as illustrated in FIG. 3. For example, in another preferred embodiment, each functional means may be implemented through cooperation of the image forming apparatus 110 and the server apparatus 120.

[0041] Next, the trained model 324 will be described. In the present embodiment, the following two separate phases are provided: a learning phase in which the trained model 324 is generated, and a use phase in which device information is input to the trained model 324 to infer a problem. FIG. 4 is a diagram illustrating an example of the learning phase and the use phase according to the present embodiment. In FIG. 4, a region surrounded by a broken line indicates the learning phase, and a region surrounded by a dash-dot line indicates the use phase.

[0042] In the learning phase, first, records of inquiries about the image forming apparatuses 110 previously made to the support desk are collected as a database. The database may be a response record database that stores responses (solutions) proposed by staff of the support desk to inquiries made to the support desk. The response record database may be, for example, a database of information such as models with which the users have experienced problems due to unclear usage and the like, applications, screens displayed at the time of occurrence of the problems, and job statuses, specific inquiries from the users, and solutions proposed by the staff of the support desk to the inquiries.

[0043] An example of the response record database will now be described with reference to FIG. 5. FIG. 5 is a diagram illustrating an example of the response record database according to the present embodiment.

[0044] As illustrated in FIG. 5, the response record database stores an ID for identifying data of a response record, a model name of the image forming apparatus 110, an application, a display screen, a job status, an inquiry, and a solution to a problem of the inquiry in association with each other. The items illustrated in FIG. 5 are one example and do not limit the embodiment. The response record database may include items other than those illustrated in FIG. 5. Thus, for example, the response record database may include an attribute of a user who has made the inquiry, such as an organization or a department to which the user belongs or a job title of the user.

[0045] In FIG. 5, the item "ID" is a code for identifying data of a response record. An ID is uniquely assigned to one inquiry. The ID can be assigned when, for example, the response record is stored in the database by the support desk.

[0046] In FIG. 5, the item "model name" is the name of a model used by the user at the time of the inquiry recorded in a response report generated by the support desk. That is, the item "model name" refers to the model of the image forming apparatus 110 with which trouble about usage has occurred and about which an inquiry has been previously made. The model name can be acquired from the response report generated by the support desk.

[0047] In FIG. 5, the item "application" is the name of an MFP application used by the user at the time of the inquiry recorded in the response report generated by the support desk. That is, the item "application" refers to an application executed when trouble occurred in the past. Examples of the application executed on the image forming apparatus 110 include copying, scanning, facsimile, and printing applications, but the embodiment is not limited thereto. The application can be acquired from the response report generated by the support desk.

[0048] In FIG. 5, the item "display screen" is the name of a screen displayed on the display 260 of the image forming apparatus 110 at the time of the inquiry recorded in the response report generated by the support desk. That is, the item "display screen" refers to a screen displayed on the display 260 of the image forming apparatus 110 when trouble occurred in the past. Examples of the display screen displayed on the display 260 of the image forming apparatus 110 include a home screen and a paper selection screen, but the embodiment is not limited thereto. The display screen displayed on the display 260 of the image forming apparatus 110 at the time of occurrence of trouble indicates the state of the image forming apparatus 110 at the point in time when the trouble occurs. Thus, the display screen contributes to understanding of a problem and proposal of a solution to the problem. The display screen can be acquired from the response report generated by the support desk.

[0049] In FIG. 5, the item "job status" is the status of an application used by the user at the time of the inquiry recorded in the response report generated by the support desk. That is, the item "job status" refers to the job status of an application running when trouble occurred in the past. Examples of the job status of an application on the image forming apparatus 110 include "aborted", which indicates cancellation of the job by the application, "canceled", which indicates cancellation of the job through a user operation, "completed", which indicates completion of the job, "pending", which indicates that the job is not started yet, "processing", which indicates that the job is being executed, and "processing stopped", which indicates that the job involving a user operation is interrupted, but the embodiment is not limited thereto. The job status can be acquired from, for example, the response report generated by the support desk or a log of the image forming apparatus 110.

[0050] In FIG. 5, the item "inquiry" is the content of an inquiry made by the user and recorded in the response report generated by the support desk. That is, the item "inquiry" refers to a problem previously experienced by the user who uses the image forming apparatus 110. Examples of the inquiry about the image forming apparatus 110 include "being unable to figure out how to set a destination", but the embodiment is not limited thereto. The inquiry can be acquired from the response report generated by the support desk.

[0051] In FIG. 5, the item "solution" is a solution presented in response to a user inquiry recorded in the response report generated by the support desk. That is, the item "solution" refers to a solution to a problem, and the solution is provided by the staff of the support desk in response to an inquiry from the user.Examples of the solution to a problem with the image forming apparatus 110 include "open the address book and set a destination", but the embodiment is not limited thereto. The solution can be acquired from the response report generated by the support desk.

[0052] The table illustrated in FIG. 5 can be created based on inquiries made to a support desk with human agents, such as a call center. For example, a user who does not know how to operate the image forming apparatus 110 makes an inquiry to a call center to solve the problem. At this time, the user transmits information (user information) to staff of the call center. Specifically, the user transmits, to the staff of the call center, the model name of the image forming apparatus 110 being used, the application running on the image forming apparatus 110, the display screen at the time of occurrence of trouble, the state of the image forming apparatus 110, such as a job status, and attributes of the user, such as the organization and department to which the user belongs and the job title of the user. In response to the inquiry from the user, the staff of the call center can acquire, for example, an operation history and an error history of the image forming apparatus 110 via a web API or the like. The staff understands the problem (the content inquired about) on the basis of the transmitted information on a situation in which the problem occurred and the acquired history information, and sends a response with a solution to the user.

[0053] After the staff sends the response, various types of information such as the model name, the application, the display screen, the job status, the inquiry, and the solution can be associated with each other. The staff stores such information in the response record database.

[0054] The description returns to FIG. 4. In response to information being stored in the response record database held by the support desk, the trained model 324 is generated by using a known machine learning means. The trained model 324 is trained such that when various types of information on a device, such as identification information of a model (e.g., a model name), an application, and a display screen, are input to the trained model 324, the trained model 324 outputs the match rate of an inquiry related with the various types of information to a previous inquiry about the device. The machine learning means can generate the trained model 324 by using the response record database as training data. Even after the trained model 324 is generated, the machine learning means can further perform machine learning and update the trained model 324 in response to an inquiry made to the support desk and an update of the response record database.

[0055] Machine learning is a technology that enables a computer to acquire a human-like learning ability, and refers to a technique in which a computer autonomously generates an algorithm used for determination such as data identification from learning data captured in advance and applies the algorithm to new data to make a prediction. A learning method to be applied for machine learning may be any one of supervised learning, unsupervised learning, semisupervised learning, reinforcement learning, and deep learning, or a combination of two or more of those learning methods. Any learning method to be applied for machine learning may be used.

[0056] Next, the use phase of the trained model 324 will be described. As illustrated in FIG. 4, in the use phase, the device information acquisition unit 314 acquires information such as a screen displayed on the display unit 316, a model name, and an application, and transmits the acquired information to the server apparatus 120 via the communication unit 315. The problem inference unit 322 inputs screen information and other device information to the trained model 324 and outputs a match rate to infer a problem. The inference result is transmitted to the image forming apparatus 110 and displayed on the display unit 316 to notify the user of the inference result.

[0057] Processing in the use phase will be described with reference to FIG. 6. FIG. 6 is a sequence diagram illustrating the processing in the use phase of the trained model 324 according to the present embodiment.

[0058] For example, when a user has trouble such as not knowing how to operate the image forming apparatus 110 during use (SI 001), in SI 002, the user presses the help button of the image forming apparatus 110. In response to pressing of the help button, in S1003, the device information acquisition unit 314 acquires information on the image forming apparatus 110, such as a model name and an application. In response to the pressing of the help button, furthermore, in SI 004, the device information acquisition unit 314 requests the display unit 316 to transmit information on the screen being displayed. In response to the request submitted in S1004, in S1005, the display unit 316 transmits information on the display screen to the device information acquisition unit 314.

[0059] In S1006, the device information acquisition unit 314, which has acquired the information on the display screen, requests the communication unit 315 to transmit the device information, such as the display screen, the model name, and the application, to the server apparatus 120. In SI 007, the communication unit 315 transmits the device information to the server apparatus 120. The data transmitted from the communication unit 315 in S1007 may include the information on the display screen and other information such as the model name of the image forming apparatus 110, the application running on the image forming apparatus 110, and the job status.

[0060] In S1008, the problem inference unit 322 of the server apparatus 120 performs a process for inferring a problem by using the information transmitted from the image forming apparatus 110 as an input to the trained model 324. Then, in SI 009, the problem inference unit 322 transmits the inference result to the notification information generation unit 323.

[0061] In S1010, the notification information generation unit 323 generates information on the display screen, based on the inference result. The notification information generation unit 323 can generate, for example, information on a screen for displaying a problem having a highmatch rate. The generated display screen may include a plurality of problems. Then, in S 1011, the notification information generation unit 323 transmits the generated screen information to the communication unit 315 of the image forming apparatus 110. In S 1012, the communication unit 315 transmits the received display screen information to the display unit 316.

[0062] In S1013, the display unit 316 displays a screen to the user. By viewing the screen displayed on the display 260 in S 1013, the user can grasp the problem that has occurred with the image forming apparatus 110 in S 1001. In S 1014, the user selects a problem to which the user desires to know a solution from among the problems displayed on the screen. In response to the selection by the user, in S 1015, the display unit 316 displays a solution to the problem selected by the user on the display 260.

[0063] The data processing in the use phase as illustrated in FIG. 6 will be described hereinafter with reference to FIG. 7. FIG. 7 is a diagram illustrating the data processing in the use phase of the trained model 324.

[0064] As illustrated in FIG. 7, the various types of device information of the image forming apparatus 110 acquired by the device information acquisition unit 314, such as the model name, the application, the display screen, and the job status, are used as inputs to the trained model 324. In the present embodiment, at least part of device information, namely, the model name (identification information), the application, and the display screen, is input to the trained model 324, and the job status may also be input to the trained model 324.

[0065] When the device information is input, the neural network including the problem inference unit 322 and the trained model 324 outputs a match rate indicating the degree to which the input device information matches the conditions included in a previous inquiry. In an example illustrated in FIG. 7(a), the following inference results are output: a match rate of 50% to the inquiry with ID 001, and a match rate of 100% to the inquiry with ID 002, for example.

[0066] The output results are sorted by the notification information generation unit 323 in descending order of match rates, and are arranged in order of display priority, as illustrated in FIG. 7(b). The notification information generation unit 323 generates information for displaying inquiries having high match rates, i.e., high display priorities, and transmits the information to the image forming apparatus 110 to present the inquiries to the user.

[0067] The tables illustrated in FIGs. 7(a) and 7(b) are for convenience of description of the embodiment, and it should be noted that the problem inference unit 322 and the notification information generation unit 323 do not necessarily output these tables.

[0068] The inference of a problem and the proposal of a solution to the problem by using the problem inference unit 322 according to the present embodiment will be described with reference to FIGs. 8Ato 8C. FIGs. 8Ato 8C are diagrams illustrating an example of the inference of a problem and the proposal of a solution to the problem according to the present embodiment.

[0069] FIG. 8A illustrates an example in which the problem inference unit 322 outputs match rates based on the input device information, and corresponds to the table illustrated in FIG. 7(a). The table illustrated in FIG. 8A is for convenience of description of the embodiment, and it should be noted that the problem inference unit 322 does not necessarily output the table as illustrated in FIG. 8A.

[0070] As illustrated in FIG. 8A, as a result of the inference performed by the problem inference unit 322, rates (match rates) at which the problem (current problem) with the image forming apparatus 110 related with the acquired device information matches the previous inquiries are output. Thus, it can be determined that an inquiry having a high match rate is likely to be the current problem that the user is experiencing. In the example illustrated in FIG. 8A, the match rate between the inquiry with ID 001 and the current problem is output as 50%, the match rate between the inquiry with ID 002 and the current problem is output as 100%, the match rate between the inquiry with ID 003 and the current problem is output as 0%, the match rate between the inquiry with ID 004 and the current problem is output as 75%, and the match rate between the inquiry with ID 005 and the current problem is output as 25%. Consideration is now given to a case where the problem inference unit 322 infers the problems with the top three match rates as current problems that the user is experiencing. In this case, the problem inference unit 322 can infer problems having high match rates, namely, the inquiries with ID 002, ID 004, and ID 001, as current problems.

[0071] The notification information generation unit 323 generates a screen for displaying a problem, based on the inference result obtained by the problem inference unit 322. As illustrated in FIG. 8B, the screen for displaying a problem includes inquiries corresponding to IDs for which the match rates are determined to be high. The screen illustrated in FIG. 8B displays the inquiry with ID 002, the inquiry with ID 004, and the inquiry with ID 001 in a selectable manner. As an example, the inquiries are displayed in descending order from the highest to lowest match rates. The user can select, through the screen illustrated in FIG. 8B, a problem that the user is experiencing with the image forming apparatus 110.

[0072] In response to the user's selection on the screen illustrated in FIG. 8B, the notification information generation unit 323 generates a screen for displaying the solution associated with the ID selected by the user. The screen for displaying the solution may have, for example, a configuration as illustrated in FIG. 8C. As a result, the user can know the solution to the current problem and appropriately solve the problem.

[0073] Operations executed by the functional means according to the present embodiment will be described with reference to FIG. 9. FIG. 9 is a flowchart illustrating an example of a process executed by the information processing system 100.

[0074] In step S2001, the process branches depending on whether a help operation has been performed for the image forming apparatus 110. The help operation according to the present embodiment will be described with reference to FIGs. lOAto 10C. FIGs. lOAto 10C are diagrams illustrating an example of screens displayed according to the present embodiment.

[0075] FIG. 10A illustrates an example of a screen displayed while an application is being operated on the image forming apparatus 110. In the example illustrated in FIG. 10A, the image forming apparatus 110 is out of paper in a manual feed tray. Consideration will now be given to a case where the user viewing the screen illustrated in FIG. 10A is unable to figure out how to address the situation. At this time, the user is experiencing a problem that the user "is requested to load the manual feed tray with paper but does not know how to do so". Then, the user presses a "help" button on the screen illustrated in FIG. lOAto address the problem. As described above, in response to the user performing a help operation, the information processing system 100 performs a process for solving the problem of the user.

[0076] The description returns to FIG. 9. If no help operation has been performed in step S2001 (NO), the process returns to step S2001 and waits for a help operation to be performed. On the other hand, if a help operation has been performed (YES), the process proceeds to step S2002.

[0077] In step S2002, the device information acquisition unit 314 of the image forming apparatus 110 acquires device information of the image forming apparatus 110. The device information acquisition unit 314 can acquire device information such as a model name, an application, a display screen, and a job status, for example. In step S2002, the device information acquisition unit 314 may acquire user information of a user who is using the image forming apparatus 110, in addition to the device information. The user information can be identified by, for example, the user performing an authentication process to use the image forming apparatus 110. The device information acquisition unit 314 can also acquire an operation history and an error history of the image forming apparatus 110 stored in the history storage unit 318.

[0078] Then, in step S2003, the problem inference unit 322 infers a problem that the user is experiencing, based on the device information and the like acquired in step S2002. In the inference of a problem, the model name of the image forming apparatus 110 and an application and a display screen on the image forming apparatus 110 are input to the trainedmodel 324, and a match rate is output. As a result, an inquiry having a high match rate is inferred as a problem.

[0079] The problem inference unit 322 may infer a problem also in consideration of the user information, the operation history, and the error history, for example. The user information includes information such as an attribute of the user. Thus, when a trained model including the attribute of the user is generated, the user information can be used to easily infer a problem unique to the organization to which the user belongs, for example. The inference accuracy can be improved. The operation history and the error history are information indicating how the image forming apparatus 110 has been used and what failures have occurred in the image forming apparatus 110. Thus, when the operation history or the error history includes an operation or an error that may cause a problem, the inference is performed in consideration of the history information. Thus, the accuracy of inference of a problem can be improved.

[0080] Then, in step S2004, the notification information generation unit 323 generates information on a screen for displaying an inquiry having a high match rate, and displays the screen on the display 260 of the image forming apparatus 110. The screen displayed in step S2004 may be, for example, a screen as illustrated in FIG. 10B. In the example illustrated in FIG. 10B, three inquiries with high match rates are displayed. The number of inquiries to be displayed is not limited to three as illustrated in FIG. 10B, and any number of inquiries may be displayed. While viewing the screen illustrated in FIG. 10B displayed in step S2004, the user can select a problem that the user is experiencing (i.e., the content to be inquired about). When the user determines that the problems displayed on the screen do not include the problem that the user is experiencing, the user may be allowed to select "display more inquiries". When the user selects "display more inquiries", the notification information generation unit 323 may display other problems having high match rates.

[0081] While the embodiment describes an example in which the inference result is displayed on the display 260, the embodiment is not limited thereto. Any means other than the display unit 316 may be used to provide a notification of the inference result. A notification means other than the display unit 316 provides a notification of the inference result. For example, a notification means such as a speaker may provide a notification of the inference result by voice.

[0082] In the embodiment described herein, since the user is experiencing a problem that the user "is requested to load the manual feed tray with paper but does not know how to do so", the user selects the inquiry "Unable to figure out how to load paper for [manual feeding]" on the screen illustrated in FIG. 10B.

[0083] The description returns to FIG. 9. When the user selects an inquiry among the inquiries displayed in step S2004, then, in step S2005, the notification information generation unit 323generates information on a screen for displaying a solution to the selected inquiry and displays the screen on the display 260 of the image forming apparatus 110. The screen displayed in step S2005 may be, for example, a screen as illustrated in FIG. 10C.

[0084] In the embodiment described herein, since the user selects the inquiry "Unable to figure out how to load paper for [manual feeding]" on the screen illustrated in FIG. 10B, the screen illustrated in FIG. 10C displays a description of "how to load paper for manual feeding". By viewing the screen illustrated in FIG. 10C, the user can understand a solution to the problem.

[0085] The description returns to FIG. 9. After the solution is displayed in step S2005, then, in step S2006, the process branches depending on whether the problem has been solved. For example, the screen illustrated in FIG. 10C includes options to allow the user to select whether the problem has been solved. In a case where the problem has been solved by the solution displayed as illustrated in FIG. 10C, the user presses a "Solved" button. In a case where the problem has not been solved by the displayed solution, the user selects a "Not Solved" button.

[0086] When it is determined in step S2006 that the problem has been solved (YES), the process ends. On the other hand, when it is determined that the problem has not been solved (NO), the process proceeds to step S2007.

[0087] In step S2007, the device information acquisition unit 314 receives an input of additional information related to the problem from the user. In step S2007, the additional information may be input by, for example, the user entering a text of a natural sentence or entering a keyword related to the problem. Examples of the text include a natural sentence such as "Usually authentication is not requested, but this time authentication was requested when opening the application". Examples of the keyword related to the problem include keywords "authentication" and "application setup". The additional information may be input by the user selecting an option displayed on the screen.

[0088] After the input of the additional information is received in step S2007, the process returns to step S2003. In the processing of step S2003 after step S2007, the problem inference unit 322 infers a problem in consideration of the additional information in addition to the device information. The inference of a problem in this way increases the likelihood of inferring a problem that could not be inferred in consideration of the device information alone. As a result, the inference accuracy can be improved. After that, the processing of step S2004 and the subsequent processing are repeated, as described above.

[0089] Another example of the process according to the present embodiment will be described with reference to FIG. 11. FIG. 11 is a flowchart illustrating another example of the processexecuted by the information processing system 100. The processing of steps S3001 to S3006 in FIG. 11 is similar to the processing of steps S2001 to S2006 in FIG. 9, and a detailed description thereof will thus be omitted.

[0090] In step S3006, the process branches depending on whether the problem has been solved. If the problem has been solved (YES), the process ends. If the problem has not been solved (NO), the process proceeds to step S3007.

[0091] In step S3007, the display unit 316 displays guidance to a human support desk. That is, since the problem of the user has not satisfactorily been solved by the solution to the inferred problem, the problem is more likely to be solved promptly by making an inquiry to support staff than by inferring the problem by the problem inference unit 322. The guidance displayed in step S3007 allows the user to send an inquiry to an appropriate contact desk. After step S3007, the process ends.

[0092] Through the processes illustrated in FIGs. 9 and 11, the information processing system 100 can propose a solution to a problem of the user. More specifically, in the embodiment described herein, device information and the like are input to a trained model to infer a problem. Thus, a handling method with a high possibility of solving an inquiry can be proposed without the user explicitly entering the inquiry.

[0093] The operations illustrated in FIGs. 9 and 11 may be implemented in any combination. In one example, if the problem has not satisfactorily been solved by the presented solution (NO in step S2006), an input of additional information is received (step S2007) to infer a problem based on the device information and the additional information (step S2003); however, if the problem still remains unsolved even through these operations, guidance for human support may be provided (step S3007). In another example, when the problem remains unsolved even after an input of additional information is repeated a predetermined number of times, guidance for human support may be provided.

[0094] The embodiment of the present disclosure described above can provide an information processing system, an information processing apparatus, an information processing method, and a program that provide a response to a problem with a device while reducing the burden on the user.

[0095] While the embodiment described above provides an example of inference of a problem using a trained model, the embodiment is not limited to this example. Thus, a problem may be inferred by using a method other than using a trained model. For example, match rates may be calculated by using a preset program in a way like regression analysis, and a problem may be inferred.

[0096] The functions in the embodiment of the present disclosure described above can be implemented by an apparatus-executable program written in C, C++, C#, Java®, or the like, and the program according to the present embodiment can be stored in an apparatus-readable recording medium such as a hard disk device, a compact disc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disc (DVD), a flexible disk, an EEPROM®, or an erasable programmable read-only memory (EPROM) and distributed, or can be transmitted via a network in a format supported by another apparatus.

[0097] The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and / or features of different illustrative embodiments may be combined with each other and / or substituted for each other within the scope of the present invention. Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.

[0098] In one aspect, an information processing system includes: acquisition means configured to acquire device information, the device information including identification information of a device used by a user, an application executed on the device, and information on a screen displayed on the device; inference means configured to infer a problem that the user has with usage of the device, based on the acquired device information; and notification means configured to provide a notification to the user based on an inference result.

[0099] In one aspect, in the information processing system, the acquisition means is configured to acquire second history information of an error of the device. The inference means is configured to infer the problem based on the device information and the second history information.

[0100] In one aspect, an information processing apparatus includes: inference means configured to infer a problem that a user has with usage of a device used by the user, based on device information, the device information including identification information of the device, an application executed on the device, and information on a screen displayed on the device; and notification information generation means configured to generate information for providing a notification based on an inference result obtained by the inference means.

[0101] In one aspect, in the above-described information processing apparatus, the inference means is configured to infer a problem that the user has with usage of the device based on input of the device information to a trained model, the trained model being trained to output a match rate between the problem and a previous inquiry about the device.

[0102] In one aspect, a recording medium storing a program for executing an information processing method, the information processing method includes: inferring a problem that a user has with usage of a device used by the user, based on device information, the device information including identification information of the device, an application executed on the device, and information on a screen displayed on the device; and generating information for providing a notification based on an inference result obtained in the inferring.

[0103] The recording medium according to claim 19, wherein the inferring includes inferring a problem that the user has with usage of the device based on input of the device information to a trained model, the trained model being trained to output a match rate between the problem and a previous inquiry about the device.

[0104] The present invention can be implemented in any convenient form, for example using dedicated hardware, or a mixture of dedicated hardware and software. The present invention may be implemented as computer software implemented by one or more networked processing apparatuses. The processing apparatuses include any suitably programmed apparatuses such as a general purpose computer, a personal digital assistant, a Wireless Application Protocol (WAP) or third-generation (3G)-compliant mobile telephone, and so on. Since the present invention can be implemented as software, each and every aspect of the present invention thus encompasses computer software implementable on a programmable device. The computer software can be provided to the programmable device using any conventional carrier medium (carrier means). The carrier medium includes a transient carrier medium such as an electrical, optical, microwave, acoustic or radio frequency signal carrying the computer code. An example of such a transient medium is a Transmission Control Protocol / Internet Protocol (TCP / IP) signal carrying computer code over an IP network, such as the Internet. The carrier medium may also include a storage medium for storing processor readable code such as a floppy disk, a hard disk, a compact disc read-only memory (CD- ROM), a magnetic tape device, or a solid state memory device.

[0105] The functionality of the elements disclosed herein may be implemented using circuitry or processing circuitry which includes general purpose processors, special purpose processors, integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or combinations thereof which are configured or programmed, using one or more programs stored in one or more memories, to perform the disclosed functionality. Processors are considered processing circuitry or circuitry as they include transistors and other circuitry therein. In the disclosure, the circuitry, units, or means are hardware that carry out or are programmed to perform the recited functionality. The hardware may be any hardware disclosed herein which is programmed or configured to carry out the recited functionality.There is a memory that stores a computer program which includes computer instructions. These computer instructions provide the logic and routines that enable the hardware (e.g., processing circuitry or circuitry) to perform the method disclosed herein. This computer program can be implemented in known formats as a computer-readable storage medium, a computer program product, a memory device, a record medium such as a CD-ROM or DVD, and / or the memory of an FPGA or ASIC.

[0106] This patent application is based on and claims priority to Japanese Patent Application No. 2024-076320, filed on May 9, 2024, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.[Reference Signs List]

[0107] 100 information processing system110 image forming apparatus120 server apparatus130 network210 CPU220 RAM230 ROM240 storage device250 communication I / F260 display270 input device280 printer device290 scanner device311 printing unit312 reading unit313 authentication processing unit314 device information acquisition unit315 communication unit316 display unit317 operation unit318 history storage unit321 communi cati on unit322 problem inference unit323 notification information generation unit324 trained model

Claims

[CLAIMS]1. An information processing system comprising: acquisition means configured to acquire device information, the device information including identification information of a device used by a user, an application executed on the device, and information on a screen displayed on the device; inference means configured to infer a problem that the user has with usage of the device, based on the acquired device information; and notification means configured to provide a notification to the user based on an inference result.

2. The information processing system according to claim 1, wherein the notification means is configured to provide the notification indicating a solution to the problem that the user has with usage of the device.

3. The information processing system according to claim 2, wherein the notification means is configured to allow the user to select whether the problem has been solved.

4. The information processing system according to claim 3, wherein in a case where it is selected that the problem has not been solved, the acquisition means is configured to receive an input of additional information, and the inference means is configured to infer the problem based on the device information and the additional information.

5. The information processing system according to claim 4, wherein in a case where it is selected that the problem has not been solved, the notification means is configured to provide the notification indicating a contact destination for human support.

6. The information processing system according to claim 1, wherein the inference means is configured to infer a problem that the user has with usage of the device based on input of the device information acquired by the acquisition means to a trained model, the trained model being trained to output a match rate between the problem and a previous inquiry about the device.

7. The information processing system according to claim 6, wherein the trained model is trained by using data of records of responses to previous inquiries about devices, and the data of the records of the responses includes identification information of devices about which inquiries have previously been made, applications executed by the devices attimes of the inquiries, and information on screens displayed on the devices at the times of the inquiries.

8. The information processing system according to claim 6, wherein the inference means is configured to infer a plurality of problems having high match rates as problems that the user has with usage of the device, the match rates being output, and the notification means is configured to notify the user of the plurality of problems in a selectable manner.

9. The information processing system according to claim 8, wherein the notification means is configured to notify the user of a solution to a problem selected from among the plurality of problems.

10. The information processing system according to claim 8, wherein the notification means is configured to allow an operation of notifying one or more problems other than the plurality of problems.

11. The information processing system according to claim 10, wherein in a case where the operation of notifying the one or more problems is performed, the notification means is configured to notify the user of a problem having a match rate that is next highest after the match rates of the plurality of problems having been notified.

12. The information processing system according to claim 1, wherein the acquisition means is configured to acquire information related to the user, and the inference means is configured to infer the problem based on the device information and the information related to the user.

13. The information processing system according to claim 1, wherein the acquisition means is configured to acquire first history information of an operation performed on the device, and the inference means is configured to infer the problem based on the device information and the first history information.

14. An information processing method comprising: inferring a problem that a user has with usage of a device used by the user, based on device information, the device information including identification information of the device, an application executed on the device, and information on a screen displayed on the device; and generating information for providing a notification based on an inference result obtained in the inferring.

15. The information processing method according to claim 14, wherein the inferring includes inferring a problem that the user has with usage of the device based on input of the device information to a trained model, the trained model being trained to output a match rate between the problem and a previous inquiry about the device.

Images