Information processing system, information processing device, information processing method and program

Abstract

To provide an appropriate handling method.SOLUTION: A user device 100 transmits image information indicating an image being displayed, to an information processing device 200. The information processing device 200 obtains error pattern information indicating an error pattern from the transmitted image information; retrieves, based on the obtained error pattern information, handling information indicating a method of handling the error pattern indicated by the error pattern information, from a database 300; and transmits it to the user device 100. The user device 100 displays the transmitted handling information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, there have been many cases where messages entered in text format are exchanged via communication devices (see, for example, Patent Document 1). In technology for exchanging such messages, chatbots have been developed and put into practical use that automatically reply with appropriate response messages in response to inquiry messages sent to them. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-84366 Summary of the Invention [Problem to be solved by the invention]

[0004] In a system that applies technology such as the chatbot described above, for example, if an error or malfunction occurs on a personal computer (PC), in order to inquire about how to resolve the issue, a message must be sent that contains a text representation of the screen displayed on the PC. This requires the user to convert visual information into text and explain it. Furthermore, even if the same phenomenon occurs and the same screen is displayed, there may be differences in the textual representations made by the users. In such cases, depending on the textual representation, there is a risk that an appropriate solution will not be found. This poses a problem in that it is not easy to obtain an appropriate solution.

[0005] An object of the present invention is to provide an information processing system, an information processing device, an information processing method, and a program that can easily obtain an appropriate countermeasure. [Means for solving the problem]

[0006] The information processing system of the present invention comprises: A system including a user device, an information processing device, and a database, The user device A display unit; an image information acquisition unit that acquires image information indicating an image displayed by the display unit; an image information transmitting unit that transmits the image information acquired by the image information acquiring unit to the information processing device; The information processing device includes: an acquisition unit that acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit; a search unit that searches the database for handling information indicating how to handle the error pattern indicated by the error pattern information, based on the error pattern information acquired by the acquisition unit, in which the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information are stored in association with each other in advance; a handling information transmission unit that transmits the handling information searched by the search unit to the user device; The display unit displays the handling information transmitted from the handling information transmission unit.

[0007] Further, the information processing device of the present invention comprises: an acquisition unit that acquires error pattern information indicating an error pattern from image information indicating an image displayed on a display unit of the user device; a search unit that searches a database that stores error pattern information and handling information indicating how to handle an error pattern indicated by the error pattern information in association with each other in advance, based on the error pattern information acquired by the acquisition unit; and The system further includes a handling information transmission unit that transmits the handling information searched by the search unit to the user device.

[0008] Further, the information processing method of the present invention comprises: A process of acquiring image information indicating an image displayed on the display unit; A process of acquiring error pattern information indicating an error pattern from the acquired image information; a process of searching for handling information indicating how to handle the error pattern indicated by the error pattern information from a database that stores the error pattern information in association with the error pattern information based on the acquired error pattern information; and outputting the retrieved handling information.

[0009] The program of the present invention also includes: A program to be executed by a computer, acquiring image information indicating an image displayed on the display unit; acquiring error pattern information indicating an error pattern from the acquired image information; a step of searching for the error pattern information from a database that stores the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information in association with each other in advance, based on the acquired error pattern information; and a procedure for outputting the retrieved handling information. [Effects of the Invention]

[0010] In the present invention, a suitable solution can be easily obtained. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a first embodiment of an information processing system according to the present invention. [Figure 2] 2 is a diagram illustrating an example of the internal configuration of the user device illustrated in FIG. 1. [Figure 3] FIG. 2 is a diagram showing an example of associations stored in the database shown in FIG. [Figure 4] 2 is a diagram illustrating an example of an internal configuration of the information processing device illustrated in FIG. 1. [Figure 5] 5 is a diagram illustrating an example of input and output of the learning model illustrated in FIG. 4. [Figure 6] 2 is a sequence diagram illustrating an example of an information processing method in the information processing system shown in FIG. 1. [Figure 7] FIG. 10 is a diagram showing an example of a display screen when the display unit displays handling information in step S8. [Figure 8] FIG. 10 is a diagram illustrating a second embodiment of an information processing system according to the present invention. [Figure 9] FIG. 9 is a diagram showing an example of associations stored in the database shown in FIG. 8. [Figure 10] 9 is a diagram illustrating an example of an internal configuration of the information processing device illustrated in FIG. 8. FIG. [Figure 11] 9 is a sequence diagram illustrating an example of an information processing method in the information processing system shown in FIG. 8. FIG. [Figure 12] FIG. 10 is a diagram showing an example of a display screen when the display unit displays handling information in step S18. [Figure 13] FIG. 10 is a diagram illustrating a third embodiment of an information processing system according to the present invention. [Figure 14] FIG. 14 is a diagram illustrating an example of the internal configuration of the information processing device illustrated in FIG. [Figure 15] 14 is a sequence diagram illustrating an example of an information processing method in the information processing system shown in FIG. 13. FIG. [Figure 16] FIG. 10 is a diagram illustrating a fourth embodiment of an information processing system according to the present invention. [Figure 17] FIG. 17 is a diagram illustrating an example of the internal configuration of the information processing device illustrated in FIG. [Figure 18] 17 is a sequence diagram illustrating an example of an information processing method in the information processing system shown in FIG. 16. [Figure 19] FIG. 10 is a diagram illustrating a fifth embodiment of an information processing system according to the present invention. [Figure 20] FIG. 20 is a diagram showing an example of associations stored in the database shown in FIG. 19. [Figure 21] FIG. 20 is a diagram illustrating an example of an internal configuration of the information processing device illustrated in FIG. [Figure 22]20 is a sequence diagram illustrating an example of an information processing method in the information processing system shown in FIG. 19. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (First embodiment)

[0013] Fig. 1 is a diagram showing a first embodiment of an information processing system of the present invention. As shown in Fig. 1, the information processing system in this embodiment includes a user device 100, an information processing device 200, and a database 300. These components are communicably connected to each other via a communication network 400. Alternatively, these components may be directly connected to each other without using the communication network 400.

[0014] The user device 100 is a device operated by a user. The user device 100 is, for example, a device equipped with a communication function, such as a notebook or desktop PC, a tablet terminal, etc. FIG. 2 is a diagram showing an example of the internal configuration of the user device 100 shown in FIG. 1. As shown in FIG. 2, the user device 100 shown in FIG. 1 has a display unit 110, an image information acquisition unit 120, and an image information transmission unit 130. Note that FIG. 2 shows only the main components related to this embodiment among the components equipped in the user device 100 shown in FIG. 1.

[0015] The display unit 110 is a general display that displays information. It goes without saying that the display unit 110 displays not only text information but also images. The display unit 110 displays the handling information transmitted from the information processing device 200. The handling information will be described later.

[0016] The image information acquisition unit 120 acquires image information indicating an image displayed by the display unit 110. For example, the image information acquisition unit 120 has a function that can save a display screen, such as a screenshot, as an image file based on an operation received from the outside by the user device 100. The image information acquisition unit 120 saves image information (image file) indicating the acquired image in memory (not shown) within the user device 100. The file format of the image file acquired by the image information acquisition unit 120 is not particularly specified.

[0017] The image information transmitting unit 130 transmits the image information acquired by the image information acquiring unit 120 to the information processing device 200. The image information transmitting unit 130 may read the image information stored in the memory by the image information acquiring unit 120 from the memory and transmit the image information to the information processing device 200. The signal format in which the image information transmitting unit 130 transmits the image information is not particularly specified.

[0018] The database 300 stores error pattern information in advance in association with handling information indicating how to handle the error pattern indicated by the error pattern information. The database 300 may be provided in the information processing device 200. The information stored in the database 300 can be edited externally. The error pattern information will be described later. Fig. 3 is a diagram showing an example of the association stored in the database 300 shown in Fig. 2. As shown in Fig. 3, the database 300 shown in Fig. 2 stores error pattern information and handling information in association with each other.

[0019] The information processing device 200 transmits handling information to the user device 100 in response to image information transmitted from the user device 100. Fig. 4 is a diagram showing an example of the internal configuration of the information processing device 200 shown in Fig. 2. As shown in Fig. 4, the information processing device 200 shown in Fig. 2 has a learning model generation unit 210, a learning model 220, an acquisition unit 230, a search unit 240, and a handling information transmission unit 250. Note that Fig. 4 shows only the main components related to this embodiment among the components provided in the information processing device 200 shown in Fig. 2.

[0020] In the learning phase of the learning model 220, the learning model generation unit 210 generates training data in which image information is associated with error pattern information. For example, the learning model generation unit 210 generates training data in which image information representing an image is associated with error pattern information (error codes) capable of identifying errors or defects, such as "A0001" and "B0001," stored as error pattern information in the database 300 as shown in FIG. 3. The learning model generation unit 210 uses the generated training data to perform machine learning to generate a learning model 220 that outputs error pattern information corresponding to the image information. In other words, the learning model generation unit 210 uses multiple pieces of image information and multiple pieces of corresponding error pattern information to perform machine learning to determine which error pattern caused the image represented by the image information to be displayed, thereby generating the learning model 220.

[0021] The learning model 220 is a trained model that has learned the relationship between image information and error pattern information. The learning model 220 outputs error pattern information when image information is input. The learning model 220 may have, for example, a neural network structure in which multiple neurons are interconnected. A neuron is an element that performs a predetermined calculation on multiple inputs and outputs one value as the calculation result. The learning model 220 is stored in a storage unit (not shown). The learning method in the learning model 220 may be a general method for generating a learning model. Figure 5 is a diagram showing an example of input / output of the learning model 220 shown in Figure 4. As shown in Figure 5, the learning model 220 shown in Figure 4 is a learning model that outputs error pattern information when image information is input.

[0022] The acquisition unit 230 acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit 130 of the user device 100. Specifically, the acquisition unit 230 inputs the image information transmitted from the image information transmission unit 130 of the user device 100 into the learning model 220, thereby acquiring error pattern information indicating an error pattern output from the learning model 220.

[0023] The search unit 240 searches the database 300 for handling information based on the error pattern information acquired by the acquisition unit 230. Specifically, the search unit 240 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out handling information associated with the error pattern information from the database 300. Here, handling information is information that indicates how to handle an error or malfunction identified by the error pattern information when it occurs. The handling information is, for example, information such as "restart the PC" or "terminate application A" as shown in FIG. 3.

[0024] The handling information transmitting unit 250 transmits the handling information searched by the searching unit 240 to the user device 100. The signal format in which the handling information transmitting unit 250 transmits the handling information is not particularly specified.

[0025] The following describes an information processing method in the information processing system shown in Fig. 1. Fig. 6 is a sequence diagram for explaining an example of an information processing method in the information processing system shown in Fig. 1.

[0026] First, when a malfunction occurs in the user device 100 operated by the user (step S1), the image information acquisition unit 120 acquires image information representing the image displayed on the display unit 110 (step S2). The timing at which the image information acquisition unit 120 acquires the image information may be when the user operates the user device 100 to activate the screenshot function. Alternatively, the timing at which the image information acquisition unit 120 acquires the image information may be when the user device 100 itself is equipped with a detection means and the detection means detects the occurrence of an error or malfunction. Next, the image information transmission unit 130 transmits the image information acquired by the image information acquisition unit 120 to the information processing device 200 (step S3). The timing at which the image information transmission unit 130 transmits the image information to the information processing device 200 is preferably as soon as possible after the image information acquisition unit 120 acquires the image information. The timing at which the image information transmission unit 130 transmits the image information to the information processing device 200 may be calculated based on the timing at which the image information acquisition unit 120 acquires the image information. Furthermore, the timing at which the image information transmitting unit 130 transmits the image information to the information processing device 200 may be triggered by a predetermined operation performed by the user on the user device 100 .

[0027] When image information is transmitted from the user device 100, the acquisition unit 230 of the information processing device 200 inputs the image information transmitted from the image information transmission unit 130 of the user device 100 to the learning model 220 (step S4). In this way, the acquisition unit 230 acquires error pattern information indicating the error pattern output from the learning model 220 (step S5). Next, the search unit 240 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out handling information associated with the error pattern information from the database 300 (step S6). Then, the handling information transmission unit 250 transmits the handling information read out from the database 300 by the search unit 240 to the user device 100 (step S7). The display unit 110 of the user device 100 displays the handling information transmitted from the handling information transmission unit 250 of the information processing device 200 (step S8).

[0028] 7 is a diagram showing an example of a display screen when the display unit 110 displays the handling information in step S8. In step S8, the display unit 110 displays the handling information transmitted from the handling information transmission unit 250, as shown in FIG. 7. The display format of the handling information on the display unit 110 is not particularly limited as long as it is readable by the user operating the user device 100. Furthermore, the handling information is not limited to text information, and may also show images including drawings, photographs (still images), and videos.

[0029] As described above, in this embodiment, when an error or malfunction occurs in the user device 100, the information processing device 200 acquires image information displayed on the screen, inputs the acquired image information into the learning model 220 to acquire error pattern information, and reads out handling information stored in the database 300 using the acquired error pattern information as a search key. The readout handling information is displayed on the user device 100. This eliminates the need to convert visual information into text information, facilitates communication, and allows the user to efficiently and easily obtain appropriate handling methods for the error or malfunction. The learning model generation unit 210 may perform reinforcement learning on the learning model 220 based on the results of the user's handling of the error or malfunction using the handling method indicated in the handling information displayed on the user device 100. In this case, the user operates the user device 100 to input result information indicating whether the error or malfunction has been resolved by using the handling method indicated in the handling information displayed on the user device 100. The learning model generation unit 210 of the information processing device 200 then performs reinforcement learning on the learning model 220 based on the result information. This makes it easier to obtain a more appropriate solution.

[0030] The learning model 220 may also output that the input image information is image information for which a model trained using pre-associated training data cannot output appropriate error pattern information. It is possible to predict that the image information input to the learning model 220 was displayed due to an unknown error or malfunction. In such a case, the information processing device 200 may externally acquire error pattern information corresponding to the image information, add the association between the image information and the acquired error pattern information to the training data of the learning model generation unit 210, and have the learning model generation unit 210 retrain the learning model 220 to generate a new learning model 220. At this time, the information processing device 200 may store the image information in a storage unit (not shown). At this time, the information processing device 200 may acquire not only the image information but also information capable of identifying its occurrence and information necessary for analysis (e.g., the date and time of occurrence, the OS version, peripheral devices used at the same time, etc.) from the user device 100, and store these together with the re-learning history in a storage unit (not shown). (Second embodiment)

[0031] Fig. 8 is a diagram showing a second embodiment of the information processing system of the present invention. As shown in Fig. 8, the information processing system in this embodiment has a user device 100, an information processing device 201, and a database 301. These components are connected to each other so that they can communicate with each other via a communication network 400. Alternatively, these components may be connected to each other directly without using the communication network 400. The user device 100 is the same as that in the first embodiment.

[0032] The database 301 stores error pattern information in advance in association with a plurality of pieces of countermeasure information indicating a plurality of countermeasure methods for the error pattern indicated by the error pattern information. The database 301 stores the correspondences with a priority order. The database 301 may be provided in the information processing device 201. FIG. 9 is a diagram showing an example of the correspondences stored in the database 301 shown in FIG. 8. As shown in FIG. 9, the database 301 shown in FIG. 8 stores error pattern information, countermeasure information, and priority order in association with each other. The priority order indicates the level of effectiveness of the countermeasure indicated by the countermeasure information for the error pattern. Specifically, the priority order indicates the degree of effectiveness of the countermeasure, i.e., the probability that the error or malfunction will be resolved by taking this countermeasure. The priority order is preset based on the functions of applications running on the user device 100 and the user's usage history. The priority order can also be changed externally. 9, the priority is shown as "high" or "low," but it may also be shown using numbers from "1" to "5," or letters from "A" to "C." Furthermore, the priority may also be shown using a probability (numerical value) that taking this action will resolve the error or malfunction.

[0033] Fig. 10 is a diagram showing an example of the internal configuration of the information processing device 201 shown in Fig. 8. As shown in Fig. 10, the information processing device 201 shown in Fig. 8 has a learning model generation unit 210, a learning model 220, an acquisition unit 230, a search unit 241, and a handling information transmission unit 251. Note that Fig. 10 shows only the main components related to this embodiment among the components included in the information processing device 201 shown in Fig. 8. The learning model generation unit 210, the learning model 220, and the acquisition unit 230 are the same as those in the first embodiment.

[0034] The search unit 241 searches for handling information from the database 301 based on the error pattern information acquired by the acquisition unit 230. Specifically, the search unit 241 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out a plurality of pieces of handling information associated with the error pattern information from the database 301. At this time, the search unit 241 also reads out from the database 301 the priority levels associated with each of the plurality of pieces of handling information that have been searched for.

[0035] The handling information transmission unit 251 transmits the handling information searched by the search unit 241 to the user device 100. At this time, the handling information transmission unit 251 transmits the multiple pieces of handling information searched by the search unit 241 to the user device 100 based on the priorities assigned to the pieces of handling information. Specifically, the handling information transmission unit 251 may transmit only the handling information with the highest priority among the multiple pieces of handling information searched by the search unit 241 to the user device 100. Alternatively, the handling information transmission unit 251 may transmit all of the multiple pieces of handling information searched by the search unit 241 together with the priorities associated with them to the user device 100. Alternatively, the handling information transmission unit 251 may transmit the multiple pieces of handling information searched by the search unit 241 to the user device 100 at predetermined intervals in descending order of priority. Alternatively, the handling information transmission unit 251 may first transmit to the user device 100 the handling information with the highest priority among the multiple pieces of handling information searched by the search unit 241, and then, if the user device 100 transmits a notification that the error or malfunction has not been resolved, transmit the handling information with the next highest priority. Alternatively, the handling information transmission unit 251 may transmit only the handling information with the highest priority if there is a large difference in the priorities of the multiple pieces of handling information (for example, if the priorities are expressed as probabilities, there is a large difference between them, such as 90% and 10%). A threshold may be set to determine whether there is a large difference, and it may be determined that there is a large difference if the difference is greater than the threshold. The signal format used by the handling information transmission unit 251 to transmit the handling information is not particularly specified.

[0036] The following describes an information processing method in the information processing system shown in Fig. 8. Fig. 11 is a sequence diagram for explaining an example of an information processing method in the information processing system shown in Fig. 8.

[0037] First, when a malfunction occurs in the user device 100 operated by the user (step S11), the image information acquisition unit 120 acquires image information indicating the image displayed on the display unit 110 (step S12). The timing at which the image information acquisition unit 120 acquires the image information may be the same as the timing in the first embodiment. Next, the image information transmission unit 130 transmits the image information acquired by the image information acquisition unit 120 to the information processing device 201 (step S13). The timing at which the image information transmission unit 130 transmits the image information to the information processing device 201 may be the same as the timing in the first embodiment.

[0038] When image information is transmitted from the user device 100, the acquisition unit 230 of the information processing device 201 inputs the image information transmitted from the image information transmission unit 130 of the user device 100 to the learning model 220 (step S14). In this way, the acquisition unit 230 acquires error pattern information indicating an error pattern output from the learning model 220 (step S15). Next, the search unit 241 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out handling information associated with the error pattern information from the database 301 (step S16). At this time, if there are multiple pieces of handling information associated with the error pattern information acquired by the acquisition unit 230, the search unit 241 also reads out the priorities associated with the pieces of handling information from the database 301. Then, the handling information transmission unit 251 transmits the handling information read out from the database 301 by the search unit 241 to the user device 100 (step S17). At this time, the handling information transmission unit 251 transmits the handling information to the user device 100 in a manner according to the priority order as described above. The display unit 110 of the user device 100 displays the handling information transmitted from the handling information transmission unit 251 of the information processing device 201 (step S18).

[0039] FIG. 12 is a diagram illustrating an example of a display screen when the display unit 110 displays the handling information in step S18. In step S18, the display unit 110 displays the handling information transmitted from the handling information transmission unit 251, as shown in FIG. 12. The handling information is displayed according to the priority of each of the handling information transmitted together with the multiple pieces of handling information from the handling information transmission unit 251. In the example shown in FIG. 12, the priority level indicated by the handling information "Change property setting 1 to 2" is higher than the priority level indicated by the handling information "Close application A." Therefore, the display shown in FIG. 12 prompts the user to take action according to the handling information "Close application A" if the error or malfunction is not resolved despite taking action according to the handling information "Change property setting 1 to 2." The display format of the handling information on the display unit 110 is not particularly limited as long as it is legible to the user operating the user device 100. Furthermore, the handling information is not limited to text information, and may also be images, including drawings, photographs (still images), and videos.

[0040] As described above, in this embodiment, when an error or malfunction occurs on the user device 100, the information processing device 201 acquires image information displayed on the screen, inputs the acquired image information into the learning model 220 to acquire error pattern information, and reads out handling information stored in the database 301 using the acquired error pattern information as a search key. The read-out handling information is displayed on the user device 100. This has the effect of eliminating the need to convert visual information into text information, facilitating communication, and allowing the user to efficiently obtain handling methods for the error or malfunction, making it easy to find an appropriate handling method. Furthermore, multiple pieces of handling information are transmitted to the user device 100 or displayed on the user device 100 based on priority. As a result, if the error or malfunction is not resolved even after taking measures according to the presented handling information, other measures can be taken promptly. (Third embodiment)

[0041] Fig. 13 is a diagram showing a third embodiment of the information processing system of the present invention. As shown in Fig. 13, the information processing system in this embodiment includes a user device 100, an information processing device 202, and a database 300. These components are communicably connected to each other via a communication network 400. Alternatively, these components may be directly connected to each other without using the communication network 400. The user device 100 and the database 300 are the same as those in the first embodiment.

[0042] Fig. 14 is a diagram showing an example of the internal configuration of the information processing device 202 shown in Fig. 13. As shown in Fig. 14, the information processing device 202 shown in Fig. 13 has an acquisition unit 232, a search unit 240, and a handling information transmission unit 250. The search unit 240 and the handling information transmission unit 250 are the same as those in the first embodiment. Note that Fig. 14 shows only the main components related to this embodiment among the components included in the information processing device 202 shown in Fig. 13.

[0043] The acquisition unit 232 acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit 130 of the user device 100. In this embodiment, the acquisition unit 232 acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit 130 using a character recognition function such as an OCR (Optical Character Recognition) function. The acquisition unit 232 performs image processing on the image information transmitted from the image information transmission unit 130, identifies text information, and acquires error pattern information from words and phrases included in the identified text information. To acquire error pattern information from words and phrases included in the text information, the acquisition unit 232 may use a trained model that has learned the relationship between words and error pattern information.

[0044] The following describes an information processing method in the information processing system shown in Fig. 13. Fig. 15 is a sequence diagram for explaining an example of an information processing method in the information processing system shown in Fig. 13.

[0045] First, when a malfunction occurs in the user device 100 operated by the user (step S21), the image information acquisition unit 120 acquires image information indicating the image displayed on the display unit 110 (step S22). The timing at which the image information acquisition unit 120 acquires the image information may be the same as in the first embodiment. Next, the image information transmission unit 130 transmits the image information acquired by the image information acquisition unit 120 to the information processing device 202 (step S23). The timing at which the image information transmission unit 130 transmits the image information to the information processing device 202 may be the same as in the first embodiment.

[0046] When image information is transmitted from the user device 100, the acquisition unit 232 of the information processing device 202 uses the OCR function to acquire error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit 130 (step S24). Next, the search unit 240 uses the error pattern information acquired by the acquisition unit 232 as a search key to search for and read out handling information associated with the error pattern information from the database 300 (step S25). Then, the handling information transmission unit 250 transmits the handling information read out from the database 300 by the search unit 240 to the user device 100 (step S26). The display unit 110 of the user device 100 displays the handling information transmitted from the handling information transmission unit 250 of the information processing device 202 (step S27). The display manner of the handling information on the display unit 110 may be the same as that in the first embodiment.

[0047] As described above, in this embodiment, when an error or malfunction occurs on the user device 100, the information processing device 202 acquires image information displayed on the screen, acquires error pattern information from the acquired image information using a character recognition function, and reads out handling information stored in the database 300 using the acquired error pattern information as a search key. The read-out handling information is displayed on the user device 100. This has the effect of eliminating the need to convert visual information into text information, facilitating communication, and allowing the user to efficiently obtain handling methods for the error or malfunction, making it easy to obtain an appropriate handling method. (Fourth embodiment)

[0048] Fig. 16 is a diagram showing a fourth embodiment of the information processing system of the present invention. As shown in Fig. 16, the information processing system in this embodiment includes a user device 100, an information processing device 203, and a database 300. These components are communicably connected to each other via a communication network 400. Alternatively, these components may be directly connected to each other without using the communication network 400. The user device 100 and the database 300 are the same as those in the first embodiment.

[0049] Fig. 17 is a diagram showing an example of the internal configuration of the information processing device 203 shown in Fig. 16. As shown in Fig. 17, the information processing device 203 shown in Fig. 16 has an acquisition unit 233, a search unit 240, and a handling information transmission unit 250. The search unit 240 and the handling information transmission unit 250 are the same as those in the first embodiment. Note that Fig. 17 shows only the main components related to this embodiment among the components included in the information processing device 203 shown in Fig. 16.

[0050] The acquisition unit 233 acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit 130 of the user device 100. In this embodiment, the acquisition unit 233 identifies the two-dimensional code information included in the image information transmitted from the image information transmission unit 130, and acquires the error pattern information indicated by the identified two-dimensional code information. The acquisition unit 233 performs image processing on the image information transmitted from the image information transmission unit 130, identifies the two-dimensional code included in the image, and acquires the error pattern information indicated by the identified two-dimensional code information.

[0051] The following describes an information processing method in the information processing system shown in Fig. 16. Fig. 18 is a sequence diagram for explaining an example of an information processing method in the information processing system shown in Fig. 16.

[0052] First, when a malfunction occurs in the user device 100 operated by the user (step S31), the image information acquisition unit 120 acquires image information indicating the image displayed on the display unit 110 (step S32). The timing at which the image information acquisition unit 120 acquires the image information may be the same as in the first embodiment. Next, the image information transmission unit 130 transmits the image information acquired by the image information acquisition unit 120 to the information processing device 203 (step S33). The timing at which the image information transmission unit 130 transmits the image information to the information processing device 203 may be the same as in the first embodiment.

[0053] When image information is transmitted from the user device 100, the acquisition unit 233 of the information processing device 203 identifies two-dimensional code information included in the image information transmitted from the image information transmission unit 130 (step S34). Then, the acquisition unit 233 acquires error pattern information indicated by the identified two-dimensional code information (step S35). Next, the search unit 240 uses the error pattern information acquired by the acquisition unit 233 as a search key to search for and read out handling information associated with the error pattern information from the database 300 (step S36). Then, the handling information transmission unit 250 transmits the handling information read out by the search unit 240 from the database 300 to the user device 100 (step S37). The display unit 110 of the user device 100 displays the handling information transmitted from the handling information transmission unit 250 of the information processing device 202 (step S38). The display manner of the handling information on the display unit 110 may be the same as that in the first embodiment.

[0054] As described above, in this embodiment, when an error or malfunction occurs on the user device 100, the information processing device 203 acquires image information displayed on the screen, identifies two-dimensional code information included in the acquired image information, acquires error pattern information indicated by the identified two-dimensional code information, and reads out handling information stored in the database 300 using the acquired error pattern information as a search key. The read-out handling information is displayed on the user device 100. This has the effect of eliminating the need to convert visual information into text information, facilitating communication, and allowing users to efficiently obtain handling methods for errors and malfunctions, making it easy for them to obtain appropriate handling methods. (Fifth embodiment)

[0055] Fig. 19 is a diagram showing a fifth embodiment of the information processing system of the present invention. As shown in Fig. 19, the information processing system in this embodiment has a user device 100, an information processing device 204, and a database 304. These components are communicably connected to each other via a communication network 400. Alternatively, these components may be directly connected to each other without using the communication network 400. The user device 100 is the same as that in the first embodiment.

[0056] The database 304 stores error pattern information in advance in association with handling information indicating how to handle the error pattern indicated by the error pattern information. At this time, the database 304 stores this association together with OS (Operating System) version information. The database 304 may be provided in the information processing device 204. Fig. 20 is a diagram showing an example of the association stored in the database 304 shown in Fig. 19. As shown in Fig. 20, the database 304 shown in Fig. 19 stores error pattern information, handling information, and OS version information in association with each other. Storing handling information for each OS version makes it possible to handle cases where different handling methods are required depending on the OS version, even for the same error pattern.

[0057] Fig. 21 is a diagram showing an example of the internal configuration of the information processing device 204 shown in Fig. 19. As shown in Fig. 21, the information processing device 204 shown in Fig. 19 has a learning model generation unit 210, a learning model 220, an acquisition unit 230, a search unit 244, a handling information transmission unit 250, and a request unit 264. Note that Fig. 21 shows only the main components related to this embodiment among the components included in the information processing device 204 shown in Fig. 19. The learning model generation unit 210, the learning model 220, the acquisition unit 230, and the handling information transmission unit 250 are the same as those in the first embodiment.

[0058] The search unit 244 searches the database 304 for handling information based on the error pattern information acquired by the acquisition unit 230. Specifically, the search unit 244 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out handling information associated with the error pattern information from the database 304. At this time, if multiple pieces of handling information associated with the error pattern information are stored, the search unit 244 reads out handling information associated with the version of the OS that launched the application that displayed the image information. The version information of the OS running on the user device 100 may be acquired in advance, or may be acquired together with the image information when the acquisition unit 230 acquires the image information from the user device 100, or may be acquired by the information processing device 204 from the user device 100 at a timing required by the acquisition unit 230.

[0059] The request unit 264 requests information required for the search by the search unit 244 from the user device 100. For example, the request unit 264 requests the above-mentioned OS version information from the user device 100.

[0060] The following describes an information processing method in the information processing system shown in Fig. 19. Fig. 22 is a sequence diagram for explaining an example of an information processing method in the information processing system shown in Fig. 19. In the following description, an example is given in which the information processing device 204 does not previously store version information of the OS running on the user device 100.

[0061] First, when a malfunction occurs in the user device 100 operated by the user (step S41), the image information acquisition unit 120 acquires image information indicating the image displayed on the display unit 110 (step S42). The timing at which the image information acquisition unit 120 acquires the image information may be the same as the timing in the first embodiment. Next, the image information transmission unit 130 transmits the image information acquired by the image information acquisition unit 120 to the information processing device 204 (step S43). The timing at which the image information transmission unit 130 transmits the image information to the information processing device 204 may be the same as the timing in the first embodiment.

[0062] When image information is transmitted from the user device 100, the acquisition unit 230 of the information processing device 204 inputs the image information transmitted from the image information transmission unit 130 of the user device 100 to the learning model 220 (step S44). In this way, the acquisition unit 230 acquires error pattern information indicating the error pattern output from the learning model 220 (step S45). Next, the search unit 244 uses the error pattern information acquired by the acquisition unit 230 as a search key to search for and read out handling information associated with the error pattern information from the database 304 (step S46). At this time, if there are multiple pieces of handling information associated with the error pattern information acquired by the acquisition unit 230 and these pieces of handling information are stored for each OS version, the request unit 264 requests OS version information from the user device 100 (step S47).

[0063] When the user device 100 notifies the information processing device 204 of the OS version information (step S48), the search unit 244 searches the database 304 for and reads out the handling information associated with the notified OS version information from the handling information searched for in step S46 (step S49). The handling information transmission unit 250 then transmits the handling information that the search unit 244 read out from the database 304 in step S49 to the user device 100 (step S50). The display unit 110 of the user device 100 displays the handling information transmitted from the handling information transmission unit 250 of the information processing device 204 (step S51). The display mode of the handling information on the display unit 110 may be the same as that in the first embodiment. At this time, the display unit 110 may also display the OS version information.

[0064] In this manner, in this embodiment, when an error or malfunction occurs on the user device 100, the information processing device 204 acquires image information displayed on the screen, inputs the acquired image information into the learning model 220 to acquire error pattern information, and reads out handling information stored in the database 304 using the acquired error pattern information as a search key. The read-out handling information is then displayed on the user device 100. This eliminates the need to convert visual information into text information, facilitates communication, and allows the user to efficiently and easily obtain appropriate handling methods for the error or malfunction. Furthermore, if there are multiple pieces of handling information for a single error pattern that differ depending on the OS version, the handling information corresponding to the OS version running on the user device 100 is sent to the user device 100 and displayed. This allows for more detailed handling methods to be obtained.

[0065] In the above embodiment, an example has been given in which troubleshooting information for resolving errors or malfunctions is displayed, but instead of displaying the information, output may be performed using audio, printing from a printer, or transmission to another device. Furthermore, the image information acquired by the image information acquisition unit 120 is not limited to image information of an image displayed on the display unit 1120, but may also be, for example, image information (image data) of an image captured by a camera provided in the user device 100, or image information (image data) of a still image or video acquired from an external device. In this case, based on the image information acquired by the image information acquisition unit 120, information indicating a defect in the product contained in the image may be provided instead of a troubleshooting method.

[0066] Although the above description has been given with each component assigned a respective function (process), this allocation is not limited to the above. Furthermore, the configuration of the components is also not limited to the above, and the above-described forms are merely examples. Furthermore, a combination of the above-described embodiments may also be used. Furthermore, in addition to the above-described processing, a method for dealing with errors and malfunctions may also be acquired based on conventionally input text information or voice information.

[0067] The processing performed by each of the information processing devices 200-204 may be performed by a logic circuit manufactured for each purpose. Alternatively, a computer program (hereinafter referred to as a program) describing the processing procedures may be recorded on a recording medium readable by each of the information processing devices 200-204, and the program recorded on the recording medium may be read and executed by each of the information processing devices 200-204. Examples of recording media readable by each of the information processing devices 200-204 include removable recording media such as magneto-optical disks, DVDs (Digital Versatile Discs), CDs (Compact Discs), Blu-ray (registered trademark) Discs, and USB (Universal Serial Bus) memories, as well as memories such as ROMs (Read Only Memory), RAMs (Random Access Memory), HDDs (Hard Disc Drives), and SSDs (Solid State Drives) built into each of the information processing devices 200-204. The program recorded on the recording medium is read by a CPU provided in each of the information processing devices 200-204, and the same processing as described above is performed under the control of the CPU. Here, the CPU operates as a computer that executes a program read from a recording medium on which the program is recorded.

[0068] Furthermore, a program in which the above-described processing contents are written as procedures may be executed by a CPU provided in the user device 100 as an application that runs within the user device 100. In this case, for example, a user operating the user device 100 may drag and drop an image file acquired by the image information acquisition unit 120 onto the application, and the application may execute the above-described processing. In this way, the information processing devices 200 to 204 are not required. [Explanation of symbols]

[0069] 100 User equipment 110 Display section 120 Image information acquisition unit 130 Image information transmission unit 200-204 Information processing equipment 210 Learning model generation unit 220 Learning Model 230,232,233 Acquisition Department 240,241,244 Search section 250,251 Handling information transmission unit 264 Request part 300,301,304 database 400 Communication Network

Claims

1. A system including a user device, an information processing device, and a database, The user device A display unit; an image information acquisition unit that acquires image information indicating an image displayed by the display unit; an image information transmitting unit that transmits the image information acquired by the image information acquiring unit to the information processing device; The information processing device includes: an acquisition unit that acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit; a search unit that searches the database for handling information indicating how to handle the error pattern indicated by the error pattern information, based on the error pattern information acquired by the acquisition unit, in which the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information are stored in association with each other in advance; a handling information transmission unit that transmits the handling information searched by the search unit to the user device; the display unit displays the handling information transmitted from the handling information transmission unit, the database stores the correspondence between the error pattern information and the handling information for each version of operating software running on the user device; The search unit searches the database for the handling information corresponding to the version of operating software running on the user device based on the error pattern information acquired by the acquisition unit.

2. A system including a user device, an information processing device, and a database, The user device A display unit; an image information acquisition unit that acquires image information indicating an image displayed by the display unit; an image information transmitting unit that transmits the image information acquired by the image information acquiring unit to the information processing device; The information processing device includes: an acquisition unit that acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit; a search unit that searches the database for handling information indicating how to handle the error pattern indicated by the error pattern information, based on the error pattern information acquired by the acquisition unit, in which the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information are stored in association with each other in advance; a handling information transmission unit that transmits the handling information searched by the search unit to the user device; the display unit displays the handling information transmitted from the handling information transmission unit, the database stores the correspondence between the error pattern information and the plurality of pieces of handling information in order of priority; the search unit searches the database for the plurality of pieces of handling information associated with the error pattern information acquired by the acquisition unit; The handling information transmission unit transmits the plurality of pieces of handling information searched by the search unit to the user device based on priorities assigned to the pieces of handling information.

3. a learning model generation unit that performs machine learning on an error pattern corresponding to the image information to generate a learning model; The information processing system of claim 1 or claim 2, wherein the acquisition unit acquires error pattern information indicating the error pattern from the learning model by inputting the image information transmitted from the image information transmission unit into the learning model.

4. 3. The information processing system according to claim 1, wherein the acquisition unit acquires error pattern information indicating an error pattern from the image information transmitted from the image information transmission unit using a character recognition function.

5. The information processing system according to claim 1 or claim 2, wherein the acquisition unit identifies two-dimensional code information contained in the image information transmitted from the image information transmission unit, and acquires error pattern information indicated by the identified two-dimensional code information.

6. The information processing device includes: The information processing system according to claim 1 , further comprising a request unit that requests information required for the search from the user device by the search unit.

7. an acquisition unit that acquires error pattern information indicating an error pattern from image information indicating an image displayed on a display unit of the user device; a search unit that searches for the error pattern information from a database that stores the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information, in association with each version of operating software running on the user device, based on the error pattern information acquired by the acquisition unit; and a handling information transmission unit that transmits the handling information searched by the search unit to the user device; The search unit searches the database for the handling information corresponding to the version of operating software running on the user device based on the error pattern information acquired by the acquisition unit.

8. An acquisition unit that acquires error pattern information indicating an error pattern from image information indicating an image displayed on a display unit provided in the user device; a search unit that searches a database that stores error pattern information and handling information indicating how to handle an error pattern indicated by the error pattern information, in a manner that prioritizes the error pattern information and the handling information, based on the error pattern information acquired by the acquisition unit; and a handling information transmission unit that transmits the handling information searched by the search unit to the user device; the search unit searches the database for the plurality of pieces of handling information associated with the error pattern information acquired by the acquisition unit; The handling information transmission unit transmits the plurality of pieces of handling information searched by the search unit to the user device based on priorities assigned to the pieces of handling information.

9. A process of acquiring image information indicating an image displayed on the display unit; A process of acquiring error pattern information indicating an error pattern from the acquired image information; a process of searching for the error pattern information corresponding to the version of the operating system running on the user device from a database that stores the error pattern information and the error pattern information indicating how to deal with the error pattern indicated by the error pattern information in advance in association with each version of the operating system running on the user device having the display unit, based on the acquired error pattern information; and and outputting the retrieved handling information.

10. A process of acquiring image information indicating an image displayed on a display unit; A process of acquiring error pattern information indicating an error pattern from the acquired image information; a process of searching for the error pattern information from a database that stores the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information in advance in association with each other, with priority assigned based on the acquired error pattern information; and outputting the retrieved handling information based on the priority assigned to the handling information.

11. On the computer, acquiring image information indicating an image displayed on the display unit; acquiring error pattern information indicating an error pattern from the acquired image information; a step of searching for the error pattern information corresponding to the version of the operating system running on the user device from a database that stores the error pattern information and the error pattern information indicating how to deal with the error pattern indicated by the error pattern information in advance in association with each version of the operating system running on the user device having the display unit, based on the acquired error pattern information; and a program for executing a procedure for outputting the retrieved handling information.

12. A computer comprising: acquiring image information indicating an image displayed on the display unit; acquiring error pattern information indicating an error pattern from the acquired image information; a step of searching for the error pattern information from a database that stores the error pattern information and handling information indicating how to handle the error pattern indicated by the error pattern information in advance in association with each other with a priority order based on the acquired error pattern information; and a procedure for outputting the retrieved handling information based on the priority assigned to the handling information.

Images