Information processing systems, information presentation systems

Abstract

To reduce the impact of user skill when proposing countermeasures for anomalies. [Solution] The present invention provides an information processing system that enables communication between an information presentation system 20 for detecting equipment abnormalities and a user terminal 10 for receiving notifications of equipment abnormalities. The system includes: a search unit 24 that acquires a first abnormality image representing the equipment abnormality as an image from the user terminal, calculates the similarity between the first abnormality image and a second abnormality image representing past equipment abnormalities as images, and identifies the abnormality type associated with the second abnormality image whose similarity is above a threshold; an answer acquisition unit 26 that creates a prompt using maintenance records that preserved the equipment abnormality with respect to the identified abnormality type and acquires an answer to the prompt; and a first communication unit 21 that transmits the answer to the user terminal. The user terminal includes a second communication unit 11 for receiving the answer and a display control unit 12 for displaying the received answer.

Description

Technical Field

[0001] The present disclosure relates to an information processing system and an information presentation system.

Background Art

[0002] When an abnormality occurs in equipment such as factory equipment or production equipment, the user needs to identify the cause of the abnormality, examine the maintenance records, and determine whether countermeasures are necessary and, if so, what countermeasures to take.

[0003] There is known a damage analysis technique using an image obtained by photographing the appearance of a damaged portion and keywords (see, for example, Patent Document 1). Patent Document 1 discloses a technique for selecting a reference image similar to the appearance characteristics of a current damaged device and calculating the occurrence probability of a damage event using a keyword selected from the keywords corresponding to this reference image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the conventional technology, it has not been easy for users with insufficient skills in prompt engineering to elicit appropriate answers. A technology for eliciting a countermeasure method from an answer of a generative AI by a user inputting information regarding an abnormality to the generative AI has been put into practical use. However, in order to elicit desired information from the generative AI, skills in prompt engineering are required. Therefore, depending on the skills of the user, there may be cases where an appropriate answer to the abnormality cannot be elicited.

[0006] In view of the above issues, this disclosure provides a technology that reduces the influence of the user's skill when suggesting countermeasures for anomalies. [Means for solving the problem]

[0007] In view of the above problems, the present invention provides an information processing system that enables communication between an information presentation system for detecting equipment abnormalities and a user terminal that receives notifications of such equipment abnormalities, The information presentation system includes: a first communication unit that receives a first abnormality image representing an abnormality of the equipment as an image from the user terminal; a search unit that calculates the similarity between the first abnormality image and a second abnormality image representing a past abnormality of the equipment as an image, and identifies the abnormality type associated with the second abnormality image whose similarity is above a threshold; and an answer acquisition unit that creates a prompt using a maintenance record that preserves the abnormality of the equipment with respect to the identified abnormality type, and obtains an answer to the prompt, the first communication unit transmitting the answer to the user terminal. The user terminal includes a second communication unit for receiving the response and a display control unit for displaying the received response. [Effects of the Invention]

[0008] This disclosure can reduce the impact of user skill when proposing countermeasures against anomalies. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram illustrates the general process of processing by the information presentation system. [Figure 2] This is a system configuration diagram of an example of an information processing system. [Figure 3] This figure shows the hardware configuration of an example of an information display system and user terminal. [Figure 4] This diagram shows an example of the functional configuration of a user terminal and an information presentation system. [Figure 5] This diagram illustrates the configuration of the anomaly management database. [Figure 6]This diagram illustrates the structure of a vector database. [Figure 7] This is an example flowchart illustrating the process by which an information presentation system builds a vector database. [Figure 8] This diagram illustrates the process in the inference phase where the answer acquisition unit generates an answer based on the prompt. [Figure 9] This diagram illustrates the training phase in which the model is trained to output responses to prompts. [Figure 10] This is an example flowchart illustrating user actions when a user uses an information presentation system. [Figure 11] This flowchart illustrates an example of a process that searches the anomaly management database and vector database using an anomaly image and text data entered by the user when an anomaly occurs, and generates a response. [Figure 12] This figure shows an example of an anomaly management screen displayed on a user terminal. [Modes for carrying out the invention]

[0010] The following describes an information presentation system and the information presentation method performed by the information presentation system as an example of the form in which this disclosure will be implemented.

[0011] <Overview of Key Features> First, we will explain the general features of the information presentation system in this disclosure.

[0012] The information presentation system disclosed in this disclosure manages image data of abnormality types and abnormal sounds (hereinafter referred to as "abnormal images") in association. The information presentation system calculates the similarity between abnormal images generated by the equipment and abnormal images managed in the abnormality management DB (database) using an arbitrary method (e.g., SSD: Sum of Squared Difference), and identifies abnormality types whose similarity exceeds a threshold.

[0013] When the user inputs text data (an example of the first text data) explaining an abnormal situation, the information presentation system may calculate the similarity between the vector data (an example of the first vector data) converted from the text data and the vector data in the vector database. The vector data in the vector database is also pre-associated with an abnormal type.

[0014] · The information presentation system displays a list of identified abnormal types and accepts the user's selection of an abnormal type. The information presentation system obtains details of the abnormality, the cause of occurrence, countermeasures, etc. from the past maintenance records of the selected abnormal type. The information presentation system embeds the information included in the maintenance records into a prompt for a large language model to present information such as countermeasures for the occurred abnormality.

[0015] Figure 1 is a diagram for explaining an outline of the processing by the information presentation system. (1) The information presentation system detects an abnormality in the equipment and presents an abnormal image 101 at the time of abnormality occurrence to the user terminal. The abnormality detection is performed in a method corresponding to the abnormality. For example, when the equipment is a breaker or an electromagnetic valve, an abnormal sound related to the abnormality occurs at the time of abnormality. Therefore, a microphone is installed in the equipment in advance, and the abnormality is detected by analyzing the sound acquired by the microphone. The information presentation system converts the sound into the relationship between frequency and intensity by Fourier transform and visualizes the frequency with a certain or more intensity with time on the horizontal axis. In the figure, the white part is the frequency with a certain or more intensity. Note that a part of the abnormal image in the figure is trimmed for convenience of explanation. The information presentation system detects an abnormality when a certain or more intensity is detected at a predetermined frequency. Alternatively, when detecting an abnormality in a semiconductor chip, the information presentation system may detect scratches, dents, etc. by image processing.

[0016] The user sets the abnormal image on the screen. Further, the user can input text data 102 such as what kind of abnormality has occurred and the information the user wants to confirm.

[0017] (2) The information presentation system calculates the similarity between the abnormal image recorded in the abnormality management DB31 and the abnormal image in (1), and identifies the abnormality type of the abnormal image whose similarity is above a threshold. An abnormality type is a type of abnormality classified according to the cause of the abnormality and the method of dealing with it. The abnormalities in the abnormality management DB31 are pre-classified into several abnormality types. The abnormality type is registered by the user when dealing with it. Since the methods of dealing with the same abnormality type are likely to be the same, the information presentation system first narrows down the abnormality type.

[0018] Furthermore, the information presentation system converts text data describing the situation beyond what the user has entered into vector data using an appropriate embedding model. Vector DB41 stores the vector data converted from the text data included in the maintenance records. The information presentation system searches Vector DB41 for vector data similar to the vector data converted from the text data entered by the user.

[0019] (3) The information presentation system presents the user with a list of abnormality types in which the similarity between the abnormal image and at least one of the text data entered by the user is above a threshold. In this embodiment, both the abnormal image and the text data entered by the user are used for the search, but either one alone may be used for the search.

[0020] (4) The information presentation system retrieves and presents each item (cause, action taken, etc.) of the maintenance record corresponding to the abnormality type selected by the user. Note that selecting an abnormality type is not mandatory, and the information presentation system may present the maintenance record of the abnormality type with the highest degree of similarity.

[0021] (5) When a user performs an operation to retrieve a response from the large language model, the information presentation system searches for maintenance records of the anomaly type selected by the user (vector data in vector DB41) using the vector data converted from the text data entered by the user. The information presentation system starts the interaction by including the text data of the maintenance record that is associated with vector data whose similarity is above a threshold in the prompt of the large language model. The information presentation system generates a response from the large language model and displays it on the screen. This response includes the response to the anomaly. The prompt may use the text data entered by the user.

[0022] In this way, the information presentation system includes information contained in maintenance records identified based on anomaly images in the prompts, allowing users to easily retrieve information buried in maintenance records as answers from a large-scale language model, even without expertise in prompt engineering. Since users do not need to thoroughly review past maintenance records, the system can present the cause and solution to problems regardless of the user's skill level.

[0023] <About Terminology> Equipment refers to objects, devices, buildings, etc., necessary for a particular purpose, and can be either movable or immovable property.

[0024] Maintenance refers to keeping equipment and other facilities in good working order, including managing them to prevent breakdowns and repairing them when they occur. Maintenance is sometimes referred to as upkeep.

[0025] An abnormality refers to a state in which some kind of malfunction occurs in equipment, meaning it is not functioning normally. An abnormality means that the equipment cannot be used without maintenance work, or even if it can be used, there is a high probability that it will become unusable very soon.

[0026] Representing equipment abnormalities with images means visualizing information obtained during an abnormality using some method. In this embodiment, examples include images that associate time with sound frequency obtained by a microphone during an abnormality, or images of the equipment's appearance.

[0027] An abnormality type is identification information that classifies abnormalities according to certain criteria and indicates abnormalities that share common characteristics. For example, in the case of an abnormality where a foreign object is found in a circuit breaker, the size, hardness, and type of the foreign object would be the abnormality type.

[0028] A maintenance record is a record of information from the time an anomaly was detected until the worker performed maintenance to fix it.

[0029] Generative AI is a type of artificial intelligence that performs various natural language processing tasks, such as text generation, question answering, text classification, sentiment analysis, information extraction, and text summarization. Generative AI can create content in various formats, including text, images, audio, video, and program code. For example, it can learn patterns and relationships based on large datasets, understand the grammar, meaning, and context of a language, and generate text that meets specified conditions.

[0030] <Example System Configuration> Referring to Figure 2, the system configuration of the information processing system 100 according to this embodiment will be described. Figure 2 is a system configuration diagram of an example of the information processing system 100. The information processing system 100 shown in Figure 2 includes a user terminal 10 and an information presentation system 20. The user terminal 10 can communicate with the information presentation system 20 via networks N1 and N2.

[0031] The user terminal 10 is located in a facility such as a company or a home. Alternatively, the user terminal 10 may be carried by the user. The user terminal 10 is connected to network N2. Network N2 may be a LAN, Wi-Fi®, Wide Area Ethernet®, 4G, 5G, 6G, or other mobile phone network. Network N1 is a wide-area network such as the Internet or WAN. The user terminal 10 does not need to be constantly connected to network N2; it only needs to be connected when using the information presentation system 20. Furthermore, the user terminal 10 may be connected to the information presentation system 20 without going through networks N1 and N2. In other words, the user terminal 10 may also be a console terminal for the information presentation system 20.

[0032] The information presentation system 20 is one or more information processing devices. When an abnormality occurs in the equipment, the information presentation system 20 notifies the user terminal 10 of an alert and presents a method for dealing with the abnormality. Abnormality detection and the presentation of a method for dealing with the abnormality may be performed by different systems. The information presentation system 20 may be a server device. A server is a computer or software that performs the function of providing information and processing results in response to requests from a client. The information presentation system 20 may be a web server that provides web applications, or a separate web server may exist and this web server communicates with the user terminal 10. A web application is an application that runs on a web browser.

[0033] The information presentation system 20 has the functionality of a generative AI. The generative AI provides a service that allows users to converse with artificial intelligence (AI) in natural language. Large Language Models (LLMs) are known as such generative AIs. Large Language Models are natural language processing models trained using a large amount of text data. The generative AI takes in a vast amount of text data in advance and learns knowledge from it using deep learning, reinforcement learning, etc. The generative AI uses this knowledge to provide response messages to prompts. A prompt is the text data that is input to the generative AI.

[0034] It should be noted that the information presentation system 20 does not necessarily have a generating AI; a separate generating AI may exist. In this case, the information presentation system 20 utilizes the services of the generating AI via the API (Application Programming Interface) that the generating AI exposes.

[0035] Known generative AIs or systems incorporating generative AIs include GPT-3, GPT-4, Transformer, and BERT (all registered trademarks). Other large-scale language models may also be used.

[0036] User terminal 10 is, for example, a terminal device (an example of an information processing device) used by a user. A user is a person who uses the information presentation system 20, for example, a person who deals with abnormalities (a person who maintains the system). A web browser or a native application runs on user terminal 10. A native application is an application that cannot be run unless it is installed on user terminal 10. The user can operate user terminal 10 to use various services provided by the information presentation system 20.

[0037] The user terminal 10 can be any information processing device capable of running applications, such as a PC (Personal Computer), smartphone, or tablet device. Such information processing devices may also include, for example, an electronic whiteboard, network-connected home appliances, or a wearable PC.

[0038] <Example Hardware Configuration> Referring to Figure 3, the hardware configuration of the information presentation system 20 and the user terminal 10 will be described. Figure 3 is a diagram showing the hardware configuration of an example of the information presentation system 20 and the user terminal 10. As shown in Figure 3, the information presentation system 20 and the user terminal 10 are built by a computer 500, which includes a CPU 501, ROM 502, RAM 503, HD (Hard Disk) 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, bus line 510, keyboard 511, pointing device 512, optical drive 514, and media I / F 516. In addition, the computer 500 may have a GPU (Graphics Processing Unit), etc.

[0039] Of these components, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs used to drive the CPU 501, such as the IPL. The RAM 503 is used as the work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501. The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network I / F 509 is an interface for data communication using the network N2. The bus line 510 is an address bus, data bus, etc., for electrically connecting each component, such as the CPU 501 shown in Figure 3.

[0040] The keyboard 511 is a type of input means equipped with multiple keys used for inputting characters, numbers, or various instructions. The pointing device 512 is a type of input means used for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The optical drive 514 controls the reading or writing of various data to the optical storage medium 513, which is an example of a removable recording medium. The optical storage medium is a DVD, CD, etc. The media interface 516 controls the reading or writing (storage) of data to the recording medium 515, such as flash memory.

[0041] <About the features> Next, with reference to Figure 4, the functional configuration of the user terminal 10 and the information presentation system 20 will be described. Figure 4 is a diagram showing an example of the functional configuration of the user terminal 10 and the information presentation system 20.

[0042] <<User Terminal>> The user terminal 10 includes a communication unit 11, a display control unit 12, and an operation reception unit 13. Each of these functional units is a function or means realized by the CPU 501 shown in Figure 3 executing instructions contained in one or more programs installed on the user terminal 10. For example, the communication unit 11, the display control unit 12, and the operation reception unit 13 may be realized by a web browser and a web application. If the user terminal 10 runs a native application, these may be realized by the native application.

[0043] The communication unit 11 sends and receives various types of information to and from the information display system 20. In this embodiment, the communication unit 11 receives abnormal images from the information display system 20 and transmits text data entered by the user to the information display system 20. The communication unit 11 is an example of a second communication unit.

[0044] The display control unit 12 interprets screen information from various screens transmitted from the information presentation system 20 and displays it on the display 506. This screen displays abnormal images, text data entered by the user, a list of abnormality types, and responses from a large-scale language model (methods for dealing with abnormalities).

[0045] The operation reception unit 13 receives various user operations on the various screens displayed on the display 506.

[0046] <<Information Display System>> The information presentation system 20 includes a communication unit 21, an anomaly detection unit 22, an information management unit 23, a search unit 24, an output unit 25, and a response acquisition unit 26. Each of these functional units of the information presentation system 20 is a function or means realized by the CPU 501 shown in Figure 3 executing instructions included in one or more programs installed in the information presentation system 20. The information presentation system 20 also has an anomaly management DB 31 and a vector DB 41, which are built on the HD 504 and RAM 503 shown in Figure 3. The anomaly management DB 31 and vector DB 41 do not necessarily have to be present in the information presentation system 20; they only need to be on a network that the information presentation system 20 can access.

[0047] The communication unit 21 sends and receives various types of information to and from the user terminal 10. In this embodiment, the communication unit 21 sends abnormal images to the user terminal 10 and receives text data entered by the user from the user terminal 10. The communication unit 21 is an example of a first communication unit.

[0048] The anomaly detection unit 22 detects anomalies in the equipment based on data acquired from the equipment. As described above, in the case of anomaly detection based on sound, the anomaly detection unit 22 detects an anomaly when a white pixel with a certain intensity or higher is displayed in a predetermined frequency band in an anomaly image with time on the horizontal axis and frequency on the vertical axis. Alternatively, the anomaly detection unit 22 may perform a Fourier transform on the sound data collected by the microphone and detect an anomaly when a certain intensity or higher is detected at a predetermined frequency. Furthermore, when detecting anomalies based on an appearance image or the like, the anomaly detection unit 22 may detect an anomaly by comparing it with an image without an anomaly and detecting an anomaly if there is a difference of a certain amount or more. In any case, the anomaly detection unit 22 may detect anomalies using a model trained on a CNN (Convolutional Neural Network) or the like that outputs whether there is an anomaly or not in response to an image input.

[0049] The information management unit 23 registers information related to maintenance records in the anomaly management DB 31 and registers vector data in the vector DB 41. The information management unit 23 also reads information that matches the search criteria from the anomaly management DB 31 or the vector DB 41.

[0050] The search unit 24 converts the input data (text data) transmitted from the user terminal 10 into vector data and performs a search of the vector DB 41. The search unit 24 also searches the anomaly management DB 31 using the anomaly image transmitted from the user terminal 10.

[0051] The output unit 25 generates display information so that the user terminal 10 can display search results and other information on its screen. This display information includes HTML files, XML files, CSS files, JavaScript (registered trademark), etc., and is the information used to display the error management screen described later. The display information is provided as a web application that the user terminal 10 executes.

[0052] The response acquisition unit 26 generates a response using a generation AI. The generation AI has learned in advance how to handle anomalies. The generation AI may be integrated with the information presentation system 20, or a commercially available generation AI service that is available on the network separately from the information presentation system 20 may be used. The response acquisition unit 26 communicates with the generation AI system via an API.

[0053] <<Error management DB>> Figure 5 illustrates the configuration of the Anomaly Management DB 31. As shown in Figure 5, the Anomaly Management DB 31 is a relational database that manages anomaly images and maintenance records. The Anomaly Management DB 31 includes an anomaly image table 32, a maintenance record table 33, an equipment master 34, anomaly type master 35, and an operator master 36.

[0054] The abnormal image table 32 is a table for managing abnormal images. A table is an area that stores data represented in a tabular format. Each row of a table is called a record, each column is called a column, and each cell is called a field. The abnormal image table 32 has an image URL and a record ID associated with the image ID. The abnormal image table 32 may also be managed by adding other attributes. • The image ID is the identification information for abnormal images. The image URL is the URL where the abnormal image is stored. The record ID is the identification information for the maintenance record table 33. The abnormal image table 32 can associate abnormal images with abnormal type IDs by referencing the maintenance record table 33 using the record ID as a key. Multiple abnormal images may be associated with a single maintenance record table 33. Past abnormal images can be referenced as image files.

[0055] The maintenance record table 33 is a table that manages maintenance records performed for individual anomalies. The maintenance record table 33 has the date and time, device ID, anomaly type ID, anomaly details, cause, action taken, and worker ID, associated with the record ID. The maintenance record table 33 may also be managed with additional attributes. The date and time indicated is the date and time the anomaly occurred. The device ID is identification information for the equipment where the malfunction occurred (e.g., circuit breaker, electromagnetic valve, etc.). The maintenance record table 33 uses the device ID as a key to refer to the device master 34. The abnormality type ID is an ID assigned to each type of abnormality and serves as identification information for the abnormality type. The maintenance record table 33 references the device type master using the abnormality type ID as a key. Multiple maintenance record tables 33 may exist with the same abnormality type. The abnormality type indicates the nature of the abnormality. For example, in the case of an abnormality where foreign matter is mixed into a circuit breaker, the abnormality type ID is determined by the size, hardness, and type of the foreign matter. Similarly, if the abnormality is a scratch, the abnormality type is determined by the size, depth, and shape of the scratch. Abnormalities with the same abnormality type tend to have similar corresponding actions. • The cause is the reason that caused the malfunction, and is the cause identified during maintenance and entered by the user. • The actions taken by the user to maintain the system after an abnormality occurred (repair, setting changes, cleaning, parts replacement, etc.) were entered by the user. • The worker ID is the identification information of the worker who performed the maintenance. The maintenance record table 33 references the worker master 36 using the worker ID as the key.

[0056] The device master 34 is a table for managing equipment. The device master 34 has equipment names associated with equipment IDs. The device master 34 may also manage other attributes. • The equipment name is a general designation for the equipment.

[0057] The Abnormal Type Master 35 is a table that manages abnormal types. The Abnormal Type Master 35 has an abnormal type name associated with an abnormal type ID. The Abnormal Type Master 35 may also manage other attributes. • The abnormal type name is a general designation for the abnormal type.

[0058] Worker Master 36 is a table for managing workers. Worker Master 36 contains the worker's name, years of experience, and skill level, associated with the worker ID. Other attributes may also be added to manage the worker master data. • The name is the name of the worker. • Years of experience refer to the worker's length of service with the company or the number of years since they started performing the relevant duties. • Skill level refers to the worker's proficiency in their assigned tasks.

[0059] <<Vector DB>> Figure 6 illustrates the configuration of the vector DB 41. The vector DB 41 is a database that stores vector data converted from chunks, which are created by dividing the abnormality details, causes, and response details of the maintenance record table 33 into chunks. The content of the text divided into chunks depends on the format of the maintenance record, so if other text data such as subject lines or supplementary information is included, these may also be included in the processing. The vector DB 41 is an example of a vector data storage unit. A chunk is a short piece of data divided to make the original data or information easier to handle. For example, it may be divided into chunks of a certain length, such as 100 to several hundred characters, or it may be divided into semantic units such as paragraphs or items within the information (which may be limited to a certain length).

[0060] As shown in Figure 6, each record in the vector DB41 contains a base chunk ID, vector data, the original text data corresponding to the vector data, the URL where the maintenance record body resides, as well as an anomaly type ID, record ID, creation date and time, and skill level. These data are merely examples, and may not include the data shown in Figure 6, nor may they include other data not shown in Figure 6. The chunk ID is an identifier that identifies the original text (one record of the vector data). Vector data is obtained by converting the original text into a distributed representation (an example of the second type of vector data). A distributed representation is a representation of text data using fixed-length vectors. The conversion to a distributed representation is performed by an existing embedding model. When the embedding model converts to a distributed representation, there is an upper limit to the maximum length of the chunk (the number of characters in the original text), so the original text may be divided into abnormal content, cause, and corresponding content at this upper limit. Note that while BERT® and Doc2Vec are known embedding models, any method can be used to convert to a distributed representation.

[0061] Converting the original text into distributed representations allows for the numerical representation of the meaning and content of the original text, and enables comparison using vector data. Traditional text-based searching and comparison methods have drawbacks, such as failing to find a match if even a single character is different, or finding a match if even a single character is the same. In contrast, comparing using vector data allows for searching information based on the degree of similarity in meaning and content of the original text. Since distributed representations are vector data, the original text can be represented as points in a vector space. The similarity of vector data can be calculated using existing methods, such as cosine similarity. The original text corresponds to the chunks mentioned above. That is, the original text is the original text data corresponding to the vector data. The original text is an example of the second text data. The URL indicates the URL where the original conservation record that forms the basis of the chunk resides. The anomaly type ID, record ID, creation date and time, and skill level are metadata copied from the anomaly management DB31.

[0062] As shown in Figure 6, since an anomaly type ID is associated with the vector data, the vector data can be filtered by the anomaly type ID. Furthermore, since a skill level is associated with the vector data, the vector data can be filtered by a skill level specified by the user. This improves the reliability of prompts. Additionally, since the creation date and time are associated with the vector data, the vector data can be filtered within a date and time range specified by the user. Other data besides the data shown in Figure 6 may be registered in the vector DB41.

[0063] Vector data is stored in the vector DB41 in this manner. Text data entered by the user is also converted into vector data, and the vector data in the vector DB41 is searched (the similarity between two vector data can be calculated). The information presentation system 20 adds text data corresponding to vector data with a similarity above a threshold as a prompt, thereby allowing the system to retrieve the abnormality details, causes, and countermeasures buried in the maintenance record (table) in the answer.

[0064] <<Process to build a vector database>> Figure 7 is a flowchart illustrating the process by which the information presentation system 20 constructs the vector DB 41.

[0065] S1: The information management unit 23 of the information presentation system 20 divides the abnormality details, cause, and response details into chunks (original text) of an appropriate number of characters according to the layout of the maintenance record.

[0066] S2: Information Management Unit 23 assigns a data ID to uniquely identify the original text.

[0067] S3: Information Management Unit 23 converts the original text into vector data using an appropriate embedding model. The embedding model is not particularly restricted as long as conversion to distributed representation is possible.

[0068] S4: Information Management Unit 23 associates chunk IDs, original text, and vector data, and manages them by linking them with metadata such as the record ID, creation date and time, and skill level of the maintenance record containing the original text. The number of pages in the maintenance record may also be managed.

[0069] With the vector DB41, the search unit 24 can not only search the anomaly management DB31 using anomaly images, but also search maintenance records using text data entered by the user, and can identify the type of anomaly by combining these search results.

[0070] <Generating answers using a generation AI system> Next, referring to Figure 8, we will explain the process by which the response acquisition unit 26 generates an answer to a prompt using the generation AI. Figure 8 is a diagram illustrating the flow in which the response acquisition unit 26 generates an answer based on the prompt during the inference phase. We will briefly explain using Transformer as an example of the generation AI. Transformer is equipped with an encoder 48 and a decoder 49.

[0071] First, the encoder 48 converts the input text into a vector with contextual features. When generating the first character, the decoder 49 takes this input text vector and a special symbol representing the beginning of the sentence. <bos>Upon receiving input, special characters <bos>The system infers the word that follows. A word that follows is a word that has been learned to be highly relevant. For example, in Figure 8(a), the input sentence is "A buzzer sounds from the circuit breaker..." <bos>In response, decoder 49 output "of the breaker". Decoder 49 has a fully connected layer and a softmax function on its output side, and outputs the generation probability of all candidate words (all learned words). The word with the highest generation probability, or the word that is most likely to be in the same sequence as words generated so far based on grammar and dictionaries, is output.

[0072] When inferring word 2 following word 1, the same vector is provided from encoder 48 to decoder 49, and special characters and word 1 are input to decoder 49. Decoder 49 outputs word 2 following word 1. For example, in Figure 8(b), the input sentence is "Buzzer from the breaker...", <bos>In response to "of the circuit breaker," decoder 49 outputted "of the circuit breaker" and "the humming sound."

[0073] When inferring word 3 following word 2, the same vector is provided from encoder 48 to decoder 49, and special characters, word 1, and word 2 are input to decoder 49. Decoder 49 outputs word 3 following word 2. For example, in Figure 8(c), the input sentence is "Buzzer from the breaker...", <bos>In response to "the circuit breaker's" and "humming sound," the decoder 49 output "the circuit breaker's," "humming sound," and "coil core." The answer acquisition unit 26 generates the answer by repeating this process until a special symbol indicating completion is output.

[0074] Figure 9 illustrates the training phase in which the model is trained to output responses to prompts. Figure 9(a) shows one of the training datasets. The training dataset consists of an input sentence and a target. The model is trained to output the target in response to the input sentence.

[0075] It is assumed that the model has been pre-trained. During pre-training, the learning device (any information processing device) generates fill-in-the-blank questions from a corpus or text data on the internet and performs self-supervised learning to predict the words that will fill in the blanks. The words to be generated are provided as training signals to the output of decoder 49 (fully connected layer and softmax layer), and the loss is propagated to learn parameters such as weights and biases between nodes. After pre-training, fine-tuning the parameters of the machine learning model for a specific task is called fine-tuning.

[0076] Figure 9(b) illustrates the operation of the encoder and decoder during the learning phase. During the learning phase (fine-tuning), the encoder 48 receives input (the entire text), and the decoder 49 receives input one word at a time. The decoder 49 outputs the probability of each word using the softmax function. The loss between this probability and a training signal (one-hot vector) containing only the words to be inferred is calculated, and the weights within the decoder 49 are learned using backpropagation. Note that learning can be performed in batches. For example... <bos>Given the input "the humming sound of the circuit breaker," the decoder 49 outputs "the humming sound of the circuit breaker" and "coil core." However, since "the humming sound of the circuit breaker" has already been learned, "coil core" is learned as the correct output.

[0077] The learning device may also be trained to associate input text with targets using only the decoder 49, without using the encoder 48.

[0078] <Processing or operation flow> Next, we will explain the user operations on the user terminal 10 based on Figure 10. This explanation is based on the anomaly management screen 50 in Figure 12. Figure 10 is a flowchart illustrating the user operations when a user uses the information presentation system 20. The process in Figure 10 is executed when some kind of anomaly occurs.

[0079] S11: An alert for an anomaly is issued to the user terminal 10. Since the alert includes an anomaly image, if the user wants to be shown how to deal with the anomaly, they set the anomaly image on the anomaly management screen 50. If multiple anomaly images are obtained for a single anomaly, the user can select one or more anomaly images. The operation reception unit 13 of the user terminal 10 accepts the setting of the anomaly image.

[0080] S12: The user also inputs information describing the anomaly that occurred, or information the user wishes to confirm, as text data into the anomaly management screen 50. The operation reception unit 13 of the user terminal 10 accepts the input. The user may omit inputting text data.

[0081] S13: The information presentation system 20 presents the user terminal 10 with abnormality types whose similarity between the abnormal image set by the user and the abnormal image in the abnormal image table 32 is above a threshold. The information presentation system 20 also presents the user terminal 10 with abnormality types whose similarity between the vector data converted from the text data entered by the user and the vector data in the vector DB 41 is above a threshold. The user terminal 10 displays a list of abnormality types whose similarity is above the threshold. The user selects an abnormality type that will be helpful in dealing with the current abnormality, and the operation reception unit 13 of the user terminal 10 accepts the selection.

[0082] S14: If the user requests an answer generated by a large-scale language model, an operation to request the generation of an answer by the generation AI is input to the user terminal 10. The operation reception unit 13 of the user terminal 10 receives the operation input.

[0083] S15: In response to the user's request, the information presentation system 20 initiates a dialogue.

[0084] S16: The response acquisition unit 26 of the information presentation system 20 generates a prompt based on the abnormality content, cause, and response content of the maintenance record table 33, which is identified by the abnormality type selected by the user. Alternatively, the response acquisition unit 26 may generate a prompt based on the text data entered in step S12. The response acquisition unit 26 may also search for vector data in the vector DB 41 corresponding to the abnormality type selected by the user using the vector data converted from the text data entered in step S12. This allows the system to identify maintenance records similar to the text data entered by the user, based only on the abnormality type selected by the user. The response acquisition unit 26 displays the generated response to this prompt on the user terminal 10. The user confirms the response.

[0085] S17: Depending on the user's satisfaction with the response, they decide whether to continue the conversation.

[0086] S18: If the dialogue continues, the user enters a new question into the user terminal 10. At this time, the question may consist only of text data, or it may include image data such as another abnormal image. If image data is included, a multimodal large-scale language model is used to generate an answer based on maintenance records corresponding to the same abnormality type.

[0087] S19: If the user does not wish to continue the dialogue, they may either proceed from step S13 to re-select the abnormality type, or they may terminate the process.

[0088] <Processing or operation of the information presentation system> Next, the processing or operation of the information presentation system 20 will be described based on Figure 11. Figure 11 is a flowchart illustrating the process of searching the anomaly management DB 31 and vector DB 41 using the anomaly image and text data entered by the user when an anomaly occurs, and generating a response. Note that some of the processing in Figure 11 overlaps with that in Figure 10.

[0089] S21: First, the anomaly detection unit 22 detects an anomaly. The method for detecting the anomaly can be any method appropriate to the anomaly. The information presentation system 20 notifies the user terminal 10 that an anomaly has occurred, along with an anomaly image. The display control unit 12 of the user terminal 10 displays the anomaly image.

[0090] S22: When an anomaly occurs, the user sets an anomaly image on the anomaly management screen 50 (see Figure 12) and inputs information explaining the anomaly and information the user wants to confirm as text data in the form of questions. The operation reception unit 13 of the user terminal 10 receives the setting of the anomaly image and the input of the text data. The communication unit 11 of the user terminal 10 sends the set anomaly image and text data to the information presentation system 20 along with the search request. Note that inputting text data is not mandatory.

[0091] S23: When the communication unit 21 of the information presentation system 20 receives these, the search unit 24 first calculates the similarity between the abnormal image stored in the abnormal image table 32 and the abnormal image received from the user terminal 10. Methods for calculating the similarity between images include comparing pixel values ​​(e.g., SSD), comparing SIFT features, histogram matching, and comparing structural similarity (SSIM) indices. Alternatively, the relationship between two images may be determined by an image similarity judgment model. For example, an image similarity judgment model generated by metric learning is trained to decrease the distance between image data of the same class while increasing the distance between image data of different classes. The search unit 24 inputs two abnormal images into this image similarity judgment model and can determine whether the abnormal images are related by comparing the output distance with a threshold.

[0092] The search unit 24 pre-sets a threshold for similarity, and if the similarity is above the threshold, it identifies the record ID as an abnormal image related to the anomaly that occurred in the present.

[0093] S24: Meanwhile, the search unit 24 converts the text data entered by the user into vector data using an embedding model. The embedding model is the same as the one that generated the vector data in the vector DB 41.

[0094] S25: The search unit 24 calculates the similarity between the vector data converted in step S24 and the vector data in vector DB 41, and identifies the record ID associated with the vector data whose similarity is determined to be above a threshold. The similarity of the vector data can be any index used to calculate the similarity between vector data, such as cosine similarity, L1 distance, L2 distance, etc.

[0095] S26: The search unit 24 searches the maintenance record table 33 using the record ID identified based on the abnormal image and the record ID identified based on the input text data, and obtains the abnormality type ID included in the search results.

[0096] S27: The output unit 25 of the information presentation system 20 searches the abnormal type master 35 using the abnormal type ID and obtains the abnormal type name. The communication unit 21 of the information presentation system 20 sends a list of abnormal types, including the abnormal type name, to the user terminal 10. The communication unit 11 of the user terminal 10 receives the list of abnormal types, and the display control unit 12 displays the list of abnormal types on the abnormal management screen 50 (list display field 52 in Figure 12). At this time, the output unit 25 may also display the number of maintenance records corresponding to each abnormal type, the most recent date and time (previous occurrence date and time), etc., as reference information.

[0097] S28: The user selects an arbitrary abnormality type from the list of abnormality type names displayed on the abnormality management screen 50. The operation reception unit 13 of the user terminal 10 accepts the selection. The communication unit 11 transmits the selected abnormality type name (abnormality type ID) to the information presentation system 20. The output unit 25 generates a summary of each item of the maintenance record corresponding to the selected abnormality type (abnormality content, cause, and response content in the maintenance record table 33) using a large-scale language model. If there are multiple maintenance records of the same abnormality type, the output unit 25 generates a summary for each of them. The output unit 25 may also obtain the full text instead of a summary.

[0098] The communication unit 21 transmits abnormal images, maintenance record summaries, etc., classified under the abnormality type selected by the user to the user terminal 10. The communication unit 11 of the user terminal 10 receives the abnormal images and maintenance record summaries, etc., and the display control unit 12 displays the abnormal images and maintenance record summaries, etc., on the abnormality management screen 50 (maintenance record section 53 in Figure 12).

[0099] S29: If the user requests an answer from the large-scale language model, the process proceeds to step S30; otherwise, the process proceeds to S36.

[0100] S30: The search unit 24 of the information presentation system 20 filters the vector data in the vector DB 41 by the abnormality type (abnormality type ID) selected by the user. By narrowing down the vector data by abnormality type, vector data that are simply similar in abnormality image but have different abnormality types, and vector data that are similar to vector data converted from input text data but have different abnormality types can be excluded from the search.

[0101] S31: The search unit 24 searches the vector data of the vector DB 41, which has been narrowed down by filtering, against the vector data from step S24. This makes it possible to identify the original text within the maintenance record that is likely to contain the maintenance record for the current anomaly.

[0102] S32: The search unit 24 retrieves the original text corresponding to the vector data whose similarity is above the threshold. If there are multiple vector data with similarity above the threshold, there will be multiple original texts.

[0103] S33: The response acquisition unit 26 includes the original text obtained in step S32 in the prompt of the large-scale language model and obtains the response from the large-scale language model. The response acquisition unit 26 may further include the text data (corresponding to the question) entered by the user in step S22 in the prompt. If there are multiple original texts obtained in step S32, all of them may be included in the prompt. The process of searching the database with the text data entered by the user and generating a response with a prompt based on the text data and search results is called RAG (Retrieval-Augmented Generation).

[0104] The output unit 25 acquires the generated response. The communication unit 21 sends the response to the user terminal 10. The communication unit 11 of the user terminal 10 receives the response, and the display control unit 12 displays it on the error management screen 50 (response text 71 in Figure 12).

[0105] S34: The information presentation system 20 determines whether or not to continue the dialogue in response to the user's actions.

[0106] S35: If the user continues the conversation, the user enters a new question as text data into the anomaly management screen 50 based on the answer. The user may also enter image data.

[0107] S36: If the user does not continue the dialogue and selects another anomaly type name, the process returns to step S28, and the information presentation system 20 displays the anomaly image, maintenance record summary, etc., again for this anomaly type. If the user does not select another anomaly type name, the process in Figure 11 ends.

[0108] <Example of display on the error management screen> Figure 12 shows an example of an anomaly management screen 50 displayed by the user terminal 10. The anomaly management screen 50 is a screen that allows the user to set an anomaly image and input text data, and to display how to deal with the anomaly in an interactive format. The anomaly management screen 50 has a setting field 51, a list display field 52, a maintenance record field 53, and an interactive field 54. Note that the list display field 52, the maintenance record field 53, and the interactive field 54 do not have to be displayed on a single screen; they may be displayed via screen transitions or as pop-ups.

[0109] The settings section 51 includes an abnormal image section 55, a text section 56, and a search button 57. The abnormal image section 55 is set with an abnormal image related to the abnormality that occurred (an example of a first abnormal image). The abnormal image may be automatically displayed in the abnormal image section 55 when an abnormality occurs, or it may be pasted manually by the user.

[0110] Text field 56 is where the user enters text data describing the abnormal situation or questions. Note that entering text data is not mandatory.

[0111] The search button 57 is used to display a list of anomaly type names that are similar to the anomaly image and text data. When the search button 57 is pressed, a list of anomaly types will be displayed in the list display area 52.

[0112] The list display area 52 displays a list of anomaly types that match the search criteria based on anomaly images (and text data). In Figure 12, types A through C are displayed as anomaly types. The number of occurrences for each anomaly type is also displayed. The number represents the number of times this anomaly type has occurred in the past (the number in the maintenance record table 33 that has the same anomaly type ID). In addition, the date and time of the last occurrence are displayed for each anomaly type. Users can decide which anomaly type to select based on the number of occurrences, the date and time of occurrence, etc.

[0113] Furthermore, each anomaly type has a skill level input field 77. Users can enter their skill level in the input field 77. This allows the search unit 24 to search for vector data that corresponds to the anomaly type and skill level selected by the user (allowing filtering of anomaly types by skill level). Users can view only maintenance records handled by users with a certain level of proficiency. The response acquisition unit 26 can also generate prompts using these maintenance records. Additionally, users may further filter anomaly types by specifying a date and time.

[0114] The user selects one anomaly type that will be helpful in addressing the current anomaly. If there is only one anomaly type, no selection is necessary. Alternatively, the search unit 24 may automatically select the anomaly type with the highest similarity to the anomaly image. In this case as well, the user does not need to select an anomaly type.

[0115] When a user selects an anomaly type, the maintenance record field 53 displays the maintenance record anomaly image 61, image file list 62, etc. The maintenance record anomaly image 61 is an anomaly image corresponding to the selected anomaly type (an example of a second anomaly image). The image file list 62 is a list of files containing anomaly images corresponding to the selected anomaly type. The image file selected in the image file list 62 is displayed in the maintenance record anomaly image 61.

[0116] The maintenance record anomaly image 61 may display the anomaly image with the highest similarity to the anomaly image of the anomaly that occurred this time. Alternatively, all anomaly images with the anomaly type selected by the user may be displayed as thumbnails or the like in the maintenance record anomaly image 61. Alternatively, the maintenance record anomaly image 61 may display the anomaly image related to the last (previous) anomaly among the anomaly types selected by the user.

[0117] The Anomaly Details column 63 displays the anomaly details contained in the maintenance record table 33 for the anomaly type selected by the user. If there are multiple maintenance record tables 33 for the same anomaly type, the Anomaly Details column 63 may display each one individually or a summary of them. The anomaly details may be the full text or a summary.

[0118] The cause column 64 displays the cause registered in the maintenance record table 33 for the type of anomaly selected by the user. If there are multiple maintenance records for the same anomaly type, the cause column 64 may display each one individually or a summary of them. The cause may be the full text or a summary.

[0119] The Action Details field 65 displays the action details registered in the maintenance record table 33 for the type of anomaly selected by the user. If there are multiple maintenance records for the same anomaly type, the Action Details field 65 may display each one individually or show a summary of them. The action details may be the full text or a summary.

[0120] The interactive mode button 66 is pressed when the user wants to further review maintenance information in an interactive format regarding the maintenance record for the abnormality type selected by the user. By pressing the interactive mode button 66, the user can start a conversation in the conversation field 54.

[0121] Note that the text field 56 and the response text 71 may be displayed in the dialogue field 54 even without pressing the dialogue mode button 66. The response text 71 can be displayed if the user selects an anomaly type, or it can be displayed based on the maintenance record associated with the anomaly image that has the highest similarity to the anomaly image, even if the user does not select an anomaly type.

[0122] Immediately after pressing the dialogue mode button 66, the response text 71 is displayed in the dialogue field 54. The text data from the text field 56 may be displayed before the response text 71. As explained in Figure 11, when displaying the response text 71, the search unit 24 performs the following processing: It searches for vector data identified by the abnormality type selected by the user in the list display field 52 using the vector data converted from the text data in the text field 56. The response acquisition unit 26 generates the response text 71 using the original text associated with vector data whose similarity is above a threshold as a prompt. The user can input a question text 72 in response to this response text 71. The prompt may include the text data from the text field 56.

[0123] The answer acquisition unit 26 generates an answer 73 in response to the user's question 72 and displays it in the dialogue field 54. The answer acquisition unit 26 can also convert the question 72 into vector data and search the vector DB 41 to identify a matching record ID. The answer acquisition unit 26 can also display the maintenance record 76 identified by the record ID along with the answer 73. The maintenance record 76 has a link (the URL where the actual maintenance record 76 exists) embedded in it, and when the user clicks it, the entire maintenance record can be displayed. Similar processing is possible for the text data in the text field 56, in which case the maintenance record 76 with the embedded link is attached to the answer 71.

[0124] Furthermore, the user can continue the conversation by entering further questions (question 74). The number of responses is optional. The user presses the end button (question 75) to end the conversation. The user may also restart the conversation by pressing the conversation mode button (question 66) or by starting over from selecting the abnormality type in the list display area (question 52).

[0125] <Main effects> The information presentation system 20 of this embodiment includes information contained in maintenance records acquired based on abnormal images in the prompt, allowing users to easily extract information buried in maintenance records as a response from a large-scale language model, even without expertise in prompt engineering. Since users do not need to thoroughly review past maintenance records, the system can present the cause and solution to problems regardless of the user's skill level.

[0126] [Other application examples] The present invention is not limited to the embodiments specifically disclosed above, and various modifications and changes are possible without departing from the scope of the claims. It goes without saying that the information presentation system 20 described in this embodiment is just one example, and there are various system configurations depending on the application and purpose.

[0127] For example, the information display system 20 may notify the user terminal 10 of the anomaly detection via email or social media. Alternatively, the user may connect to the information display system 20 using the user terminal 10 that received the anomaly notification, or they may connect another terminal device to the information display system 20.

[0128] For example, the configuration example shown in Figure 4 is divided according to its main function in order to facilitate understanding of the processing performed by the information presentation system 20. The present invention is not limited by the way the processing units are divided or the names of those units. The processing of the information presentation system 20 can be further divided into many more processing units depending on the processing content. Furthermore, it is possible to divide the system so that one processing unit includes even more processing.

[0129] Each of the functions of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0130] The apparatus described in the examples represents only one of several computing environments for carrying out the embodiments disclosed herein. In one embodiment, the information presentation system 20 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein.

[0131] Furthermore, the information presentation system 20 can combine the disclosed processing steps in various ways. Each element of the information presentation system 20 may be combined into a single device or divided into multiple devices. Also, each processing performed by the information presentation system 20 may be performed by a terminal device. [Explanation of symbols]

[0132] 10 user terminals 20 Information Presentation System 100 Information Processing Systems< / bos> < / bos> < / bos> < / bos> < / bos> < / bos>

Claims

1. An information processing system that enables communication between an information display system for detecting equipment abnormalities and a user terminal that receives notifications of said equipment abnormalities, The aforementioned information presentation system is A first communication unit receives a first abnormality image representing the abnormality of the equipment in image form from the user terminal, A search unit calculates the similarity between the first abnormal image and a second abnormal image representing past abnormalities of the equipment, and identifies the type of abnormality associated with the second abnormal image whose similarity is above a threshold. The system includes a response acquisition unit that creates a prompt using a maintenance record that preserves the equipment abnormality with respect to the identified abnormality type, and acquires a response to the prompt, The first communication unit transmits the response to the user terminal. The aforementioned user terminal is A second communication unit that receives the aforementioned response, A display control unit that displays the received response, An information processing system having

2. If there are multiple second abnormal images whose similarity is equal to or greater than a threshold, and the abnormality types associated with the multiple second abnormal images are different, The first communication unit transmits the list of abnormality types to the user terminal. The display control unit displays the list of abnormality types, The user terminal has an operation reception unit that accepts the selection of the abnormality type, The second communication unit transmits the selected abnormality type to the information presentation system. The response acquisition unit creates a prompt using the maintenance record relating to the selected abnormality type. The information processing system according to claim 1.

3. The display control unit displays the first abnormal image, The operation reception unit receives input of first text data describing the abnormality of the equipment, The second communication unit transmits the first text data to the information presentation system. The aforementioned information presentation system is The device has a vector data storage unit that stores, in association with, the second text data obtained by dividing one maintenance record into multiple second text data, the second vector data converted from the second text data, and the anomaly type. The search unit calculates the similarity between the first vector data converted from the first text data and the second vector data, and identifies the anomaly type associated with the second vector data whose similarity is equal to or greater than a threshold. The first communication unit transmits to the user terminal the abnormality type identified in the first abnormal image and a list of the abnormality types identified in the first text data. The response acquisition unit creates a prompt using the second text data to which the abnormality type selected by the user is associated. The information processing system according to claim 2.

4. The search unit calculates the similarity between the second vector data to which the abnormality type selected by the user is associated and the first vector data, and identifies the second text data associated with the second vector data whose similarity is equal to or greater than a threshold. The response acquisition unit creates a prompt using the second text data which is associated with the second vector data whose similarity is equal to or greater than a threshold. The information processing system according to claim 3.

5. The response acquisition unit further creates the prompt using the first text data. The information processing system according to claim 3 or 4.

6. The vector data storage unit stores the date and time when the anomaly occurred, in association with the second text data and the second vector data. The display control unit displays the date and time when the last occurrence of the abnormality of the abnormality type occurred, along with the list of abnormality types. The information processing system according to claim 3.

7. The vector data storage unit associates the second text data and the second vector data with the date and time the anomaly occurred and the skill level of the worker corresponding to the anomaly. The display control unit displays the error type, the date and time when the error of that error type last occurred, and an input field for the skill level. The search unit calculates the similarity between the first vector data and the second vector data to which the skill level entered by the user and the anomaly type selected by the user are associated, and identifies the second text data to which the similarity is equal to or greater than a threshold. The response acquisition unit creates a prompt using the second text data which is associated with the second vector data whose similarity is equal to or greater than a threshold. The information processing system according to claim 3.

8. When the user terminal displays a list of abnormality types, accepts the selection of one abnormality type, and transmits the selected abnormality type to the information presentation system, The first communication unit transmits to the user terminal the full text or summary of the maintenance record to which the abnormality type selected by the user is associated. The display control unit displays the full text or a summary of the maintenance record. The information processing system according to claim 2 or 3.

9. The first communication unit transmits the second abnormal image, to which the abnormality type selected by the user is associated, to the user terminal. The display control unit causes the second abnormal image to be displayed. The information processing system according to claim 8.

10. When the display control unit displays the received response, Display a link to the maintenance record associated with the second vector data whose similarity to the first vector data is above a threshold. The information processing system according to claim 3.

11. An information display system that can communicate with user terminals and detect equipment abnormalities, A first communication unit receives a first abnormality image representing the abnormality of the equipment in image form from the user terminal, A search unit calculates the similarity between the first abnormal image and a second abnormal image representing past abnormalities of the equipment, and identifies the type of abnormality associated with the second abnormal image whose similarity is above a threshold. The system includes a response acquisition unit that creates a prompt using a maintenance record that preserves the equipment abnormality with respect to the identified abnormality type, and acquires a response to the prompt, The first communication unit transmits the response to the user terminal. An information presentation system having [a certain feature].

Images