Program, information processing method, and information processing apparatus

Abstract

This invention provides a program, an information processing method, and an information processing device that efficiently obtain responses from a language model to queries related to acquired power transmission and distribution equipment. [Solution] In the power transmission and distribution equipment analysis support system S, a program stored in the memory unit of the information processing device 1 acquires queries related to power transmission and distribution equipment, generates answers to the queries using a language model that utilizes the acquired queries and technical information from a technical information database that stores technical information related to power transmission and distribution equipment, and causes the information processing device, which is a computer, to execute a process to output the generated answers.

Description

Technical Field

[0001] The present invention relates to a program, an information processing method, and an information processing apparatus.

Background Art

[0002] For example, Patent Document 1 describes matters related to electrical equipment such as transformers, cables, circuit breakers, capacitors, etc., and further mentions a method for evaluating the deterioration of electrical insulating oil used in these electrical equipment.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the method disclosed in Patent Document 1, no consideration is given to efficiently obtaining an answer from a language model for a query regarding the acquired power transmission and distribution facilities.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide a program or the like that can efficiently obtain an answer from a language model for a query regarding the acquired power transmission and distribution facilities.

Means for Solving the Problems

[0006] A program according to an aspect of the present disclosure causes a computer to execute a process of acquiring a query regarding power transmission and distribution facilities, generating an answer to the query using a language model that uses the acquired query and technical information in a technical information database storing technical information regarding power transmission and distribution facilities, and outputting the generated answer.

[0007] In this embodiment, the power transmission and distribution equipment includes, for example, circuit breakers, transformers, TVRs (Thyristor Voltage Regulators), or SVRs (Step Voltage Regulators), and the parts of this power transmission and distribution equipment to be analyzed (observation points) are identified, for example, during periodic maintenance inspections or when responding to faults. The maintenance personnel of the power transmission and distribution equipment send queries regarding the analysis targets (observation points) of the power transmission and distribution equipment being maintained to the information processing device, for example, via an information terminal such as a smartphone used by the maintenance personnel. The control unit of the information processing device generates answers to the queries using a language model that utilizes the acquired queries and technical information from a technical information database that stores technical information related to the power transmission and distribution equipment, thereby enabling efficient acquisition of the answers.

[0008] A program according to one aspect of this disclosure acquires image data of the power transmission and distribution equipment and the query including the target of analysis of the power transmission and distribution equipment, extracts reference information related to the acquired image data and the target of analysis from the technical information database, and generates the response by providing the extracted reference information to the language model.

[0009] In this embodiment, the control unit of the information processing device acquires image data (query image data) of power transmission and distribution equipment, either as an appendix to or as part of a query related to power transmission and distribution equipment. This image data (query image data) may be captured, for example, by a camera built into an information terminal such as a smartphone used by the maintenance personnel, and transmitted from the information terminal to the information processing device as an attachment to the query entered via the information terminal. The control unit of the information processing device may also vectorize the acquired image data (query image data) in the same way as the query to generate an image embedding vector, and then search the technical information database using the text embedding vector obtained by vectorizing the query and the image embedding vector. As a search result based on the image data (query image data), the control unit of the information processing device extracts image data similar to the query image data from the image data stored in the technical information database as reference image data (reference information). In this case, the reference image data becomes part of the reference information, i.e., constitutes a part of the reference information. Furthermore, the control unit of the information processing device may also extract document data (text data) similar to the query image data from the document data (text data) stored in the technical information database as reference information, based on the image data (query image data). In this way, by including image data (query image data) as part of the query in addition to text-based queries and performing a search on the technical information database, a multimodal search process can be performed, enabling a suitable search for queries made by maintenance personnel of power transmission and distribution equipment.

[0010] A program according to one aspect of this disclosure uses a language model to output the response, including the analysis method and the past analysis results, using prompts that include image data of the power transmission and distribution equipment, a query including the target of analysis of the power transmission and distribution equipment, an instruction to extract reference information related to the query by referring to the technical information database, and an instruction to output an analysis method and past analysis results for the target of analysis.

[0011] In this embodiment, the control unit of the information processing device generates a prompt for input to the language model based on the acquired query. That is, the query constitutes at least a part of the prompt. Alternatively, the query may be applied as the prompt itself. The prompt includes image data of the power transmission and distribution equipment (query image), a query including the power transmission and distribution equipment as the target of analysis, an instruction to extract reference information related to the query by referring to a technical information database, and an instruction to output the analysis method, past analysis results, and considerations for the target of analysis. Therefore, by providing the prompt to the language model, the language model can be efficiently made to output a response including the analysis method, past analysis results, and considerations.

[0012] A program according to one aspect of the present disclosure generates an extended prompt based on the extracted reference information and the prompt, and obtains the answer from the language model by inputting the generated extended prompt into the language model.

[0013] In this embodiment, the query may be written in colloquial language, such as, "The power transmission and distribution equipment is a transformer, the object of analysis is the copper wire terminal portion, the matter to be clarified is the cause of damage to the copper wire terminal, search the technical information database based on these matters, and output the details of the analysis method, past analysis results, and considerations, taking the search results into account," and may be used as a prompt input to a Large Language Model (LLM). In other words, the query constitutes at least a part of the prompt. In this case, the prompt may define the user's role, such as being a maintenance worker for power transmission and distribution equipment or an expert in the analysis of power transmission and distribution equipment. That is, the prompt includes four elements: command, context, input data, and output indicator, and the query may also include these elements. The control unit of the information processing device acquires the query sent from an information terminal such as a smartphone used by the maintenance worker, and derives reference information by performing a search on the technical information database stored in the memory unit of the information processing device using the query. The technical information database is stored in the memory unit of the information processing device, and contains various technical information related to power transmission and distribution equipment. This technical information includes, for example, incident reports for past failure incidents, maintenance and inspection reports showing the results of maintenance and inspection, and test reports showing the results of various tests conducted during the development phase of the power transmission and distribution equipment. Furthermore, the technical information may also include product information of the power transmission and distribution equipment, or characteristics or specifications of various components used in the power transmission and distribution equipment, and information on operating methods for inspection devices or analytical instruments used to inspect or analyze power transmission and distribution equipment or components of the power transmission and distribution equipment. The control unit of the information processing device generates an extended prompt that incorporates derived reference information into the prompt generated based on the query entered by the maintenance personnel of the power transmission and distribution equipment, and inputs this extended prompt into a language model, which is a generative learning model with natural language interpretation capabilities. This language model may be, for example, a large-scale language model (LLM) such as ChatGPT implemented on an external server (LLM) that is communicably connected to the information processing device via an external network such as the Internet.Alternatively, the language model may be a private LLM implemented in the information processing device. In this way, the control unit of the information processing device may, based on the acquired query, search the technical information database and generate an extended prompt that takes the search results into account, construct an execution environment that performs this series of processes (sequence) using, for example, an LLM framework such as LangChain. The language model (LLM) that receives the extended prompt generates an answer to the query from the power transmission and distribution equipment maintenance personnel by using the information it has pre-trained, along with the prompt and reference information (search results from the technical information database) included in the extended prompt, and transmits the generated answer to the information processing device. The control unit of the information processing device retrieves the answer from the language model (LLM) and outputs (transmits) the retrieved answer to, for example, an information terminal used by the power transmission and distribution equipment maintenance personnel. In this way, when a query is entered by a maintenance worker of power transmission and distribution equipment, the language model (LLM) is questioned with content (extended prompts) that takes into account the search results (reference information related to the query) from the technical information database where technical information about power transmission and distribution equipment is stored. As a result, even maintenance workers with relatively little experience can efficiently search the technical information stored in the technical information database and derive (extract) reference information related to the query. Furthermore, since the search results (reference information related to the query) from the technical information database are also input to the language model (LLM), even if the language model (LLM) has not been trained or retrained (fine-tuned) using that reference information, it can obtain an appropriate answer using the reference information.

[0014] A program according to one aspect of this disclosure includes technical information stored in the technical information database, which includes image data, and extracts image data by searching the technical information database using the query, and extracts reference information including the extracted image data.

[0015] In this embodiment, the technical information stored in the technical information database includes image data such as photographs, and this image data may be vectorized together with the text data annotated to the image data. In this case, the repository in the technical information database that stores this image data may be configured, for example, with CLIP (Contrastive Language-Image Pre-training). The control unit of the information processing device performs a search on the technical information database using a query entered by a maintenance worker of the power transmission and distribution equipment, thereby extracting image data related to the query (reference image data) from the image data stored in the technical information database and deriving it as part of the reference information. In this case, the control unit of the information processing device may derive as reference information image data whose similarity (cosine similarity, Euclidean distance, dot product similarity measure) is equal to or greater than a predetermined threshold, based on the similarity between the embedded vector vectorized from the prompt generated based on the query and the embedded vector vectorized from the image data or the text data annotated to the image data. In this way, because the technical information database stores image data (technical information), the image data (reference image data) can also be efficiently derived (extracted) based on queries by maintenance personnel of power transmission and distribution equipment, and the answer that takes into account the extracted image data (reference image data) can be obtained from the language model (LLM).

[0016] A program according to one aspect of this disclosure includes, in which the technical information stored in the technical information database includes analysis methods corresponding to the subject of analysis and past cases related to the subject of analysis, and the response from the language model is generated using the analysis methods and past cases extracted from the technical information database based on the query.

[0017] In this embodiment, queries entered by maintenance personnel of power transmission and distribution equipment include information such as where the target of analysis is located, which analysis method is effective for that target, and what the cause is, regarding incidents such as failures that occur in the power transmission and distribution equipment. In response, the technical information stored in the technical information database contains analysis methods appropriate to the target of analysis, such as the operation method of a scanning electron microscope, and past cases related to the cause of incidents, such as analysis results or reports for incidents that have occurred in the past. These past cases describe matters concerning various cause of incidents, such as metal fatigue, overcurrent, overvoltage, or external damage, along with the corresponding incidents. Therefore, by searching the technical information database based on prompts generated based on queries that include matters concerning the target of analysis, analysis methods and past cases highly relevant to the query (prompt) can be extracted (derived) as reference information. Since the language model is input with extended prompts that include this reference information, including analysis methods and past cases, the language model can generate answers that include summaries or considerations using the analysis methods and past cases based on this reference information, and can provide maintenance personnel with useful answers (technical information to deal with incidents).

[0018] A program relating to one aspect of this disclosure includes, in which the analysis method includes matters described in the operating manual for the analytical instrument, and in which the past cases include matters described in reports on past incidents in power transmission and distribution equipment.

[0019] In this embodiment, the analytical methods included in the technical information stored in the technical information database are, for example, matters described in the operating manuals of various analytical instruments; that is, the technical information database contains document data that are operating manuals for various analytical instruments. Such analytical instruments include, for example, optical microscopes, scanning electron microscopes (SEMs), X-ray CT scanners, X-ray diffractometers, Fourier transform infrared spectrophotometers (FT-IRs), or atomic absorption spectrometers. The past cases included in the technical information stored in the technical information database are, for example, matters described in various reports, etc., previously issued or created by organizations to which power transmission and distribution equipment maintenance personnel belong; that is, the technical information database contains document data such as various reports. In this way, the technical information database contains document data such as operating manuals for analytical instruments and reports, etc., that are already held by organizations to which power transmission and distribution equipment maintenance personnel belong, so that the processing man-hours for registering data to the technical information database can be relatively reduced, while effectively utilizing the document data that organizations currently hold.

[0020] A program according to one aspect of this disclosure registers the response obtained from the language model into the technical information database.

[0021] In this embodiment, the control unit of the information processing device can increase the amount of technical information stored in the technical information database by outputting the response obtained from the language model to the information terminal of the maintenance personnel and registering it in the technical information database. The response obtained from the language model is based on an extended prompt that takes into account the search results in the technical information database, and will include a portion of the search results, but is also expected to include matters that the language model has recognized through pre-training. Therefore, the response from the language model will include matters other than the search results in the technical information database, and will also include summaries or considerations generated based on these matters, so it is expected that it will include new content for the technical information stored in the technical information database. In response to this, the control unit of the information processing device can recursively register the response from the language model in the technical information database, thereby performing the registration of the response generated based on the search results as a series of processes, in conjunction with the search process for the technical information database. While general database registration processes require human effort, the control unit of the information processing device can efficiently register technical information in the technical information database by integrating the search and registration for the technical information database, thereby efficiently improving the usefulness of the technical information database.

[0022] A program relating to one aspect of this disclosure comprises a technical information database configured by RAG (Retrieval-Augmented Generation) and including a vector index into which the technical information is vectorized.

[0023] In this embodiment, the technical information database stored in the memory unit of the information processing device is configured using RAG (Retrieval-Augmented Generation). The source data of the technical information stored in the technical information database is expected to be multimodal data consisting of different formats, such as reports on past incidents, operating manuals for analytical instruments, specifications for power transmission and distribution equipment, or characteristic or specification documents for various parts used in power transmission and distribution equipment. In contrast, since the case database is configured using RAG, even if the stored case information is multimodal data consisting of different formats and is a group of data that is not normalized or structured (unstructured data), a vector index can be generated by inputting the unstructured data into the embedding model provided by RAG, and a search process can be performed using this vector index. The vector index is a vector database that stores sentence embedding vectors, which are vectorized at the sentence level or the like within the technical information. A search of the vector index (vector database) in the RAG may be performed by querying using, for example, cosine similarity, Euclidean distance, or dot product similarity measure to extract information that has a similarity level of or higher. Furthermore, the project database may also be constructed using CLIP (Contrastive Language-Image Pre-training), and in this case, it may include a vector database that stores image embedding vectors generated by vectorizing each image contained in the technical information. By constructing the project database using RAG, or a combination of RAG and CLIP, efficient search processing can be performed on technical information consisting of unstructured data.

[0024] An information processing method according to one aspect of this disclosure involves causing a computer to perform the following processes: acquire a query relating to power transmission and distribution equipment, generate a response to the query using a language model that utilizes the acquired query and technical information from a technical information database storing technical information relating to power transmission and distribution equipment, and output the generated response.

[0025] In this aspect, it is possible to provide an information processing method that efficiently obtains an answer from a language model for a query regarding the acquired power transmission and distribution equipment.

[0026] An information processing apparatus according to an aspect of the present disclosure includes an acquisition unit that acquires a query regarding power transmission and distribution equipment, a generation unit that generates an answer to the query using the acquired query and technical information in a technical information database that stores technical information regarding power transmission and distribution equipment by means of a language model, and an output unit that outputs the generated answer.

[0027] In this aspect, it is possible to provide an information processing apparatus that efficiently obtains an answer from a language model for a query regarding the acquired power transmission and distribution equipment.

Advantages of the Invention

[0028] It is possible to provide a program or the like that efficiently obtains an answer from a language model for a query regarding the acquired power transmission and distribution equipment.

Brief Description of the Drawings

[0029] [Figure 1] It is a schematic diagram for explaining the outline of a power transmission and distribution equipment analysis support system according to Embodiment 1. [Figure 2] It is a block diagram showing the configuration of an information processing apparatus. [Figure 3] It is a functional block diagram exemplifying functional units included in the control unit of an information processing apparatus. [Figure 4] It is a flowchart exemplifying a processing procedure by the control unit of an information processing apparatus. [Figure 5] It is an explanatory diagram exemplifying an input screen for inputting a query. [Figure 6] It is an explanatory diagram exemplifying a prompt generated based on a query. [Figure 7] It is an explanatory diagram exemplifying an answer from a language model. [Modes for carrying out the invention]

[0030] (Embodiment 1) The embodiments will be described below with reference to the drawings. Figure 1 is a schematic diagram illustrating the overview of the power transmission and distribution equipment H analysis support system S according to Embodiment 1. Figure 2 is a block diagram showing the configuration of the information processing device 1. The power transmission and distribution equipment H analysis support system S is configured with the information processing device 1 as the main device, and the information processing device 1 is connected to an information terminal T, such as a smartphone used by a maintenance worker of the power transmission and distribution equipment H, via an external network such as the Internet. Alternatively, the information terminal T may be smart glasses or a wearable device, and input to the information terminal T may be by voice input. From the information terminal T, queries regarding the power transmission and distribution equipment H are transmitted to the information processing device 1 in accordance with the operation of the maintenance worker, along with images (query image data) taken of the parts (observation points) that are the target of analysis of events that occurred in the power transmission and distribution equipment H. Furthermore, the information processing device 1 is connected to an external server G (LLM server) on which a large-scale language model (LLM) is implemented, via an external network such as the Internet.

[0031] The information processing device 1 generates a prompt based on the acquired query, searches the technical information database based on the prompt, and generates an extended prompt that includes reference information as a result of the search results. The information processing device 1 outputs the extended prompt to the large-scale language model implemented on the external server G, and acquires generated data (answers) from the external server G that are generated according to the content of the extended prompt. The information processing device 1 transmits the answers acquired from the external server G to the information terminal T.

[0032] The imaging device C is, for example, a CMOS camera. The imaging device C may also be, for example, a camera built into an information terminal T such as a smartphone held by a maintenance worker of the power transmission and distribution equipment H, or it may be connected via the input / output IF 14 of the information processing device 1. The information terminal T of the maintenance worker of the power transmission and distribution equipment H and the information processing device 1 are connected in a way that allows communication, for example, via an external network such as the Internet or a LAN. The parts (observation points) of the events occurring in the power transmission and distribution equipment H captured by the imaging device C that are to be analyzed are acquired by the information processing device 1 by transmission from the information terminal T of the maintenance worker, acquisition via the input / output IF 14 of the information processing device 1, or offline acquisition via a storage medium such as a memory card.

[0033] The information processing device 1 is a computer capable of various information processing and information transmission / reception, such as a server device or a personal computer. The server device includes not only a single server device but also a cloud server device or a virtual server device composed of multiple computers. The information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, and an input / output IF 14.

[0034] The control unit 11 has one or more arithmetic processing units equipped with timing functions such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, etc. by reading and executing the program P (program product) stored in the storage unit 12.

[0035] The storage unit 12 includes volatile storage areas such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), and flash memory, as well as non-volatile storage areas such as EEPROM or hard disk. The storage unit 12 pre-stores programs P (program products) and data referenced during processing, and also stores various data, including intermediate data, generated during processing. The programs P (program products) stored in the storage unit 12 may be programs P (program products) read from a recording medium M that the information processing device 1 can read. Alternatively, programs P (program products) may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 12. As will be described in detail later, the storage unit 12 stores a technical information database.

[0036] The communication unit 13 is a communication module or communication interface for communicating with information terminals T etc. via wired or wireless methods such as Ethernet®, for example, a narrow-area wireless communication module such as Wi-Fi® or Bluetooth®, or a wide-area wireless communication module such as 4G or 5G. The control unit 11 communicates with information terminals T such as smartphones held by maintenance personnel of power transmission and distribution equipment H, or with external servers G (LLM servers) via the communication unit 13, for example, through an external network such as the Internet or a LAN.

[0037] Figure 3 is a functional block diagram illustrating the functional units included in the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 functions as an acquisition unit 111, a reference information derivation unit 112, an extended prompt generation unit 113, an answer acquisition unit 114, and an output unit 115 by executing a program P stored in the storage unit 12.

[0038] The acquisition unit 111 acquires queries entered via an information terminal T used by maintenance personnel of the power transmission and distribution equipment H from the information terminal T. The queries may, for example, constitute part of a prompt written in colloquial language, or they may be in the form of a prompt that also includes prerequisites for the query. The prompts are treated as input data to a Large-Scale Language Model (LLM). The queries may request answers that include analysis and considerations regarding events (occurring events) that have occurred in the power transmission and distribution equipment H, such as, "Power transmission and distribution equipment H is a transformer, the target of analysis is the copper wire terminal portion, the matter to be clarified is the cause of damage to the copper wire terminal, please search the technical information database based on these matters, and output the details of the analysis method, past analysis results, and considerations, taking into account the search results." Alternatively, the acquisition unit 111 may acquire queries entered from a query input screen. The query input screen will be described later. The acquisition unit 111 generates a prompt that substantially includes the content of the query by adding the command, context, and output, which in this embodiment are elements that constitute a standard phrase, to the acquired query.

[0039] In addition to queries composed of text data, the acquisition unit 111 may also acquire image data (query image data) of the power transmission and distribution equipment H to be analyzed from the information terminal T. This query image data is captured, for example, by an imaging device C such as a CMOS camera built into the information terminal T, and includes the part or component in the power transmission and distribution equipment H where the event occurred. The acquisition unit 111 stores the prompt generated based on the query acquired from the information terminal T, or the prompt and query image data, in the storage unit 12, and outputs it to the reference information derivation unit 112, the extended prompt generation unit 113, and other functional units responsible for subsequent processes.

[0040] The reference information extraction unit 112 performs a search process on the technical information database based on the prompt obtained from the acquisition unit 111, and extracts technical information related to the prompt from the technical information stored in the technical information database as reference information. The technical information database is stored in the storage unit 12 of the information processing device 1. The technical information database stores past analysis results and considerations, such as incident reports for past failure incidents, maintenance and inspection reports showing the results of maintenance and inspection, and test reports showing the results of various tests conducted during the development phase of the power transmission and distribution equipment H. Furthermore, the technical information database stores operating manuals for various analytical instruments, such as optical microscopes, scanning electron microscopes (SEMs), X-ray CT scanners, X-ray diffractometers, Fourier transform infrared spectrophotometers (FT-IRs), and atomic absorption spectrometers, as analytical methods. Furthermore, the technical information database stores operating manuals, specifications, or standards for various parts used in the power transmission and distribution equipment H.

[0041] The technical information database is constructed, for example, by RAG (Retrieval-Augmented Generation) or a combination of RAG and CLIP (Contrastive Language-Image Pre-training), and includes a vector index in which the technical information is vectorized. In this case, text data among the technical information stored in the technical information database may be vectorized into sentence embedding vectors and stored in RAG. Image data among the technical information stored in the technical information database may be vectorized into image embedding vectors and stored in CLIP. In this case, if past reports, etc., contain both text data and image data, the text data extracted from the report may be stored in RAG, and the image data extracted from the report may be stored in CLIP. In this way, the technical information database can be constructed by a combination of RAG and CLIP to form a multimodal vector index (vector database). Alternatively, the technical information database may be constructed, for example, by a text-based full-text search database using morphological analysis, etc. Alternatively, the technical information database may be constructed by a vector database such as RAG and a full-text search database.

[0042] The reference information derivation unit 112 vectorizes the prompt from the acquisition unit 111 and uses the vectorized prompt to perform a search on the technical information database. Alternatively, if image data (query image data) is attached to the prompt, the query image data may also be vectorized. The reference information derivation unit 112 performs a search on the technical information database (vector database) composed of RAG and CLIP, for example, using cosine similarity, Euclidean distance, or dot product similarity measure, to derive technical information (text data and image data) that has a similarity of a predetermined level or higher to the prompt (query image data), and outputs it to the extended prompt generation unit 113.

[0043] The reference information derivation unit 112 may, when searching the technical information database using query image data, derive attribute information of the power transmission and distribution equipment H contained in the query image data as text data, and then search the technical information database using a sentence embedding vector obtained by vectorizing the attribute information (text data). That is, the information processing device 1 has an attribute information model implemented that outputs attribute information of the power transmission and distribution equipment H contained in the image data when image data is input, and the reference information derivation unit 112 obtains attribute information of the power transmission and distribution equipment H by inputting query image data into the attribute information model. This attribute information includes, for example, the type, name, classification, model, product specifications, or part name of the part where the event occurred.

[0044] The extended prompt generation unit 113 generates an extended prompt from the acquisition unit 111 using reference information from the reference information derivation unit 112. That is, the extended prompt includes both the prompt and the reference information. The extended prompt generation unit 113 may, for example, generate an extended prompt that instructs the system to respond to a prompt containing query image data by taking into account or considering the reference information, which includes reference image data. The extended prompt may also include a statement that it defines the maintenance personnel for the power transmission and distribution equipment H as an item indicating its own role. Alternatively, the role may be included in the prompt and inherited by the extended prompt. The extended prompt generation unit 113 outputs the generated extended prompt to the response acquisition unit 114.

[0045] The response acquisition unit 114 inputs the extended prompt from the extended prompt generation unit 113 into a language model configured, for example, a large-scale language model (LLM), thereby generating a response that reflects the reference information, which is the search result in the technical information database. The large-scale language model (LLM) is implemented, for example, in an external server G (LLM server), which is a different device that becomes the information processing device 1.

[0046] The language model (Large-Scale Language Model: LLM) implemented on the external server G (LLM server) generates and outputs a response to a query from the maintenance personnel of the power transmission and distribution equipment H in response to an extended prompt, that is, in response to the reference information (reference image data) included in or accompanying the extended prompt. In this process, the response will take into account or consider reference information, including reference image data which is the search result in the technical information database, and therefore the response will include matters related to the reference image data. The response acquisition unit 114 acquires the response from the external server G (LLM server).

[0047] In this embodiment, the response acquisition unit 114 uses a large-scale language model (LLM) implemented on an external server G (LLM server) as the language model, but is not limited to this. The language model (large-scale language model: LLM) may be implemented on the information processing device 1. In this case, the control unit 11 of the information processing device 1 may generate a large-scale language model (private LLM) by training a neural network composed of transformers, etc., with a corpus generated using a large amount of document data acquired from, for example, the Internet. The control unit 11 of the information processing device 1 may perform instruction tuning or fine tuning on the private LLM thus generated using various technical information related to the power transmission and distribution equipment H. Alternatively, the language model (large-scale language model: LLM) may be implemented on an information terminal T used by maintenance personnel of the power transmission and distribution equipment H.

[0048] The output unit 115 outputs (transmits) the answers obtained by the answer acquisition unit 114 to an information terminal T, such as a smartphone, used by the maintenance personnel of the power transmission and distribution equipment H, via the communication unit 13. By obtaining the answers to their questions, the personnel can recognize analysis methods appropriate to the subject of analysis and past cases related to the causes of events that occurred at the power transmission and distribution equipment H. Since the answers include reference information (reference image data) which are search results from the technical information database, it can provide useful information for the personnel when performing analysis work on the events that occurred.

[0049] The output unit 115 may store the outputted response in the technical information database. By adding the responses generated for each individual event to the technical information database in a recursive or regression manner, the amount of technical information stored in the database can be efficiently increased.

[0050] Figure 4 is a flowchart illustrating the processing procedure performed by the control unit 11 of the information processing device 1. The control unit 11 of the information processing device 1 receives operator input, for example, from a keyboard connected to the input / output I / F 14, and performs the following processing based on the received operation.

[0051] The control unit 11 of the information processing device 1 acquires a query (S101). The control unit 11 of the information processing device 1 acquires a query regarding an event occurring in the power transmission and distribution equipment H from, for example, an information terminal T such as a smartphone used by a maintenance worker of the power transmission and distribution equipment H. In this case, the query may be accompanied by an image (query image data) of the part (observation point) that is the subject of analysis of the event. The control unit 11 of the information processing device 1 stores the acquired query (query image data) in the storage unit 12. The control unit 11 of the information processing device 1 may also acquire a query entered on an input screen displayed on the information terminal T, for example.

[0052] Figure 5 is an explanatory diagram illustrating an input screen for entering a query. The input screen includes, for example, a query description area and a query image attachment area. In the query description area, information regarding the equipment type and the subject of analysis is entered, such as, "Power transmission and distribution equipment H is a transformer, the subject of analysis is the copper wire terminal portion, and the matter to be clarified is the cause of damage to the copper wire terminal." In the query image attachment area, an image related to the query entered in the query description area (query image) is entered. Thus, the input screen may be configured so that only information relating to the input (input data) elements in the prompt is entered.

[0053] The control unit 11 of the information processing device 1 generates a prompt based on the acquired query (S102). The control unit 11 of the information processing device 1 generates a prompt by adding or appending elements (commands, context, outputs) that are positioned as standard phrases, based on the query and query image entered via the input screen displayed on the information terminal T. In other words, the prompt substantially includes the query (query image).

[0054] Figure 6 is an explanatory diagram illustrating a prompt generated based on a query. In this embodiment, a prompt consists of four elements: command, context, input (input data), and output (output instruction). The command element may state, for example, "For the event {#input}, output a response in the format of {#output}." The context element may state, for example, "You are a skilled maintenance worker or analysis expert regarding power transmission and distribution equipment. For the event in {#input} below, you are required to output analysis methods and causes for an event occurring in power transmission and distribution equipment by searching and referring to the technical information database." The input element may state, for example, "Power transmission and distribution equipment H is a transformer. The object of analysis is the copper wire terminal portion. The matter to be clarified is the cause of damage to the copper wire terminal. Images of the damaged copper wire terminal portion {xxxx01.png,xxx02.png} are attached." Regarding the output elements, for example, it might state, "Please output details of the analysis method, past analysis results regarding the causes, and discussion."

[0055] The control unit 11 of the information processing device 1 derives reference information based on the generated prompt (S103). The control unit 11 of the information processing device 1 performs a search on a technical information database, for example, composed of RAG or CLIP, based on the generated prompt (query image data), thereby deriving technical information related to the prompt or similar to the event that is the target of the prompt as reference information (reference image data). This technical information includes, for example, analytical methods according to the object of analysis, such as the operation method of a scanning electron microscope, and past cases related to the causes of the event, such as past analysis results or reports.

[0056] The control unit 11 of the information processing device 1 generates an extended prompt based on the derived reference information (S104). The control unit 11 of the information processing device 1 generates an extended prompt by adding or considering the derived reference information to the generated prompt.

[0057] The control unit 11 of the information processing device 1 obtains a response to an extended prompt (S105). The control unit 11 of the information processing device 1 inputs an extended prompt, which includes reference information (reference image data) that is a search result from a technical information database, to a language model (large-scale language model: LLM) implemented on an external server G, for example. The language model generates and outputs a response that incorporates the reference information (reference image data) in response to the extended prompt. The control unit 11 of the information processing device 1 obtains this response from the language model (large-scale language model: LLM) implemented on the external server G.

[0058] Figure 7 is an explanatory diagram illustrating the response from the language model. The response includes the analysis method, past analysis results, and consideration of the cause of the incident. In the illustration of this embodiment, the incident that occurred at the power transmission and distribution equipment H (incident) is, for example, damage to the copper wire terminal portion of the transformer. In this case, chemical analysis is performed to investigate the cause of the damage and to consider measures to prevent recurrence. The analysis method includes matters related to the output of the analysis method manual. Past analysis results include matters related to relevant past reports. URL links may be set for such reports. Such URL links may be set to the report file, or to relevant pages, chapters, or paragraphs within the file. Furthermore, past analysis results may include relevant images (reference image data). The cause of the incident includes a summary or details of the estimated cause of the incident based on matters extracted from relevant past reports.

[0059] The analysis method manual output includes prerequisite information such as, "To analyze the fracture surface of the copper wire and estimate the cause of failure, the following analytical instruments are generally used," depending on the location of the incident (the copper wire terminal portion of the transformer). Furthermore, it includes specific details of the analytical instruments, such as, "Scanning electron microscope (SEM): SEM observes the microstructure of the fracture surface at high resolution... / Optical microscope: Optical microscope observes the macrostructure of the fracture surface..." Past similar results include reference information (past analysis results, reports, reference image data) found in the technical information database, along with images showing the relevant reference image data (Reference Images 1, 2, 3), and a summary and discussion of the causes derived from the past analysis results (XXX report).

[0060] The control unit 11 of the information processing device 1 outputs the response obtained from the language model (Large-Scale Language Model: LLM) and registers it in the technical information database (S106). The control unit 11 of the information processing device 1 outputs the obtained response to the information terminal T of the maintenance worker. The information terminal T, having received the response from the information processing device 1, displays the response on the display of the information terminal T. Furthermore, the control unit 11 of the information processing device 1 may also register the obtained response in the technical information database. In this case, the control unit 11 of the information processing device 1 may also register the query in association with the response in the technical information database. Furthermore, the control unit 11 of the information processing device 1 may obtain the maintenance worker's evaluation of the response from the language model (Large-Scale Language Model: LLM) from the information terminal T used by the maintenance worker, and register the evaluation in association with the response in the technical information database. Furthermore, the control unit 11 of the information processing device 1 may obtain, for example, the final report completed by the maintenance worker using the response from the information terminal T used by the maintenance worker, and register the final report in association with the response in the technical information database.

[0061] In this embodiment, the information processing device 1 searches the technical information database and generates an extended prompt based on the acquired query, but this is not limited to this. The external server G itself may search the technical information database and generate an extended prompt based on the query sent from the information processing device 1. In other words, the division of roles between the information processing device 1 and the external server G is not limited to what is described above, and the series of processes in the flowchart described above may be carried out through the cooperation and division of processing between the information processing device 1 and the external server G.

[0062] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended.

[0063] Regarding the multiple claims described in the patent claims, they can be combined with each other regardless of the form of citation. The patent claims include multiple dependent claims that depend on multiple claims. The patent claims do not include multiple dependent claims that depend on multiple dependent claims, but multiple dependent claims that depend on multiple dependent claims may be included. [Explanation of Symbols]

[0064] S Power transmission and distribution equipment analysis support system, H Power transmission and distribution equipment, T Information terminal, C Imaging device, G External server (LLM server), 1 Information processing device, 11 Control unit, 111 Acquisition unit, 112 Reference information derivation unit, 113 Extended prompt generation unit, 114 Answer acquisition unit, 115 Output unit, 12 Storage unit, P Program (program product), M Recording medium, 13 Communication unit, 14 Input / output IF

Claims

1. Retrieve queries related to power transmission and distribution equipment. Using the acquired query and the technical information in the technical information database that stores technical information related to power transmission and distribution equipment, a language model is used to generate a response to the query. Output the generated answer. A program that instructs a computer to perform a process.

2. The image data of the aforementioned power transmission and distribution equipment and the query including the target of analysis of the aforementioned power transmission and distribution equipment are obtained. The acquired image data and reference information related to the subject of analysis are extracted from the technical information database. The extracted reference information is provided to the language model to generate the answer. The program according to claim 1.

3. Using image data of the aforementioned power transmission and distribution equipment, a query including the target of analysis of the aforementioned power transmission and distribution equipment, a command to extract reference information related to the query by referring to the technical information database, and a prompt including a command to output the analysis method and past analysis results for the target of analysis, The language model outputs the response, which includes the analysis method and the past analysis results. The program according to claim 1.

4. Based on the extracted reference information and the prompt, an extended prompt is generated by extending the prompt. By inputting the generated extended prompt into the language model, the response is obtained from the language model. The program according to claim 3.

5. The technical information stored in the aforementioned technical information database includes image data, The aforementioned query extracts image data by searching the aforementioned technical information database. Extract the aforementioned reference information, including the extracted image data. The program according to claim 2.

6. The technical information stored in the aforementioned technical information database includes analytical methods corresponding to the subject of analysis, and past cases related to the subject of analysis. The response from the language model is generated using the analysis method and past examples extracted from the technical information database based on the query. The program according to claim 3.

7. The aforementioned analytical method includes the matters described in the operating manual for the analytical instrument, The aforementioned past cases include matters described in reports on past incidents in power transmission and distribution facilities. The program according to claim 6.

8. The response obtained from the language model is registered in the technical information database. The program according to claim 1.

9. The aforementioned technical information database is constructed using RAG (Retrieval-Augmented Generation) and includes a vector index in which the technical information is vectorized. The program according to any one of claims 1 to 7.

10. On the computer, Retrieve queries related to power transmission and distribution equipment. Using the acquired query and the technical information in the technical information database that stores technical information related to power transmission and distribution equipment, a language model is used to generate a response to the query. Output the generated answer. An information processing method that involves having a computer perform a task.

11. An acquisition unit that obtains queries related to power transmission and distribution equipment, A generation unit generates a response to the query using a language model that utilizes the acquired query and technical information from a technical information database that stores technical information related to power transmission and distribution equipment. An output unit that outputs the generated answer and An information processing device equipped with the following features.

Images