Data registration device, data search device, program, and data registration and search system

Abstract

A data registration device for registering a retrieval unit of data in a database. The data registration device includes: a division unit for dividing a document into retrieval units of data; a summary acquisition unit for acquiring a first summary text of a first body text included in the retrieval units of data by using a first machine learning model; a synthesis unit for generating first synthetic data of the first summary text of the document and second synthetic data of the first body text and the first synthetic data; a first feature vector acquisition unit for acquiring a feature vector of the second synthetic data by using a second machine learning model; and a registration unit for registering, in a database, a retrieval unit of data including the second synthetic data and the feature vector of the second synthetic data.

Description

[Technical Field]

[0001] The present disclosure relates to a data registration device, a data search device, a program, and a data registration and search system. [Background technology]

[0002] In recent years, chatbot (question answering) and search technologies based on large-scale language models have been advancing, as exemplified by ChatGPT. Conventionally, chatbot and search technologies based on large-scale language models have used large-scale language models to generate feature vectors and use them for searches.

[0003] Patent Document 1 discloses a technique for document vectors that indicate the topic of a document (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-99774 Summary of the Invention [Problem to be solved by the invention]

[0005] Large-scale language models may have limitations on the length of strings that can be input. Documents, such as electronic laboratory notebooks, that are intended for use with chatbots and search technologies based on large-scale language models may contain a large number of characters, and may exceed the length of strings that can be input to large-scale language models.

[0006] Documents that exceed the string length that can be input to a large-scale language model can be split into multiple pieces of data to make the string length suitable for input to a large-scale language model. However, the split data may lack the content (topics) of the entire document.

[0007] Conventional chatbot and search technologies based on large-scale language models have had the problem that if the content of an entire document is missing from the segmented data, the user may not be able to obtain the answers or search results they expect.

[0008] An object of the present disclosure is to provide a data registration device, a data search device, a program, and a data registration and search system that can further improve the accuracy of answers or searches. [Means for solving the problem]

[0009] The present disclosure has the following configuration.

[0010] [1] A data registration device that registers search unit data in a database, a division unit that divides a document into data of the search unit; a summary acquisition unit that acquires a first summary text of a first main text included in the search unit data using a first machine learning model; a synthesis unit for generating first synthesis data of the first summary text of the document and second synthesis data of the first body text and the first synthesis data; a first feature vector acquisition unit that acquires a feature vector of the second combined data using a second machine learning model; a registration unit that registers the second combined data and the search unit data including a feature vector of the second combined data in the database; A data registration device having:

[0011] [2] a file acquisition unit that acquires a file associated with the search unit data; and the summary acquisition unit acquires a second summary text of a second body text included in the file using the first machine learning model; the synthesis unit generates third synthesis data of the first body text and the second summary text of the document; the first feature vector acquisition unit acquires a feature vector of the third combined data using the second machine learning model; The registration unit registers the third combined data and the search unit data including a feature vector of the third combined data in the database. [1] The data registration device according to the present invention.

[0012] [3] The registration unit further registers meta information of the document. [1] or [2]. A data registration device.

[0013] [4] The dividing unit divides the document into one or more sections into the search unit data. A data registration device according to any one of [1] to [3].

[0014] [5] The dividing unit divides the document into the search unit data so that the character string length does not exceed a character string length that can be input to the second machine learning model. A data registration device according to any one of [1] to [4].

[0015] [6] The document is an electronic lab notebook. A data registration device according to any one of [1] to [5].

[0016] [7] A data search device for searching for search unit data registered in a database by the data registration device described in [2], an input text acquisition unit that acquires input text to be used for search; a second feature vector acquisition unit that acquires a feature vector of the input text using the second machine learning model; a search unit that uses the feature vector of the input text to search for data in the search units registered in the database that has a similar feature vector to the second composite data or the third composite data; A data search device having:

[0017] [8] an answer output unit that outputs an answer sentence generated by a third machine learning model based on the data of the search unit searched by the search unit; The data search device according to [7], further comprising:

[0018] [9] The search unit searches for data in the search unit that includes a feature vector of the second synthetic data or the third synthetic data that is similar to the feature vector of the input text, according to a vector similarity between the feature vector of the input text and the feature vector of the second synthetic data or the third synthetic data included in the data in the search unit that is registered in the database. [8] The data search device according to [8].

[0019]

[10] a chatbot function unit that receives the input text from a user and displays the answer sentence; The data search device according to [8], further comprising:

[0020]

[11] The third machine learning model is a trained language model that generates the answer sentence that matches the input text based on the data of the search unit searched by the search unit.

[10] The data search device according to claim 1.

[0021]

[12] A data registration device that registers search unit data in a database, a dividing step of dividing the document into the search unit data; a summary obtaining step of obtaining a first summary text of a first main text included in the data of the search unit using a first machine learning model; generating first synthesized data of the first summary text of the document and second synthesized data of the first body text and the first synthesized data; a first feature vector acquisition step of acquiring a feature vector of the second synthetic data using a second machine learning model; a registration step of registering the second composite data and the search unit data including the feature vector of the second composite data in the database; A program that executes the following.

[0022]

[13] A data registration and search system that registers data in search units in a database and searches for the data in search units registered in the database, a division unit that divides a document into data of the search unit; a summary acquisition unit that acquires a first summary text of a first main text included in the search unit data using a first machine learning model; a synthesis unit for generating first synthesis data of the first summary text of the document and second synthesis data of the first body text and the first synthesis data; a first feature vector acquisition unit that acquires a feature vector of the second combined data using a second machine learning model; a registration unit that registers the second combined data and the search unit data including a feature vector of the second combined data in the database; an input text acquisition unit that acquires input text to be used for search; a second feature vector acquisition unit that acquires a feature vector of the input text using the second machine learning model; a search unit that uses the feature vector of the input text to search for data in the search units registered in the database that has a feature vector similar to that of the second composite data; A data registration and search system having the above. [Effects of the Invention]

[0023] According to the present disclosure, the accuracy of answers or searches can be further improved. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 3] FIG. 4 is an explanatory diagram of an example of processing of the data registration device according to the embodiment; [Figure 4] FIG. 1 is an image diagram of an example of an electronic experiment notebook. [Figure 5] FIG. 10 is an explanatory diagram of an example of summary text that reflects the topic of the entire note. [Figure 6] FIG. 10 is an illustration of an example of a prompt to input into a large scale language that has been trained. [Figure 7] FIG. 4 is an explanatory diagram of an example of processing of the data registration device according to the embodiment; [Figure 8] FIG. 10 is an explanatory diagram of an example of a file attachment operation to an electronic experiment notebook. [Figure 9] 1 is a functional configuration diagram of an example of a data registration device according to an embodiment of the present invention; [Figure 10] FIG. 10 is a diagram illustrating an example of data structure in units of search registered in a database. [Figure 11] 1 is a functional configuration diagram of an example of a data search device according to an embodiment of the present invention; [Figure 12] 10 is a flowchart illustrating an example of processing performed by the data search device according to the present embodiment. [Figure 13] This is an example of a UI displayed on a user terminal. [Figure 14] FIG. 10 is an illustration of an example of a prompt to input into a large scale language that has been trained. DETAILED DESCRIPTION OF THE INVENTION

[0025] Next, an embodiment of the present invention will be described in detail, but the present invention is not limited to the following embodiment.

[0026] <System configuration> Fig. 1 is a configuration diagram of an example of an information processing system 1 according to this embodiment. The information processing system 1 shown in Fig. 1 processes an electronic lab notebook as an example of a document. An electronic lab notebook, which is an example of a document, is digital data in which a large amount of data and knowledge is recorded as information, such as research and development results, internal and external technical information, internal procedures, and various know-how.

[0027] The amount of characters per electronic lab notebook varies depending on the creator or topic. In an electronic lab notebook, information is recorded in a specific unit, such as a section, within a single notebook. In this embodiment, an example is described in which information is recorded in a specific unit, such as a section, within a single notebook. Files can also be attached to an electronic lab notebook.

[0028] The information processing system 1 comprises an electronic experiment notebook storage device 10, a data registration device 12, a data search device 14, a database 16, and one or more user terminals 18, all of which are connected to each other so as to enable data communication via a network 20. The network 20 is, for example, a local area network (LAN) or the Internet.

[0029] The electronic experiment notebook storage device 10 is an example of an information processing device that stores electronic experiment notebooks and allows users who operate user terminals 18 to share the electronic experiment notebooks.

[0030] The data registration device 12 is an example of an information processing device that processes the electronic experiment notebooks stored in the electronic experiment notebook storage device 10 into search unit data as described below, and then registers the data in the database 16.

[0031] The data search device 14 is an example of an information processing device that searches for data in search units registered in the database 16 by the data registration device 12, as described below. The data search device 14 may provide a chatbot function to a user operating the user terminal 18.

[0032] The chatbot function accepts input text of a question from a user operating the user terminal 18. The chatbot function may display an answer sentence generated based on search unit data searched from the database 16 as described below on the user terminal 18 as a response to the question from the user.

[0033] The database 16 is an example of an information processing device to which data in search units is registered from the data registration device 12. The database 16 accepts a search for data in search units from the data search device 14 and returns search unit data as a result of the search.

[0034] The user terminal 18 is an example of an information processing device operated by a user, such as a PC, tablet terminal, or smartphone. The user can operate the user terminal 18 and use the chatbot function provided by the data search device 14. By using the chatbot function provided by the data search device 14 to search for data of a search unit registered in the database 16, the user can easily find the electronic experiment notebook containing the information they wish to view from the electronic experiment notebook storage device 10.

[0035] The data registration device 12 and the data search device 14 may be realized as a data registration and search system 22. The data registration and search system 22 may be configured by integrating the data registration device 12 and the data search device 14 shown in Fig. 1. The data registration and search system 22 shown in Fig. 1 may be provided as a cloud service.

[0036] Furthermore, the database 16 may be configured to be included in the data registration device 12, the data search device 14, or the data registration and search system 22. The data registration device 12, the data search device 14, and the data registration and search system 22 may have a trained large-scale language model, or may use a trained large-scale language model that another device has. The trained large-scale language model is a natural language processing model trained using a large amount of text data. A fine-tuned trained model may be used as the trained large-scale language model. For example, BERT or GPT can be used as the large-scale language model.

[0037] The electronic experiment notebook storage device 10, data registration device 12, data search device 14, database 16, and data registration and search system 22 shown in FIG. 1 can be realized by a workstation or a PC (Personal Computer) or the like.

[0038] 1 is an example. The configuration of the information processing system 1 varies depending on the application and purpose. For example, the electronic experiment notebook storage device 10, the data registration device 12, the data search device 14, the database 16, and the data registration and search system 22 may be realized by multiple computers or as a cloud service.

[0039] <Hardware configuration> For example, the electronic experiment notebook storage device 10, data registration device 12, data search device 14, database 16, one or more user terminals 18, and data registration and search system 22 shown in FIG. 1 can be realized by a computer 500 having the hardware configuration shown in FIG.

[0040] 2 is a diagram showing an example of the hardware configuration of a computer 500 according to this embodiment. The computer 500 includes, for example, an input device 501, an output device 502, an external I / F 503, a RAM (Random Access Memory) 504, a ROM (Read Only Memory) 505, a CPU (Central Processing Unit) 506, a communication I / F 507, and an auxiliary storage device 508, all of which are interconnected by a bus B. The input device 501 and the output device 502 may be connected to each other for use.

[0041] The input device 501 is a device that accepts user operations, and is, for example, a touch panel, operation keys, buttons, a keyboard, or a mouse. The output device 502 has a device that displays a screen and a device that outputs sound. The device that displays a screen is, for example, a liquid crystal display. The device that outputs sound is, for example, a speaker. The communication I / F 507 is an interface that allows the computer 500 to perform data communication.

[0042] The auxiliary storage device 508 is an example of a non-volatile storage device that stores programs and data. The auxiliary storage device 508 is, for example, a hard disk drive (HDD) or a solid state drive (SSD). The programs are, for example, an operating system (OS), which is basic software, and applications that provide various functions on the OS.

[0043] The external I / F 503 is an interface with an external device. The external device is a recording medium 503a or the like. The computer 500 can read programs and data from the recording medium 503a via the external I / F 503. The recording medium 503a is, for example, a flexible disk, a CD, a DVD, an SD memory card, or a USB memory.

[0044] The ROM 505 stores the BIOS, OS settings, network settings, etc., which are executed when the computer 500 starts up. The RAM 504 is a working memory for temporarily storing programs and data. The CPU 506 reads programs and data from the ROM 505 or the auxiliary storage device 508 into the RAM 504 and executes the processes, thereby realizing various functions described below.

[0045] <Processing Overview> 3 is an explanatory diagram of an example of processing by the data registration device 12 according to this embodiment. The data registration device 12 reads out the electronic experiment notebook 1000 stored in the electronic experiment notebook storage device 10.

[0046] 4 is a diagram illustrating an example of an electronic lab notebook 1000. The electronic lab notebook 1000 includes meta information and one or more sections. The meta information includes, for example, the title of the electronic lab notebook 1000, the creator, the creation date and time, the update date and time, the disclosure scope, tags, and the like. The disclosure scope may be, for example, a department or a project.

[0047] A section corresponds to, for example, a chapter or section of a paper or report. A section may include, for example, a title, text, images, tables, and compound information. Various application files can also be attached to a section. The main text may include text extracted from a section, text extracted from an image using OCR (Optical Character Recognition / Reader), and text extracted from a table.

[0048] The main text may also be text that has been written by interpreting non-verbal information such as images. The main text may not only encompass the text contained in table cells, but may also be text that has been written by interpreting a table with reference to row and column headings, etc. If a file is attached, the main text may also be text extracted from the file. A file attached to a section is an example of a file associated with data in a search unit.

[0049] In this embodiment, the main text is acquired in units of sections. The reason for acquiring the main text in units of sections is as follows.

[0050] One topic is described in one electronic lab notebook 1000. If the registration unit of the database 16 is the electronic lab notebook 1000, the information in one record of the database 16 will be data that reflects the topic of the entire electronic lab notebook 1000, and there will be no problem. Also, if the search method is a general full-text search (string matching), there will be no problem because the entire electronic lab notebook 1000 can be registered in the database 16 and then searched.

[0051] In this embodiment, vector similarity is used as a search method. By performing a search using vectors, it is possible to obtain semantically similar data even if the search query and the text to be searched do not perfectly match. Feature vectorization of text is performed using a trained large-scale language model. The trained large-scale language model used for feature vectorization of text may have a limit on the length of strings that can be input. If the string length exceeds the length that can be input to the trained large-scale language model, the text to be feature vectorized is divided so that it does not exceed the input length, or the strings that exceed the input length are deleted. Note that mechanical division in units of a set number of characters (e.g., 5,000 characters) has drawbacks such as division in the middle of a sentence.

[0052] Therefore, it is preferable to divide the text to be converted into feature vectors at topic transition points rather than mechanically. The electronic laboratory notebook 1000 is written by dividing it into sections. Each section is often constructed by dividing it at topic transition points that further subdivide a notebook, such as background, method, or results.

[0053] Therefore, in this embodiment, when dividing one note, the section boundaries are used to divide the note while taking advantage of the transition points of the topic.

[0054] There is another reason for acquiring the main text in sections: One electronic laboratory notebook 1000 is written about one topic (the topic of the entire notebook), and each section describes a subtopic that details the topic of the entire notebook.

[0055] In a search scenario, there are times when you want to search for granular information, such as "How to generate XX." In a database where the entire note is registered as a single piece of data, the user must read the entire searched note to find the part that says "How to generate XX."

[0056] Furthermore, when using a chatbot function based on a large-scale language model, answers to users who ask questions are generated using a trained large-scale language model trained using technology such as RAG (Retrieval Augmented Generation). It is desirable that the data input to the trained large-scale language model contain as little information as possible other than the "method for generating XX," such as background and results. This is because if the input to the trained large-scale language model contains a lot of unintended information, the answer generated by the trained large-scale language model may deviate from the user's intent. Furthermore, if the input to the trained large-scale language model contains a lot of unintended information, it takes longer for the trained large-scale language model to generate an answer.

[0057] Thus, in this embodiment, in order to obtain highly accurate answers that reflect the user's search intent or question intent, it is desirable to subdivide the entire notebook into one or more sections and register them in the database 16. Note that the term "section" is just an example, and any other term may be used as long as it is a specific unit that can divide the entire notebook into small topics.

[0058] Returning to Figure 3, the electronic lab notebook 1000 is divided into search units of data, with one section 1004-1 or multiple consecutive sections 1004-2 to 1004-3 as one unit. In Figure 3, one section 1004-1 with a large number of text characters and two consecutive sections 1004-2 to 1004-3 with a small number of text characters are divided into search units of data.

[0059] The main texts 1006-1 and 1006-2 are obtained from the segmented search unit data. The main texts 1006-1 and 1006-2 are examples of first main texts. The summary text 1008 is generated from the main texts 1006-1 and 1006-2 using a trained large-scale language model (an example of a first machine learning model).

[0060] It is desirable to set an upper limit on the number of characters in the summary text 1008 generated from the main texts 1006-1 and 1006-2, as shown in the following formula (1).

[0061]

number

[0062]

number

[0063] Furthermore, the summary text 1008 may be generated by reflecting the topic of the entire note, as shown in Figure 5. Figure 5 is an explanatory diagram of an example of summary text that reflects the topic of the entire note.

[0064] 5 shows an example in which key phrases obtained from the note title and the character string of the entire note are used as the topic of the entire note, and are used as auxiliary information for instructing the generation of summary text 1008. Existing algorithms such as RAKE or TopicRank can be used to extract key phrases.

[0065] The trained large-scale language model may be instructed to generate a summary text by inputting the prompt shown in Fig. 6 into the trained large-scale language. Fig. 6 is an explanatory diagram of an example of a prompt to be input into the trained large-scale language. The prompt is a statement that specifies conditions for the trained large-scale language model to generate a summary text.

[0066] In this manner, in this embodiment, a summary text 1008 may be generated that is in line with the topic of the entire note.

[0067] Summary text 1010 of the entire notebook is an example of first synthesized data that combines all summary texts 1008 generated from one electronic experiment notebook 1000. Summary text 1010 of the entire notebook is text that outputs key points that clarify the topic and flow of the entire notebook. Summary text 1010 of the entire notebook may be in prose or may be in bullet points.

[0068] Furthermore, the summary text 1010 of the entire note is converted into a feature vector using a large-scale language that has already been learned, so the approximate number of characters is expressed by equation (3).

[0069]

number

[0070] The database 16 stores meta information 1002, summary text of the entire note plus main text 1012-1, and search unit data 1014-1 including the feature vector of summary text of the entire note plus main text 1012-1.

[0071] The database 16 also stores meta information 1002, summary text of the entire note plus main text 1012-2, and search unit data 1014-2 including the feature vector of summary text of the entire note plus main text 1012-2.

[0072] The feature vector of the summary text + main text 1012-1 of the entire note is generated from the summary text + main text 1012-1 of the entire note by using a trained large-scale language model (an example of a second machine learning model).

[0073] In addition, the feature vector of the summary text + main text 1012-2 of the entire note is generated from the summary text + main text 1012-2 of the entire note by using a trained large-scale language model (an example of a second machine learning model).

[0074] The feature vector of the summary text of the entire note + main text 1012-1 and the feature vector of the summary text of the entire note + main text 1012-2 are used by the data search device 14 when searching the database 16.

[0075] In the database 16 shown in FIG. 3, the summary text 1010 of the entire note is reflected in the feature vector, and therefore the accuracy of searches of the search unit data 1014-1 and 1014-2 is improved.

[0076] For example, when the main texts 1006-1 and 1006-2 are obtained from divided data, the entire contents of the electronic lab notebook 1000 may be lost. Also, when the main texts 1006-1 and 1006-2 are obtained from divided data, the flow (context) continuing from the previous and next sections of the electronic lab notebook 1000 may be lost.

[0077] If the entire contents of the electronic experiment notebook 1000 are missing, or if the flow (context) from the previous and next sections is lost, data may be retrieved for a search unit that does not match the intention (context) at the time of the search and the entire contents of the electronic experiment notebook 1000.

[0078] In this embodiment, summary text 1008 of main text 1006-1 and 1006-2 obtained from the divided search unit data is synthesized to generate summary text 1010 of the entire notebook. Therefore, in this embodiment, the content of the entire electronic lab notebook 1000 or the surrounding context is reflected in the feature vector, improving the accuracy of searching the search unit data 1014-1 and 1014-2.

[0079] In this embodiment, a file attached to an electronic experiment notebook 1000 is processed, for example, as shown in Figure 7. Figure 7 is an explanatory diagram of an example of processing by the data registration device 12 according to this embodiment. The data registration device 12 reads out the electronic experiment notebook 1000 stored in the electronic experiment notebook storage device 10.

[0080] 7 includes meta information 1002, section 1004-4, and section 1004-5. Section 1004-5 has file 1020 attached. File 1020 is an example of a file associated with search unit data. File 1020 may be any of various application files, or a linked file containing linked information.

[0081] The electronic lab notebook 1000 is divided into search units of data, with sections 1004-4 and 1004-5 as one unit. The main text 1006-3 is obtained from the data of the divided section 1004-4. The main text 1006-4 is obtained from the data of the divided section 1004-5. The file 1020 is obtained from section 1004-5.

[0082] The main text 1018 of the entire notebook is a combination of the main texts 1006-3 to 1006-4 obtained from all sections 1004-4 to 1004-5 included in one electronic lab notebook 1000. Meta information 1022 is meta information for file 1020. The main text 1024 is an example of a second main text included in the file. The summary text 1026 is generated from the main text 1024 using a trained large-scale language model (an example of a first machine learning model). The summary text 1026 is an example of a second summary text. The summary text 1026 is generated from the entire main text 1024. If the number of characters in the main text 1024 exceeds the upper limit of the number of characters that can be input into the large-scale language model used to generate the summary, the main text 1024 may be divided into multiple parts so that the number of characters does not exceed the upper limit, and multiple summary texts may be generated.

[0083] The main text 1018 of the entire notebook may be the first composite data that combines all of the summary texts 1008 generated from one electronic laboratory notebook 1000. The main text of the entire notebook plus summary text 1028 is an example of the third composite data that combines the main text 1018 of the entire notebook and the summary text 1026.

[0084] The database 16 stores meta information 1002 of the note, the main text of the entire note plus summary text 1028, and search unit data 1014-3 including the feature vector of the main text of the entire note plus summary text 1028.

[0085] Furthermore, the database 16 stores meta information 1022 of the file, main text 1024 of the file, and search unit data 1014-4 including a feature vector of the main text 1024 of the file.

[0086] The feature vector of the summary text 1026 of the body text 1024 of the file attached to section 1004-5 is generated from the body text 1024 of the file using a trained large-scale language model (an example of a second machine learning model).

[0087] The feature vector of the summary text 1026 of the main text 1024 of the file is used by the data search device 14 when searching the database 16. In the database 16 shown in Fig. 7, the summary text 1026 of the main text 1024 of the file 1020 attached to the electronic experiment notebook 1000 is reflected in the feature vector, thereby improving the accuracy of searches of the search unit data 1014-3 and 1014-4.

[0088] The main text 1024 obtained from the file 1020 may be obtained from a portion that is in line with at least one of the contents of the entire electronic lab notebook 1000, the contents of sections 1004-4 to 1004-5, and the contents of the preceding and following sections. The main text 1024 obtained in this manner excludes descriptions that are unrelated to the entire contents of the electronic lab notebook 1000. The main text 1024 obtained from the file 1020 may also be translated into text that matches the language of the entire electronic lab notebook 1000 and output.

[0089] In this embodiment, the character string length (number of characters) of the entire note's main text plus summary text 1028 is reduced by generating summary text 1026 of the main text 1024 obtained from the attached file 1020. Furthermore, in this embodiment, the content of the attached file 1020 is reflected in the feature vector, improving the accuracy of searches of the search unit data 1014-3 and 1014-4.

[0090] The attached file 1020 may be handled as follows:

[0091] The file 1020 can be used as a single search unit. The main text, summary text, and feature vector of the entire note or section may include the main text and summary text of the file 1020. By supporting the attached file 1020 in this way, data processing becomes possible with the contents of the attached file 1020 independent of the contents of the note.

[0092] The attached file 1020 may be handled as follows:

[0093] The contents of file 1020 are incorporated into the search unit data as part of the information of the attached notes and sections. The contents of file 1020 do not constitute search unit data by themselves. The main text and summary text of file 1020 are generated according to the contents of the main text of the entire note or section. The data of file 1020 may be registered as information of one section as separate data (separate record) from the main text of the section.

[0094] The attached file 1020 may be handled as follows:

[0095] An attached file 1020 may be useful as additional information for a note or section, but there may be cases where only a portion of the file 1020 is desired to be used as reference information. A user who is a note creator attaches the file 1020 to an electronic lab notebook 1000, as shown in Figure 8. Figure 8 is an explanatory diagram of an example of the operation of attaching a file to an electronic lab notebook.

[0096] The user attaches a file 1020 in a file format such as PDF (Portable Document Format) to the electronic experiment notebook 1000 by dragging and dropping or the like.

[0097] The user selects the page they want to display from the thumbnail images of the pages in the attached file. The thumbnail image of the page selected by the user is displayed in the electronic experiment notebook 1000. The attached file itself and the selected page information are stored in the background of the electronic experiment notebook 1000.

[0098] The page of a file displayed as a thumbnail image in the electronic experiment notebook 1000 is information that is highly relevant to the contents of the corresponding note and section. In other words, the page of a file that is not displayed as a thumbnail image in the electronic experiment notebook 1000 is information that is less relevant to the corresponding note and section. Therefore, the contents of the page selected by the user may be used as additional information for the note and section.

[0099] In the case of FIG. 8, the character strings written on the pages "Page-2" and "Page-3" selected by the user, or the character strings converted into text using image2text (image semantic interpretation), become the main text of the file 1020. In the case of FIG. 8, summary text may be generated for the main text of the attached file 1020 by reflecting at least one of the topic of the entire note and the topic of the section. Furthermore, in the case of FIG. 8, summary text may be generated for the main text of the attached file 1020 by additionally reflecting the topic of the entire attached file 1020. In this case, summary text that reflects the contents of the entire note and the entire attached file 1020 can be obtained.

[0100] <Functional configuration> The functional configurations of the data registration device 12 and the data search device 14 according to this embodiment will be described.

[0101] 9 is a functional configuration diagram of an example of the data registration device 12 according to this embodiment. Note that in the configuration diagram of FIG. 9, parts that are not necessary for explaining this embodiment are omitted as appropriate.

[0102] The data registration device 12 shown in Figure 9 has a data acquisition unit 30, a division unit 32, a text acquisition unit 34, a summary acquisition unit 36, a synthesis unit 38, a first feature vector acquisition unit 40, a registration unit 42, a file acquisition unit 44, and a trained large-scale language model storage unit 46.

[0103] The data acquisition unit 30 acquires electronic experiment notebook data from the electronic experiment notebook storage device 10. The division unit 32 divides the electronic experiment notebook data, which is an example of a document, into search unit data. The file acquisition unit 44 acquires files attached to the data divided into search units as search unit data.

[0104] The data in search units may be data in registration units of the database 16. The data in search units do not have to be in electronic experiment notebook units. The dividing unit 32 divides the acquired electronic experiment notebook into multiple section groups, such as one or multiple section units.

[0105] The main text acquisition unit 34 acquires main text for each registration unit in the database 16 from the data of the electronic experiment notebook divided by the division unit 32 or from the file acquired by the file acquisition unit 44. The main text that the main text acquisition unit 34 acquires from the data of the electronic experiment notebook is mainly text. The main text acquisition unit 34 may acquire text extracted by OCR of the image data of the electronic experiment notebook, text extracted by speech recognition of attached audio data, text generated from image data using artificial intelligence (AI), and text generated from audio data using AI.

[0106] The summary acquisition unit 36 ​​uses a trained large-scale language model to acquire summary text of the main text of the registration unit in the database 16 acquired by the main text acquisition unit 34. The trained large-scale language model can use a generation AI such as GPT.

[0107] The synthesis unit 38 synthesizes the summary text (hereinafter referred to as the first summary text) of the main text (hereinafter referred to as the first main text) of the registration unit acquired by the summary acquisition unit 36 ​​into summary text of the entire notebook (hereinafter referred to as the first synthetic data).The synthesis unit 38 also synthesizes the first main text and the first synthetic data into second synthetic data.If there is a file attached to the electronic experiment notebook 1000, the synthesis unit 38 synthesizes the first main text and the summary text of the attached file into third synthetic data.

[0108] The first feature vector acquisition unit 40 acquires the feature vector of the second combined data using the trained large-scale language model. If there is a file attached to the electronic lab notebook 1000, the first feature vector acquisition unit 40 acquires the feature vector of the third combined data using the trained large-scale language model. The trained large-scale language model storage unit 46 stores the trained large-scale language model used by the first feature vector acquisition unit 40.

[0109] The search process of the data search device 14 uses the similarity between the feature vector of the second composite data or the feature vector of the third composite data acquired by the first feature vector acquisition unit 40. By using a large-scale multilingual trained language model, the search process of the data search device 14 can perform searches regardless of whether the query input is in Japanese or English, for example.

[0110] The registration unit 42 registers the search unit data including the second composite data and the feature vector of the second composite data in the database 16. If there is a file attached to the electronic lab notebook 1000, the registration unit 42 registers the search unit data including the third composite data and the feature vector of the third composite data in the database 16. The registration unit 42 may include meta information of the electronic lab notebook in the search unit data to be registered in the database 16. The meta information of the electronic lab notebook may include information such as a link destination URL (Uniform Resource Locator) that identifies the original electronic lab notebook stored in the electronic lab notebook storage device 10. The registration unit 42 may acquire information that may be lost due to feature vectorization, for example, by named entity extraction, and register the information in the database 16.

[0111] The data search device 14 may perform the process of registering data in units of search into the database 16 by batch processing, for example, or may perform the process at the timing instructed by an administrator or the like.

[0112] 10 is a diagram showing an example of the structure of search unit data to be registered in database 16. The structure of search unit data includes the following items: data ID, link URL, note title, creator name, creator affiliation, section title, creation date, update date, data category, keywords, main text, summary text, and feature vector.

[0113] In the item "Main text", for example, the first main text is registered. In the item "Summary text", for example, the first composite data is registered. In the item "Feature vector", for example, the feature vector of the second composite data or the feature vector of the third composite data is registered. In the item "Link destination URL", information that identifies the original electronic experiment notebook stored in the electronic experiment notebook storage device 10 is registered. Other items are registered using meta information, key phrase extraction results, etc.

[0114] 11 is a functional configuration diagram of an example of the data search device 14 according to this embodiment. Note that the configuration diagram in FIG. 11 appropriately omits parts that are not necessary for explaining this embodiment.

[0115] The data search device 14 shown in FIG. 11 includes an input text acquisition unit 50, a second feature vector acquisition unit 52, a search unit 54, an answer output unit 56, a chatbot function unit 58, and a trained large-scale language model storage unit 60.

[0116] The input text acquisition unit 50 acquires input text to be used for search. The input text acquisition unit 50 acquires input text (question sentence) that the chatbot function unit 58 accepts as input from a user. For example, in the case of repeating questions and answers like a chatbot, a series of questions and answers may be used as a query.

[0117] The second feature vector acquisition unit 52 uses a trained large-scale language model to acquire the feature vector of the input text acquired by the input text acquisition unit 50. The second feature vector acquisition unit 52 acquires the feature vector using a trained large-scale language model similar to that used by the first feature vector acquisition unit 40.

[0118] The search unit 54 uses the feature vector of the input text acquired by the second feature vector acquisition unit 52 to search for search unit data with similar feature vectors from the search unit data registered in the database 16, using cosine similarity or the like.

[0119] The answer output unit 56 outputs an answer sentence generated by a trained large-scale language model (an example of a third machine learning model) based on the data in search units searched by the search unit 54.

[0120] The chatbot function unit 58 receives input text from the user and displays and presents to the user the answer sentence output by the answer output unit 56. The trained large-scale language model storage unit 60 stores the trained large-scale language model used by the second feature vector acquisition unit 52 and the answer output unit 56.

[0121] The data search device 14 according to this embodiment uses the similarity between the feature vector of the data in the search unit registered in the database 16 and the feature vector of the input text received from the user for the search. By using the feature vector, the data search device 14 according to this embodiment is able to perform a search that takes into account variations in the meaning of sentences, notations or expressions, or multiple languages. The similarity of the feature vectors can be calculated using the cosine similarity between two feature vectors, or the like.

[0122] FIG. 12 is a flowchart showing an example of the processing of the data search device 14 according to this embodiment.

[0123] In step S10, the input text acquisition unit 50 of the data search device 14 accepts input of input text (question or query) to be used for search from the user. The data search device 14 may acquire the input text accepted by the chatbot function unit 58 from the user.

[0124] In step S12, the second feature vector acquisition unit 52 performs feature vectorization on the input text acquired by the input text acquisition unit 50 for a search using the similarity of the feature vectors. The input text to be feature vectorized may be the question Q2 displayed in the example UI (User Interface) shown in Fig. 13, or may be the question Q1, answer A1, and question Q2 resulting from the question and answer session so far. Fig. 13 shows an example of the UI displayed on the user terminal 18.

[0125] In step S14, the search unit 54 uses the feature vector of the input text acquired by the second feature vector acquisition unit 52 in step S12 to search for search unit data with similar feature vectors from the search unit data registered in the database 16.

[0126] The search in step S14 may be performed after narrowing down the search results by items such as the creation date or the creator name shown in Fig. 10. For information that is difficult to search using feature vectors, a search targeting, for example, full-text search or named entity extraction may also be used.

[0127] In step S16, the answer output unit 56 outputs an answer sentence based on the data of the search unit searched by the search unit 54. The answer output unit 56 may output the note title of the data of the search unit, the part of the query that corresponds to the note title, etc. as the search result.

[0128] The answer output unit 56 may output an answer sentence generated by the trained large-scale language model based on the data in search units searched by the search unit 54. The answer output unit 56 generates an answer sentence that matches the question sentence based on the data in search units searched in step S14.

[0129] The trained large-scale language model may be instructed to generate an answer sentence that matches the question sentence by, for example, inputting the prompt shown in Fig. 14 into the trained large-scale language. Fig. 14 is an explanatory diagram of an example of a prompt to be input into the trained large-scale language. In the example of Fig. 14, the trained large-scale language model can be instructed to generate an answer sentence that matches the intent of the user's question based on the top N search unit data retrieved from database 16 based on the similarity of feature vectors.

[0130] In step S18, the chatbot function unit 58 displays the response sentence generated in step S16 on, for example, the user terminal 18, etc., to present it to the user.

[0131] Therefore, according to the information processing system 1 of this embodiment, even if data divided from an electronic experiment notebook is registered in the database 16 as search unit data, the accuracy of answers or searches can be improved so that the answers or search results expected by the user can be presented.

[0132] Although the present embodiment has been described above, it will be understood that various modifications in form and detail are possible without departing from the spirit and scope of the claims. Although the present invention has been described based on the examples, the present invention is not limited to the above examples and various modifications are possible within the scope of the claims. This application claims priority from basic application No. 2023-212963 filed with the Japan Patent Office on December 18, 2023, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0133] 1. Information Processing Systems 10. Electronic experiment notebook storage device 12 Data registration device 14 Data retrieval device 16 Databases 18 User terminal 22 Data Registration and Search System 32 Division 36 Summary Acquisition Section 38 Synthesis Section 40 First feature vector acquisition unit 42 Registration Department 44 File Acquisition Section 50 Input text acquisition section 52 Second feature vector acquisition unit 54 Search Section 56 Answer output section 58 Chatbot Function Department 500 computers

Claims

1. A data registration device that registers data in units of search into a database, a division unit that divides a document into data of the search unit; a summary acquisition unit that acquires a first summary text of a first main text included in the search unit data using a first machine learning model; a synthesis unit that generates first synthesized data by synthesizing all of the first summary text of the document, and second synthesized data of the first body text and the first synthesized data; a first feature vector acquisition unit that acquires a feature vector of the second combined data using a second machine learning model; a registration unit that registers the second combined data and the search unit data including a feature vector of the second combined data in the database; A data registration device having:

2. a file acquisition unit that acquires a file associated with the search unit data; and the summary acquisition unit acquires a second summary text of a second body text included in the file using the first machine learning model; the synthesis unit generates third synthesis data of the first body text and the second summary text of the document; the first feature vector acquisition unit acquires a feature vector of the third combined data using the second machine learning model; The registration unit registers the third combined data and the search unit data including a feature vector of the third combined data in the database.

2. The data registration device according to claim 1.

3. The registration unit further registers meta information of the document.

3. The data registration device according to claim 1 or 2.

4. The division unit divides the document into data for each of one or more sections.

3. The data registration device according to claim 1 or 2.

5. The dividing unit divides the document into the search unit data so that the length of the character string does not exceed a length that can be input to the second machine learning model.

3. The data registration device according to claim 1 or 2.

6. The document is an electronic lab notebook 3. The data registration device according to claim 1 or 2.

7. 3. A data search device for searching search unit data registered in a database by the data registration device according to claim 2, comprising: an input text acquisition unit that acquires input text to be used for search; a second feature vector acquisition unit that acquires a feature vector of the input text using the second machine learning model; a search unit that uses the feature vector of the input text to search for data in the search units registered in the database that has a similar feature vector to the second composite data or the third composite data; A data search device having:

8. an answer output unit that outputs an answer sentence generated by a third machine learning model based on the data of the search unit searched by the search unit; 8. The data retrieval device according to claim 7, further comprising:

9. The search unit searches for data in the search unit that includes a feature vector of the second synthetic data or the third synthetic data that is similar to the feature vector of the input text, according to a vector similarity between the feature vector of the input text and the feature vector of the second synthetic data or the third synthetic data that is included in the data in the search unit registered in the database.

9. The data search device according to claim 8.

10. a chatbot function unit that receives the input text from a user and displays the answer sentence; 9. The data search device according to claim 8, further comprising:

11. The third machine learning model is a trained language model that generates the answer sentence that matches the input text based on the data of the search unit searched by the search unit.

11. The data search device according to claim 10.

12. A data registration device that registers search unit data in a database, a dividing step of dividing the document into the search unit data; a summary obtaining step of obtaining a first summary text of a first main text included in the search unit data using a first machine learning model; a combining step of generating first combined data by combining all of the first summary text of the document, and second combined data of the first body text and the first combined data; a first feature vector acquisition step of acquiring a feature vector of the second synthetic data using a second machine learning model; a registration step of registering the second composite data and the search unit data including a feature vector of the second composite data in the database; A program that executes the following.

13. A data registration and search system that registers data in search units in a database and searches for the data in search units registered in the database, a division unit that divides a document into data of the search unit; a summary acquisition unit that acquires a first summary text of a first main text included in the search unit data using a first machine learning model; a synthesis unit that generates first synthesized data by synthesizing all of the first summary text of the document, and second synthesized data of the first body text and the first synthesized data; a first feature vector acquisition unit that acquires a feature vector of the second combined data using a second machine learning model; a registration unit that registers the second combined data and the search unit data including a feature vector of the second combined data in the database; an input text acquisition unit that acquires input text to be used for search; a second feature vector acquisition unit that acquires a feature vector of the input text using the second machine learning model; a search unit that searches the search unit data registered in the database for search unit data having a similar feature vector to the second composite data, using the feature vector of the input text; A data registration and search system having the above.

Images