Learning system, learning method, estimation system, estimation method, and program
The learning system enhances document retrieval efficiency and accuracy by combining multiple document elements using a feature extraction model, addressing the time-consuming requirement for large data sets in conventional methods.
Patent Information
- Application Number
- JP2022193049
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-12-01
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-12-01
AI Technical Summary
Conventional methods for estimating documents similar to a search query require large amounts of document data for high learning efficiency and estimation accuracy, which is time-consuming.
A learning system that extracts and combines multiple elements from document data using a feature extraction model to increase similarity between query and document features, allowing for efficient learning and estimation without extensive data preparation.
Improves learning efficiency and estimation accuracy by utilizing multiple document elements, enabling accurate document retrieval without the need for large data sets.
Smart Images

Figure 0007588629000002 
Figure 0007588629000003 
Figure 0007588629000004
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a learning system, a learning method, an estimation system, an estimation method, and a program. [Background technology]
[0002] There is known a technique for estimating documents having contents similar to a search keyword (query) input by a user.
[0003] For example, Non-Patent Documents 1 and 2 below describe a technology for extracting features of query data using a feature extraction model, and calculating the similarity between the features of the query data and features of one element contained in each of a plurality of document data that have been extracted in advance using the feature extraction model, thereby estimating documents with content similar to the query.
[0004] The techniques described in Non-Patent Document 1 and Non-Patent Document 2 include, in addition to an estimation stage in which the above estimation is performed, a learning stage in which the feature extraction model is trained. Specifically, in the learning stage, the feature extraction model is used to extract features of query data and features of an element of document data corresponding to the query data. Then, the feature extraction model is trained so as to increase the similarity between the features of the query data and the features of an element of document data corresponding to the query data. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Wataru Sakata, Tomohide Shibata, Ribeka Tanaka, and Sadao Kurohashi. 2019. FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). Association for Computing Machinery, New York, NY, USA, 1113-1116. [Non-Patent Document 2] Seo, J.; Lee, T.; Moon, H.; Park, C.; Eo, S.; Aiyanyo, ID; Park, K.; So, A.; Ahn, S.; Park, J. Dense-to-Question and Sparse-to-Answer: Hybrid Retriever System for Industrial Frequently Asked Questions. Mathematics 2022, 10, 1335. Summary of the Invention [Problem to be solved by the invention]
[0006] In order to obtain high learning efficiency and estimation accuracy in the above-mentioned conventional technology, it is desirable to prepare a large amount of document data in the learning and estimation stages, but preparing such a large amount of document data is extremely time-consuming.
[0007] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a learning system, a learning method, an estimation system, an estimation method, and a program that are capable of achieving high learning efficiency and estimation accuracy without preparing a large amount of document data. [Means for solving the problem]
[0008] (1) A learning system according to the present disclosure includes an acquisition unit that acquires query data and acquires the first element, the second element, and combined data of the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; and a feature extraction model that extracts features of the query data and the corresponding document data from each of a pair of the query data and the first element of the corresponding document data, a pair of the query data and the second element of the corresponding document data, and a pair of the query data and the combined data of the corresponding document data. a pair of the feature amounts of the query data and the feature amounts of the first element of the corresponding document data, a pair of the feature amounts of the query data and the feature amounts of the second element of the corresponding document data, and a pair of the feature amounts of the query data and the feature amounts of the combined data of the corresponding document data; and a learning means that executes learning of the feature extraction model so as to increase a similarity between the pair of the feature amounts of the query data and the feature amounts of the first element of the corresponding document data, a similarity between the pair of the feature amounts of the query data and the feature amounts of the second element of the corresponding document data, and a similarity between the pair of the feature amounts of the query data and the feature amounts of the combined data of the corresponding document data.
[0009] (2) In the learning system of (1), the acquisition means further acquires the first element, the second element, and the combined data from non-corresponding document data that is included in the plurality of document data and does not correspond to the query data, and the extraction means further extracts, using the feature extraction model, a pair of features of the query data and the first element of the non-corresponding document data from each of a pair of the query data and the first element of the non-corresponding document data, a pair of the query data and the second element of the non-corresponding document data, and a pair of the combined data of the query data and the non-corresponding document data. , a pair of features of the query data and features of the second element of the non-corresponding document data, and a pair of features of the query data and features of the combined data of the non-corresponding document data, and the learning means may further perform learning of the feature extraction model so that a similarity between the pair of features of the query data and features of the first element of the non-corresponding document data, a similarity between the pair of features of the query data and features of the second element of the non-corresponding document data, and a similarity between the pair of features of the query data and features of the combined data of the non-corresponding document data are reduced.
[0010] (3) In the learning system of (1) or (2), the acquisition means may further acquire the first element and the second element from first document data included in the plurality of document data, and the extraction means may further use the feature extraction model to extract a pair of a feature of either the first element or the second element of the first document data and a feature of the other of the first element or the second element of the first document data from a pair of either the first element or the second element of the first document data and the other of the first element or the second element of the first document data, and the learning means may further perform learning of the feature extraction model so as to increase a similarity between a pair of the feature of either the first element or the second element of the first document data and the feature of the other of the first element or the second element of the first document data.
[0011] (4) In the learning system of (3), the acquisition means may further acquire the first element or the second element from second document data included in the plurality of document data, and the extraction means may further use the feature extraction model to extract a pair of a feature of either the first element or the second element of the first document data and a feature of either the first element or the second element of the second document data from a pair of either the first element or the second element of the first document data and either the first element or the second element of the second document data, and the learning means may further perform learning of the feature extraction model so that a similarity between a pair of the feature of either the first element or the second element of the first document data and the feature of either the first element or the second element of the second document data is reduced.
[0012] (5) In the learning system of (3) or (4), the acquisition means may further acquire the combined data from the first document data, and the extraction means may further use the feature extraction model to extract a pair of a feature of any one of the first element, the second element or the combined data of the first document data and a feature of any one of the first element, the second element or the combined data of the first document data from a pair of any one of the first element, the second element or the combined data of the first document data and the other one of the first element, the second element or the combined data of the first document data, and the learning means may further perform learning of the feature extraction model so as to increase a similarity between a pair of the feature of any one of the first element, the second element or the combined data of the first document data and the other one of the first element, the second element or the combined data of the first document data.
[0013] (6) The estimation system according to the present disclosure includes a first feature acquisition means for acquiring features of the first element of each of a plurality of document data having a first element and a second element, the features being extracted from the first element of the plurality of document data using a feature extraction model; a combined feature acquisition means for acquiring features of the combined data of each of the plurality of document data, the combined data being extracted from combined data obtained by combining the first element and the second element of each of the plurality of document data using the feature extraction model; a query feature extraction means for extracting features of the query data from query data using the feature extraction model; a calculation means for calculating, for each of the plurality of document data, a first similarity which is a similarity between a pair of features of the first element and features of the query data, and for calculating a combined similarity which is a similarity between a pair of features of the combined data and features of the query data; and a selection means for selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated by the calculation means for each of the plurality of document data.
[0014] (7) In the estimation system of (6), the selection means may select one or more of the plurality of document data based on a representative value of the first similarity and the combined similarity for each of the plurality of document data.
[0015] (8) In the estimation system of (6) or (7), the estimation system may further include a second feature acquisition means for acquiring features of the second elements of each of the plurality of document data extracted from the second elements of each of the plurality of document data using the feature extraction model, wherein the calculation means further calculates a second similarity for each of the plurality of document data, which is a similarity between a pair of the features of the second element and the features of the query data, and the selection means may select one or more of the plurality of document data based further on the second similarity calculated by the calculation means for each of the plurality of document data.
[0016] (9) In the estimation system according to any one of (6) to (8), the feature extraction model acquires training query data, and acquires the first element, the second element, and the combined data from corresponding document data corresponding to the training query data; and extracts, using the feature extraction model, a pair of features of the training query data and the first element of the corresponding document data, a feature of the first element of the corresponding document data, and a feature of the previous element from each of a pair of the training query data and the first element of the corresponding document data, a pair of the training query data and the second element of the corresponding document data, and a pair of the training query data and the combined data of the corresponding document data. extracting a pair of features of the training query data and features of the second element of the corresponding document data, and a pair of features of the training query data and features of the combined data of the corresponding document data, respectively; and performing learning of the feature extraction model such that a similarity between the pair of the features of the training query data and features of the first element of the corresponding document data, a similarity between the pair of the features of the training query data and features of the second element of the corresponding document data, and a similarity between the pair of the features of the training query data and features of the combined data of the corresponding document data are increased.
[0017] (10) A learning method according to the present disclosure includes the steps of acquiring query data and acquiring the first element, the second element, and combined data of the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; and extracting features of the query data and the corresponding document data from each of a pair of the query data and the first element of the corresponding document data, a pair of the query data and the second element of the corresponding document data, and a pair of the query data and the combined data of the corresponding document data, using a feature extraction model. The method includes an extraction step of extracting a pair of the features of the query data and the first element, a pair of the features of the query data and the second element of the corresponding document data, and a pair of the features of the query data and the features of the combined data of the corresponding document data, respectively; and a learning step of executing learning of the feature extraction model so as to increase a similarity between the pair of the features of the query data and the features of the first element of the corresponding document data, a similarity between the pair of the features of the query data and the features of the second element of the corresponding document data, and a similarity between the pair of the features of the query data and the features of the combined data of the corresponding document data.
[0018] (11) The estimation method according to the present disclosure includes a first feature acquisition step of acquiring features of the first element of each of a plurality of document data having a first element and a second element, the features being extracted from the first element of the plurality of document data using a feature extraction model; a combined feature acquisition step of acquiring features of the combined data of each of the plurality of document data, the combined data being extracted from combined data obtained by combining the first element and the second element of each of the plurality of document data using the feature extraction model; a query feature extraction step of extracting features of the query data from query data using the feature extraction model; a calculation step of calculating, for each of the plurality of document data, a first similarity which is a similarity between a pair of features of the first element and features of the query data, and calculating a combined similarity which is a similarity between a pair of features of the combined data and features of the query data; and a selection step of selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated by the calculation step for each of the plurality of document data.
[0019] (12) A program according to the present disclosure includes an acquisition unit that acquires query data, and acquires the first element, the second element, and combined data of the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; and a feature extraction unit that extracts features of the query data and the corresponding document data from a pair of the query data and the first element of the corresponding document data, a pair of the query data and the second element of the corresponding document data, and a pair of the query data and the combined data of the corresponding document data, using a feature extraction model. the computer is made to function as an extraction means for extracting a pair of features of the query data and the first element of the corresponding document data, a pair of features of the query data and the second element of the corresponding document data, and a pair of features of the query data and the combined data of the corresponding document data; and a learning means for executing learning of the feature extraction model so as to increase the similarity between the pair of the features of the query data and the first element of the corresponding document data, the similarity between the pair of the features of the query data and the second element of the corresponding document data, and the similarity between the pair of the features of the query data and the combined data of the corresponding document data.
[0020] (13) A program according to the present disclosure includes a first feature acquisition means for acquiring a feature of each of a plurality of document data including a first element and a second element, the feature being extracted from the first element of each of the plurality of document data using a feature extraction model; The computer is caused to function as: a combined feature acquisition means for acquiring features of the combined data of each of the plurality of document data extracted using the feature extraction model from combined data obtained by combining the first element and the second element of each of the plurality of document data; a query feature extraction means for extracting features of the query data using the feature extraction model from query data; a calculation means for calculating, for each of the plurality of document data, a first similarity which is the similarity between a pair of the feature of the first element and the feature of the query data, and for calculating a combined similarity which is the similarity between a pair of the feature of the combined data and the feature of the query data; and a selection means for selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated by the calculation means for each of the plurality of document data. [Brief description of the drawings]
[0021] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of the estimation system. [Diagram 2] FIG. 13 is a diagram illustrating an example of a screen transition of an FAQ search service page. [Diagram 3] FIG. 2 is a functional block diagram showing an example of functions realized by the estimation system. [Figure 4] FIG. 2 is a diagram showing an overview of processing executed in a learning device. [Diagram 5] FIG. 2 is a diagram showing an overview of processing executed in a learning device. [Figure 6] FIG. 2 is a diagram showing an example of an original training data set. [Figure 7] FIG. 13 is a diagram showing an example of a processed learning dataset. [Figure 8] FIG. 1 shows a schematic diagram of training with contrastive loss. [Figure 9] FIG. 2 is a diagram showing an overview of a process executed in the estimation device. [Figure 10] FIG. 10 is a diagram illustrating an example of a method for creating a feature amount database. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] [1. Hardware configuration of the estimation system] An example of an estimation system including a learning device and an estimation device according to the present disclosure will be described below. FIG. 1 is a diagram showing an example of a hardware configuration of the estimation system. As shown in FIG. 1, the estimation system S includes a learning device 10 and an estimation device 20, which are connected to a network N such as the Internet or a LAN. Although FIG. 1 shows one learning device 10 and one estimation device 20, there may be a plurality of these. Furthermore, FIG. 1 shows a case where the learning device 10 and the estimation device 20 are different from each other, but they may be the same device. Furthermore, the learning device 10 and the estimation device 20 may belong to different systems.
[0023] Study device 10 is a computer operated by a user. For example, study device 10 is a personal computer, a server computer, a mobile information terminal (including a tablet computer and a wearable terminal), a mobile phone (including a smartphone), etc. Study device 10 includes a control unit 11, a memory unit 12, a communication unit 13, an operation unit 14, and a display unit 15.
[0024] The control unit 11 has at least one processor. The control unit 11 executes processing according to programs and data stored in the storage unit 12. The storage unit 12 includes a main storage unit and an auxiliary storage unit. For example, the main storage unit is a volatile memory such as a RAM, and the auxiliary storage unit is a non-volatile memory such as a ROM, an EEPROM, a flash memory, or a hard disk.
[0025] The communication unit 13 is a communication interface for wired or wireless communication, and performs data communication via the network N. The operation unit 14 is an input device for performing operations, and is, for example, a touch panel, a pointing device such as a mouse, a keyboard, or a button. The display unit 15 is, for example, a liquid crystal display unit or an organic EL display unit. The display unit 15 displays an image according to an instruction from the control unit 11.
[0026] The estimation device 20 is a computer operated by a user. The user of the learning device 10 and the user of the estimation device 20 may be the same or different. For example, the estimation device 20 is a personal computer, a server computer, a mobile information terminal (including a tablet computer and a wearable terminal), or a mobile phone (including a smartphone). The estimation device 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The physical configurations of the control unit 21, the storage unit 22, the communication unit 23, the operation unit 24, and the display unit 25 may be similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 14, and the display unit 15, respectively.
[0027] The programs and data described as being stored in the storage units 12 and 22 may be supplied via the network N. The hardware configuration of each of the learning device 10 and the estimation device 20 is not limited to the above example, and various hardware may be applied. For example, a reading unit (e.g., an optical disk drive or a memory card slot) that reads a computer-readable information storage medium and an input / output unit (e.g., a USB port) for inputting and outputting data to and from an external device may be included. For example, the programs and data stored in the information storage medium may be supplied to each computer via the reading unit or the input / output unit.
[0028] [2. Overview of the estimation system] Based on a query input by a user, the estimation system S estimates a document from among multiple documents whose content is similar to the query. The query is mainly character information, but a numerical value or the like may also be used as the query. This character information is sometimes called a keyword or a search string. For example, the character information may include only one word (search term), or may include multiple words. When the character information includes multiple words, the multiple words may be separated by spaces. In this case, the multiple words may be used in an AND condition or an OR condition. Furthermore, for example, the character information is not limited to a word, and may be a sentence or a part of a sentence.
[0029] In this embodiment, a case will be described in which the estimation system S is applied to a frequently asked questions (hereinafter, referred to as "FAQ") search service on an EC site. For example, the estimation system S acquires character information input by a user to his / her user terminal as a query, and estimates one or more FAQ documents with contents similar to the query. Note that the estimation system S can be applied to any scene other than FAQ search, and can be applied to various scenes such as, for example, paper search and patent document search.
[0030] For example, when a user operates his / her own user terminal to access the estimation device 20, the top screen of the FAQ search service is displayed on the display unit of the user terminal. Note that the screens described below may be displayed as screens of an application stored in the user terminal, or may be displayed as screens of a web browser. In addition, although a screen for a smartphone is taken as an example in this embodiment, a screen with another layout, such as for a desktop or tablet, may also be displayed.
[0031] FIG. 2 is a diagram showing an example of screen transitions on the FAQ search service page. An input form F for inputting a query Q and a button B for executing a search are displayed on a top screen G1 shown on the left side of FIG. 2. When a user inputs the query Q into the input form F and selects the button B, the estimation device 20 searches for FAQ documents with contents similar to the query Q. In general, the estimation device 20 determines the degree of similarity between each of a plurality of FAQ documents stored in a database and the query Q, and selects a certain number of FAQ documents from the plurality of FAQ documents in descending order of similarity to the query Q. As shown in FIG. 2, a list of titles T of the selected FAQ documents is displayed as a search result SR on the top screen G1. Note that, in this embodiment, a case will be described in which FAQ documents are selected in descending order of similarity to the query Q, but FAQ documents may be selected in ascending order of similarity to the query Q.
[0032] When the user selects one of the titles T of FAQ documents displayed as the search result SR, the screen transitions from the top screen G1 to the FAQ screen G2 shown on the right side of Fig. 2. The FAQ screen G2 displays the FAQ document corresponding to the selected title T. The FAQ document includes a title T, which is a first element, an answer A, which is a second element, and a category C, which is a third element. Note that the FAQ document may include elements other than those mentioned above, and may include, for example, keywords, helpfulness, number of views, etc. as its elements.
[0033] Title T corresponds to the question text of each FAQ document. Title T can also be said to be the title or summary of each FAQ document. Answer A corresponds to the answer text of each FAQ document. Answer A can also be said to be the body of each FAQ document.
[0034] Category C is the classification to which each FAQ document belongs, and is determined according to the content of the FAQ document. Category C is, for example, "Login / Member Registration", "Points", "How to Order", etc. In this embodiment, the classes to which each FAQ document belongs are classified into multiple stages, from general to specific. In other words, category C is a hierarchical category, and is determined, for example, as "EC Site Top > Inquiry Top > Login / Member Registration".
[0035] As explained in the Background section, in the conventional technology, the degree of similarity between an FAQ document and a query Q is determined based on the degree of similarity between the feature of one element of the FAQ document (e.g., title T or answer A) extracted using a feature extraction model and the feature of the query Q. However, although an FAQ document has multiple elements, i.e., multiple aspects, the conventional technology takes into account only one of those aspects. Therefore, it is difficult to obtain a high estimation accuracy with the conventional technology, and it has been considered desirable to prepare a large amount of FAQ document data to improve the estimation accuracy.
[0036] In this regard, the estimation device 20 of the estimation system S according to this embodiment determines the degree of similarity between each FAQ document and query Q based on the similarity between each of the features of title T, the features of answer sentence A, the features of category C, the features of the combination of title T and answer sentence A, the features of the combination of title T and category C, the features of the combination of answer sentence A and category C, and the features of the combination of title T, answer sentence A and category C, which are extracted using the feature extraction model, and the features of query Q. In this way, the degree of similarity between each FAQ document and query Q is determined taking into account not only one aspect of the FAQ document but multiple aspects thereof, thereby making it possible to improve the estimation accuracy without preparing a large amount of FAQ document data.
[0037] Also, as explained in the Background section, in the conventional technology, in the learning stage, the feature amount of a query Q and the feature amount of an element of an FAQ document corresponding to the query Q are extracted using a feature amount extraction model. Then, the feature amount extraction model is trained so as to increase the similarity between the feature amount of query Q and the feature amount of an element of the FAQ document corresponding to query Q. However, in such conventional technology, since only one piece of learning data can be obtained from one piece of FAQ document data, it is difficult to achieve high learning efficiency, and it has been considered desirable to prepare a large amount of FAQ document data in order to improve learning efficiency.
[0038] In this regard, the learning device 10 of the estimation system S according to the present embodiment extracts the feature amount of the query Q, and the feature amount of the title T, the feature amount of the answer sentence A, the feature amount of the category C, the feature amount of the combination of the title T and the answer sentence A, the feature amount of the combination of the title T and the category C, the feature amount of the combination of the answer sentence A and the category C, and the feature amount of the combination of the title T, the answer sentence A and the category C of the FAQ document corresponding to the query Q, using a feature amount extraction model. Then, the learning of the feature amount extraction model is executed so that the similarity between the feature amount of the query Q and each of the feature amounts of the title T, the feature amount of the answer sentence A, the feature amount of the category C, the feature amount of the combination of the title T and the answer sentence A, the feature amount of the combination of the title T and the category C, the feature amount of the combination of the answer sentence A and the category C of the FAQ document corresponding to the query Q is increased. As a result, multiple learning data can be obtained from one FAQ document data, and learning efficiency can be improved without preparing a large amount of FAQ document data. Hereinafter, the estimation system S will be described in detail.
[0039] [3. Functions realized by the estimation system] 3 is a functional block diagram showing an example of functions realized by the estimation system. Here, functions realized by the learning device 10 and functions realized by the estimation device 20 will be described.
[0040] [3-1. Functions realized by the learning device] In the learning device 10, a data storage unit 100, an acquisition unit 101, an extraction unit 102, and a learning unit 103 are realized. The data storage unit 100 is realized mainly by the storage unit 12, and the acquisition unit 101, the extraction unit 102, and the learning unit 103 are each realized mainly by the control unit 11. Hereinafter, the learning device 10 will be described with reference to Figs. 4 and 5. Figs. 4 and 5 are diagrams showing an overview of the processing executed in the learning device. The processing shown in Figs. 4 and 5 is executed by the control unit 11 operating in accordance with a program stored in the storage unit 12.
[0041] [Data storage section] The data storage unit 100 stores the feature extraction model FE and the learning database LDB.
[0042] The feature extraction model FE is a machine learning model that extracts features of character information from character information such as words and sentences. As the feature extraction model FE, various known machine learning models can be used, and for example, BERT (Bidirectional Encoder Representations from Transformers) and LaBSE (Language-agnostic BERT Sentence Embedding) can be used as the feature extraction model FE.
[0043] In this embodiment, the feature amount is specifically a vector (embedding vector) embedded in a vector space called a latent space. As will be described in detail later, in the latent space, two words or sentences with similar meanings are embedded in positions close to each other, and two words or sentences with dissimilar meanings are embedded in positions far from each other (see FIG. 8). The embedding vector is also called a distributed representation.
[0044] In this embodiment, the feature extraction model FE does not mean the concept of feature extraction, but means electronic data generated by the concept. That is, the feature extraction model FE includes a program and parameters. Note that the feature extraction model FE may mean only either the program or the parameters.
[0045] The program included in the feature extraction model FE defines the processing content (commands or calculation formulas) that the computer should execute. The parameters include values that are adjusted by learning, such as weighting coefficients and biases. A feature extraction model FE before learning includes parameters of initial values, and a learned feature extraction model FE includes parameters adjusted by learning.
[0046] In this embodiment, a case will be described in which the trained feature extraction model FE is stored in the data storage unit 200 of the estimation device 20, but the trained feature extraction model FE may also be stored in the data storage unit 100 of the learning device 10. In addition, for example, the data storage unit 100 may store a program required for the learning process and a formula (Formula 1) described later. This formula may be written as part of the program.
[0047] The learning database LDB stores an original learning dataset LDS-o. Fig. 6 is a diagram showing an example of the original learning dataset. As shown in Fig. 6, the original learning dataset LDS-o includes query data QD and FAQ document data DD. Although details will be described later, the original learning dataset LDS-o is not used for learning as it is, but is processed into a processed learning dataset LDS-p by the acquisition unit 101 and then used for learning.
[0048] The FAQ document data DD has, as its elements, title data TD (first element), answer sentence data AD (second element), and category data CD. In the original learning dataset LDS-o, each of the multiple query data QD is associated with FAQ document data DD indicating the FAQ with the content closest to the query data QD. From another perspective, the original learning dataset LDS-o can also be said to have corresponding FAQ document data DD-m (corresponding document data) corresponding to each of the multiple query data QD, and non-corresponding FAQ document data DD-n (non-corresponding document data) not corresponding to each of the multiple query data QD (see FIG. 4).
[0049] 6, for convenience, each of the multiple query data QD is referred to as "Query 1" etc., but in reality, the query data QD is character information such as words or sentences that are expected to be entered by a user when searching. For example, "Query 1" may be a sentence such as "When will the product arrive?" and "Query 2" may be a sentence such as "I forgot my password."
[0050] Similarly to the query data QD, the title data TD, answer data AD, and category data CD are also character information such as words or sentences written in the FAQ document. For example, the title data TD ("Title 1"), answer data AD ("Answer 1"), and category data CD ("Category 1") corresponding to the above example "Query 1" ("When will the product arrive?") may be a sentence such as "About the product delivery period", "Please check the product sales page for any explanations regarding the delivery period and delivery method...", or "EC site top > Inquiry top > Transaction-related problems". The query data QD, title data TD, answer data AD, and category data CD may include numerical values, etc.
[0051] [Acquisition Department]
[0052] Hereinafter, the acquiring unit 101 will be described with reference to Fig. 7 in addition to Fig. 3 to Fig. 5. Fig. 7 is a diagram showing an example of a processed learning dataset.
[0053] The acquiring unit 101 acquires the processed learning dataset LDS-p from the original learning dataset LDS-o. As shown in FIG. 7, the processed learning dataset LDS-p includes query data QD, title data TD, answer sentence data AD, and category data CD, similar to the original learning dataset LDS-o. The processed learning dataset LDS-p further includes title-answer sentence combined data TAD obtained by combining the title data TD and the answer sentence data AD, title-category combined data TCD obtained by combining the title data TD and the category data CD, answer sentence-category combined data ACD obtained by combining the answer sentence data AD and the category data CD, and title-answer sentence-category combined data TACD obtained by combining the title data TD, the answer sentence data AD, and the category data CD. That is, the processed learning dataset LDS-p includes the query data QD and seven types of data derived from the FAQ document data DD.
[0054] Specifically, the title-answer sentence combination data TAD is obtained by combining the title data TD and the answer sentence data AD by string combination. In this embodiment, this string combination is performed by inserting a [SEP] token between two sentences. For example, if the title data TD is "About the delivery period of the product", by adding "Please check the product sales page for an explanation on the delivery time and delivery method..." as the answer sentence data AD at the end, the title-answer sentence combination data TAD of "'About the delivery period of the product' '[SEP]' 'Please check the product sales page for an explanation on the delivery time and delivery method..." can be obtained. The order of combination may be reversed from the above. Also, the string combination may be performed by the learning device 10, or the learning device 10 may acquire the title-answer sentence combination data TAD combined by another device. Also, the string combination may be performed automatically or manually. The title-category combination data TCD, the answer sentence-category combination data ACD, and the title-answer sentence-category combination data TACD are also obtained by the same string combination as above.
[0055] 4, the acquiring unit 101 acquires query data QD, corresponding FAQ document data DD-m corresponding to the query data QD, and non-corresponding FAQ document data DD-n not corresponding to the query data QD from the original learning data set LDS-o of the learning database LDB. For example, if the query data QD is "When will the product arrive?", the title data TD-m of the corresponding FAQ document data DD-m may be "Regarding the product delivery period," and the title data TD-n of the non-corresponding FAQ document data DD-n may be "Regarding how to use points."
[0056] Furthermore, the acquiring unit 101 acquires title data TD-m, answer sentence data AD-m, and title-answer sentence data TAD-m from the corresponding FAQ document data DD-m as shown in Fig. 4. Also, the acquiring unit 101 acquires title data TD-n, answer sentence data AD-n, and combined title-answer sentence data TAD-n from the non-corresponding FAQ document data DD-n.
[0057] For ease of explanation, Fig. 4 does not show how the acquiring unit 101 acquires the category data CD, the title-category combined data TCD, the answer sentence-category combined data ACD, and the title-answer sentence-category combined data TACD, but in reality, the acquiring unit 101 acquires these data. In the following explanation, processing of these data will be omitted, but in reality, these data are subjected to the same processing as that performed on the title data TD, the answer sentence data AD, and the title-answer sentence combined data TAD. The acquiring unit 101 may acquire only the query data QD, the title data TD, the answer sentence data AD, and the title-answer sentence combined data TAD without acquiring the above data.
[0058] 4, the query data QD, the title data TD-m of the corresponding FAQ document data DD-m, and the title data TD-n of the non-corresponding FAQ document data DD-n form a first input group IG1. Similarly, the query data QD, the answer sentence data AD-m of the corresponding FAQ document data DD-m, and the answer sentence data AD-n of the non-corresponding FAQ document data DD-n form a second input group IG2. In addition, the query data QD, the combined title-answer sentence data TAD-n of the corresponding FAQ document data DD-m, and the combined title-answer sentence data TAD-n of the non-corresponding FAQ document data DD-n form a third input group IG3.
[0059] Now, moving to Fig. 5, the acquiring unit 101 acquires the first FAQ document data DD-a (first document data) and the second FAQ document data DD-b (second document data) from the original learning data set LDS-o of the learning database LDB. Figs. 4 and 5 show a case where the first FAQ document data DD-a and the corresponding FAQ document data DD-m are different FAQ document data, and the second FAQ document data DD-b and the non-corresponding FAQ document data DD-n are different FAQ document data. Note that the first FAQ document data DD-a and the corresponding FAQ document data DD-m may be the same FAQ document data, and the second FAQ document data DD-b and the non-corresponding FAQ document data DD-n may be the same FAQ document data.
[0060] Furthermore, as shown in Fig. 5, the acquiring unit 101 acquires answer sentence data AD-a and title data TD-a from the first FAQ document data DD-a, and acquires title data TD-b or answer sentence data AD-b from the second FAQ document data DD-b. Specifically, as shown in Fig. 5, the acquiring unit 101 acquires answer sentence data AD-a and title data TD-a from the first FAQ document data DD-a, and acquires title data TD-b from the second FAQ document data DD-b.
[0061] In addition, as shown in Figure 5, the acquisition unit 101 acquires answer sentence data AD-a and combined title-answer sentence data TAD-a from the first FAQ document data DD-a, and acquires combined title-answer sentence data TAD-b from the second FAQ document data DD-b.
[0062] Here, the answer sentence data AD-a and title data TD-a of the first FAQ document data DD-a and the title data TD-b of the second FAQ document data DD-b form a fourth input group IG4. Also, the answer sentence data AD-a and combined title-answer sentence data TAD-a of the first FAQ document data DD-a and combined title-answer sentence data TAD-b of the second FAQ document data DD-b form a fifth input group IG5.
[0063] In addition, in FIG. 5, only the state where the acquiring unit 101 acquires the fourth input group IG4 and the fifth input group IG5 is shown, but actually, the acquiring unit 101 acquires a plurality of input groups other than these. That is, for example, the acquiring unit 101 actually acquires an input group formed by the category data CD-a and the answer sentence data AD-a of the first FAQ document data DD-a and the answer sentence data AD-b of the second FAQ document data DD-b. It is to be noted that the combination of data in such an input group is not limited to this example. In the following explanation, the processing of the input groups other than the fourth input group IG4 and the fifth input group IG5 is omitted, but actually, the input group is subjected to the same processing as that performed on the fourth input group IG4 and the fifth input group IG5. It is to be noted that the acquiring unit 101 may acquire only the fourth input group IG4 and the fifth input group IG5 without acquiring the above input groups.
[0064] [Extraction part] As shown in FIG. 4, the extraction unit 102 uses the feature extraction model FE from each of the first input group IG1, the second input group IG2, and the third input group IG3 to extract a first output group OG1 formed by the feature QF of the query data, the feature TF-m of the title data of the corresponding FAQ document data DD-m, and the feature TF-n of the title data of the non-corresponding FAQ document data DD-n, a second output group OG2 formed by the feature QF of the query data, the feature AF-m of the answer sentence data of the corresponding FAQ document data DD-m, and the feature AF-n of the answer sentence data of the non-corresponding FAQ document data DD-n, and a third output group OG3 formed by the feature QF of the query data, the feature TAF-m of the combined title-answer sentence data of the corresponding FAQ document data DD-m, and the feature TAF-n of the combined title-answer sentence data of the non-corresponding FAQ document data DD-n.
[0065] Moving on to Figure 5, the extraction unit 102 uses the feature extraction model FE to extract from the fourth input group IG4 and the fifth input group IG5 a fourth output group OG4 formed by the feature AF-a of the answer sentence data and the feature TF-a of the title data of the first FAQ document data DD-a and the feature TF-b of the title data of the second FAQ document data DD-b, and a fifth output group OG5 formed by the feature AF-a of the answer sentence data of the first FAQ document data DD-a and the feature TAF-a of the combined title-answer sentence data of the second FAQ document data DD-b.
[0066] [Learning Department] The learning unit 103 will be further described below with reference to Fig. 8. Fig. 8 is a diagram that illustrates learning using contrastive loss. As shown in Fig. 8, the learning unit 103 executes learning of the feature extraction model FE so that the similarity between the pair of the feature QF of the query data and the feature TF-m of the title data of the corresponding FAQ document data DD-m, the similarity between the pair of the feature QF of the query data and the feature AF-m of the answer sentence data of the corresponding FAQ document data DD-m, and the similarity between the pair of the feature QF of the query data and the feature TAF-m of the combined title-answer sentence data of the corresponding FAQ document data DD-m are increased.
[0067] Furthermore, as shown in FIG. 8, the learning unit 103 performs learning of the feature extraction model FE so as to reduce the similarity between the pair of the feature QF of the query data and the feature TF-n of the title data of the non-compliant FAQ document data DD-n, the similarity between the pair of the feature QF of the query data and the feature AF-n of the answer sentence data of the non-compliant FAQ document data DD-n, and the similarity between the pair of the feature QF of the query data and the feature TAF-n of the combined title-answer sentence data of the non-compliant FAQ document data DD-n.
[0068] As shown in FIG. 4, in this embodiment, the learning unit 103 executes learning of the feature extraction model FE using a contrastive loss L shown in the following formula 1. The contrastive loss L is a similarity sim(QF,F i The larger the similarity sim(QF,F j The smaller n) is, the smaller it becomes.
[0069]
number
[0070] Here, F i -m is the feature of the i-th corresponding FAQ document data DD-m, and F j i and j are natural numbers less than or equal to 7. i -m is any one of the features TF-m of the title data of the corresponding FAQ document data DD-m, the features AF-m of the answer data, the features CF-m of the category data, the features TAF-m of the combined title-answer data, the features TCF-m of the combined title-category data, the features ACF-m of the combined answer data, or the features TACF-m of the combined title-answer-category data.
[0071] Similarly, F j -n is any one of the features TF-n of the title data, the features AF-n of the answer data, the features CF-n of the category data, the features TAF-n of the combined title-answer data, the features TCF-n of the combined title-category data, the features ACF-n of the combined answer data-category data, or the features TACF-n of the combined title-answer data-category data of the non-compliant FAQ document data DD-n.
[0072] The similarity can be obtained by, for example, calculating the cosine similarity or distance between the features. The learning unit 103 adjusts the parameters of the feature extraction model FE using a known parameter adjustment method such as backpropagation, gradient descent, Adam, etc., so that the contrast loss L becomes small. Note that, as the contrast loss L, any function can be used in addition to the one shown in Equation 1.
[0073] Moreover, the learning unit 103 executes learning of the feature extraction model FE so that the similarity between the pair of the feature AF-a of the answer sentence data of the first FAQ document data DD-a and the feature TF-a of the title data of the first FAQ document data DD-a, and the similarity between the pair of the feature AF-a of the answer sentence data of the first FAQ document data DD-a and the feature TAF-a of the combined title-answer sentence data of the first FAQ document data DD-a, are increased. Furthermore, the learning unit 103 executes learning of the feature extraction model FE so that the similarity between the pair of the feature AF-a of the answer sentence data of the first FAQ document data DD-a and the feature TF-b of the title data of the second FAQ document data DD-b, and the similarity between the feature AF-a of the answer sentence data of the first FAQ document data DD-a and the feature TAF-b of the combined title-answer sentence data of the second FAQ document data DD-b, are decreased. As for the learning method itself, a method using contrast loss can be used as described above using FIG. 8 and Equation 1, but is not limited thereto.
[0074] According to the learning device 10 of this embodiment described above, multiple pieces of learning data can be obtained from one piece of FAQ document data, thereby improving learning efficiency without the need to prepare a large amount of FAQ document data.
[0075] Furthermore, in the learning device 10 according to this embodiment, in addition to learning using a group formed by the feature amount of the query data QD, the feature amount of an element of the corresponding FAQ document data DD-m, and the feature amount of an element of the non-corresponding FAQ document data DD-n (see FIG. 4), learning is performed using a group formed by the feature amount of one element of the first FAQ document data DD-a, the feature amount of another element of the first FAQ document data DD-a, and the feature amount of one element of the second FAQ document data DD-b (see FIG. 5). That is, the learning in the learning device 10 according to this embodiment uses learning data that does not include the query data QD, based on the implicit relationship that the similarity between feature amounts belonging to the same FAQ document data is large and the similarity between feature amounts belonging to different FAQ document data is small. By using such learning data, the number of query data QD to be prepared for learning can be reduced.
[0076] [3-2. Functions realized by the estimation device] Returning to FIG. 3, in the estimation device 20, a data storage unit 200, an acquisition unit 201, an extraction unit 202, a calculation unit 203, and a selection unit 204 are realized. The data storage unit 200 is realized mainly by the storage unit 22, and the acquisition unit 201, the extraction unit 202, the calculation unit 203, and the selection unit 204 are each realized mainly by the control unit 21. Hereinafter, the estimation device 20 will be described with reference to FIG. 9. FIG. 9 is a diagram showing an overview of the processing executed in the estimation device. Note that the processing shown in FIG. 9 is executed by the control unit 11 operating in accordance with a program stored in the storage unit 12.
[0077] [Data storage section] The data storage unit 200 stores a trained feature extraction model FE and a feature database FDB. Specifically, the data storage unit 200 stores the feature extraction model FE whose parameters have been adjusted by training in the above-mentioned training device 10. Note that the feature extraction model FE stored in the data storage unit 200 does not necessarily have to be trained in the training device 10.
[0078] The feature amount database FDB will be described below with reference to Fig. 10. Fig. 10 is a diagram for explaining an example of a method for creating a feature amount database. In this embodiment, a case will be described in which the estimation device 20 creates the feature amount database FDB, but the feature amount database FDB may be created by an external device.
[0079] As shown in Fig. 10, the estimation device 20 acquires an FAQ document data set DDS including a plurality of FAQ document data DD, and acquires title data TD, answer sentence data AD, and title-answer sentence combined data TAD from each of the plurality of FAQ document data DD. Note that, for convenience of explanation, in Fig. 10, the acquisition of the category data CD, the title-category combined data TCD, the answer sentence-category combined data ACD, and the title-answer sentence-category combined data TACD is omitted, but in reality, the estimation device 20 acquires these. In the following explanation, processing of these data will be omitted, but in reality, these data are subjected to the same processing as that performed on the title data TD, the answer sentence data AD, and the title-answer sentence combined data TAD. Note that the estimation device 20 may acquire only the query data QD, the title data TD, the answer sentence data AD, and the title-answer sentence combined data TAD without acquiring these.
[0080] The estimation device 20 extracts features TF of the title data, features AF of the answer sentence data, and features TAF of the combined title-answer sentence data from the title data TD, answer sentence data AD, and combined title-answer sentence data TAD, respectively, using the trained feature extraction model FE. The features of each of the multiple FAQ document data DD thus extracted are stored in the feature database FDB, and are then stored in the data storage unit 200.
[0081] [Acquisition Department] Turning now to FIG. 9, the acquiring unit 201 acquires the feature amount TF of the title data, the feature amount AF of the answer sentence data, and the feature amount TAF of the combined title-answer sentence data of each of the multiple FAQ document data DD from the feature amount database FDB.
[0082] [Extraction part] The extraction unit 202 extracts features EQF of the query data EQD to be estimated from the query data EQD to be estimated by using the trained feature extraction model FE. The query data EQD to be estimated is, for example, data corresponding to a query input by a user into an input form F of an FAQ search service page.
[0083] [Calculation section] The calculation unit 203 calculates, for each of the multiple FAQ document data DD, a similarity (first similarity) between a pair of feature TF of the title data and feature EQF of the query data to be estimated, a similarity (second similarity) between a pair of feature AF of the answer sentence data and feature EQF of the query data to be estimated, and a similarity (combined similarity) between a pair of feature TAF of the combined title-answer sentence data and feature EQF of the query data to be estimated. The similarity can be found, for example, by calculating the cosine similarity or distance between the features.
[0084] [Selection] The selection unit 204 selects one or more of the multiple FAQ document data DD based on the similarity between the feature TF of the title data and the feature EQF of the query data to be estimated for each of the multiple FAQ document data DD, the similarity between the feature AF of the answer sentence data and the feature EQF of the query data to be estimated, and the similarity between the feature TAF of the combined title-answer sentence data and the feature EQF of the query data to be estimated. The selected FAQ document data DD may be displayed on the FAQ search service page, for example, in a form as shown in Fig. 2. In this embodiment, a case will be described in which the selection unit 204 selects five of the multiple FAQ document data DD as shown in Fig. 9, but the number of FAQ document data DD selected by the selection unit 204 is not limited to this example.
[0085] Specifically, the selection unit 204 selects one or more of the multiple FAQ document data DD based on a representative value of the similarity between the pair of feature TF of the title data and feature EQF of the query data to be estimated, the similarity between the pair of feature AF of the answer sentence data and feature EQF of the query data to be estimated, and the similarity between the pair of feature TAF of the combined title-answer sentence data and feature EQF of the query data to be estimated, for each of the multiple FAQ document data DD.
[0086] More specifically, for each of the multiple FAQ document data DD, the selection unit 204 determines the similarity between the pair of the feature TF of the title data and the feature EQF of the query data to be estimated, the similarity between the pair of the feature AF of the answer sentence data and the feature EQF of the query data to be estimated, and a representative value of the feature TAF of the combined title-answer sentence data and the feature EQF of the query data to be estimated, and selects one or more of the multiple FAQ document data DD in descending order of the representative value. In this embodiment, the selection unit 204 selects five of the multiple FAQ document data DD in descending order of the representative value. Note that the selection unit 204 may select five of the multiple FAQ document data DD in ascending order of the representative value.
[0087] In this embodiment, the representative value is the similarity between the pair of feature TF of the title data and feature EQF of the query data to be estimated, the similarity between the pair of feature AF of the answer sentence data and feature EQF of the query data to be estimated, and the average value of the feature TAF of the combined title-answer sentence data and the feature EQF of the query data to be estimated, but the representative value may also be the median, maximum value, etc.
[0088] According to the estimation device 20 of this embodiment described above, the degree of similarity between each FAQ document and a query is determined by taking into consideration not only one aspect of the FAQ document but multiple aspects of the FAQ document, so that it is possible to improve the estimation accuracy without preparing a large amount of FAQ document data.
[0089] According to the embodiments of the present invention described above, a learning system, a learning method, an estimation system, an estimation method, and a program are provided that are capable of achieving high learning efficiency and estimation accuracy without preparing a large amount of document data. [Explanation of symbols]
[0090] S Estimation system, N Network, 10 Learning device, 11, 21 Control unit, 12, 22 Memory unit, 13, 23 Communication unit, 14, 24 Operation unit, 15, 25 Display unit, 100 Data storage unit, 101 Acquisition unit, 102 Extraction unit, 103 Learning unit, 20 Estimation device, 200 Data storage unit, 201 Acquisition unit, 202 Extraction unit, 203 Calculation unit, 204 Selection unit, G1 Top screen, G2 FAQ screen, Q Query, T Title, A Answer text, C Category, F Input form, B Button, SR Search result, FE Feature extraction model, LDB Learning database, LDS-o Original learning dataset, LDS-p Processed learning dataset, L Contrast loss, DD, DD-m, DD-n, DD-a, DD-b FAQ document data, QD Query data, TD, TD-m, TD-n, TD-a, TD-b Title data, AD, AD-m, AD-n, AD-a, AD-b Answer data, CD, CD-m, CD-n, CD-a, CD-b Category data, TAD, TAD-m, TAD-n, TAD-a, TAD-b Combined title-answer data, TCD Combined title-category data, ACD Combined answer-category data, TACD Combined title-answer-category data, IG1, IG2, IG3, IG4, IG5 Input group, Features of QF query data, TF, TF-m, TF-n, TF-a, TF-b Features of title data, AF, AF-m, AF-n, AF-a, AF-b Features of answer data, CF-m, CF-n Features of category data, TAF, TAF-m, TAF-n, TAF-a, TAF-b Features of title-answer combined data, TCF-m,TCF-n Features of title-category combined data, ACF-m,ACF-n Features of answer-category combined data, TACF-m,TACF-n Features of title-answer-category combined data, OG1,OG2,OG3,OG4,OG5 Output group, FDB Feature database, DDS FAQ document dataset, EQD Query data to be estimated, EQF Feature of query data to be estimated
Claims
1. an acquiring means for acquiring query data and acquiring the first element, the second element, and combined data obtained by combining the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; an extraction means for extracting, using a feature extraction model, a pair of features of the query data and features of the first element of the corresponding document data, a pair of features of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data, from each of a pair of the query data and the first element of the corresponding document data, a pair of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data; a learning means for executing learning of the feature extraction model so as to increase a similarity between a pair of the feature of the query data and the feature of the first element of the corresponding document data, a similarity between a pair of the feature of the query data and the feature of the second element of the corresponding document data, and a similarity between a pair of the feature of the query data and the feature of the combined data of the corresponding document data; A learning system having the above configuration.
2. The acquiring means further acquires the first element, the second element, and the combined data from non-corresponding document data that is included in the plurality of document data and does not correspond to the query data; the extraction means further uses the feature extraction model to extract, from each of the pair of the query data and the first element of the non-corresponding document data, the pair of the query data and the second element of the non-corresponding document data, and the pair of the query data and the combined data of the non-corresponding document data, a pair of the feature of the query data and the feature of the first element of the non-corresponding document data, a pair of the feature of the query data and the feature of the second element of the non-corresponding document data, and a pair of the feature of the query data and the feature of the combined data of the non-corresponding document data; the learning means further executes learning of the feature extraction model so that a similarity between a pair of the feature of the query data and the feature of the first element of the non-corresponding document data, a similarity between a pair of the feature of the query data and the feature of the second element of the non-corresponding document data, and a similarity between a pair of the feature of the query data and the feature of the combined data of the non-corresponding document data are reduced. The learning system according to claim 1 .
3. The acquiring means further acquires the first element and the second element from a first document data included in the plurality of document data; the extraction means further extracts, from a pair of either the first element or the second element of the first document data and the other of the first element or the second element of the first document data, a pair of a feature of either the first element or the second element of the first document data and a feature of the other of the first element or the second element of the first document data, using the feature extraction model; The learning means further executes learning of the feature extraction model so as to increase a similarity between a pair of a feature of either the first element or the second element of the first document data and a feature of the other of the first element or the second element of the first document data. A learning system according to claim 1 or 2.
4. The acquiring means further acquires the first element or the second element from second document data included in the plurality of document data; the extraction means further extracts, from a pair of either the first element or the second element of the first document data and either the first element or the second element of the second document data, a pair of a feature of either the first element or the second element of the first document data and a feature of either the first element or the second element of the second document data, using the feature extraction model; the learning means further executes learning of the feature extraction model so that a similarity between a pair of a feature of either the first element or the second element of the first document data and a feature of either the first element or the second element of the second document data becomes small. The learning system according to claim 3 .
5. The acquiring means further acquires the combined data from the first document data, the extraction means further uses the feature extraction model to extract a pair of a feature of any one of the first element, the second element or the combined data of the first document data and a feature of any one of the first element, the second element or the combined data of the first document data from a pair of any one of the first element, the second element or the combined data of the first document data and the other one of the first element, the second element or the combined data of the first document data; The learning means further executes learning of the feature extraction model so as to increase a similarity between a pair of a feature of any one of the first element, the second element, or the combined data of the first document data and another feature of any one of the first element, the second element, or the combined data of the first document data. The learning system according to claim 3 .
6. a first feature amount acquiring means for acquiring a feature amount of each of the first elements of a plurality of document data, the feature amount being extracted from each of the first elements of the plurality of document data having a first element and a second element by using a feature amount extraction model; a combined feature acquisition means for acquiring a feature of each of the combined data of the plurality of document data, the feature extraction model being extracted from combined data obtained by combining the first element and the second element of each of the plurality of document data; a query feature extraction means for extracting features of the query data from the query data by using the feature extraction model; a calculation means for calculating, for each of the plurality of document data, a first similarity which is a similarity between a pair of a feature amount of the first element and a feature amount of the query data, and a combined similarity which is a similarity between a pair of a feature amount of the combined data and a feature amount of the query data; a selection means for selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated by the calculation means for each of the plurality of document data; The estimation system includes:
7. the selecting means selects one or more of the plurality of document data based on the representative value of the first similarity and the combined similarity for each of the plurality of document data. The estimation system according to claim 6.
8. a second feature acquisition means for acquiring a feature of each of the second elements of the plurality of document data, the feature being extracted from the second elements of each of the plurality of document data by using the feature extraction model; The calculation means further calculates, for each of the plurality of document data, a second similarity which is a similarity between a pair of a feature amount of the second element and a feature amount of the query data; the selecting means selects one or more of the plurality of document data further based on the second similarity calculated by the calculating means for each of the plurality of document data. The estimation system according to claim 6 or 7.
9. The feature extraction model is acquiring learning query data, and acquiring the first element, the second element, and the combined data from corresponding document data corresponding to the learning query data; using a feature extraction model, extracting a pair of the feature amounts of the training query data and the feature amounts of the first element of the corresponding document data, a pair of the feature amounts of the training query data and the feature amounts of the second element of the corresponding document data, and a pair of the feature amounts of the training query data and the feature amounts of the combined data of the corresponding document data, from each of the pair of the training query data and the first element of the corresponding document data, the pair of the training query data and the second element of the corresponding document data, and the pair of the training query data and the combined data of the corresponding document data; executing learning of the feature extraction model so as to increase a similarity between a pair of the feature amount of the training query data and the feature amount of the first element of the corresponding document data, a similarity between a pair of the feature amount of the training query data and the feature amount of the second element of the corresponding document data, and a similarity between a pair of the feature amount of the training query data and the feature amount of the combined data of the corresponding document data; is generated by The estimation system according to claim 6 or 7.
10. an acquiring step of acquiring query data and acquiring the first element, the second element, and combined data obtained by combining the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; an extraction step of extracting, using a feature extraction model, a pair of features of the query data and features of the first element of the corresponding document data, a pair of features of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data, from each of a pair of the query data and the first element of the corresponding document data, a pair of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data; a learning step of learning the feature extraction model so as to increase a similarity between a pair of the feature of the query data and the feature of the first element of the corresponding document data, a similarity between a pair of the feature of the query data and the feature of the second element of the corresponding document data, and a similarity between a pair of the feature of the query data and the feature of the combined data of the corresponding document data; A learning method having:
11. a first feature acquisition step of acquiring a feature of each of a plurality of document data, the feature being extracted from each of the first elements of the plurality of document data having a first element and a second element by using a feature extraction model; a combined feature acquisition step of acquiring features of each of the combined data of the plurality of document data, the combined data being extracted by using the feature extraction model from combined data obtained by combining the first element and the second element of each of the plurality of document data; a query feature extraction step of extracting features of the query data from the query data by using the feature extraction model; a calculation step of calculating, for each of the plurality of document data, a first similarity which is a similarity between a pair of a feature amount of the first element and a feature amount of the query data, and calculating a combined similarity which is a similarity between a pair of a feature amount of the combined data and a feature amount of the query data; a selection step of selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated in the calculation step for each of the plurality of document data; The estimation method includes:
12. an acquiring means for acquiring query data and acquiring the first element, the second element, and combined data obtained by combining the first element and the second element from corresponding document data that is included in a plurality of document data each having a first element and a second element and corresponds to the query data; an extraction means for extracting, using a feature extraction model, a pair of features of the query data and features of the first element of the corresponding document data, a pair of features of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data, from each of a pair of the query data and the first element of the corresponding document data, a pair of the query data and features of the second element of the corresponding document data, and a pair of features of the query data and features of the combined data of the corresponding document data; a learning means for executing learning of the feature extraction model so as to increase a similarity between a pair of the feature of the query data and the feature of the first element of the corresponding document data, a similarity between a pair of the feature of the query data and the feature of the second element of the corresponding document data, and a similarity between a pair of the feature of the query data and the feature of the combined data of the corresponding document data; A program that makes a computer function as a
13. a first feature amount acquiring means for acquiring a feature amount of each of the first elements of a plurality of document data, the feature amount being extracted from each of the first elements of the plurality of document data having a first element and a second element by using a feature amount extraction model; a combined feature acquisition means for acquiring a feature of each of the combined data of the plurality of document data, the feature being extracted from combined data obtained by combining the first element and the second element of each of the plurality of document data using the feature extraction model; a query feature extraction means for extracting features of the query data from the query data by using the feature extraction model; a calculation means for calculating, for each of the plurality of document data, a first similarity which is a similarity between a pair of a feature amount of the first element and a feature amount of the query data, and a combined similarity which is a similarity between a pair of a feature amount of the combined data and a feature amount of the query data; a selection means for selecting one or more of the plurality of document data based on the first similarity and the combined similarity calculated by the calculation means for each of the plurality of document data; A program that makes a computer function as a
Citation Information
Patent Citations
Method and system for similar case retrieval and storage medium
CN115293145A
Question answering device
JP2021124824A
Method for searching for item based on burying similarity, computer device, and computer program
JP2022173084A
Information processing device, information processing method, and information processing program
WO2022209313A1
