Learning system, attribute value extraction system, learning method, and program

Abstract

To enhance the accuracy of an attribute value extraction model.SOLUTION: A model storage unit (200) of a learning system (1) stores an attribute value extraction model for extracting a second attribute value from first data including the second attribute value on a first item for estimation, using a first query including at least one first attribute value associated with a first attribute on the first item. A second query acquisition unit (203) acquires a second query from which at least a part of a plurality of third attribute values associated with a second attribute on a second item for learning is excluded. A learning unit (204) learns on the attribute value extraction model on the basis of second data including a fourth attribute value on the second item and the second query.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a learning system, an attribute value extraction system, a learning method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for extracting attribute values ​​related to various items such as products or content. For example, Non-Patent Documents 1 and 2 describe a technique for inputting item data related to an item and a query including attributes related to the item to a model corresponding to a trained attribute value extraction model, and extracting information corresponding to a named entity such as an attribute value output from the model. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Qifan Wang, Li Yang, Bhargav Kanagal, Sumit Sanghai, D.Sivakumar, Bin Shu, Zac Yu, and Jon Elsas. 2020. Learning to extract attribute value from product via question answering: A multi-task approach. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 47-55, Online. ACM. [Non-Patent Document 2] Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020. A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849-5859, Online. Association for Computational Linguistics. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the accuracy of the attribute values ​​output from the models of Non-Patent Document 1 and Non-Patent Document 2 may decrease depending on the query. For example, in the technology of Non-Patent Document 1, it is assumed that an attribute name is input as a query. For example, in the technology of Non-Patent Document 2, it is assumed that a definition and description of the extraction target is input as a query. For example, when a simple word is input as a query, the models of Non-Patent Document 1 and Non-Patent Document 2 may not be able to recognize the meaning of the query, and the accuracy of the attribute values ​​output from the models may decrease.

[0005] One of the objectives of this disclosure is to improve the accuracy of the attribute value extraction model. [Means for solving the problem]

[0006] The learning system according to the present disclosure includes a model storage unit that stores an attribute value extraction model for extracting a second attribute value from first data including a second attribute value related to a first item to be estimated, using a first query including at least one first attribute value associated with a first attribute related to the first item, a second query acquisition unit that acquires a second query in which at least a portion of a plurality of third attribute values ​​associated with a second attribute related to a second item to be learned is excluded, and a learning unit that learns about the attribute value extraction model based on second data including a fourth attribute value related to the second item and the second query. Effect of the Invention

[0007] According to the present disclosure, accuracy from the attribute value extraction model is improved. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a learning system. [Diagram 2] 11A and 11B are diagrams illustrating an example of attributes and attribute values ​​acquired from product data. [Diagram 3] FIG. 13 is a diagram illustrating an example of an attribute value extraction model using a question answering model. [Figure 4] FIG. 4 is a diagram illustrating an example of an attribute value extraction model according to the present embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of a method for making an attribute value extraction model learn about incompleteness of knowledge. [Figure 6] A functional block diagram showing an example of functions realized by the learning system. [Figure 7] FIG. 2 is a diagram illustrating an example of a training database. [Figure 8] FIG. 4 is a diagram illustrating an example of an attribute database. [Figure 9] FIG. 2 is a flow chart showing an example of processing executed in the learning system. [Figure 10] FIG. 2 is a flow chart showing an example of processing executed in the learning system. [Figure 11]13 is an example of a learning system and an attribute value extraction system according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [1. Overall configuration of the learning system] An example of an embodiment of a learning system and an attribute value extraction system according to the present disclosure will be described. In this embodiment, an example is given of a learning system that also includes the functions of an attribute value extraction system, but the learning system and the attribute value extraction system may be separate systems. A case in which the learning system and the attribute value extraction system are separate systems will be described in a modified example below.

[0010] 1 is a diagram showing an example of the overall configuration of a learning system. For example, the learning system 1 includes a server 10, a learning terminal 20, and an estimated terminal 30. Each of the server 10, the learning terminal 20, and the estimated terminal 30 can be connected to an arbitrary network N such as the Internet or a LAN.

[0011] The server 10 is a server computer. The control unit 11 includes at least one processor. The storage unit 12 includes a volatile memory such as a RAM and a non-volatile memory such as a flash memory. The communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication.

[0012] The learning terminal 20 is a computer that learns an attribute value extraction model described below. For example, the learning terminal 20 is a personal computer, a smartphone, or a tablet terminal. The hardware configurations of the control unit 21, the memory unit 22, and the communication unit 23 are similar to those of the control unit 11, the memory unit 12, and the communication unit 13, respectively. The operation unit 24 is an input device such as a touch panel or a mouse. The display unit 25 is a liquid crystal display or an organic EL display.

[0013] The estimation terminal 30 is a computer that uses a trained attribute value extraction model described later. For example, the estimation terminal 30 is a personal computer, a smartphone, or a tablet terminal. The hardware configurations of the control unit 31, the storage unit 32, the communication unit 33, the operation unit 34, and the display unit 35 are the same as those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 24, and the display unit 25, respectively.

[0014] The programs stored in the storage units 12, 22, and 32 may be supplied via the network N. Each computer may also include at least one of a reading unit (e.g., a memory card slot) that reads a computer-readable information storage medium and an input / output unit (e.g., a USB port) for inputting and outputting data to and from an external device. For example, a program stored in an information storage medium may be supplied via at least one of the reading unit and the input / output unit.

[0015] Furthermore, the learning system 1 may include at least one computer. The computers included in the learning system 1 are not limited to the example in FIG. 1. For example, the estimation terminal 30 may exist outside the learning system 1. The learning system 1 may include only the server 10 and the learning terminal 20. The learning system 1 may include only either the server 10 or the learning terminal 20. The learning system 1 may include either the server 10 or the learning terminal 20, and another computer.

[0016] [2. Overview of the learning system] In this embodiment, the processing executed by the learning system 1 will be described by taking as an example a case where attribute values ​​according to product attributes are acquired from product data related to the product. A product is an object that is the subject of commercial transaction. In this embodiment, a product in an online shopping mall is taken as an example, but the product itself may be any product, for example, a product traded in an internet auction, an online flea market, or a real store.

[0017] Product data is information related to product details. For example, product data includes characters, numbers, other symbols, images, sounds, videos, or combinations of these. In this embodiment, products in an online shopping mall are taken as an example, so any text entered by a store staff member (e.g., the title of the product) corresponds to the product data. For example, the contents of the product data can be viewed from the website or application of the online shopping mall.

[0018] An attribute is a category for classifying products. An attribute may also be called a category or genre. An attribute may be defined hierarchically. When an attribute is defined hierarchically, the higher the attribute, the more abstract the meaning of the attribute, and the lower the attribute, the more specific the meaning of the attribute. Hereinafter, when simply describing an attribute, it means the name of the attribute. An attribute is expressed by letters, numbers, other symbols, or a combination of these. An attribute may be anything that can classify products from a specific perspective, and may be defined from any perspective. For example, an attribute may be information such as brand, color, material, size, function, pattern, or place of origin.

[0019] An attribute value is a value indicating a specific classification. A plurality of attribute values ​​are prepared in advance for an attribute. At least one of the plurality of prepared attribute values ​​is assigned to a product. The plurality of prepared attribute values ​​are candidates for attribute values ​​to be assigned to a product. An attribute value is expressed by letters, numbers, other symbols, or a combination of these. For example, when uploading product data for a certain product, a store staff member specifies an attribute and at least one of the plurality of prepared attribute values ​​for this attribute. The specified attribute and attribute value are associated with the product data for this product.

[0020] For example, various brand names are prepared as attribute values ​​for the attribute "brand." A product of a certain brand is assigned an attribute value indicating the brand name of the brand of the product from among the multiple attribute values ​​prepared for the attribute "brand." For example, various colors such as black, white, and yellow are prepared in advance as attribute values ​​for the attribute "color." A product that looks black is assigned an attribute value indicating "black" from among the multiple attribute values ​​prepared for the attribute "color." The same is true for other attributes; any attribute value can be prepared in advance and assigned to a product.

[0021] FIG. 2 is a diagram showing an example of attributes and attribute values ​​acquired from product data. For example, when the learning terminal 20 accesses a website of an online shopping mall, a product page P including product data is displayed on the display unit 25. For example, the product page P displays product data such as the store name, the product title, an image of the product, and a detailed description of the product. If the store staff specifies attributes and attribute values ​​when uploading the product data, the specified attributes and attribute values ​​are also displayed as product data on the product page P. The product attributes and attribute values ​​are used as indexes during searches.

[0022] If appropriate attribute values ​​according to the attributes of a product can be extracted from product data, various effects can be achieved. For example, the burden on store personnel can be reduced because the store personnel do not need to specify attribute values. For example, even if the store personnel mistakenly specifies an inappropriate attribute value, it can be replaced with an appropriate attribute value obtained from the product data. For example, new attribute values ​​can be extracted from product data even if they are not currently available as attribute values. For example, it is possible to deeply understand customer preferences or marketplaces and use them in marketing.

[0023] Therefore, the learning system 1 extracts attribute values ​​from the product data by using an attribute value extraction model for extracting attribute values ​​from the product data. The attribute value extraction model is a model that uses machine learning. Various methods can be used for the machine learning itself. For example, any of the machine learning methods, such as supervised learning, semi-supervised learning, and unsupervised learning, can be used for the attribute value extraction model.

[0024] In this embodiment, an attribute value extraction model using a question answering model (QA model) based on a model called BERT is taken as an example, but the attribute value extraction model may be a model that extracts attribute values ​​by inputting an entity, such as product data, and a query, and there is no limitation on the type of the model. For example, the attribute value extraction model may be a so-called Transformer-based model other than BERT, or a neural network model that was used before the advent of Transformer.

[0025] Fig. 3 is a diagram showing an example of an attribute value extraction model using a question answering model. The question answering model M1 and the attribute value extraction model M2 in Fig. 3 are conventional models, but the conventional models will be described in order to understand the basic mechanisms. For details of the question answering model M1 and the attribute value extraction model M2, please refer to Non-Patent Document 1 and Non-Patent Document 2 listed as prior art documents, or references described therein. Note that in these documents, models equivalent to the question answering model M1 and the attribute value extraction model M2 may be called by other names.

[0026] For example, when a news article and a query are input, the question-answering model M1 outputs a response corresponding to the query. The question-answering model M1 has learned the relationship between the news article and the query, and the response corresponding to the query. The query input to the question-answering model M1 is a question about the content of the news article. The response output from the question-answering model M1 is an answer to the question, which is the query. The question-answering model M1 estimates a portion of the news article that is appropriate as an answer, and outputs that portion.

[0027] In the example of Fig. 3, a news article N10 about the lifting of the state of emergency in Japan and a query Q11 asking when the state of emergency will be lifted are input to the question-answering model M1. The question-answering model M1 estimates that the part "September 30th" in the news article N10 is appropriate as the date when the state of emergency will be lifted, and outputs a response R12 that can identify this part. Such a question-answering model M1 can also be used for the purpose of extracting attribute values ​​from product data.

[0028] For example, the attribute value extraction model M2 is a model created by reusing the question answering model M1. When product data and a query are input, the attribute value extraction model M2 outputs an attribute value as a response to the query. The attribute value extraction model M2 has learned the relationship between the product data and the query, and the attribute values ​​included in the product data. The query input to the attribute value extraction model M2 is an attribute of the product indicated by the product data. The query can be said to be a question asking about a part of the product data that includes an attribute value corresponding to the attribute. The response output from the attribute value extraction model M2 is an attribute value corresponding to the attribute that is the query. The attribute value extraction model M2 estimates an appropriate part as an attribute value from the product data, and outputs the part.

[0029] In the example of Fig. 3, product data D20 related to the characteristics of a bag and a query Q21 including the attribute "brand" are input to the attribute value extraction model M2. In Fig. 3, an image of the product is also input to the attribute value extraction model M2, but in reality, only the text portion of the product data D20 is input to the attribute value extraction model M2. The image portion of the product data D20 may be converted into a combination of numerical values ​​such as a feature vector via a feature extractor based on a machine learning model such as a neural network, and then input to the attribute value extraction model M2.

[0030] For example, before the product data D20 is uploaded to an online shopping mall, the product data D20 and a query Q21 are input to the attribute value extraction model M2. The attribute value extraction model M2 estimates that the "BBB bag" part of the product data is appropriate as an attribute value V22 (here, a brand name) corresponding to the attribute "brand" included in the query Q21, and outputs the inference. For example, other attributes such as the attribute "color", the attribute "material", the attribute "size", or the attribute "function" can also be input as a query to the attribute value extraction model M2.

[0031] For example, since product attributes are often simple words, the query input to the attribute value extraction model M2 may be shorter and less specific than the query input to the question answering model M1. For this reason, the attribute value extraction model M2 may not be able to recognize the meaning of the query. If the attribute value extraction model M2 cannot recognize the meaning of the query, the accuracy of the output from the attribute value extraction model M2 will also decrease. In particular, in the case of an online shopping mall, when training data for the attribute value extraction model M2 is created based on product data, there tends to be more attributes for popular products and less training data for other products. Such sparseness of the training data is also one of the factors that cause the accuracy of the output from the attribute value extraction model M2 to decrease.

[0032] Taking the attribute "function" as an example, a product such as an air conditioner may have functions such as "timer" and "energy saving." A product such as a bag may have attributes such as "waterproof," "mildew-proof," or "stain-resistant." For this reason, the same attribute "function" may have various meanings. The attribute value extraction model M2 may not be able to recognize the meaning of a short and abstract query such as the attribute "function," and may not be able to output an appropriate attribute value. This is the same for other attributes, and the accuracy of the output from the attribute value extraction model M2 may decrease depending on the quality of the query.

[0033] Therefore, in this embodiment, the query is expanded by using not only the attributes of the product but also attribute values ​​previously associated with the attributes of the product as the query. This makes the meaning of the query more specific, and the attribute value extraction model M2 can easily recognize the meaning of the query, which is believed to improve the quality of the query. It is believed that if the quality of the query is improved, the problem of sparseness of the training data described above can also be solved.

[0034] Fig. 4 is a diagram showing an example of an attribute value extraction model of this embodiment. The attribute value extraction model M3 is not a conventional technology but a new technology. The attribute value extraction model M3 is similar to the attribute value extraction model M2 in that it is a model that outputs a response according to the query when product data and a query are input, but differs from the attribute value extraction model M2 in the content of the query. In the example of Fig. 4, the product data D30 is the same as the product data D20, but the query Q31 includes not only attributes but also attribute values, and is therefore more extended than the query Q21.

[0035] For example, assume that the product indicated by the product data D30 is a bag, and the product attribute input as the query Q31 is "function". As described above, the attribute "function" is a short and abstract attribute, and has various meanings related to products other than bags. For this reason, the query Q31 including the attribute "function" and the attribute value "mold prevention" prepared in advance for the bag-related attribute "function" is input so that the attribute value extraction model M3 can recognize that the meaning is a bag-related attribute. Note that, although omitted in FIG. 4, in this embodiment, all attribute values ​​associated with the attribute "function" are included in the query Q31, not just the attribute value "mold prevention".

[0036] Although the product in Fig. 4 does not have an anti-mold function, the attribute value "anti-mold" included in the query Q31 enables the attribute value extraction model M3 to recognize the meaning of the attribute "function". For example, the attribute value extraction model M3 can recognize that the attribute "function" included in the query Q31 is not a function such as an air conditioner as described above, but a function related to a bag. Therefore, the attribute value extraction model M3 can output, from the product data D30, an attribute value V32 such as "waterproof", which is a function name having an expression similar in meaning or notation to the attribute value "anti-mold" included in the query Q31.

[0037] Other than the example in Fig. 4, when another expression that is semantically or notationally close to the attribute value "waterproof" to be extracted is given as a query, such as the attribute value "stain-resistant" (function to prevent stains), the attribute value extraction model M3 can output the attribute value "waterproof" from the product data. This is also true when other queries including short and abstract attributes, such as the attribute "kind" or "type", are input to the attribute value extraction model M3, not limited to the attribute "function".

[0038] As described above, in this embodiment, the query is expanded by utilizing the attribute values ​​associated with the attributes of the product. This also applies to other attributes such as the attribute "brand", attribute "color", attribute "material", attribute "size", or attribute "function". For example, if it is possible to cover all the attributes and attribute values ​​of products traded in an online shopping mall, it is considered that it will be possible to handle any product.

[0039] However, since there are a wide variety of products in online shopping malls, it is not realistic to completely cover all attributes and attribute values. For example, among the products traded in online shopping malls, there are attributes and attribute values ​​that the administrator of the online shopping mall does not fully understand. For this reason, in actual operation, it is necessary to use incomplete knowledge that does not completely cover all attributes and attribute values.

[0040] For example, if the attribute value extraction model M3 has not learned that knowledge used in actual operation is incomplete, the attribute value extraction model M3 may estimate an attribute value assuming that an input query is completely correct. In this case, for example, the attribute value extraction model M3 may estimate an attribute value based on a simple string match with an attribute value included in the query. In this case, it is considered that unknown attributes and attribute values ​​cannot be handled. Therefore, in this embodiment, the attribute value extraction model M3 is intentionally made to learn the incompleteness of knowledge, thereby enabling it to handle actual operation.

[0041] Fig. 5 is a diagram showing an example of a method for making the attribute value extraction model M3 learn about incompleteness of knowledge. For example, the training database DB1 stores a large number of combinations of product data (product titles in Fig. 5) of products to be learned by the attribute value extraction model M3, and the attributes and attribute values ​​of the products. In the example of Fig. 5, the product data of a battery-related product includes a character string such as "ABC Battery 12V 14AH SLA Rechargeable." "ABC Battery" is the attribute value of the attribute "Brand." "14AH" is the attribute value of the attribute "Nominal Capacity."

[0042] For example, the attribute database DB2 stores many pairs of attributes and attribute values ​​that exist in the training database DB1. If there are 100 attribute values ​​such as "1ah" to "100ah" as attribute values ​​of the attribute "nominal capacity" stored in the training database DB1, pairs of the attribute "nominal capacity" and these 100 attribute values ​​are stored in the attribute database DB2. It is assumed that the attribute database DB2 also stores many pairs of other attributes.

[0043] If the attribute value extraction model M3 is trained using a query Q40 that includes all 100 attribute values ​​of the attribute "nominal capacity", the attribute value extraction model M3 may recognize that the knowledge input to it is complete. In this case, for example, the attribute value extraction model M3 may focus only on the attribute values ​​and not on the attributes, or may be unable to extract unknown attribute values. For example, when an unknown attribute is input as a query, or when an attribute with few available attribute values ​​is input as a query, the accuracy of the attribute value extraction model M3 may decrease.

[0044] Therefore, in this embodiment, two methods are used to make the attribute value extraction model M3 learn about the incompleteness of knowledge. In the first method, instead of making the attribute value extraction model M3 learn all the attribute values ​​stored in the attribute database DB2, some of the attribute values ​​are intentionally excluded and then made to learn by the attribute value extraction model M3. Hereinafter, the first method is called the knowledge dropout method. In the knowledge dropout method, the attribute value extraction model M3 can learn about the incompleteness of knowledge by intentionally reducing the number of attribute values ​​that are actually usable. In other words, the knowledge dropout method is a method for intentionally reducing the amount of knowledge to be learned by the attribute value extraction model M3.

[0045] In the second method, the attribute value extraction model M3 is trained without using any of the attribute values ​​stored in the attribute database DB2. However, if no attribute values ​​are used, there is a possibility that the accuracy will decrease as in the conventional attribute value extraction model M2 described in Fig. 3. Therefore, in this embodiment, for each product whose product data is stored in the training database DB1, two types of learning are performed: learning using the knowledge dropout method and learning in which all attribute values ​​are removed and not used.

[0046] In the second technique, a special token that indicates the availability of an attribute value is included in the query. A token is a unit of a word. In the example of FIG. 5, a token is a unit of a word separated by a space. A token may be a unit of a word separated by a special symbol instead of a space. A token may be a word that has some meaning, but looking at a certain token alone does not necessarily mean that the meaning of the token is understandable to humans.

[0047] Hereinafter, the special token used in the second method is called a knowledge token. Furthermore, the second method is called the knowledge token method. In the example of Fig. 5, the knowledge token is expressed by character strings such as [Seen] and "Unseen". When the knowledge token is [Seen], it means that the attribute value is available. When the knowledge token is [Unseen], it means that the attribute value is not available.

[0048] In this embodiment, it is assumed that an attribute stored in the attribute database DB2 always has a paired attribute value. For this reason, in the learning stage, a pseudo knowledge token of [Unseen] is associated with an attribute whose attribute value is actually available. In the example of FIG. 5, as in the input I42 to the attribute value extraction model M3 during learning, the attribute "nominal capacity" actually has 100 attribute values, but the attribute value is pseudo-assigned to the knowledge token of [Unseen] as if no attribute value exists. In addition, as in the input I41, a query including a knowledge token of [Seen] using the knowledge dropout technique (a query including an attribute value that was not excluded) is also learned by the attribute value extraction model M2.

[0049] As described above, in this embodiment, the knowledge dropout method and the knowledge token method are used to make the attribute value extraction model M3 learn about the incompleteness of knowledge. By making the attribute value extraction model M3 learn about the incompleteness of knowledge, attribute value extraction suitable for the actual operation environment can be executed, and the accuracy of the attribute value extraction model M3 is improved. The learning system 1 will be described in detail below.

[0050] [3. Functions realized by the learning system] 6 is a functional block diagram showing an example of functions realized by the learning system 1. In this embodiment, a case will be described in which a learning function related to learning the attribute value extraction model M3 is realized by the learning terminal 20. A case will be described in which an estimation function related to estimation using the learned attribute value extraction model M3 is realized by the estimation terminal 30. Hereinafter, in order to distinguish between the products, product data, attributes, attribute values, and queries used during estimation and learning, the products, product data, attributes, attribute values, and queries are given names as follows:

[0051] [Terms used in estimation] First product: The product to be processed by the trained attribute value extraction model M3 First data: Product data for the first product 1st attribute: 1st product attribute 1st attribute value: The attribute value associated with the 1st attribute First query: Query used for estimation Second attribute value: Attribute value extracted from the first data

[0052] [Terms used during study] Second product: Product used in training the attribute value extraction model M3 Secondary data: Product data for the secondary product Secondary Attribute: Secondary product attribute 3rd attribute value: The attribute value associated with the 2nd attribute Second query: Query used during training Fourth attribute value: The attribute value included in the second data that is the correct answer during learning

[0053] [3-1. Functions realized by the server] The model storage unit 100 is realized by the storage unit 12. The model storage unit 100 stores a trained attribute value extraction model M3. The trained attribute value extraction model M3 is an attribute value extraction model M3 for which training by a learning unit 204 described later has been completed. In this embodiment, a case will be described in which the estimation terminal 30 downloads and uses the trained attribute value extraction model M3 stored in the model storage unit 100, but the estimation terminal 30 may use the trained attribute value extraction model M3 online without downloading the trained attribute value extraction model M3. The model storage unit 100 may store a training database DB1 and an attribute database DB2 described later.

[0054] [3-2. Functions realized on the learning device] The model storage unit 200 is realized by the storage unit 22. The third attribute value acquisition unit 201, the probability determination unit 202, the second query acquisition unit 203, and the learning unit 204 are mainly realized by the control unit 21. These functions are examples of learning functions.

[0055] [Model memory section] The model storage unit 200 stores data necessary for learning the attribute value extraction model M3. For example, the model storage unit 200 stores an attribute value extraction model M3 for extracting a second attribute value from first data including a second attribute value related to a first product using a first query including at least one first attribute value associated with a first attribute related to a first product for estimation. The first product is an example of a first item. Therefore, the description of the first product can be replaced with the description of the first item.

[0056] An item is an object to which attributes and attribute values ​​are assigned. An item may be a tangible object such as a product, or an intangible object such as a service or data. An item may be any object and is not limited to a product. For example, an item may be content related to accommodations, content related to restaurants, e-books, videos, music, websites, or other content. For example, an item may be an introduction to a service such as a financial service or a communication service, a post on an SNS, a message such as an e-mail, or other document.

[0057] The first item is an item among the above items that is to be estimated by the trained attribute value extraction model M3. The first item is an item that is to be extracted as a second attribute value. The first item is an item from which a second attribute value has not yet been extracted from the first data. The first item may coincidentally be the same as the second item described below, but in principle, it is different from the second item.

[0058] The first data is data including some content related to the first item. The first data is data related to details of the first item. The first data may be data according to the type of the first item, and is not limited to the title of the first product. For example, if the first item is an accommodation facility, the title or description of the accommodation facility or room may correspond to the first data. For example, if the item is an electronic book, the actual data portion of the electronic book may correspond to the first data. In this embodiment, the first data includes a first character string related to a description of the first product. The attribute value extraction model M3 is a model related to natural language processing for extracting a second attribute value from the first character string.

[0059] For example, the model storage unit 200 stores the attribute value extraction model M3 before learning. The attribute value extraction model M3 before learning is an attribute value extraction model M3 with parameters of initial values. After the learning unit 204 starts learning, the model storage unit 200 stores the attribute value extraction model M3 in the middle of learning. After learning is completed, the model storage unit 200 stores the learned attribute value extraction model M3. In addition to the attribute value extraction model M3, the model storage unit 200 also stores a training database DB1 and an attribute database DB2.

[0060] FIG. 7 is a diagram showing an example of the training database DB1. The training database DB1 is a database in which data that can be training data for the attribute value extraction model M3 is stored. The training data is data related to the relationship between the second data, the second query, and the fourth attribute value. For example, the second data, the second attribute included in the second query, and the fourth attribute are associated with each other in the training database DB1. The training data includes a pair of an input portion input to the attribute value extraction model M3 during learning and an output portion to be output from the attribute value extraction model M3. In FIG. 7, only the second attribute of the second query is shown. The third attribute value included in the second query is filled in from the attribute database DB2.

[0061] The input portion of the training data has the same format as the data input to the trained attribute value extraction model M3. Therefore, the format of the first data and the first query input at the time of estimation is the same as the format of the second data and the second query input at the time of learning. In this embodiment, the first data and the first query, and the second data and the second query are each in a character format, but they may be in any format such as numbers, other symbols, or a combination of these and characters. For example, if the feature of the character is input to the attribute value extraction model M3 instead of the character, the feature of the second data and the second query may be included as the input portion of the training data.

[0062] The output part of the training data has the same format as the data output from the trained attribute value extraction model M3. Therefore, the format of the output at the time of estimation is the same as the format of the output at the time of learning. In this embodiment, a case where partial identification information described later is output will be described, but the second attribute value and the fourth attribute value themselves may be output. In this embodiment, a case where the second attribute value and the fourth attribute value are both in character format will be described, but these may be in any format such as numbers, other symbols, or a combination of these and characters. For example, if the attribute value extraction model M3 does not output a character string indicating the attribute value but outputs an ID or number that can identify the attribute value, the output part of the training data may include an ID or number that can identify the fourth attribute value.

[0063] In this embodiment, a case will be described in which products actually sold at an online shopping mall correspond to the first product and the second product. That is, product data actually uploaded to an online shopping mall corresponds to the first data and the second data. For example, the first data and the second data are the title, description, or a combination of the products sold at the online shopping mall. The first data includes characters entered by a user. Note that the training data may be manually created by an administrator of the online shopping mall, rather than being created based on products actually sold at the online shopping mall.

[0064] The second query is a question for asking the attribute value of the second product. For example, the second query may include a second attribute and a third attribute value associated with the second attribute. The second attribute is an attribute associated with the second product. For example, the second attribute includes characters indicating the attribute designated by the store staff. The second attribute may be designated by the manager of the online shopping mall, or a tool for extracting the second attribute may be used. In this embodiment, the third attribute value included in the second query is determined by a knowledge dropout method, so that only the second attribute to be included in the second query is indicated in the training database DB1. The third attribute value to be included in the second query is obtained from the attribute database DB2 described later.

[0065] The fourth attribute value is a correct attribute value for the second product. The fourth attribute value may be any one of a plurality of third attribute values ​​associated with the second attribute, or may be an attribute value that does not exist among the plurality of third attribute values. For example, the fourth attribute value includes characters indicating an attribute value designated by a store staff member. The fourth attribute value may not be designated by a store staff member, but may be designated by an online shopping mall manager, or may be extracted by a previously created attribute value extraction model M3. However, since the fourth attribute value extracted from the previously created attribute value extraction model M3 may include an incorrect fourth attribute value, the accuracy may be worse than that of the attribute value extraction model M3 trained based on training data created manually.

[0066] FIG. 8 is a diagram showing an example of the attribute database DB2. The attribute database DB2 is a database in which attribute values ​​prepared in advance for each of a plurality of attributes are stored. The attribute database DB2 can also be said to be a dictionary of attributes and attribute values. For example, the attribute database DB2 stores a large number of pairs of attributes and attribute values. An attribute stored in the attribute database DB2 can be a first attribute or a second attribute. An attribute value stored in the attribute database DB2 can be a first attribute value or a third attribute value. The attribute database DB2 may store frequencies, which will be described later.

[0067] In this embodiment, the attribute database DB2 is created based on the training database DB1. For example, the learning terminal 20 counts combinations of the second attribute included in the input part of the training data and the fourth attribute value that is the output part corresponding to the input part. Based on the counting result, the learning terminal 20 stores pairs of the second attribute and the fourth attribute value in the attribute database DB2. The learning terminal 20 may store only the fourth attribute value having a frequency equal to or higher than a threshold value for a certain second attribute in the attribute database DB2.

[0068] The attribute database DB2 can be created by any method and does not have to be created based on the training database DB1. For example, the learning terminal 20 may create the attribute database DB2 based on a product database related to products sold at an online shopping mall. For example, an administrator of the online shopping mall may manually create the attribute database DB2. For example, a list of attributes and attribute values ​​that can be assigned to products sold at an online shopping mall may be used as the attribute database DB2.

[0069] [Third attribute value acquisition section] The third attribute value acquisition unit acquires at least one third attribute value included in the second query. For example, the third attribute value acquisition unit 201 acquires a plurality of third attribute values ​​associated with the third attribute from an attribute database DB2 in which a plurality of third attribute values ​​are stored. The attribute database DB2 is an example of a second database. Therefore, the description of the attribute database DB2 can be replaced with the second database. The second database may be called by another name as long as it is a database in which various third attribute values ​​are stored.

[0070] In this embodiment, the third attribute value acquisition unit 201 acquires all the third attribute values ​​stored in the attribute database DB2 in which a plurality of third attribute values ​​are stored. For example, the third attribute value acquisition unit 201 refers to the attribute database DB2 and acquires all the third attribute values ​​associated with the second attribute of a certain second product. Taking the battery in FIG. 5 as an example of the second product, the second attribute "nominal capacity" has 100 third attribute values ​​such as "1ah" to "100ah" stored in the attribute database DB2, so the third attribute value acquisition unit 201 acquires all of these 100 third attribute values. The third attribute value acquisition unit 201 similarly acquires all the third attribute values ​​for other second attributes from the attribute database DB2.

[0071] The third attribute value acquiring unit 201 may acquire only a part of the third attribute values ​​stored in the attribute database DB2. For example, the third attribute value acquiring unit 201 may randomly acquire a predetermined number of third attribute values ​​from among all the third attribute values. For example, the third attribute value acquiring unit 201 may acquire a predetermined number of third attribute values ​​from among all the third attribute values ​​in descending order of frequency in the training database DB1. For example, the third attribute value acquiring unit 201 may acquire third attribute values ​​whose frequency in the training database DB1 is equal to or greater than a threshold value from among all the third attribute values.

[0072] For example, the third attribute value acquiring unit 201 may acquire the third attribute value from the training database DB1 or another database, instead of acquiring the third attribute value from the attribute database DB2. The third attribute value may be specified by an administrator of an online shopping mall. For example, when the administrator specifies the third attribute value one by one during learning of the attribute value extraction model M3, the third attribute value acquiring unit 201 may acquire the third attribute value specified one by one. When only one third attribute value is available, the third attribute value acquiring unit 201 may acquire only one third attribute value.

[0073] [Probability determination section] The probability determination unit 202 determines the probability that the third attribute value is excluded. This probability is the probability that the third attribute value included in the tentative second query will not be included in the final second query in the knowledge dropout method. This probability may be a common probability for all the third attribute values ​​or may be determined randomly, but in this embodiment, it is assumed to be a probability according to the frequency of the third attribute value.

[0074] The frequency is the frequency with which the third attribute value is used in an actual product. In this embodiment, the frequency of a certain third attribute value is the frequency with which the third attribute value appears in the training database DB1. The frequency can also be referred to as the number of occurrences in the training database DB1. Although omitted in FIG. 8, the attribute database DB2 also stores the frequency of the third attribute value. For example, the learning terminal 20 tallies the frequency with which the third attribute value appears in the training database DB1, and stores it in the attribute database DB2.

[0075] For example, the probability determination unit 202 determines the probability of each third attribute value based on the frequency of the third attribute value in a training database DB1 in which a plurality of second data are stored. The training database DB1 is an example of a first database. Therefore, the description of the training database DB1 can be replaced with the first database. The first database may be called by another name as long as it is a database in which the third attribute values ​​of various second products are stored. For example, a product database of an online shopping mall may correspond to the first database. That is, the frequency in another database may be used.

[0076] For example, the probability determination unit 202 determines the probability of the third attribute value such that the higher the frequency of the third attribute value, the lower the probability of the third attribute value. Conversely, the probability determination unit 202 determines the probability of the third attribute value such that the lower the frequency of the third attribute value, the higher the probability of the third attribute value. In this way, it becomes difficult to exclude a frequently used third attribute value, making it easier to have the attribute value extraction model M3 learn the frequently used third attribute value.

[0077] The data indicating the relationship between the frequency and the probability is stored in advance in the model storage unit 200. This data may be in any format, for example, in the form of a mathematical formula or a table. This data may be a part of a model using machine learning or a program. The probability determination unit 202 determines the probability of the third attribute value so that the probability is associated with the frequency of the third attribute value.

[0078] For example, the probability determination unit 202 may determine the probability of each third attribute value based on a predetermined exclusion rate and the frequency of the third attribute value. The exclusion rate is a default probability. The exclusion rate may vary depending on the third attribute value, but in this embodiment, the exclusion rate is common to all third attribute values. The exclusion rate may be specified by an administrator of an online shopping mall, or may be dynamically determined based on an external database.

[0079] For example, if the exclusion rate is r and the frequency of a certain third attribute value is nv, the probability determination unit 202 determines the probability of this third attribute value as r nv It should be noted that the method of calculating the probability may be other calculation methods and is not limited to the example of this embodiment. For example, the probability determination unit 202 may determine the value obtained by dividing the exclusion rate r by the frequency nv of a certain third attribute value as the probability of this third attribute value. For example, the probability determination unit 202 may determine the value obtained by subtracting the frequency nv of a certain third attribute value from the exclusion rate r as the probability of this third attribute value. For example, the probability determination unit 202 may determine the probability without using the exclusion rate r.

[0080] [Second query acquisition part] The second query acquisition unit 203 acquires a second query. In this embodiment, in order to express incompleteness of knowledge, third attribute values ​​that are actually available for learning are intentionally excluded, so the second query acquisition unit 203 acquires a second query in which at least some of the multiple third attribute values ​​associated with the second attribute related to the second product for learning are excluded.

[0081] Excluding the third attribute value means not including the third attribute value in the second query. In other words, intentionally not including a third attribute value that is actually usable for learning in the second query corresponds to excluding the third attribute value. Although the third attribute value exists in the attribute database DB2, not including it in the second query corresponds to excluding the third attribute value. Exclusion of the third attribute value can also be referred to as disabling or deleting the third attribute value.

[0082] In this embodiment, the second query includes the second attribute and excludes at least a part of the multiple third attribute values. That is, the second query includes both the second attribute and the third attribute value. The second query may not include the second attribute and may include the third attribute values ​​that are not excluded from the multiple third attribute values. If a certain number of third attribute values ​​are available and the meaning of the second query can be recognized to some extent based on only the third attribute values, the second query may not include the second attribute.

[0083] If the query Q40 in FIG. 5 corresponds to the second query, it includes a knowledge token, an attribute token corresponding to the second attribute, an attribute value token corresponding to the third attribute value, and an SEP token. The SEP token is a special token indicating a separation. For example, there is an SEP token indicating a separation between the second data and the second attribute (in FIG. 5, the SEP token between the title token and the knowledge token), an SEP token indicating a separation between the second attribute and the third attribute value (in FIG. 5, the SEP token between the attribute token and the attribute value token), and an SEP token indicating a separation between the third attribute values ​​(in FIG. 5, the SEP token between the attribute value tokens). Note that the CLS token in FIG. 5 is a special token placed at the beginning. The hidden state corresponds to a memory for holding calculations inside the attribute value extraction model M3. The hidden state may hold information such as an embedded expression.

[0084] In the knowledge dropout method, the second query acquisition unit 203 excludes only a portion of the multiple third attribute values. The second query acquisition unit 203 acquires a second query including the remaining third attribute values ​​that have not been excluded from the multiple third attribute values. For example, if there are k third attribute values ​​(k is an integer equal to or greater than 2) associated with a certain second attribute, excluding 1 or more but less than k third attribute values ​​from the k third attribute values ​​corresponds to excluding only a portion of the multiple third attribute values. The number or ratio of the excluded third attribute values ​​may be determined in advance or may change dynamically.

[0085] In this embodiment, the second query acquisition unit 203 determines, for each third attribute value, whether or not to exclude the third attribute value based on a probability associated with the third attribute value. The second query acquisition unit 203 excludes only a portion of the multiple third attribute values ​​that is determined to be excluded based on the probability. For example, the second query acquisition unit 203 determines, for each third attribute value, whether or not to exclude the third attribute value based on a probability associated with the third attribute value according to the frequency of the third attribute value.

[0086] In the example of the probability determination unit 202 described above, the probability of the third attribute value is r nv The second query acquisition unit 203 calculates the probability r nv Whether or not to exclude the third attribute value is determined based on the third attribute value. Various lottery algorithms can be used for this determination. The second query acquisition unit 203 excludes the third attribute value determined to be excluded from the second query so that the third attribute value is not included in the second query. The second query acquisition unit 203 does not exclude the third attribute value from the second query so that the third attribute value not determined to be excluded is included in the second query.

[0087] In this embodiment, since all the third attribute values ​​are acquired for the time being, the second query acquisition unit 203 excludes only a portion of all the third attribute values. The second query acquisition unit 203 acquires a second query including the remaining third attribute values ​​that have not been excluded from all the third attribute values. For example, the second query acquisition unit 203 acquires a tentative second query including all the third attribute values ​​for the time being. The second query acquisition unit 203 excludes the third attribute values ​​that are determined to be excluded from the tentative second query, and acquires a final second query.

[0088] For example, the second query acquisition unit 203 acquires a second query including a plurality of third attribute values. This second query is a tentative second query. The second query acquisition unit 203 may replace a portion of the plurality of third attribute values ​​included in the second query with another value, thereby excluding the portion of the third attribute values. The other value may be a predetermined value, for example, a predetermined character string. The other value may be called padding. The other value may be a character string that has no particular meaning. Here, the other value is assumed to be a character string such as [PAD].

[0089] For example, the second query after the excluded third attribute value is replaced with another value is the final second query. In the example of input I41 in FIG. 5, third attribute values ​​such as "1ah", "4ah", and "5ah" are excluded. In the example of FIG. 5, a case is shown in which the excluded third attribute value is deleted from the tentative third query without being replaced with another value, but, for example, another value for replacing the excluded "1ah" may be inserted before "2ah" in input I41. Similarly, another value for replacing the excluded "4ah" and "5ah" may be inserted between "3ah" and "6ah".

[0090] The second query described above is a second query used in the knowledge dropout method. In the knowledge token method, the second query acquisition unit 203 acquires a first type of second query including a knowledge token of [Seen] indicating that the third attribute value is available and at least a part of the multiple third attribute values. The knowledge token of [Seen] is an example of first identification information. Therefore, the description of the knowledge token of [Seen] can be read as first identification information.

[0091] The first identification information only needs to indicate that the third attribute value is available, and any information other than the knowledge token of [Seen] can be used. For example, it may be a character string other than [Seen], or may be a numerical value or other symbol. However, the token used as the first identification information is not included in the title, attribute name, and attribute value. Since it is not known what title, attribute name, and attribute value will be input when making an estimation using the trained attribute value extraction model M3, the first identification information is made to be known as the first identification information by enclosing it in [], etc. The identifier indicating that it is the first identification information is not limited to [] and may be another symbol. The second query of the first type is a second query including the first identification information. The second query of the first type is a second query including the third attribute value. In the example of FIG. 5, the part of the input I41 after [Seen] corresponds to the second query of the first type.

[0092] In addition, in the knowledge token method, the second query acquisition unit 203 acquires a second query of a second type including second identification information indicating that the third attribute value is not available and excluding all of the multiple third attribute values. The knowledge token of [Unseen] is an example of second identification information. Therefore, the description of the knowledge token of [Unseen] can be read as the second identification information.

[0093] The second identification information only needs to indicate that the third attribute value is not available, and any information other than the knowledge token of [Unseen] can be used. For example, it may be a character string other than [Unseen], or may be a numerical value or other symbol. The second identification information is also made recognizable as the second identification information by enclosing it in [ ], as with the first identification information. The second query of the second type is a second query including the second identification information. In the example of FIG. 5, the portion of the input I42 after [Unseen] corresponds to the second query of the second type. In the example of FIG. 5, the second query of the second type includes only the knowledge token and the second attribute.

[0094] The knowledge tokens [Seen] and [Unseen] may be inserted at any positions, and the positions at which these knowledge tokens are inserted are not limited to the example of Fig. 5. For example, these knowledge tokens may be inserted after the second attribute or after the third attribute value. The knowledge tokens may be inserted at predetermined positions.

[0095] [Learning Department] The learning unit 204 learns the attribute value extraction model M3 based on the second data including the fourth attribute value related to the second product and the second query. The second product is an example of the second item. Therefore, the description of the second product can be read as the second item. The second item is an item used in learning the attribute value extraction model M3 among the items described in the description of the first item. The second item is an item for which the correct fourth attribute value has been identified. The second item is an item used as training data for the attribute value extraction model M3.

[0096] The second data is data including some content related to the second item. The second data is data related to details of the second item. The second data may be data according to the type of the second item, and is not limited to the title of the second product. For example, if the second item is an accommodation facility, the title or description of the accommodation facility or room may correspond to the second data. For example, if the item is an electronic book, the actual data portion of the electronic book may correspond to the second data. In this embodiment, the second data includes a second character string related to a description of the second product.

[0097] For example, the learning unit 204 learns the attribute value extraction model M3 so that an output portion of training data is output when an input portion of training data is input. The learning is to adjust the parameters of the attribute value extraction model M3. The learning itself can use various algorithms according to the attribute value extraction model M3, and may take the form of fine tuning performed on a pre-trained Transformer-based model such as BERT, and an algorithm such as backpropagation or gradient descent may be used to find optimal parameters for other models.

[0098] In this embodiment, the learning unit 204 performs learning so that when the second data and the second query are input to the attribute value extraction model M3, the attribute value extraction model M3 outputs a start point Pb and an end point Pe that can identify the part of the fourth attribute value in the second data. The start point Pb and the end point Pe are examples of partial identification information. Therefore, the part explaining the start point Pb and the end point Pe can be read as partial identification information. In the example of FIG. 5, the part of the fourth attribute value in the inputs I41 and I42 is the part "100Ah". For example, when a character is used as a unit of a token, learning is performed so that the part "1" becomes the start point Pb and the part "h" becomes the end point Pe. For example, when a word is used as a unit of a token and "100Ah" is divided into "100" and "Ah", the part "100" becomes the start point Pb and the part "Ah" becomes the end point Pe. For example, if "100Ah" is divided into "100", "##A", and "##h", with subwords as the unit of tokens, the "100" part will be the starting point Pb, and the "##h" part will be the end point Pe. In this case, the "##" attached to the subword means that the subword was connected to the previous subword in the original string. As shown in Figure 5, learning is performed so that slight variations in spelling, such as "AH", "ah", and "Ah", can be absorbed.

[0099] The partial identification information may be information that can identify a portion of the second data that includes the fourth attribute value. For example, a pair of information indicating the number of the character immediately before the fourth attribute value and information indicating the number of the character immediately after the fourth attribute value may correspond to the partial identification information. For example, the partial identification information may include information that can identify not only the start point Pb and the end point Pe of the fourth attribute value, but also characters in between. The attribute value extraction model M3 may output the fourth attribute value itself, instead of the start point Pb and the end point Pe. In this case, the learning unit 204 learns the attribute value extraction model M3 so that the fourth attribute value included in the second data is output when the second data and the second query are input.

[0100] In the knowledge token method, the learning unit 204 performs a first learning on the attribute value extraction model M3 based on the second data and the second query of the first type. In this embodiment, the first learning is assumed to be the same as the learning in the knowledge dropout method. The learning unit 204 performs a second learning on the attribute value extraction model M3 based on the second data and the second query of the second type. The learning unit 204 performs two learnings for a certain second product: the first learning using the second query of the first type and the second learning using the second query of the second type.

[0101] In this embodiment, the learning unit 204 performs the first learning and the second learning by using multi-domain learning in which the availability of the third attribute value is regarded as a domain. In the multi-domain learning of this embodiment, the presence or absence of attribute value knowledge is regarded as a domain, and inputs including [Seen] or [Unseen] are artificially prepared as data for each domain, and learning of the attribute value extraction model M3 for the two domains is performed. For example, the learning unit 204 performs the above multi-domain learning inspired by the multi-domain learning method of "Denny Britz, Quoc Le, and Reid Pryzant. 2017. Effective domain mixing for neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 118-126, Copenhagen, Denmark. Association for Computational Linguistics."

[0102] As a result of the first learning, when a query of the same type as the first type is input, the first domain corresponding to the first type is used in the estimation. As a result of the second learning, when a query of the same type as the second type is input, the second domain corresponding to the second type is used in the estimation. The first domain is estimated by focusing on the attribute value in the query rather than the second domain. The second domain is estimated by focusing on the attribute in the query rather than the first domain. The first domain is estimated by placing more importance on knowledge than the second domain. The learning unit 204 may perform learning using the second query of the first type and the second query of the second type using a method other than multi-domain learning.

[0103] When learning of the unlearned attribute value extraction model M3 stored in the model storage unit 200 is completed, the learning unit 204 records the learned attribute value extraction model M3 in the model storage unit 200. The learning unit 204 transmits the learned attribute value extraction model M3 to the server 10. When the server 10 receives the learned attribute value extraction model M3, it records the learned attribute value extraction model M3 in the model storage unit 100.

[0104] [3-3. Functions realized by the estimated device] The model storage unit 300 is realized by the storage unit 32. The availability determination unit 301, the first query acquisition unit 302, and the second attribute value extraction unit 303 are mainly realized by the control unit 31. These functions are examples of estimation functions.

[0105] [Model memory section] The model storage unit 300 stores the trained attribute value extraction model M3. For example, when the estimation terminal 30 downloads the trained attribute value extraction model M3 from the server 10, the estimation terminal 30 records the trained attribute value extraction model M3 in the model storage unit 300. The model storage unit 300 also stores an attribute database DB2. This attribute database DB2 may also be downloaded from the server 10.

[0106] [Availability determination section] The availability determination unit 301 determines the availability of at least one first attribute value associated with the first attribute related to the first product for estimation. In this embodiment, depending on the first attribute, the first attribute value may not be available. For example, when an unknown first attribute is input as the first query, the first attribute value is not available. For this reason, the second attribute for learning is stored in the attribute database DB2 in principle, but the first attribute for estimation is not necessarily stored in the attribute database DB2. In addition, even if the first attribute is stored in the attribute database DB2, if the first attribute value does not exist, it is not necessarily associated with the first attribute.

[0107] For example, the availability determination unit 301 refers to the attribute database DB2 and determines whether or not the first attribute exists. If the availability determination unit 301 determines that the first attribute does not exist in the attribute database DB2, it determines that the first attribute value is not available. If the availability determination unit 301 determines that the first attribute exists in the attribute database DB2, it determines whether or not the first attribute value associated with the first attribute exists in the attribute database DB2.

[0108] When the availability determination unit 301 determines that the first attribute value associated with the first attribute does not exist in the attribute database DB2, it determines that the first attribute value is not available. When the availability determination unit 301 determines that the first attribute value associated with the first attribute exists in the attribute database DB2, it determines that the first attribute value is available. Note that the method of determining availability is not limited to the example of this embodiment. For example, even if the first attribute value associated with the first attribute exists, the availability determination unit 301 may determine that the first attribute value is not available if the number of the first attribute values ​​is less than a threshold value or the frequency of the first attribute value is less than a threshold value.

[0109] In this embodiment, the case where the store staff specifies the first attribute is taken as an example, but the first attribute may be acquired in some form, and may be acquired by other methods. For example, the manager of an online shopping mall may specify the first attribute. For example, the first attribute may be acquired by estimating the first attribute from the first data based on a machine learning model in which the relationship between product data and attributes is learned. In another example, another attribute related to the attribute specified by the store staff for the first product may be acquired as the first attribute. The first data is similar, and can be acquired by any method other than being specified by the store staff. For example, the first data and the first attribute may be stored in the estimation terminal 30 or another computer, and the stored first data and first attribute may be acquired.

[0110] [First query acquisition part] The first query acquisition unit 302 acquires a first query including at least one first attribute value associated with a first attribute related to a first product for estimation. In this embodiment, the first query includes, in principle, the first attribute and at least one first attribute value. However, since the first attribute value is not always available, in this case, the first query may include the first attribute but not the first attribute value. When the first attribute value is available, the first query may include only one first attribute.

[0111] In the knowledge token method, when it is determined that the first attribute value is available, the first query acquisition unit 302 acquires a third type of first query including third identification information indicating that the first attribute value is available and at least one first attribute value. The third identification information differs from the first identification information in that it indicates the availability of the first attribute value, but is similar to the first identification information in other respects. For this reason, in this embodiment, the third identification information is a character string of [Seen]. The third type differs from the first type in that it is a type of first query for estimation, but the format itself as data is similar to the first type. The details of the third identification information and the third type are similar to the details of the first identification information and the first type.

[0112] When it is not determined that the first attribute value is available, the first query acquisition unit 302 acquires a fourth type of first query including fourth identification information indicating that the first attribute value is not available. The fourth identification information differs from the second identification information in that it indicates the availability of the first attribute value, but is similar to the second identification information in other respects. For this reason, in this embodiment, the fourth identification information is a character string of [Unseen]. The fourth type differs from the second type in that it is a type of first query for estimation, but the format itself as data is similar to the fourth type. Details of the fourth identification information and the fourth type are similar to those of the second identification information and the second type.

[0113] In this embodiment, the knowledge dropout method is assumed to be used during learning, not during estimation. Therefore, when it is determined that the first attribute value is available, the first query acquisition unit 302 acquires the first query so that all available first attribute values ​​are included in the first query. The first query may include only a part of the available first attribute values. In this case, the first query acquisition unit 302 may acquire a first query including only the most frequent first attribute value. For example, the first query acquisition unit 302 may acquire a first query including a randomly selected first attribute value, a first query including all first attribute values ​​whose frequency is equal to or greater than a threshold, or a first query including a predetermined number of first attribute values ​​in descending order of frequency. For example, the first attribute value may be included in the first query in an embedded representation state.

[0114] [Second attribute value extraction part] The second attribute value extraction unit 303 extracts the second attribute value from the first data based on the first data including the second attribute value related to the first product, the first query, and the trained attribute value extraction model M3. For example, the second attribute value extraction unit 303 inputs the first data and the first query to the trained attribute value extraction model M3. The trained attribute value extraction model M3 calculates an embedding vector (embedded expression) of the first data and the first query, and outputs a start point Pb and an end point Pe of the second attribute value based on the calculated embedding vector. The second attribute value extraction unit 303 extracts a portion of the first data from the start point Pb to the end point Pe as the second attribute value.

[0115] In the knowledge token method, when it is determined that the first attribute value is usable, the second attribute value extraction unit 303 extracts the second attribute value from the first data based on the first data including the second attribute value related to the first product, the first query of the third type, and the trained attribute value extraction model M3. The second attribute value extraction unit 303 inputs the first data and the first query of the third type to the trained attribute value extraction model M3. The trained attribute value extraction model M3 outputs the start point Pb and the end point Pe of the second attribute value according to these embedding vectors.

[0116] When it is determined that the first attribute value is not available, the second attribute value extraction unit 303 extracts the second attribute value from the first data based on the first data, the fourth type first query, and the trained attribute value extraction model M3. The second attribute value extraction unit 303 inputs the first data and the fourth type first query to the trained attribute value extraction model M3. The trained attribute value extraction model M3 outputs a start point Pb and an end point Pe of the second attribute value according to these embedding vectors. Note that, when the attribute value extraction model M3 outputs the second attribute value itself instead of outputting the start point Pb and the end point Pe, the second attribute value extraction unit 303 may acquire the second attribute value output from the attribute value extraction model M3.

[0117] [4. Processing performed by the learning system] Figures 9 and 10 are flow diagrams showing an example of processing executed by the learning system 1. The processing in Figures 9 and 10 is executed by the control units 11, 21, and 31 operating in accordance with programs stored in the storage units 12, 22, and 32, respectively. When the processing in Figures 9 and 10 is executed, the training database DB1 and the attribute database DB2 are assumed to have been created in advance.

[0118] 9, the learning terminal 20 determines a second product to be learned from among the second products whose second data are stored in the training database DB1 (S1). The second product to be learned is a second product that is a target of processing from S2 onward. For example, from among the second products whose second data are stored in the training database DB1, a second product that has not yet been trained by the attribute value extraction model M3 is determined as the learning target.

[0119] The learning terminal 20 refers to the training database DB1 and acquires the second data and the second attribute of the second product to be learned (S2). The learning terminal 20 refers to the attribute database DB2 and acquires all the third attribute values ​​associated with the second attribute acquired in S2 (S3). The learning terminal 20 acquires a tentative second query including the second attribute acquired in S2 and all the third attribute values ​​acquired in S3 (S4). Although a knowledge token may be inserted at the point of S4, it is assumed that the knowledge token is not inserted yet here. An SEP token is inserted between the second attribute and the third attribute value and between the third attribute values.

[0120] The learning terminal 20 determines the probability that each third attribute value will be excluded based on the exclusion rate and the frequency of the third attribute value (S5). As described above, the exclusion rate is specified in advance. The frequency is assumed to be stored in the attribute database DB2. The learning terminal 20 determines whether each third attribute value should be excluded based on the probability of the third attribute value (S6). It is determined in S6 that some of all the third attribute values ​​will be excluded. Note that there are cases where it is determined with an extremely low probability that all the third attribute values ​​will be excluded, but this point is not taken into consideration in this embodiment.

[0121] The learning terminal 20 acquires a second query of a first type by replacing the third attribute values ​​determined to be excluded in S6 from the provisional second query acquired in S4 with other values ​​(e.g., padding tokens) (S7). In S7, the knowledge token of [Seen] is also inserted into the second query. The learning terminal 20 acquires a second query of a second type from the provisional second query acquired in S4, in which all the third attribute values ​​have been excluded (S8). In S8, the knowledge token of [Unseen] is also inserted into the second query.

[0122] The learning terminal 20 uses multi-domain learning to learn the attribute value extraction model M3 based on the second data, the first type second query, and the second type second query acquired in S2 (S9). In S9, the learning terminal 20 learns the attribute value extraction model M3 so that when the second data of the second product to be learned and the first type second query are input, the start point Pb and the end point Pe of the fourth attribute value in the second data are output. The learning terminal 20 learns the attribute value extraction model M3 so that when the second data of the second product to be learned and the second type second query are input, the start point Pb and the end point Pe of the fourth attribute value in the second data are output.

[0123] The learning terminal 20 judges whether or not learning of the attribute value extraction model M3 is completed (S10). The judgment of S10 may be performed based on a predetermined condition. For example, the learning of the attribute value extraction model M3 may be completed when all the second data stored in the training database DB1 have been used in learning, or the learning of the attribute value extraction model M3 may be completed when a certain number of second data have been used in learning.

[0124] In S10, when it is not determined that the learning of the attribute value extraction model M3 is completed (S10: N), the process returns to S1. In this case, the second product to be the next learning target is determined, and the learning of the attribute value extraction model M3 is continued. When it is determined that the learning of the attribute value extraction model M3 is completed (S10: Y), the learning terminal 20 completes the learning of the attribute value extraction model M3, and transmits the learned attribute value extraction model M3 to the server 10 (S11). When the server 10 receives the learned attribute value extraction model M3 (S12), it records the learned attribute value extraction model M3 in the storage unit 12 (S13). Thereafter, the learned attribute value extraction model M3 becomes available from the estimation terminal 30.

[0125] 10, the estimation terminal 30 downloads the trained attribute value extraction model M3 from the server 10 and records it in the storage unit 22 (S14). The estimation terminal 30 acquires the first data of the first product pre-stored in the storage unit 32 and the first attribute specified by the store staff (S15). The estimation terminal 30 refers to the attribute database DB2 and determines the availability of the first attribute value (S16).

[0126] When it is determined that the first attribute value is usable (S16: Yes), the estimated terminal 30 acquires all the first attribute values ​​associated with the first attribute (S17). The estimated terminal 30 acquires a third type first query including the first attribute acquired in S15 and all the first attribute values ​​(S18). The estimated terminal 30 extracts a second attribute value from the first data based on the first data, the third type first query, and the learned attribute value extraction model M3 (S19), and this process ends. In S19, estimation is performed using the first domain of the attribute value extraction model M3.

[0127] In S16, if it is not determined that the first attribute value is usable (S16: not possible), the estimation terminal 30 acquires a fourth type first query that does not include the first attribute value but includes the first attribute (S20). The estimation terminal 30 extracts a second attribute value from the first data based on the first data, the fourth type first query, and the learned attribute value extraction model M3 (S21), and this process ends. In S21, estimation is performed using the second domain of the attribute value extraction model M3.

[0128] According to the learning system 1 of the present embodiment, learning is performed on the attribute value extraction model M3 based on the second data including the fourth attribute value and the second query in which at least a part of the multiple third attribute values ​​associated with the second attribute of the second product is excluded. As a result, by excluding the third attribute value that is actually available for learning, the attribute value extraction model M3 can learn the incompleteness of knowledge, thereby improving the accuracy of the attribute value extraction model M3. For example, in actual operation, it is necessary to use incomplete knowledge such as the attribute database DB2 that is not completely covered. By having the attribute value extraction model M3 learn this incompleteness, even if an unknown first attribute is input or a first attribute with almost no first attribute value is input in the attribute database DB2, the attribute value extraction model M3 can extract the second attribute value from the first data. Furthermore, since the query can be expanded more than the conventional attribute value extraction model M2, the accuracy of the attribute value extraction model M3 is also improved in this respect.

[0129] In addition, in the learning system 1, the first query includes the first attribute and at least one first attribute value. The second query includes the second attribute, and at least some of the multiple third attribute values ​​are excluded. This allows not only the first attribute value but also the first attribute to be included in the first query, and allows the second attribute to be included in the second query, thereby achieving more effective query expansion. As a result, the accuracy of the attribute value extraction model M3 is improved.

[0130] Furthermore, the learning system 1 excludes only a portion of the multiple third attribute values, and obtains a second query including the remaining third attribute values ​​that have not been excluded from the multiple third attribute values. This allows a certain number of third attribute values ​​to be included in the second query, making it possible to have the attribute value extraction model M3 learn about the incompleteness of knowledge while realizing query expansion.

[0131] Furthermore, the learning system 1 determines whether or not to exclude each third attribute value based on the probability associated with the third attribute value, and excludes only a portion of the multiple third attribute values ​​that are determined to be excluded based on the probability. This makes it easier to manage how many third attribute values ​​are left unexcluded, so that the third attribute values ​​can be excluded as the administrator expects. For example, it is possible to prevent more third attribute values ​​than the administrator expects from being excluded, or to prevent fewer third attribute values ​​than the administrator expects from being excluded. As a result, the attribute value extraction model M3 can learn the unnaturalness of knowledge as the administrator intends.

[0132] Furthermore, the learning system 1 determines whether or not to exclude each third attribute value based on a probability associated with the third attribute value according to the frequency of the third attribute value. This makes it possible to make it more difficult for the third attribute value to be excluded the higher the frequency of the third attribute value, for example, making it easier to have the attribute value extraction model M3 learn frequently occurring third attribute values.

[0133] Furthermore, the learning system 1 determines the probability of each third attribute value based on a predetermined exclusion rate and the frequency of the third attribute value. The exclusion rate makes it easier to manage how many third attribute values ​​are left, so that the administrator can exclude third attribute values ​​as intended.

[0134] Furthermore, the learning system 1 excludes only a portion of all the third attribute values, and obtains a second query including the remaining third attribute values ​​that have not been excluded from all the third attribute values, thereby allowing the attribute value extraction model M3 to learn a certain number of third attribute values.

[0135] Furthermore, the learning system 1 acquires a second query including multiple third attribute values, and replaces some of the multiple third attribute values ​​included in the second query with other values ​​to exclude the some third attribute values. This allows the attribute value extraction model M3 to learn about incompleteness of knowledge by including the other values ​​in the second query.

[0136] Furthermore, the learning system 1 performs a first learning on the attribute value extraction model M3 based on the second data and the second query of the first type, and performs a second learning on the attribute value extraction model M3 based on the second data and the second query of the second type. This makes it possible to make the attribute value extraction model M3 learn the third attribute value while more efficiently learning the incompleteness of knowledge. Furthermore, compared to creating separately the attribute value extraction model M3 when the first attribute value is available and the attribute value extraction model M3 when the first attribute value is not available, they can be consolidated into one attribute value extraction model M3, thereby reducing the management burden of the attribute value extraction model M3.

[0137] Furthermore, the learning system 1 performs the first learning and the second learning by using multi-domain learning in which the availability of the third attribute value is regarded as a domain. This allows the estimation process to be switched in one attribute value extraction model M3 depending on the availability of the first attribute value at the time of actual estimation. This allows the attribute value extraction model M3 to be a hybrid model that can handle both cases where the first attribute value is available and cases where it is not available.

[0138] Furthermore, the learning system 1 performs learning so that when the second data and the second query are input to the attribute value extraction model M3, the attribute value extraction model M3 outputs partial identification information capable of identifying a portion of the fourth attribute value in the second data. This makes it possible to identify which portion of the first data is the second attribute value during actual estimation, making it easier to extract the second attribute value from the first data.

[0139] In addition, the first item is a first product for estimation, and the second item is a second product for learning, in the learning system 1. This allows for accurate extraction of attribute values ​​of products traded in online shopping malls.

[0140] Furthermore, the learning system 1 extracts the second attribute value from the first data based on the first data including the second attribute value related to the first product, the first query including the first attribute value, and the trained attribute value extraction model M3. This allows for query expansion, so that the second attribute value can be extracted with high accuracy.

[0141] Furthermore, when it is determined that the first attribute value is available, the learning system 1 extracts the second attribute value from the first data based on the first data including the second attribute value related to the first item, the first query of the third type, and the trained attribute value extraction model M3. When it is not determined that the first attribute value is available, the learning system 1 extracts the second attribute value from the first data based on the first data, the first query of the fourth type, and the trained attribute value extraction model M3. This makes it possible to extract the second attribute value with high accuracy in both cases, that is, when the first attribute value is available and when the first attribute value is not available.

[0142] [5. Modifications] The present disclosure is not limited to the above-described embodiment, and may be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0143] FIG. 11 is an example of a learning system and an attribute value extraction system in a modified example. As shown in FIG. 11, the learning system 1 and the attribute value extraction system 2 may be separate systems. The attribute value extraction system 2 is a system that can use a trained attribute value extraction model created by the learning system 1. In the example of FIG. 11, the learning system 1 includes a learning terminal 20. The attribute value extraction system 2 includes a server 10 and an estimation terminal 30. The functions of the server 10, the learning terminal 20, and the estimation terminal 30 are as described in the embodiment. The attribute value extraction system 2 may include only the estimation terminal 30 without including the server 10.

[0144] For example, in the embodiment, a case where both the knowledge dropout method and the knowledge token method are used has been described, but the learning terminal 20 may learn the attribute value extraction model M3 using only the knowledge dropout method without using the knowledge token method. In this case, the first query and the second query do not include a knowledge token. For example, the learning terminal 20 may learn the attribute value extraction model M3 using only the knowledge token method without using the knowledge dropout method. In this case, the first query may include all third attribute values ​​without excluding the third attribute value.

[0145] For example, the functions described as being realized by the learning terminal 20 may be realized by the server 10, the estimation terminal 30, or another computer, or may be shared among multiple computers. For example, the functions described as being realized by the estimation terminal 30 may be realized by the server 10, the learning terminal 20, or another computer, or may be shared among multiple computers.

[0146] [6. Notes] For example, the learning system and the attribute value extraction system can be configured as follows.

[0147] (1) a model storage unit that stores an attribute value extraction model for extracting a second attribute value from first data including a second attribute value related to a first item for estimation, by using a first query including at least one first attribute value associated with the first attribute related to the first item; a second query acquisition unit that acquires a second query in which at least a part of a plurality of third attribute values ​​associated with a second attribute related to a second item for learning is excluded; a learning unit that performs learning on the attribute value extraction model based on second data including a fourth attribute value related to the second item and the second query; A learning system including: (2) the first query includes the first attribute and the at least one first attribute value; the second query includes the second attribute and excludes at least a portion of the third attribute values; A learning system as described in (1). (3) The second query acquisition unit is excluding only a portion of the plurality of third attribute values; obtaining the second query including the remaining third attribute values ​​that have not been excluded from the plurality of third attribute values; A learning system according to (1) or (2). (4) The second query acquisition unit is determining, for each of the third attribute values, whether to exclude the third attribute value based on a probability associated with the third attribute value; excluding only a portion of the plurality of third attribute values ​​that is determined to be excluded based on the probability; (3) A learning system as described above. (5) the learning system further includes a probability determination unit that determines, for each of the third attribute values, the probability of the third attribute value based on a frequency of the third attribute value in a first database in which a plurality of the second data are stored; the second query acquisition unit determines, for each of the third attribute values, whether or not to exclude the third attribute value based on the probability associated with the third attribute value and corresponding to the frequency of the third attribute value; (4) A learning system as described above. (6) the probability determination unit determines, for each of the third attribute values, the probability of the third attribute value based on a predetermined exclusion rate and the frequency of the third attribute value; (5) A learning system as described above. (7) The learning system further includes a third attribute value acquisition unit that acquires all of the third attribute values ​​stored in a second database in which the plurality of third attribute values ​​are stored, The second query acquisition unit is Excluding only a portion of all of the third attribute values; obtaining the second query including the remaining third attribute values ​​that have not been excluded from among all the third attribute values; A learning system according to any one of (3) to (6). (8) The second query acquisition unit is obtaining the second query including the third attribute values; replacing a part of the plurality of third attribute values ​​included in the second query with another value, thereby excluding the part of the third attribute values; A learning system according to any one of (3) to (7). (9) Depending on the first attribute, the first attribute value may not be available; The second query acquisition unit is obtaining the second query of a first type including first identification information indicating that the third attribute value is available and at least a portion of the plurality of third attribute values; obtaining the second query of a second type including second identification information indicating that the third attribute value is not available and excluding all of the third attribute values; The learning unit is a first learning of the attribute value extraction model based on the second data and the second query of the first type; performing second learning on the attribute value extraction model based on the second data and the second query of the second type; A learning system according to any one of (1) to (8). (10) the learning unit performs the first learning and the second learning by using multi-domain learning in which availability of the third attribute value is regarded as a domain. A learning system as described in (9). (11) the learning unit performs the learning such that, when the second data and the second query are input to the attribute value extraction model, the attribute value extraction model outputs partial identification information capable of identifying a portion of the fourth attribute value in the second data. A learning system according to any one of (1) to (10). (12) the first item is a first commodity for estimation; the second item is a second product for learning; the first data includes a first string relating to a description of the first product; the second data includes a second string relating to a description of the second product; the attribute value extraction model is a model related to natural language processing for extracting the second attribute value from the first character string; A learning system according to any one of (1) to (11). (13) An attribute value extraction system capable of using a trained attribute value extraction model created by the learning system according to any one of (3) to (8), a first query acquisition unit that acquires a first query including at least one first attribute value associated with a first attribute related to a first item for estimation; a second attribute value extraction unit that extracts the second attribute value from the first data based on first data including a second attribute value related to the first item, the first query, and the trained attribute value extraction model; An attribute value extraction system comprising: (14) An attribute value extraction system capable of using a trained attribute value extraction model created by the learning system according to (9) or (10), an availability determination unit that determines the availability of at least one first attribute value associated with a first attribute related to a first item for estimation; a first query acquisition unit that acquires a third type of first query including third identification information indicating that the first attribute value is available and the at least one first attribute value when it is determined that the first attribute value is available, and acquires a fourth type of first query including fourth identification information indicating that the first attribute value is not available when it is not determined that the first attribute value is available; a second attribute value extraction unit that extracts the second attribute value from the first data based on first data including a second attribute value related to the first item, a first query of the third type, and the learned attribute value extraction model when it is determined that the first attribute value is available, and extracts the second attribute value from the first data based on the first data, the first query of the fourth type, and the learned attribute value extraction model when it is not determined that the first attribute value is available; An attribute value extraction system comprising: [Explanation of symbols]

[0148] 1 Learning system, 2 Attribute value extraction system, N Network, P Product page, 10 Server, 11, 21, 31 Control unit, 12, 22, 32 Memory unit, 13, 23, 33 Communication unit, 20 Learning terminal, 24, 34 Operation unit, 25, 35 Display unit, 30 Estimation terminal, P Product page, M1 Question answering model, M2, M3 Attribute value extraction model, Pb Start point, Pe End point, 100, 200 Model memory unit, 201 Third attribute value acquisition unit, 202 Probability determination unit, 203 Second query acquisition unit, 204 Learning unit, 300 Model memory unit, 301 Availability determination unit, 302 First query acquisition unit, 303 Second attribute value extraction unit, N10 News article, R12 Response, D20, D30 Product data, DB1 Training database, DB2 Attribute database, I41, I42 Input,Q11,Q21,Q31,Q40 queries,V22,V32 attribute values.

Claims

1. a model storage unit that stores an attribute value extraction model for extracting a second attribute value from first data including a second attribute value related to a first item to be estimated, by using a first query including a first attribute related to the first item and at least one first attribute value associated with the first attribute; a second query acquisition unit that acquires a second query including a second attribute related to a second item for learning and a third attribute value that has not been excluded from among a plurality of third attribute values ​​associated with the second attribute; a learning unit that causes the attribute value extraction model to learn training data including, as an input portion, second data including a fourth attribute value related to the second item and associated with the second attribute, and the second query, and including, as an output portion, information capable of identifying the fourth attribute value in the second data; A learning system including:

2. The second query acquisition unit is excluding only a portion of the plurality of third attribute values; obtaining the second query including the remaining third attribute values ​​that have not been excluded from the plurality of third attribute values; The learning system according to claim 1 .

3. The second query acquisition unit is determining, for each of the third attribute values, whether to exclude the third attribute value based on a probability associated with the third attribute value; excluding only a portion of the plurality of third attribute values ​​that is determined to be excluded based on the probability; The learning system according to claim 2 .

4. the learning system further includes a probability determination unit that determines, for each of the third attribute values, the probability of the third attribute value based on a frequency of the third attribute value in a first database in which a plurality of the second data are stored; the second query acquisition unit determines, for each of the third attribute values, whether or not to exclude the third attribute value based on the probability associated with the third attribute value and corresponding to the frequency of the third attribute value; The learning system according to claim 3 .

5. the probability determination unit determines the probability of each of the third attribute values ​​based on a predetermined exclusion rate and the frequency of the third attribute value; The learning system according to claim 4.

6. The learning system further includes a third attribute value acquisition unit that acquires all of the third attribute values ​​stored in a second database in which the plurality of third attribute values ​​are stored, The second query acquisition unit is excluding only a portion of all the third attribute values; obtaining the second query including the remaining third attribute values ​​that have not been excluded from among all the third attribute values; The learning system according to claim 2 .

7. The second query acquisition unit is obtaining the second query including the third attribute values; replacing a part of the plurality of third attribute values ​​included in the second query with another value, thereby excluding the part of the third attribute values; The learning system according to claim 2 .

8. Depending on the first attribute, the first attribute value may not be available; The second query acquisition unit is obtaining the second query of a first type including first identification information indicating that the third attribute value is available and at least a portion of the plurality of third attribute values; obtaining the second query of a second type including second identification information indicating that the third attribute value is not available, the second query excluding all of the third attribute values; The learning unit is a first learning of the attribute value extraction model based on the second data and the second query of the first type; performing second learning on the attribute value extraction model based on the second data and the second query of the second type; A learning system according to any one of claims 1 to 7.

9. the learning unit performs the first learning and the second learning by using multi-domain learning in which availability of the third attribute value is regarded as a domain. The learning system according to claim 8.

10. the learning unit performs the learning such that, when the second data and the second query are input to the attribute value extraction model, the attribute value extraction model outputs partial identification information capable of identifying a portion of the fourth attribute value in the second data. A learning system according to any one of claims 1 to 7.

11. the first item is a first commodity for estimation; the second item is a second product for learning; the first data includes a first string relating to a description of the first product; the second data includes a second string relating to a description of the second product; the attribute value extraction model is a model related to natural language processing for extracting the second attribute value from the first character string; A learning system according to any one of claims 1 to 7.

12. An attribute value extraction system capable of using a trained attribute value extraction model created by the learning system according to claim 2, a first query acquisition unit that acquires a first query including at least one first attribute value associated with a first attribute related to a first item for estimation; a second attribute value extraction unit that extracts the second attribute value from the first data based on first data including a second attribute value related to the first item, the first query, and the trained attribute value extraction model; An attribute value extraction system comprising:

13. An attribute value extraction system capable of using a trained attribute value extraction model created by the learning system according to claim 8, an availability determination unit that determines availability of at least one first attribute value associated with a first attribute related to a first item for estimation; a first query acquisition unit that acquires a third type of first query including third identification information indicating that the first attribute value is available and the at least one first attribute value when it is determined that the first attribute value is available, and acquires a fourth type of first query including fourth identification information indicating that the first attribute value is not available when it is not determined that the first attribute value is available; a second attribute value extraction unit that extracts the second attribute value from the first data based on first data including a second attribute value related to the first item, a first query of the third type, and the learned attribute value extraction model when it is determined that the first attribute value is available, and extracts the second attribute value from the first data based on the first data, the first query of the fourth type, and the learned attribute value extraction model when it is not determined that the first attribute value is available; An attribute value extraction system comprising:

14. A computer comprising: a second query acquisition step of acquiring a second query including a second attribute related to a second item for learning and a third attribute value that has not been excluded among a plurality of third attribute values ​​associated with the second attribute; a learning step of training data including, as an input portion, second data including a fourth attribute value related to the second item and the fourth attribute value related to the second attribute, and the second query, and including, as an output portion, information capable of identifying the fourth attribute value in the second data, using a first query including a first attribute related to a first item to be estimated and at least one first attribute value associated with the first attribute, to learn an attribute value extraction model for extracting the second attribute value from first data including the second attribute value related to the first item, by using the first query; Learn how to do it.

15. a second query acquisition unit that acquires a second query including a second attribute related to a second item for learning and a third attribute value that has not been excluded from among a plurality of third attribute values ​​associated with the second attribute; a learning unit that causes an attribute value extraction model that stores an attribute value extraction model for extracting the second attribute value from first data including a second attribute value related to the first item to learn training data including, as an input portion, second data including a fourth attribute value related to the second item, the fourth attribute value being related to the second attribute, the second query, the second data including the fourth attribute value different from the third attribute value, and the second query, and including, as an output portion, information capable of identifying the fourth attribute value in the second data, by using a first query including a first attribute related to a first item to be estimated and at least one first attribute value associated with the first attribute; A program that makes a computer function as a

Images