Information processing device, information processing method, and program

Abstract

It estimates the category of a product or other object with higher accuracy. [Solution] An information processing device includes a processing unit. For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, the processing unit extracts a first partial string included in the target string and extracts a second partial string from the first partial string using the category string. For each of the plurality of pieces of first input information, the processing unit determines a second partial string to be associated with the target string included in the first input information. The processing unit executes a learning process to learn first feature information representing features of the determined second partial string and second feature information representing features of a target string including the determined second partial string and calculated based on the first feature information, so as to increase the match rate between the category of the target string estimated from the second feature information and the category represented by the category string.

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] With the advancement of the Internet of Things (IoT), the utilization of purchasing data that shows who bought what is accelerating. In particular, information that shows product categories is expected to be used for new product development, marketing, customer profiling, and promotions. Traditionally, experts have often had to select the corresponding category from a vast collection of categories based on the product name, which can be an enormous burden when there are a large number of products. To address this issue, machine learning and data mining are increasingly being used. For example, technology has been proposed that extracts features from product names and infers categories based on the extracted features. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 021845 [Patent Document 2] Patent No. 6446602 [Patent Document 3] Japanese Patent Publication No. 2022-105126 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to provide an information processing device, an information processing method, and a program that can estimate the category of an object such as a product with higher accuracy. [Means for solving the problem]

[0005] An information processing device according to an embodiment includes a processing unit. The processing unit extracts a first partial string included in a target string for each of a plurality of pieces of first input information, the target string including a target and a category string representing a category to which the target belongs, and extracts a second partial string from the first partial string using the category string. The processing unit determines a second partial string to be associated with the target string included in the first input information for each of the plurality of pieces of first input information. The processing unit executes a learning process to learn first feature information representing a feature of the determined second partial string and second feature information representing a feature of a target string including the determined second partial string and calculated based on the first feature information, so as to increase a match rate between the category of the target string estimated from the second feature information and the category represented by the category string. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram of an information processing apparatus according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a data structure of a product master. [Figure 3] 10 is a flowchart of a process for extracting a substring according to an embodiment. [Figure 4] 10A and 10B are diagrams showing a specific example of a process for extracting a substring. [Figure 5] 10 is a flowchart of a graph generation process according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing a specific example of graph generation processing. [Figure 7] 10 is a flowchart of a learning process. [Figure 8] 10 is a flowchart of an estimation process. [Figure 9] FIG. 10 is a block diagram of an information processing apparatus according to a second embodiment. [Figure 10] 10 is a flowchart of a graph generation process according to the second embodiment. [Figure 11] FIG. 10 is a block diagram of an information processing apparatus according to a third embodiment. [Figure 12] FIG. 10 is a diagram showing an example of a category estimation method performed by a learning unit and an estimation unit. [Figure 13]10 is a flowchart of a learning process according to the third embodiment. [Figure 14] FIG. 10 is a block diagram of an information processing apparatus according to a fourth embodiment. [Figure 15] 10 is a flowchart of information processing according to the fourth embodiment. [Figure 16] 10A and 10B are diagrams illustrating examples of calculation of embedded expressions according to an embodiment. [Figure 17] FIG. 10 is a block diagram of an information processing apparatus according to a fifth embodiment. [Figure 18] 13 is a flowchart of an estimation process according to the fifth embodiment. [Figure 19] FIG. 1 is a hardware configuration diagram of an information processing apparatus according to first to fifth embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an information processing apparatus according to the present invention will be described in detail below with reference to the accompanying drawings.

[0008] In the following, an example will be described in which the target of category estimation is a product and the target string, which is a string representing the target (product), is a product name. The target and target string are not limited to a product and a product name.

[0009] One proposed technology for estimating product categories involves dividing a character string representing a product name into substrings and then using the substrings to estimate the category. Product names often have a small number of characters, and the accuracy of category estimation is likely to vary significantly depending on the method for selecting the substrings. For example, substrings are not specialized for product category estimation, so they may not contribute to improving the accuracy of category estimation.

[0010] A technology has also been proposed for estimating product categories using a method for extracting character string features using large language models (LLMs). However, since large language models are trained based on information such as web pages, they are not directly related to the category estimation task, and the accuracy of category estimation may be reduced.

[0011] (First embodiment) The information processing device of the first embodiment uses information about products for which a character string representing a category (category character string) has been obtained in advance to extract a substring of a product name and learns the feature quantities of the extracted substring. The information processing device of this embodiment then uses the learned feature quantities to estimate the category of a product designated as an estimation target. This allows the product category to be estimated with higher accuracy. Products for which a category string has been obtained in advance may be only a portion of a large number of products. This eliminates the need for experts or the like to select categories for a large number of products, thereby reducing the burden of category selection.

[0012] 1 is a block diagram showing an example of the configuration of an information processing device 100 according to the first embodiment. As shown in FIG. 1, the information processing device 100 includes a storage unit 120, an acquisition unit 101, an extraction unit 102, a graph generation unit 103, a calculation unit 104, a learning unit 105, an estimation unit 106, and an output control unit 111.

[0013] The storage unit 120 stores various information used by the information processing device 100. For example, the storage unit 120 stores a product master 121 and a partial character string 122. The partial character string 122 is a partial character string extracted from the product name by the extraction unit 102.

[0014] The product master 121 corresponds to product information obtained in advance. FIG. 2 is a diagram showing an example of the data structure of the product master 121. As shown in FIG. 2, the product master 121 includes a product name and a category name. The product name corresponds to a character string (target character string) representing the name of the product. The category name corresponds to a category character string representing the category of the product. The product master 121 corresponds to a plurality of input information ID1 (first input information) including a target character string representing the target and a category character string representing the category to which the target belongs.

[0015] At least a portion of the information included in the product master 121 may be subjected to predetermined preprocessing. For example, product names may be subjected to preprocessing such as standardizing to full-width characters, standardizing to uppercase characters, and other normalization processes. Product names may be displayed only partially to reduce the database capacity. Therefore, a preprocessing step may be performed on the product names to restore the omitted character strings by interpolating the product names. The interpolation results are registered as product names in the product master 121.

[0016] Each product name may be assigned one category (category name) or multiple categories. The product master 121 may be subjected to category preprocessing. For example, a category whose occurrence count in the product master 121 does not reach a specified number may be deleted from the product master 121 to be excluded from estimation targets. In addition, when multiple categories are treated as the same single category, information indicating the relationship between them may be registered.

[0017] The storage unit 120 can be configured with any commonly used storage medium, such as a flash memory, a memory card, a RAM (Random Access Memory), a HDD (Hard Disk Drive), an optical disk, etc. Some or all of the data (product master 121, partial character string 122) stored in the storage unit 120 may be stored in physically different storage media, or may be stored in different storage areas of the same physically stored medium.

[0018] Returning to the explanation of Fig. 1, the acquisition unit 101 acquires various information used in the information processing device 100. For example, the acquisition unit 101 acquires the product master 121, thresholds used in the processing of each unit (threshold for the number of products, threshold for ratio, etc.), extraction conditions for partial strings, and the number of dimensions of feature amounts (embedded expressions, etc.). The extraction conditions for partial strings are, for example, the number of characters in the partial strings to be extracted (2 to 6 characters, etc.).

[0019] The acquisition unit 101 may acquire information by any method, for example, a method of receiving information from an external device via a network, or a method of reading information from a storage medium.

[0020] The acquiring unit 101 may perform a predetermined pre-processing on the acquired information. For example, the acquiring unit 101 may perform the pre-processing on the product master 121 as described above.

[0021] The extraction unit 102 extracts one or more partial character strings from the input information ID1 for each of a plurality of input information ID1 included in the product master 121. The extraction unit 102 is configured to extract partial character strings that are useful for estimating a category.

[0022] For example, the extraction unit 102 extracts one or more partial character strings PT1 (first partial character strings) included in the product name (target character string) for each of the multiple pieces of input information ID 1. Next, the extraction unit 102 extracts one or more partial character strings PT2 (second partial character strings) from the one or more partial character strings PT1 included in the product name by using a category name (category character string).

[0023] For example, for each of one or more partial character strings PT1, the extraction unit 102 extracts, as a partial character string PT2, a partial character string PT1 having a judgment value greater than that of the other partial character strings PT1 from among one or more category names (category character strings) corresponding to product names containing the partial character string PT1. The judgment value is, for example, at least one of the number n of corresponding product names and the ratio (correctness rate) of the number n to the total number of product names containing the partial character string PT1.

[0024] FIG. 3 is a flowchart showing an example of the process of extracting a partial character string in this embodiment.

[0025] The extraction unit 102 extracts one or more partial character strings PT1, which are partial character strings of a specified number of characters (for example, 2 to 6 characters), from each product name included in the product master 121 (step S101).

[0026] The extraction unit 102 calculates the accuracy rate for each combination of the partial character string PT1 and the corresponding category name (step S102). As described above, the accuracy rate is the ratio of the number n of product names corresponding to the category name to the total number of product names that include the partial character string PT1.

[0027] The extraction unit 102 extracts one or more combinations whose accuracy rate is equal to or greater than a threshold value THA (step S103). The extraction unit 102 extracts a specified number of partial character strings PT1 included in the one or more extracted combinations in descending order of accuracy rate as one or more partial character strings PT2 (step S104), and ends the partial character string extraction process.

[0028] The method for extracting the partial character string PT1 in step S101 may be any method, but for example, the following method may be used. (EM1) How to enumerate all substrings of a specified length. (EM2) A method of utilizing substrings extracted by word segmentation methods such as MeCab. (EM3) A method of dividing a product name into one or more substrings PT1 using a predetermined character (such as a space) depending on the language used (such as Japanese or English). (EM4) A method for extracting substrings from product names and also extracting one or more substrings from category names as substring PT1.

[0029] The above (EM4) is a method that takes into consideration that information useful for estimating a category for a product name may also be contained in the category name. Two or more of the above methods may be used. In this case, the extraction unit 102 may adjust the threshold THA used when extracting the partial character string PT2 from the partial character string PT1 extracted by each of the multiple methods so that the values ​​are different for each of the multiple methods. This makes it possible to more appropriately extract the partial character string PT2 useful for estimating the category.

[0030] FIG. 4 is a diagram showing a specific example of the process of extracting a substring. FIG. 4 shows an example in which "beef" is focused on as the substring PT1. As shown in FIG. 4, it is expected that products containing "beef" will be included in many categories, such as the "yakiniku" category and the "salad" category. The extraction unit 102 determines that the accuracy rate of 45% for the "yakiniku" category, which has the largest number of applicable products, is greater than, for example, a threshold value THA=40%, and extracts "beef" as the substring PT2.

[0031] The extraction unit 102 also performs this process on other partial character strings PT1 (for example, "grilled meat" or "salad") to extract one or more partial character strings PT2. As shown in the flowchart of FIG. 3, the extraction unit 102 may further extract a specified number of partial character strings PT2 from the extracted partial character string PT2 in descending order of accuracy rate.

[0032] The extraction unit 102 extracts a partial string PT2 for each product name included in the product master 121 by the above process, and stores the extracted partial string PT2 in the storage unit 120 as a partial string 122. The stored partial string PT2 (partial string 122) is referenced, for example, in the graph generation process by the graph generation unit 103 and the feature (embedded expression) calculation process by the calculation unit 104.

[0033] Returning to the explanation of Fig. 1, the graph generation unit 103 generates a graph including product names and one or more partial character strings PT2 included in the product names. For example, for each of a plurality of pieces of input information ID1 included in the product master 121, the graph generation unit 103 determines one or more partial character strings PT2 to be associated with the product names included in the input information ID1. The graph generation unit 103 generates a graph including the determined partial character strings PT2 and the product names.

[0034] FIG. 5 is a flowchart showing an example of graph generation processing in the first embodiment.

[0035] The graph generating unit 103 identifies the product name including the partial string PT2 for each of the extracted one or more partial strings PT2 (step S201). As a result, for each product name, the partial string PT2 included in the product name is identified. The graph generating unit 103 may identify the partial string PT2 included in the product name for each of one or more product names included in the product master 121 from the partial strings 122 stored in the storage unit 120.

[0036] The graph generating unit 103 determines, for each product name, a partial character string PT2 to be included in the graph from among the partial character strings PT2 included in the product name (step S202). The graph generating unit 103 generates a graph including the determined partial character strings and the product name (step S203), and ends the graph generation process.

[0037] Fig. 6 shows a specific example of graph generation processing. Fig. 6 shows an example of graph generation processing for a product name 601, "shrimp and broccoli tartare." Assume that at least seven substrings included in the substring group 602 are extracted as substring PT2.

[0038] For example, when product name 601 is identified as a product name that includes each of seven partial strings, graph generation unit 103 determines the partial string as a partial string to be included in the graph for product name 601. Partial string group 611 shows an example of five partial strings determined in this manner. Graph generation unit 103 generates graph 612, which uses the determined partial strings and product names as nodes and includes edges connecting each of the partial strings to the product names.

[0039] The method for determining the substrings to be included in the graph is not limited to the above. For example, the graph generating unit 103 may determine the substrings to be included in the graph using the following method. (DM1) A method for determining, when a part of a substring is included in a product name, that substring as a substring to be included in a graph. (DM2) A method of determining whether or not strings match after converting to unify character types (kanji, hiragana, katakana, etc.). For example, in a string that contains kanji, hiragana, and katakana, converting the kanji and katakana to hiragana and then determining whether or not they match.

[0040] Returning to the explanation of Fig. 1, the calculation unit 104 calculates feature information (feature quantity) of the product name. The feature quantity may be in any format, for example, an embedded representation in which the product name is represented by a vector with a specified number of dimensions.

[0041] The calculation unit 104 calculates embedded expressions F2 (second feature information) of product names included in the graph, using embedded expressions F1 (first feature information) of partial strings included in the graph generated by the graph generation unit 103. For example, the calculation unit 104 calculates embedded expressions F2 of product names included in the graph by adding embedded expressions F1 of one or more partial strings included in the graph.

[0042] The calculation unit 104 may calculate the embedded representation of the product name by utilizing a graph analysis technique such as GNN (Graph Neural Networks). For example, the calculation unit 104 may calculate the average value of the embedded representations F1 of the substrings included in the graph as the embedded representation F2 of the product name included in the graph.

[0043] The method for calculating the embedded expression F2 is not limited to the method using a graph, and any other method may be used as long as it can calculate the embedded expression F2 from the embedded expression F1 of the substring. If a graph is not used, the graph generating unit 103 may not be provided.

[0044] At the start of learning, an initial value may be set for the embedded representation F1 of the substring. The initial value may be, for example, a value selected at random. When estimation is performed using learned feature information (embedded representation), the learned embedded representation F1 is used to calculate the embedded representation F2 of the product name.

[0045] The learning unit 105 executes a learning process to learn the embedded expression F1 of the substring and the embedded expression F2 of the product name. For example, the learning unit 105 executes the learning process so as to increase the coincidence rate between the category estimated from the embedded expression F2 and the category represented by the category name (category string) corresponding to the product name in the product master 121. The category represented by the category name corresponding to the product name in the product master 121 corresponds to the correct category.

[0046] Any method may be used to estimate a category from the embedded representation F2, but for example, a method using a pre-trained estimation model EMA may be applied. The estimation model EMA is, for example, a model that is trained to input an embedded representation and output the probability of each of multiple categories. The category probability corresponds to a value that represents the likelihood that a product corresponding to the embedded representation belongs to that category. The category with the highest probability corresponds to the category estimation result (estimated value). The estimation model EMA may also be a model that outputs the most likely category from among multiple categories.

[0047] The data representing the probability of each of the multiple categories can be interpreted as a vector containing elements whose element values ​​are the probabilities for the number of categories. In other words, the estimation model EMA can be interpreted as a model that outputs an embedded representation different from the embedded representation F2, whose number of dimensions is the number of categories.

[0048] In the learning process, the learning unit 105 updates the value of the embedded expression F1 of the partial string from which the embedded expression F2 is calculated, for example, according to backpropagation, so as to increase the coincidence rate between the category estimated from the embedded expression F2 and the category represented by the category name (category string) corresponding to the product name in the product master 121. Updating the value of the embedded expression F1 of the partial string can be interpreted as being equivalent to updating the value of the embedded expression F2 calculated using the embedded expression F1.

[0049] FIG. 7 is a flowchart showing an example of the learning process performed by the learning unit 105.

[0050] The learning unit 105 initializes the value of the embedded expression F1 for each of one or more substrings PT2 (step S301). The calculation unit 104 calculates the embedded expression F2 for each product name by propagating the value of the embedded expression F1 according to the structure of the graph generated by the graph generation unit 103 (step S302). Propagating the value of the embedded expression F1 means, for example, adding the values ​​of the embedded expressions F1 of the substrings corresponding to one or more substring nodes included in the graph, thereby calculating the embedded expression F2 for the product name node connected to the node.

[0051] The learning unit 105 inputs the calculated embedded representation F2 of the product name into the estimation model EMA to estimate the category (step S303). For example, the learning unit 105 outputs the category with the highest probability that the estimation model EMA will output as the estimation result.

[0052] The learning unit 105 updates the value of the embedded expression of each substring so that the estimated category matches the correct category (step S304).

[0053] The learning unit 105 determines whether to end learning (step S305). If it is determined not to end learning (step S305: No), the learning unit 105 returns to step S302 and repeats the process. If it is determined to end learning (step S305: Yes), the learning unit 105 ends the learning process. For example, the learning unit 105 determines to end learning when the match rate becomes equal to or greater than a threshold THB (match rate threshold) or when the number of times steps S302 to S305 have been repeated exceeds a threshold THC (number of times threshold).

[0054] Returning to the explanation of Fig. 1, the estimation unit 106 executes a category estimation process using the learned embedded representation. For the estimation process, the input information ID2 containing the product name for which a category is to be estimated is subjected to the same process as during learning, up to the calculation of the embedded representation F2.

[0055] For example, the acquisition unit 101 acquires input information ID2. The extraction unit 102 extracts one or more partial character strings PT3 (third partial character strings) included in the product name included in the input information ID2. The calculation unit 104 calculates an embedded expression F2 representing the characteristics of the product name included in the input information ID2 using an embedded expression F1 representing the characteristics of the extracted partial character string PT3.

[0056] The estimation unit 106 executes an estimation process to estimate a category of a product name using the embedded expression F2 calculated in this manner. For example, the estimation unit 106 estimates a category from the embedded expression F2 using the same estimation model as the estimation model EMA used by the learning unit 105.

[0057] FIG. 8 is a flowchart showing an example of the estimation process performed by the estimation unit 106.

[0058] The extraction unit 102 extracts partial character strings from the product names to be estimated (step S401). The graph generation unit 103 generates a graph including the product names and the extracted partial character strings (step S402). The calculation unit 104 calculates the embedded expression F2 for each product name by propagating the value of the embedded expression F1 according to the structure of the graph generated by the graph generation unit 103 (step S403).

[0059] The estimation unit 106 estimates a category by inputting the calculated embedded representation F2 of the product name into the estimation model EMA (step S404). For example, the estimation unit 106 outputs the category with the highest probability that the estimation model EMA will output as the estimation result.

[0060] Returning to the explanation of Fig. 1, the output control unit 111 controls the output of various information used in the information processing device 100. For example, the output control unit 111 outputs the category estimated by the estimation unit 106. Any method may be used to output the information, and examples of applicable methods include a method of displaying the information on a display device and a method of transmitting the information to an external device via a network.

[0061] At least a part of each of the above units (acquisition unit 101, extraction unit 102, graph generation unit 103, calculation unit 104, learning unit 105, estimation unit 106, and output control unit 111) may be realized by one or more processing units. Each of the above units is realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or may realize two or more of the units.

[0062] Furthermore, the information processing device 100 may be physically configured as one device, or may be physically configured as multiple devices. For example, the information processing device 100 may be constructed in a cloud environment. Furthermore, each unit in the information processing device 100 may be distributed across multiple devices. For example, the information processing device 100 (information processing system) may be configured to include a device (e.g., a learning device) that has functions necessary for learning (e.g., the learning unit 105) and a device (e.g., an estimation device) that has estimation functions (e.g., the estimation unit 106).

[0063] In this way, the information processing device of the first embodiment learns the feature of a substring using a category name (category string) obtained in advance, and estimates the category of a product designated as an estimation target using the learned feature, thereby enabling more accurate estimation of the product category.

[0064] (Second embodiment) In the first embodiment, a graph is generated so as to include all of the partial strings PT2 included in product names out of the partial strings 122 (partial strings PT2) stored in the storage unit 120. Multiple partial strings PT2 may include overlapping strings. In the example of FIG. 6, the partial strings "Bro" and "Broccoli" included in the partial string group 611 include the overlapping string "Bro." Furthermore, the partial strings "Tarutaru" and "Taru" included in the partial string group 611 include the overlapping string "Taru." Because overlapping strings may be included in this way, if a graph is generated so as to include all of the partial strings PT2 included in product names, the strings included in the graph will be redundant, which may reduce the accuracy of category estimation.

[0065] Therefore, the information processing device of the second embodiment determines substrings to be included in the graph so as not to be redundant. Fig. 9 is a block diagram showing an example of the configuration of an information processing device 100-2 of the second embodiment. As shown in Fig. 9, the information processing device 100-2 includes a storage unit 120, an acquisition unit 101, an extraction unit 102, a graph generation unit 103-2, a calculation unit 104, a learning unit 105, an estimation unit 106, and an output control unit 111.

[0066] In the second embodiment, the function of the graph generation unit 103-2 is different from that in the first embodiment. Other configurations and functions are the same as those in FIG. 1, which is a block diagram of the information processing device 100 in the first embodiment, so the same reference numerals are used and the description here is omitted.

[0067] For each product name, the graph generation unit 103-2 determines the substrings PT2 to be included in the graph so as not to be redundant. For example, for each product name included in a plurality of pieces of input information ID1, the graph generation unit 103-2 determines, from among combinations of one or more substrings PT2 included in the product name, the substrings PT2 included in a combination in which the number of characters in the product name that match the characters included in the substring PT2 is greater than in other combinations and the number of substrings PT2 in the combination is smaller than in other combinations. This determination method can be interpreted as a method of determining substrings PT2 so as to cover the product name with as few combinations of substrings as possible.

[0068] More specifically, the graph generating unit 103-2 may determine the partial character string PT2 as follows. That is, the graph generating unit 103-2 executes a determination process to determine, from the scanning start position (at least one of the beginning and end) of the product name, a partial character string PT2 from one or more partial character strings PT2 in which the number of characters in the product name that match the characters contained in the partial character string PT2 is greater than the number of other partial character strings PT2. The graph generating unit 103-2 removes the determined partial character string PT2 from the product name, sets the resulting character string as a new product name, and repeats the determination process. The determination process is repeatedly executed, for example, until none of the partial character strings PT2 are contained in the product name.

[0069] FIG. 10 is a flowchart showing an example of graph generation processing in the second embodiment.

[0070] The graph generating unit 103-2 identifies the product name including the partial string PT2 for each of the extracted one or more partial strings PT2 (step S501). For each product name included in the product master 121, the graph generating unit 103-2 determines as few partial strings as possible that cover the product name as partial strings PT2 to be included in the graph (step S502). The graph generating unit 103-2 generates a graph including the determined partial strings and the product name (step S503), and ends the graph generation process.

[0071] A specific example of graph generation processing in this embodiment will be described. As in Fig. 6, an example of graph generation processing for the product name 601, "shrimp and broccoli tartare," will be shown. The scanning start position is assumed to be the beginning of the product name 601.

[0072] In this case, the substring "shrimp" is determined as the first substring PT2. The character string obtained by removing the substring "shrimp" from product name 601 is set as the new product name. If the particle "no" is also removed, the new product name becomes "Broccoli Tartar."

[0073] When the new product name "Broccoli Tartare" is scanned from the beginning, the candidates for matching substring PT2 are the substrings "Bro" and "Bro". Of these two substrings, the substring "Broccoli" has the largest number of characters in the product name that match the characters contained in substring PT2, and is determined to be the next substring PT2. Similarly, of the substrings "Tartar" and "Taru", the substring "Tartar" has the largest number of characters in the product name that match the characters contained in substring PT2, and is determined to be the next substring PT2.

[0074] In the above example, the scanning start position is assumed to be the beginning, but the scanning start position may be the end, or both the beginning and the end. In the latter case, for example, a graph is generated that includes both a substring PT2 determined with the scanning start position as the beginning and a substring PT2 determined with the scanning start position as the end.

[0075] In this way, the information processing device of the second embodiment can determine the substrings to be included in the graph so as not to be redundant, thereby enabling more accurate estimation of the category.

[0076] (Third embodiment) In the above-described embodiment, the embedded representation of each product name is calculated based on the relationship between the product name and the substring. However, the embedded representation of the product name can also be calculated by other methods. A typical method is to use a language model such as LLM. The information processing device of the third embodiment further uses the feature calculated using the language model to learn the feature, and then uses the learned feature to estimate a category.

[0077] 11 is a block diagram showing an example of the configuration of an information processing device 100-3 according to the third embodiment. As shown in FIG. 11, the information processing device 100-3 includes a storage unit 120, an acquisition unit 101, an extraction unit 102, a graph generation unit 103, a calculation unit 104-3, a learning unit 105-3, an estimation unit 106-3, and an output control unit 111.

[0078] In the third embodiment, the functions of a calculation unit 104-3, a learning unit 105-3, and an estimation unit 106-3 are different from those in the first embodiment. The other configurations and functions are the same as those in the block diagram of the information processing device 100 in the first embodiment shown in FIG. 1, and therefore the same reference numerals are used and the description thereof will be omitted here.

[0079] The calculation unit 104-3 differs from the calculation unit 104 of the above embodiment in that it further includes a function of calculating an embedded expression F3 of a product name (third feature information) using a language model. The language model is a model trained to receive an input of a product name and output an embedded expression F3 of the product name.

[0080] The learning unit 105-3 performs a learning process so as to increase the rate of match between the category estimated using the embedded expression F3 and the embedded expression F2 and the category represented by the category name (category string) corresponding to the product name in the product master 121.

[0081] For example, the learning unit 105-3 uses the estimation model EMA to estimate the probability of each of multiple categories from the embedded expression F2 in the same manner as in the above embodiment. Furthermore, the learning unit 105-3 uses the estimation model EMB to estimate the probability of each of multiple categories from the embedded expression F3. The estimation model EMB is a model that is trained to input, for example, the embedded expression F3 and output the probability of each of multiple categories. The learning unit 105-3 calculates an estimated category result for the product name by adding the probabilities calculated for each of the embedded expression F2 and the embedded expression F3 for each category.

[0082] The number of dimensions of the embedded expression F3 output by the language model does not necessarily match the number of dimensions of the embedded expression F2. For this reason, as in the above example, the learning unit 105-3 calculates the probabilities of multiple categories from the embedded expression F3 using an estimation model EMB that is different from the estimation model EMA that inputs the embedded expression F2. If the number of dimensions of the embedded expression F2 and the number of dimensions of the embedded expression F3 match, the learning unit 105-3 may execute the above procedure using the estimation model EMA instead of the estimation model EMB. The learning unit 105-3 may calculate an estimation result of the category for the product name by inputting the value obtained by adding the embedded expression F2 and the embedded expression F3 into the estimation model EMA.

[0083] The estimation unit 106-3 estimates a category using the embedded expression F3 and the embedded expression F2. The estimation unit 106-3 can estimate a category using the embedded expression F3 and the embedded expression F2 using the same method as the learning unit 105-3.

[0084] 12 is a diagram showing an example of a category estimation method by the learning unit 105-3 and the estimation unit 106-3. Although the learning unit 105-3 will be described below as an example, the same procedure is also applied to the estimation unit 106-3.

[0085] The calculation unit 104-3 calculates an embedded expression 1211 (embedded expression F2) using a graph from the product name 1200. Furthermore, the calculation unit 104-3 calculates an embedded expression 1212 (embedded expression F3) using a language model from the product name 1200. In Fig. 12, an example is shown in which the embedded expression 1211 has 50 dimensions and the embedded expression 1212 has 768 dimensions.

[0086] The learning unit 105-3 inputs the embedded representation 1211 into the estimation model EMA to obtain an estimation result 1221 that indicates the probability of each category. Furthermore, the learning unit 105-3 inputs the embedded representation 1212 into the estimation model EMB to obtain an estimation result 1222 that indicates the probability of each category. In Fig. 12, an example in which the number of categories is 100 is shown.

[0087] The learning unit 105-3 adds the estimation result 1221 and the estimation result 1222 together to calculate the final estimation result 1230 of the category for the product name 1200.

[0088] Next, the learning process by the information processing device 100-3 of the third embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the learning process in the third embodiment.

[0089] Steps S601 to S602 are the same as steps S301 to S302 in the information processing apparatus 100 of the first embodiment, and therefore a description thereof will be omitted.

[0090] In this embodiment, the calculation unit 104-3 uses a language model to calculate an embedded expression F3 for each product name (step S603). The learning unit 105-3 inputs the embedded expression F2 into an estimation model EMA to estimate a category, and inputs the embedded expression F3 into an estimation model EMB to estimate a category (step S604). The learning unit 105-3 adds the estimation results of the two categories to calculate a final estimation result for each product name (step S605).

[0091] Steps S606 and S607 are the same as steps S304 and S305 in the information processing apparatus 100 of the first embodiment, and therefore a description thereof will be omitted.

[0092] Although an example using one language model has been described so far, the number of language models is not limited to one, and may be two or more.

[0093] In this way, in the third embodiment, the feature amounts calculated using the language model are further used to learn the feature amounts, and the learned feature amounts can be used to estimate the category.

[0094] (Fourth embodiment) In the above-described embodiment, the sum (or average) of the embedded expressions of the partial strings included in the product name is calculated as the embedded expression of the product name. On the other hand, in the case of product names in Japanese, for example, the partial strings included at the end of the product name often contain important information for category estimation. The information processing device of the fourth embodiment calculates the feature amount (embedded expression) of the product name using a weight according to the appearance position of the partial string within the product name.

[0095] 14 is a block diagram showing an example of the configuration of an information processing device 100-4 according to the fourth embodiment. As shown in FIG. 14, the information processing device 100-4 includes a storage unit 120, an acquisition unit 101, an extraction unit 102, a graph generation unit 103, a calculation unit 104-4, a learning unit 105, an estimation unit 106, and an output control unit 111.

[0096] In the fourth embodiment, the function of the calculation unit 104-4 is different from that in the first embodiment. Other configurations and functions are the same as those in FIG. 1, which is a block diagram of the information processing device 100 in the first embodiment, so the same reference numerals are used and the description here is omitted.

[0097] The calculation unit 104-4 calculates the embedded expression F2 of the product name based on the embedded expression F1 of one or more substrings PT2 included in the product name, which is weighted according to the position where the substring PT2 appears within the product name.

[0098] For example, if the product name is "Shrimp and Broccoli Salad," the "salad" category is likely to be more appropriate than the "shrimp" category for the product name. Therefore, rather than simply calculating the sum (or average) of the embedded representations of the substrings, the weight of the substrings is adjusted according to their positions, which is expected to improve the accuracy of category estimation. For example, the calculation unit 104-4 calculates the weight so that the closer the substring PT2 is to the beginning of the product name, the smaller the weight, and the closer it is to the end, the larger the weight. For example, the ratio of the position of the final character of the substring to the length of the product name (number of characters) can be used as such a weight. In the above example, the weight of the substring "shrimp" is (2 / 13) ≒ 0.15, and the weight of the substring "salad" is (13 / 13) = 1.

[0099] The calculation unit 104-4 may calculate normalized weights such that the sum of the weight values ​​for each product is 1. The weights may be calculated by multiplying the above ratio by a constant CA and a constant CB. The weights may be calculated by normalizing the value of the above ratio by a constant CA and a constant CB. The constants CA and CB may be set using a machine learning method such as deep learning so that the accuracy of category estimation is optimized.

[0100] The method of calculating the weight may be changed depending on the language used. For example, in English, a substring closer to the beginning of a product name is considered to be more relevant to the category. In such a case, a weight may be used in which the closer the appearance position is to the beginning of the product name, the larger the value, and the closer the appearance position is to the end, the smaller the value.

[0101] Next, information processing by the information processing device 100-4 of the fourth embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of information processing in the fourth embodiment.

[0102] Step S801 is the same process as step S301 in the information processing apparatus 100 of the first embodiment, and therefore a description thereof will be omitted.

[0103] The calculation unit 104-4 calculates the weight of the partial string (step S802). The calculation unit 104-4 calculates the embedded expression F2 for each product name (step S803). At this time, the calculation unit 104-4 calculates the embedded expression F2 for the product name node connected to one or more partial string nodes included in the graph by adding values ​​obtained by multiplying the embedded expressions F1 of the partial strings corresponding to the nodes by the weights corresponding to the partial strings.

[0104] Steps S804 to S806 are the same as steps S303 to S305 in the information processing apparatus 100 of the first embodiment, and therefore a description thereof will be omitted.

[0105] Fig. 16 is a diagram showing an example of calculation of an embedded expression F2 according to this embodiment. Fig. 16 shows an example of calculation of an embedded expression F2 for a product name "shrimp and broccoli salad." In addition, in the example of Fig. 16, normalized weights are used.

[0106] For example, the ratio of the substring "shrimp" to the length of the product name is 0.15, and the normalized weight is 0.08. The calculation unit 104-4 calculates the embedded representation F2 of the product name "shrimp and broccoli salad" as follows, based on the weight of each substring and the graph 1612: Embedded Representation F2 = Embedding expression of “shrimp” × 0.08 + 0.38 embedded expressions for "broccoli" + 0.54 embedded expressions for "salad"

[0107] The graph in the lower right of Figure 16 is a graph that schematically illustrates the addition of embedded expressions. Embedded expressions 1621, 1622, and 1623 correspond to values ​​obtained by multiplying the embedded expressions for "shrimp," "broccoli," and "salad" by weights, for example. Adding these values ​​results in embedded expression 1630, which is embedded expression F2 for the product name "shrimp and broccoli salad."

[0108] By performing such processing, it is possible to distinguish between two product names that contain the same substring "salad," for example, one in which "salad" appears in the first half and one in which "salad" appears in the second half.

[0109] In this way, in the fourth embodiment, the feature amount (embedded expression) of the product name is calculated using a weight according to the appearance position of the substring in the product name, which can further improve the accuracy of category estimation.

[0110] (Fifth embodiment) The information processing device of the fifth embodiment corrects the probability of a category by referring to the hierarchical structure between categories. The correction of the probability can be applied both during learning and during estimation. The following mainly describes an example in which the correction is performed during estimation.

[0111] There may be a large number of product name categories. In such cases, there is a concern that estimation accuracy may decrease if each category is estimated independently. On the other hand, when there are a large number of categories, multiple categories are often related to each other. For example, if there are a fish category, a tuna category, and a salmon category, the fish category corresponds to the category that is a higher-level concept than the tuna category and the salmon category.

[0112] By utilizing such information on the hierarchical structure between categories, it is expected that the accuracy of category estimation will be improved. In this embodiment, for example, instead of directly using the probability that each product belongs to the fish category, the estimated probability of the fish category is calculated as the linear sum of the probability that the product belongs to the salmon category, which is a sub-concept of the fish category, and the probability that the product belongs to the tuna category.

[0113] 17 is a block diagram showing an example of the configuration of an information processing device 100-5 according to the fifth embodiment. As shown in FIG. 17, the information processing device 100-5 includes a storage unit 120, an acquisition unit 101, an extraction unit 102, a graph generation unit 103, a calculation unit 104, a learning unit 105, an estimation unit 106, a correction unit 107-5, and an output control unit 111.

[0114] The fifth embodiment differs from the first embodiment in that a correction unit 107-5 is added. Other configurations and functions are the same as those of the information processing device 100 of the first embodiment shown in FIG. 1, and therefore the same reference numerals are used and the description thereof will be omitted.

[0115] The correction unit 107-5 corrects the probabilities of at least some of the multiple categories output by the estimation model EMA used by the learning unit 105 and the estimation unit 106. For example, when a category CTA (first category) among the multiple categories is a superordinate concept of one or more categories CTB (second category) among the multiple categories, the correction unit 107-5 corrects the probability of the category CTA to a linear sum of the probabilities of one or more categories CTB.

[0116] Instead of the linear sum, an average value may be used. Furthermore, weights used in calculating the linear sum may be set using AI (Artificial Intelligence) technology such as deep learning.

[0117] Next, the estimation process performed by the information processing device 100-5 according to the fifth embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of the estimation process according to the fifth embodiment.

[0118] Steps S901 to S904 are the same as steps S401 to S404 in the information processing apparatus 100 of the first embodiment, and therefore description thereof will be omitted.

[0119] The correction unit 107-5 identifies a set of categories related to the target category (step S905). The target category is, for example, a category designated as a target for which the probability is to be corrected, and corresponds to the above-mentioned category CTA. For example, the correction unit 107-5 identifies one or more categories (category CTB) that correspond to subordinate concepts of the target category (category CTA) by referring to information representing the hierarchical relationship between multiple categories. The set of category CTB corresponds to the set of categories related to the target category. The correction unit 107-5 corrects the probability of the target category (category CTA) to a linear sum of the probabilities of the category CTB included in the set (step S906), and terminates the estimation process.

[0120] The target category is not limited to a category corresponding to a higher-level concept. For example, the target category may be a category indicating that the product does not belong to any category (hereinafter referred to as the "other category"). The "other" category often does not have a substring specific to that category, making it difficult to identify products that belong to the "other" category with high accuracy. On the other hand, if the probability that the product belongs to a category other than the "other" category is low, it can be considered that the probability that the product belongs to the "other" category is high.

[0121] For example, the correction unit 107-5 identifies one or more categories designated as categories belonging to the "other" category, and calculates the probability of the "other" category by linearly adding the probabilities of the identified one or more categories.

[0122] In this way, the information processing device of the fifth embodiment can correct the probability of a category by referring to the hierarchical structure between categories.

[0123] The category estimation results obtained by the above-described embodiments can be used, for example, as follows.

[0124] (Example 1) If the maximum value of the probability for each category does not reach the specified threshold THD (second threshold), the product (product name) being estimated does not belong to any of the specified categories, and can be considered to have a high probability of belonging to a new category or other category.

[0125] Therefore, for example, the estimation unit 106 may determine that the product name category is not included in the multiple categories when the probability of each of the multiple categories is smaller than the threshold THD.

[0126] If the maximum value of the probabilities for each category does not reach the specified threshold THD, the method of the embodiment may be interpreted as indicating that the category cannot be estimated with high accuracy. Therefore, in such a case, an expert or the like may select (label) a category for the product name. For example, the output control unit 111 may output information indicating that the target product name is not included in any category. This may prompt an expert to perform labeling or other processing.

[0127] (Example of use 2) If the probabilities of multiple categories have multiple peaks, the target product name may be considered to belong to a new category that combines elements of multiple categories corresponding to the multiple peaks.

[0128] Therefore, for example, when the probability for two or more categories CTC (third category) among the multiple categories is greater than or equal to a threshold PHE (first threshold), the estimation unit 106 may generate a single category CTD (fourth category) that integrates the two or more categories CTC.

[0129] (Example 3) By comparing the category names in the product master 121 with the category names of the categories estimated by the embodiment, it is possible to detect what error patterns exist regarding the estimation of categories.

[0130] For example, the estimation unit 106 may identify an error pattern by comparing the category estimated by the estimation process of the embodiment with the category (correct category) indicated by the category name in the product master 121. The output control unit 111 may output the identified pattern.

[0131] (Example 4) Even if the accuracy rate of category estimation for each of a large number of products is low, the accuracy rate of category estimation may be increased by limiting the products to those for which the maximum value of the probability of each category is equal to or greater than the threshold value PHF (third threshold). To be able to determine such a situation, for example, the learning unit 105 may calculate the proportion of correct products among the products for which the maximum value of the probability of each category is equal to or greater than the threshold value PHF (third threshold). The output control unit 111 may output information visualizing the calculated proportion.

[0132] As described above, according to the first to fifth embodiments, the category of an object such as a product can be estimated with higher accuracy.

[0133] Next, the hardware configuration of the information processing apparatus according to the first to fifth embodiments will be described with reference to Fig. 19. Fig. 19 is an explanatory diagram showing an example of the hardware configuration of the information processing apparatus according to the first to fifth embodiments.

[0134] The information processing device of the first to fifth embodiments includes a control device such as a CPU (Central Processing Unit) 51, a storage device such as a ROM (Read Only Memory) 52 and a RAM (Random Access Memory) 53, a communication I / F 54 that connects to a network and communicates, and a bus 61 that connects each part.

[0135] The programs executed by the information processing apparatuses of the first to fifth embodiments are provided in advance in the ROM 52 or the like.

[0136] The programs executed by the information processing devices of the first to fifth embodiments may be configured to be provided as a computer program product by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0137] Furthermore, the programs executed by the information processing apparatuses of the first to fifth embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by the information processing apparatuses of the first to fifth embodiments may be provided or distributed via a network such as the Internet.

[0138] The programs executed by the information processing devices of the first to fifth embodiments can cause a computer to function as each unit of the information processing device described above. In this computer, the CPU 51 can read the programs from a computer-readable storage medium onto a main storage device and execute the programs.

[0139] A configuration example of the embodiment will be described below. (Configuration example 1) For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings corresponding to the target character string included in the first input information; executing a learning process to learn first feature information representing the features of the determined second partial string and second feature information representing the features of the target string including the determined second partial string and calculated based on the first feature information, so as to increase the coincidence rate between the category of the target string estimated from the second feature information and the category represented by the category string; Processing section An information processing device comprising: (Configuration example 2) The processing unit extracting, as the second partial string, a first partial string for which a determination value, which is at least one of the number of corresponding target strings and the ratio of the number of corresponding target strings to the total number of target strings including the first partial string, is greater than other first partial strings, from among the one or more category strings corresponding to the target string including the first partial string, The information processing device according to configuration example 1. (Configuration example 3) The processing unit further extracting one or more substrings included in the category string as the first substring; The information processing device according to configuration example 1 or 2. (Configuration example 4) The processing unit generating a graph including the target string and the determined second substring; calculating the second feature information of the target string included in the graph based on the first feature information of the second partial string included in the graph; The information processing device according to any one of configuration examples 1 to 3. (Configuration Example 5) The processing unit For each of the target strings included in the plurality of pieces of first input information, determine, from among combinations of one or more of the second partial strings included in the target string, the second partial strings included in a combination in which the number of characters in the target string that match characters included in the second partial string is greater than in other combinations and the number of the second partial strings in the combination is smaller than in other combinations. The information processing device according to any one of configuration examples 1 to 4. (Configuration Example 6) The processing unit performing a determination process from a scanning start position that is at least one of the beginning and end of the target string to determine, from one or more of the second partial strings, a second partial string that has a greater number of characters in the target string that match characters contained in the second partial string than other second partial strings; and repeating the determination process using a character string obtained by removing the determined second partial character string from the target character string as the new target character string. The information processing device according to any one of configuration examples 1 to 4. (Configuration Example 7) The processing unit calculating the second feature information of the target string based on the first feature information of one or more of the second partial strings included in the target string, the first feature information being weighted according to the appearance position of the one or more second partial strings included in the target string within the target string; The information processing device according to any one of configuration examples 1 to 6. (Configuration Example 8) The processing unit calculating the third feature information of the target string including the extracted second partial string using a language model trained to receive the target string and output third feature information representing a feature of the target string; performing the learning process so as to increase a matching rate between the category of the target character string estimated using the third feature information and the second feature information and the category represented by the category character string; The information processing device according to any one of configuration examples 1 to 7. (Configuration Example 9) The processing unit extracting one or more third substrings included in the target string contained in the second input information using second input information including the target string; calculating the second feature information representing the feature of the target string included in the second input information based on the first feature information representing the feature of the extracted third partial string; performing an estimation process for estimating a category of the target character string using the second feature information; The information processing device according to any one of configuration examples 1 to 8. (Configuration Example 10) The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; When a first category among the plurality of categories is a superordinate concept of one or more second categories among the plurality of categories, the probability of the first category is modified to a linear sum of the probabilities of one or more second categories; The information processing device according to configuration example 9. (Configuration Example 11) The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; generating one fourth category by integrating the two or more third categories when the probabilities for two or more third categories among the plurality of categories are equal to or greater than a first threshold value; The information processing device according to configuration example 9. (Configuration Example 12) The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; When the probabilities of each of the plurality of categories are all smaller than a second threshold, it is determined that the category of the target character string is not included in the plurality of categories. The information processing device according to configuration example 9. (Configuration Example 13) The processing unit Identifying an error pattern by comparing the category estimated by the estimation process with a correct category. The information processing device according to configuration example 9. (Configuration Example 14) The processing unit Estimating a probability of each of a plurality of categories from the second feature information; Calculating the proportion of the objects that are correct among the objects whose probability is equal to or greater than a third threshold; The information processing device according to any one of configuration examples 1 to 13. (Configuration Example 15) The processing unit performing a predetermined preprocessing on at least some of the plurality of pieces of first input information; extracting the first partial character string and the second partial character string for each of the plurality of pieces of first input information that have been preprocessed; The information processing device according to any one of configuration examples 1 to 14. (Configuration Example 16) Using input information including a target string representing a target, extract one or more substrings included in the target string; calculating second feature information representing a feature of the target character string based on first feature information representing a feature of the extracted partial character string; performing an estimation process for estimating a category of the target character string using the second feature information; Processing section An information processing device comprising: (Configuration Example 17) An information processing method executed by an information processing device, For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings to be associated with the target character string included in the first input information; a step of executing a learning process to learn first feature information representing features of the determined second partial string and second feature information representing features of the target string including the determined second partial string and calculated based on the first feature information, so as to increase the coincidence rate between the category of the target string estimated from the second feature information and the category represented by the category string; An information processing method including: (Configuration Example 18) On the computer, For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings to be associated with the target character string included in the first input information; a step of executing a learning process to learn first feature information representing features of the determined second partial string and second feature information representing features of the target string including the determined second partial string and calculated based on the first feature information, so as to increase the coincidence rate between the category of the target string estimated from the second feature information and the category represented by the category string; A program to execute.

[0140] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0141] 100, 100-2, 100-3, 100-4, 100-5 Information processing equipment 101 Acquisition Department 102 Extraction part 103,103-2 Graph generation unit 104, 104-3, 104-4 Calculation section 105,105-3 Learning Department 106,106-3 Estimation part 107-5 Correction section 111 Output control section 120 Storage section

Claims

1. For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings to be associated with the target character string included in the first input information; a learning process is executed to learn first feature information representing the features of the determined second partial character string and second feature information representing the features of the target character string including the determined second partial character string and calculated based on the first feature information, so that a matching rate between the category of the target character string estimated from the second feature information and the category represented by the category character string becomes large; Processing section An information processing device comprising:

2. The processing unit extracting, as the second partial string, a first partial string for which a determination value, which is at least one of the number of corresponding target strings and the ratio of the number of corresponding target strings to the total number of target strings including the first partial string, is greater than other first partial strings, among the one or more category strings corresponding to the target string including the first partial string; The information processing device according to claim 1 .

3. The processing unit further extracting one or more substrings included in the category string as the first substring; The information processing device according to claim 1 .

4. The processing unit generating a graph including the target string and the determined second substring; calculating the second feature information of the target string included in the graph based on the first feature information of the second partial string included in the graph; The information processing device according to claim 1 .

5. The processing unit For each of the target strings included in the plurality of pieces of first input information, determine, from among combinations of one or more of the second partial strings included in the target string, the second partial strings included in a combination in which the number of characters in the target string that match the characters included in the second partial string is greater than in other combinations and the number of the second partial strings in the combination is smaller than in other combinations. The information processing device according to claim 1 .

6. The processing unit performing a determination process from a scanning start position that is at least one of the beginning and end of the target string to determine, from one or more of the second partial strings, a second partial string that has a greater number of characters in the target string that match characters included in the second partial string than other second partial strings; and repeating the determination process using a character string obtained by removing the determined second partial character string from the target character string as the new target character string. The information processing device according to claim 1 .

7. The processing unit calculating the second feature information of the target string based on the first feature information of one or more second partial strings included in the target string, the first feature information being weighted according to the appearance position of the one or more second partial strings included in the target string within the target string; The information processing device according to claim 1 .

8. The processing unit calculating the third feature information of the target string including the extracted second partial string using a language model trained to receive the target string and output third feature information representing a feature of the target string; performing the learning process so as to increase a matching rate between the category of the target character string estimated using the third feature information and the second feature information and the category represented by the category character string; The information processing device according to claim 1 .

9. The processing unit extracting one or more third substrings included in the target string included in the second input information using second input information including the target string; calculating second feature information representing features of the target character string included in the second input information based on the first feature information representing features of the extracted third partial character string; performing an estimation process for estimating a category of the target character string using the second feature information; The information processing device according to claim 1 .

10. The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; When a first category among the plurality of categories is a superordinate concept of one or more second categories among the plurality of categories, the probability of the first category is modified to a linear sum of the probabilities of one or more second categories. The information processing device according to claim 9 .

11. The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; generating one fourth category by integrating the two or more third categories when the probabilities for two or more third categories among the plurality of categories are equal to or greater than a first threshold value; The information processing device according to claim 9 .

12. The processing unit outputting a probability for each of a plurality of categories as a result of the estimation process; When the probabilities of each of the plurality of categories are all smaller than a second threshold, it is determined that the category of the target character string is not included in the plurality of categories. The information processing device according to claim 9 .

13. The processing unit Identifying an error pattern by comparing the category estimated by the estimation process with a correct category. The information processing device according to claim 9 .

14. The processing unit Estimating a probability of each of a plurality of categories from the second feature information; Calculating the proportion of the objects that are correct among the objects whose probability is equal to or greater than a third threshold value; The information processing device according to claim 1 .

15. The processing unit performing a predetermined preprocessing on at least some of the plurality of pieces of first input information; extracting the first partial character string and the second partial character string for each of the plurality of pieces of first input information that have been preprocessed; The information processing device according to claim 1 .

16. Using input information including a target string representing a target, extract one or more substrings included in the target string; calculating second feature information representing a feature of the target character string based on first feature information representing a feature of the extracted partial character string; performing an estimation process for estimating a category of the target character string using the second feature information; Processing section An information processing device comprising:

17. An information processing method executed by an information processing device, For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings to be associated with the target character string included in the first input information; a step of executing a learning process to learn first feature information representing features of the determined second partial string and second feature information representing features of the target string including the determined second partial string and calculated based on the first feature information, so as to increase the coincidence rate between the category of the target string estimated from the second feature information and the category represented by the category string; An information processing method including:

18. On the computer, For each of a plurality of pieces of first input information including a target string representing an object and a category string representing a category to which the object belongs, extracting one or more first partial strings included in the target string, and extracting one or more second partial strings from the one or more first partial strings included in the target string using the category string; determining, for each of the plurality of pieces of first input information, one or more second partial character strings to be associated with the target character string included in the first input information; a step of executing a learning process to learn first feature information representing features of the determined second partial string and second feature information representing features of the target string including the determined second partial string and calculated based on the first feature information, so as to increase the coincidence rate between the category of the target string estimated from the second feature information and the category represented by the category string; A program to execute.

Images