Information processing device, information processing system, and information processing method

Abstract

To prevent arbitrary determination by automatically extracting feature quantities effective for prediction in an information processing device.SOLUTION: The information processing device has: a feature quantity extraction model construction section for constructing a feature quantity extraction model that predicts results on the basis of a plurality of pieces of first unstructured data and result information with regard to each of the pieces of first unstructured data; a feature quantity extraction section for inputting a plurality of pieces of second unstructured data to the feature quantity extraction model and extracting a feature quantity that contributes to accuracy of result prediction; and a result prediction model construction section for constructing a result prediction model that predicts a result on the basis of supplementary information to each of the feature quantity and the plurality of pieces of second unstructured data and the result information of each of the pieces of second unstructured data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, and an information processing method. [Background technology]

[0002] Appropriate budget / actual management is a necessary process for planning and optimizing personnel resources. To properly implement budget / actual management, it is necessary to accurately predict the probability that the project you are currently trying to acquire will be concluded in the future, as well as the progress rate of the project.

[0003] However, because it is difficult to accurately predict whether a deal will be concluded and its progress, proper budget and actual management has become a challenge for many companies, resulting in lower sales and missed opportunities.The main reason why it is difficult to predict whether a deal will be concluded and its progress is that the success rate changes depending on the target customer's situation at the time, and prediction algorithms that only refer to data on past similar deals have poor accuracy.

[0004] The customer situation here refers to the customer's situation, which changes from project to project, such as whether the customer is positive about the proposal or the existence of competitors.

[0005] This customer situation is best understood by the sales staff who are in daily contact with customers to close deals. Therefore, by extracting customer situation information from sales records recorded during daily sales activities, the accuracy of the deal prediction algorithm can be improved.

[0006] Patent document 1 discloses a method for providing an information processing device that can use document files created for a case or a group of messages exchanged between multiple users regarding the case when predicting the outcome of the case, such as whether or not the case will be concluded. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2021-149844 Summary of the Invention [Problem to be solved by the invention]

[0008] In Patent Document 1, information is extracted based on multiple rules from document files or message groups created for a case, and an outcome for a case is predicted using artificial intelligence that has learned to predict the outcome of the case corresponding to the information in the document files or message groups for each case.

[0009] However, with the above technology, the feature rules are manually determined, which results in arbitrary criteria, and the created feature values ​​are not necessarily effective for prediction.In addition, as the number of target customers and projects increases, new rules must be added, which requires a considerable amount of work.

[0010] An object of the present invention is to prevent arbitrary judgments by automatically extracting feature amounts that are effective for prediction in an information processing device. [Means for solving the problem]

[0011] An information processing device according to one embodiment of the present invention is an information processing device having a memory unit and a calculation unit, wherein the memory unit stores unstructured information related to a target to be predicted, result information related to the target to be predicted, and incidental information related to the target to be predicted, and the calculation unit has: a feature extraction model construction unit that constructs a feature extraction model that predicts a result based on a plurality of first unstructured data that are a part of the unstructured information and the result information for each of the first unstructured data; a feature extraction unit that inputs a plurality of second unstructured data that are the remaining part of the unstructured information to the feature extraction model and extracts features that contribute to the prediction accuracy of the result; and a result prediction model construction unit that constructs a result prediction model that predicts the result based on the features, the incidental information for each of the plurality of second unstructured data, and the result information for each of the second unstructured data. [Effects of the Invention]

[0012] According to one aspect of the present invention, an information processing device can prevent arbitrary decisions by automatically extracting feature amounts that are effective for prediction. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an information processing device and an information processing system according to a first embodiment. [Figure 2] 10 is a flowchart illustrating an example of processing in a feature extraction model construction unit in the information processing device. [Figure 3] FIG. 2 is a diagram illustrating an example of a data structure of unstructured information. [Figure 4] FIG. 10 is a diagram illustrating an example of a data structure of result information. [Figure 5] FIG. 2 is a diagram showing an example of document data of each case. [Figure 6] 10 is a flowchart illustrating an example of processing in a feature extraction unit and a result prediction model construction unit in the information processing device. [Figure 7] FIG. 10 is a diagram illustrating an example of a data structure of additional information. [Figure 8] FIG. 10 is a diagram illustrating an example of an input data structure of a result prediction model. [Figure 9] FIG. 10 is a diagram illustrating an example of a data structure of prediction accuracy calculated based on a plurality of feature sets. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of an information processing device and an information processing system according to a second embodiment. [Figure 11] 10 is a flowchart illustrating an example of processing in a prediction contribution information selection unit in the information processing device. [Figure 12A] FIG. 10 is a diagram illustrating an example of document data from which information contributing to prediction is selected. [Figure 12B] FIG. 10 is a diagram illustrating an example of a data structure for selecting predicted contribution information. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Example]

[0015] First Embodiment A configuration of an information processing device and an information processing system according to a first embodiment will be described with reference to FIG. The information processing system 100 includes an information processing device 101 communicably connected via a network 102, a user terminal 103 used by a user, and a database 104 storing information on a prediction target.

[0016] The network 102 enables the user terminal 103, the database device 104, and the information processing device 101 to communicate with each other.

[0017] The user terminal 103 is an information processing device such as a PC (Personal Computer). The user inputs predetermined information into the user terminal 103, and the information processing device 101 outputs the results calculated. As a specific example, in the case of predicting whether a case will be concluded in sales activities, the input information is the case number for which budget / actual management is to be performed, and the output information is the probability of concluding the case. Here, data related to the case for which budget / actual management is to be performed, which is processed by the information processing device 101, is stored in the database device 104.

[0018] The information processing device 101 is an information processing device that automatically extracts features that contribute to improving the accuracy of outcome prediction from unstructured data and constructs a model that predicts outcomes based on the features. Here, unstructured data is, for example, free-format data.

[0019] The information processing device 101 includes a storage unit 110, a calculation unit 120, and a communication unit 130. The storage unit 110 stores unstructured information 111, which is unstructured data related to the prediction target, result information 112, which is result information related to the prediction target, and other incidental information 113, which is information related to the prediction target that includes at least structured data.

[0020] The calculation unit 120 includes a feature extraction model construction unit 121 , a feature extraction unit 122 , and a result prediction model construction unit 123 .

[0021] The feature extraction model construction unit 121 constructs a machine learning model that predicts results from a first plurality of unstructured data and result information for each of the first unstructured data. The feature extraction unit 122 inputs a second plurality of unstructured data to the machine learning model and acquires intermediate layer data. The result prediction model construction unit 123 predicts results from the intermediate layer data, data including at least structured data for each of the second plurality of unstructured data, and result information for each of the second unstructured data. Here, the first unstructured data is a portion of the unstructured information 111, and the second unstructured data is the remaining portion of the unstructured information 111.

[0022] The communication unit 130 communicates with the information processing device 101 and other devices via the network 102 . For example, the information processing device 101 is configured by a computer, the calculation unit 120 is configured by a processor, and the storage unit 110 is configured by a memory.

[0023] The feature extraction model construction unit 121, the feature extraction unit 122, and the result prediction model construction unit 123 are each configured with a program, and operate as functional units that provide predetermined functions by a processor processing these programs. For example, the processor functions as the feature extraction model construction unit 121 by processing in accordance with the feature extraction model construction program. The same applies to the other programs.

[0024] In this first example, we assume a case where we want to predict whether a deal will be concluded in sales activities. In addition, the unstructured data used is only document data that records interactions between salespeople and customers, impressions of salespeople, etc.

[0025] Here, the unstructured data is not limited to document data, and other formats such as images and audio may also be used. Multiple types of unstructured data may also be used. In this case, appropriate measures may be taken, such as increasing the number of machine learning models constructed by the feature extraction model construction unit 121 by the amount of unstructured data.

[0026] 2 shows a processing flow in the feature extraction model construction unit 121. In the following embodiment, the function of the feature extraction model construction unit 121 will be described according to this flow. First, in step S202, document data of past cases stored in the unstructured information 111 is read. At this time, instead of reading all the data, the document data of some of the cases is read and used to build a machine learning model that extracts features. Furthermore, the document data of the remaining cases is used to build a prediction model in the result prediction model building unit 123. In this Example 1, these pieces of data are referred to as a first plurality of document data and a second plurality of document data, respectively.

[0027] FIG. 3 shows an example of data stored in the unstructured information 111, which contains document files for each information item. The data 301 storing the document files for each case is composed of a case number column 311 and an attached document data column 312 .

[0028] Information relating to a case number that identifies a case is stored in the case number column 311. Document data relating to each case is stored in the attached document file column 312.

[0029] Here, the stored document files may be of one type or multiple types. In the first embodiment, as an example, a processing method using one type of data that records interactions between a sales representative and a customer, the sales representative's impressions, etc. is shown. When using multiple types, an appropriate method may be selected, such as a method of combining multiple document data to form one document data, or a method of treating multiple document data as separate data and building a machine learning model.

[0030] Next, in step S203, result information of the case corresponding to the document data read in step S202 is read from the result information 112.

[0031] FIG. 4 shows an example of data stored in the result information 112, which is the result information of each case. The data 401 storing the result information of each case is composed of a case number column 411 and a case success / failure result column 412. The case number column 411 stores information related to the case number that identifies the case. The case success / failure result 412 stores information indicating the success / failure result of the target case.

[0032] In the first embodiment, binary information indicating whether or not a deal has been concluded is stored in order to predict whether or not a deal will be concluded in sales activities. However, there is no limitation on the format of the result information processed by this information processing device, and for example, if the prediction target is the progress rate of a project, continuous values ​​such as real numbers or probability values ​​may be used.

[0033] Next, in step S204, a machine learning model is constructed to predict whether or not a deal will be concluded based on the document data read in step S202 and the result information of each deal read in step S203.

[0034] In this first example, a machine learning model having an internal neural network is assumed as the machine learning model having an intermediate layer. A neural network is also known as a multi-perceptron, and is a technology that has the ability to solve linearly inseparable problems by stacking multiple perceptrons in multiple layers. Here, the intermediate layer may be any of the multiple perceptron layers used to calculate the prediction result.

[0035] FIG. 5 shows an example of document data for each case used in the first embodiment. The document data 501 of each case is structured by a case number column 511 and a data content column 512. In the first embodiment, a machine learning model is constructed based on the document data shown in FIG.

[0036] Therefore, first, preprocessing of document data is performed so that a machine learning model can be constructed. In this Example 1, as a preprocessing method, morphological analysis is used, which divides each sentence constituting a document into words constituting the document and divides all words contained in the document into word types and word information, and one-hot vectorization is used, which quantifies document data.

[0037] A one-hot vector is a vector in which one element out of all elements is 1 and all other elements are 0. When converting document data into a one-hot vector, each element of the vector corresponds to a word or character type. The number of dimensions of a one-hot vector can be determined in advance based on the character types corresponding to each language, or after counting the word types in the document data to be processed, but it can be determined as appropriate depending on the analysis at that time.

[0038] Furthermore, there are many other preprocessing methods for inputting document data into a machine learning model, such as embedding vectorization, and the method is not limited to the method of this embodiment.

[0039] Furthermore, if image data is used as unstructured data, preprocessing such as color correction and image resizing may be possible. If audio data is used, preprocessing such as standardization of audio signals and conversion to spectrogram format may be possible.

[0040] The feature extraction model construction unit 121 constructs a machine learning model with an intermediate layer based on the unstructured data and the result information according to the above flow, and constructs a prediction model that predicts the result for the input unstructured data. In this Example 1, this model is called a feature extraction model, and this model is mainly used to extract features that contribute to the accuracy of result prediction from unstructured data such as document data.

[0041] 6 shows the processing flow in the feature extraction unit 122 and the prediction model construction unit 123. In FIG. 6, steps S602 and S603 are processed by the feature extraction unit 122, and steps S604 and S605 are processed by the prediction model construction unit 123.

[0042] In this first embodiment, the functions of the feature extraction unit 122 and the prediction model construction unit 123 will be described in accordance with this flow.

[0043] First, in step S602, document data of past cases stored in the unstructured information 111 is read. At this time, a second plurality of document data that was not used to build a machine learning model in the feature extraction model building unit 121 is read.

[0044] Next, in step S603, the document data read in step S602 is input to a feature extraction model. Then, the intermediate layer values ​​of the model are obtained as feature vectors. At this time, the read document data is subjected to the same preprocessing as when constructing the feature extraction model.

[0045] The acquired feature vector is a vector used when predicting results using a feature extraction model, and is therefore a feature that contributes to the accuracy of the result prediction. With this method, the information processing device of the present invention can automatically extract features that contribute to improving the accuracy of result predictions from unstructured data, without the need to manually determine feature rules.

[0046] Next, in step S604, result information of the case corresponding to the document data read in step S602 is read from the result information 112, and other case-related information related to the case is read from the other incidental information 113.

[0047] FIG. 7 shows an example of data in which other incidental information for a case is recorded and stored in the other incidental information 113. Other case-related information 701 is made up of a case number column 711 , a customer business type column 712 , a customer base column 713 , a contract amount column 714 , and a product type column 715 .

[0048] The case number field 711 stores case number information that identifies the case. The customer industry field 712 stores information about the customer's industry. The industry has predetermined categories such as "finance," "automotive," and "medical." The customer base field 713 stores information about the country in which the customer is based. The base has predetermined categories such as "Japan," "America," and "China."

[0049] The contract amount field 714 stores information about the contract amount of the project. The product type field 715 stores information about the type of product sold or proposed in the project. Product types have predetermined categories such as "Product X," "Product Y," and "Product Z."

[0050] This data may also include unstructured data. In that case, the unstructured data can be converted into numerical data and then into structured data using the one-hot vectorization method described above.

[0051] Next, in step S605, a result prediction model for predicting whether or not the case will be concluded is constructed based on the feature vector acquired in step S603, the case result information acquired in step S604, and other case-related information.

[0052] The types of algorithms for the constructed predictive models include, for example, decision trees, random forests, support vector machines, and neural networks.

[0053] To build the above-mentioned prediction model, in this step, the feature vector and other case-related information are first combined using the case number entered in the case number column as a key to create input data for the prediction model.

[0054] Figure 8 shows an example of the input data for the prediction model, which is data combining feature vectors and other additional case information for each case number. The input data 801 for the prediction model is composed of a case number field 811, a customer industry field 812, a customer headquarters field 813, a contract amount field 814, a product type field 815, and document features 816. The document features 816 are feature vectors for each document data item of each case extracted in S603, and the number of features N varies depending on the size of the intermediate layer of the constructed feature extraction model.

[0055] Next, the created input data is preprocessed using a preprocessing method appropriate for the prediction model to be used. For example, if an algorithm that processes numerical data, such as a neural network, is used as the prediction model, the input data 801 contains text data in the customer industry field, so labeling is performed based on predetermined classifications. There are no restrictions on the preprocessing method, and the preprocessing method may be changed as appropriate. After that, a result prediction model is constructed based on the input data and the result information of the case.

[0056] When predicting the success rate of a certain case based on this model, the document data of the case to be predicted is input into the feature extraction unit, a feature vector is obtained, and this feature vector is combined with other case-related information that has been preprocessed in the same way as when building the result prediction model, and then input into the prediction model to calculate the success rate of that case.

[0057] In the information processing apparatus 101 of the first embodiment, a result prediction model can be constructed based on the feature amount that contributes to improving the prediction accuracy of the result automatically extracted from unstructured data by this method.

[0058] In addition, in this first embodiment, the prediction model construction unit 123 may have a function of selecting information to be used as input data using the prediction accuracy when constructing a prediction model. Below, an embodiment of a method for constructing a prediction model after selecting information to be used as input data will be described.

[0059] First, to calculate the prediction accuracy, the input data 801 and the result information of the case acquired in step S604 are divided into data to be used for training (training data) and data to calculate the accuracy (verification data).

[0060] Then, several feature sets are created by selecting some of the features included in the input data, and a prediction model is constructed for each feature set using training data, after which the prediction accuracy is calculated using validation data. Here, there are no restrictions on the method for creating feature sets, and it is possible to determine them randomly, or to define rules in advance, such as always using other additional information about the case.

[0061] 9 shows an example of prediction accuracy calculated based on a plurality of created feature sets. In this Example 1, the prediction correctness rate of the possibility of contracting a deal is used as the prediction accuracy. The validity rate table 901 of deal success or failure calculated using each feature set is composed of a feature set number column 911 , a used feature column 912 , and a validity rate column 913 .

[0062] For example, the first row of table 901 shows that the accuracy rate of the prediction model when all features contained in input data 801 (customer industry, customer headquarters, ..., document feature 1, ..., document feature (N-1), document feature N) are used is 90%.

[0063] In addition, the second row of Table 901 shows that the accuracy rate of the prediction model is 95% when using "customer industry, customer headquarters, ..., document feature 1, ..., document feature (N-1)" from the features contained in the input data 801, excluding document feature N.

[0064] Furthermore, the third row of Table 901 shows that the accuracy rate of the prediction model is 75% when using the features contained in the input data 801, excluding the customer's headquarters, namely, "customer industry, contract amount, ..., document feature 1, ..., document feature (N-1)."

[0065] Therefore, in this Example 1, since the accuracy of the prediction model constructed based on feature set B is high, a prediction model is constructed and used using feature set B.

[0066] In this way, by selecting information to be used as features, it becomes possible to build a prediction model that is more accurate and allows for faster processing. In addition, in this Example 1, as a method of calculating accuracy, the data is simply divided into training data and validation data, and prediction accuracy is calculated, but there is no limitation on the accuracy calculation method, and accuracy calculation by cross-validation, etc., can also be performed. [Example]

[0067] Next, a method for selecting a portion of unstructured data that contributes to prediction when constructing a result prediction model using an information processing device according to the second embodiment will be described.

[0068] Unstructured data can contain noise-like information that does not contribute to predicting outcomes. If an outcome prediction model is built with this information included, it can result in issues such as longer construction times and reduced prediction accuracy.

[0069] Therefore, in the information processing device of Example 2, information that contributes to prediction and that should be used to train the result prediction model can be selected from the unstructured data before the feature extraction unit 122 extracts a feature vector from the unstructured data, so that information such as noise that does not contribute to predicting the result is not used as training data for the result prediction model.

[0070] This process can remove noise-like information from the training data for the outcome prediction model, which has the effect of reducing the training time for the outcome prediction model and improving prediction accuracy.

[0071] The configuration of an information processing device and an information processing system according to the second embodiment will be described with reference to FIG. An information processing system 1000 of the second embodiment includes an information processing device 1001 communicably connected via a network 1002, a user terminal 1003 used by a user, and a database 1004 storing information to be predicted.

[0072] The network 1002 enables the user terminal 1003, the database device 1004, and the information processing device 1001 to communicate with each other.

[0073] The user terminal 1003 is an information processing device such as a PC (Personal Computer). The user inputs predetermined information into the user terminal 1003, which outputs the results calculated by the information processing device 1001. As a specific example, in the case of predicting whether a case will be concluded in sales activities, the input information is the case number for which budget / actual management is desired, and the output information is the probability of concluding the case. Here, data related to the case for which budget / actual management is desired to be processed by the information processing device 1001 is stored in the database device 1004.

[0074] The information processing device 1001 is an information processing device that automatically extracts features that contribute to improving the accuracy of outcome prediction from unstructured data and constructs a model that predicts outcomes based on the features. The information processing device 1001 has a storage unit 1010, a calculation unit 1020, and a communication unit 1030.

[0075] The storage unit 1010 stores unstructured information 1011, which is unstructured data related to the prediction target, result information 1012, which is result information related to the prediction target, and other incidental information 1013, which is information including at least structured data related to the prediction target.

[0076] The calculation unit 1020 has a feature extraction model construction unit 1021, a feature extraction unit 1022, a result prediction model construction unit 1023, and a prediction contribution information selection unit 1024. The feature extraction model construction unit 1021, the feature extraction unit 1022, and the result prediction model construction unit 1023 of the second embodiment have the same functions as the feature extraction model construction unit 121, the feature extraction unit 122, and the result prediction model construction unit 123 of the first embodiment in Fig. 1, and therefore a description thereof will be omitted.

[0077] The newly added prediction contribution information selection unit 1024 selects information that contributes to prediction accuracy from unstructured data based on the machine learning model constructed by the feature extraction model construction unit 1021.

[0078] For example, the information processing device 1001 is configured by a computer, the calculation unit 1020 is configured by a processor, and the storage unit 1010 is configured by a memory.

[0079] The feature extraction model construction unit 1021, the feature extraction unit 1022, the result prediction model construction unit 1023, and the prediction contribution information selection unit 1024 are each configured with a program, and by processing these programs by a processor, they operate as functional units that provide predetermined functions. For example, the processor functions as the prediction contribution information selection unit 1024 by processing in accordance with the prediction contribution information selection program. The same applies to the other programs.

[0080] The communication unit 1030 communicates with the information processing device 1001 and other devices via the network 1002 .

[0081] As such, the information processing device 1001 and information processing system 1000 of Example 2 in Figure 10 have the same configuration as the information processing device 101 and information processing system 100 of Example 1 in Figure 1, except that a predicted contribution information selection unit 1024 has been newly added.

[0082] In this example 2, as in the example 1, it is assumed that the likelihood of a deal being concluded in sales activities is predicted. In addition, the unstructured data used is only document data that records interactions between salespeople and customers, impressions of salespeople, etc.

[0083] First, a machine learning model for predicting whether a deal will be concluded based on document data is constructed using the feature extraction model construction unit 1021 by the same process as in the first embodiment.

[0084] 11 shows a processing flow in the prediction contribution information selection unit 1024. In the following embodiment, the function of the prediction contribution information selection unit 1024 will be described according to this flow. First, in step S1102, the document data stored in the unstructured information 1011 is read in. At this time, a second plurality of document data that was not used to build the feature extraction model is read in.

[0085] Next, in step S1103, the document data read in step S1102 is input into a feature extraction model to calculate the deal success probability. At this time, the read document data is subjected to the same preprocessing as when the feature extraction model was constructed. In this Example 2, it is assumed that the deal success probability calculated by inputting the document data read in step S1102 into the feature extraction model is 95%.

[0086] Next, in step S1104, the probability of closing a deal is calculated when a certain word is masked from the document data used as input data in step S1103. Masking here means that a certain word in the text is not used as input data to the feature extraction model.

[0087] In this second embodiment, one-hot vectorization is performed, so the one-hot vector for the masked word becomes a vector in which all elements are 0. In this case, if the document data contains two or more words to be masked, they are masked simultaneously and used as input data. This process makes it possible to calculate the deal closing probability based on a sentence in which a certain word has been removed from the document data. This process is performed for each word contained in the document data, and the predicted probability when each word is masked is saved.

[0088] FIG. 12A shows an example of document data from which information contributing to prediction is selected, and FIG. 12B shows an example of a table recording the predicted probability and the range of change in predicted probability when each word included in the document data is masked.

[0089] The sentence "We received a positive response to our proposal" included in the document data 1201 means that the customer is satisfied with the proposal, and is thought to contribute to the accuracy of the prediction of whether or not the deal will be concluded. On the other hand, the sentence "After that, I headed off to a meeting with another company" clearly does not contribute to the accuracy of the prediction of whether or not the deal will be concluded. The information processing device 1001 aims to remove such sentences.

[0090] Table 1202 is composed of a masked word column 1211, a predicted probability column 1212, and a variation range of predicted probability column 1213. Here, the variation range of predicted probability column 1213 stores the absolute value of the difference between the deal closing probability calculated by inputting the original sentence calculated in step S1103 as input data into the feature extraction model, and the deal closing probability calculated by inputting the sentence calculated in step S1104 with the words listed in the masked word column 1211 masked as input data into the feature extraction model.

[0091] For example, the first row of table 1202 shows the change range of the deal success probability and predicted probability when the word "proposal" contained in document data 1201 is masked. Since the deal success probability when "proposal" is masked is 93%, the change range of the predicted probability is 2%.

[0092] The second row of table 1202 shows the change in the deal closing probability and predicted probability when the word "positive" contained in document data 1201 is masked. Because the deal closing probability was 75% when "positive" was masked, the change in predicted probability was 20%.

[0093] The third row of table 1202 shows the change range of the deal closing probability and predicted probability when the word "reaction" contained in document data 1201 is masked. Since the deal closing probability when "reaction" is masked is 90%, the change range of the predicted probability is 5%.

[0094] If the calculated prediction probability changes significantly, the word can be determined to be a word that contributes to predicting deal success, because simply removing that word from the document data means that the prediction probability has changed significantly from the prediction based on the original text.

[0095] Therefore, in step S1105, a threshold value for the range of change in prediction probability is set, and words that exceed that threshold value are selected as words that contribute to predicting the success of a deal, and sentences containing those words are output as input data for the feature extraction unit 1022.

[0096] In this Example 2, the threshold value for the change range of the prediction probability is set to 10%. As a result, it is assumed that, based on the results of Table 1202, only the word "positive" is selected as a word that contributes to the prediction.

[0097] Therefore, a sentence such as "I received a positive response to my proposal," which includes the selected word, is selected as input data for the feature extraction unit 1022. On the other hand, a sentence such as "After that, I headed off to a meeting with another company," which does not include the word "positive," is determined to be a sentence that does not contribute to predicting whether or not the deal will be concluded, and is not selected as input data for the feature extraction unit 1022.

[0098] Thereafter, the sentences selected by this process are output as input data for the feature extraction unit 1022. The subsequent processes are performed by executing the flow in Fig. 6 as described in the first embodiment. This makes it possible to select information that contributes to prediction and that should be used as training data for the model before the feature extraction unit 1022 acquires feature vectors from unstructured data.

[0099] In the second embodiment, words in a sentence are masked, prediction probabilities are calculated, and it is determined whether each word contributes to prediction based on the range of change, but the present invention is not limited to this method.

[0100] Furthermore, in this second embodiment, since document data was used as an example, the target to be masked was a word. When other unstructured data is handled using a similar method, it is possible to deal with it by appropriately changing the target to be masked. For example, when handling an image as unstructured data, a method can be considered in which a specific range of pixels or a part of the image is masked, and the value of the masked pixels or part of the image is set to 0 and used as input data.

[0101] In the case of audio data, a method is conceivable in which data for a certain time is masked, and the amplitude value of the signal at the masked time is set to 0 and used as input data.

[0102] According to the above embodiment, when constructing a model for predicting a certain outcome using unstructured data such as document data, it is possible to automatically extract features from the unstructured data that contribute to improving the accuracy of outcome prediction, and to construct a model for predicting the outcome based on those features. [Explanation of symbols]

[0103] 100 Information Processing Systems 101 Information processing equipment 102 Network 103 User Terminals 104 databases 110 Storage section 120 Arithmetic section 121 Feature Extraction Model Construction Unit 122 Feature Extraction Unit 123 Outcome Prediction Model Building Department 130 Communications Department 1024 Prediction contribution information selection unit

Claims

1. An information processing device having a storage unit and a calculation unit, The storage unit unstructured information about the target to be predicted; Result information regarding the prediction target; and additional information relating to the prediction target; The calculation unit a feature extraction model construction unit that constructs a feature extraction model that predicts a result based on a plurality of first unstructured data that are a part of the unstructured information and the result information for each of the first unstructured data; a feature extraction unit that inputs a plurality of second unstructured data, which are the remaining part of the unstructured information, into the feature extraction model and extracts features that contribute to the prediction accuracy of the results; a result prediction model construction unit that constructs a result prediction model that predicts the result based on the feature amount, the auxiliary information of each of the plurality of second unstructured data, and the result information of each of the second unstructured data; An information processing device comprising:

2. The feature extraction model construction unit The information processing apparatus according to claim 1 , wherein a machine learning model having an intermediate layer is constructed as the feature extraction model.

3. The feature extraction unit 3. The information processing apparatus according to claim 2, wherein a plurality of the second unstructured data are input to the feature extraction model, and the feature is extracted by acquiring the value of the intermediate layer as a feature vector.

4. The result prediction model construction unit The information processing apparatus according to claim 1 , wherein the feature quantities that satisfy a predetermined prediction accuracy are selected, and the result prediction model is constructed using the selected feature quantities.

5. The result prediction model construction unit 5. The information processing apparatus according to claim 4, wherein the feature amount is selected using a prediction correctness rate of the result as the prediction accuracy.

6. 2. The information processing apparatus according to claim 1, further comprising a prediction contribution information selection unit that selects prediction contribution information that contributes to the prediction accuracy from the unstructured information based on the feature extraction model.

7. The unstructured information is A plurality of document data relating to a predetermined activity is included, The result information is The results of the predetermined activities include whether or not the project will be concluded, The additional information is The information processing device according to claim 6 , further comprising structured data relating to at least the prediction target.

8. The prediction contribution information selection unit inputting the plurality of document data as the second unstructured data into the feature quantity extraction model to calculate a deal success probability of the deal success probability; masking each word contained in the document data and calculating a predicted probability when each word is masked; Selecting the words whose change range of the predicted probability compared to the probability of contracting the deal exceeds a predetermined threshold as words that contribute to the prediction of whether or not the deal will be concluded; 8. The information processing apparatus according to claim 7, wherein the document data including the words that contribute to the prediction is input to the feature extraction model.

9. The unstructured information is The predetermined activity includes the document data related to sales activities, The result information is 9. The information processing apparatus according to claim 8, wherein the predetermined activity includes a result of the sales activity as to whether or not the deal is concluded.

10. An information processing system in which an information processing device and a user terminal are connected via a network, The user terminal: A user inputs predetermined information, and the information processing device outputs the results of the calculation. The information processing device includes: A memory unit, a calculation unit, and a communication unit are included. The calculation unit a feature extraction model construction unit that constructs a feature extraction model that predicts a result based on a plurality of first unstructured data that are part of unstructured information and result information for each of the first unstructured data; a feature extraction unit that inputs a plurality of second unstructured data, which are the remaining part of the unstructured information, into the feature extraction model and extracts features that contribute to the prediction accuracy of the results; a result prediction model construction unit that constructs a result prediction model that predicts the result based on the feature amount, auxiliary information of each of the plurality of second unstructured data, and result information of each of the second unstructured data, The communication unit An information processing system, characterized in that communication is performed between the information processing device and the user terminal via the network.

11. The feature extraction model construction unit of the information processing device 11. The information processing system according to claim 10, wherein a machine learning model having an intermediate layer is constructed as the feature extraction model.

12. The feature extraction unit of the information processing device 12. The information processing system according to claim 11, wherein a plurality of the second unstructured data are input to the feature extraction model, and the feature is extracted by acquiring values ​​of the intermediate layer as feature vectors.

13. a storage step of storing unstructured information related to the prediction target, result information related to the prediction target, and auxiliary information related to the prediction target; a feature extraction model that predicts a result based on a plurality of first unstructured data that are part of the unstructured information and the result information for each of the first unstructured data; a feature extraction model construction step for constructing the a feature extraction step of inputting a plurality of second unstructured data, which are the remaining part of the unstructured information, into the feature extraction model and extracting features that contribute to the prediction accuracy of the results; a result prediction model construction step of constructing a result prediction model that predicts the result based on the feature amount, the auxiliary information of each of the plurality of second unstructured data, and the result information of each of the second unstructured data; An information processing method comprising:

Images