Information processing device, information processing method, and computer program

Abstract

To provide an information processing device for performing accurate corporation evaluation on the basis of document data on a corporation.SOLUTION: An information processing device has: a corporation evaluation phrase storage unit for storing a corporation evaluation phrase composed of two or more words related to corporation evaluation; a text data preprocessing unit for acquiring documents on a corporation, extracting text data from the documents, and generating a plurality of fragment data by dividing the extracted text data by the number of prescribed words; and a corporation evaluation unit for calculating a corporation evaluation value by a corporation evaluation model being a learned neural network where learning is performed so as to calculate the corporation evaluation value on the basis of the corporation evaluation phrase.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a computer program. [Background technology]

[0002] In recent years, it has become common to evaluate corporations, including companies, local governments, and university corporations. Corporate evaluations involve analyzing and evaluating corporations not limited to numerical data such as operating profit and sales revenue, which appear in financial statements, but also qualitative information not found in financial statements. Qualitative information includes, for example, human capital such as employee abilities and skills; manufacturing capital such as infrastructure for product manufacturing; intellectual capital such as patents, know-how, and technological capabilities; natural capital, which refers to renewable and non-renewable natural resources such as plants, air, and animals; and social capital, which refers to relationships between people, such as trust and norms. By including such qualitative information not found in financial statements, a multifaceted corporate evaluation can be performed.

[0003] As a way to evaluate corporations based on qualitative information, it is being considered to evaluate corporations from three perspectives, for example, ESG (Environment, Social, and Governance).

[0004] For example, in Patent Document 1, news articles are collected and classified by date and company, and similarity between news articles is analyzed. For news articles with similarity above a threshold value, After clustering, the technology disclosed classifies news articles into the categories of E (environment), S (society), and G (governance), and calculates ESG risk on a cluster-by-cluster basis to calculate an ESG corporate evaluation score. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-504789 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology in Patent Document 1 collects news from the Internet, which creates a problem in that the ESG risk scores differ between companies that do not make the news and companies that do, resulting in scores that deviate from reality.

[0007] In addition, because it was based on the frequency of ESG-related words, there was also the problem that if a word that is actually related to ESG is not registered as an ESG-related word, the correct score could not be calculated. Also, because it is based on word frequency, if the same word is used repeatedly in a short news article, the score could be higher than the actual efforts of the company.

[0008] Therefore, in the present invention, for each corporate evaluation item, a phrase consisting of two or more words related to that corporate evaluation item is registered, and based on the registered corporate evaluation phrase, the degree of relevance is calculated as a corporate evaluation value from fragment data consisting of a predetermined number of words, thereby performing corporate evaluation without being influenced by the frequency of word occurrence. [Means for solving the problem]

[0009] The present invention provides an information processing device having a corporate evaluation phrase memory unit that stores corporate evaluation phrases consisting of two or more words related to corporate evaluation, a text data preprocessing unit that acquires documents related to corporations, extracts text data from the documents, and generates multiple fragment data by dividing the extracted text data by a predetermined number of words, and a corporate evaluation unit that calculates a corporate evaluation value using a corporate evaluation model 150, which is a trained neural network that has been trained to calculate a corporate evaluation value based on the fragment data contained in the documents and the corporate evaluation phrases.

[0010] The text data preprocessing unit may also generate fragment data by dividing the text data into a predetermined number of words while shifting the text data word by word.

[0011] Furthermore, the corporation evaluation phrase storage unit may store a plurality of corporation evaluation phrases in association with the corporation evaluation items, and the corporation evaluation unit may calculate a corporation evaluation value for each of the corporation evaluation items.

[0012] In addition, the trained neural network of the corporate evaluation unit may use part of BERT.

[0013] The system further includes a corporate evaluation phrase adding unit that adds new corporate evaluation phrases to the corporate evaluation phrase storage unit, and the corporate evaluation phrase adding unit uses a corporate evaluation model trained based on corporate evaluation phrases and fragment data generated from learning documents by the text preprocessing unit to calculate the corporate evaluation relevance of fragment data generated from documents related to the acquired corporation, calculates the number of occurrences of phrases corresponding to fragment data calculated to have corporate evaluation relevance, and adds a predetermined number of phrases, starting with the phrases that appear most frequently, to the corporate evaluation phrase storage unit as corporate evaluation phrases.

[0014] The corporate evaluation phrase adding unit further calculates the similarity between frequently occurring phrases and corporate evaluation phrases stored in the corporate evaluation phrase storage unit, and adds frequently occurring phrases having a similarity equal to or greater than a predetermined value to the corporate evaluation phrase storage unit.

[0015] The corporate evaluation phrase adding unit further re-learns the corporate evaluation model based on the corporate evaluation phrases stored in the corporate evaluation word storage unit to which the new corporate evaluation phrases have been added and on fragment data generated from the learning documents.

[0016] Furthermore, the phrase corresponding to the fragment data may be the two words that appear next to the fragment data.

[0017] In the information processing device of the present invention, the corporate evaluation unit may calculate the corporate evaluation value of a document related to a corporation by averaging the corporate evaluation values ​​calculated for all fragment data included in the document related to a corporation.

[0018] The information processing method of the present invention provides an information processing method executed by a computer, comprising the steps of: storing corporate evaluation phrases, which are phrases related to corporate evaluation; acquiring documents related to corporations, extracting text data from the documents, and generating multiple fragment data by dividing the extracted text data by a predetermined number of words; and calculating a corporate evaluation value using the fragment data contained in the documents and a corporate evaluation model, which is a trained neural network trained to calculate a corporate evaluation value based on the corporate evaluation phrases.

[0019] The computer program of the present invention provides a computer program that causes a computer to execute the following steps: storing corporate evaluation phrases, which are phrases related to corporate evaluation; acquiring documents related to corporations, extracting text data from the documents, and dividing the extracted text data into a predetermined number of words to generate multiple pieces of fragment data; and calculating a corporate evaluation value using the fragment data contained in the documents and a corporate evaluation model, which is a trained neural network trained to calculate a corporate evaluation value based on the corporate evaluation phrases. [Effects of the Invention]

[0020] According to the present invention, by calculating a corporate evaluation value using a corporate evaluation model trained with corporate evaluation phrases consisting of two or more words related to corporate evaluation, it is possible to calculate an accurate evaluation value regardless of the frequency of occurrence of words related to corporate evaluation. Furthermore, by using BERT as part of the neural network that constitutes the corporate evaluation model, it is possible to calculate an evaluation that takes into account the context of the entire sentence.

[0021] Furthermore, the corporate evaluation model, which is a trained neural network used to calculate evaluation values, is a phrase that is assigned to fragment data that has calculated a high corporate evaluation value.Phrases that appear frequently are sequentially added and registered as corporate evaluation phrases for the corporate evaluation item in question, and the neural network can be re-trained, so that accurate evaluation values ​​can be calculated even if there are trends or terms that change daily. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device according to the present invention. [Figure 2] FIG. 2 is a flowchart showing the process of generating fragment data in the text data preprocessing unit 110 of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of fragment data generated in the fragment data generation process performed by the text data preprocessing unit 110. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of the configuration of a neural network of the corporate valuation model 150 according to the present invention. [Figure 5] FIG. 5 is a diagram showing an example of a database in which corporate evaluation phrases are stored in the corporate evaluation phrase storage unit 130. As shown in FIG. [Figure 6] FIG. 6 is a flow diagram showing an outline of the update learning of the corporate evaluation model 150 of the present invention. [Figure 7] FIG. 7 is a block diagram showing a model of the procedure for learning the corporate evaluation model 150 for a certain corporate evaluation item in the corporate evaluation model learning stage 601. As shown in FIG. [Figure 8] FIG. 8 is a block diagram showing a procedure for additionally storing corporate evaluation phrases in the corporate evaluation phrase storage unit 130 in the additional storage step 602 of corporate evaluation phrases. [Figure 9] FIG. 9 is a block diagram illustrating the process of calculating a corporate evaluation in the corporate evaluation unit 140. [Figure 10]FIG. 10 is a diagram showing an example of the hardware configuration of the information processing device 100 according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functions and configurations are designated by the same reference numerals, and redundant description will be omitted.

[0024] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device according to the present invention. The information processing device 100 according to the present invention includes a text data preprocessing unit 110, a corporation evaluation phrase adding unit 120, a corporation evaluation phrase storage unit 130, a corporation evaluation unit 140, and a corporation evaluation model 150. The text data preprocessing unit 110 acquires a document related to a corporation input to the information processing device 100, extracts text data from the document, and generates multiple fragment data by dividing the extracted text data into a predetermined number of words. The document data refers to, for example, corporate documents used in corporation evaluation, such as annual reports and investor information, which have been digitized.

[0025] If the corporate document data is PDF data or image data, the text data preprocessing unit 110 may extract text data from such data. Furthermore, the text data preprocessing unit 110 performs a cleaning process on the text data when generating fragment data. This cleaning process involves removing tags and punctuation marks that may become noise, such as uppercase and lowercase letters, commas, and periods in the case of English, before processing using a natural language processing model. The text data preprocessing unit 110 may also perform processes such as dividing sentences into words, normalizing verbs to their base forms, and removing stop words such as particles and auxiliary verbs in Japanese and articles and prepositions in English. Details of these processes will be described later.

[0026] When generating fragment data from text data extracted from a document, the text data preprocessing unit 110 divides the text data into a predetermined number of words while shifting each word. At this time, fragment data may be generated by dividing the text data into a predetermined number of words while shifting each word after processing such as cleaning, normalization, and stop word removal has been performed on the text data. By dividing the text data while shifting each word, the content contained in the fragment data differs by only one word from the fragment data generated previously. Note that, although only one word is used here, this is not a limitation, and fragment data may also be generated by dividing into a predetermined number of words while shifting two words at a time.

[0027] The corporate evaluation phrase adding unit 120 adds new corporate evaluation phrases to the corporate evaluation phrase storage unit 130. A corporate evaluation phrase is a phrase consisting of at least two words that is considered to be highly relevant to a certain corporate evaluation item in a certain corporate evaluation item. The corporate evaluation phrase adding unit 120 trains the corporate evaluation model 150, which is a neural network, based on one or more corporate evaluation phrases stored in the corporate evaluation phrase storage unit 130 and multiple fragment data generated from training documents.

[0028] Furthermore, based on the corporate evaluation model 150 thus trained, the corporate evaluation phrase adding unit 120 calculates the corporate evaluation relevance of each piece of fragment data generated from newly acquired corporate documents and calculates the frequency of occurrence of phrases corresponding to fragment data calculated to have a corporate evaluation relevance. A phrase corresponding to fragment data may be the two words that appear after the last word included in the fragment data, or the two words immediately preceding a word included in the fragment data. While a phrase is defined as two words here, any phrase longer than two words, including three words, may be used. The corporate evaluation phrase adding unit 120 labels such phrases as phrases corresponding to fragment data. The corporate evaluation phrase adding unit 120 may add and store a predetermined number of phrases, in descending order of frequency of occurrence, as corporate evaluation phrases in the corporate evaluation phrase storage unit 130.

[0029] The corporation evaluation phrase adding unit 120 may be configured to calculate the similarity between frequently occurring phrases and the corporation evaluation phrases stored in the corporation evaluation phrase storage unit 130, and add frequently occurring phrases having a similarity equal to or greater than a predetermined value to the corporation evaluation phrase storage unit. By additionally storing only those frequently occurring phrases whose similarity to corporation evaluation phrases already stored in the corporation evaluation phrase storage unit 130 is equal to or greater than a predetermined value, it is possible to store only phrases that are highly relevant to the corporation evaluation item as corporation evaluation phrases.

[0030] The corporation evaluation phrase adding unit 120 further retrains the corporation evaluation model 150, which is a trained neural network, based on the corporation evaluation phrases stored in the corporation evaluation phrase storage unit 130 to which the newly added corporation evaluation phrases have been added and the fragment data generated by the text data preprocessing unit 110 from the training documents. Specifically, the corporation evaluation model 150 is retrained using the fragment data generated from the training documents so as to increase the corporation evaluation value of the fragment data associated with the newly added corporation evaluation phrases. By using the corporation evaluation model 150 retrained by the corporation evaluation phrase adding unit 120 in the corporation evaluation unit 140, it is possible to calculate a corporation evaluation value with higher accuracy. Furthermore, the corporation evaluation phrase adding unit 120 may perform processing to add further corporation evaluation phrases to the corporation evaluation phrase storage unit 130 using the retrained corporation evaluation model 150.

[0031] The corporate evaluation phrase storage unit 130 stores corporate evaluation phrases consisting of two or more words related to corporate evaluation. The corporate evaluation phrase storage unit 130 has multiple corporate evaluation items and stores multiple corporate evaluation phrases in association with each corporate evaluation item. The corporate evaluation phrase storage unit 130 also stores phrases that the corporate evaluation phrase adding unit 120 has decided to add as corporate evaluation phrases in association with the corporate evaluation items.

[0032] The corporate evaluation unit 140 calculates a corporate evaluation value using the corporate evaluation model 150, which is a trained neural network trained to calculate a corporate evaluation value based on fragment data included in documents related to a corporation and corporate evaluation phrases. The corporate evaluation unit 140 may also calculate a corporate evaluation value for each corporate evaluation item using the corporate evaluation model 150 for each corporate evaluation item.

[0033] Furthermore, the corporate evaluation model 150, which is the trained neural network of the corporate evaluation unit 140, may use part of BERT (Bidirectional Encoder Representations from Transformers), a natural language processing model. When calculating a corporate evaluation value, the corporate evaluation unit 140 calculates the corporate evaluation value by averaging the corporate evaluation values ​​calculated for all fragment data from documents related to the corporation. The corporate evaluation unit 140 calculates the corporate evaluation value using the corporate evaluation model 150 trained by the corporate evaluation phrase addition unit 120.

[0034] The corporate evaluation model 150 is a prediction model composed of a neural network that calculates the degree of relevance to a certain corporate evaluation item as a corporate evaluation value. There are multiple neural networks for calculating corporate evaluation values, and the number of such neural networks is equal to the number of corporate evaluation items. The corporate evaluation model 150 is trained by the corporate evaluation phrase addition unit 120, and the trained corporate evaluation model 150 is used by the corporate evaluation unit 140 to calculate the corporate evaluation value.

[0035] The corporation evaluation model 150 is also used when the corporation evaluation phrase adding unit 120 adds and stores corporation evaluation phrases in the corporation evaluation phrase storage unit 130. The trained corporation evaluation model 150 calculates a corporation evaluation value to determine the corporation evaluation relevance of the phrase corresponding to the fragment data. In addition, the corporation evaluation model 150 for a certain corporation evaluation item may be retrained by the corporation evaluation phrase adding unit 120 when a corporation evaluation phrase is added and stored in the corporation evaluation phrase storage unit 130 for the same corporation evaluation item.

[0036] 2 is a flowchart showing the process of generating fragment data in the text data preprocessing unit 110 of the present invention. When the text data preprocessing unit 110 receives document data related to a corporation, such as an annual report or an investor information document, input to the information processing unit 100, it extracts text data from the document data (step S201). The text data, which is the document content, is extracted not only from text data but also from various data file formats such as DOC and PDF.

[0037] Next, the text data preprocessing unit 110 performs a cleaning process on the extracted text data (step S202). The cleaning process is a process for removing noise from the text data when subsequent natural language processing is performed. For example, tags such as html, line breaks, extra spaces and lines, and punctuation marks are removed. In the case of English, to eliminate the distinction between uppercase and lowercase letters, uppercase letters are converted to lowercase, and commas and periods are removed. Subscripts and superscripts are also converted to normal-sized characters. For example, CO2 is converted to CO2. If necessary, special characters such as symbols and emojis are also removed so that only text information remains. Note that the above is an example, and the information to be deleted is not limited to these.

[0038] The text data that has undergone the cleaning process is subjected to word normalization processing by the text data preprocessing unit 110 (step S203). Here, the words contained in the text data are standardized to reduce the amount of calculation and memory usage in subsequent processing. The normalization process, for example, standardizes spelling and variations in notation, and standardizes differences between full-width and half-width characters, hiragana, kanji, and katakana. It also performs processing to return verbs and nouns to their basic forms.

[0039] The text data preprocessing unit 110 then removes stop words (step S204). In the stop word removal process, words that are not subject to processing in natural language processing are removed as stop words, thereby reducing the data size. Words to be removed as stop words include, for example, particles, auxiliary verbs, English articles (e.g., a, an, the), prepositions (e.g., on), and conjunctions (e.g., and). These stop words may be removed by storing a list of words to be removed in advance and removing them from the text data. The stop word list may be set by the user, or an existing list may be used.

[0040] The text data preprocessing unit 110 performs tokenization (step S205). Tokenization involves dividing a sentence into words, which are its constituent elements, and converting the words into numerical codes corresponding to the words so that they can be input into the corporate reputation model 150, which is a natural language processing model. In this embodiment, since the BERT Transformer is used as part of the neural network that constitutes the corporate reputation model 150, the BERT tokenizer is used to tokenize (divide into words) the text data based on the BERT dictionary, and convert the tokens into numerical codes. These numbers are codes assigned in BERT to correspond to each word. The BERT used is the BERT for the language corresponding to the language of the sentence.

[0041] The text data preprocessing unit 110 performs fragmentation (step S206). Specifically, the text data preprocessing unit 110 divides the text data into a predetermined number of words to generate fragment data. The predetermined number of words is, for example, 60 words, but is not limited to this. If fragment data is composed of a short number of words, the accuracy of the evaluation calculation by the corporate evaluation model 150 decreases, while longer fragment data increases the calculation time. Therefore, the predetermined number of words is set according to the processing speed of the computer, such as the CPU. For this reason, the number of words included in the fragment data should be between 30 and 150 words, and more preferably between 40 and 80 words. Fragment data is generated by dividing tokenized text data that has undergone preprocessing such as cleaning, normalization, and stop word removal. At this time, the text data preprocessing unit 110 divides the tokenized text data into a predetermined number of words by shifting each word.

[0042] FIG. 3 is a diagram showing an example of fragment data generated in the fragment data generation process performed by the text data preprocessing unit 110. The data shown here is tokenized text data. Therefore, each word has been converted into a number, which is the code number corresponding to the word. In the fragmentation process, fragment data is generated by dividing the data into a predetermined number of words, one word at a time, shifting the data by one code at a time. Therefore, fragment data 1 and fragment data 2 are fragment data that differ by one word, but contain the same number of words. In this way, fragment data is generated by sequentially fragmenting text data contained in document data related to corporations.

[0043] FIG. 4 is a diagram showing an example of a neural network configuration of the corporate valuation model 150 according to the present invention. The neural network of the corporate valuation model 150 is generated for each corporate evaluation item. Therefore, there will be as many corporate valuation model 150 neural networks as there are corporate evaluation items. The corporate valuation model 150 according to the present invention is, for example, a Transformer consisting of eight layers. The corporate valuation model 150 outputs vector quantities as corporate valuation values ​​for sentence fragments consisting of 60 words contained in the input fragment data. The eight layers are an input layer 401, an embedding block layer 402, a transformer block layer 403, a global average pooling (Global Average Pooling to One Dimension) layer 404, a first dropout layer 405, a first fully connected (Dense) layer 406, a second dropout layer 407, and a second fully connected layer 408.

[0044] Here, an example will be described in which fragment data contains information on 60 words. The corporate reputation model 150 receives fragment data. The input layer 401 receives 60 tokenized tensors corresponding to the 60 words as fragment data. The embedding block layer 402, for example, is configured with a BERT embedding layer, which reprojects words into multiple dimensions while reducing the tensor dimension and clustering similar words. In the case of the English BERT, the vocabulary size is 30,522, so the matrix is ​​converted to a 60 × 30,522 matrix. However, each row is a one-hot vector. For example, when encoding the integer 0, the first element is 1, but all other elements are 0. Applying this matrix directly to a neural network would result in slow processing. To speed up processing, the embedding block layer 402 converts the 60 × 30,522 matrix to one with 60 × 768 dimensions. This embedding block layer 402 has 1,109,482,240 parameters and is trained.

[0045] The transformer block layer 403 is configured based on a self-attention mechanism, for example, multi-head attention, and has the function of emphasizing necessary information and patterns, such as attention expressions. The transformer block layer 403 includes, for example, eight-head attention, which executes eight head attentions in parallel, and a position-wise feed-forward network. The transformer block layer focuses only on keywords, attention expressions, and patterns related to the corporate evaluation items targeted by the corporate evaluation model 150. This transformer block layer 403 has, for example, 18,946,592 parameters and is trained.

[0046] The global average pooling layer 404 is a one-dimensional layer that reduces the dimension by averaging. The 60 elements are projected onto a point in the 768-dimensional space defined by the embedding block layer 402.

[0047] In order to prevent overlearning, first dropout layer 405 and second dropout layer 407 set, for example, 10% of the output to 0. Note that the output percentage set to 0 is not limited to 10%.

[0048] The first fully connected layer (Dense Layer) 406 and the second fully connected layer 408 are used to connect the neural network. The first fully connected layer 406 reduces the dimension from 768-dimensional space to 64. The first fully connected layer 406 is configured, for example, with a ReLU (Rectified Linear Unit) function and has 49,216 parameters.

[0049] The second fully connected layer 408 reduces the dimension to one and outputs a numerical value that is the evaluation value of a certain corporate evaluation item. The numerical value is between 0 and 1. The fully connected layer 408 is also configured with a sigmoid function and has 130 parameters.

[0050] FIG. 5 is a diagram showing an example of a database storing corporate evaluation phrases in the corporate evaluation phrase storage unit 130. The corporate evaluation phrase storage unit 130 stores corporate evaluation phrases consisting of two or more words related to corporate evaluation. A plurality of corporate evaluation phrases are stored in association with corporate evaluation items. In FIG. 5, three corporate evaluation items are shown as examples: "Green house gas," "Community," and "Air pollution," and a plurality of corporate evaluation phrases are stored in association with these corporate evaluation items. For example, five word pairs are stored for "green house gas": (green, gas), (gas, emission), (reduce, CO), (goal, zero), and (warning, climate).

[0051] Note that the number of corporate evaluation phrases stored is not limited to five, and more may be stored. Also, while only three corporate evaluation items are illustrated here, the number is not limited to this. For example, corporate evaluation phrases may be stored for 13 ESG-related evaluation items: air pollution, greenhouse gases, water consumption, mining consumption, safety and health, human rights, work environment, community, domestic job creation, domestic reflux rate, governance risk, production cost, and economic ripple effect. Furthermore, since there are as many corporate evaluation models 150 as there are corporate evaluation items, there will be as many corporate evaluation models 150 as there are corporate evaluation items stored in the corporate evaluation phrase storage unit.

[0052] The corporate evaluation phrases stored in association with corporate evaluation items are corporate evaluation phrases that appear in sentences that have a high degree of relevance to a certain corporate evaluation item. Fragment data corresponding to a corporate evaluation phrase stored in the corporate evaluation phrase storage unit 130 has a high degree of relevance to that corporate evaluation item. A corporate evaluation phrase corresponding to a certain piece of fragment data is two words that are located close to the fragment data, for example, two words that appear in a sentence following a word included in the fragment data, or may be two words that appear immediately before a word included in the fragment data. Note that although two words are used here, this is not limited to this, and three words may also be used.

[0053] The corporate evaluation phrases stored in the corporate evaluation phrase storage unit 130 are not only pre-stored corporate evaluation phrases, but also corporate evaluation phrases that are determined by the corporate evaluation phrase adding unit 120 to have a high degree of relevance to the corporate evaluation item.

[0054] FIG. 6 is a flow diagram outlining the update learning of the corporate evaluation model 150 of the present invention. Update learning of the corporate evaluation model 150 of the present invention has two stages: a corporate evaluation model learning stage (601) and an additional storage stage for corporate evaluation phrases (602). The corporate evaluation model learning stage (601) is the learning of the corporate evaluation model 150, and the initial learning of the corporate evaluation model 150 begins from this corporate evaluation model learning stage. Prior to the initial learning of the corporate evaluation model, several phrases highly relevant to corporate evaluation items are pre-registered in the corporate evaluation phrase storage unit 130. The corporate evaluation model 150 is then trained based on the fragment data generated from the learning document by the text data preprocessing unit 110 and the corporate evaluation phrases stored in the corporate evaluation phrase storage unit 130.

[0055] The corporate evaluation phrase additional storage stage (602) uses the corporate evaluation model 150 learned in the corporate evaluation model learning stage (601) to input fragment data generated from documents related to the acquired learning corporation, and additionally stores corporate evaluation phrases that meet specified conditions in the corporate evaluation phrase storage unit 130.

[0056] If a corporate evaluation phrase is additionally stored in the corporate evaluation phrase storage unit 130 in the corporate evaluation phrase additional storage step (602), the process returns to the corporate evaluation model learning step (601). The corporate evaluation model 150 is re-trained using the corporate evaluation phrases stored in the updated corporate evaluation phrase storage unit 130. Note that if no corporate evaluation phrase is additionally stored in the corporate evaluation phrase additional storage step (602), the process does not return to the corporate evaluation model learning step 601.

[0057] 7 is a block diagram showing the procedure for learning the corporate evaluation model 150 for a certain corporate evaluation item in the corporate evaluation model learning stage 601. Before the corporate evaluation model learning stage, the learning document prepared for learning is first converted into text data by the text data preprocessing unit 110, which is then divided into a predetermined number of words to generate tokenized fragment data. The fragment data generated by the text data preprocessing unit 110 is then input sequentially to the corporate evaluation phrase adding unit 120 (701).

[0058] Next, the corporation evaluation phrase adding unit 120 assigns a phrase corresponding to the input fragment data as a label for that fragment data (702). Here, as an example, the two words that appear in the sentence following the word included in the fragment data are assigned as the label for that fragment data. In other words, if the fragment data contains information for 60 words, the phrase corresponding to that fragment data would be the 61st and 62nd words, assuming the first word included in the fragment data in the sentence is word 1. As shown in Figure 3, the two words following the word data (tokenized) that make up fragment data 1 become the phrase corresponding to that fragment data and are assigned as the label.

[0059] The corporate evaluation phrase adding unit 120 determines whether the phrase labeled in the fragment data is a corporate evaluation phrase that is a corporate evaluation item in the corporate evaluation model 150 being trained and is stored in the same corporate evaluation item in the corporate evaluation phrase storage unit 130 (703). The corporate evaluation phrase storage unit 130 stores in advance several corporate evaluation phrases that are highly relevant to the corporate evaluation item. If the phrase labeled in the fragment data is stored in the corporate evaluation phrase storage unit 130, the corporate evaluation phrase adding unit 120 assigns an evaluation value of "1" to the fragment data. If the phrase is not stored, the corporate evaluation phrase adding unit 120 assigns an evaluation value of "0" to the fragment data.

[0060] For example, when training the corporate evaluation model 150 for Community, if the phrase labeled to the fragment data is "local, community," the same phrase "local, community" is stored in the corporate evaluation phrase storage unit 130, and therefore a "1" is assigned as the evaluation value for that fragment data. However, when training the corporate evaluation model 150 for Green House Gas, "local, community" is not stored for Green House Gas, and therefore the evaluation value for the fragment data is "0." In this way, as training data for training the corporate evaluation model 150 for each corporate evaluation item, fragment data and corresponding phrases are assigned evaluation values ​​of "0" or "1" for that fragment data, and training data is generated. Note that when determining whether a phrase labeled to fragment data is a corporate evaluation phrase stored in the corporate evaluation phrase storage unit 130, the order of the two words may be taken into consideration. For example, if "local, community" is stored in the corporate evaluation phrase storage unit 130 and the phrase labeled to the fragment data is "community, local," the order is reversed, and therefore the phrase is not stored, and a "0" may be assigned.

[0061] The corporate evaluation phrase adding unit 120 trains the neural network, which is the corporate evaluation model 150, by sequentially inputting the generated fragment data (704). At this time, the fragment data for training is prepared so that the number of fragment data to which "0" is assigned is greater than the number of fragment data to which "1" is assigned, for example, the number of fragment data to which "0" is assigned is 20 times the number of fragment data to which "1" is assigned, and weights are added during the training stage. The corporate evaluation model 150 is trained, and the parameters of the corporate evaluation model 150 are adjusted so that the correct evaluation value is output.

[0062] This training is performed using cross-validation (705). Specifically, k-fold cross-validation is performed. That is, cross-validation is performed by using 10% of the fragment data generated from the training document data as validation data and the remaining 90% as training data. The 10% validation data is set so that the ratio of the number of fragment data labeled "0" to the number of fragment data labeled "1" is the same as the ratio in the remaining 90% training data. Furthermore, cross-validation may be performed in different epochs with different allocations of training data and validation data.

[0063] The corporate evaluation model 150 is repeatedly trained. For example, training may be performed for 50 epochs. To avoid overtraining, training may be terminated early after 20 epochs. When training is completed after a predetermined number of epochs, the process proceeds to the additional storage stage 602 of corporate evaluation phrases.

[0064] 8 is a block diagram showing the procedure for additionally storing corporate evaluation phrases in the corporate evaluation phrase storage unit 130 in the corporate evaluation phrase additional storage step 602. This corporate evaluation phrase additional storage step is also performed for each corporate evaluation item. First, the corporate evaluation phrase adding unit 120 accepts input of fragment data generated by the text data preprocessing unit 110 from the corporate document data for training (801). The fragment data input here may be, for example, fragment data generated from the corporate document data for training that was not used in the training step 601 of the corporate evaluation model 150.

[0065] The corporation evaluation phrase adding unit 120 assigns a phrase corresponding to the input fragment data as a label for that fragment data (802). This label assignment is the same process as 702 in the corporation evaluation model learning stage 601.

[0066] The corporate evaluation phrase adding unit 120 inputs the fragment data into the corporate evaluation model 150 for the target corporate evaluation item (803). Once input, the corporate evaluation relevance, which is the evaluation value of the fragment data by the corporate evaluation model 150, is output (804). As explained in Figure 4, the output from the corporate evaluation model 150 is expressed as a number between 0 and 1. At this time, if the output result from the corporate evaluation model 150 is 0.5 or greater, the corporate evaluation phrase adding unit 120 corrects it to "1," indicating that there is a corporate evaluation relevance, and if it is less than 0.5, it corrects it to "0," indicating that there is no corporate evaluation relevance. This is to widely extract phrases that may be related to the corporate evaluation item. Fragment data assigned "0" will not be used in this corporate evaluation phrase addition and storage stage.

[0067] Next, for the fragment data to which "1" has been assigned, the corporation evaluation phrase adding unit 120 determines whether the phrase labeled with that fragment data is stored in the corporation evaluation phrase storage unit 130 in the same corporation evaluation item as the corporation evaluation model 150 for which the evaluation value was calculated (805). If the phrase is already stored in the corporation evaluation phrase storage unit 130, the corporation evaluation phrase adding and storing step 602 ends. If the phrase is not stored in the corporation evaluation phrase storage unit 130, the process proceeds to the next step.

[0068] The corporation evaluation phrase adding unit 120 extracts frequently occurring phrases that are not stored in the corporation evaluation phrase storage unit 130 (806). More specifically, when the number of phrases labeled with fragment data assigned a "1" based on the evaluation value of a certain corporation evaluation item by the corporation evaluation model 150 and that are not stored in the corporation evaluation phrase storage unit 130 reaches a certain number, the number of times those phrases appear is counted and the phrases are ranked in descending order of frequency of appearance (806). For example, the phrases are ranked by frequency of appearance, and the top 10 or top 30 phrases are extracted in descending order of frequency of appearance.

[0069] The corporation evaluation phrase adding unit 120 calculates (807) the cosine similarity between the extracted phrase of the additional storage candidate and the corporation evaluation phrase included in the corresponding corporation evaluation item in the corporation evaluation phrase storage unit 130. For each of the 10 or 30 phrases extracted in order of frequency of appearance, such as the top 10 or top 30, the cosine similarity is calculated with the corporation evaluation phrase stored in the corporation evaluation phrase storage unit 130. For example, BERT is used to calculate the cosine similarity.

[0070] The corporation evaluation phrase adding unit 120 determines whether the cosine similarity calculated for the candidate phrase for additional storage is equal to or greater than a predetermined value. If it is equal to or greater than the predetermined value, the phrase is stored in the corresponding corporation evaluation item in the corporation evaluation phrase storage unit 130 (808). If it is less than the predetermined value, the phrase is determined to have low relevance to the corporation evaluation item and is not stored. The predetermined value is the cosine similarity calculated in 807. For example, if the cosine similarity is 60% or greater, the phrase is stored in the corporation evaluation phrase storage unit 130, and if it is less than 60%, it is not stored. The cosine similarity is calculated between all corporation evaluation phrases stored in the corporation evaluation item, and the average cosine similarity calculated between each corporation evaluation phrase and the candidate phrase for additional storage is calculated. Whether to store the phrase for additional storage may be determined based on whether the average cosine similarity calculated is equal to or greater than a predetermined value.

[0071] When a new phrase is stored in the corporate evaluation phrase storage unit 130, the process returns to the corporate evaluation model learning stage 601, and the corporate evaluation model 150 for the corporate evaluation item for which the phrase has been newly added and stored is re-learned. The process may be terminated by returning to the corporate evaluation model learning stage 601 and re-learning, or the process of the corporate evaluation phrase additional storage stage (602) may be performed again using the re-learned corporate evaluation model.

[0072] FIG. 9 is a block diagram illustrating the process of calculating a corporate evaluation in the corporate evaluation unit 140. The corporate evaluation unit 140 calculates a corporate evaluation value using the corporate evaluation model 150 trained by the corporate evaluation phrase adding unit 120. First, corporate document data to be used for corporate evaluation is input to the information processing device 100. The input corporate document data is converted into text data, if necessary, in the text data preprocessing unit 110, and the text data is divided to generate fragment data. The fragment data generated by the text data preprocessing unit 110 is input to the corporate evaluation unit 140 (901).

[0073] Next, the corporate evaluation unit 140 calculates a corporate evaluation value for the input fragment data using the trained corporate evaluation model 150 (902). The trained corporate evaluation model 150 is a corporate evaluation model 150 trained based on the training fragment data and the corporate evaluation phrases stored in the corporate evaluation phrase storage unit 130, and there is a trained corporate evaluation model 150 for each corporate evaluation item. Therefore, when calculating corporate evaluation values ​​for all corporate evaluation items, one piece of fragment data is input into each corporate evaluation model 150, and a corporate evaluation value for each corporate evaluation item is obtained.

[0074] The corporate evaluation unit 140 calculates the corporate evaluation value of the corporation by averaging the corporate evaluation values ​​calculated for each fragment data included in one or more corporate document data (903). Specifically, for example, the overall corporate evaluation for a certain corporate evaluation item in one or more corporate document data is calculated using the following formula:

[0075]

number

[0076] Here, Tendency jis the trend of one or more corporate document data entered for the corporate evaluation items, i.e., the corporate evaluation value. The corporate evaluation value is expressed as a number between 0 and 1, with the closer to 1 the higher the relevance. j is the corporate evaluation item, and i and N are the number of fragment data extracted from the corporate document. The one or more documents are, for example, annual reports or integrated reports. The overall corporate evaluation is calculated for each corporate evaluation item based on all the corporate evaluation values ​​of the fragment data contained in the documents used to evaluate the corporation. The output of the overall corporate evaluation is not simply a numerical output, but can also be configured to be output and displayed as a radar chart or bar graph based on the corporate evaluation values ​​calculated for each corporate evaluation item.

[0077] Fig. 10 is an example of a hardware configuration diagram constituting an information processing device 100 according to the present invention. As shown in Fig. 10, the computer constituting the information processing device 100 is configured by connecting a CPU 11, a communication interface 12 connected to a network such as the Internet, a ROM 13, a RAM 14, a hard disk drive 15, an input / output interface 16, a display unit 17 connected to the input / output interface 16, a pointing device 18, and a keyboard 19 to a bus. An external storage device 20 such as a USB memory can be connected to the input / output interface 16.

[0078] The display unit 17 is, for example, a display device such as a liquid crystal display, etc. The pointing device 18 is, for example, a mouse or a trackball.

[0079] When a series of processes is executed by a program, for example, the functions of the text data preprocessing unit 110, the corporate evaluation phrase adding unit 120, the corporate evaluation unit 140, and the corporate evaluation model 150 are stored in the ROM 13 or the hard disk drive 15 as a computer program for corporate evaluation, and are executed by the CPU 11 to perform various functions. In addition, the corporate evaluation phrase storage unit 130 is stored in the ROM 13 or the hard disk drive 15 as part of the computer program for corporate evaluation.

[0080] The computer program for corporate evaluation is installed in the information processing device 100 by connecting an external storage device 20, such as a USB memory stick, in which the computer program for corporate evaluation is stored, to the input / output interface 16. The computer program may be installed in the information processing device 100 from the communication interface 12 via a network, or may be pre-installed in the information processing device itself, for example, in a ROM 13 in which the computer program is recorded.

[0081] Furthermore, the learning document data used for corporate evaluation and the document data for corporate evaluation may be stored in the external storage device 20, or may be stored in one or more other servers or storage devices on the network. The information processing device 100 may be configured to transmit the calculated corporate evaluation value in response to a corporate evaluation request received from an external information processing device via the network from the communication interface 12.

[0082] Furthermore, the description of this embodiment is merely an example, and components described in a single form may be implemented in multiple distributed forms, and similarly, components described in a distributed form may be implemented in a combined form. [Explanation of symbols]

[0083] 100 Information processing device 110 Text data preprocessing section 120 Corporate Evaluation Phrase Addition Section 130 Corporate Evaluation Phrase Memory 140 Corporate Evaluation Department 150 Corporate Valuation Model

Claims

1. a corporate evaluation phrase storage unit that stores corporate evaluation phrases each consisting of two or more words related to corporate evaluation; a text data preprocessing unit that acquires a document related to a corporation, extracts text data from the document, and generates a plurality of fragment data by dividing the extracted text data by a predetermined number of words; An information processing device having the fragment data contained in the document and a corporate evaluation unit that calculates a corporate evaluation value using a corporate evaluation model, which is a trained neural network trained to calculate a corporate evaluation value based on the corporate evaluation phrase.

2. The information processing apparatus according to claim 1 , wherein the text data preprocessing unit generates the fragment data by dividing the text data into a predetermined number of words while shifting the text data word by word.

3. 2. The information processing apparatus according to claim 1, wherein the corporation evaluation phrase storage unit stores a plurality of the corporation evaluation phrases in association with corporation evaluation items, and the corporation evaluation unit calculates a corporation evaluation value for each corporation evaluation item.

4. 2. The information processing device according to claim 1, wherein the trained neural network of the corporation evaluation unit uses a part of BERT.

5. The information processing device according to claim 1 comprises: a corporate evaluation phrase adding unit for adding a new corporate evaluation phrase to the corporate evaluation phrase storage unit; The corporate evaluation phrase adding unit An information processing device characterized in that a corporate evaluation model trained based on the corporate evaluation phrases and fragment data generated by the text data preprocessing unit from learning documents calculates the corporate evaluation relevance of the fragment data generated from documents related to the acquired corporation, calculates the number of occurrences of phrases corresponding to fragment data calculated to have the corporate evaluation relevance, and adds a predetermined number of phrases, in descending order of frequency of occurrence, as the corporate evaluation phrases to the corporate evaluation phrase storage unit.

6. The information processing device according to claim 5, characterized in that the corporate evaluation phrase adding unit further calculates the similarity between the frequently occurring phrase and the corporate evaluation phrase stored in the corporate evaluation phrase storage unit, and adds frequently occurring phrases having a similarity equal to or greater than a predetermined value to the corporate evaluation phrase storage unit.

7. The information processing device described in claim 5, characterized in that the corporation evaluation phrase addition unit further re-trains the corporation evaluation model based on the corporation evaluation phrases stored in the corporation evaluation word memory unit to which the new corporation evaluation phrases have been added and the fragment data generated from the learning document.

8. 6. The information processing apparatus according to claim 5, wherein the phrase corresponding to the fragment data is made up of two words that appear next to the fragment data.

9. The information processing apparatus according to claim 1 , wherein the corporate evaluation unit calculates the corporate evaluation value of the document related to the corporation by averaging the corporate evaluation values ​​calculated for all fragment data included in the document related to the corporation.

10. storing corporate evaluation phrases, which are phrases related to corporate evaluation; a step of acquiring a document related to a corporation, extracting text data from the document, and dividing the extracted text data into a predetermined number of words to generate a plurality of fragment data; An information processing method executed by a computer, comprising: a step of calculating a corporate evaluation value using a corporate evaluation model, which is a trained neural network trained to calculate a corporate evaluation value based on the fragment data contained in the document and the corporate evaluation phrase.

11. storing corporate evaluation phrases, which are phrases related to corporate evaluation; a step of acquiring a document related to a corporation, extracting text data from the document, and dividing the extracted text data into a predetermined number of words to generate a plurality of fragment data; A computer program characterized by causing a computer to execute the steps of: calculating a corporate evaluation value using a corporate evaluation model, which is a trained neural network trained to calculate a corporate evaluation value based on the fragment data contained in the document and the corporate evaluation phrase.

Images