Report evaluation method, and information processing device and program for executing said method

Abstract

To provide an evaluation technique capable of objectively evaluating reports, particularly integrated reports.SOLUTION: An information processing device is employed for the evaluation in this application. A logistic regression analysis is performed to create an evaluation model for co-occurrence pairs of terms contained within sentences described in existing selected reports and existing non-selected reports (Step S10). For the report under evaluation, the principal component score obtained from the logistic regression analysis is applied to the evaluation model as an explanatory variable, thereby evaluating the objective variable (Step S50).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for objectively evaluating the quality of a report, and is particularly suitable for evaluating an integrated report. [Background technology]

[0002] In recent years, ESG investment has been gaining attention. ESG investment is an investment that takes into account a company's ESG (environmental, social, and governance) information in addition to its financial information. Institutional investors gather ESG information from integrated reports issued by companies and use it to make investment decisions.

[0003] Generally, when writing a piece of writing, the author's subjectivity is always included. It is very difficult to objectively evaluate such writing. In response to this, a system for objectively evaluating writing has been proposed (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-140337 Summary of the Invention [Problem to be solved by the invention]

[0005] Even for integrated reports, it is difficult to objectively evaluate the quality of the report because there is no set format and the report can be edited freely, and the content varies depending on the industry and business type.

[0006] However, reports, especially integrated reports, have content that is unique to general writing. Even if a general writing evaluation system is applied, it is unclear whether it can be properly evaluated, taking into account the unique aspects of the report.

[0007] The present invention is intended to solve the above-mentioned problems, and aims to provide a technique for objectively evaluating reports, particularly integrated reports. [Means for solving the problem]

[0008] The inventors of the present application focused on three relationships: "integrated report," "logistic regression analysis," and "co-occurrence pairs."

[0009] Generally, the reason why it is difficult to objectively evaluate a piece of writing is that it is difficult to establish objective evaluation criteria. In other words, the evaluation criteria will vary depending on who is making the judgment.

[0010] In response to this, the Government Pension Investment Fund selects and publishes "excellent integrated reports" every year. Because the Government Pension Investment Fund is a public institution and the actual decision-makers are institutional investors, it can be said that the reports are as objective as possible.

[0011] Furthermore, we noted that integrated reports can be clearly divided into two types: those selected as "excellent integrated reports" and those not.

[0012] Regression analysis is used when trying to find a certain causal relationship between uncertain phenomena. One type of regression analysis is logistic regression analysis. Logistic regression analysis is a statistical method that predicts the probability of two dependent variables occurring from multiple explanatory variables. For example, if the dependent variable is the occurrence / non-occurrence of a disease, the cause of the disease occurrence can be estimated.

[0013] In general, it is difficult to clearly distinguish between good and bad writing, and it is hard to imagine applying logistic regression analysis to writing evaluation. However, taking into account the characteristics of integrated reports, we found that logistic regression analysis can be applied to quality evaluation.

[0014] In addition, logistic regression analysis always returns a value between 0 and 1 as the objective variable, making the evaluation results intuitive. This is preferable because it prevents investors from being confused about the outcome.

[0015] Furthermore, we found that there are certain trends in the content of integrated reports, which is a phenomenon unique to integrated reports, and that these trends can be extracted more reliably by focusing on co-occurrence pairs within a single sentence.

[0016] The present invention was arrived at through the above thought process.

[0017] The report evaluation method of the present invention uses an information processing device and includes the steps of: performing logistic regression analysis on terms contained in existing selected reports and existing non-selected reports to create an evaluation model; and applying the evaluation model to the report to be evaluated and evaluating it.

[0018] Generally, it is difficult to clearly distinguish between good and bad writing, but in cases where a clear distinction can be made between selected and non-selected reports, such as integrated reports, logistic regression analysis can be applied, allowing for an objective evaluation.

[0019] In the above invention, preferably, a logistic regression analysis is performed on the co-occurrence pairs of the terms contained in one sentence.

[0020] Compared with analyzing words, analyzing co-occurrence pairs improves evaluation accuracy.

[0021] In the above invention, preferably, the top N co-occurrence pairs of the terms contained in a sentence in existing selected reports are selected. The total number of existing selected reports and existing non-selected reports is d, and the number of selected co-occurrence pairs is N. A d-row, N-column matrix A relating to the occurrence probability is created. Principal component analysis is performed on matrix A to extract the first principal component through the K-th principal component. A d-row, K-column matrix B relating to the principal component scores from the first principal component through the K-th principal component is created. For matrix B, the objective variable for existing selected reports is set to 1, and the objective variable for existing non-selected reports is set to 0, and the regression coefficient for each principal component is calculated to create an evaluation model.

[0022] This eliminates unnecessary information and reduces the amount of calculation.

[0023] In the above invention, preferably, the report to be evaluated is evaluated using the first principal component score to the K-th principal component score of the report to be evaluated as explanatory variables.

[0024] This returns a numerical value between 0 and 1 as the objective variable, allowing for an objective and intuitive evaluation.

[0025] In the above invention, preferably, report A highly evaluated by the evaluation method of the present invention is compared with report B lowly evaluated, and advice on corrections to report B is given.

[0026] This allows for appropriate advice to be given to reports that are given a low rating.

[0027] In the above invention, preferably, the difference in occurrence probability for the N co-occurrence pairs is calculated for report A that has been highly rated by the evaluation method of the present application and report B that has been lowly rated, and advice including a specific co-occurrence pair from among the N co-occurrence pairs is generated based on the difference.

[0028] This allows for appropriate advice to be given to reports that have been given a low rating.In addition, by comparing reports with each other rather than with an average trend, accurate comparisons can be made.

[0029] The information processing device of the present invention has an evaluation model creation function that performs logistic regression analysis on terms contained in existing selected reports and existing non-selected reports to create an evaluation model, and a report evaluation function that applies the evaluation model to the report to be evaluated and evaluates it.

[0030] If selected and unselected reports can be clearly separated, logistic regression analysis can be applied, which allows for objective evaluation.

[0031] The program of the present invention causes an information processing device to perform a process of performing logistic regression analysis on terms contained in existing selected reports and existing non-selected reports to create an evaluation model, and a process of applying the evaluation model to the report to be evaluated and evaluating it.

[0032] If selected and unselected reports can be clearly separated, logistic regression analysis can be applied, which allows for objective evaluation. [Effects of the Invention]

[0033] The evaluation technique according to the present invention makes it possible to objectively evaluate reports, particularly integrated reports.

[0034] According to the evaluation technique of the present invention, it is possible to provide appropriate advice for reports that have been given a low evaluation. [Brief explanation of the drawings]

[0035] [Figure 1] Overall flow diagram [Figure 2] Equipment hardware configuration diagram [Figure 3] Evaluation model creation functional block diagram [Figure 4] Evaluation model creation flow chart [Figure 5] Evaluation model creation image [Figure 6] Evaluation model creation image [Figure 7] Top 30 co-occurring pairs image [Figure 8] Matrix A image [Figure 9] Principal component analysis image [Figure 10] Principal component score diagram [Figure 11] Matrix B image [Figure 12] Evaluation target evaluation unit functional block diagram [Figure 13] Evaluation target evaluation flow chart [Figure 14] Principal component score diagram [Figure 15] Evaluation target evaluation image [Figure 16] Advice generation flow diagram [Figure 17] Difference image DETAILED DESCRIPTION OF THE INVENTION

[0036] ~Summary~ Figure 1 is an overall flow diagram of the present application. The evaluation method of the present application comprises a step of creating an evaluation model (step S10) and a step of applying the evaluation model to evaluate the evaluation target X (step S50). Furthermore, a step of providing advice when a low evaluation is given (step S70) may be added.

[0037] To avoid confusion in the explanation, function 1 (see FIG. 3) that executes the step of creating an evaluation model (step S10) and function 5 (see FIG. 12) that executes the step of evaluating the evaluation object X (step S50) and the step of providing advice (step S70) are configured separately. Function 1 and function 5 may be implemented in a single computing device, or may be implemented in separate devices and connected to each other via a network.

[0038] Function 1 and Function 5 are functions implemented in an information processing device, which may be, for example, a personal computer. Any device may be used as long as it can execute the arithmetic processing described below.

[0039] 2 is a hardware configuration diagram of information processing device 100. Device 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a non-volatile memory 104, and an I / O 105, and each component is connected to each other via a bus 106.

[0040] The CPU 101 executes each of the functions 1 and 5 based on a program. The ROM 102 stores the program. The RAM 103 is used as a temporary work area for the CPU 101. The non-volatile memory 104 is an example of a storage device that maintains stored information even if the power supply is cut off. For example, it is a semiconductor memory or a hard disk. The non-volatile memory 104 does not need to be built-in and may be removable and portable. The I / O 105 is an input / output interface.

[0041] To the I / O 105, for example, a communication unit 107, an input unit 108, a display unit 109, etc. are connected.

[0042] The communication unit 107 is connected to a communication line and has a communication protocol for communicating data with the outside. For example, text information for each report may be input via communication.

[0043] The input unit 108 is a touch panel, a keyboard, a mouse, or the like, and is used to input operation instructions and the like.

[0044] The display unit 109 is, for example, a liquid crystal display, an organic EL display, a projector, or the like, and displays the information processing results, etc. The results may be displayed on another terminal display via a communication line, including a printer, etc. The evaluation of the final report, etc., is displayed on the display unit 109.

[0045] ~Creating an evaluation model~ 3 is a functional block diagram of the evaluation model creation function 1. The CPU 101 executes each function via a program. The processing content and processing procedure of each function will be described later using a flowchart (FIG. 4).

[0046] The evaluation model creation function 1 comprises an analysis object extraction unit 10 and a logistic regression analysis unit 20. The analysis object extraction unit 10 extracts analysis objects from each report that serves as the basis for creating the model. The logistic regression analysis unit 20 analyzes the analysis objects.

[0047] The evaluation model creation function 1 can read data from the storage unit 30 as needed, and can write data to the storage unit 30. The storage unit 30 is a functional block diagram showing storage devices 102, 103, and 104, which are hardware components.

[0048] The analysis target extraction unit 10 comprises a report reading unit 11 , a dictionary reference unit 12 , a co-occurrence pair extraction unit 13 , a conditional probability calculation unit 14 , and a top 30 selection unit 15 .

[0049] Fig. 4 is a flowchart for creating an evaluation model. Fig. 5 is an image diagram of the processing content in the first half of the flow. The processing content of each function of the analysis target extraction unit 10 will be explained using Fig. 4 and Fig. 5.

[0050] The report reading unit 11 reads the selected 50 reports (examples) and the unselected 150 reports (examples), totaling 200 reports (examples), in the form of text data (step S11).

[0051] Each report is assigned a report identification number between 1 and 200, and either a "1" indicating that it has been selected or a "0" indicating that it has not been selected.

[0052] If the report has not been converted into text data, the printed information is converted into text data. The report reading unit 11 may read the report via external communication. The read text data is stored in the storage unit 30.

[0053] The storage unit 30 may store a dictionary in advance. The dictionary in this application may be, for example, a thesaurus and a compound noun dictionary. "CO2" and "carbon dioxide" are synonyms. "SDGs" and "sustainable development goals" are synonyms. Furthermore, a characteristic of integrated reports is that compound nouns are frequently found. Compound nouns are often new words, and their meanings can become ambiguous when they are divided. Therefore, several compound nouns are registered in the dictionary in advance.

[0054] The dictionary reference unit 12 performs morphological analysis on the read text data while referring to a dictionary, and extracts terms (step S12). Note that although dictionary reference is not essential, subsequent accuracy verification has confirmed that the evaluation accuracy is significantly improved depending on whether or not a dictionary is referenced.

[0055] The co-occurrence pair extraction unit 13 extracts co-occurrence pairs of terms contained in one sentence (from the character following a punctuation mark to the next punctuation mark) (step S13). Note that although it is possible to create an evaluation model similar to that of the present application using single words as terms, subsequent accuracy verification has confirmed that the evaluation accuracy is significantly improved by using co-occurrence pairs.

[0056] After determining all co-occurrence pairs and their occurrence frequencies, the conditional probability calculation unit 14 calculates the conditional probability of each co-occurrence pair (step S14). As a specific example, Equation 1 is shown in FIG.

[0057] In Equation 1, Na is the total number of target integrated reports (200 examples), and Nb is the number of selected integrated reports (50 examples). Overall, Ea is the number of times co-occurrence pair E was mentioned, and Eb is the number of times co-occurrence pair E was mentioned in the selected integrated reports. This makes it possible to suppress the extraction of co-occurrence pairs that are common to all, and to extract co-occurrence pairs that are unique to the selected integrated reports.

[0058] The top 30 selection unit 15 sorts the co-occurrence pairs in descending order of probability based on the results of the conditional probability calculation unit 14, and selects the top 30 (examples) co-occurrence pairs with the highest occurrence probability (step S15). The top 30 co-occurrence pairs are assigned pair numbers and stored in the storage unit 30 (see image in FIG. 7).

[0059] The intermediate data generated by the co-occurrence pair extraction unit 13 and the conditional probability calculation unit 14 may be left stored in the storage unit 30 or may be deleted as appropriate.

[0060] The logistic regression analysis unit 20 comprises an occurrence probability matrix A creation unit 21 , a principal component analysis unit 22 , a principal component score matrix B creation unit 23 , a regression coefficient estimation unit 24 , and an evaluation model creation unit 25 .

[0061] Fig. 4 is a flowchart showing the creation of an evaluation model. Fig. 6 is an image diagram showing the processing content of the latter half of the flow. The processing content of each function of the logistic regression analysis unit 20 will be explained using Fig. 4 and Fig. 6.

[0062] The occurrence probability matrix A creation unit 21 calculates the occurrence probability Ratio for each pair number (pair=1 to 30) and each report identification number (d=1 to 200). pair,d (Equation 2 is shown in FIG. 6) is obtained, and a matrix A with 200 rows and 30 columns is created (step S21). An image of matrix A is shown in FIG.

[0063] In Equation 2, Freq is the number of occurrences of the co-occurrence pair pair that appears in the integrated report d, and Total is the total number of occurrences of all co-occurrence pairs that appear in the integrated report d. In other words, the occurrence probability Ratio pair,d is the occurrence rate of a specific co-occurrence pair in report d.

[0064] The principal component analysis unit 22 extracts 30 principal components by performing principal component analysis on the matrix A (step S22), and also calculates the contribution rate and factor burden of each principal component.

[0065] Figure 9 shows the results of principal component analysis. 30 principal components are extracted for the 30 co-occurrence pairs. Each principal component is sorted in descending order of factor load.

[0066] The principal component score matrix B creation unit 23 calculates the principal component score PCS for each principal component and each report identification number (d=1 to 200). pc,d Request.

[0067] FIG. 10 is an image diagram of the calculation of principal component scores. For the sake of convenience, in FIG. 10, the principal component scores are calculated for the first to thirtieth principal components. In this application, for example, principal components with a cumulative contribution rate of less than 60% are extracted. As a result, the first to third principal components are extracted.

[0068] The principal component score matrix B creation unit 23 creates principal component scores PCS for each of the first to third principal components and each of the report identification numbers (d=1 to 200). pc,d Then, a matrix B with 200 rows and 3 columns is created (step S23). An image of matrix B is shown in FIG.

[0069] The basic evaluation model is shown in Equation 4 (shown in Figure 6). Equation 4 is a general logistic regression model. The exponent (absolute value) of e in Equation 4 is shown in Equation 3.

[0070] The regression coefficient estimation unit 24 sets the dependent variable of the selected report to "1" and the dependent variable of the non-selected report to "0," and estimates the regression coefficients α0, α1, α2, and α3 in Equation 3 based on matrix B (step S24).

[0071] The evaluation model creation unit 25 creates an evaluation model by applying the regression coefficients α0, α1, α2, and α3 to Equation 4 (step S25). In Equation 4, the principal component scores become explanatory variables. The evaluation model according to Equation 4 is stored in the storage unit 30.

[0072] The intermediate data generated in the occurrence probability matrix A generation unit 21, the principal component analysis unit 22, the principal component score matrix B generation unit 23, and the regression coefficient estimation unit 24 may be left stored in the memory unit 30 or may be deleted as appropriate.

[0073] ~Evaluation Target X Rating~ 12 is a functional block diagram of the evaluation object X evaluation function 5. The CPU 101 executes each function via a program. The processing content and processing procedure of each function will be described later using a flowchart (FIG. 13).

[0074] The evaluation object X evaluation function 5 comprises an analysis object extraction unit 50 and a logistic regression analysis unit 60. An advice generation unit 70 may be included as an additional component. The analysis object extraction unit 50 extracts an analysis object from the report X that is the evaluation object. The logistic regression analysis unit 60 applies the analysis object to the evaluation model obtained by the logistic regression analysis unit 20 and evaluates it.

[0075] The evaluation object X evaluation function 5 can be read from the storage unit 30 and written to the storage unit 30 as needed.

[0076] The analysis target extraction unit 50 comprises a report reading unit 51, a dictionary reference unit 52, and a co-occurrence pair extraction unit 53. Fig. 13 is a flowchart of evaluation of evaluation target X. The processing contents of each function of the analysis target extraction unit 50 will be explained using the first half of Fig. 13.

[0077] The report reading unit 51 reads the report X to be evaluated in the form of text data (step S51). Since the report reading unit 51 performs almost the same processing as the report reading unit 11, it may be performed by the report reading unit 11. Note that the information "X" indicating that the report is to be evaluated is attached.

[0078] The dictionary reference unit 52 performs morphological analysis on the read text data while referring to a dictionary, and extracts terms (step S52). Since this performs almost the same processing as the dictionary reference unit 12, the dictionary reference unit 12 may perform this processing.

[0079] The co-occurrence pair extracting unit 53 extracts co-occurrence pairs of terms contained in one sentence (step S53). Since the co-occurrence pair extracting unit 53 performs substantially the same processing as the co-occurrence pair extracting unit 13, the co-occurrence pair extracting unit 53 may perform the processing.

[0080] That is, the analysis target extraction unit 50 applies part of the processing of the analysis target extraction unit 10 to the evaluation target report X.

[0081] The logistic regression analysis unit 60 comprises an occurrence probability calculation unit 61, a principal component score calculation unit 63, and an evaluation unit 66. The processing contents of each function of the logistic regression analysis unit 60 will be explained using the latter half of FIG.

[0082] The occurrence probability calculation unit 61 retrieves the top 30 co-occurrence pairs from the storage unit 30, and calculates the occurrence probability Ratio for each pair number (pair=1 to 30) in the report identification number x. pair,x (Equation 5 is shown in FIG. 14) is calculated (step S61).

[0083] In Equation 5, Freq is the number of occurrences of the co-occurrence pair pair that appears in the integrated report x, and Total is the total number of occurrences of all co-occurrence pairs pair that appear in the integrated report x obtained in step S53. That is, the occurrence probability Ratio pair,x is the occurrence rate of a specific co-occurrence pair in the report X to be evaluated.

[0084] The processing of the occurrence probability calculation unit 61 is the same as part of the processing of the occurrence probability matrix A creation unit 21, and therefore may be performed by the occurrence probability matrix A creation unit 21.

[0085] The principal component score calculation unit 63 obtains the principal component scores PCS pc,x of the first to third principal components for the report identification number x (step S63).

[0086] Fig. 14 shows an image of calculation of principal component scores for report X to be evaluated. The factor burdens are retrieved from the storage unit 30 as calculated in step S22. In step S23, the first to third principal components are extracted.

[0087] The processing of principal component score calculation section 63 is the same as part of the processing of principal component score matrix B creation section 23, and therefore may be performed by principal component score matrix B creation section 23.

[0088] The evaluation unit 66 retrieves the evaluation model (Equation 4) from the storage unit 30 and calculates the principal component scores PCS of the first to third principal components as explanatory variables. pc,x is input to find the objective variable. A numerical value between 0 and 1 is returned as the objective variable. This numerical value becomes the evaluation of the evaluation object X (step S66). The evaluation of the evaluation object X is stored in the storage unit 30.

[0089] Figure 15 is an image diagram of the evaluation of evaluation object X. The evaluation is expressed as a numerical value between 0 and 1. The closer the quality of the selected report is to the evaluation "1", and the further away it is from the quality of the selected report, the closer it is to the evaluation "0". In the example shown, an evaluation of "0.39" indicates a low evaluation.

[0090] The logistic regression analysis unit 60 applies part of the processing of the logistic regression analysis unit 20 to the report X to be evaluated.

[0091] The intermediate data generated by the report reading unit 51, the dictionary reference unit 52, the co-occurrence pair extraction unit 53, the occurrence probability calculation unit 61, and the principal component score calculation unit 63 may be left stored in the memory unit 30 or may be deleted as appropriate.

[0092] ~Effects~ The report evaluation is the objective variable of the logistic regression analysis and is expressed as a value between 0 and 1. This provides institutional investors with objective and intuitive information for making decisions.

[0093] Objectivity is ensured by having the report evaluation performed by the information processing device 100. On the other hand, the subjectivity of the institutional investor is also ensured by defining the integrated report selected by the institutional investor as a "good report."

[0094] ~Advice Generation Background~ The above embodiment is intended to allow investors to objectively evaluate integrated reports. On the other hand, if a business company that issues an integrated report creates a high-quality integrated report, it will attract the attention of institutional investors as an investment target for ESG investment. Therefore, it is important for business companies and report preparers to create a high-quality integrated report.

[0095] Incidentally, the Government Pension Investment Fund selects and publishes "excellent integrated reports" every year, but the selection criteria are not made public.

[0096] As a result, even if a report writer tries to create an excellent report, he or she is unable to determine how to write the report specifically. It is also difficult for the writer to objectively evaluate the report he or she has created.

[0097] The evaluation system of the present application allows for objective evaluation. However, if a low evaluation is given, it is difficult to determine how to improve the evaluation.

[0098] ~Advice Generation Details~ The evaluation object X evaluation function 5 may have an advice generation unit 70 as an additional component (see FIG. 12). The advice generation unit 70 is made up of a comparison object identification unit 71, a difference unit 72, and an advice unit 73. FIG. 16 is an advice generation flow diagram. The processing contents of each function of the advice generation unit 70 will be described with reference to FIG. 16.

[0099] If the evaluation is equal to or lower than a threshold value (for example, 0.45), the evaluation unit 66 may determine that the evaluation is low and activate the advice generation unit 70 (step S71). pair,x is called (step S72).

[0100] The comparison target specifying unit 71 specifies the closest comparison target Y from among the selected reports Nos. 1 to 50 based on the type of business or field (step S73).

[0101] Furthermore, the occurrence probability Ratio for each pair number (pair=1 to 30) in the comparison report Y calculated in step S21 is stored in the storage unit 30. pair,Y is called (step S74).

[0102] The difference unit 72 calculates the appearance probability Ratio associated with a low evaluation. pair,x and the probability of occurrence of high evaluations pair,Y The difference between these is calculated (step S75), normalized to fall between 0 and 1 (step S76), and co-occurrence pairs whose normalized difference value exceeds a threshold are extracted (step S77). Figure 17 is a diagram illustrating the difference.

[0103] When extracting co-occurrence pairs related to advice, the difference between the occurrence probability Ratio pair,x related to a low rating and the occurrence probability Ratio pair,Y related to a high rating is negative, so it is taken as an absolute value. Also, if the difference between the occurrence probability Ratio pair,x related to a low rating and the occurrence probability Ratio pair,Y related to a high rating is positive, it may be extracted as a co-occurrence pair that should be deleted or reduced from the integrated report.

[0104] The advice unit 73 uses an external sentence generation artificial intelligence or the like to generate advice based on the extracted co-occurrence pairs (step S78). The advice is displayed on the terminal of the creator of the evaluation target X.

[0105] This allows the creator of the evaluation target X to understand the cause of the low evaluation and make specific corrections. In other words, since the advice includes the extracted co-occurrence pairs, the creator of the evaluation target X can simply add sentences that include the co-occurrence pairs.

[0106] In addition, by comparing with the closest comparable entity Y in the same industry or field, the adjustments can be made more specific.

[0107] ~Other~ The present application is not limited to the above-described embodiment, and various modifications are possible within the scope of the technical concept of the present invention.

[0108] Although the evaluation model creation function 1 and the evaluation object X evaluation function 5 have been described as being configured separately, they may be configured as the same.

[0109] In the above embodiment, each process is executed by the information processing device 100, but a network may be formed by a plurality of distributed devices.

[0110] For example, a network is formed between the server, the terminals of evaluators such as institutional investors, and the terminals of report creators. Input / output processing is performed on each terminal, and other processing is performed on the server.

[0111] An integrated report is used as an example, but this can also be applied to other reports, etc., as long as there is a clear distinction between good and bad statements.

[0112] The numerical values ​​in the above embodiment are merely examples for ease of understanding, and are not limiting. [Explanation of symbols]

[0113] 1. Evaluation model creation function 5 Evaluation Target X Evaluation Function 10 Analysis target extraction section 11 Dictionary Reference Section 12 Input section 13 Co-occurrence pair extraction section 14 Conditional Probability Calculation Unit 15 Top 30 Selection Section 20 Logistic Regression Analysis Section 21. Occurrence probability matrix A creation section 22 Principal component analysis section 23 Principal component score matrix B creation section 24 Regression coefficient estimation section 25 Evaluation Model Creation Department 30 Storage section 50 Analysis target extraction section 51 Dictionary Reference Section 52 Input section 53 Co-occurrence pair extraction unit 60 Logistic Regression Analysis Section 61 Occurrence probability calculation section 63 Principal component score calculation section 66 Evaluation Department 70 Advice Generation Unit 71 Comparison target identification section 72 Difference part 73 Advisory Department 100 Information processing device 101 CPU 102 ROM 103 RAM 104 Non-volatile memory 105 Input / Output Interface Bus 106 107 Communication Unit 108 input units 109 Display Unit

Claims

1. Using an information processing device having an evaluation model creation function and a report evaluation function, a step of performing a logistic regression analysis on co-occurrence pairs of the terms contained in a sentence described in the existing selection report and the existing non-selection report by the evaluation model creation function to create an evaluation model; a step of applying the evaluation model to the report to be evaluated by the report evaluation function and evaluating the report; Equipped with In the step of creating the evaluation model, In the existing selection report, the top N co-occurrence pairs of the terms contained in one sentence with the highest occurrence probability are selected; Let d be the total number of existing selected reports and existing non-selected reports, and N be the number of selected co-occurrence pairs. Create a d-row, N-column matrix A relating to the occurrence probability. Principal component analysis is performed on matrix A, and the first to Kth principal components are extracted. A matrix B having d rows and K columns relating to the principal component scores from the first principal component to the K-th principal component is created; For matrix B, the objective variable for existing selected reports is set to 1, and the objective variable for existing non-selected reports is set to 0, and the regression coefficient for each principal component is calculated. Create a rating model A report evaluation method characterized by:

2. In the evaluating step, The report to be evaluated is evaluated using the first principal component score to the Kth principal component score of the report to be evaluated as explanatory variables.

2. The report evaluation method according to claim 1.

3. The report evaluation function compares report A, which is highly evaluated by the evaluation method described in claim 1, with report B, which is lowly evaluated; Provide advice on correcting the above report B A report advice method characterized by:

4. The report evaluation function uses the evaluation method described in claim 1 to evaluate report A, which has been highly evaluated, and report B, which has been poorly evaluated. Calculate the difference in occurrence probability for the N co-occurrence pairs, Based on the difference, advice including a specific co-occurrence pair among the N co-occurrence pairs is generated. A report evaluation method characterized by:

5. an evaluation model creation function that performs logistic regression analysis on co-occurrence pairs of the terms contained in a sentence in an existing selection report and an existing non-selection report to create an evaluation model; a report evaluation function that applies the evaluation model to a report to be evaluated and evaluates it; Equipped with The evaluation model creation function includes: In the existing selection report, the top N co-occurrence pairs of the terms contained in one sentence with the highest occurrence probability are selected; Let d be the total number of existing selected reports and existing non-selected reports, and N be the number of selected co-occurrence pairs. Create a d-row, N-column matrix A relating to the occurrence probability. Principal component analysis is performed on matrix A, and the first to Kth principal components are extracted. A matrix B having d rows and K columns relating to the principal component scores from the first principal component to the K-th principal component is created; For matrix B, the objective variable for existing selected reports is set to 1, and the objective variable for existing non-selected reports is set to 0, and the regression coefficient for each principal component is calculated. Create a rating model 1. An information processing device comprising:

6. The report evaluation function includes: The report to be evaluated is evaluated using the first principal component score to the Kth principal component score of the report to be evaluated as explanatory variables.

6. The information processing apparatus according to claim 5,

7. A process of performing a logistic regression analysis on co-occurrence pairs of the terms contained in a sentence in the existing selection report and the existing non-selection report to create an evaluation model; A process of applying the evaluation model to a report to be evaluated and evaluating it; causing an information processing device to execute the above; In the process of creating the evaluation model, In the existing selection report, the top N co-occurrence pairs of the terms contained in one sentence with the highest occurrence probability are selected; Let d be the total number of existing selected reports and existing non-selected reports, and N be the number of selected co-occurrence pairs. Create a d-row, N-column matrix A relating to the occurrence probability. Principal component analysis is performed on matrix A, and the first to Kth principal components are extracted. A matrix B having d rows and K columns relating to the principal component scores from the first principal component to the K-th principal component is created; For matrix B, the objective variable for existing selected reports is set to 1, and the objective variable for existing non-selected reports is set to 0, and the regression coefficient for each principal component is calculated. Create a rating model A program characterized by:

8. In the evaluation process, The report to be evaluated is evaluated using the first principal component score to the Kth principal component score of the report to be evaluated as explanatory variables.

8. The program according to claim 7,

Images