Information analysis system, information analysis method, and program

Abstract

This information analysis system comprises: a collection unit that collects a plurality of pieces of document data; a reception unit that receives an input of a keyword; an extraction unit that extracts text data including the keyword from the document data on the basis of the keyword; an analysis unit that analyzes the text data and weights each word and each pair of words included in the text data; and an output unit that, on the basis of the weighting, outputs analysis results including a co-occurrence network in which the number of occurrences of each word is represented by the size of a circle and the number of occurrences of each pair of words is represented by the thickness of a line between the words.

Description

Information analysis system, information analysis method, and program

[0001] The present invention relates to an information analysis system, an information analysis method, and a program.

[0002] There are information analysis systems that analyze and visualize a variety of large amounts of text information. For example, technologies such as text mining have been developed as a method to make it easier for personnel to analyze a variety of large amounts of text information, and by classifying and visualizing the information, personnel can efficiently extract the information they need (see, for example, Non-Patent Document 1).

[0003] "AI Text Mining by User Local," [Retrieved June 1, 2024], Internet,<URL:https: / / textmining.userlocal.jp / >

[0004] For example, when formulating a company's business strategy, it is more important to accurately grasp and analyze information that has not yet been recognized than information that is generally recognized. However, conventional technologies have the problem of being unable to accurately extract potentially useful information that has not been recognized by the public from various documents.

[0005] The embodiments of the present invention have been made in consideration of the above-mentioned problems, and enable accurate extraction of potentially useful information in an information analysis system that analyzes and visualizes various document data.

[0006] In order to solve the above-mentioned problems, an information analysis system according to an embodiment of the present invention includes a collection unit that collects multiple literature data, a reception unit that receives input of keywords, an extraction unit that extracts text data containing the keywords from the literature data based on the keywords, an analysis unit that analyzes the text data and weights words and word pairs included in the text data, and an output unit that outputs analysis results based on the weighting, including a co-occurrence network in which the number of times the words appear is represented by the size of a circle and the number of times the word pairs appear is represented by the thickness of the line between the words.

[0007] According to an embodiment of the present invention, it becomes possible to accurately extract potentially useful information in an information analysis system that analyzes and visualizes various document data.

[0008] FIG. 1 is a diagram illustrating an example of the configuration of an information analysis system according to the present embodiment. FIG. 2 is a diagram illustrating an overview of processing according to the present embodiment. FIG. 3 is a diagram illustrating an example of a co-occurrence network generated by a conventional method. FIG. 4 is a flowchart illustrating an example of information analysis processing according to Example 1. FIG. 5 is a diagram illustrating an example of information analysis processing according to Example 1. FIG. 6 is a diagram illustrating an example of a weighting change operation according to Example 1. FIG. 7 is a flowchart illustrating an example of information analysis processing according to Example 2. FIG. 8 is a diagram illustrating an example of information analysis processing according to Example 2. FIG. 9 is a diagram illustrating an example of the hardware configuration of a computer.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0010] <Background and Issues> For example, in the process of formulating and developing a company's business strategy, the person in charge collects and analyzes various text information and uses it to plan and develop the optimal strategy. However, due to the large amount of text information and the expertise of the person in charge, it is currently difficult to accurately extract important information that is not widely recognized.

[0011] Technologies such as text mining have been developed as a method to make it easier for personnel to analyze large amounts of various text information, and services are known that allow personnel to efficiently extract the information they need by classifying and visualizing the information (see, for example, Non-Patent Document 1).

[0012] For example, when formulating a business strategy for a company, it is more important to accurately grasp and analyze information that is not yet widely known than information that is generally known. However, there is a problem in that ordinary searches or the technology disclosed in Non-Patent Document 1 are unable to accurately extract information that is not widely known.

[0013] Therefore, the information analysis system 1 according to this embodiment has a configuration as shown in FIG. 1, for example, in order to be able to accurately extract potentially useful information.

[0014] <Configuration Example> Fig. 1 is a diagram showing a configuration example of an information analysis system according to this embodiment. The information analysis system 1 is a system that analyzes and visualizes various literature data. In the example of Fig. 1, the information analysis system 1 is configured by an information analysis device 100. However, this is just one example, and the information analysis system 1 may also be configured by multiple information processing devices.

[0015] The information analysis device 100 is an information processing device having a computer configuration, or a system configured with multiple computers. The information analysis device 100 executes a predetermined program to realize, for example, each functional configuration as shown in FIG. 1. In the example of FIG. 1, the information analysis system 1 and the information analysis device 100 have each functional configuration, such as a collection unit 101, an input unit 102, an extraction unit 103, an analysis unit 104, an output unit 105, and an information DB (Database) 106. Note that at least a portion of each of the above functional configurations may be realized by hardware.

[0016] The collection unit 101 executes a collection process for collecting a plurality of literature data. For example, the collection unit 101 collects a large amount of various literature data issued by the government, public institutions, think tanks, etc. from the Internet, etc., and stores the data in the information DB 106, etc. Note that the literature data collected by the collection unit 101 may also include literature data other than those described above (for example, news releases, media articles, etc.).

[0017] The input unit 102 executes an input process for accepting input of keywords. For example, the input unit 102 accepts input of keywords to be analyzed by a user. As a specific example, the user specifies keywords related to the management of their company. Note that the input unit 102 may be called, for example, a reception unit that accepts input of keywords.

[0018] The extraction unit 103 executes an extraction process to extract text data including a keyword from the document data collected by the collection unit 101, based on the keyword received by the input unit 102. For example, the extraction unit 103 extracts documents, sentences, etc. including the keyword from the information DB 106, based on the keyword received by the input unit 102.

[0019] The analysis unit 104 analyzes the text data extracted by the extraction unit 103 and performs an analysis process to weight words and word pairs included in the text data. For example, the analysis unit 104 calculates the appearance frequency of words used in the extracted text data and the relationships between words. The analysis unit 104 also weights the calculated appearance frequency of words and the relationships between words using a predetermined algorithm. Specific examples of the processing performed by the analysis unit 104 will be described later.

[0020] As another example, the input unit 102 may further receive an input of a field to be analyzed, and the analysis unit 104 may weight words and word pairs related to the field to be analyzed.

[0021] The output unit 105 executes an output process to output the analysis results based on the weighting by the analysis unit 104. For example, the output unit 105 outputs the analysis results including a co-occurrence network in which the size of a circle represents the frequency of occurrence of a word and the thickness of the line between the words represents the frequency of occurrence of a word pair. Here, "output" includes, for example, displaying on a display unit, transmitting to another device, or storing in a memory unit or the like. Furthermore, the co-occurrence network is a network in which words are used as vertices and words are connected by lines (edges) to represent the relationships between words.

[0022] The information DB 106 is, for example, a database that stores literature data and the like collected by the collection unit 101. Note that the information DB 106 may be provided outside the information analysis device 100, or may be provided outside the information analysis system 1 (for example, on the cloud, etc.).

[0023] 2 is a diagram for explaining an outline of the process according to this embodiment. This diagram shows an outline of an example of the process executed by the information analysis system 1.

[0024] The collection unit 101 collects documents and the like published by the government, public organizations, and / or think tanks, for example, from the Internet, and stores them in the information DB 106 (step S1).

[0025] The input unit 102 receives input of keywords related to the management of the company by the user (step S2). The keywords related to the management of the company are an example of keywords that are branching points in a scenario or words related thereto.

[0026] Based on the input keyword, the extraction unit 103 extracts text data containing the keyword from the information DB 106 (step S3).

[0027] The analysis unit 104 analyzes the text data extracted by the extraction unit 103 and weights words and word pairs (step S4). For example, the analysis unit 104 calculates the appearance frequency of words included in the text data extracted by the extraction unit 103 and the relationships between words. The analysis unit 104 also weights the calculated appearance frequency of words and the relationships between words using an algorithm.

[0028] The output unit 105 outputs the analysis result obtained by the analysis unit 104 (step S5). The analysis result includes, for example, a co-occurrence network 300 according to this embodiment, as shown in FIG.

[0029] 3 is a diagram for explaining an example of an analysis result according to this embodiment. The output unit 105 displays, for example, a co-occurrence network 300 as shown in FIG. 3 on a display screen showing the analysis result.

[0030] Co-occurrence network 300 is a co-occurrence network in which the size of a circle 301 represents the number of occurrences of weighted words, and the thickness of a line 302 represents the number of occurrences of the weighted word bare, with a keyword 304 at the center, which is a branching point of the scenario. By applying weighting in this way, it becomes easier to discover or extract potential candidates (signposts) of important information that are difficult for users to notice.

[0031] Figure 4 shows an image of a co-occurrence network generated by a conventional method. The co-occurrence network 400 generated by a conventional method has a keyword 404, which is the branching point of the scenario, at its center, with circles 401 representing words as its vertices, and each circle 401 is connected by a line (edge) 402 to represent the relationships between words. However, because the circles 401 are not weighted, there is a problem in that it is difficult for the user to discover and extract candidates 403 for potentially important information.

[0032] In this way, according to this embodiment, it is possible to accurately present potentially useful information that is difficult for the user to notice.

[0033] <Processing Flow> Next, a specific processing flow of the information analysis method according to this embodiment will be described.

[0034] 5 is a flowchart illustrating an example of information analysis processing according to Example 1. This processing illustrates an example of information analysis processing executed by the information analysis system 1 described with reference to FIG.

[0035] In step S501, the collection unit 101 collects a plurality of pieces of literature data and stores the collected literature data in the information DB 106. The literature data collected by the collection unit 101 includes, for example, data on a large amount of literature issued by the government, public institutions, and / or think tanks, etc.

[0036] In step S502, the input unit 102 receives input from the user of a branching point in a scenario or a keyword that is a word related thereto.

[0037] In step S503, the extraction unit 103 extracts text data containing the keyword from the information DB 106 based on the received keyword.

[0038] Next, the analysis unit 104 analyzes the extracted text data using natural language processing. For example, the analysis unit 104 executes the processes of steps S504 to S507.

[0039] In step S504, the analysis unit 104 performs a co-occurrence analysis on the extracted text data to quantify the frequency of occurrence of words and word pairs. For example, the analysis unit 104 counts the number of times a word appears and the number of word pairs (co-occurring words) that appear in one sentence.

[0040] 6A shows an example 601 of words in a sentence. In the example of FIG. 6A, the analysis unit 104 identifies the underlined words "global warming," "we," "humanity," "urgent," and "issue" as words in the sentence. The analysis unit 104 counts how many times each word appears in the extracted text data.

[0041] 6B shows an example of word pairs 602. In the example of FIG. 6B, the analysis unit 104 combines words in the sentence to form word pairs such as (global warming, us) (global warming, humanity) (global warming, urgent) (global warming, issue) (we, humanity) (we, urgent) (we, issue) (humanity, urgent) (humanity, issue) (urgent, issue). The analysis unit 104 counts the number of times each word pair appears in the extracted text data.

[0042] In step S505, the analysis unit 104 sorts the quantified word and word pair appearance frequencies in descending order, performs regression analysis on the changes, and obtains an approximate expression. For example, the analysis unit 104 sorts the number of word appearances and the number of word pairs in descending order, as shown in FIG. 7 .

[0043] 7, the horizontal axis represents words or word pairs, and the vertical axis represents frequency, with black dots 701 representing words or word pairs sorted in descending order of frequency. The analysis unit 104 performs regression analysis on the changes in the black dots 701 representing word pairs to find an approximation of y = f(x) (solid line 702).

[0044] In step S506, the analysis unit 104 obtains an equation that is symmetrical about the center of the appearance frequency distribution of the word and word pair for the obtained approximate equation. For example, for the approximate equation y = f(x) in Figure 78, the analysis unit 104 obtains a weighting function y = g(x) (dashed line 704) that is symmetrical about the center line 703 on the x-axis of the appearance frequency distribution of the word (and word pair), as shown in Figure 7.

[0045] In step S507, the analysis unit 104 weights the occurrence frequencies of words and word pairs based on the calculated symmetric formula. For example, the analysis unit 104 performs a weighted calculation of the occurrence counts of words and word pairs based on the weighting function y=g(x) calculated in step S505.

[0046] Based on the calculation results, a co-occurrence network is constructed with the number of occurrences of a word representing the size of the circle and the number of word pairs representing the thickness of the lines connecting the words, thereby making it possible to create, for example, the co-occurrence network 300 according to this embodiment described in Fig. 3. Note that the creation of the co-occurrence network 300 according to this embodiment may be performed by the analysis unit 104 or the output unit 105.

[0047] In step S508, the output unit 105 outputs the analysis result including, for example, the co-occurrence network 300 according to this embodiment described with reference to FIG. 3 based on the weighting. For example, the output unit 105 displays (outputs) the analysis result on a display unit. Alternatively, the output unit 105 may transmit (output) the analysis result to a user terminal or the like that is capable of communicating via a network and display it. Furthermore, the output unit 105 may store (output) the analysis result in a storage unit or the like.

[0048] Fig. 8 is a diagram for explaining an example of a weighting change operation according to Example 1. Preferably, as shown in Fig. 8, the user can apply a new weighting function y = h(x) by moving the dashed center line 703 in the graph 700 to a dashed center line 801.

[0049] For example, the output unit 105 displays the co-occurrence network 300 shown in FIG. 3 and the graph 800 shown in FIG. 8, allowing the user to change the co-occurrence network 300 to make it easier to view.

[0050] As described above, according to the first embodiment, in an information analysis system that analyzes and visualizes various document data, it becomes possible to accurately extract potentially useful information based on keywords entered by a user.

[0051] [Example 2] Fig. 9 is a flowchart showing an example of information analysis processing according to Example 2. This processing shows another example of information analysis processing executed by, for example, the information analysis system 1 described in Fig. 1. Note that, among the processing in Fig. 9, the processing in steps S501 to S504 is similar to the information analysis processing according to Example 1 described in Fig. 5, and therefore description thereof will be omitted here.

[0052] In step S901, the input unit 102 accepts a user's designation of a field to be analyzed. For example, the user designates a field to be analyzed from among the fields of STEEP (society, technology, economy, environment, and politics). Note that the input unit 102 may also accept input of a field to be analyzed when accepting input of a keyword in step S502.

[0053] In step S902, the analysis unit 104 sorts the appearance frequencies of words and word pairs related to the specified field in descending order of frequency, performs regression analysis on the changes, and obtains an approximate expression. For example, the analysis unit 104 sorts the appearance frequencies of words and word pairs in descending order of frequency, as shown in FIG. 10 .

[0054] 10, the horizontal axis represents words or word pairs, and the vertical axis represents frequency, and black dots 1001 representing words or word pairs related to a specified field are sorted in descending order of frequency. The analysis unit 104 performs regression analysis on the changes in the black dots 1001 representing word pairs to find an approximation of y = f(x) (solid line 1002).

[0055] In step S903, the analysis unit 104 obtains an equation that is symmetrical about the center of the appearance frequency distribution of the word and word pair for the obtained approximate equation. For example, for the approximate equation y=f(x) in Fig. 108, the analysis unit 104 obtains a weighting function y=g(x) (dashed line 1004) that is symmetrical about the center line 1003 on the x-axis of the appearance frequency distribution of the word (and word pair), as shown in Fig. 10.

[0056] In step S904, the analysis unit 104 weights the occurrence frequencies of words and word pairs based on the calculated symmetric formula. For example, the analysis unit 104 performs a weighted calculation of the occurrence counts of words and word pairs based on the weighting function y=g(x) calculated in step S903.

[0057] Based on the calculation results, a co-occurrence network is constructed with the number of occurrences of a word as the size of the circle, and the number of word pairs as the thickness of the lines connecting the words, thereby making it possible to create, for example, the co-occurrence network 300 according to this embodiment described in Figure 3 in a specified field. Note that in Example 2 as well, the co-occurrence network 300 may be created by the analysis unit 104 or the output unit 105.

[0058] In step S905, the output unit 105 outputs the analysis results including the co-occurrence network 300 according to this embodiment in the specified field based on the weighting. For example, the output unit 105 displays (outputs) the analysis results on a display unit. Alternatively, the output unit 105 may transmit (output) the analysis results to a user terminal or the like that is capable of communicating via a network and display the results. Furthermore, the output unit 105 may store (output) the analysis results in a storage unit or the like.

[0059] In the second embodiment, too, as explained in FIG. 8, a new weighting function y=h(x) can be applied by moving the dashed center line 703.

[0060] As described above, according to Example 2, in an information analysis system that analyzes and visualizes various literature data, it becomes possible to accurately extract potentially useful information based on keywords entered by the user and the field to be analyzed.

[0061] <Hardware Configuration> The information analysis device 100 according to this embodiment has, for example, the hardware configuration of a computer 1100 as shown in Fig. 11. Note that the information analysis system 1 may be realized by one or more computers 1100. Furthermore, the computer is not limited to a physical machine and may be, for example, a virtual machine on a cloud.

[0062] Fig. 11 is a diagram showing an example of the hardware configuration of a computer. In the example of Fig. 11, a computer 1100 includes a drive device 1101, an auxiliary storage device 1102, a memory device 1103, a CPU 1104, an interface device 1105, a display device 1106, an input device 1107, and an output device 1108, which are all connected to each other via a bus B. The computer 1100 may further include another processor such as a GPU (Graphics Processing Unit).

[0063] A program for implementing processing on the computer 1100 is provided by a recording medium 1111, such as a CD-ROM or a memory card. When the recording medium 1111 storing the program is set in the drive device 1101, the program is installed from the recording medium 1111 to the auxiliary storage device 1102 via the drive device 1101. However, the program does not necessarily have to be installed from the recording medium 1111, but may be downloaded from another computer via a network. The auxiliary storage device 1102 stores the installed program as well as necessary files, data, etc.

[0064] The memory device 1103 reads and stores a program from the auxiliary storage device 1102 when an instruction to start the program is received. The CPU 1104 realizes, for example, functions related to the information analysis device 100 in accordance with the program stored in the memory device 1103. The interface device 1105 is used as an interface for connecting to a communication network or the like. The display device 1106 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1107 is composed of a keyboard, a mouse, buttons, and / or a touch panel, and is used to input various operation instructions. The output device 1108 outputs calculation results.

[0065] The CPU 1104 may be another processor such as a DSP (Digital Signal Processor), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0066] <Effects of the embodiment> According to the present embodiment, in an information analysis system that analyzes and visualizes various document data, it becomes possible to accurately extract potentially useful information.

[0067] For example, when formulating a business strategy for a company, it is more important to accurately grasp and analyze information that is not yet widely known than information that is generally known. The information analysis system 1 according to this embodiment can accurately extract such information that is not widely known by the public using ordinary searches or conventional techniques.

[0068] Summary of Embodiments This specification discloses at least the following information analysis system, information analysis method, program, and information analysis device: (Item 1) An information analysis system comprising: a collection unit that collects a plurality of literature data; an input unit that accepts input of keywords; an extraction unit that extracts text data containing the keywords from the literature data based on the keywords; an analysis unit that analyzes the text data and weights words and word pairs included in the text data; and an output unit that outputs, based on the weighting, an analysis result including a co-occurrence network in which the frequency of occurrence of the words is represented by the size of a circle and the frequency of occurrence of the word pairs is represented by the thickness of the line between the words. (Item 2) The information analysis system described in Item 1, wherein the input unit further accepts input of a field to be analyzed; the analysis unit weights the words and word pairs related to the field to be analyzed; and the output unit outputs, based on the weighting, an analysis result including the co-occurrence network of the field to be analyzed. (Clause 3) An information analysis method in which a computer executes the following steps: a collection process for collecting multiple pieces of literature data, an input process for accepting input of keywords, an extraction process for extracting text data containing the keywords from the literature data based on the keywords, an analysis process for analyzing the text data and weighting words and word pairs included in the text data, and an output process for outputting analysis results including a co-occurrence network in which the number of times the words appear is represented by the size of a circle and the number of times the word pairs appear is represented by the thickness of the line between the words, based on the weighting. (Clause 4) A program, or a storage medium storing a program, that causes a computer to execute the information analysis method described in clause 3.(Clause 5) An information analysis device comprising: a collection unit that collects a plurality of literature data; an input unit that accepts input of keywords; an extraction unit that extracts text data including the keywords from the literature data based on the keywords; an analysis unit that analyzes the text data and weights words and word pairs included in the text data; and an output unit that outputs, based on the weighting, an analysis result including a co-occurrence network in which the number of times the words appear is represented by the size of a circle and the number of times the word pairs appear is represented by the thickness of the line between the words.

[0069] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0070] REFERENCE SIGNS LIST 1 Information analysis system 100 Information analysis device 101 Collection unit 102 Input unit 103 Extraction unit 104 Analysis unit 105 Output unit 106 Information DB 300 Co-occurrence network 1100 Computer

Claims

1. An information analysis system comprising: a collection unit that collects multiple literature data; an input unit that accepts input of keywords; an extraction unit that extracts text data containing the keywords from the literature data based on the keywords; an analysis unit that analyzes the text data and weights words and word pairs contained in the text data; and an output unit that outputs analysis results based on the weighting, including a co-occurrence network in which the number of times the words appear is represented by the size of a circle and the number of times the word pairs appear is represented by the thickness of the line between the words.

2. The information analysis system of claim 1, wherein the input unit further accepts input of a field to be analyzed, the analysis unit weights the words and word pairs related to the field to be analyzed, and the output unit outputs analysis results including the co-occurrence network in the field to be analyzed based on the weighting.

3. An information analysis method in which a computer executes the following steps: a collection process for collecting multiple literature data; an input process for accepting input of keywords; an extraction process for extracting text data containing the keywords from the literature data based on the keywords; an analysis process for analyzing the text data and weighting words and word pairs contained in the text data; and an output process for outputting, based on the weighting, an analysis result including a co-occurrence network in which the number of times the words appear is represented by the size of a circle and the number of times the word pairs appear is represented by the thickness of the line between the words.

4. A program for causing a computer to execute the information analysis method according to claim 3.

Images