Analytical systems, analytical instruments, analytical methods, and analytical programs

Abstract

This invention provides an analysis system, analysis device, analysis method, and analysis program that can suppress the increase in computational load when analyzing an object containing multiple analytical data. [Solution] The analysis device 1 acquires multiple analysis data included in the object to be analyzed (S104), derives feature quantities for each element of each acquired analysis data that can be represented as N-dimensional vector data (S107), selects M elements whose contribution rate of the feature quantities of the N elements constituting the derived N-dimensional vector data is higher than a predetermined value (S108), derives the distance between each analysis data based on the M-dimensional vector data composed of the selected feature quantities (S109), performs hierarchical clustering on each analysis data based on the derived distance between each analysis data (S111), and outputs the result of the hierarchical clustering as the analysis result (S112).

Description

Technical Field

[0001] This application relates to an analysis system for analyzing an analysis target, an analysis device used in the analysis system, an analysis method using the analysis device, and an analysis program for realizing the analysis device.

Background Art

[0002] In the 2010s, the need for data scientists was called for in society, and education to cultivate talents with such skills was carried out at universities. Currently, corporate data scientists are required to analyze text data that is collected in large quantities every day through customer service chatbots and corporate SNS. In view of such a situation, various methods have been proposed for displaying as a visible diagram that hierarchically represents clusters of data (for example, see Patent Document 1). Patent Document 1 proposes a text mining method or the like in which the results of analyzing text data are arranged on a two-axis analysis result screen and grouped.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the method disclosed in Patent Document 1, when the number of elements constituting the target analysis data is 3 or more, it is necessary to project onto a two-dimensional plane or calculate for selecting two axes with high contribution rates. Although it is also possible to arrange the analysis results in a three-dimensional space and display them on a two-dimensional screen, even in that case, the same problem occurs when the number of elements constituting the analysis data is 4 or more. Such a problem of calculation load becomes a more serious problem as data science progresses and the number of elements that can be derived as features increases.

[0005] This invention has been made in view of the above circumstances, and the primary objective of this application is to disclose an analysis system capable of reducing the computational load.

[0006] Furthermore, this application also aims to provide analytical equipment used in the analytical system disclosed herein.

[0007] Furthermore, this application aims to provide an analytical method using the analytical apparatus disclosed herein.

[0008] Furthermore, this application aims to provide an analytical program for realizing the analytical apparatus disclosed herein. [Means for solving the problem]

[0009] To solve the above problems, the analysis system disclosed in this application is an analysis system equipped with an analysis device for analyzing an object to be analyzed, and includes: an analysis data acquisition means for acquiring multiple analysis data included in the object to be analyzed; a feature quantity derivation means for deriving feature quantities for each element of each acquired analysis data that can be represented as N-dimensional (N is a natural number) vector data; a distance derivation means for deriving the distance between each analysis data based on the derived N-dimensional vector data; a clustering means for performing hierarchical clustering processing on each analysis data based on the derived distance between each analysis data; and a result output means for outputting the result of the hierarchical clustering processing as an analysis result.

[0010] Furthermore, the analysis system further includes display means for displaying images, and the result output means is characterized by including: a first display means for displaying an image of a dendrogram in which each analysis data processed hierarchically is a leaf node; a boundary position receiving means for receiving input for an instruction operation to indicate the position on the dendrogram of a boundary line that divides each leaf node into cluster units; a second display means for displaying an image in which the boundary line passing through the input position is superimposed on the dendrogram; and a third display means for displaying an image in which each leaf node is displayed in a different display method for each cluster divided by the boundary line.

[0011] Furthermore, in the analysis system, the object of analysis includes text data containing language data having meaning as natural language, the analysis data acquisition means acquires morphemes obtained by morphological analysis of the text data included in the object of analysis as analysis data, the acquired analysis data includes text data in sentence units containing one or more morphemes, and the feature quantity derivation means derives features by performing correspondence analysis on the acquired analysis data based on a cross-analysis table that quantifies the commonalities of morphemes included in each sentence.

[0012] The distance derivation means further comprises a selection means that selects M elements (where M is a natural number less than or equal to N) from among the N elements constituting the N-dimensional vector data of each analysis data derived by the feature derivation means, in order from the element with the highest contribution rate, such that the cumulative value of the contribution rate is greater than or equal to a predetermined value. The distance derivation means is characterized by deriving the distance between each analysis data using M-dimensional vector data composed of the feature quantities of the M elements selected from among the N elements constituting the N-dimensional vector data.

[0013] Furthermore, the analytical apparatus disclosed herein is an analytical apparatus comprising an analytical apparatus for analyzing an object to be analyzed, and is characterized by including: an analytical data acquisition means for acquiring a plurality of analytical data included in the object to be analyzed; a feature quantity derivation means for deriving feature quantities for each element of each acquired analytical data that can be represented as N-dimensional (N is a natural number) vector data; a distance derivation means for deriving the distance between each analytical data based on the derived N-dimensional vector data; a clustering means for performing hierarchical clustering processing on each analytical data based on the derived distance between each analytical data; and a result output means for outputting the results of the hierarchical clustering processing as an analysis result.

[0014] Furthermore, the analysis method disclosed herein is an analysis method using an analysis device for analyzing an object to be analyzed, and is characterized by including a plurality of steps: an analysis data acquisition step of acquiring multiple analysis data included in the object to be analyzed; a feature quantity derivation step of deriving feature quantities for each element that can be represented as N-dimensional (N is a natural number) vector data for each acquired analysis data; a distance derivation step of deriving the distance between each analysis data based on the derived N-dimensional vector data; a clustering step of performing hierarchical clustering on each analysis data based on the derived distance between each analysis data; and a result output step of outputting the result of the hierarchical clustering as an analysis result.

[0015] Furthermore, the analysis program disclosed herein is an analysis program that causes a computer to analyze an object to be analyzed, and is characterized by causing the computer to execute a plurality of steps, including: an analysis data acquisition step of acquiring a plurality of analysis data contained in the object to be analyzed; a feature quantity derivation step of deriving feature quantities for each element of each acquired analysis data that can be represented as N-dimensional (N is a natural number) vector data; a distance derivation step of deriving the distance between each analysis data based on the derived N-dimensional vector data; a clustering step of performing hierarchical clustering processing on each analysis data based on the derived distance between each analysis data; and a result output step of outputting the result of the hierarchical clustering processing as an analysis result. [Effects of the Invention]

[0016] The analysis system disclosed in this application has excellent effects, such as reducing the computational load, by analyzing the target of analysis based on the distance between multiple analysis data, which are represented as N-dimensional (N is a natural number) vector data. [Brief explanation of the drawing]

[0017] [Figure 1] This is a schematic diagram conceptually illustrating an example of the analysis system disclosed in this application. [Figure 2]It is a block diagram schematically showing a configuration example of a main device included in the analysis system disclosed in the present application. [Figure 3] It is a flowchart showing an example of analysis processing of an analysis device and a terminal device included in the analysis system disclosed in the present application. [Figure 4] It is a chart conceptually showing an example of an analysis target in the analysis system disclosed in the present application. [Figure 5] An example of a cross-tabulation table that is the basis of correspondence analysis is shown. [Figure 6] It is a chart conceptually showing an example of the relationship between each analysis data and feature amount in the analysis processing of the analysis system disclosed in the present application. [Figure 7] It is an explanatory diagram conceptually showing an example of hierarchical clustering processing in the analysis system disclosed in the present application. [Figure 8] It is an explanatory diagram showing an example of an analysis result output in the analysis system disclosed in the present application. [Figure 9] It is a flowchart showing an example of auxiliary analysis processing of an analysis device and a terminal device included in the analysis system disclosed in the present application. [Figure 10] It is an explanatory diagram showing an example of an analysis result in which a boundary line output in the analysis system disclosed in the present application is superimposed and displayed. [Figure 11] It is an explanatory diagram showing an example of an analysis result in which a boundary line output in the analysis system disclosed in the present application is superimposed and displayed.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0019] <Application Example> The analysis method disclosed in this application is used for analyzing large amounts of data, such as text data containing language data that has meaning as natural language, using hierarchical clustering. More specifically, the analysis method disclosed in this application can be used by corporate data scientists to analyze the large amounts of text data collected daily through network services such as customer service chatbots and corporate social networking services (SNS). By enabling the analysis of such text data, the analysis method disclosed in this application can be utilized for purposes such as product development and marketing. Below, with reference to the drawings, an analysis system to which the analysis method disclosed in this application is applied will be described.

[0020] <System Configuration> Figure 1 is a schematic diagram conceptually illustrating an example of the analysis system disclosed in this application. The analysis method disclosed in this application can be implemented in various forms, such as a web service using analysis device 1, or a standalone system that can be run by analysis device 1 alone. Figure 1 illustrates a form in which the analysis system is executed as a web service using analysis device 1, which is a computer such as a server computer. Analysis device 1 is connected to a communication network NW such as a LAN (Local Area Network), WAN (Wide Area Network), dedicated communication network, or the Internet. Various devices such as terminal device 2, communication device 3, and storage device 4 are connected to the communication network NW. Terminal device 2, as illustrated in Figure 1, is a computer such as a personal computer operated by a user who uses the web service provided by analysis device 1. Communication device 3 is a computer such as a web server computer that provides other web services such as SNS. Storage device 4 is a computer such as a database server computer that stores various databases used for analysis. Users can access analysis device 1 by operating terminal device 2 and enjoy web services such as analysis processing provided by analysis device 1. The analysis device 1, in accordance with commands from the terminal device 2, acquires data to be analyzed, including a large amount of data such as text data, analyzes the acquired data, and outputs the results of the analysis as analysis results. The output of analysis results by the analysis device 1 is performed, for example, by sending analysis result information indicating the analysis results to the terminal device 2.

[0021] <Example Hardware Configuration> Figure 2 is a schematic block diagram showing an example of the configuration of the main components of the analysis system disclosed in this application. The analysis device 1 used in the analysis system disclosed in this application is configured using various computers, such as a server computer capable of communication. The analysis device 1 includes various components such as a control unit 10, a storage unit 11, and a communication unit 12.

[0022] The control unit 10 is a processor such as a CPU (Central Processing Unit) that includes various circuits such as information processing circuits, timing circuits, and register circuits, and performs processing to control the entire device.

[0023] The storage unit 11 is a storage unit composed of non-volatile memory such as hard disks, SSDs (Solid State Drives), RAID (Redundant Arrays of Inexpensive Disks), and flash memory, and volatile memory such as various types of RAM (Random Access Memory). The storage unit 11 stores various programs, such as the operating system (OS) and application programs that run on the operating system. As application programs, it stores various programs such as the analysis program 110 for realizing the analysis apparatus 1 disclosed in this application. In addition, the storage unit 11 can store various types of data, and can also store various data such as data on analysis targets acquired from other devices such as the terminal device 2, communication device 3, and storage device 4.

[0024] The communication unit 12 is a communication device such as a LAN adapter, which connects to a communication network NW, communicates with various devices, and can access various types of information.

[0025] A computer with the various configurations exemplified above operates as an analysis device 1 by reading various programs, such as the analysis program 110, stored in the memory unit 11 under the control of the control unit 10, and executing various steps contained in the read program.

[0026] Terminal device 2 comprises various components such as a control unit 20, a storage unit 21, a communication unit 22, an input unit 23, and an output unit 24. The input unit 23 is a device used for operation input such as a keyboard or mouse. The output unit 24 is an output device such as a liquid crystal display (display means) or a printer (printing means). The input unit 23 and the output unit 24 may be integrated and implemented using a device such as a liquid crystal touch panel.

[0027] <Software Processing Example> Next, the processing of the various devices in the analysis system disclosed in this application will be described. First, the analysis processing in the analysis system disclosed in this application will be described. Analysis processing is the main processing in the analysis system disclosed in this application. Figure 3 is a flowchart showing an example of the analysis processing of the analysis device 1 and terminal device 2 in the analysis system disclosed in this application. A user operating terminal device 2 operates terminal device 2 to access analysis device 1 via a communication network NW in order to have analysis device 1 perform analysis processing. The user operates the input unit 23 of terminal device 2 to perform analysis start operations such as specifying the analysis target to be analyzed and specifying the analysis method. Specifying the analysis target can be exemplified by specifying a URL, specifying a tab, or specifying a specific search word for collecting posts when collecting text data from SNS. Also, when specifying digitized text as the analysis target, it can be exemplified by specifying for reading processing and specifying the storage location. In the following explanation, an example will be described in which the user specifies an analysis target that includes text data containing language data that has meaning as natural language.

[0028] The terminal device 2, under the control of the control unit 20, receives an input for an analysis start operation from the input unit 23 (S101), and transmits an analysis start command based on the received operation from the communication unit 22 to the analysis device 1 via the communication network NW (S102).

[0029] The analysis device 1 performs analysis processing by executing the analysis program 110 stored in the memory unit 11 at the control unit 10. Specifically, the control unit 10 of the analysis device 1 receives an analysis start command from the communication unit 12 (S103) and acquires the analysis target specified in the received analysis start command (S104). The acquisition of analysis data in step S104 is a process such as reading the specified analysis target included in the received analysis start command. Specifically, this is a process such as acquiring analysis targets uploaded from the terminal device 2, reading analysis targets from SNS provided by the communication device 3, and reading analysis targets stored in the database of the storage device 4. In step S104, the analysis targets acquired from SNS are collected in units of utterance groups (groups) that are referred to by names such as groups, forums, threads, chat rooms, and spaces where specific or unspecified multiple people converse (write). Each collected utterance group contains, for example, the content of the conversation as data for each person's utterance (writing). Data for individual utterances is associated with identification information such as utterance identification information, speaker identification information, and utterance group identification information.

[0030] The control unit 10 performs morphological analysis on the acquired object to be analyzed (S105) and acquires the resulting multiple morphemes as analysis data (S106). The analysis device 1 acquires analysis data that has been decomposed into morphemes through the processing in steps S105 to S106. In steps S105 to S106, the analysis device 1 does not simply acquire the morphemes as they are, but if the morphemes are verbs, adjectives, adjectival verbs, or other predicates that have conjugated forms, it is also possible to add processing to convert the morphemes into their base forms and, if necessary, decompose them into stems and suffixes. Furthermore, the morphemes acquired as analysis data are associated with the part of speech obtained through morphological analysis, and, if they are predicates that have been converted, with related information such as the original form, stem, or suffix.

[0031] The control unit 10 derives feature quantities of elements that can be represented as N-dimensional vector data by performing analyses such as correspondence analysis on each acquired analysis data (S107). In step S107, feature quantities of multiple elements (multiple axes) are derived for each analysis data by performing correspondence analysis assuming various elements (axes) that represent the feature quantities. The feature quantities of multiple elements (multiple axes) are numerical values ​​that can be represented as N-dimensional vector data. Although N is a natural number, it is desirable to set it to 4 or more, as the analysis method disclosed in this application is effective for analyzing data that cannot be represented by known representation methods such as 2D and 3D. The elements in the analysis may be known elements or unknown elements, and furthermore, elements that combine known elements and unknown elements may be defined. Examples of known elements include morphemes (words) themselves, as well as other elements such as parts of speech, speakers, specific themes, and forms before or after conjugation. However, these elements may be selected elements for analysis rather than being axis elements. For example, by selecting the part of speech "verb" when deriving features, the analysis will be performed only on verbs. Alternatively, if an unknown element is used as the axis, the element may be derived using unsupervised machine learning. In step S107, if the source of the analysis data is, for example, a group of utterances obtained from social media, a cross-tabulation table is created that shows the relationship between the morphemes contained in the utterances and each utterance using numerical values ​​indicating the presence or absence of commonality. Then, correspondence analysis is performed based on the cross-tabulation table for each utterance. This derives feature quantities of elements that can be represented as N-dimensional vector data.

[0032] The control unit 10 derives the contribution rates of the N elements that constitute the N-dimensional vector data for each of the derived analysis data (S108). Then, the control unit selects M elements from the N elements in order from the elements with the highest contribution rates, such that the cumulative value of the derived contribution rates is equal to or greater than a predetermined value such as 0.8 (S109). In step S109, M is a natural number less than or equal to N. In order to effectively utilize the analysis method disclosed in this application for analysis targets for which known representation methods are difficult, it is desirable that M be 4 or greater. In step S109, elements are selected in order from the elements with the highest contribution rates, and the contribution rates of the selected elements are accumulated to derive a cumulative value. Then, when the cumulative value is equal to or greater than the predetermined value, the elements selected up to that point are selected as M elements. The processing in step S109 makes it possible to select effective elements while excluding elements that may be noise, thereby increasing reliability.

[0033] The control unit 10 derives the distance between each analysis data based on M-dimensional vector data composed of features selected from N dimensions (S110). Step S110 is a process in which the analysis data, represented by M(N) features corresponding to each element of the coordinate axes, is placed in an M(N)-dimensional virtual space composed of M(N) coordinate axes, and the distance between each analysis data in the virtual space is calculated.

[0034] The control unit 10 performs hierarchical clustering on each analysis data based on the distance between each derived analysis data (S111). Step S111 is a process that groups the analysis data based on the distance between each derived analysis data, and further performs the grouping hierarchically.

[0035] Then, the control unit 10 of the analysis device 1 outputs analysis result information that displays the results of the hierarchical clustering process as the analysis result (S112). The output of step S112 is executed as a process to transmit the analysis result information from the communication unit 12 to the terminal device 2 via the communication network NW.

[0036] The terminal device 2, under the control of the control unit 20, receives analysis result information at the communication unit 22 (S113), and outputs the analysis results based on the received analysis result information from the output unit 24 (S114). Step S114 is the process of displaying the analysis results as an image of a dendrogram, for example, with each analysis data processed using hierarchical clustering as a leaf node, on the output unit 24, i.e., the liquid crystal display which is the display means.

[0037] This section describes a specific example of the analysis process performed by the analysis system disclosed in this application. Figure 4 is a conceptual diagram illustrating an example of an analysis target in the analysis system disclosed in this application. Figure 4 shows an example of an analysis target acquired in step S104 of the analysis process. Figure 4 illustrates an analysis target acquired as a group of utterances from SNS, where the content of the conversation is included as text data of utterance units in the "utterance" item. Each utterance is assigned an utterance identification information in the "No." item, an utterance identification information in the "speaker" item, and an utterance group identification information in the "group" item. In this application, a single line of conversation as exemplified in Figure 4 is treated as an utterance, but the term "sentence" here does not necessarily correspond to a grammatical "sentence." For example, a set of text data sent in a single transmission process on SNS is used as the unit of a "sentence."

[0038] The object to be analyzed, as illustrated in Figure 4, undergoes morphological analysis in step S105. For example, the utterance "Hmm. I feel like water and salt are important... (laugh)" from "No. 1" is morphologically analyzed as follows: "Hmm", ".", "Water", "and", "Salt", "is", "important", "feels", "feels", "..." (ellipsis), ".", "(", "laugh", and ")". Through the morphological analysis in step S105, the utterance is divided into morphemes (words). The morphemes obtained through morphological analysis include interjections such as "Hmm", nouns such as "Salt", conjunctions such as "and", as well as punctuation marks such as ".", and symbols such as "..." and "(".

[0039] Morphological analysis associates various pieces of information derived during the morphological analysis process with morphemes as related information. An example of related information associated with morphemes is shown below.

[0040] Hmm; interjection,*,*,*,*,*,Hmm,Hmm,Hmm . ; Symbol, period, *, *, *, *, . , . , . Water; noun, general, *, *, *, *, water, Mizu, Mizu to: particle, coordinating particle, *, *, *, *, to, to, to Salt; noun, general, *, *, *, *, salt, shio, shio ga; particle, case particle, general, *, *, *, ga, ga Important; noun, adjectival noun stem, *, *, *, *, important, taisetsu, taisetsu na; auxiliary verb, *, *, *, special, da, nominal connection, da, na, na Ki; noun, non-independent, general, *, *, *, Ki, Ki ga; particle, case particle, general, *, *, *, ga, ga する; verb, independent, *, *, sa-hen verb - suru, base form, suru, suru …; Symbol, General, *, *, *, *, …, …, … …; Symbol, General, *, *, *, *, …, …, … . ; Symbol, period, *, *, *, *, . , . , . ( ; symbol, opening parentheses, *, *, *, *, (, ( Lol; noun, proper noun, person's name, name, *, *, lol, Emi, Emi ) : symbol, closing parenthesis, *, *, *, *,),),)

[0041] The related information above is an example of related information associated with each morpheme obtained through morphological analysis and other processing of the sentence, "Hmm. I feel like water and salt are important... (lol)." In the related information above, verbs, particles, auxiliary verbs, and other predicates are broken down into stems and suffixes, and for suffixes, the base form is associated as one of the related pieces of information. For example, the adjectival noun "taisetsu na" (important) is broken down into the stem "taisetsu" and the suffix "na," and the base form of the suffix, "da," is associated with it. The symbol "*" indicates that there is no corresponding item. As described above, related information includes information such as part of speech, other grammatical types, base form, common name, other or searchable names, etc.

[0042] In step S105, the morphologically analyzed data is subjected to correspondence analysis in step S107 to derive feature quantities. Figure 5 is a conceptual diagram showing an example of a cross-tabulation table generated in the analysis process of the analysis system disclosed in this application. Figure 5 shows an example of a cross-tabulation table that forms the basis of correspondence analysis. The cross-tabulation table in the analysis process is created as a cross-tabulation table that shows the relationship between the morphemes contained in an utterance and each utterance using numerical values ​​indicating the presence or absence of commonality. The presence or absence of commonality can be quantified, for example, by setting "1" if a common morpheme is included and "0" if it is not included. The cross-tabulation table exemplified in Figure 5 is based on the results of the morphological analysis of the data analyzed as exemplified in Figure 4. In the cross-tabulation table exemplified in Figure 5, all the morphemes (words) to be compared contained in all utterances are listed in the top row, each utterance is listed in the leftmost column for each utterance, and the relationship between the morphemes to be compared and each morpheme contained in each utterance is shown. In other words, the cell at the intersection of the column where the morphemes to be compared are shown at the top and the row extending to the right of the utterance column will show "1" or "0" depending on whether the morphemes to be compared are included in the utterance. In the cross-tabulation table exemplified in Figure 5, "verbs," "adjectives," "auxiliary verbs," "particles," "interjections," and "symbols" were selected as parts of speech for the derivation of features, and these are listed as the morphemes to be compared at the top. Furthermore, for verbs whose conjugation changes, the ending is shown in its base form. For example, "taranai" included in utterance "No. 7" is shown at the top as its base form, "taru." Since the aforementioned related information is associated with each morpheme included in each utterance, the cell at the intersection of the "taru" column and utterance "No. 7" shows "1." In this way, a cross-tabulation table for the reference utterance is created by quantifying the commonality between the morphemes included in all utterances and the morphemes included in each utterance. The cross-tabulation table illustrated in Figure 5 is just one example; it can be designed as appropriate, for example, by adding columns that show commonalities among speakers in addition to morphemes. Correspondence analysis from the cross-tabulation table and the derivation of features can be performed using commonly available processing methods.

[0043] Figure 6 is a conceptual diagram illustrating an example of the relationship between each analysis data and feature in the analysis process of the analysis system disclosed in this application. Figure 6 shows the relationship between morphemes treated as each analysis data and N (N-dimensional) feature quantities derived in step 107 of the analysis process. Morphemes (words) such as "um," ".", and "and" decomposed by morphological analysis are shown as analysis data in the leftmost column, and for each morpheme, the feature quantities of N elements in the horizontal direction are shown in association. In addition, for each element, the feature quantities arranged vertically are the values ​​on the coordinate axis of that element. The numerical values ​​in the top row of each element indicate the contribution rate of each element derived in step S108 of the analysis process. In the diagram illustrated in Figure 6, each element is sorted in descending order of contribution rate. In step S109 of the analysis process, M elements with high contribution rates are selected such that the cumulative value of the contribution rates is greater than or equal to a predetermined value such as 0.8. More specifically, step S109 is a process that repeatedly selects elements in order of their contribution rate, calculates the cumulative value of the contribution rate of the selected elements, and continues selecting elements until the cumulative value of the contribution rate exceeds a predetermined value. Conventional analysis methods using correspondence analysis select two or three elements and represent them on a two-axis graph that can be represented on a plane or on a three-axis coordinate space that can be represented as a three-dimensional object. This invention increases the cumulative contribution rate and enables a more accurate representation by selecting more elements (M≧3 or M≧4) than conventional analysis methods.

[0044] Figure 7 is a conceptual diagram illustrating an example of hierarchical clustering processing in the analysis system disclosed in this application. Figure 7 conceptually shows the hierarchical clustering processing described as steps S110 to S111 of the analysis process. Each morpheme (W1, W2, W3, ...) treated as analysis data constitutes M-dimensional vector data using M selected features. Figure 5 conceptually shows the M-dimensional vector data arranged as arrows in an M-dimensional virtual space. Hierarchical clustering processing is then performed to group the data based on the distance between each vector data. In Figure 7, the hierarchical clustering is shown as ellipses.

[0045] Figure 8 is an explanatory diagram showing an example of the analysis results output by the analysis system disclosed in this application. Figure 8 shows the analysis results output from the output unit 24, which is shown as step S114 of the analysis process. In Figure 8, morphemes (words) such as "shift," "suitable," and "soak in" obtained from SNS provided by the communication device 3 are arranged vertically. Note that morphemes also include symbols such as "(" and ")". The analysis result is shown as a dendrogram with each morpheme that becomes the analysis data as a leaf node. Branches extend to the right from each leaf node, and for analysis data that can be grouped, the branches are combined to form a binary tree, and the whole thing is a tree diagram. The further to the left the merging and branching positions of each branch are, the closer the position of the original M-dimensional vector is, indicating a stronger relationship.

[0046] As described above, the analysis process by the analysis system disclosed in this application is performed. Note that the process exemplified above is merely an example; for example, whether or not to convert the conjugation form, and the setting of the comparison targets after the conjugation conversion, can be set as appropriate.

[0047] Next, the auxiliary analysis processing in the analysis system disclosed in this application will be described. The auxiliary analysis processing is one of the various tools used to further analyze the results of the analysis processing in the analysis system disclosed in this application. Figure 9 is a flowchart showing an example of the auxiliary analysis processing of the analysis device 1 and terminal device 2 provided in the analysis system disclosed in this application. In order for the analysis device 1 to perform the auxiliary analysis processing, the user operates the input unit 23 to the terminal device 2 to input a boundary line instruction command that indicates the position of the boundary line that divides each leaf node in the dendrogram displayed as an analysis result into cluster units.

[0048] The terminal device 2, under the control of the control unit 20, receives a boundary line instruction command input from the input unit 23 (S201), and transmits a boundary line instruction command based on the received operation from the communication unit 22 to the analysis device 1 via the communication network NW (S202).

[0049] The analysis device 1 performs auxiliary analysis processing by executing an analysis program 110 stored in the memory unit 11 at the control unit 10. Specifically, the control unit 10 of the analysis device 1 receives a boundary line instruction command from the communication unit 12 (S203), and generates auxiliary analysis result information to display the analysis result based on the received boundary line instruction command (S204). The analysis result in step S204 is, for example, a dendrogram illustrated in Figure 8, with the boundary lines superimposed. The boundary lines are lines that indicate the positions where each leaf node shown in the dendrogram is divided into cluster units. The boundary lines are superimposed on the dendrogram so as to pass through the positions indicated by the boundary line instruction command.

[0050] Then, the control unit 10 outputs the generated auxiliary analysis result information (S205). The output of step S205 is executed as a process to transmit the auxiliary analysis result information from the communication unit 12 to the terminal device 2 via the communication network NW.

[0051] The terminal device 2, under the control of the control unit 20, receives auxiliary analysis result information via the communication unit 22 (S206), and outputs the analysis result with the boundary lines superimposed from the output unit 24 based on the received auxiliary analysis result information (S207). Step S207 is the process of displaying the image with the boundary lines superimposed on the dendrogram on the output unit 24, i.e., the liquid crystal display which is the display means. In step S207, the control unit 20 of the terminal device 2 displays an image on the output unit 24 that shows each leaf node and / or branch in a different display method for each cluster divided by the boundary lines, for example, in a different color for each cluster. The user who views the image checks the output image and confirms the analysis results divided by cluster. Furthermore, the user operates the input unit 23 to move the boundary lines as needed and performs a boundary line movement command to move the position of the boundary lines to the desired position.

[0052] The terminal device 2, under the control of the control unit 20, receives a boundary line movement command input from the input unit 23 (S208), and transmits the boundary line movement command based on the received operation from the communication unit 22 to the analysis device 1 via the communication network NW (S209).

[0053] The control unit 10 of the analysis device 1 receives a boundary line movement command from the communication unit 12 (S210), moves the boundary line to the position indicated by the boundary line movement command, and generates auxiliary analysis result information to display the superimposed analysis result (S211). Then, the control unit 10 outputs the generated auxiliary analysis result information (S212).

[0054] The terminal device 2, under the control of the control unit 20, receives auxiliary analysis result information via the communication unit 22 (S213), and outputs the analysis result with the moved boundary line superimposed from the output unit 24 based on the received auxiliary analysis result information (S214). In step S214, the output analysis result shows that the boundary line position has moved, and each leaf node and branch is displayed in a different display method for each cluster divided by the moved boundary line. The user who views the image views the output image, confirms the analysis result divided by cluster, and performs operations to request further processing such as moving the boundary line again or outputting morphemes (words) that become analysis data classified for each divided cluster. The terminal device 2 and the analysis device 1 perform various analysis and output processing according to the commands based on the commands from the further operations. For example, as one of the analysis and output processing, the terminal device 2 and the analysis device 1 can output data in a format such as CSV (Comma Separated Values) that shows a list of morphemes (words), their parts of speech, and features for each cluster.

[0055] Figures 10 and 11 are explanatory diagrams showing an example of analysis results with the boundary lines output by the analysis system disclosed in this application superimposed. Figure 10 is the output result based on the auxiliary analysis result information output from the output unit 24 in step S207, and Figure 11 is the output result output in step S214. For example, when the boundary line is located at the position exemplified in Figure 10, "find" and "enter" are classified into the same cluster, and the other analysis data are independent of each other. When the boundary line is moved to the position exemplified in Figure 11, clusters containing multiple morphemes are further generated, such as the cluster of "even" and "kick", the cluster of "make" and "hook", and the cluster of "get wet", "wow", "while", "say", and "pick". In the actual output image, the analysis data and branches are displayed in different colors, different line types, and other different display methods for each cluster.

[0056] As described above, the auxiliary analysis process by the analysis system disclosed in this application is performed.

[0057] As detailed above, the analysis method disclosed in this application has excellent effects, such as reducing the computational load, by performing analyses such as clustering based on the distance between multiple analysis data shown as N-dimensional (or selected M-dimensional) vector data.

[0058] Furthermore, the analysis method disclosed in this application offers excellent advantages, such as the ability to move the boundary lines as needed, enabling analysis at various levels.

[0059] <Experimental Results> For example, when dealing with analytical data containing multidimensional elements using previous analysis methods, the data was reduced to 2D (or 3D) feature quantities for the purpose of visualization, for instance, using methods such as principal component analysis. Therefore, the computation of converting multidimensional feature quantities to 2D (or 3D) was extremely computationally intensive.

[0060] The inventors of this application conducted comparative experiments using a standalone device that eliminated external disturbances such as communication load in order to conduct experiments on computational load. For the experiments, a Hewlett-Packard® "EliteDesk® 800 G8 SF" was used as the experimental analysis device. The CPU was an Intel® "Core i7 11700 processor" and the memory was 8GB. The OS was "Windows® 10 Pro". The data analyzed was text data obtained from social networking services (SNS). The data analyzed consisted of 8233 lines of utterances collected on SNS.

[0061] Experiments with an experimental analysis device showed that using conventional methods, the device froze due to increased computational load and was unable to output results. In contrast, the experimental analysis device using the analysis method disclosed in this application was able to output analysis results in a total of 66 seconds, with 51 seconds for analysis and 15 seconds for display (output). In this experiment, 639 features were obtained from 8233 lines of utterances, of which 621 features were selected.

[0062] The present invention is not limited to the embodiments described above and can be implemented in various other forms. Therefore, the embodiments described above are merely illustrative in all respects and should not be interpreted restrictively. The technical scope of the present invention is described by the claims and is not restricted in any way by the text of the specification. Furthermore, any modifications and changes falling within the equivalent scope of the claims are all within the scope of the present invention.

[0063] For example, the above embodiment shows an implementation as an analysis system using an analysis device 1 and a terminal device 2, but the analysis system disclosed in this application is not limited to this and can be realized in various system configurations. For example, the analysis system disclosed in this application can be developed into various forms, such as an analysis device 1 in which the analysis device 1 and terminal device 2 are integrated, or an analysis device 1 formed by linking multiple server computers. Furthermore, even when the analysis system disclosed in this application is configured as an analysis device 1 and a terminal device 2, it is possible to design as appropriate which device will execute each process. For example, auxiliary analysis processing can be performed by terminal device 2 alone. In that case, terminal device 2 will execute the analysis program 110 and effectively operate as analysis device 1.

[0064] Furthermore, although the above embodiment shows a configuration in which Japanese text data is the target of analysis, the analysis system disclosed in this application is not limited to this, and can analyze text data of various natural languages, such as natural languages ​​other than Japanese, or languages ​​from multiple countries.

[0065] Furthermore, although the above embodiment shows a form in which text data is the target of analysis, the analysis system disclosed in this application is not limited to this, and can analyze various targets as long as each analysis data is data containing multidimensional elements. For example, the analysis system disclosed in this application can analyze various data such as audio, images, and videos. Also, when analyzing various analysis data other than qualitative data such as text data, it is possible to use analysis methods appropriate to the target of analysis for element analysis. Specifically, for measurable data such as audio, it is possible to develop various forms, such as decomposing the data into feature quantities for each element using factor analysis, rather than just correspondence analysis. [Explanation of Symbols]

[0066] 1 Analyzer 10 Control Unit 11 Storage section 110 Analysis Programs 12 Communications Department 2 Terminal devices 20 Control Unit 21 Memory section 22 Communications Department 23 Input section 24 Output section 3. Communication device 4 Storage device Network

Claims

1. An analytical system equipped with an analytical device for analyzing the target of analysis, An analytical data acquisition means for acquiring multiple analytical data included in the target of analysis, A feature derivation means for deriving feature quantities for each element of each acquired analysis data that can be represented as N-dimensional (N is a natural number) vector data, A distance derivation means for deriving the distance between each analysis data based on the derived N-dimensional vector data, A clustering means that performs hierarchical clustering of each analysis data based on the distance between each derived analysis data, A result output means that outputs the results of hierarchical clustering processing as analysis results. An analytical system characterized by including [this].

2. Further equipped with a means for displaying images, The result output means is, A first display means that displays an image of a dendrogram on the display means, in which each analysis data processed using hierarchical clustering is used as a leaf node, A boundary position receiving means that receives input for an instruction operation that specifies the position on the dendrogram of the boundary line that divides each leaf node into cluster units, A second display means for displaying an image in which the boundary line passing through the input location is superimposed on the dendrogram, A third display means that displays an image showing each leaf node in a different display method for each cluster divided by the aforementioned boundary line. including The analytical system according to feature 1.

3. The subject of analysis includes text data containing language data that has meaning as natural language, The aforementioned data acquisition means acquires morphemes obtained by performing morphological analysis on the text data included in the target of analysis as data for analysis. The acquired analytical data includes text data in sentence units containing one or more morphemes. The feature extraction means derives features by performing a correspondence analysis on the acquired analysis data based on a cross-analysis table that quantifies the commonalities of morphemes contained in each sentence. The analytical system according to claim 1 or 2, characterized by the features described above.

4. The feature derivation means further comprises a selection means that selects M elements (where M is a natural number less than or equal to N) from among the N elements constituting the N-dimensional vector data of each analysis data derived by the feature derivation means, in order from the element with the highest contribution rate, such that the cumulative value of the contribution rate is greater than or equal to a predetermined value. The distance derivation means derives the distance between each analysis data using M-dimensional vector data, which is composed of feature quantities of M elements selected from the N elements that make up the N-dimensional vector data. The analytical system according to claim 1 or 2, characterized by the features described above.

5. An analytical apparatus equipped with an analytical device for analyzing the object to be analyzed, An analytical data acquisition means for acquiring multiple analytical data included in the target of analysis, A feature derivation means for deriving feature quantities for each element of each acquired analysis data that can be represented as N-dimensional (N is a natural number) vector data, A distance derivation means for deriving the distance between each analysis data based on the derived N-dimensional vector data, A clustering means that performs hierarchical clustering of each analysis data based on the distance between each derived analysis data, A result output means that outputs the results of hierarchical clustering processing as analysis results. An analytical apparatus characterized by including [a specific component].

6. An analytical method using an analytical instrument to analyze the target of analysis, An analysis data acquisition step to obtain multiple analysis data included in the target of analysis, For each acquired analysis data, a feature derivation step is performed to derive the feature quantities of each element that can be represented as N-dimensional (N is a natural number) vector data, Based on the derived N-dimensional vector data, a distance derivation step is performed to derive the distance between each analysis data, A clustering step is performed to perform hierarchical clustering of each analysis data based on the distance between each derived analysis data, A result output step that outputs the results of hierarchical clustering processing as analysis results. An analytical method characterized by including multiple steps to perform the following actions.

7. An analysis program that uses a computer to analyze the object of analysis, On the computer, An analysis data acquisition step to obtain multiple analysis data included in the target of analysis, For each acquired analysis data, a feature derivation step is performed to derive the feature quantities of each element that can be represented as N-dimensional (N is a natural number) vector data, Based on the derived N-dimensional vector data, a distance derivation step is performed to derive the distance between each analysis data, A clustering step is performed to perform hierarchical clustering of each analysis data based on the distance between each derived analysis data, A result output step that outputs the results of hierarchical clustering processing as analysis results. An analysis program characterized by performing multiple steps, including those mentioned above.

Images