Information processing system, information processing method, and information processing program

Abstract

To generate retrieval keywords with a high degree of association with a subject of a document.SOLUTION: An information processing system has a processor for executing a program, and a storage device for storing the program. The processor executes conversion processing of converting natural language data into a query that can be analyzed by a predetermined natural language data analysis model, analysis processing of analyzing the query obtained by converting the natural language data analysis model by the conversion processing and outputting an analysis result, first calculation processing of calculating a first contribution value indicating a degree of contribution to the analysis result output by the analysis processing for words in the natural language data, and acquisition processing of acquiring a keyword candidate that is a candidate imparted to the natural language data on the basis of the first contribution value calculated by the first calculation processing.SELECTED DRAWING: Figure 4A

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program for processing information.

Background Art

[0002] In recent years, the performance of machine learning models for analyzing data of sentences or texts described in natural language (referred to as "natural language data") stored in formats such as voice and text has been rapidly improving. Various documents that have been created and then discarded within companies are expected to generate new value by being stored in a database, called up in response to requests, and becoming the analysis targets of machine learning models.

[0003] Documents created within a company for such purposes have characteristics such as each document being relatively long compared to general web searches and the terms used being similar for each company. Therefore, when calling these documents, it is not always optimal to apply the full-text search technology commonly used in web searches. If a heading word is set for each document at the time of database storage and a search regarding the heading word is performed, it is often sufficient in terms of search accuracy and search efficiency.

[0004] On the other hand, selecting an inappropriate word as the heading word of a document causes false detection in searches. Especially when a large number of documents are generated and stored within a company, if the heading words are not carefully selected, a large number of documents with little relevance to the heading words are called up as false positives, and there is a problem that the document search no longer functions effectively.

[0005] Patent Document 1 discloses an apparatus for creating an index for searching a database based on collected information. The apparatus includes a dictionary including a plurality of phrases associated with each other for each specific concept, an input interface unit that receives an input of a text sentence, a text processing unit that extracts a plurality of words from the text sentence as tokens, a directed graph generation unit that generates a directed graph representing a connection relationship between the extracted plurality of tokens, a directed graph search unit that searches within the directed graph while referring to the dictionary and expands the directed graph when a search target phrase is found in the dictionary, and an index creation unit that creates an index based on the plurality of tokens in the directed graph.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] In the technology disclosed in Patent Document 1, when a search target phrase is found in a pre-created dictionary, the directed graph generated from the input text sentence is searched, and an index (heading word) is created based on the tokens in the directed graph, but the usage and importance of the search target phrase in the input text sentence are not referred to. Therefore, if the search target phrase is included in a part with a low relevance to the theme in the input text sentence, the created index is likely to have a low relevance to the theme of the input text, which may cause false detection during search.

[0008] One object of the present disclosure is to provide an information processing system, an information processing method, and an information processing program capable of assigning search heading words with a high relevance to the theme of a document.

Means for Solving the Problems

[0009] An information processing system which is an aspect of the invention disclosed in the present application is an information processing system having a processor that executes a program and a storage device that stores the program, wherein the processor performs a conversion process of converting natural language data into a query analyzable by a predetermined natural language data analysis model, an analysis process of analyzing the query converted by the conversion process by the natural language data analysis model and outputting an analysis result, a first calculation process of calculating a first contribution value representing the degree of contribution to the analysis result output in the analysis process for words in the natural language data, and an acquisition process of acquiring a candidate heading word which is a candidate to be assigned to the natural language data based on the first contribution value calculated in the first calculation process of the previous term.

Effect of the Invention

[0010] According to a typical embodiment of the present invention, it is possible to provide an information processing system, an information processing method, and an information processing program that can assign search heading words with a high degree of relevance to the subject of a document. Problems, configurations, and effects other than those described above will be clarified by the description of the following examples.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 5C

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12

Figure 13

BEST MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment

[0013] FIG. 1 is a block diagram of the information processing system 100 according to Embodiment 1. FIG. 2 is a block diagram showing the hardware configuration of the computer included in the information processing system 100.

[0014] Referring to FIG. 1, the information processing system 100 includes computers 100-1 and 100-2. The natural language analysis system includes computers 100-1 and 100-2 and a terminal 101. The computers 100-1 and 100-2 and the terminal 101 are connected to a communication network 102. The communication network 102 is a facility that enables data transmission and reception via a wired or wireless line.

[0015] The computer 100-1 includes a text analysis unit 111 and a headline candidate presentation unit 112, and is a computer that takes natural language data as input and outputs headline candidates. The computer 100-2 is a computer that can access a natural language analysis model store 120 and an explainable AI program store 130.

[0016] The natural language analysis model store 120 stores one or more natural language analysis models that analyze queries processed from natural language data and output analysis results. In particular, the natural language analysis model may be a model that classifies queries into several classes, a regression model that outputs some numerical values, or a model that outputs analysis results in other formats.

[0017] The explainable AI program store 130 stores one or more programs of explainable AI technologies that can extract a plurality of phrases from a query and calculate the first contribution value for each of the extracted plurality of phrases. Here, the first contribution value is a value representing the degree of contribution (the magnitude of influence) of a phrase to the analysis result obtained by the natural language analysis model analyzing the query. The larger the first contribution value of a phrase, the greater the influence it has on the analysis result of the query Qi. Also, the program of the explainable AI technology is preferably a program that can calculate a set of phrases formed by combining a plurality of phrases in the query and the second contribution value of this set of phrases. Here, the second contribution value is a value representing the degree of contribution of the interaction of the plurality of phrases (the relationship between the plurality of phrases) constituting the set of phrases to the analysis result obtained by the natural language analysis model analyzing the query. The so-called "interaction value" of the explainable AI technology may correspond to the second contribution value. The program of the explainable AI technology may be, for example, a program of an algorithm such as Baseline SHAP, Attention value, ACD, etc.

[0018] Referring to FIG. 2, the computer 100-1 has a processor 201, a main memory device 202, a secondary storage device (storage device) 203, a network interface 204, an input device 205, and an output device 206.

[0019] The secondary storage device 203 is a device that stores data in a writable and readable manner. The data used for calculation processing and the results of calculation processing are stored in the secondary storage device 203. Also, the secondary storage device 203 stores a text analysis program 111A and a heading word candidate presentation program 112A.

[0020] The processor 201 is a device that reads the data stored in the secondary storage device 203 into the main memory device 202 and executes programs. The functions of the text analysis unit 111 and the heading word candidate presentation unit 112 in the computer 100-1 are realized by the processor 201 reading the text analysis program 111A and the heading word candidate presentation program 112A stored in the secondary storage device 203 into the main memory device 202 and executing them.

[0021] The network interface 204 is a device that transmits and receives data to and from other devices via the communication network 102. The network interface 204 can transmit the data of the processing result of the processor 201 to other devices via the communication network 102. Also, the network interface 204 can receive data to be used for the processing of the processor 201 from other devices via the communication network 102.

[0022] The input device 205 is a device that accepts operations by the user, such as a keyboard or a mouse, and acquires the information input by the user's operation. The output device 206 is a device that outputs information, such as a display or a speaker, and presents the information to the user, for example, by displaying it on the screen.

[0023] The computer 100-2 and the terminal 101 can be configured by using the same hardware resources as the computer 100-1. In the computer 100-2, the natural language analysis model store 120 and the explainable AI program store 130 are stored in the auxiliary storage device 203.

[0024] Returning to FIG. 1, the computer 100-1 includes a text analysis unit 111 and a candidate heading word presentation unit 112. The computer 100-1 generates candidate heading words for the input text T, which is natural language data, based on the setting information I. When generating candidate heading words for voice data or the like that is not text data, the voice data or the like may be converted into text in advance by a known method and stored as the input text T.

[0025] The setting information I is set for a user or the like who operates the terminal 101. FIG. 3 is an explanatory diagram showing an example of a setting screen for displaying the setting information I. As an example, the setting screen 300 in FIG. 3 can include a query task setting column 310, a model specification column 320, a query number setting column 330, a contribution value calculation method setting column 340, a candidate adoption threshold setting column 350, a combination weight setting column 360, and the like. The user can select conditions by clicking on check boxes (such as check box 311). The check boxes selected by the user are displayed in black, and the unselected check boxes are displayed in white.

[0026] The query task setting column 310 is a column for setting the task (analysis content) to be analyzed by the natural language analysis model N. The query is created by processing the input text T. The composition of the query changes according to the analysis content of the task for which the natural language analysis model N analyzes the query. In FIG. 3, it is shown that "Next Sentence Prediction" corresponding to the black-filled check box 312 is selected as the analysis content of the task. The user can also select "Masked Language Model" or "Classification" depicted in the task setting column 310 of FIG. 3.

[0027] The model specification column 320 is a column for specifying the natural language analysis model N to be read from and used by the natural language analysis model store 120. The user can specify the natural language analysis model N by clicking on the button 321 depicted as "Reference" in the model specification column 320. In FIG. 3, the reference column 322 in the model specification column 320 is blank, indicating that the default natural language analysis model N related to "Next Sentence Prediction" selected in the query task setting column 310 is selected.

[0028] The query number setting column 330 is a column for setting the number of text segments Ti to be split from the input text T. Also, the query Qi is created by processing the text segment Ti. In FIG. 3, it shows that the input text T is split into eight (column 331) to generate eight text segments Ti. Note that it is also possible to automatically set the number of the text segments Ti from the length of the input text T or the natural language analysis model N or the metadata associated with the natural language analysis model N.

[0029] The contribution value calculation method setting column 340 is a column for setting the calculation method when calculating the first contribution value from the output of the natural language analysis model N (the analysis result of the query Qi) for the words Pi1, Pi2 to Pim in the query Qi (text segment Ti). In this column, for example, algorithms of so-called explainable AI (eXplainable AI) technologies such as Baseline SHAP, Attention value, ACD, etc. illustrated in FIG. 3, and information such as conditions related to the calculation can be set. Needless to say, the methods that can be adopted as explainable AI are not limited to the methods illustrated here. FIG. 3 shows that "ACD" corresponding to the black-filled checkbox 343 is selected.

[0030] Also, some or all of the algorithms that can be set in this column can be highlighted or hidden according to the applicability calculated from the natural language analysis model N or the metadata associated with the natural language analysis model N or the information related to the query Qi.

[0031] The candidate adoption threshold setting column 350 is a column for setting the extraction conditions for adopting the words Pi1, Pi2 to Pim in the query Qi (text segment Ti) as heading word candidates according to the calculated first contribution value. In FIG. 3, the condition of "adopting words with a first contribution value of 0.20 (column 352) or more as heading word candidates" corresponding to the white-filled checkbox 351 is not selected, and it shows that the condition of "adopting words with the top eight (column 354) first contribution values as heading word candidates" corresponding to the black-filled checkbox 353 is selected as the extraction condition.

[0032] In this way, for example, in this column, the extraction condition can be set as the threshold value for the first contribution value. As the threshold value, statistical values such as the average value and the standard deviation can be calculated from the first contribution values calculated for the phrases Pi1, Pi2 to Pim, and candidates for the extraction condition can be set as the threshold values for the statistical values. Also, for example, in this column, the maximum number of phrases to be adopted as candidate headings can be set.

[0033] The combination weight setting column 360 is a column for setting the extraction condition for the second contribution value of the phrase combination. In this column, for example, the maximum number of phrases (performing interaction) constituting the phrase combination to be calculated can be set. Also, for example, in this column, the extraction condition can be set as the threshold value for the second contribution value. As the threshold value, statistical values such as the average value and the standard deviation can be calculated from the second contribution value, and it is also possible to set the threshold value for the statistical value. Also, for example, in this column, the maximum number of phrase combinations to be adopted as candidates can be set.

[0034] In FIG. 3, the check box 361 is white, indicating that the setting is to calculate the phrase combination and the second contribution value. Also, the check box 362 is white, indicating that the condition "the maximum number of phrases constituting the phrase combination to be calculated is up to 4 (column 363)" is not set. Also, both the check box 364 and the check box 366 are black, indicating that both "phrase combinations with a second contribution value of 0.20 (column 365) or more" corresponding to the check box 364 and "phrase combinations with a rank from the top of the second contribution value of 8 (column 367) or more" corresponding to the check box 366 are set as extraction conditions.

[0035] The user makes settings for some or all of the query task setting field 310, model specification field 320, query number setting field 330, contribution value calculation method setting field 340, candidate adoption threshold setting field 350, and combination weight setting field 360, and presses the save button 301, whereby the setting information I depicted on the setting screen 300 is saved. The saved setting information I is applied to the processing by the text analysis unit 111. At the same time, these setting information can be stored in the auxiliary storage device 203.

[0036] Also, the GUI (Graphical User Interface) shown in the setting screen 300 of FIG. 3 is an example. The setting screen is not limited to the setting screen 300 shown in FIG. 3. Also, the user does not necessarily have to set all the setting information I, and may automatically determine the settings in an appropriate manner.

[0037] The input text T and the setting information I are stored in a device that can transmit and receive data with the computer 100-1 and the terminal 101. The user operates the terminal 101 to transmit, from the terminal 101 to the computer 100-1, the information on the location where the input text T is stored, the information on the location where the setting information I is stored, and the processing start command for creating the heading word candidates. Instead of the user transmitting from the terminal 101 to the computer 100-1 in this way, the user may input to the computer 100-1. Also, the computer 100-1 may automatically read the default setting information I.

[0038] FIG. 4A is a flowchart showing an example of a text analysis procedure by the text analysis unit 111 of the information processing system 100.

[0039] When the information processing system 100 receives a processing start command created by the user inputting from the terminal 101 or the like, it acquires the specified input text T and the setting information I created by the user operating the terminal 101 or stored in another device via the communication network 102 (step S401).

[0040] Next, based on the acquired setting information I, the information processing system 100 reads the natural language analysis model N from the natural language analysis model store 120, and also reads the program of the explainable AI technology from the explainable AI program store 130 (step S402).

[0041] Next, the information processing system 100 divides part or all of the input text T into text pieces T1, T2 to Tn based on the acquired setting information I (step S403). Note that this division does not necessarily require dividing all parts of the input text T into any of the text pieces T1, T2 to Tn. Also, this division does not prevent there from being overlapping parts among the text pieces T1, T2 to Tn.

[0042] Next, the information processing system 100 starts loop processing for the divided text piece Ti (step S404). The processing shown between S404 at the start of the loop and S407 at the end of the loop is repeated for each text piece Ti. In step S404, the information processing system 100 selects one text piece Ti from the unprocessed text pieces of T.

[0043] Next, based on the acquired setting information I, the information processing system 100 processes part or all of the text segment Ti into a query Qi that can be analyzed by the natural language analysis model N (step S405). Here, "processing" means, for example, when the natural language analysis model N is configured as a Masked Language Model, masking a part of the text segment Ti so that it can be input data for the natural language analysis model N to infer the masked word. In this way, by replacing some of the words and phrases in the text segment (natural language data) Ti with (masks) to process it into the query Qi, an appropriate query Qi can be provided. Also, when the natural language analysis model N is configured as a model that directly accepts natural language data as input data and classifies topics (Classification), the text segment Ti may be used directly as the query Qi without any operation. Further, when the natural language analysis model N is configured to make inferences regarding Next Sentence Prediction, it is processed into the query Qi by operations such as partially swapping the order of the sentences in the text segment Ti. In this way, by swapping the order of some of the sentences in the text segment (natural language data) Ti to process it into the query Qi, an appropriate query Qi can be provided. Note that the processing is not limited to these examples.

[0044] Next, for the output (analysis result) obtained by having the natural language analysis model N analyze the query Qi, the information processing system 100 calculates word groups Pi1, Pi2, …, Pim within the text segment Ti (query Qi), calculates the first contribution value to the output (analysis result) from word groups Pi1, Pi2, …, Pim, and further calculates a word group combination formed by grouping a plurality of word groups among Pi1, Pi2, …, Pim and its second contribution value (step S406). Here, since the word groups Pi1, Pi2, …, Pim calculated from the query Qi are the word groups included in the query Qi, they are the word groups included in the text segment Ti that is the source of the query Qi and the input text T. Also, the word groups Pi1, Pi2, …, Pim may be words having meanings or units formed by concatenating them, or may be units having no meaning individually created mechanically from the appearance frequencies in external data (for example, the learning data when the natural language analysis model N previously performed supervised learning).

[0045] Also, here, the calculation of the first contribution value to the output (analysis result) refers to an operation of quantitatively obtaining, as the contribution from the word groups Pi1, Pi2, …, Pim within the text segment (natural language data) Ti, the value of the output result (analysis result) itself or the basis on which the likelihood value related to the output result was calculated. For this operation, for example, so-called explainable AI technology can be used. Therefore, using the program of the explainable AI technology read out in step S402, the word groups Pi1, Pi2, …, Pim within the text segment Ti (query Qi) and their first contribution value are calculated. Thereby, an appropriate first contribution value can be calculated.

[0046] When, for example, the natural language analysis model N is configured as a Masked Language Model by the explainable AI technology, the degree of contribution (first contribution value) of the words Pi1, Pi2 to Pim in the text segment Ti can be obtained in the process of the natural language analysis model N inferring the masked word. When the natural language analysis model N is configured as a model for classifying the topics of natural language data, the degree of contribution (first contribution value) of the words Pi1, Pi2 to Pim in Ti can be obtained in the process of the classification. When the natural language analysis model N is configured to make inferences regarding Next Sentence Prediction, the degree of contribution (first contribution value) of the words Pi1, Pi2 to Pim in the text segment Ti can be obtained in the process of the inference. Needless to say, the configuration of the natural language analysis model N to which the explainable AI technology can be applied is not limited to these examples.

[0047] Also, as forms of the contribution value, there are a first contribution value and a second contribution value. The information processing system 100 calculates a set of words and its second contribution value using the program of the explainable AI technology based on the setting information I (refer to the combination weight setting column 360 in FIG. 3).

[0048] Next, the information processing system 100 determines whether the processing for all the text segments Ti has been completed (step S407).

[0049] If it is determined that the processing for all the text segments Ti has not been completed, the information processing system 100 returns to step S404 and continues the processing.

[0050] When it is determined that the processing has been completed for all text pieces Ti, the information processing system 100 extracts heading word candidates A1, A2, …, Ak from word groups P11, P12, …, Pnm, and also extracts heading word candidates for groups of word groups from groups of word groups (step S408). That is, among word groups P11 to Pnm, word groups that satisfy the extraction conditions of the setting information I (refer to the candidate adoption threshold setting column 350 in FIG. 3) are extracted as heading word candidates A1, A2, …, Ak. Also, among groups of word groups, groups of word groups that satisfy the extraction conditions for groups of word groups of the setting information I (refer to the combination weight setting column 360 in FIG. 3) are extracted as heading word candidates for groups of word groups. Note that hereinafter, the heading word candidates A1, A2, …, Ak may also be referred to as high contribution value word groups A1, A2, …, Ak.

[0051] Next, the information processing system 100 transmits the heading word candidates A1, A2, …, Ak calculated in step S408, the heading word candidates for groups of word groups, and their second contribution values to the heading word candidate presentation unit 112 and ends the processing (step S409).

[0052] FIG. 4B is a flowchart showing an example of a procedure for presenting heading word candidates by the heading word candidate presentation unit 112 of the information processing system 100.

[0053] The heading word candidate presentation unit 112 of the information processing system 100 acquires and stores the heading word candidates A1, A2, …, Ak, the heading word candidates for groups of word groups, and their second contribution values from the text analysis unit 111 (step S451).

[0054] Next, the information processing system 100 presents the heading word candidates A1, A2 to Ak and the heading word candidates of the phrase sets and their second contribution values to the user using a display or a speaker and ends the process (step S452). Here, the information processing system 100 outputs (transmits) the information such as the heading word candidates A1, A2 to Ak to be presented to the terminal 101, and causes the terminal 101 to display it on the display of the terminal 101 and present it to the user who operates the terminal 101. Here, the information processing system 100 may cause the terminal 101 to present it by voice using the speaker of the terminal 101. Further, the information processing system 100 may display the information such as the heading word candidates A1, A2 to Ak to be presented on the display of the computer 100-1, or may present it by voice using the speaker of the computer 100-1.

[0055] Note that in the above, in the example of the text analysis procedure shown in the flowchart of FIG. 4A and the example of the procedure for presenting the heading word candidates shown in the flowchart of FIG. 4B, the phrase sets and their second contribution values, the heading word candidates of the phrase sets and their second contribution values are calculated and presented, but it is not necessarily required to calculate them, nor is it necessary to present them.

[0056] FIG. 5A is an explanatory diagram showing an example of a heading word candidate display screen according to the first embodiment. The heading word candidate display screen 500 is displayed on the display of the terminal 101 based on the information created by the heading word candidate presentation unit 112. The heading word candidate display screen 500 has a heading word candidate display column 501 and a combination score display column 502. In the heading word candidate display screen 500 of FIG. 5A, the user can select the heading word candidates to be set as the heading words by clicking on the check boxes (such as the check box 511). The check boxes selected by the user are displayed in black, and the unselected check boxes are displayed in white.

[0057] The heading word candidate display column 501 displays some or all of the heading word candidates A1, A2 to Ak. In the heading word candidate display column 501 of FIG. 5A, the heading word candidates "Customization Support", "Additional Add-on", and "Wide Area Common" are shown as the heading word candidates, and these words have a higher first contribution value the more to the left they are. Also, in FIG. 5A, it is shown that the heading word candidates "Additional Add-on" and "Wide Area Common" corresponding to the blacked-out check boxes 512 and 513 are selected as the heading words.

[0058] In this column, for example, the first contribution value can be displayed together with the heading word candidates A1, A2 to Ak. Also, in this column, for example, a list including some or all of the heading word candidates A1, A2 to Ak may be displayed, or after displaying some or all of the text T, the words Pi1, Pi2 to Pim may be displayed, and furthermore, some or all of the locations of the heading word candidates A1, A2 to Ak may be highlighted and displayed. Also, in this column, for example, after displaying some or all of the query Qi, the words Pi1, Pi2 to Pim may be displayed, and furthermore, some or all of the locations of the heading word candidates A1, A2 to Ak may be highlighted and displayed. By displaying as described above, the user can more easily select the heading word from the heading word candidates.

[0059] Note that, as described above, the words Pi1, Pi2 to Pim and their first contribution values, and the first contribution values of the heading word candidates A1, A2 to Akm are transmitted (output) from the text analysis unit 111 to the heading word candidate presentation unit 112 as necessary, and further transmitted (output) from the heading word candidate presentation unit 112 to the terminal 101.

[0060] The combination score display column 502 displays the candidate headwords for the phrase sets. In this column, for example, a list including some or all of the candidate headwords for the phrase sets may be displayed. Also, the candidate headwords for the phrase sets can be displayed in association with their second contribution values. Further, in this column, for example, after displaying some or all of the text T, some or all of the positions of the candidate headwords for the phrase sets may be highlighted and displayed. Also, in this column, for example, after displaying some or all of the query Qi, some or all of the positions of the candidate headwords for the phrase sets may be highlighted and displayed. By displaying as described above, the user can more easily select the headwords for the phrase sets from the candidate headwords for the phrase sets.

[0061] Note that in the above, the phrase sets (other than the candidate headwords for the phrase sets) and their second contribution values are, if necessary, transmitted from the text analysis unit 111 to the headword candidate presentation unit 112, and further transmitted (output) from the headword candidate presentation unit 112 to the terminal 101.

[0062] In the combination score display column 502 of FIG. 5A, it is shown that the candidate headword for the phrase set corresponding to the white check box 521, "night batch version management", is not selected as the headword for the phrase set, and its second contribution value is "+0.18". Also, in FIG. 5A, it is shown that the candidate headword for the phrase set corresponding to the black check box 522, "add-on unique specification existing system", is selected as the headword for the phrase set, and its second contribution value is "+0.28".

[0063] Also, when the user presses the adoption button 503, a command is sent to save the selected heading word candidate words and the set of phrases of the heading word candidate as heading words to a search database (not shown) for the text T, and the heading words can be associated with the text T and saved in the search database (not shown). Alternatively, metadata for the text T can be created for the selected heading word candidate words and the set of phrases of the heading word candidate and stored in the secondary storage device 203. The metadata can also include the first contribution values of the phrases Pi1, Pi2 to Pim and the second contribution values of the set of phrases (the heading word candidate of the set of phrases and the rest).

[0064] Next, among the example text analysis procedures by the text analysis unit 111 of the information processing system 100 shown as a flowchart in FIG. 4A, examples of the central processes of step S403, steps S405 to S406 will be described in detail with reference to FIGS. 5B and 5C.

[0065] First, as shown in FIG. 5B, the input text T is divided into text pieces T1, T2 to Tn (step S403), and each of the further divided text pieces T1, T2 to Tn is processed to create queries Q1, Q2 to Qn (step S405).

[0066] Next, in step S406, the program of the explainable AI technology uses the query Qi and the natural language analysis model N to obtain the phrases Pi1, Pi2 to Pim in the query Qi. Since the query Qi is created by processing the text piece Ti, the phrases Pi1, Pi2 to Pim are the phrases included in the text piece Ti and the input text T.

[0067] Next, the first contribution value is calculated. In FIG. 5C, as an example, the natural language analysis model N is configured to make inferences regarding Next Sentence Prediction, and the analysis of the natural language analysis model N is used to infer whether the order of the 7th sentence (sentence 7 in the figure) from the beginning of the query Qi and the 8th sentence (sentence 8 in the figure) is correct.

[0068] In the program of the explainable AI technology, as shown in FIG. 5C, the query Qi is analyzed by the natural language analysis model N, and an analysis result is obtained that the order of the seventh sentence from the beginning and the eighth sentence is 50% correct. And before the seventh sentence from the beginning, the word "park" is written, and after the eighth sentence, the word "cloud" is written.

[0069] Furthermore, in the program of the explainable AI technology, a replacement query Qt1 is created by replacing the word "park" before the seventh sentence from the beginning of the query Qi with a predetermined word (replacement word), and the replacement query Qt1 is analyzed by the natural language analysis model N. Assume that the analysis result 1 (replacement analysis result) obtains a result that the order of the seventh sentence from the beginning and the eighth sentence is 50% correct. Since the analysis result 1 is the same as the analysis result (50%) of the query Qi, it is considered that the word "park" replaced with the predetermined word (replacement word) in the replacement query Qt1 has little influence on the analysis result of the query Qi, and its first contribution value is small.

[0070] Furthermore, in the program of the explainable AI technology, a replacement query Qt2 is created by replacing the word "cloud" after the eighth sentence from the beginning of the query Qi with a predetermined word (replacement word), and the replacement query Qt2 is analyzed by the natural language analysis model N, and an analysis result 2 (replacement analysis result) that the order of the seventh sentence from the beginning and the eighth sentence is 70% correct is obtained. The analysis result 2 has increased by 20% from the analysis result (50%) of the query Qi, and it is considered that the word "cloud" replaced with the predetermined word (replacement word) in the replacement query Qt2 has a great influence on the analysis result of the query Qi, and its first contribution value is a relatively large value.

[0071] In the program of the explainable AI technology, for each of the various words in query Qi, the above-described processes (1. Create a replacement query by replacing some of the words in query Qi with replacement words. 2. Analyze the replacement query using the natural language analysis model N to obtain a replacement analysis result.) are repeated. Then, by performing operations such as comparing the words replaced with replacement words and the replacement analysis results, a first contribution value for each of the phrases Pi1, Pi2 to Pim in query Qi (within text segment Ti) is calculated.

[0072] Note that the method for calculating the second contribution value varies depending on the algorithm of the explainable AI technology, and the explanation thereof is omitted here.

[0073] The input text T (sentence) is created based on the theme. Therefore, when the natural language analysis model N analyzes query Qi, the order of words in query Qi, the content of query Qi, the order of sentences in query Qi, etc., which are the objects referred to by the natural language analysis model N, and the analysis result of the natural language analysis model N change according to the theme of the input text T. And since the keywords regarding the theme of the input text T have a high relevance to the theme of the input text T, they have a great influence on the objects referred to by the natural language analysis model N when analyzing query Qi and the analysis result of the natural language analysis model N. For this reason, the keywords regarding the theme of the input text T are considered to be phrases with large first contribution values. Also, since the candidate heading words calculated by the information processing system 100 are phrases with relatively large first contribution values, the candidate heading words calculated by the information processing system 100 are considered to correspond to the keywords regarding the theme of the input text T. Therefore, the candidate heading words are considered to have a high relevance to the theme of the input text T. In other words, the information processing system 100 can appropriately estimate candidate heading words with a high degree of relevance to the theme of the input text T (sentence). And since the heading word is selected from the candidate heading words, the information processing system 100 can assign a search heading word with a high degree of relevance to the theme of the document.

[0074] As described above, according to Example 1, it is possible to strongly contribute to the analysis of the natural language analysis model N for the text T and present to the user the words and phrases estimated to be important in the text T (see Fig. 5A). Therefore, the user can grasp that the said words and phrases are candidate headwords when storing the text T in the search database. Also, by presenting candidate headwords to the user, it becomes easier for the user to select a headword from the candidate headwords.

[0075] In particular, in the query task setting field 310 (see Fig. 3), by setting a task that refers to the global semantic information of the text (the context of many parts of the text) such as Next Sentence Prediction and selecting the natural language analysis model N suitable for the task, it is possible to appropriately identify the words and phrases that significantly contribute to the reference of the global semantic information of the text and adopt them as candidate headwords when storing the text T in the search database.

[0076] In particular, when storing the text T in the search database, it may be necessary to calculate a weight score associated with the headword. By using the first contribution value of the words and phrases Pi1, Pi2 to Pim and the second contribution value of the combination of words and phrases as the criteria for calculating the search weight score associated with the headword when storing the text T in the search database, it is possible to refer to the semantic importance of the said words and phrases within the text and the increase or decrease in importance due to the words and phrases being used in the said combination, and reflect it in the priority display of search results, etc.

[0077] In particular, by generating candidate headwords, it becomes easier to create headwords for the article and assign headwords to the article. As a result, it is possible to reduce the energy required to assign headwords to the article and the amount of carbon dioxide emissions generated, and suppress global warming.

Example

[0078] In Example 2, for the phrases A1, A2 to Ak identified as candidate keywords in Example 1, they are matched with an externally created keyword dictionary to provide candidate keywords in a more intuitive and useful form closer to the phrases used by the user when searching. In Example 2, for the purpose of mainly explaining the differences from Example 1, the same components as those in Example 1 are denoted by the same reference numerals, and their descriptions are omitted.

[0079] FIG. 6A is a block diagram of the information processing system 100A according to Example 2. FIG. 6B is a block diagram showing the hardware configuration of the information processing system 100A. Referring to FIG. 6A, in the information processing system 100A of Example 2, the computer 100-1 includes a dictionary matching unit 610 in addition to the text analysis unit 111 and the candidate keyword presentation unit 112. The computer 100-2 includes a keyword dictionary store 620 in addition to the natural language analysis model store 120 and the explainable AI program store 130.

[0080] The keyword dictionary store 620 stores a keyword dictionary D which is a list of keywords created manually by the user or acquired from outside. That is, the keyword dictionary D is keyword dictionary data including one or more keywords that can be assigned to the input text T (natural language data).

[0081] Referring to FIG. 6B, in the information processing system 100A, the computer 100-1 stores in the auxiliary storage device 203 a text analysis program 111A, a candidate keyword presentation program 112A, a dictionary matching program 610A, and a candidate keyword list 611A. The computer 100-2 stores in the auxiliary storage device 203 the natural language analysis model store 120, the explainable AI program store 130, and the keyword dictionary store 620. The function of the dictionary matching unit 610 in the computer 100-1 is realized by the processor 201 reading out the dictionary matching program 610A stored in the auxiliary storage device 203 into the main storage device 202 and executing it.

[0082] The heading word candidate list 611A is a database of heading candidates that can register by associating a high contribution value word A to be adopted as a heading word, the first contribution value of the high contribution value word A, and information indicating whether the high contribution value word A is included in the heading word dictionary D.

[0083] In the second embodiment, the text analysis unit 111 transmits the input text T, the words P11, P12 to Pnm, their first contribution values, the high contribution value words (heading word candidates) A1, A2 to Ak, their first contribution values, and the setting information I to the dictionary matching unit 610. Also, the text analysis unit 111 transmits the heading word candidates of the word sets and their second contribution values to the heading word candidate presentation unit 112. Further, the setting information I in the second embodiment includes a new word adoption criterion in addition to the setting information I in the first embodiment.

[0084] FIG. 7 is an explanatory diagram showing an example of a setting screen for analysis conditions according to the second embodiment.

[0085] In the second embodiment, as an example, the setting screen 300 can include a new word adoption criterion setting column 910 in addition to a query task setting column 310, a model designation column 320, a query number setting column 330, a contribution value calculation method setting column 340, a candidate adoption threshold setting column 350, and a combination weight setting column 360. In the setting screen 400, the check boxes selected by the user are displayed in black, and the unselected check boxes are displayed in white.

[0086] The new word adoption criterion setting column 910 is a column for setting a new word adoption criterion, which is a condition for determining whether a high contribution value word A can be regarded as a word not included in the heading word dictionary D (a new word). That is, among the words stored in the heading word dictionary D, if a word d with a large overlapping part with the high contribution value word A (a word similar to the high contribution value word A) and the high contribution value word A satisfy the new word adoption criterion, the high contribution value word A is regarded as a word not included in the heading word dictionary D, and if not satisfied, the high contribution value word A is regarded as a word included in the heading word dictionary D.

[0087] Further, in FIG. 7, it shows that the condition of "a phrase with a duplication rate of the high contribution value phrase A and the word d of 0.20 (column 912) or less" corresponding to the white-filled checkbox 911 is not used as the new word adoption criterion. Also, in FIG. 7, it shows that the condition of "a phrase with a duplication word count of the high contribution value phrase A and the word d of 2 (column 914) or less" corresponding to the black-filled checkbox 913 is used as the new word adoption criterion. Further, in FIG. 7, it shows that the condition of "a phrase with an edit distance between the high contribution value phrase A and the word d of 5 (column 916) or more" corresponding to the white-filled checkbox 915 is not used as the new word adoption criterion.

[0088] In this column, for example, this condition can be set as a threshold value regarding the duplication count in character units between the high contribution value phrase A and the word d and the duplication rate compared to the word length. Also, in this column, for example, this condition can be set as a threshold value regarding the edit distance between the high contribution value phrase A and the word d. As these threshold values, statistical values such as an average value and a standard deviation can be calculated using the phrases included in the text T or the heading word dictionary D, and set as the threshold values regarding the statistical values. Also, the statistical values may be calculated based on the co-occurrence relationship in a separately prepared corpus or the text T.

[0089] FIG. 8 is a flowchart showing an example of a dictionary matching procedure by the dictionary matching section 610 according to the second embodiment. Hereinafter, an example of the dictionary matching procedure will be described along the flowchart of FIG. 8 with reference to FIG. 9. Note that FIG. 9 is a diagram showing an example of information created between step S704 and step S711 in tabular form. Although FIG. 9 shows an example of information in tabular form, it is not necessary that data is actually created in such a form inside the dictionary matching section 610, and it is not necessary that information for each high contribution value phrase A is aggregated in this way.

[0090] The information processing system 100A (dictionary matching unit 610) receives and stores the input text T, the phrases P11 to Pnm and their first contribution values, the high contribution value phrases (candidate headwords) A1, A2 to Ak and their first contribution values, and the setting information I that are transmitted from the text analysis unit 111 to the dictionary matching unit 610. Also, it reads the headword dictionary D from the headword dictionary store 620 (step S701).

[0091] Next, the information processing system 100A (dictionary matching unit 610) starts loop processing for the high contribution value phrase A that is the received candidate headword (step S702). The processing shown between the loop start step S702 and the loop end step S711 is repeated for each high contribution value phrase Ai. In step S702, the information processing system 100A selects one high contribution value phrase Ai from the unprocessed high contribution value phrases A.

[0092] Next, the information processing system 100A refers to the headword dictionary D and selects a word d that has the most overlapping parts with the high contribution value phrase Ai among the words stored in the headword dictionary D (step S703). Here, the overlapping parts may be extracted based on, for example, the number of overlaps in character units, the overlap rate compared to the length of the word, or may be extracted based on the edit distance, etc., or may be calculated based on the co-occurrence relationship in a separately prepared corpus or the text T. Also, the word d is a word similar to the candidate headword extracted from the headword dictionary D (headword dictionary data).

[0093] Next, the information processing system 100A determines whether the high contribution value phrase Ai and the selected word d satisfy the new word adoption criteria set in the setting information I (step S704). If it is determined that the high contribution value phrase Ai and the selected word d satisfy the new word adoption criteria (step S704: YES), it proceeds to step S705, and if it is determined that the contribution value phrase Ai and the selected word d do not satisfy the new word adoption criteria (step S704: NO), it proceeds to step S706.

[0094] Next, the information processing system 100A adds the high contribution value phrase Ai to the heading word candidate list 611A (step S705). After that, the information processing system 100A proceeds to step S711. Here, since the high contribution value phrase Ai was determined to satisfy the new word adoption criteria in step S704 (step S704: YES), the contribution value phrase Ai is regarded as a word not included in the heading word dictionary D (heading word dictionary data). The information processing system 100A associates the high contribution value phrase Ai, the first contribution value of the high contribution value phrase Ai, and information indicating that the high contribution value phrase Ai is not included in the heading word dictionary D, and registers them in the heading word candidate list 611A.

[0095] In the row of the information 810 in FIG. 9, “ceramics” is shown as an example of the high contribution value phrase Ai in step S705 (when the high contribution value phrase Ai and the word d satisfy the new word adoption criteria).

[0096] Returning to FIG. 8, next, the information processing system 100A determines whether the high contribution value phrase Ai has a portion not included in the word d (step S706). If it is determined that the high contribution value phrase Ai has a portion not included in the word d (step S706: YES), the process proceeds to step S707. If it is determined that the high contribution value phrase Ai does not have a portion not included in the word d (step S706: NO), the process proceeds to step S708.

[0097] Next, the information processing system 100A changes the division of the phrases Pi1, Pi2 to Pim for calculating the first contribution value in the text piece Ti, divides the high contribution value phrase Ai into a portion overlapping with the word d and a portion not overlapping with the word d, and then sends an instruction to re - execute the first contribution value calculation to the text analysis unit 111 (step S707). The dictionary matching unit 610 receives the phrase that is again regarded as a heading word candidate, and continues the loop process for the high contribution value phrases after step S702 for the received phrase.

[0098] In the row of information 820 in FIG. 9, as an example of the high contribution value phrase Ai in step S707 (when the high contribution value phrase A does not have a part included in word d), "Shinbashi Station" is shown as an example of the high contribution value phrase Ai, and "Shinbashi" is shown as an example of word d. In this example, the overlapping part between the high contribution value phrase Ai and word d is "Shinbashi", and the non-overlapping part is "Station".

[0099] Returning to FIG. 8, next, the information processing system 100A determines whether word d has a part not included in the high contribution value phrase Ai (step S708). If it is determined that word d has a part not included in the high contribution value phrase A (step S708: YES), the process proceeds to step S710. If it is determined that word d does not have a part not included in the high contribution value phrase A (step S708: NO), the process proceeds to step S709.

[0100] Next, the information processing system 100A adds word d to the heading word candidate list 611A (step S709). Then, the information processing system 100A proceeds to step S711. When adding word d to the heading word candidate list 611A, the information processing system 100A associates word d (as a heading word candidate), the first contribution value of the high contribution value phrase Ai, and the information indicating that the high contribution value phrase Ai is included in the heading word dictionary D, and registers them in the heading word candidate list 611A. In this way, replacing the high contribution value phrase Ai with word d (a word similar to the heading word candidate) and registering it in the heading word candidate list 611A means, in other words, processing the high contribution value phrase (heading word candidate) Ai based on word d similar to the heading word candidate to make it word d. Also, this is to transform the heading word candidate Ai extracted as the high contribution value phrase based on word d in the heading word dictionary.

[0101] Next, the information processing system 100A sets the part of the word d that is not included in the high contribution value word group Ai as the word d′, receives the first contribution value of the word d′ from the text analysis unit 111, and determines whether the value obtained by adding the first contribution value of the high contribution value word group Ai satisfies the extraction condition for adopting the heading word candidate set in the setting information I (refer to the candidate adoption threshold setting column 350 in FIG. 7) (step S710). If it is determined that the added value satisfies the extraction condition for adopting the heading word candidate set in the setting information I (step S710: YES), the process proceeds to step S709. If it is determined that the added value does not satisfy the extraction condition for adopting the heading word candidate set in the setting information I (step S710: NO), the process proceeds to step S711.

[0102] In the row of the information 830 in FIG. 9, as an example of the high contribution value word group Ai in step S710 (when the word d does not have a part d′ that is not included in the high contribution value word group A), "sound" is shown as an example of the high contribution value word group Ai, "phonetic characters" is shown as an example of the word d, and "characters" is shown as an example of the word group d′.

[0103] Here, instead of simply adding the first contribution value or a statistical value calculated from the first contribution value (such as the average value or standard deviation of the first contribution value) for the addition, the information processing system 100A changes the division of the word groups Pi1, Pi2 to Pim from which the first contribution value is calculated in the text piece Ti, and sends a command to the text analysis unit 111 to re - execute the calculation of the first contribution value for the word group corresponding to the word d, and may make a determination about the value.

[0104] Returning to FIG. 8, next, the information processing system 100A determines whether the processing for all the high contribution value word groups Ai has been completed (step S711).

[0105] If it is determined that the processing for all the high contribution value word groups Ai has not been completed, the information processing system 100A returns to step S702 and continues the processing.

[0106] When it is determined that the processing for all high contribution value phrases Ai has been completed, the information processing system 100A transmits the information registered in the created heading word candidate list 611A to the heading word candidate presentation unit 112 and ends the processing (step S712). Note that the heading word candidate for the phrase set and its second contribution value are transmitted from the text analysis unit 111 to the heading word candidate presentation unit 112.

[0107] The heading word candidate presentation unit 112 (information processing system 100A) presents the heading word candidates to the user based on the information registered in the heading word candidate list 611A. The heading word candidate presentation unit 112 (information processing system 100A) associates information indicating that the word is not included in the heading word dictionary D with the heading word candidates in the heading word candidate list for which such information is associated, and presents it to the user (for example, the new dictionary addition candidate display button 1001 in FIG. 10).

[0108] FIG. 10 is an explanatory diagram showing an example of a heading word candidate display screen according to Example 2 when the heading word candidates, the heading word candidates for the phrase sets, and their second contribution values are displayed on the display of the computer 100-1 or the terminal 101.

[0109] In Example 2, the heading word candidate display screen 500 can include a new dictionary addition candidate display button 1001 depicted as "New!" in addition to the heading word candidate display column 501 and the combination score display column 502.

[0110] The new dictionary addition candidate display button 1001 is displayed for words (words considered to be) that are not included in the heading word dictionary D among the heading word candidates displayed in the heading word candidate display column 501. Also, when the user presses the new dictionary addition candidate display button 1001, it is possible to send a command to add and save the word to the heading word dictionary D. Thereby, it is possible to easily register the heading word candidates not registered in the heading word dictionary D in the heading word dictionary D.

[0111] Thus, according to Example 2, since the heading word candidates extracted as high contribution value phrases can be transformed based on the words in the externally created heading word dictionary by matching with the heading word dictionary, it is possible to provide heading words in a more intuitive and useful form that is closer to the phrases used by the user when searching.

[0112] In addition, words not included in the heading word dictionary among the high contribution value phrases A can be specified (see the new dictionary addition candidate display button 1001 in FIG. 10), and can be added to the heading word dictionary. Therefore, when there are omissions in the externally created heading word dictionary or when there are changes over time, it is possible to prompt the update of the heading word dictionary.

Example

[0113] In Example 3, an example is shown in which, for the natural language analysis model N stored in the natural language analysis model store 120 in Example 1, annotations by the user and metadata related to the operating user are acquired to create the natural language analysis model N. In Example 3, in order to mainly explain the differences from Example 1, the same components as in Example 1 or Example 2 are denoted by the same reference numerals, and the description thereof is omitted.

[0114] FIG. 11A is a block diagram of the information processing system 100B according to Example 3. FIG. 11B is a block diagram showing the hardware configuration of the information processing system 100B. Referring to FIG. 11A, in the information processing system 100B according to Example 3, the computer 100-2 includes, in addition to the natural language analysis model store 120 and the explainable AI program store 130, a temporary storage text data store (learning database) 1110 and a natural language analysis model learning unit 1120.

[0115] The temporary storage text data store 1110 stores a so-called teacher data when learning the natural language analysis model in a sufficient amount for learning the natural language analysis model, but it is not always necessary to store a sufficient amount.

[0116] Referring to FIG. 11B, in the information processing system 100B, the computer 100-1 stores the text analysis program 111A and the heading word candidate presentation program 112A in the auxiliary storage device 203. Also, the computer 100-2 stores in the auxiliary storage device 203, in addition to the natural language analysis model store 120 and the explainable AI program store 130, the temporary storage text data store 1110 and the natural language analysis model learning program 1120A. The function of the natural language analysis model learning unit 1120 in the computer 100-2 is realized by the processor 201 reading out the natural language analysis model learning program 1120A stored in the auxiliary storage device 203 into the main storage device 202 and executing it.

[0117] The user operates the terminal 101 to input information on the location where the input text T' (learning text) is stored, user information L', and a storage instruction to store the input text T' in the temporary storage text data store 1110, and causes the terminal 101 to transmit these from the terminal 101 to the computer 100-1. The input text T' is natural language data used as learning data when learning the natural language analysis model. Also, the user information L' may not be transmitted. As will be described later, the user information L' is processed by the natural language analysis model learning unit 1120 into annotation data (correct answer data). As the user information L', for example, the following two types of information can be cited. 1. Annotation information regarding the input text T' input by the user operating the terminal 101. 2. Information about the creator of the input text T' (for example, information regarding job type and position).

[0118] For example, a user who operates the terminal 101 creates an input text T′ and transmits (provides) the input text T′ to the information processing system 100B. Further, the information processing system 100B may access the terminal 101 and obtain information regarding the occupation and position of the user (the creator of the input text T′) from the operation history and login information of the user's terminal 101, and create user information L′. In this case, the user information L′ is information about the creator of the input text T′ and is also information about the provider of the input text T′ (learning text) to the temporary storage text data store (learning database) 1110. Further, in this case, even when the user cannot input annotation information, the information processing system 100B creates the user information L′.

[0119] FIG. 12 is a flowchart showing an example of natural language analysis model learning by the natural language analysis model learning unit 1120 according to the third embodiment.

[0120] The information processing system 100B acquires an input text T′ specified by the user operating the terminal 101 (step S1201).

[0121] Next, the information processing system 100B acquires user information L′ regarding the user who operated the terminal 101 in step S1201 (step S1202). This user information L′ may be annotation information regarding the input text T′ clearly input by the user operating the terminal 101. Further, this user information L′ may be information regarding the occupation and position of the user operating the terminal 101, acquired by the information processing system 100B from the operation history and login information of the terminal 101.

[0122] Next, the information processing system 100B stores the text T′ and the user information L′ in the temporary storage text data store 1110 (step S1203). Here, the natural language analysis model learning unit 1120 can perform the necessary processing for using the input text T′ as input data and the user information L′ as annotation data when creating a natural language analysis model by so-called supervised learning, and then store it in the temporary storage text data store 1110.

[0123] Next, the information processing system 100B performs supervised learning using the input text T′ as input data and the user information L′ as annotation data (labels) to create a natural language analysis model (step S1204). Here, the supervised learning trains the natural language data analysis model to infer the user information L′. The supervised learning at this time can use a generally known method for creating a natural language analysis model. In particular, so-called transfer learning can be performed on a pre-trained language model created externally to create a natural language analysis model. Also, when the amount of learning text stored in the temporary storage text data store 1110 is not sufficient, supervised learning may be started when a predetermined amount of text T′ (learning text) is stored in the temporary storage text data store 1110 or when a command is received from the user.

[0124] Next, the information processing system 100B stores the created natural language analysis model in the natural language analysis model store 120 and ends the process.

[0125] As described above, according to the third embodiment, even when a natural language analysis model suitable for a task of referring to global (many parts) semantic information of text is not prepared, it is possible to accept annotation addition from the user and create a natural language analysis model suitable for extracting candidate headings.

[0126] Furthermore, even when the user cannot explicitly input annotation information, learning data for a task of referring to global semantic information for inferring the occupation or position of the user who created the text from the text can be constructed, and a natural language analysis model suitable for extracting candidate headwords can be created.

[0127] Note that the present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Also, the configuration of another embodiment may be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with another configuration may be made.

[0128] FIG. 13 is an overall configuration diagram according to Embodiment 2 and Embodiment 3. As shown in FIG. 13, the configurations of all the embodiments may be added.

[0129] In addition, each of the above-described configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them, for example, by an integrated circuit, or may be realized in software by a processor interpreting and executing a program for realizing each function.

[0130] Information such as programs, tables, and files for realizing each function can be stored in a storage device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0131] Also, the control lines and information lines show those considered necessary for explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it may be considered that almost all the configurations are interconnected.

Explanation of Symbols

[0132] 100, 100A, 100B: Information Processing System 101: Terminal 102: Communication Network 111: Text Analysis Unit 112: Heading Word Candidate Presentation Unit 120: Natural Language Analysis Model Store 130: Explainable AI Program Store 201: Processor 202: Main Memory Device 203: Auxiliary Memory Device 204: Network Interface 205: Input Device 206: Output Device 610: Dictionary Matching Unit 620: Heading Word Dictionary Store 1110: Temporary Storage Text Data Store 1120: Natural Language Analysis Model Learning Unit

Claims

1. An information processing system having a processor that executes a program and a storage device that stores the program, wherein the processor in the case where a predetermined natural language data analysis model is a natural language data analysis model that uses the processed natural language data as input data, performs a conversion process of converting the natural language data into a query that is the processed natural language data, an analysis process of analyzing the query converted by the conversion process by the natural language data analysis model and outputting an analysis result, a first calculation process of calculating a first contribution value representing the degree of contribution to the analysis result output by the analysis process for words in the natural language data, An information processing system characterized by executing an acquisition process of acquiring candidate heading words that are candidates to be assigned to the natural language data based on the first contribution value calculated by the first calculation process in the previous term.

2. The information processing system according to claim 1, wherein the processor executes a first output process of outputting and presenting the candidate heading words acquired by the acquisition process. An information processing system characterized by this.

3. The information processing system according to claim 1, In the conversion process, the processor rearranges the order of some words or sentences in the natural language data and converts it into a query. An information processing system characterized by this.

4. The information processing system according to claim 1, a second calculation process of calculating a set of word groups obtained by grouping a plurality of word groups in the natural language data into one set and a second contribution value representing the degree of contribution due to the interaction of the plurality of words constituting the word group to the analysis result output from the analysis process, An information processing system characterized by executing a second output process of outputting and presenting a candidate phrase group heading word that is a candidate to be assigned to the natural language data and its second contribution value based on the second contribution value calculated by the second calculation process.

5. The information processing system according to claim 1, is able to access heading word dictionary data including one or more heading words that can be assigned to the natural language data, The processor refers to the heading word dictionary data, extracts words similar to the candidate heading words acquired in the acquisition process from the heading word dictionary data, and executes a processing process of processing the candidate heading words based on the similar words. An information processing system characterized by this.

6. The information processing system according to claim 1, being able to access heading word dictionary data including one or more heading words that can be assigned to the natural language data, wherein the processor refers to the heading word dictionary data, extracts a word similar to the heading word candidate obtained in the acquisition process from the heading word dictionary data, determines whether the heading word candidate and the similar word satisfy a new word adoption criterion, and when it is determined that the heading word candidate and the similar word satisfy the new word adoption criterion, outputs and presents a message indicating that the heading word candidate is not included in the heading word dictionary data in association with the heading word candidate, and executes a third output process. An information processing system characterized by the above.

7. The information processing system according to claim 1, being able to access a learning database that stores learning data, wherein the processor performs information acquisition processing for acquiring learning text and information on the creator of the learning text or information on the provider of the learning text to the learning database, and executes a training process for training a natural language data analysis model to infer information on the creator or provider obtained by the information acquisition process using the learning text as input data. An information processing system characterized by the above.

8. The information processing system according to claim 1, wherein the first calculation process extracts a plurality of words from the query, creates a replacement query in which some of the words in the query are replaced with predetermined replacement words, obtains a replacement analysis result obtained by analyzing the replacement query with the natural language data analysis model, and based on the obtained replacement analysis result, calculates the first contribution value for the words in the query. An information processing system characterized by using a predetermined explainable AI (Artificial Intelligence) technology capable of performing a first contribution value calculation process.

9. An information processing method executed by an information processing system having a processor that executes a program and a storage device that stores the program, wherein the processor in the case of a natural language data analysis model that uses processed natural language data as input data, performs a conversion process of converting natural language data into a query that is the processed natural language data, and performs an analysis process of analyzing the query converted by the conversion process by the natural language data analysis model and outputting an analysis result. A first calculation process for calculating a first contribution value representing the degree of contribution to the analysis result output by the analysis process for each word in the natural language data; An information processing method characterized by executing an acquisition process for acquiring candidate heading words that are candidates to be assigned to the natural language data based on the first contribution value calculated by the first calculation process in the previous stage. **Claim 10** In a processor that executes a program, When the natural language data is a natural language data analysis model that uses the processed natural language data as input data, a conversion process for converting the natural language data into a query that is the processed natural language data, An analysis process for analyzing the query converted by the conversion process by the natural language data analysis model and outputting an analysis result; A first calculation process for calculating a first contribution value representing the degree of contribution to the analysis result output by the analysis process for each word in the natural language data; An acquisition process for acquiring candidate heading words that are candidates to be assigned to the natural language data based on the first contribution value calculated by the first calculation process in the previous stage; An information processing program for causing the above to be executed.

Images