Information processing apparatus, system, method, computer program product, and storage medium

Abstract

The invention relates to an information processing apparatus, an information processing system, an information processing method, a computer program product, and a storage medium, and aims to evaluate the unnatural degree of text data at a high speed. This information processing device is provided with: a reference distribution acquisition unit for acquiring a reference distribution indicating the tendency of prescribed language information; an object distribution generation unit for generating an object distribution of the language information extracted from the text data; and an evaluation value calculation unit for calculating an evaluation value indicating a difference between the reference distribution and the object distribution.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, a computer program product, and a storage medium. Background Art

[0002] The technology of building language models using neural networks is widely known. This technology can perform self-learning using only unannotated text data as learning data. In this case, the more learning data there is, the higher the performance of the language model.

[0003] For example, in the huge amount of text data represented by texts on the Internet, there are many unnatural articles that are different from the articles that humans usually write or say. If unnatural articles are used as learning data, the performance of the language model will be reduced, so it is effective to remove unnatural articles from the learning data.

[0004] For example, Patent Document 1 discloses an article evaluation system for evaluating the content of an article represented by a character string. The article evaluation system not only divides the character string in the evaluation object data into a plurality of continuous morphemes, but also associates each morpheme with its own word class, and classifies the article based on the frequency of occurrence of attributes corresponding to the pattern of continuous word classes and the coefficient assigned to each attribute.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2008-269430

[0006] However, the prior art has the problem that it takes a long time to evaluate the unnaturalness of text data. For example, in order to accurately detect the unnaturalness of text data, the previous cleaning method needs to use a complex process such as a dedicated judgment model. Summary of the invention

[0007] In view of the above technical problems, the present invention provides an embodiment, which aims to evaluate the unnaturalness of text data at high speed.

[0008] The present invention provides an information processing device according to an embodiment, characterized in that it comprises: a baseline distribution acquisition unit for acquiring a baseline distribution representing a tendency of specified language information; an object distribution generation unit for generating an object distribution of the language information extracted from text data; and an evaluation value calculation unit for calculating an evaluation value representing a difference between the baseline distribution and the object distribution.

[0009] An effect of one embodiment of the present invention is to evaluate the unnaturalness of text data at high speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1It is a schematic diagram showing an example of the overall structure of the information processing system according to the first embodiment.

[0011] Figure 2 This is an example of a computer hardware block diagram.

[0012] Figure 3 This is a functional block diagram of an example of the information processing system according to the first embodiment.

[0013] Figure 4 This is an example of a baseline distribution graph.

[0014] Figure 5 This is an example of a natural article object distribution map.

[0015] Figure 6 This is an example of an object distribution graph for unnatural articles.

[0016] Figure 7 This is an example of a graph showing the deviation of natural articles.

[0017] Figure 8 This is an example of a graph showing the deviation of unnatural articles.

[0018] Fig. 9 This is a flowchart of an example of the information processing method according to the first embodiment.

[0019] Fig.10 This is a schematic diagram of an example of a result screen in the first embodiment.

[0020] Fig.11 This is a functional block diagram of an example of the information processing system of Modification Example 1.

[0021] Fig.12 This is a functional structure block diagram of an example of an information processing system according to variation 2.

[0022] Fig.13 This is a functional block diagram of an example of the information processing system according to the second embodiment.

[0023] Fig.14 This is a flowchart of an example of the information processing method in the second embodiment.

[0024] Fig.15 This is a schematic diagram of an example of a result screen in the second embodiment.

[0025] Fig.16 This is a functional block diagram of an example of the information processing system according to the third embodiment.

[0026] Fig.17 This is a flowchart of an example of the information processing method according to the third embodiment.

[0027] Fig.18This is a schematic diagram of an example of a result screen in the third embodiment.

[0028] Explanation of symbols

[0029] 10 Information processing device

[0030] 101 Benchmark distribution storage unit

[0031] 102 Text acquisition unit

[0032] 103 Language Information Extraction Department

[0033] 104 Object distribution generation unit

[0034] 105 Reference distribution acquisition unit

[0035] 107 Result Output Section

[0036] 108 Text storage unit

[0037] 109 Reference distribution generation unit

[0038] 110 Text Deletion Department

[0039] 111 Text Segmentation

[0040] 20 Terminal Devices

[0041] 201 Text storage unit

[0042] 202 Text sending department

[0043] 203 Display control unit

[0044] 1000 Information Processing Systems DETAILED DESCRIPTION

[0045] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the drawings, components having the same functions are given the same reference numerals, and repeated descriptions are omitted.

[0046] [First embodiment]

[0047] The first embodiment of the present invention is an information processing system for evaluating the unnaturalness of text data. The information processing system of the first embodiment has the function of outputting a score (hereinafter also referred to as an "unnaturalness score") representing the unnaturalness of text data as an evaluation object (hereinafter also referred to as "evaluation object text").

[0048] In this embodiment, unnatural articles are all articles except natural articles. Natural articles are articles spoken or written by humans to convey information. For example, unnatural articles include texts that list the names of prefectures in Japan, such as "Hokkaido, Aomori Prefecture, Akita Prefecture, Iwate Prefecture, Yamagata Prefecture, Miyagi Prefecture, Fukushima Prefecture, ... Okinawa Prefecture". Another example of unnatural articles is texts that repeat specific phrases, such as "Can't run away, can't run away, can't run away...can't run away". In such articles, it can be seen that the frequency of occurrence of specific words or punctuation marks is abnormally high, or features such as the absence of auxiliary words are found.

[0049] In the prior art, in order to evaluate the unnaturalness of text data, it is necessary to perform complex processing using, for example, a dedicated judgment model, which results in a long processing time. The purpose of this embodiment is to evaluate the unnaturalness of text data at high speed.

[0050] <Overall Structure of Information Processing System>

[0051] refer to Figure 1 , describing the overall structure of the information processing system of the first embodiment. Figure 1 It is a schematic diagram showing an example of the overall structure of the information processing system according to the first embodiment.

[0052] like Figure 1 As shown, the information processing system 1000 of the first embodiment includes an information processing device 10 and one or more terminal devices 20. The information processing device 10 and the terminal device 20 are respectively connected to a communication network N1. Each device connected to the communication network N1 can communicate with each other.

[0053] The communication network N1 is constructed by a wired communication network such as the Internet, LAN (Local Area Network), or WAN (Wide Area Network). In addition to wired communication, the communication network N1 also includes a network based on wireless communication such as wireless LAN, short-range wireless communication, or mobile communication such as WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), or 5G (5th Generation).

[0054] The information processing device 10 is an information processing device for evaluating an evaluation object text. The information processing device 10 receives a text set including a plurality of evaluation object texts from a terminal device 20. The information processing device 10 evaluates each evaluation object text included in the text set, and sends screen data for indicating the evaluation result of the evaluation object text to the terminal device 20.

[0055] The terminal device 20 is an information processing device operated by the user of the information processing system 1000. A text set is pre-stored in the terminal device 20. The terminal device 20 sends the text set to the information processing device 10 according to the user operation. The terminal device 20 presents the result screen to the user based on the screen data received from the information processing device 10.

[0056] The computer is an example of an information processing device 10 and an example of a terminal device 20. The terminal device 20 is not limited to a computer as long as it has a communication function. The terminal device 20 can be, for example, an image forming device (printer, FAX device, MFP (Multifunction Peripheral / Product / Printer), scanning device), PJ (Projector), IWB (Interactive White Board, a whiteboard with an electronic blackboard function that can communicate with each other), digital signage and other output devices, HUD (Head Up Display) equipment, industrial machinery, camera devices, sound collection devices, medical machinery, network appliances, automobiles (Connected Car), notebook PCs (Personal Computers), mobile phones, smart phones, tablet terminals, game consoles, PDAs (Personal Digital Assistants), digital cameras, wearable PCs or desktop PCs, etc.

[0057] Figure 1 The configuration of the information processing system 1000 is shown as an example, and other configurations are also possible. For example, the information processing device 10 can be implemented by a single information processing device, or can be a system implemented by multiple information processing devices. The information processing system 1000 includes various devices that perform at least one of input and output of electronic data, and these devices can use various services provided by the information processing device 10.

[0058] <Hardware Configuration of Information Processing System>

[0059] See also Figure 2 , describing the hardware configuration of each device included in the information processing system of the first embodiment.

[0060] Computer hardware components

[0061] As an example, the information processing device 10 and the terminal device 20 in the first embodiment may be realized by a computer. Figure 2 This is an example of a computer hardware block diagram.

[0062] like Figure 2As shown, the computer 500 includes a CPU 501, a ROM 502, a RAM 503, an HD 504, a HDD (Hard Disk Drive) controller 505, a display 506, a peripheral connection I / F (Interface) 508, a network I / F 509, a bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514 and a media I / F 516.

[0063] The CPU 501 controls the overall operation of the computer. The ROM 502 stores programs such as IPL for driving the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501.

[0064] The display 506 displays various information such as a cursor, menu, window, text or image. The peripheral connection I / F 508 is an interface for connecting various external devices. In this case, the peripheral is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the communication network N1. The bus 510 is an interface for electrically connecting Figure 2 The address bus and data bus of each component such as the CPU 501 shown.

[0065] The keyboard 511 is an input device having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 512 is an input device for selecting and executing various instructions, selecting a processing object, moving a cursor, etc. The DVD-RW drive 514 controls the reading or writing of various data in a DVD-RW 513 as an example of a removable recording medium. In addition, it is not limited to DVD-RW, and it can also be a DVD-R, etc. The medium I / F 516 controls the reading or writing (storage) of data in a recording medium 515 such as a flash memory.

[0066] <Functional structure of information processing system>

[0067] refer to Figures 3 to 8 , describing the functional structure of the information processing system of the first embodiment. Figure 3 This is a functional block diagram of an example of the information processing system according to the first embodiment.

[0068] Functional structure of information processing device

[0069] like Figure 3As shown, the information processing device 10 of the first embodiment includes a reference distribution storage unit 101 , a text acquisition unit 102 , a language information extraction unit 103 , an object distribution generation unit 104 , a reference distribution acquisition unit 105 , an evaluation value calculation unit 106 , and a result output unit 107 .

[0070] The reference distribution storage unit 101 uses, for example, Figure 2 The data stored in the HD 504 is implemented by the HDD controller 505. For example, the reading or writing of the data stored in the HD 504 is performed.

[0071] The text acquisition unit 102, the language information extraction unit 103, the object distribution generation unit 104, the reference distribution acquisition unit 105, the evaluation value calculation unit 106, and the result output unit 107 are executed by the CPU 501 from, for example, Figure 2 This is achieved by the processing of the program expanded from HD 504 to RAM 503 as shown.

[0072] The reference distribution storage unit 101 stores a reference distribution in advance. The reference distribution is a distribution indicating the tendency of predetermined language information. For example, the reference distribution is a distribution generated by extracting predetermined language information from a natural article collection collected in advance.

[0073] For example, the reference distribution is a distribution generated by extracting specified language information from a collection of a large number of articles collected in advance. In this case, the collection of articles includes unnatural articles. For example, it can be considered that most of the articles collected from the Internet are natural articles, so even if a small number of unnatural articles are included, it will not have a significant impact on the shape of the distribution.

[0074] For example, the reference distribution is information indicating the relationship between the word frequency and the word frequency rank. In this case, the specified language information is the word frequency. The word frequency refers to data that divides the text data into word units and counts the frequency of occurrence of each word unit. The word frequency rank refers to data that indicates the rank when the frequency of occurrence of each word unit is arranged in descending order.

[0075] Here, the vocabulary unit refers to a word or an arbitrary unit equivalent to a word. The vocabulary unit is a processing unit called a token in a language model. The frequency of occurrence is a numerical value that counts the number of times each vocabulary unit appears in text data.

[0076] Another example of the reference distribution is the part-of-speech distribution. In this case, the specified language information is the frequency of occurrence of each part of speech. The part-of-speech distribution refers to the distribution of the proportion of each part of speech when the article is divided into words and each word is classified according to the part of speech.

[0077] The text acquisition unit 102 acquires the evaluation target text. The text acquisition unit 102 may also acquire the evaluation target text by dividing the text set into arbitrary units. The text acquisition unit 102 may also receive the text set from the terminal device 20. The text acquisition unit 102 may also read the text set stored in the medium 515 through the medium I / F 506.

[0078] The language information extraction unit 103 extracts predetermined language information from the evaluation target text acquired by the text acquisition unit 102. When the text acquisition unit 102 acquires a plurality of evaluation target texts, the language information extraction unit 103 extracts language information from each of the plurality of evaluation target texts. The predetermined language information is the same as the language information used when generating the reference distribution.

[0079] The object distribution generating unit 104 generates an object distribution for each evaluation object text based on the language information extracted by the language information extracting unit 103. The object distribution is the same distribution as the reference distribution. That is, an example of the object distribution is information indicating the relationship between the word frequency and the word frequency rank. Another example of the object distribution is the part-of-speech distribution.

[0080] The reference distribution acquisition unit 105 acquires the reference distribution by reading the reference distribution stored in the reference distribution storage unit 101 .

[0081] The evaluation value calculation unit 106 calculates the evaluation value of the evaluation object document based on the object distribution generated by the object distribution generation unit 104 and the reference distribution obtained by the reference distribution acquisition unit 105. The evaluation value is, for example, the deviation between the reference distribution and the object distribution. The deviation is a value indicating the difference between the reference distribution and the object distribution.

[0082] When the reference distribution and the target distribution are information indicating the relationship between the word frequency and the word frequency rank, the evaluation value calculation unit 106 may also perform standardization processing on the target distribution and the reference distribution, respectively, calculate the difference between the standardized target distribution and the standardized reference distribution, and use the difference as the evaluation value. The difference may be, for example, the sum of the absolute values ​​of the differences in the frequency of occurrence of each level, or the sum of the squares of the differences in the frequency of occurrence of each level.

[0083] The result output unit 107 sends screen data for displaying the result screen to the terminal device 20. The result screen is a screen that displays the evaluation result of the target document. The screen data is screen data written in, for example, HTML (Hyper Text Markup Language) and may include an application program written in JavaScript (registered trademark) or the like.

[0084] Functional composition of terminal device

[0085] like Figure 3 As shown, the terminal device 20 of the first embodiment includes a text storage unit 201 , a text transmission unit 202 , and a display control unit 203 .

[0086] The text storage unit 201 uses, for example, Figure 2 The data stored in the HD 504 is realized by the HDD controller 505, for example.

[0087] The text transmission unit 202 and the display control unit 203 execute the following operations by, for example, causing the CPU 501 to execute Figure 2 This is achieved by the processing of the program expanded from HD 504 to RAM 503 as shown.

[0088] The text storage unit 201 stores a text collection in advance. The text collection may be information collected in advance from the Internet or the like through electronic processing, or may be information electronically processed using optical character recognition (OCR) or the like on a printed document. The text collection may also select text data that meets specific conditions according to the type of task implemented using the language model. The specific conditions refer to, for example, articles containing specified words or articles on websites belonging to a specified category.

[0089] The text transmission unit 202 transmits the text set read from the text storage unit 201 to the information processing device 10 in response to the user's operation.

[0090] The display control unit 203 displays a screen to be presented to the user on the display 506 based on the screen data received from the information processing device 10. The screens displayed by the display control unit 203 include a result screen showing the evaluation result of the evaluation target document.

[0091] (Benchmark Distribution)

[0092] Figure 4 is a diagram of an example benchmark distribution. Figure 4 The graph shown is a distribution of word frequencies in descending order, where the vertical axis is the logarithm of the word frequency and the horizontal axis is the logarithm of the word frequency rank.

[0093] In a sufficiently large text dataset, the frequency of a word is inversely proportional to the order in which it appears (Zipf's law). Figure 4 As shown, if we plot the word frequency and word frequency rank on a double logarithmic graph, we get a straight line that descends to the right.

[0094] (Object Distribution)

[0095] Figure 5 This is a diagram of the object distribution of an example of a natural article. Figure 5 The diagram shown is similar to Figure 4 The same is the distribution of word frequencies in descending order, where the vertical axis is the logarithm of the word frequency and the horizontal axis is the logarithm of the word frequency rank.

[0096] The distribution of objects extracted from natural articles about word frequencies has the same shape as the benchmark distribution. Figure 5 As shown, the object distribution of natural articles is the same as the benchmark distribution, which is a straight line descending to the right.

[0097] Figure 6 This is a diagram of the object distribution of an example of unnatural articles. Figure 5 The diagram shown is similar to Figure 4 The same is the distribution of word frequencies in descending order, where the vertical axis is the logarithm of the word frequency and the horizontal axis is the logarithm of the word frequency rank.

[0098] The more unnatural the distribution of word frequencies extracted from unnatural articles is, the greater the deviation from the baseline distribution. Figure 6 As shown, the object distribution of the unnatural article has deformed shapes and is not a straight line.

[0099] (Degree of deviation)

[0100] Figure 7 This is a diagram of the deviation of a natural article. Specifically, Figure 7 Yes Figure 4 The benchmark distribution shown and Figure 5 The distribution of objects shown together form a realistic diagram. Figure 7 In order to make the scale of the benchmark distribution and the target distribution consistent, the distributions of both are standardized.

[0101] Figure 7 The shaded area in the figure is the range of the difference between the reference distribution and the target distribution. The deviation can be the area of ​​the shaded area or the sum of the squares of the differences in the frequency rank of each word. Figure 7 As shown, regarding the deviation of natural articles, the area of ​​the mesh part is relatively small, and it can be seen that it is a very small value.

[0102] Figure 8 This is a diagram of the deviation of an example of unnatural articles. Specifically, Figure 8 Yes Figure 4 The benchmark distribution shown and Figure 6 The distribution of objects shown is combined to show the graph. Figure 8 Zhongyu Figure 7 The distribution of the two is standardized.

[0103] Figure 8The shaded area in the middle is the range of differences between the benchmark distribution and the target distribution. Figure 8 As shown, regarding the deviation of unnatural articles, the shaded area is relatively large and the value is also large.

[0104] <Processing order of information processing method>

[0105] refer to Fig. 9 An information processing method executed by the information processing system according to the first embodiment will be described. Fig. 9 This is a flowchart of an example of the information processing method according to the first embodiment.

[0106] In step S1, the user of the information processing system 1000 performs an evaluation operation of text data. The operation of evaluating text data may be, for example, an operation of selecting a text data evaluation menu on a menu screen displayed on the display 506 of the terminal device 20. Then, the information processing system 100 receives the evaluation operation and sends the operation to the terminal device 20.

[0107] When receiving the operation of evaluating the text data, the text transmission unit 202 of the terminal device 20 reads the text set from the text storage unit 201 according to the operation. Then, the text transmission unit 202 transmits the read text set to the information processing device 10.

[0108] In step S2, the text acquisition unit 102 of the information processing device 10 receives a text set from the terminal device 20. Next, the text acquisition unit 102 acquires one or more evaluation target texts from the received text set. The text acquisition unit 102 may acquire all evaluation target texts contained in the text set, or may acquire a portion of the evaluation target texts. Then, the text acquisition unit 102 sends the acquired evaluation target texts to the language information extraction unit 103.

[0109] In step S3, the language information extraction unit 103 of the information processing device 10 receives the evaluation object text from the text acquisition unit 102. Next, the language information extraction unit 103 extracts predetermined language information from the evaluation object text. Then, the language information extraction unit 103 sends the language information extracted from the evaluation object text to the object distribution generation unit 104.

[0110] Specifically, the language information extraction unit 103 extracts language information as follows. As an example, the vocabulary frequency is extracted here. First, the language information extraction unit 103 divides the evaluation object text into vocabulary units. Then, the language information extraction unit 103 calculates the frequency of occurrence of each vocabulary unit in the evaluation object text. When the text acquisition unit 102 acquires multiple evaluation object texts, the language information extraction unit 103 extracts the vocabulary frequency of each of the multiple evaluation object texts.

[0111] In step S4, the object distribution generating unit 104 of the information processing device 10 receives the language information of the evaluation object text from the language information extracting unit 103. Next, the object distribution generating unit 104 generates the object distribution about the evaluation object text based on the language information of the evaluation object text. Specifically, the object distribution generating unit 104 arranges the frequency of occurrence of each vocabulary unit extracted in step S3 in descending order. Then, the object distribution generating unit 104 sends the object distribution about the evaluation object text to the evaluation value calculating unit 106.

[0112] In step S5, the reference distribution acquisition unit 105 of the information processing device 10 reads the reference distribution from the reference distribution storage unit 101. Next, the reference distribution acquisition unit 105 sends the read reference distribution to the evaluation value calculation unit 106.

[0113] In step S6, the evaluation value calculation unit 106 of the information processing device 10 receives the object distribution of the evaluation object text from the object distribution generation unit 104. The evaluation value calculation unit 106 also receives the reference distribution from the reference distribution acquisition unit 105. Next, the evaluation value calculation unit 106 calculates the evaluation value of the evaluation object text based on the reference distribution and the object distribution. Then, the evaluation value calculation unit 106 sends the evaluation value of the evaluation object text to the result output unit 107.

[0114] Specifically, the evaluation value calculation unit 106 calculates the evaluation value of the evaluation object text as follows. First, the evaluation value calculation unit 106 normalizes the reference distribution, and at the same time, the evaluation value calculation unit 106 normalizes the object distribution. Next, the evaluation value calculation unit 106 calculates the difference between the normalized object distribution and the normalized reference distribution. Thus, the evaluation value calculation unit 106 obtains the evaluation value of the evaluation object text.

[0115] In step S7 , the result output unit 107 of the information processing device 10 receives the evaluation value of the evaluation object document from the evaluation value calculation unit 106 . Next, the result output unit 107 generates screen data for displaying the result screen and transmits the generated screen data to the terminal device 20 .

[0116] When generating screen data, the result output unit 107 embeds the evaluation result of the evaluation object document into the screen data. The result output unit 107 can embed the evaluation value itself as the evaluation result into the screen data, or embed the score after processing the evaluation value as the evaluation result into the screen data.

[0117] In step S8, the display control unit 203 of the terminal device 20 receives screen data for displaying the result screen from the information processing device 10. Next, the display control unit 203 displays the result screen on the display 506 based on the received screen data.

[0118] Results screen

[0119] See also Fig.10 , illustrating the result screen of the first embodiment. Fig.10 This is a schematic diagram of an example of a result screen in the first embodiment.

[0120] like Fig.10 As shown, the result screen 600 has a text display column 601 and a score display column 602 .

[0121] Information indicating the evaluation target text is displayed in the text display field 601. The information indicating the evaluation target text may be, for example, a file name, a file path, a serial number of the text, a predetermined character at the beginning of the text, or the like.

[0122] The evaluation result of each evaluation target document is displayed in the score display column 602. The evaluation result may be the evaluation value itself or a score after the evaluation value is processed. Fig.10 The unnaturalness score is classified into four levels (0 to 3 black stars) and is displayed in the score display field 602. As an example of the unnaturalness score, the greater the number of black stars, the more unnatural the article is.

[0123] The score is not limited to the above content, and the numerical range and display method of the score can be arbitrarily designed. For example, the size of the unnaturalness score can be reversed, and the larger the display value, the more natural the article is.

[0124] <Effects of the First Embodiment>

[0125] The information processing device 10 of this embodiment calculates an evaluation value of the difference between a reference distribution representing a prescribed language information tendency and an object distribution of language information extracted from text data. The information processing device 10 can evaluate text data by easily calculating the distribution of contrasting language information. From one aspect, the effect of this embodiment is that the unnaturalness of text data can be evaluated at high speed.

[0126] The language information of this embodiment can also be information representing the relationship between the frequency of occurrence of each word and the frequency of occurrence rank. As is well known, the frequency of occurrence of each word is inversely proportional to the frequency of occurrence rank. The effect of this embodiment is that the unnaturalness of text data can be evaluated with high accuracy.

[0127] The information processing device 10 of this embodiment can also calculate the difference between the object distribution after standardization and the reference distribution after standard post-processing, and use the difference as the evaluation value. The consistency of the scale of the object distribution and the reference distribution can control the difference within a small range. The effect of this embodiment is that the unnaturalness of text data can be evaluated at high speed and high precision.

[0128] The information processing device 10 of this embodiment can also obtain a set of text data and calculate evaluation values ​​for each text data included in the set. This embodiment has the effect that even if there are many evaluation target texts, the unnaturalness of each evaluation target text can be evaluated at high speed.

[0129] <Modification 1>

[0130] The first embodiment describes the evaluation of text data using a pre-prepared reference distribution. In contrast, Modification 1 will describe the generation of a reference distribution based on a pre-prepared reference text.

[0131] The information processing system of Modification 1 will be described below, focusing on the differences between Modification 1 and the first embodiment.

[0132] <Functional structure of information processing system>

[0133] refer to Fig.11 , describing an example functional structure of an information processing system in variant example 1. Fig.11 This is a functional block diagram of an example of the information processing system of Modification Example 1.

[0134] Functional structure of information processing device

[0135] like Fig.11 As shown, the information processing device 10 of the modification example 1 includes a text acquisition unit 102, a language information extraction unit 103, an object distribution generation unit 104, an evaluation value calculation unit 106, a result output unit 107, a text storage unit 108, and a reference distribution generation unit 109. That is, the information processing device 10 of the modification example 1 is different from the first embodiment in that the former does not include the reference distribution storage unit 101 and the reference distribution acquisition unit 105, but includes the text storage unit 108 and the reference distribution generation unit 109.

[0136] The text storage unit 108 stores text data (hereinafter also referred to as "reference text") for generating a reference distribution. The reference text is a collection of a large number of articles collected in advance. The reference text may include data collected electronically in advance from the Internet, etc., or data obtained by electronicizing a printed document using optical text recognition, etc.

[0137] The reference distribution generating unit 109 generates a reference distribution based on the reference text read from the text storage unit 108. Specifically, the reference distribution generating unit 109 generates the reference distribution as follows. First, the reference distribution generating unit 109 divides the reference text into vocabulary units. Then, the reference distribution generating unit 109 calculates the appearance frequency of each vocabulary unit in the reference text. Next, the reference distribution generating unit 109 arranges the appearance frequency of each vocabulary unit in descending order.

[0138] The evaluation value calculation unit 106 in this embodiment calculates the evaluation value of the evaluation target document based on the target distribution generated by the target distribution generation unit 104 and the reference distribution generated by the reference distribution generation unit 109. The evaluation value and its calculation method are the same as those in the first embodiment.

[0139] <Effects of Modification 1>

[0140] The information processing device 10 of this modification generates a distribution of language information extracted from a reference text and uses the distribution as a reference distribution. This modification has the advantage that the degree of unnaturalness of text data can be evaluated using a reference distribution generated based on an arbitrary reference text.

[0141] <Modification 2>

[0142] Modification 1 describes the generation of a reference distribution based on a previously prepared reference text. Modification 2 describes the generation of a reference distribution using a text set to be evaluated as a reference text.

[0143] The following describes the information processing system of Modification 2, focusing on the differences between Modification 2 and Modification 1.

[0144] <Functional structure of information processing system>

[0145] refer to Fig.12 An example functional configuration of the information processing system according to Modification 2 will be described. Fig.12 This is a functional structure block diagram of an example of an information processing system according to variation 2.

[0146] like Fig.12 As shown, the information processing device 10 of the modification example 2 includes a text acquisition unit 102, a language information extraction unit 103, an object distribution generation unit 104, an evaluation value calculation unit 106, a result output unit 107, and a reference distribution generation unit 109. That is, the information processing device 10 of the modification example 2 is different from the modification example 1 in that the text storage unit 108 is not included.

[0147] The reference distribution generating unit 109 in this embodiment generates reference distributions of a plurality of evaluation target texts based on the language information extracted by the language information extracting unit 103. In other words, the reference distribution generating unit 109 generates reference distributions by taking the text set of the evaluation target texts as reference texts. If the language information is the vocabulary frequency, the reference distribution generating unit 109 sums up the frequency of occurrence of each vocabulary unit extracted from each evaluation target text, and arranges the sum of the frequency of occurrence of each vocabulary unit in descending order.

[0148] <Effects of Modification 2>

[0149] The information processing device 10 of this modification generates a distribution of language information extracted from a set of text data as a reference distribution. This modification has the effect that text data for generating a reference distribution does not need to be collected separately from text data to be evaluated.

[0150] [Second embodiment]

[0151] The first embodiment has been described in which an unnatural degree score is outputted to evaluate the unnatural degree of text data as an evaluation object. The second embodiment will be described in which an unnatural article is deleted from a collection of text data according to the evaluation result of the unnatural degree of the text data. In other words, the information processing system implementing the clearing process to the text data collection is described in the second embodiment.

[0152] The information processing system of the second embodiment will be described below focusing on the differences from the first embodiment.

[0153] refer to Fig.13 , describing an example functional structure of the information processing system of the second embodiment. Fig.13 This is a functional block diagram of an example of the information processing system according to the second embodiment.

[0154] Functional structure of information processing device

[0155] like Fig.13 As shown, the information processing device 10 of the second embodiment includes a reference distribution storage unit 101, a text acquisition unit 102, a language information extraction unit 103, an object distribution generation unit 104, a reference distribution acquisition unit 105, an evaluation value calculation unit 106, a result output unit 107, and a text deletion unit 110. That is, the information processing device 10 of the second embodiment is different from the first embodiment in that the former further includes the text deletion unit 110.

[0156] The text deletion unit 110 deletes a portion of the evaluation target texts from the plurality of evaluation target texts acquired by the text acquisition unit 102 based on the evaluation values ​​obtained by the evaluation value calculation unit 106. The text deletion unit 110 outputs information indicating whether to delete the evaluation target text (hereinafter also referred to as "deletion flag") for each evaluation target text.

[0157] <Processing flow of information processing method>

[0158] refer to Fig.14 An information processing method executed by the information processing system according to the second embodiment will be described. Fig.14 This is a flowchart of an example of the information processing method according to the second embodiment.

[0159] The processing of steps S11 to S16 is similar to the processing of steps S1 to S6 in the first embodiment (see Fig. 9However, in step S16, the evaluation value calculation unit 106 sends the evaluation value of the evaluation object text to the result output unit 107 and the text deletion unit 110.

[0160] In step S17, the text deletion unit 110 of the information processing device 10 receives the evaluation value of the evaluation target text from the evaluation value calculation unit 106. Next, the text deletion unit 110 deletes a portion of the evaluation target text from the plurality of evaluation target texts acquired in step S12 based on the evaluation value of the evaluation target text. Then, the text deletion unit 110 sends a deletion flag indicating whether to delete each evaluation target text to the result output unit 107.

[0161] As an example, the text deleting unit 110 may compare the evaluation value of each evaluation target text with a predetermined threshold value and delete the evaluation target text having an evaluation value greater than the threshold value. The criteria for deleting the evaluation target text are not limited to the above, and the evaluation target text may be deleted using any criteria.

[0162] For example, the text deletion unit 110 can compare the unnaturalness score after the evaluation value is processed with a specified threshold value, and delete the evaluation object text whose unnaturalness score is above the threshold value. For another example, the text deletion unit 110 can also arrange the evaluation object texts in descending order of the evaluation value, and delete the evaluation object texts that are away from the beginning by a specified number. For another example, the text deletion unit 110 can also arrange the evaluation object texts in descending order of the evaluation value, and delete the evaluation object texts within a specified proportion from the top.

[0163] In step S18, the result output unit 107 of the information processing device 10 receives the evaluation value of the evaluation object text from the evaluation value calculation unit 106. Then, the result output unit 107 receives the deletion flag from the text deletion unit 110. Next, the result output unit 107 generates screen data for displaying the result screen. When generating the screen data, the result output unit 107 embeds the evaluation result including the deletion flag into the screen data. Then, the result output unit 107 sends the generated screen data to the terminal device 20.

[0164] In step S19, the display control unit 203 of the terminal device 20 receives screen data for displaying the result screen from the information processing device 10. Next, the display control unit 203 deletes a portion of the comparison target text from the text set stored in the text storage unit 201 according to the deletion flag included in the screen data. Then, the display control unit 203 displays the result screen on the display 506 according to the received screen data.

[0165] Results screen

[0166] refer to Fig.15 The result screen of the second embodiment will be described. Fig.15This is a schematic diagram of an example of a result screen in the second embodiment.

[0167] like Fig.15 As shown, the result screen 600 of the second embodiment includes a text display column 601, a score display column 602, and a logo display column 603. That is, the result screen 600 of the second embodiment differs from the first embodiment in that the logo display column 603 is further included.

[0168] In the flag display column 603 , it is displayed whether each evaluation object text is to be deleted. Fig.15 The mark display bar 603 shown only indicates that the evaluation object text is deleted, but the display method of the deletion mark is not limited to this. In addition, for example, the mark display bar 603 may not be set on the result screen 600, but the evaluation object text to be deleted may be grayed out. For another example, the mark display bar 603 may not be set on the result screen 600, but the evaluation object text to be deleted may be set to non-display (i.e., only the evaluation object text not to be deleted is displayed on the result screen 600).

[0169] <Effects of the Second Embodiment>

[0170] The information processing device 10 of this embodiment deletes text data with high evaluation values ​​from a set of text data. This has the effect of being able to perform a cleaning process to delete unnatural text data at high speed.

[0171] [Third Embodiment]

[0172] The first embodiment describes outputting an unnaturalness score for evaluating the degree of unnaturalness of text data as an evaluation target. In contrast, the third embodiment will describe dividing a text data set based on an evaluation result of evaluating the degree of unnaturalness of text data.

[0173] The information processing system of the third embodiment will be described below, focusing on the differences from the first embodiment.

[0174] <Functional Configuration of Information Processing Device>

[0175] refer to Fig.16 An example functional configuration of the information processing system according to the third embodiment will be described. Fig.16 It is a functional block diagram of the information processing system according to the third embodiment.

[0176] Functional structure of information processing device

[0177] like Fig.16As shown, the information processing device 10 of the third embodiment includes a reference distribution storage unit 101, a text acquisition unit 102, a language information extraction unit 103, an object distribution generation unit 104, a reference distribution acquisition unit 105, an evaluation value calculation unit 106, a result output unit 107, and a text segmentation unit 111. That is, the information processing device 10 of the third embodiment is different from the first embodiment in that it further includes a text segmentation unit 111.

[0178] The text segmentation unit 111 segments the plurality of evaluation target texts acquired by the text acquisition unit 102 into a plurality of partial sets according to the evaluation values ​​obtained by the evaluation value calculation unit 106. The text segmentation unit 111 outputs information indicating the partial set to which each evaluation target text belongs (hereinafter also referred to as "segmentation information") for each evaluation target text.

[0179] <Processing steps of information processing method>

[0180] refer to Fig.17 An information processing method executed by the information processing system according to the third embodiment will be described. Fig.17 This is a flowchart of an example of the information processing method according to the third embodiment.

[0181] The processing from step S21 to step S26 is similar to the processing from step S1 to step S6 in the first embodiment (see Fig. 9 However, in step S26, the evaluation value calculation unit 106 sends the evaluation value of the evaluation object text to the result output unit 107 and the text segmentation unit 111.

[0182] In step S27, the text segmentation unit 111 of the information processing device 10 receives the evaluation value of the evaluation target text from the evaluation value calculation unit 106. Next, the text segmentation unit 111 segments the plurality of evaluation target texts obtained in step S22 into a plurality of partial sets according to the evaluation values ​​of the evaluation target texts. Then, the text segmentation unit 111 sends segmentation information indicating the partial set to which each evaluation target text belongs to the result output unit 107.

[0183] As an example, the text segmentation unit 111 can compare the evaluation value of each evaluation object text with the upper limit value and the lower limit value specified for multiple partial sets, and classify the comparison object text according to the partial set whose evaluation value falls between the upper limit value and the lower limit value. The segmentation criteria of the evaluation object text are not limited to the above, and the evaluation object text can also be segmented using any criteria.

[0184] For example, the text segmentation unit 111 can process the evaluation value of each comparison object text into an unnaturalness score, and segment the comparison object text into partial sets corresponding to the unnaturalness score (for example, partial set A when the unnaturalness score = 1, partial set B when the unnaturalness score = 2, etc.). The text segmentation unit 111 can also, for example, arrange the comparison object texts in ascending or descending order of the evaluation value, and segment the comparison object texts according to the proportion from the top (for example, the top 25% is partial set A, the top 25% to 50% is partial set B, etc.).

[0185] In step S28, the result output unit 107 of the information processing device 10 receives the evaluation value of the evaluation object text from the evaluation value calculation unit 106. Next, the result output unit 107 receives the segmentation information from the text segmentation unit 111. Next, the result output unit 107 generates screen data for displaying the result screen. When generating the screen data, the result output unit 107 embeds the evaluation result including the segmentation information into the screen data. Then, the result output unit 107 sends the generated screen data to the terminal device 20.

[0186] In step S29, the display control unit 203 of the terminal device 20 receives the screen data for displaying the result screen from the information processing device 10. Next, the display control unit 203 divides the text set stored in the text storage unit 201 into partial sets based on the division information included in the screen data. Then, the display control unit 203 displays the result screen on the display 506 based on the received screen data.

[0187] Results screen

[0188] refer to Fig.18 , illustrating the result screen of the third embodiment. Fig.18 This is a schematic diagram of an example of a result screen in the third embodiment.

[0189] like Fig.18 As shown, the result screen 600 in the third embodiment includes a text display field 601, a score display field 602, and a split display field 604. That is, the result screen 600 in the third embodiment differs from the first embodiment in that the split display field 604 is further included.

[0190] Information indicating the partial set to which each evaluation target text belongs is displayed in the segmentation display field 604 . Fig.18The segmentation display bar 604 shown shows information on segmenting each evaluation target text into partial sets A to D corresponding to the unnaturalness scores. The display method of the segmentation information is not limited to this. For example, the segmentation display bar 604 may not be provided on the result screen 600, and at least one of the text color and the background color may be displayed in a color corresponding to the partial set to which the evaluation target text belongs.

[0191] <Effects of the Third Embodiment>

[0192] The information processing device 10 of this embodiment can segment the set of text data according to the evaluation value. Using the segmented set of text data, text data used for learning a language model can be selected according to the degree of unnaturalness or the amount of data. The effect of this embodiment is that it can perform processing that can flexibly select the required text data at a high speed.

[0193] [Replenish]

[0194] The functions of the above-described embodiments can be implemented by one or more processing circuits. The "processing circuit" herein includes: a processor that performs the functions through software programming, such as a processor installed by a circuit; and an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and existing circuit modules and other devices designed to perform the above-mentioned functions.

[0195] The device cluster described in the embodiments only represents one of multiple computing environments for implementing the embodiments disclosed herein. In some embodiments, the information processing device 10 includes multiple computing devices, such as a server cluster. Multiple computing devices can communicate with each other through any type of communication link (including a network, shared memory, etc.) to perform the processing disclosed herein.

[0196] The following are examples of embodiments of the present invention.

[0197] [1] An information processing device, comprising:

[0198] A reference distribution acquisition unit for acquiring a reference distribution indicating a tendency of predetermined language information;

[0199] an object distribution generating unit, configured to generate an object distribution of the language information extracted from the text data; and

[0200] An evaluation value calculation unit is configured to calculate an evaluation value indicating a difference between the reference distribution and the target distribution.

[0201] [2] The information processing device according to [1], characterized in that the language information is information indicating the relationship between the frequency of appearance of each word and the rank of the frequency of appearance.

[0202] [3] The information processing device according to [2] is characterized in that the evaluation value calculation unit performs standardization processing on the object distribution and the reference distribution respectively, calculates the difference between the object distribution after standardization and the reference distribution after standardization, and uses the difference as the evaluation value.

[0203] [4] The information processing device according to any one of [1] to [3], characterized in that:

[0204] It further comprises a text acquisition unit, which is used to acquire the set of text data;

[0205] The evaluation value calculation section calculates the evaluation value of each of the text data included in the set of text data.

[0206] [5] The information processing device according to [4] is characterized in that the reference distribution acquisition unit generates a distribution of the language information extracted from the set of text data and uses the distribution as the reference distribution.

[0207] [6] The information processing device according to [4] or [5] is characterized by further comprising a text deletion unit for deleting the text data with a high evaluation value from the set of text data.

[0208] [7] The information processing device according to [4] or [5] above is characterized by further comprising a text segmentation unit for segmenting the set of text data according to the evaluation value.

[0209] [8] An information processing system in which an information processing terminal operated by a user and an information processing device are capable of communicating via a network, characterized in that:

[0210] The information processing device comprises:

[0211] A reference distribution acquisition unit for acquiring a reference distribution indicating a tendency of predetermined language information;

[0212] An object distribution generating unit, used for generating an object distribution of the language information extracted from the text data;

[0213] an evaluation value calculation section for calculating an evaluation value indicating a difference between the reference distribution and the target distribution; and

[0214] a result output unit for sending screen data representing the evaluation result of the text data to the information processing terminal,

[0215] The information processing terminal has:

[0216] a text sending unit, configured to send the text data to the information processing device; and

[0217] A display control unit is configured to display a screen indicating an evaluation result of the text data based on the screen data received from the information processing device.

[0218] [9] An information processing method, characterized in that a computer can execute the following steps:

[0219] Obtaining a reference distribution indicating a tendency of specified language information;

[0220] generating an object distribution of the language information extracted from the text data; and

[0221] An evaluation value representing a difference between the reference distribution and the target distribution is calculated.

[0222]

[10] A computer program product, characterized in that it enables a computer to execute the following steps:

[0223] Obtaining a reference distribution indicating a tendency of specified language information;

[0224] generating an object distribution of the language information extracted from the text data; and

[0225] An evaluation value representing a difference between the reference distribution and the target distribution is calculated.

[0226] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various changes or modifications are allowed within the scope of the gist of the present invention.

Claims

1. An information processing device, characterized in that: have: A reference distribution acquisition unit for acquiring a reference distribution indicating a tendency of predetermined language information; an object distribution generating unit, configured to generate an object distribution of the language information extracted from the text data; and An evaluation value calculation unit is configured to calculate an evaluation value indicating a difference between the reference distribution and the target distribution.

2. The information processing device according to claim 1, characterized in that The language information is information indicating the relationship between the appearance frequency of each word and the rank of the appearance frequency.

3. The information processing device according to claim 2, characterized in that: The evaluation value calculation unit performs normalization processing on the target distribution and the reference distribution, calculates a difference between the target distribution after the normalization processing and the reference distribution after the normalization processing, and uses the difference as the evaluation value.

4. The information processing device according to any one of claims 1 to 3, characterized in that: It further comprises a text acquisition unit, which is used to acquire the set of text data; The evaluation value calculation section calculates the evaluation value of each of the text data included in the set of text data.

5. The information processing device according to claim 4, characterized in that The reference distribution acquisition unit generates a distribution of the language information extracted from the set of text data, and uses the distribution as the reference distribution.

6. The information processing device according to claim 4, characterized in that The device further includes a text deleting unit for deleting the text data having a high evaluation value from the set of text data.

7. The information processing device according to claim 4, characterized in that: The system further includes a text segmentation unit configured to segment the set of text data according to the evaluation value.

8. An information processing system, wherein an information processing terminal operated by a user and an information processing device can communicate via a network, characterized in that: The information processing device comprises: A reference distribution acquisition unit for acquiring a reference distribution indicating a tendency of predetermined language information; An object distribution generating unit, used for generating an object distribution of the language information extracted from the text data; an evaluation value calculation unit configured to calculate an evaluation value indicating a difference between the reference distribution and the target distribution; as well as a result output unit for sending screen data representing the evaluation result of the text data to the information processing terminal, The information processing terminal has: a text sending unit, configured to send the text data to the information processing device; and A display control unit is configured to display a screen indicating an evaluation result of the text data based on the screen data received from the information processing device.

9. An information processing method, characterized in that: The following steps are available for your computer: obtaining a reference distribution indicating a tendency of the specified language information; generating an object distribution of the language information extracted from the text data; and An evaluation value representing a difference between the reference distribution and the target distribution is calculated.

10. A computer program product, characterized in that The following steps are available for your computer: obtaining a reference distribution indicating a tendency of the specified language information; generating an object distribution of the language information extracted from the text data; and An evaluation value representing a difference between the reference distribution and the target distribution is calculated.

11. A computer-readable storage medium, characterized in that: It contains programs that allow the computer to perform the following steps: obtaining a reference distribution indicating a tendency of the specified language information; generating an object distribution of the language information extracted from the text data; and An evaluation value representing a difference between the reference distribution and the target distribution is calculated.

Images