Training device, training method, and program

Abstract

The purpose of the present invention is to perform machine learning with higher accuracy when training a machine learning model using summary source text and summary text.　For this purpose, the present disclosure is a training device for training a machine learning model to infer summary text from summary source text, the training device including: an acquisition unit that acquires summary source text and acquires summary text as ground truth data; a first exclusion unit that excludes a sentence containing a predetermined noun from the summary text if the predetermined noun in the summary text is not included in the summary source text; and a machine learning unit that trains the machine learning model on the basis of the summary source text and the summary text after processing by the first exclusion unit.

Description

Learning device, learning method, and program

[0001] The present disclosure relates to techniques for filtering out noise from training data that is summarized text.

[0002] In recent years, there has been a growing need for summaries to efficiently grasp the overview of large documents and to select the information that should be read from them. To address this need, API (Application Programming Interface) services that generate summaries have been actively released.

[0003] Furthermore, summarization using machine learning requires a high-quality summary dataset of hundreds of thousands of items. Therefore, in many cases, datasets for machine learning are created using news articles, for which data is readily available, and their three-line headlines as summaries. For example, Non-Patent Document 1 discloses a technique for creating pseudo-data using predetermined news articles and their three-line summaries, and using the pseudo-data for machine learning.

[0004] Information Processing Society of Japan Research Report Vol.2017-DBS-165 No.33 (Vol.2017-IFAT-128No.33) 2017 / 9 / 30 "Neural document summarization focusing on TL;DR three-line summaries" (Kodaira Tomonori, Komachi Mamoru)

[0005] However, there are cases where the summary text contains information that is unrelated to the news article itself (the original text), which can become noise in machine learning.

[0006] The present disclosure has been made in consideration of the above-mentioned circumstances, and aims to perform more accurate machine learning when training a machine learning model using a source text and a summarized text.

[0007] In order to achieve the above object, the invention of claim 1 is a learning device that trains a machine learning model that infers summary text from summary source text, the learning device having: an acquisition unit that acquires the summary source text and the summary text as correct answer data; a first exclusion unit that excludes sentences containing a predetermined noun in the summary text from the summary text if the predetermined noun is not included in the summary source text; and a machine learning unit that trains the machine learning model based on the summary source text and the summary text after processing by the first exclusion unit.

[0008] As described above, the present invention has the effect of enabling more accurate machine learning.

[0009] FIG. 1 is an overall configuration diagram of a communication system according to an embodiment; FIG. 2 is an electrical hardware configuration diagram of a learning device according to an embodiment; FIG. 3 is an electrical hardware configuration diagram of an input device according to an embodiment; FIG. 4 is a functional configuration diagram of a learning device; FIG. 5 is a conceptual diagram showing a dictionary table constituting a dictionary DB; FIG. 6 is a flowchart showing the processing or operation of a learning device; and FIG.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [System Configuration of the Embodiment] First, an outline of the configuration of a communication system according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to the present embodiment.

[0012] 1, a communication system 10 of this embodiment is constructed by a learning device 30 and an input device 50. The input device 50 is managed and used by a user. The learning device 30 and the input device 50 can communicate with each other via a communication network 100 such as the Internet.

[0013] The communication network 100 may include a dedicated network such as an ISP (Internet Service Provider) network managed and / or operated by a telecommunications carrier. The connection form of the communication network 100 may be either wireless or wired.

[0014] The learning device 30 is composed of one or more computers. When the learning device 30 is composed of multiple computers, it may be referred to as a "learning device" or a "learning system."

[0015] The learning device 30 acquires a machine learning dataset (source text and summary text as correct answer data) from the input device 50. The learning device 30 then removes noise from the summary text in two stages, and then trains a machine learning model for inferring the summary text from the source text.

[0016] The input device 50 is a computer, and FIG. 1 shows a laptop computer as an example. In FIG. 1, a user operates the input device 50. The input device 50 transmits a machine learning dataset (source text to be summarized, and summary text as supervised data) to the learning device 30. Note that the learning device 30 may input the machine learning dataset (source text to be summarized, and summary text as supervised data) directly through user operation, rather than from the input device 50.

[0017] [Hardware Configuration] <Hardware Configuration of Learning Device> Next, the electrical hardware configuration of the learning device 30 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the learning device.

[0018] As shown in FIG. 2, the learning device 30 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an SSD (Solid State Drive) 304, an external device connection I / F (Interface) 305, a network I / F 306, a media I / F 309, and a bus line 310.

[0019] Of these, the CPU 301 controls the overall operation of the learning device 30. The ROM 302 stores programs used to drive the CPU 301, such as an IPL (Initial Program Loader). The RAM 303 is used as a work area for the CPU 301.

[0020] The SSD 304 reads or writes various data under the control of the CPU 301. Note that instead of the SSD 304, a hard disk drive (HDD) may be used.

[0021] The external device connection I / F 305 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0022] The network I / F 306 is an interface for performing data communication via the communication network 100 .

[0023] The media I / F 309 controls reading and writing (storing) of data from and to a recording medium 309m such as a flash memory, etc. The recording medium 309m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0024] The bus line 310 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 301 shown in FIG.

[0025] <Hardware Configuration of Input Device> Next, the electrical hardware configuration of the input device 50 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the electrical hardware configuration of the input device.

[0026] As shown in FIG. 3, the input device 50 is a computer and includes a CPU 501, a ROM 502, a RAM 503, an SSD 504, an external device connection I / F (Interface) 505, a network I / F 506, a display 507, an input device 508, a media I / F 509, and a bus line 510.

[0027] Of these, the CPU 501 controls the overall operation of the input device 50. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501.

[0028] The SSD 504 reads or writes various data under the control of the CPU 501. Note that instead of the SSD 504, a hard disk drive (HDD) may be used.

[0029] The external device connection I / F 505 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB memory, and a printer.

[0030] The network I / F 506 is an interface for performing data communication via the communication network 100 .

[0031] The display 507 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays various images.

[0032] The input device 508 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like.

[0033] The media I / F 509 controls reading and writing (storing) of data from and to a recording medium 509m such as a flash memory, etc. The recording medium 509m includes DVDs and Blu-ray Discs (registered trademarks).

[0034] The bus line 510 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 501 shown in FIG.

[0035] [Functional Configuration of Learning Device] Next, the functional configuration of the learning device 30 according to this embodiment will be described with reference to Figures 4 and 5. Figure 4 is a diagram showing the functional configuration of the learning device.

[0036] 4, the learning device 30 includes an acquisition unit 31, a first exclusion unit 32, a second exclusion unit 33, and a machine learning unit 34. These units each have a function realized by an instruction from the CPU 301 in FIG. 2 based on a program. In addition, the RAM 303 or the SSD 304 stores a machine learning model 40 and a dictionary DB 41.

[0037] <Dictionary DB> Fig. 5 is a conceptual diagram showing a dictionary table that constitutes the dictionary DB. As shown in Fig. 5, the dictionary table manages proper nouns and abbreviations in association with each other. "Proper nouns" include company names, personal names, product names, service names, etc. "Abbreviations" also include nicknames, pen names, etc. For example, a proper noun is "Nippon Telegraph and Telephone Corporation" and an abbreviation is "NTT (registered trademark)."

[0038] <Functional Configuration> (Acquisition Unit) The acquisition unit 31 acquires a learning set (source text to be summarized, and summarized text as correct answer data) from the input device 50 .

[0039] (First Exclusion Unit) The first exclusion unit 32 acquires the source text and the summary text from the acquisition unit 31 and determines whether a predetermined noun contained in the summary text is also contained in the source text. Specifically, the first exclusion unit 32 extracts the predetermined noun by performing morphological analysis on the summary text. The predetermined noun may be a company name, a personal name, a product name, a service name, etc. The first exclusion unit 32 then searches the source text using the extracted predetermined noun as a search key to determine whether the source text contains a noun that is the same as the predetermined noun.

[0040] Furthermore, if the source text does not contain a noun identical to the predetermined noun, the first excluding unit 32 further determines whether the predetermined noun in the summary text is an abbreviation of a proper noun in the source text. Specifically, the first excluding unit 32 searches the dictionary DB 41 using the predetermined noun as a search key to determine whether the predetermined noun is managed as an abbreviation in the dictionary DB 41. If the predetermined noun is managed as an abbreviation, the first excluding unit 32 retrieves a proper noun corresponding to the abbreviation. Then, the source text is searched using the proper noun as a search key to determine whether the source text contains the same proper noun. If the same proper noun is found, the first excluding unit 32 determines that the predetermined noun in the summary text is an abbreviation of the proper noun in the source text. On the other hand, if the same proper noun is not found, the first excluding unit 32 determines that the predetermined noun in the summary text is not an abbreviation of the proper noun in the source text.

[0041] Furthermore, if the first exclusion unit 32 determines that a predetermined noun contained in the summary text is not an abbreviation of a proper noun contained in the source text, it excludes the sentence containing the predetermined noun from the summary text. On the other hand, if the first exclusion unit 32 determines that a predetermined noun contained in the summary text is an abbreviation of a proper noun contained in the source text, it does not exclude the sentence containing the predetermined noun from the summary text.

[0042] The first exclusion unit 32 also repeatedly performs the above-mentioned determination for all predetermined nouns in the summary text.

[0043] As described above, if there is a specific noun that is not included in the original text and is not included in the summary text as an abbreviation of a proper noun in the original text, the first exclusion unit 32 excludes (deletes) sentences that include this specific noun, thereby eliminating noise. For example, if the author's name is included only in the summary text and not in the original text, the author's name will be excluded because it will become noise in machine learning.

[0044] On the other hand, if the summary text contains a predetermined noun that is not included in the source text but is included as an abbreviation of a proper noun in the source text, the first exclusion unit 32 suppresses the exclusion of noise by not excluding sentences containing this predetermined noun. For example, if the source text contains a proper noun such as a company name or personal name, but the summary text contains an abbreviation, this abbreviation will not be excluded as it will not become noise in machine learning.

[0045] (Second Exclusion Unit) The second exclusion unit 33 excludes character groups from each sentence in the summary text whose relationship with each sentence in the source text is below a predetermined value. These "characters" include symbols. Furthermore, "character groups" include not only sentences but also URLs (Uniform Resource Locators) or email addresses. Specifically, the second exclusion unit 33 uses ROUGE (Recall-Oriented Understudy for Gisting Evaluation) technology to exclude from the summary text any sentences for which the ROUGE between the source text and each sentence in the summary text is below a predetermined threshold, as sentences with low relationship. In this case, the second exclusion unit 33 measures how many words from the source text are included in the summary text, and excludes character groups containing a certain percentage or less of characters as noise.

[0046] For example, on English news sites, the summary text may contain email addresses that are not included in the original text being summarized. However, such email addresses are excluded because they become noise in machine learning.

[0047] (Machine Learning Unit) The machine learning unit 34 associates the source text to be summarized with the summarized text as the correct data from which noise has been removed by the processing performed by the first removal unit 32 and the second removal unit 33, and uses this as input data (learning data), and performs machine learning on the machine learning model 40. In this way, the machine learning unit 34 generates a trained machine learning model.

[0048] [Processing or Operation of Learning Device] Next, the processing or operation of the learning device 30 in the learning phase and estimation phase will be described with reference to Figures 6 and 7. Figures 6 and 7 are flowcharts showing the processing or operation of the learning device.

[0049] S11: The acquisition unit 31 acquires a learning set (original text to be summarized, and summarized text as correct answer data) from the input device 50.

[0050] S12: The first exclusion unit 32 determines whether a predetermined noun in the summary text also exists in the original text. If the predetermined noun in the summary text also exists in the original text (S12; YES), the process proceeds to step 16 in FIG. 7.

[0051] S13: On the other hand, if the specified noun in the summary text is not present in the original text (S12; NO), the first exclusion unit 32 further determines whether the specified noun in the summary text is an abbreviation of a proper noun in the original text.

[0052] S14: If the specified noun in the summary text is not an abbreviation of a proper noun in the original text (S13; NO), the first exclusion unit 32 excludes sentences (text) containing the specified noun from the summary text and proceeds to the next process S16.

[0053] S15: On the other hand, if the predetermined noun in the summary text is an abbreviation of a proper noun in the original text (S13; YES), the first exclusion unit 32 does not exclude sentences (text) containing the predetermined noun from the summary text, and proceeds to the next step S16. Note that the first exclusion unit 32 may omit steps S13 and S15, and thereby exclude the predetermined noun from the summary text by step S15 if the predetermined noun in the summary text is not included in the original text (S12; NO).

[0054] S16: If the first exclusion unit 32 has not yet performed the determinations in steps S12 and S13 for all of the specified nouns in the summary text (NO), it returns to step S12 and continues performing steps S12 to S15 for the unprocessed specified nouns.

[0055] S17: If the first exclusion unit 32 has performed the determinations in steps S12 and S13 for all of the predetermined nouns in the summary text (S16; YES), the second exclusion unit 33 uses ROUGH technology to exclude, from each sentence in the summary text, groups of characters whose relationship with each sentence in the source text is equal to or less than a predetermined value. Note that the second exclusion unit 33 may perform its processing before the first exclusion unit.

[0056] S18: The machine learning unit 34 executes machine learning of the machine learning model 40 based on the original text to be summarized and the supervised data, which is the summarized text after processing by the first exclusion unit 32 and the second exclusion unit 33.

[0057] As described above, according to this embodiment, the first exclusion unit 32 excludes, from the machine learning summary text, sentences that include a predetermined noun that is not included in the summary source text as machine learning noise, thereby achieving the effect of being able to exclude noise from the summary text that serves as ground truth data for machine learning.

[0058] Furthermore, the first exclusion unit 32 does not treat sentences containing abbreviations of proper nouns in the source text as noise in machine learning and does not exclude them from the machine learning summary text, thereby achieving the effect of enabling more accurate machine learning.

[0059] Furthermore, the second exclusion unit 33 utilizes ROUGE to exclude sentences whose ROUGE for each sentence in the source text and the summary text is below a predetermined threshold value from the summary text as correct answer data, as sentences with low relevance, thereby achieving the effect of enabling even more accurate machine learning.

[0060] [Supplementary Note] The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations), for example.

[0061] (1) The learning device 30 can be realized by a computer and a program, but this program can also be recorded on a (non-temporary) recording medium or provided via a communication network 100 such as the Internet.

[0062] (2) In the above embodiment, a smartphone is shown as an example of the input device 50, but this is not limited to this and may be, for example, a laptop PC, a tablet terminal, a smart watch, a camera capable of communication, etc.

[0063] (3) The CPUs 301 and 501 serving as processors may each be single or multiple.

[0064] REFERENCE SIGNS LIST 10 Communication system 30 Learning device 31 Acquisition unit 32 First exclusion unit 33 Second exclusion unit 34 Machine learning unit 40 Machine learning model 41 Dictionary DB (an example of a dictionary management unit) 50 Input device

Claims

1. A learning device that trains a machine learning model that infers summary text from source text, comprising: an acquisition unit that acquires source text and summary text as correct answer data; a first exclusion unit that excludes sentences containing a predetermined noun in the summary text from the summary text if the predetermined noun is not included in the source text; and a machine learning unit that trains the machine learning model based on the source text and the summary text after processing by the first exclusion unit.

2. The learning device described in claim 1, wherein if the specified noun in the summary text is not included in the original text, and the specified noun in the summary text is an abbreviation of a proper noun in the original text, the first exclusion unit does not exclude sentences containing the specified noun from the summary text.

3. A learning device as described in claim 2, further comprising a dictionary management unit in which proper places of interest are associated with abbreviations of the proper nouns, and the first exclusion unit uses the dictionary management unit to determine whether the specified noun in the summary text is an abbreviation of a proper noun in the source text.

4. A learning device according to any one of claims 1 to 3, comprising a second exclusion unit that uses Recall-Oriented Understudy for Gisting Evaluation to exclude, from each sentence in the summary text, a group of characters whose relationship with each sentence in the source text is equal to or less than a predetermined value, and wherein the machine learning unit trains the machine learning model based on the source text and the summary text after processing by the second exclusion unit.

5. A learning method performed by a learning device that trains a machine learning model that infers summary text from source text, wherein the learning device performs the following steps: an acquisition process that acquires source text and summary text as correct answer data; a first exclusion process that, if a predetermined noun in the summary text is not included in the source text, excludes sentences that include the predetermined noun from the summary text; and a machine learning process that trains the machine learning model based on the source text and the summary text after the first exclusion process.

6. A program for causing a computer to execute the method according to claim 5.

Images