Information processing device and information processing method

Abstract

Device (10) for determining the correspondence between an article content text that is displayed or read aloud on a screen and for displaying the image on the screen, which comprises: a unit (11a, 11b) for morphological analysis to carry out a morphological analysis in order to divide each of an article content text contained in an article and a caption of each of images into morphemes; a sentence part acquisition unit (12) that divides the article content text into sentence parts based on a result of the morphological analysis carried out by the morphological analysis unit (11b); a caption summary unit (14) that generates a caption summary by summarizing the caption; a correspondence determination unit (13) that determines a correspondence between each of the sentence parts of the article content text and the images by calculating a correlation between the image caption summary and each of the sentence parts of the article content text on the basis of the result of the morphological analysis carried out by the unit (11a) for morphological analysis; an output unit (22, 23) that displays the article content text on the screen or reads the article content text aloud, and that displays the corresponding images; and a control unit (21) that controls the output unit (22, 23) to display the image corresponding to the sentence part on the basis of the match obtained by the match determination unit (13) when a sentence part of the article content text is read aloud or displayed.

Description

TECHNICAL AREA

[0001] The present invention relates to an information processing device and an information processing method for processing information. STATE OF THE ART

[0002] Many applications that display text, news articles, or blog posts on a screen are in practical use. For example, news applications are used to display news on the screen of a vehicle navigation system, a display device in an elevator, or a smartphone.

[0003] In many cases, an article displayed by the aforementioned application includes an image as supplementary information. Furthermore, a caption explaining the image's content may also be added.

[0004] Adding an image to an article makes it easier for the user to understand the article's content. However, if an image is displayed that doesn't correspond to either a sentence shown on the screen or a sentence currently being read aloud, the user will be confused and the sentence's content will be difficult to understand.

[0005] For example, in a case where several images are added to an article, if all images are displayed in the same way, it is difficult to understand which image the sentence currently displayed or read aloud explains. Furthermore, in a case where the screen area is small, such as that of a smartphone, and only one of several images can be displayed at a time, it is necessary to display an image corresponding to the sentence currently displayed or read aloud.

[0006] An example of a technique for selecting an image adapted to a document is described in patent document 1. In a document decoration support system of patent document 1, a document is analyzed, characteristic words are extracted, rating values ​​for template images are calculated, and the template images are displayed in descending order of their rating values.

[0007] Furthermore, an example of a technique for highlighting a sentence that is currently being read aloud is described in patent document 2. A display control device of patent document 2 extracts text areas containing text from display data, calculates a correlation between the interior of each of the text areas and each writing element contained in the writing data to be read aloud in order to determine a correspondence between the text area and the writing element, and displays the text area highlighted according to the writing element through which speech data is being output.

[0008] Non-patent document 1 presents a method for measuring the semantic similarity of texts, based on a corpus-based measure of semantic word similarity as well as on a normalized and modified version of the Longest Common Subsequence (LCS) string-matching algorithm.

[0009] Non-patent document 2 proposes a novel model capable of constructing representations of morphologically complex words from their morphemes. It combines recursive neural networks (RNNs), where each morpheme represents a basic unit, with neural language models (NLMs) to incorporate contextual information when learning morphologically sensitive word representations. CITATION LIST PATENT LITERATURE Patent document 1: Publication No. 2011-113289 of an unexamined Japanese patent application Patent document 2: Publication No. 2012-185567 of an unexamined Japanese patent application. NON-PATENT LITERATURE Non-Patent Document 1: ISLAM Aminul; INKPEN, Diana. Semantic text similarity using corpus-based word similarity and string similarity. ACM Transactions on Knowledge Discovery from Data (TKDD), 2008, Volume 2, No. 2, pp. 1-25. DOI: 10.1145 / 1376819 Non-Patent Document 2: LUONG, Minh-Thang; SOCHER, Richard; MANNING, Christopher D. Better Word Representations with Recursive Neural Networks for Morphology. In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning. 2013. pp 104-113. SUMMARY OF THE INVENTIONAL PROBLEM

[0010] In the aforementioned patent documents 1 and 2, a correspondence with an image is achieved for each input unit, such as a document or a sentence. Therefore, a problem arises in that, although an application displays article content text and an image on the screen using the technique described in patent document 1, if multiple images are provided for a single article, the article content text and the image currently displayed on the screen do not correspond. Similarly, a problem arises in that, although an application displays an image while article content text is being read aloud using the technique described in patent document 2, if multiple images are provided for a single article, the currently read-aloud article content text and the image displayed on the screen do not correspond.

[0011] The present invention was made to solve the aforementioned problems, and it is therefore an object of the present invention to provide a technique for correctly determining the correspondence between an article content text that is displayed or read aloud on a screen and an image displayed on the screen. SOLUTION TO THE PROBLEM

[0012] An information processing device according to the present invention comprises: a morphological analysis unit that performs a morphological analysis to divide each element of an article's content text and each image caption into morphemes; a sentence part acquisition unit that divides the article's content text into sentence parts based on a result of the morphological analysis performed by the morphological analysis unit; and an image caption summary unit that generates an image caption summary by summarizing the image caption.and a correspondence determination unit that determines the correspondence between each of the sentence parts of the article content text and the images by calculating a correlation between the image caption summary and each of the sentence parts of the article content text based on the result of the morphological analysis carried out by the morphological analysis unit. ADVANTAGEOUS EFFECTS OF THE INVENTION

[0013] According to the present invention, since a correlation between a caption and each of the sentence parts of an article content text is calculated and a correspondence between each of the sentence parts of the article content text and an image is determined, a correspondence between an article content text that is displayed or read aloud on a screen and an image displayed on the screen can be correctly determined. BRIEF DESCRIPTION OF DRAWINGS Fig. Figure 1 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 1 of the present invention; Fig. Figure 2 is a view showing an example of a hardware configuration of the voice guidance device that uses the information processing device according to embodiment 1 of the present invention; Fig. Figure 3 is a view showing another example of a hardware configuration of the voice guidance device that uses the information processing device according to embodiment 1 of the present invention; Fig. Figure 4 is a view showing an example of an article to be processed by the information processing device according to the embodiment of the present invention; Fig. Figure 5 is a flowchart showing the operation of the information processing device according to embodiment 1 of the present invention; Fig. 6 is a flowchart that details step ST4 of Fig. 5 shows; Fig. Figure 7 is a table showing the number of matching independent words between each clause and each heading according to embodiment 1 of the present invention; Fig. Figure 8 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 2 of the present invention; Fig. Figure 9 is a flowchart showing the operation of the information processing device according to embodiment 2 of the present invention; Fig. Figure 10 is a view showing an example of an article to be processed by the information processing device according to embodiment 2 of the present invention; Fig. 11 is a table showing the number of matching independent words between each clause and each caption in a case where the article of Fig. 10 is processed by the information processing device according to embodiment 1 of the present invention; Fig. 12 is a table showing the number of matching independent words between each clause and each caption in a case where the article of Fig. 10 is processed by the information processing device according to embodiment 2 of the present invention; Fig. Figure 13 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 3 of the present invention; Fig. Figure 14 is a flowchart showing the operation of the information processing device according to embodiment 3 of the present invention; Fig. 15 is a view that is a result of the dependency analysis of the in Fig. Article 4 shown, which was carried out by a dependency analysis unit of the information processing device according to embodiment 3 of the present invention; Fig. Figure 16 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 4 of the present invention; Fig. Figure 17 is a flowchart showing the operation of the information processing device according to embodiment 4 of the present invention; Fig. Figure 18 shows an example of an article to be processed by the information processing device according to embodiment 4 of the present invention and an article content summary generated by a content summary unit; Fig. Figure 19 is a block diagram showing an example of the configuration of a voice guidance device that uses an information processing device according to embodiment 5 of the present invention; Fig. Figure 20 is a flowchart showing the operation of the information processing device according to embodiment 5 of the present invention; Fig. Figure 21 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 6 of the present invention; Fig. Figure 22 is a flowchart showing the operation of the information processing device according to embodiment 6 of the present invention; Fig. Figure 23 is a view showing an example of word significance level information in the information processing device according to embodiment 6 of the present invention; Fig. Figure 24 is a block diagram showing an example of a configuration of a voice guidance device that uses an information processing device according to embodiment 7 of the present invention. Fig. Figure 25 is a view showing an example of a hardware configuration of the voice guidance device that uses the information processing device according to embodiment 7 of the present invention; and Fig. Figure 26 is a view showing another example of a hardware configuration of the voice guidance device using the information processing device according to embodiment 7 of the present invention. DESCRIPTION OF EXAMPLES OF EXECUTION

[0014] In order to explain the present invention in detail, some embodiments of the invention are described below with reference to the accompanying drawings. Example 1

[0015] Fig. Figure 1 is a block diagram showing an example configuration of a speech guidance device 20 that uses an information processing device 10 according to embodiment 1 of the present invention. The information processing device 10 includes a morphological analysis unit 11a, a morphological analysis unit 11b, a sentence part acquisition unit 12, and a matching unit 13. In the example of Fig. In 1, the information processing device 10 is integrally configured with the speech guidance device 20. The speech guidance device 20 comprises the information processing device 10, a control unit 21, a speech output unit 22, and a display unit 23.

[0016] The Fig. 2 and Fig. Figure 3 shows examples of a hardware configuration of the speech guidance device 20, which uses the information processing device 10 according to embodiment 1 of the present invention. Each of the functions of the morphological analysis unit 11a, the morphological analysis unit 11b, the sentence part acquisition unit 12, the matching unit 13, and the control unit 21 in the speech guidance device 20 is implemented by a processing circuit. That is, the speech guidance device 20 contains a processing circuit for implementing each of the aforementioned functions. The processing circuit can be a processor 100 that can execute a program stored in a memory 102, or it can be a processing circuit 107 as dedicated hardware.

[0017] In the case where the processing circuit is processor 100, as in Fig. As shown in Figure 2, each of the functions of the morphological analysis unit 11a, the morphological analysis unit 11b, the sentence part acquisition unit 12, the match determination unit 13, and the control unit 21 is implemented by software, firmware, or a combination of software and firmware. The software or firmware is written as a program, and the program is stored in memory 102. Alternatively, the program can be stored on a disk drive 103. The processor 101 implements the function of each of the units by reading and executing the program stored in memory 102. That is, the speech guidance device 20 contains memory 102 for storing a program, which is shown in a flowchart of Fig. The steps shown, or similar ones, which will be described later, are performed as a result when the program is executed by processor 101. Furthermore, it should be noted that this program causes a computer to perform operations or procedures implementing the Morphological Analysis Unit 11a, the Morphological Analysis Unit 11b, the Sentence Part Acquisition Unit 12, the Matching Unit 13, and the Control Unit 21.

[0018] Here, processor 101 is a central processing unit (CPU), a processing device, an arithmetic device, a microprocessor, a microcomputer, or the like.

[0019] The memory 102 can be a non-volatile or volatile semiconductor memory, such as random access memory (RAM), read-only memory (ROM), erasable programmable ROM (EPROM), or flash memory, or it can be an optical disc such as a Compact Disc (CD) or a Digital Versatile Disc (DVD).

[0020] In the case where the processing circuit uses hardware for dedicated use, as in Fig. As shown in Figure 3, the processing circuit 107 is, for example, a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (EPGA), or a combination of these circuits. The functions of the morphological analysis unit 11a, the morphological analysis unit 11b, the sentence part acquisition unit 12, the matching unit 13, and the control unit 21 can be implemented by several processing circuits 107, or the functions of these units can be implemented in their entirety by a single processing circuit 107.

[0021] Some of the functions of the morphological analysis unit 11a, the morphological analysis unit 11b, the sentence part acquisition unit 12, the match determination unit 13, and the control unit 21 can be implemented by software or firmware, and some of these can be implemented by hardware for dedicated use. In this way, the processing circuit in the speech guidance device 20 can implement any of the aforementioned functions using software, firmware, hardware, or a combination of software, firmware, and hardware.

[0022] The speech output unit 22 in the speech guidance device 20 is a loudspeaker 104. The display unit 23 in the speech guidance device 20 is a display device 105. The processor 101 or the processing circuit 107, the disk drive 103, the loudspeaker 104 and the display device 105 are connected via a bus 106 or the like in such a way that the sending and receiving of information can be carried out.

[0023] Next, the operation of the speech guidance device 20, which uses the information processing device 10, will be explained.

[0024] Fig. Figure 4 shows an example of an article to be processed by the information processing device 10 according to embodiment 1 of the present invention. The article contains information about an article content text, images, and image captions. An image caption is a sentence that explains the content of an image with which the caption is paired.

[0025] Fig. Figure 5 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 1 of the present invention. The following assumes a case in which the information pieces of the Fig. The information from the article shown in section 4 is entered into the information processing device 10.

[0026] In step ST1, unit 11a for morphological analysis performs a morphological analysis of each caption in order to divide the caption into morphemes, and obtains a reading text assigned to each of the morphemes and a language part of each of the morphemes.

[0027] In step ST2, unit 11b for morphological analysis performs a morphological analysis of the article content text in order to divide the article content text into morphemes, and obtains a reading text that is assigned to each of the morphemes, and a language part of each of the morphemes.

[0028] In step ST3, the Sentence Part Acquisition Unit 12 divides the article content text into sentence parts based on the result of the morphological analysis of the article content text, the result obtained by Unit 11b for morphological analysis. More precisely, Sentence Part Acquisition Unit 12 determines whether each of the morphemes in the article content text is an independent word or a dependent word, taking into account the part of speech and conjugation, and obtains sentence parts, each of which is a combination of morphemes. According to a very simplified definition, a sentence part has a form in which zero or more dependent words follow a sequence of one or more independent words. The processing performed by Sentence Part Acquisition Unit 12 can be implemented using a technique described, for example, in "Nyumon Shizen Gengo Shori (Natural Language Processing with Python)" (Steven Bird et al. 4th ed.).as described in the O'Reilly Japan edition, January 17, 2012, pages 498-499.

[0029] In step ST4, the matching unit 13 calculates a correlation between each image caption and each of the sentence parts in the article content text. This is based on the result of the morphological analysis of the image caption, obtained by unit 11a for morphological analysis, and the sentence parts in the article content text, acquired by the sentence part acquisition unit 12. This determines a match between each of the sentence parts in the article content text and an image. More precisely, the matching unit 13 calculates a correlation between each of the sentence parts in the article content text and each image caption and identifies the image corresponding to a caption with the highest correlation to the sentence part as the image corresponding to a sentence part.

[0030] Next, details of step ST4 will be explained.

[0031] Fig. 6 is a flowchart that details step ST4 in Fig. Figure 5 shows that a caption attached to one of the several images included in the article is referred to as a caption c. Furthermore, one of the several clauses contained in the article's body text is referred to as a clause p.

[0032] In step ST41, the matching unit acquires 13 independent words in caption c based on the language part of caption c, where the language part was determined by unit 11a for morphological analysis, and generates a list of independent words. There are different procedures for acquiring independent words. For example, matching unit 13 can acquire all morphemes except stop words (postpositive particles and auxiliary verbs) as independent words, or it can acquire only nouns as independent words.

[0033] In step ST42, the matching unit 13 calculates a correlation between the sentence part p of the article content text and the image caption c. There are various methods for calculating the correlation. For example, the matching unit 13 obtains a correlation between the sentence part p and the image caption c based on how many words in the list of independent words of the image caption c the sentence part p contains.

[0034] In step ST43, the matching unit 13 identifies the caption c, which has the highest correlation with the sentence fragment p, as a matching image. The matching unit 13 then outputs the match between the sentence fragment and the image.

[0035] Fig. Figure 7 is a table showing the number of matching independent words between each sentence part and each caption in embodiment 1 of the present invention. “ / ” in each caption and “ / ” in each sentence part indicate split positions between morphemes. According to Fig. 7. The correlation between the phrase “Yamada / Taro / shi / no” and the caption “Yamada / Taro / shi” is “3”, and is higher than the correlation “0” with each of the other captions “SatojHanakojmotojBoeisho” and “Suzuki / Ichiro / moto / Somusho”. Therefore, the image corresponding to the phrase “Yamada / Taro / shi / no” is the one with the caption “Yamada / Taro / shi”.

[0036] In addition to the aforementioned method, one method for calculating the correlation is, for example, a method for calculating a value between each sentence part and each caption, where the value is called the Levenshtein distance or the edit distance, and a method for expressing each sentence part and each caption as vectors using the "word2vec" method and acquiring the similarity between the vectors.

[0037] The control unit 21 controls the speech output unit 22 and the display unit 23 for displaying the corresponding image when a sentence fragment of the article content text is read aloud, based on the match between each image and each sentence fragment, the match being determined by the match detection unit 13. The speech output unit 22 reads the article content text aloud under the control of the control unit 21. The display unit 23 displays the image under the control of the control unit 21.

[0038] For example, during the time when the speech output unit 22 displays “Yamada / Taro / shi / no”, which is the first part of the sentence in Fig. 7. While the speech output unit 22 is reading aloud, display unit 23 shows the image with the caption "Yamada / Taro / shi". Next, while speech output unit 22 is reading aloud the sentence parts from the second part "jishoku / ni" to the tenth part "doto / shuin / giin / no", display unit 23 does not show an image. Next, while speech output unit 22 is reading aloud the eleventh part "Sato / Hanako / moto / Boeisho / ni / taishi / ", display unit 23 shows the image with the caption "Sato / Hanako / moto / Boeisho".

[0039] Alternatively, during the time when the speech output unit 22 reads the sentence parts from the second sentence part “jishoku / ni” to the tenth sentence part “doto / shuin / giin / no”, the display unit 23 can continue to display the image to which the caption “Yamada / Taro / shi” is added, corresponding to the first sentence part “Yamada / Taro / shi / no”.

[0040] Although in embodiment 1 the voice guidance device 20 is configured to read the article content text aloud, such voice reading is not necessarily required. For example, a configuration may be provided in which each sentence part is highlighted instead of being read aloud, and an image corresponding to the highlighted sentence part is displayed. Furthermore, a configuration may be provided in which, when part of the article content text is displayed on the screen, an image corresponding to the currently displayed part of the article content text is displayed or highlighted.

[0041] As described above, the information processing device 10 according to embodiment 1 is configured to include: a morphological analysis unit 11a, 11b, which performs a morphological analysis to divide each of the article content text and each of the images into morphemes; a clause acquisition unit 12, which divides the article content text into clauses based on a result of the morphological analysis performed by the morphological analysis unit 11b; and a correspondence determination unit 13, which determines a correspondence between each of the clauses of the article content text and the images by calculating a correlation between the image caption and each of the clauses of the article content text based on the result of the morphological analysis performed by the morphological analysis unit 11a.As a result, a match between an article content text displayed or read aloud on the screen and an image displayed on the screen can be correctly determined. Example 2

[0042] Fig. Figure 8 is a block diagram showing an example configuration of a voice guidance device 20 that uses an information processing device 10 according to embodiment 2 of the present invention. The information processing device 10 according to embodiment 2 has a configuration in which a caption summary unit 14 is added to the information processing device 10 according to embodiment 2. Fig. 1. The embodiment shown in example 1 has been added. Fig. 8 are the same or corresponding components as those in Fig. 4 are designated by the same reference symbols, and an explanation of the components is omitted below.

[0043] The caption summary unit 14 is provided by the one in Fig. The processor 101 shown in Figure 2 implements a program stored in memory 102. Alternatively, the caption summary unit 14 is replaced by the one shown in Figure 2. Fig. 3 processing circuit 107 implemented.

[0044] Fig. Figure 9 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 2 of the present invention. Since the processes in steps ST1 to ST3 of Fig. 9 the same as those in steps ST1 to ST3 of Fig. If there are 5 steps, the explanation of the steps will be omitted.

[0045] In step ST1, the subsequent step ST5, the caption summarization unit 14 generates a caption summary by summarizing each caption using information about a language element, and so on, from the result of the caption's morphological analysis, the result of which was obtained by unit 11a for morphological analysis. Various methods exist for summarizing a caption, such as using a dependency structure and using term frequency-inverse document frequency (TF-IDF). The IDF is a value predetermined for each word and represents the frequency of its occurrence. TF is a value that shows the frequency of a word's occurrence within a text to be read aloud. TF-IDF is calculated by multiplying the TF value and the IDF value.

[0046] Here, Caption Summary Unit 14 uses an extraction procedure, as a caption summary, of a group of nouns that are at least part of a caption, based on the result of a morphological analysis of the caption. In particular, in many cases, a noun or noun phrase appears at the end of a caption for an image in a news article or the like, which is a characteristic referred to in Japanese as "taigendome," so this procedure using such a characteristic is effective.

[0047] In step ST4A, a match determination unit 13 calculates a correlation between the caption summary obtained by the caption summary unit 14 and each of the sentence parts of the article content text, where the sentence parts were obtained by a sentence part acquisition unit 12, based on the caption summary and each of the sentence parts of the article content text, thereby determining a match between each of the sentence parts of the article content text and an image.

[0048] Fig. Figure 10 is a view showing an example of an article to be processed by the information processing device 10 according to embodiment 2 of the present invention. Fig. 11 is a table showing the number of matching independent words between each sentence part and each caption in the case where the information processing device 10 according to embodiment 1 of the present invention processes the article of Fig. 10 processed. Fig. Figure 12 is a table showing the number of matching independent words between each sentence part and each caption in the case where the information processing device 10 according to embodiment 2 of the present invention processes the article of Fig. 10 processed.

[0049] In embodiment 1, the correlation unit 13 calculates a correlation using the result of a morphological analysis of each entire image caption. However, in the case of the Fig. In the 10 articles shown, if a caption consisting of a long sentence is used without any processing, the correlation will not be calculated correctly. As in Fig. As shown in 11, although an image shows “Saito Jiro bengoshi (jurist Jiro Saito)”, there is a high correlation with “Ofuna Kosan”, which is not directly related to the image, and as a consequence the image to which the caption “Ofuna / Kosan / keiei / jin / to / no / kyogi / go / , / kisha / dan / no / shuzai / ni / oji / ru / sogyoke / gawa / dairinin / no / Saito / Jiro / bengoshi / ” is added, is selected as the image corresponding to the phrase “Ofuna / Kosan / no”.

[0050] On the other hand, in embodiment 2, correlations are calculated using the caption summaries obtained by summarizing the captions, as shown in Fig. 12 is shown. Fig. 12. Groups of nouns “Sait / Jiro / bengoshi” and “Mitsubishi / Taro / shacho”, which appear at the ends of the captions, are extracted by the caption summary unit 14 as the caption summaries. As a consequence, the calculation of a correlation between a disturbance part contained in each caption and each of the clauses of the article content is eliminated, and the determination of the correspondence is carried out more accurately.

[0051] As described above, the information processing device 10 according to embodiment 2 is configured to include a caption summarization unit that generates a caption summary by summarizing the caption. The matching unit determines a match between each of the sentence parts of the article content text and the images by calculating a correlation between the caption summary and each of the sentence parts of the article content text. Consequently, by generating a caption summary, a match between article content text displayed on the screen or read aloud and an image displayed on the screen can be correctly determined. Example 3

[0052] Fig. Figure 13 is a block diagram showing an example configuration of a voice guidance device 20 that uses an information processing device 10 according to embodiment 3 of the present invention. The information processing device 10 according to embodiment 3 has a configuration in which a dependency analysis unit 15 is connected to the information processing device 10 according to embodiment 3. Fig. 8 Example 2 has been added. Fig. 13 are the same or corresponding components as those in Fig. 8 are designated by the same reference symbols, and an explanation of the components is omitted below.

[0053] The dependency analysis unit 15 is provided by the in Fig. The processor 101 shown in Figure 2 implements a program stored in memory 102. Alternatively, the dependency analysis unit 15 can be replaced by the one shown in Figure 2. Fig. 3 Processing circuit 107 shown implemented.

[0054] Fig. Figure 14 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 3 of the present invention. Since the processes in steps ST1 to ST3 and ST5 are shown in Fig. 14 the same as those in steps ST1 to ST3 and ST5 in Fig. Since there are 9 steps, an explanation of these steps is omitted.

[0055] In step ST3 and the subsequent step ST6, the dependency analysis unit 15 establishes a dependency relationship between sentence parts of an article content text based on the sentence parts obtained by the sentence part acquisition unit 12 and divides the article content text into meaning blocks.

[0056] In step ST4B, based on caption summaries obtained through a caption summary unit 14 and the article content meaning blocks obtained through the dependency analysis unit 15, a correspondence determination unit 13 calculates a correlation between each of the caption summaries and each of the article content meaning blocks, thereby determining a correspondence between each of the article content meaning blocks and an image.

[0057] Fig. 15 is a view that is a result of a dependency analysis of the data in the data, performed by the dependency analysis unit 15 of the information processing unit 10 according to embodiment 3 of the present invention. Fig. The article shown in Figure 4 illustrates this. It is subsequently assumed that captions not shown in the drawings have been summarized to create caption summaries “Sato Hanako moto Boeisho (former Minister of Defense Hanako Sato)” and “Suzuki Ichiro moto Somusho (former Minister of Foreign Affairs and Communications Ichiro Suzuki)”.

[0058] The dependency analysis result has a tree structure in which the last sentence fragment "mitoshi / ." is the root, and this structure is called the dependency analysis tree. It is evident that when the Matching Unit 13 calculates a correlation between this dependency analysis tree and each of the image caption summaries, the following three sentence fragments, "Sato / Hanako / moto / Boeisho / ni / taishi / ," "Somusho / mo," and "Suzuki / Ichiro / shi / ga," which are each in bold, show high correlations. The Matching Unit 13 obtains a meaning block, which is a collection of sentence fragments above each of the sentence fragments with a high correlation in the dependency analysis tree; that is, it branches out extending in one direction away from the root and can, as a consequence, determine a match between the meaning block and an image. Fig. 15 is the image, to which the caption summary “Sato Hanako moto Boeisho (former Defense Minister Hanako Sato)” is attached, in accordance with the meaning block “Minshuto kenren o hihan shi nagara shutsuba hyomei shi ta doto syuin giin no Sato Hanako moto Boeisho ni taishi (in opposition to former Defense Minister Hanako Sato, who is a lower house member of the Minshu Party and who announced her candidacy while criticizing the Minshu Party Prefectural Federation,)”. Furthermore, the image, to which the caption summary “Suzuki Ichiro moto Somusho (former Minister of the Interior and Communications Ichiro Suzuki)” is attached, is brought into accordance with the meaning block “zen ken chiji de Somusho mo tsutome ta Suzuki Ichiro shi ga (Mr. Ichiro Suzuki, who is a former prefectural governor and also served as a minister of the Department of the Interior and Communications)”.

[0059] As a method of acquiring a meaning block, there are other methods, such as a method of dividing an article at the position of a period (.) or a comma (,). Furthermore, if no image has been brought into agreement with a meaning block, it is possible to use a method of continuously bringing an image into agreement with a clause in another meaning block preceding the meaning block, until the next time a clause with a high correlation with an image is determined.

[0060] In embodiment 2, a correlation is calculated with each caption summary for each clause of the article content, and a corresponding image is determined for each clause. However, when an image is matched to each clause, a situation arises where the time period for displaying the image becomes extremely short. In contrast, by performing a dependency analysis as in embodiment 3, a relationship between a modifying clause and a modified clause can be found, and a "meaning block" containing multiple clauses can be obtained. Therefore, it becomes possible to determine a match with an image for a longer sequence of clauses and for a sequence of clauses containing a less inaccurate meaning.

[0061] As mentioned above, the information processing device 10 according to embodiment 3 is configured to include: a caption summarization unit 14, which generates a caption summary by summarizing the caption; and a dependency analysis unit 15, which divides the article content text into meaning blocks by acquiring a dependency relationship between the sentence parts of the article content text. The correspondence determination unit 13 determines the correspondence between each of the meaning blocks of the article content text and the images by calculating a correlation between the caption summary and each of the meaning blocks of the article content text. As a consequence, the correspondence between an article content text displayed or read aloud on the screen and an image displayed on the screen can be determined more accurately.Furthermore, it is possible to avoid the time period for displaying an image becoming extremely short, and consequently, it is possible to improve visibility.

[0062] Although embodiment 3 shows the configuration in which the information processing device 10 contains the caption summary unit 14, the information processing device need not contain the caption summary unit 14. In this case, the correspondence determination unit 13 calculates a correlation between a caption and each of the meaning blocks of the article content text, thereby determining the correspondence between each of the meaning blocks of the article content text and an image. Example 4

[0063] Fig. Figure 16 is a block diagram showing an example configuration of a voice guidance device 20 that uses an information processing device 10 according to embodiment 4 of the present invention. The information processing device 10 according to embodiment 4 has a configuration in which a content text summary unit 16 is connected to the information processing device 10 of the [unclear text]. Fig. 13, embodiment 3, is added. Fig. 16 are the same or corresponding components as those in Fig. 13 are designated by the same reference symbols, and an explanation of these components is omitted below.

[0064] Content summary unit 16 is provided by the one in Fig. 2 implements processor 101, which executes a program stored in memory 102. As an alternative, the content text summary unit 16 is replaced by the one shown in Fig. 3 Processing circuit 107 shown implemented.

[0065] In embodiments 1 to 3, it is possible that when an article's content is summarized, a section corresponding to an image is not included in the summary because a match with an image caption is not taken into account. However, since such a section, to which an image is attached in an article, is considered important, it is preferable that a section corresponding to an image to be displayed is included in the article's summary.

[0066] Fig. Figure 17 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 4 of the present invention. Since the processes in steps ST1 to ST3, ST5 and ST6 in Fig. 17 the same as those such as steps ST1 to ST3, ST5 and ST6 in Fig. Since there are 14, an explanation of these steps is omitted.

[0067] In step ST6, the subsequent step ST7, the content text summary unit 16 generates an article content text summary by summarizing an article content text using the result of a dependency analysis obtained by the dependency analysis unit 15.

[0068] Fig. Figure 18 shows an example of an article to be processed by the information processing device 10 according to embodiment 4 of the present invention and an article content summary generated by the content text summary unit 16. The bolded part of the article content is the article content summary.

[0069] There are several methods for summarizing article content. For example, Content Summary Unit 16 calculates the importance levels of words in each sentence, defines the sum of the importance levels of the words in the sentence as the sentence's importance level, and extracts a string of characters within a specified number of characters from a sentence with a higher importance level. The TF-IDF value, for example, is used as the importance level of a word. Content Summary Unit 16 can be configured to increase the likelihood of a sentence with a higher correlation to a caption summary being included in the article content summary by increasing the importance level of each word in the caption summary obtained through Caption Summary Unit 14.With such a configuration, it becomes possible to facilitate the generation of an article content text summary that describes the content of an image.

[0070] In step ST4C, using the caption summaries obtained through the caption summary unit 14 and the article content summary obtained through the content summary unit 16, the matching unit 13 calculates a correlation between each of the caption summaries and each of the sentence parts contained in the article content summary, thereby determining a match between each of the sentence parts of the article content summary and an image.

[0071] In the speech guidance device 20, which contains the content text summary unit 16, a sentence that is read aloud by a speech output unit 22 or displayed on the screen by a display unit 23 can be an article content text or an article content text summary.

[0072] As described above, the information processing device 10 according to embodiment 4 is configured to include: a caption summary unit 14, which generates a caption summary by summarizing the caption; and a content summary unit 16, which generates an article content summary by summarizing the article content. The correspondence determination unit 13 determines a correspondence between each of the sentence parts contained in the article content summary and the images by calculating a correlation between the caption and each of the sentence parts contained in the article content summary.As a consequence, even if a caption and the article content text are combined, a match between the article content text displayed or read aloud on the screen and an image displayed on the screen can be correctly determined.

[0073] Although embodiment 4 shows the configuration in which the information processing device 10 contains the caption summary unit 14, the information processing device need not contain the caption summary unit 14. In such a case, the matching unit 13 calculates a correlation between a caption and each of the sentence parts contained in the article content summary, thereby determining a match between each of the sentence parts of the article content summary and an image.

[0074] Furthermore, the matching unit 13 can calculate a correlation between a caption or caption summary and each of the meaning blocks contained in the article content summary, thereby determining a match between each of the meaning blocks of the article content summary and an image. Example 5

[0075] Fig. Figure 19 is a block diagram showing an example configuration of a voice guidance device 20 that uses an information processing device 10 according to embodiment 5 of the present invention. While the information processing device 10 according to embodiment 5 has the same configuration as the information processing device 10 according to the embodiment 5 of the present invention, the information processing device 10 in embodiment 19 has the same configuration as the information processing device 10 according to the embodiment 5 of the present invention. Fig. In the embodiment 4 shown in Figure 16, the information processing device 10 according to embodiment 5 differs from that of embodiment 4 in that an arrow is directed from the content text summary unit 16 to the image caption summary unit 14.

[0076] Fig. Figure 20 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 5 of the present invention. Since the processes in steps ST1 to ST3, ST4C, ST6 and ST7 are shown in Fig. 20 the same as those in steps ST1 to ST3, ST4C, ST6 and ST7 in Fig. Since there are 17, an explanation of these steps is omitted.

[0077] In step ST7 and the subsequent step ST8, the caption summary unit 14 generates a caption summary by summarizing the caption using a result of the morphological analysis of each caption, obtained by a morphological analysis unit 11a, and an article content summary obtained by a content summary unit 16. The caption summary unit 14 generates the caption summary to include the article content summary based on, for example, TF-IDF or the like, as in embodiment 4.

[0078] As a consequence, in a case where the area for displaying a caption or the like is small, by increasing the significance level of each word or phrase contained in the article content summary, the likelihood is increased that the word or phrase will be included in a caption summary, and as a consequence, the correspondence between the currently read-aloud article content summary and an image becomes more accurate and easier to recognize.

[0079] Furthermore, the caption summary unit 14 outputs each caption summary to a control unit 21. When each of the sentence parts of the article content summary is read aloud, the control unit 21 controls the display unit 23 to display the image corresponding to the sentence part and a caption summary obtained by the caption summary unit 14 together on the screen. The display unit 23 displays the image and the caption summary under the control of the control unit 21.

[0080] As described above, in embodiment 5, the caption summary unit 14 is configured to generate a caption summary that includes an article content summary. As a consequence, even when a caption and article content are summarized, a match between the article content displayed or read aloud on the screen and an image displayed on the screen can be correctly determined.

[0081] In the confirmation of embodiment 5, the match determination unit 13 determines a match between each of the sentence parts of the article content text summary and an image by calculating a correlation between an image caption summary and each of the sentence parts contained in the article content text summary. Alternatively, the match determination unit 13 can be configured to determine the match between each of the meaning blocks of the article content text summary and an image by calculating a correlation between an image caption summary and each of the meaning blocks contained in the article content text summary.

[0082] Furthermore, although in the configuration of embodiment 5 the display unit 23 displays a caption summary generated by the caption summary unit 14, it is not necessarily required to display the caption summary on the screen.

[0083] Furthermore, in embodiments 2 to 4, the display unit 23 can be configured to display the caption summary on the screen, as in embodiment 5. Example 6

[0084] Fig. Figure 21 is a block diagram showing an example configuration of a voice guidance device 20 that uses an information processing device 10 according to embodiment 6 of the present invention. The information processing device 10 according to embodiment 6 has a configuration in which a second matching unit 17 is added to the information processing device 10 of the embodiment 6. Fig. 19, embodiment 5, is added. Fig. 21 are the same or corresponding components as those in Fig. 19 are designated by the same reference numerals, and an explanation of these components is omitted below. The content text summary unit 16 can generate an article content text summary such that it contains a caption summary as in embodiment 4. Alternatively, the caption summary unit 14 can generate a caption summary such that it contains an article content text summary as in embodiment 5.

[0085] The second conformity determination unit 17 is provided by the one in Fig. The processor 101 shown in Figure 2 implements a program stored in memory 102. Alternatively, the second matching unit 17 is implemented by the one shown in Figure 2. Fig. 3 Processing circuit 107 shown implemented.

[0086] Since in the configuration of embodiment 5 a match is determined between a sentence part or a block of meaning and an image, one consequence is a case in which the frequency of a display switch between images is high.

[0087] In embodiment 6, if several sentence parts exist within a single sentence and several images correspond to that single sentence, the second matching unit 17 ascertains the degree of meaning of each of the sentence parts and, as an image corresponding to the sentence, determines an image corresponding to a sentence part with a higher degree of meaning. A unit with which an image is matched is not limited to a single sentence and can be a block of sentences with one or more sentences, such as a paragraph, or it can be an article content summary generated by summarizing article content.

[0088] Fig. Figure 22 is a flowchart illustrating the operation of the information processing device 10 according to embodiment 6 of the present invention. Since the processes in steps ST1 to ST8 are described in Fig. 22 the same as those in steps ST1 to ST8 in Fig. Since there are 20, an explanation of these steps is omitted.

[0089] In step ST9, following step ST4C, the second matching unit 17 checks, based on the match between each of the sentence fragments in the article content text and an image (the match being determined by a matching unit 13), whether there are multiple images, each containing a match with a sentence block, that contain one or more sentences in the article content text. If there are multiple images, each containing a match with such a sentence block, the second matching unit 17 determines the meaning level of each of the sentence fragments in the article content text based on word meaning level information that indicates the meaning level of each word.Then, the second matching unit 17 determines that a single image corresponding to the clause with the highest degree of importance in the aforementioned sentence block is an image that matches the aforementioned sentence block, and outputs the match between the aforementioned sentence block and the single image. Conversely, if the number of images that match the aforementioned sentence block is one, the second matching unit 17 outputs the match between the aforementioned sentence block and this image.

[0090] The second matching unit 17 can acquire the meaning level of each meaning block, instead of acquiring the meaning level of each sentence part.

[0091] The word meaning level information is, for example, the TF-IDF value. In this case, the second matching unit 17 calculates the meaning levels of the words contained in each clause or meaning block and determines the sum of the meaning levels of the words contained in the clause or meaning block as the meaning level of the clause or meaning block.

[0092] Furthermore, word meaning level information can be determined by other means, for example, by setting the meaning levels of words contained in a previously processed article using historical information to be higher or lower. Additionally, word meaning level information can be determined using information generated by acquiring data about user-generated speech, which lists keywords preferred by the user, by setting the meaning levels of words contained as keywords to a higher value.Since words that appear in previously viewed articles—that is, words that appear in articles the user has frequently viewed in the past—are considered content of interest to the user, increasing the meaning of such a word increases the likelihood that the user will select a section of content of interest. On the other hand, users don't necessarily need to repeatedly view the same articles; they may also want to read an article with new content they haven't seen before.In such a case, by lowering the importance levels of words contained in a previously viewed article, the likelihood of the user selecting words they already know decreases, while increasing the likelihood of selecting content they are unfamiliar with. Based on the above considerations, the importance levels of words contained in a previously viewed article can be increased or decreased according to the purpose. As a consequence, an image whose display is suitable for the user can be preferentially shown.

[0093] Fig. Figure 23 is a view showing an example of the word meaning level information in the information processing device 10 according to embodiment 6 of the present invention. In this word meaning level information, the word meaning level of "Sato Hanako" is determined as "6", and the word meaning level of "Suzuki Ichiro" is determined as "2".

[0094] For example, a case is assumed in which a single image represents a single sentence in the Fig. The image, to which the caption summary “Sato Hanako moto Boeisho (former Defense Minister Hanako Sato)” is attached, is in accordance with the meaning block “Minshuto kenren o hihan shi nagara shutsuba hyomei shi ta doto syuin giin no Sato Hanako moto Boeisho ni taishi, (in opposition to former Defense Minister Hanako Sato, a lower house member of the Minshu Party, who announced her candidacy while criticizing the Minshu Party Prefectural Federation,)” through the conformity determination unit 13.Furthermore, the image, to which the caption summary “Suzuki Ichiro moto Somusho (former Minister of the Interior and Communications Ichiro Suzuki)” is attached, is in accordance with the meaning block “zen ken chiji de Somusho mo tsutome ta Suzuki Ichiro shi ga (Mr. Ichiro Suzuki, who is a former prefectural governor and also served as a minister of the Department of the Interior and Communications)”. That is, two images are in accordance with the one in . Fig. 15 individual sentences shown. Since the in Fig. Given that 23 word meaning levels are used, and the meaning level of the meaning block containing "Sato Hanako" is higher than that of the meaning block containing "Suzuki Ichiro", the image with the caption "Sato Hanako moto Boeisho (former Defense Minister Hanako Sato)" is selected as the image corresponding to this single sentence.

[0095] As described above, the information processing device 10 according to embodiment 6 is configured to include a second matching unit 17. When several images, each matching a sentence block containing several sentences in the article content text, are identified, the unit determines the meaning level of each sentence part or meaning block of the article content text based on word meaning level information indicating the meaning level of each word. It then identifies an image corresponding to a sentence part or meaning block with the highest meaning level in that sentence block as the image that matches the sentence block. As a consequence, a single image most suitable for a sentence or text can be determined.Furthermore, it is possible to avoid the time period for displaying an image becoming extremely short, and consequently it is possible to improve visibility. Example 7

[0096] Fig. Figure 24 is a block diagram showing an example configuration of a speech guidance device 20 that uses an information processing device 10 according to embodiment 7 of the present invention. The information processing device 10 according to embodiment 7 has a configuration in which a speech acquisition unit 24 and a speech recognition unit 25 are added to the information processing device 10 according to embodiment 7. Fig. 1. The following are added to the embodiment shown in example 1. Fig. 24 are the same or corresponding components as those in Fig. 1 are designated by the same reference symbols, and an explanation of these components is omitted below.

[0097] In embodiment 7, a string of characters obtained by performing speech recognition of speech data is entered into the information processing device 10 instead of an article content text.

[0098] The Fig. 25 and Fig. Figures 26 are views showing examples of a hardware configuration of the voice guidance device 20, which uses the information processing device 10 according to embodiment 7 of the present invention. Fig. 25 and Fig. 26 are the same or corresponding components as those in Fig. 2 are designated by the same reference symbols, and an explanation of these components is omitted below.

[0099] In the Fig. In the hardware configuration example shown in Figure 25, the speech acquisition unit 24 in the information processing device 10 is a sensor 108, such as a microphone, that detects a sound. The speech recognition unit 25 in the information processing device 10 is implemented by a processor 101, which executes a program stored in the memory 102.

[0100] Alternatively, the speech recognition unit 25 can be implemented by the processing circuit 107, which is hardware for dedicated use, as in the case of the one in Fig. 26 hardware configuration example shown.

[0101] The speech guidance device 20 contains the speech acquisition unit 24, which acquires speech uttered by a user as speech data, and the speech recognition unit 25, which generates a character string by performing speech recognition on the speech data acquired by the speech acquisition unit 24. The character string generated by the speech recognition unit 25 is input into the morphological analysis unit 11b of the information processing device 10. The information processing device 10 performs the same process on the character string obtained by performing speech recognition on the speech data as it does on an article content text.

[0102] As a consequence, the voice guidance device 20 can display or emphasize an image corresponding to what the user has uttered. This voice guidance device 20 can be used for a presentation or the like. Furthermore, if information equipment is controlled based on a speech recognition result and a button is selected by voice, the voice guidance device 20 can be used for the purpose of highlighting a corresponding button image.

[0103] In embodiments 1 to 6, it is also possible to provide a configuration in which a character string obtained by speech recognition of speech data is entered into the information processing device 10 instead of an article content text.

[0104] Furthermore, in the configuration of each of embodiments 1 to 7, unit 11a for morphological analysis performs a morphological analysis of an image caption, and unit 11b for morphological analysis performs a morphological analysis of an article body text. Alternatively, a configuration can be provided in which a single unit for morphological analysis performs a morphological analysis of both an image caption and an article body text.

[0105] Furthermore, although in embodiments 1 to 7 the information processing device 10 and the speech guidance device 20 are explained, which have Japanese as their target language, the target language of the information processing device 10 and the speech guidance device 20 is not limited to Japanese.

[0106] Furthermore, in the configuration of each of embodiments 1 to 7, the information processing device 10 is used for the voice guidance device 20, which reads aloud an article content text. Alternatively, a configuration can be provided in which the information processing device 10 is used for an information display device that displays an article content text on a screen.

[0107] Furthermore, in the configuration of each of the embodiments 1 to 7, the voice guidance device 20 includes the information processing device 10. Alternatively, the information processing device 10 can be configured separately from the voice guidance device 20 and can be connected to the voice guidance device 20 in order to be able to send and receive information to each other.

[0108] Unlike the configurations described above, any combination of the embodiments described above can be made, various modifications can be made to each component of each embodiment, and each component of each embodiment can be omitted within the scope of the invention. COMMERCIAL APPLICABILITY

[0109] Since the information processing device according to the present invention correctly determines a correspondence between an article content text that is displayed or read aloud on a screen and an image displayed on the screen, it is suitable for use in an information display device that displays information on a screen and a voice guidance device that reads information aloud, wherein these devices include a vehicle navigation system, a display device in an elevator, a smartphone and a device that reads aloud a television program, etc. REFERENCE MARK LIST

[0110] 10 Information processing device, 11a Morphological analysis unit, 11b Morphological analysis unit, 12 Sentence part acquisition unit, 13 Matching unit, 14 Caption summarization unit, 15 Dependency analysis unit, 16 Content text summarization unit, 17 Second matching unit, 20 Speech guidance device, 21 Control unit, 22 Speech output unit, 23 Display unit, 24 Speech acquisition unit, 25 Speech recognition unit, 101 Processor, 102 Memory, 103 Disk drive, 104 Loudspeaker, 105 Display device, 106 Bus, 107 Processing circuit, and 108 Sensor.

Claims

[1] Device (10) for determining the correspondence between an article content text that is displayed or read aloud on a screen and for displaying the image on the screen, which comprises: a unit (11a, 11b) for morphological analysis to carry out a morphological analysis in order to divide each of an article content text contained in an article and a caption of each of images into morphemes; a sentence part acquisition unit (12) that divides the article content text into sentence parts based on a result of the morphological analysis carried out by the morphological analysis unit (11b); a caption summary unit (14) that generates a caption summary by summarizing the caption; a correspondence determination unit (13) that determines a correspondence between each of the sentence parts of the article content text and the images by calculating a correlation between the image caption summary and each of the sentence parts of the article content text on the basis of the result of the morphological analysis carried out by the unit (11a) for morphological analysis; an output unit (22, 23) that displays the article content text on the screen or reads the article content text aloud, and that displays the corresponding images; and a control unit (21) that controls the output unit (22, 23) to display the image corresponding to the sentence part on the basis of the match obtained by the match determination unit (13) when a sentence part of the article content text is read aloud or displayed. [2] Device (10) for determining the correspondence between an article content text that is displayed or read aloud on a screen and for displaying the image on the screen, which comprises: a unit (11a, 11b) for morphological analysis which performs a morphological analysis to divide each of an article content text contained in an article and a caption of each of images into morphemes; a sentence part acquisition unit (12) that divides the article content text into sentence parts based on a result of the morphological analysis carried out by the unit (11b) for morphological analysis; a dependency analysis unit (15) that divides the article content text into meaning blocks by establishing a dependency relationship between the sentence parts of the article content text; a correspondence determination unit (13) that determines a correspondence between each of the meaning blocks of the article content text and the images by calculating a correlation between the image caption and each of the meaning blocks of the article content text; an output unit (22, 23) that displays the article content text on the screen or reads the article content text aloud, and that displays the corresponding images; and a control unit (21) that controls the output unit (22, 23) to display the image corresponding to the sentence part on the basis of the match obtained by the match determination unit (13) when a sentence part of the article content text is read aloud or displayed. [3] Device (10) according to claim 2, which further comprises: a caption summary unit (14) which generates a caption summary by summarizing the caption, wherein the correspondence determination unit (13) determines a correspondence between each of the meaning blocks of the article content text and the images by calculating a correlation between the caption summary and each of the meaning blocks of the article content text. [4] Device (10) for determining the correspondence between an article content text that is displayed or read aloud on a screen and for displaying the image on the screen, which comprises: a unit (11a, 11b) for morphological analysis which performs a morphological analysis to divide each of an article content text contained in an article and a caption of each of images into morphemes; a sentence part acquisition unit (12) that divides the article content text into sentence parts based on a result of the morphological analysis carried out by the unit (11b) for morphological analysis; a content text summary unit (16) that generates an article content text summary by summarizing the article content text; a caption summary unit (14) that generates a caption summary by summarizing the caption; a correspondence determination unit (13) that determines a correspondence between each of the sentence parts included in the article content text summary and the image by calculating a correlation between the image caption summary and each of the sentence parts included in the article content text summary based on the result of the morphological analysis carried out by the morphological analysis unit (11a); an output unit (22, 23) that displays the article content text on the screen or reads the article content text aloud, and that displays the corresponding images; and a control unit (21) that controls the output unit (22, 23) to display the image corresponding to the sentence part on the basis of the match obtained by the match determination unit (13) when a sentence part of the article content text is read aloud or displayed. [5] Device (10) according to claim 4, wherein the caption summary unit (14) generates the caption summary in such a way that it contains the article content text summary. [6] Device (10) for determining the correspondence between an article content text that is displayed or read aloud on a screen and for displaying the image on the screen, comprising: a unit (11a, 11b) for morphological analysis, which performs a morphological analysis to divide each of an article content text contained in an article and a caption of each of images into morphemes; a sentence part acquisition unit (12) that divides the article content text into sentence parts based on a result of the morphological analysis carried out by the unit (11b) for morphological analysis; and a correspondence determination unit (13) that determines a correspondence between each of the sentence parts of the article content text and the images by calculating a correlation between the image caption and each of the sentence parts of the article content text on the basis of the result of the morphological analysis carried out by the unit (11a) for morphological analysis; a second matching unit (17) which, if there are multiple images, each containing a match with a sentence block containing one or more sentences in the article content text, obtains a meaning level for each of the sentence parts of the article content text or a meaning level for each of the meaning blocks of the article content text based on word meaning level information showing a meaning level for each word, and identifies an image corresponding to a sentence part or meaning block with the highest meaning level in the meaning block as an image matching the sentence block; and an output unit (22, 23) that displays the article content text on the screen or reads the article content text aloud, and that displays the corresponding images; and a control unit (21) that controls the output unit (22, 23) to display the image corresponding to the sentence part on the basis of the match obtained by the match determination unit (13) when a sentence part of the article content text is read aloud or displayed.

Images