Information processing device, output method, and output program

The information processing device addresses the challenge of redundant and grammatically inappropriate voice recognition outputs by generating summary sentences from voice data, enhancing user understanding through concise and coherent outputs.

WO2024241570A1PCT designated stage expired Publication Date: 2024-11-28MITSUBISHI ELECTRIC CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/019496
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing voice recognition technologies struggle with outputting information that is redundant, grammatically inappropriate, or repetitive, making it difficult for users to understand the recognized content, especially in scenarios like train announcements or spoken language in tourist spots.

Method used

An information processing device that acquires voice data, performs speech recognition, and generates a summary sentence using word vectors or learned models to extract keywords, which are then applied to templates to create a concise and coherent output, either displayed or spoken back to the user.

Benefits of technology

The device effectively outputs information that is easier for users to understand by condensing redundant or repetitive content into clear, grammatically correct summaries, improving user comprehension.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023019496_28112024_PF_FP_ABST
    Figure JP2023019496_28112024_PF_FP_ABST
Patent Text Reader

Abstract

An information processing device (100) comprises: an acquisition unit (120) that acquires voice data; a voice recognition execution unit (130) that executes voice recognition by using the voice data; a generation unit (140) that generates summary text of the voice data by using a keyword in data acquired by the voice recognition or a word vector based on said data; and an output unit (150) that outputs the summary text.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device, output method, and output program

[0001] The present disclosure relates to an information processing device, an output method, and an output program.

[0002] Speech recognition technology is known. For example, Patent Document 1 describes a technology related to speech recognition technology.

[0003] International Publication No. 2017 / 065266

[0004] An utterance may be speech-recognized, and the speech recognition result may be displayed or output as audio. The content of the utterance may have the following characteristics: The content is redundant The same words are repeated The content is not grammatically correct spoken language The content of the utterance may have such characteristics. Therefore, if an utterance having such characteristics is speech-recognized, and the speech recognition result is displayed or output as audio, it may be difficult for the user to understand the information.

[0005] The objective of the present disclosure is to output information that is easily understandable to the user.

[0006] According to one aspect of the present disclosure, there is provided an information processing device including: an acquisition unit that acquires voice data; a voice recognition execution unit that performs voice recognition using the voice data; a generation unit that generates a summary of the voice data using keywords in the data obtained by the voice recognition or word vectors based on the data; and an output unit that outputs the summary.

[0007] According to the present disclosure, it is possible to output information that is easily understandable to the user.

[0008] 1A and 1B are diagrams illustrating a providing system according to a first embodiment; a diagram illustrating hardware included in an information processing device according to the first embodiment; a block diagram illustrating functions of the information processing device according to the first embodiment; a flowchart illustrating an example of processing executed by the information processing device according to the first embodiment; a flowchart illustrating an example of summary generation processing according to the first embodiment; a diagram illustrating an example of a management table according to the first embodiment; a flowchart illustrating an example of summary generation processing according to a first modification of the first embodiment; a flowchart illustrating an example of summary generation processing according to a second modification of the first embodiment; a flowchart illustrating an example of summary generation processing according to a third modification of the first embodiment; a diagram illustrating a providing system according to a third embodiment; a flowchart illustrating an example of processing executed by the information processing device according to the third embodiment; a block diagram illustrating functions of an information processing device according to a fourth embodiment; a flowchart illustrating an example of processing executed by the information processing device according to the fourth embodiment; a diagram illustrating an example of a management table according to the fourth embodiment;

[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1. Figures 1(A) and (B) are diagrams showing a provision system according to embodiment 1. Figure 1(A) shows the provision system. The provision system includes a microphone 10, a display 20, a speaker 30, and an information processing device 100. The microphone 10, the display 20, the speaker 30, and the information processing device 100 are connected via a network. The network may be a wired network or a wireless network.

[0011] The information processing device 100 is a device that executes an output method. The processing of the information processing device 100 will be briefly described. The information processing device 100 acquires audio data from a microphone 10. For example, the sound of the audio data may be an audio announcement on a train or in a station, or an audio announcement at a tourist spot or commercial facility. The information processing device 100 generates a summary of the audio data based on the audio data. The information processing device 100 may output the summary to a display 20. The information processing device 100 may output an audio signal of the summary to a speaker 30. This provides the summary to a user. The information processing device 100 may also translate the summary. The information processing device 100 may output the translated summary to the display 20, or may output an audio signal of the translated summary to the speaker 30.

[0012] Here, the provision system may include an output control device 200. FIG. 1B shows that the output control device 200 is connected to the display 20, the speaker 30, and the information processing device 100. The information processing device 100 may output a summary or an audio signal of the summary to the display 20 and the speaker 30 via the output control device 200. The output control device 200 may also have a translation function. If the output control device 200 has the translation function, the output control device 200 translates the input summary. The output control device 200 outputs the translated summary to the display 20. The output control device 200 also outputs an audio signal of the translated summary to the speaker 30.

[0013] Next, the hardware of the information processing device 100 will be described. Fig. 2 is a diagram showing the hardware of the information processing device of embodiment 1. The information processing device 100 is a computer. The information processing device 100 has a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0014] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0015] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0016] Next, a description will be given of functions of the information processing device 100. Fig. 3 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 100 has a storage unit 110, an acquisition unit 120, a speech recognition execution unit 130, a generation unit 140, and an output unit 150.

[0017] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, the speech recognition execution unit 130, the generation unit 140, and the output unit 150 may be realized by a processing circuit. Furthermore, some or all of the acquisition unit 120, the speech recognition execution unit 130, the generation unit 140, and the output unit 150 may be realized as program modules executed by the processor 101. For example, the program executed by the processor 101 is also referred to as an output program. For example, the output program is recorded on a recording medium.

[0018] The storage unit 110 stores various information. The functions of the acquisition unit 120, the speech recognition execution unit 130, the generation unit 140, and the output unit 150 will be described in detail later.

[0019] Next, the processing executed by the information processing device 100 will be described using a flowchart. FIG. 4 is a flowchart showing an example of processing executed by the information processing device of embodiment 1. (Step S11) The acquisition unit 120 acquires audio data. For example, the acquisition unit 120 acquires audio data from the microphone 10. The acquisition unit 120 may also acquire audio data from the microphone 10 via an external device. The external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server. Note that an illustration of the external device is omitted. The audio data may also be called an audio signal.

[0020] (Step S12) The speech recognition execution unit 130 performs speech recognition using the speech data. As a result, the information processing device 100 obtains data that is a character string. (Step S13) The information processing device 100 performs a summary generation process. For example, the generation unit 140 generates a summary of the speech data using keywords in the data obtained by speech recognition or word vectors based on the data. (Step S14) The output unit 150 outputs the summary. For example, the output unit 150 outputs the summary to the display 20. Also, for example, the output unit 150 outputs the summary of the audio signal to the speaker 30. Also, for example, the output unit 150 outputs the summary to the output control device 200.

[0021] Next, an example of a summary generation process is shown. FIG. 5 is a flowchart illustrating an example of the summary generation process according to the first embodiment. For example, the process in FIG. 5 corresponds to step S13. (Step S21) The generation unit 140 extracts keywords from data obtained by speech recognition. The extraction process will be described. For example, the generation unit 140 extracts keywords from the data using a rule base that indicates keywords. Examples of methods using a rule base include a method using a dictionary or a method using regular expressions. Furthermore, for example, the generation unit 140 extracts keywords from the data using a trained model. More specifically, when the generation unit 140 inputs the data into the trained model, the trained model outputs keywords. Note that the trained model can be created by machine learning, such as a deep neural network (DNN). For example, in machine learning, a large number of character strings are prepared as input data. Training data is prepared in which keywords corresponding to character strings in the input data are used as correct labels. Then, using the input data and training data, a trained model is created by machine learning so that the keywords that are the output data of the trained model match the correct labels.

[0022] (Step S22) The generation unit 140 detects a template to which the extracted keywords are to be applied. The template detection process will be described with reference to the drawings.

[0023] 6 is a diagram illustrating an example of a management table according to the first embodiment. For example, the management table 111 is stored in the storage unit 110. The management table 111 has fields for item number, keyword, and template. For example, if the keyword is "restart operation," the generation unit 140 detects the template for the record with item number 1.

[0024] The management table 111 may also be stored in an external device. If the management table 111 is stored in an external device, the generation unit 140 acquires the management table 111 from the external device via the acquisition unit 120.

[0025] Furthermore, the generation unit 140 may detect a template using a trained model. Specifically, when the generation unit 140 inputs a keyword to the trained model, the trained model outputs an item number in the management table 111. For example, the trained model outputs "item number 1." The generation unit 140 detects the template of the output item number as a template to which the keyword should be applied. The trained model can be created, for example, by machine learning such as DNN. For example, in machine learning, a large number of keywords are prepared as input data. Training data is prepared in which templates corresponding to the keywords in the input data are used as correct labels. Then, the trained model is created by performing machine learning using the input data and the training data so that the templates, which are output data of the trained model, match the correct labels.

[0026] (Step S23) The generation unit 140 generates a summary by applying the keywords to a template. For example, the generation unit 140 generates the summary "Train service is expected to resume."

[0027] Here, the content of an utterance may have the following characteristics: The content is redundant The same words are repeated The utterance is not grammatically correct spoken language The content of an utterance may have such characteristics. Therefore, if an utterance having such characteristics is speech-recognized and the speech recognition result is displayed or output as voice, it may be difficult for the user to understand the information.

[0028] According to the first embodiment, the information processing device 100 generates a summary based on the content of an utterance and outputs the summary. As a result, even if the content of an utterance has the above-mentioned characteristics, the information processing device 100 can output information that can be easily understood by the user.

[0029] Variation 1 of Embodiment 1. In Variation 1 of Embodiment 1, a case where a summary is generated by a different method will be described. Figure 7 is a flowchart showing an example of a summary generation process according to Variation 1 of Embodiment 1. For example, the process in Figure 7 corresponds to step S13.

[0030] (Step S31) The generation unit 140 extracts keywords from the data obtained by speech recognition.

[0031] (Step S32) The generation unit 140 uses the keyword to search for past cases that match the keyword. Note that past cases are information that have been output as summaries in the past. Here, the storage unit 110 or the external device stores multiple past cases. For example, if the storage unit 110 stores multiple past cases, the generation unit 140 searches for a past case that matches the keyword from the multiple past cases stored in the storage unit 110. Also, for example, if the external device stores multiple past cases, the generation unit 140 accesses the external device and searches for a past case that matches the keyword from the multiple past cases stored in the external device.

[0032] (Step S33) The generation unit 140 generates a summary of the past cases obtained by the search. This summary may be expressed as follows: The generation unit 140 outputs the searched past cases as a summary.

[0033] According to the first modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0034] Variation 2 of Embodiment 1 In Variation 2 of Embodiment 1, a case where a summary is generated by a different method will be described. Fig. 8 is a flowchart showing an example of a summary generation process according to Variation 2 of Embodiment 1. For example, the process in Fig. 8 corresponds to step S13.

[0035] (Step S41) The generation unit 140 extracts keywords from the data obtained by speech recognition. (Step S42) The acquisition unit 120 acquires the trained model from the storage unit 110 or an external device. (Step S43) The generation unit 140 generates a summary using the keywords and the trained model. Specifically, when the generation unit 140 inputs keywords into the trained model, the trained model outputs a summary. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large number of keywords are prepared as input data. Training data is prepared in which the summary corresponding to the keywords in the input data is used as a correct label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary, which is the output data of the trained model, matches the correct label.

[0036] According to the second modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0037] Variation 3 of Embodiment 1 In Variation 3 of Embodiment 1, a case where a summary is generated by a different method will be described. Fig. 9 is a flowchart showing an example of a summary generation process according to Variation 3 of Embodiment 1. For example, the process in Fig. 9 corresponds to step S13.

[0038] (Step S51) The acquisition unit 120 acquires the trained model from the storage unit 110 or an external device. (Step S52) The generation unit 140 generates a summary using data obtained by speech recognition and the trained model. In particular, when the generation unit 140 inputs the data into the trained model, the trained model outputs the summary. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large number of character strings are prepared as input data. Training data is prepared in which summary sentences corresponding to character strings in the input data are used as correct labels. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentences, which are output data of the trained model, match the correct labels.

[0039] According to the third modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0040] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. In the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.

[0041] First, a speaker may give a meaningfully connected speech in multiple parts. Therefore, when a summary is generated based on the speech data of one speech, the accuracy of the generated summary may be low. Therefore, in the second embodiment, a case where a more accurate summary is generated will be described.

[0042] 10 is a block diagram showing the functions of the information processing device of embodiment 2. The information processing device 100 further includes a determination unit 160 and a combination unit 170. Part or all of the determination unit 160 and the combination unit 170 may be realized by a processing circuit. Also, part or all of the determination unit 160 and the combination unit 170 may be realized as a program module executed by the processor 101. The functions of the determination unit 160 and the combination unit 170 will be described later.

[0043] Next, the processing executed by the information processing device 100 will be described using a flowchart. Fig. 11 is a flowchart showing an example of the processing executed by the information processing device of embodiment 2. The processing in Fig. 11 differs from the processing in Fig. 4 in that steps S11a and 12a to 12d are executed. Therefore, steps S11a and 12a to 12d will be described in Fig. 11. Description of the processing other than steps S11a and 12a to 12d will be omitted.

[0044] (Step S11a) The acquisition unit 120 acquires voice data for one episode. (Step S12a) The determination unit 160 determines whether the voice data acquired in step S11a is the first voice data of the same person. If the acquired voice data is the first voice data, the determination unit 160 stores data obtained by voice recognition in the storage unit 110. Then, the process proceeds to step S11a. If the acquired voice data is voice data of the same person, the process proceeds to step S12b. Note that the determination unit 160 can determine whether the acquired voice data is voice data of the same person by analysis.

[0045] (Step S12b) The determination unit 160 determines whether there is a semantic connection between the data obtained by speech recognition and the data stored in the storage unit 110. For example, the determination unit 160 determines whether there is a semantic connection using a rule base. For example, a method using a rule base is a method using a dictionary or a method using regular expressions. Furthermore, for example, the determination unit 160 vectorizes the data obtained by speech recognition and the data stored in the storage unit 110 and compares the two vectorized data to determine whether there is a semantic connection. If there is a semantic connection, the process proceeds to step S12c. If there is no semantic connection, the process proceeds to step S12d.

[0046] (Step S12c) The combining unit 170 combines the data obtained by the speech recognition with the data stored in the storage unit 110. The combining unit 170 stores the combined data in the storage unit 110. Then, the process proceeds to step S11a. (Step S12d) The acquiring unit 120 acquires the combined data from the storage unit 110. Note that the combined data is data in which multiple pieces of audio data are combined and in which content that has a connection in meaning is combined. Note that if the combined data is not stored in the storage unit 110, the generating unit 140 uses audio data for one episode.

[0047] The generation unit 140 generates a summary using the combined data. For example, in step S21, the generation unit 140 extracts keywords from the combined data. The processing from step S22 onwards is the same as the processing in FIG. 5.

[0048] According to the second embodiment, the information processing device 100 generates a summary based on a plurality of utterances that are semantically related, thereby enabling the information processing device 100 to generate a more accurate summary.

[0049] Furthermore, there are cases where a speaker inverts a sentence and utters multiple utterances that are connected in meaning. Even when a speaker speaks in this manner, the second embodiment is effective.

[0050] Here, the external device may execute steps S11a, S12, and S12a to S12c. If the external device executes the processes, in step S12d, the acquiring section 120 acquires the combined data from the external device.

[0051] The second embodiment can be combined with the first modification of the first embodiment. Specifically, in step S31, the generation unit 140 extracts keywords from the combined data. The processes from step S32 onward are the same as those in FIG. 7.

[0052] The second embodiment can be combined with the second modification of the first embodiment. Specifically, in step S41, the generation unit 140 extracts keywords from the combined data. The processes from step S42 onward are the same as those in FIG. 8 .

[0053] The second embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary using the combined data and the trained model. In particular, when the generation unit 140 inputs the combined data to the trained model, the trained model outputs a summary. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large amount of combined data is prepared as input data. Training data is prepared in which the summary corresponding to the combined data in the input data is labeled as a correct answer. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary, which is output data of the trained model, matches the correct answer label.

[0054] Embodiment 3 Next, embodiment 3 will be described. In embodiment 3, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 3, descriptions of matters common to embodiment 1 will be omitted.

[0055] 12 is a diagram showing a provision system according to the third embodiment. The provision system further includes a server 300. The information processing device 100 and the server 300 communicate with each other via a network. The network may be a wired network or a wireless network.

[0056] The server 300 stores supplemental information when generating a summary. The supplemental information is useful information. For example, the supplemental information may include a character string indicating the content of the useful information. For example, the server 300 stores information on SNS (Social Networking Service). For example, when a summary about a train is generated, the information may be information such as the reason for the train stopping or the reason for the delay. The server 300 performs filtering and extracts useful information from the SNS information. This allows the server 300 to store the supplemental information, which is useful information. The server 300 periodically updates the supplemental information. The server 300 may also periodically transmit the supplemental information to the information processing device 100. When the acquisition unit 120 of the information processing device 100 acquires the supplemental information, the acquisition unit 120 stores the supplemental information in the storage unit 110.

[0057] Next, the processing executed by the information processing device 100 will be described using a flowchart. Fig. 13 is a flowchart showing an example of the processing executed by the information processing device of embodiment 3. The processing in Fig. 13 differs from the processing in Fig. 4 in that step S12e is executed. Therefore, step S12e will be described in Fig. 13. Further, a description of the processing other than step S12e will be omitted.

[0058] (Step S12e) The acquisition unit 120 acquires supplemental information related to the data obtained by speech recognition from the storage unit 110 or the server 300. This sentence may be expressed as follows: The acquisition unit 120 acquires supplemental information related to words included in the data obtained by speech recognition from the storage unit 110 or the server 300. For example, if the data includes a route name, the acquisition unit 120 acquires supplemental information related to the route name.

[0059] The generation unit 140 generates a summary using the data obtained by speech recognition and the supplemental information. For example, in step S21, the generation unit 140 extracts keywords from the data and the supplemental information. The processing from step S22 onwards is the same as the processing in FIG. 5.

[0060] Here, for example, the content of the utterance may lack some information due to a speech recognition error in the voice announcement, an incorrect utterance, or the like. Furthermore, for example, the content of the utterance may lack a prerequisite due to an omission of the utterance, or the like. In such cases, the accuracy of a summary generated based solely on the content of the utterance may be low. Therefore, the information processing device 100 uses supplemental information to supplement the missing information in the content of the utterance. This allows a more accurate summary to be generated.

[0061] A specific example will be given. The data obtained by speech recognition includes "Tokaido Line." The content of the utterance is assumed to lack the cause of the delay on the "Tokaido Line." The supplemental information indicates the cause of the delay on the "Tokaido Line." The generation unit 140 generates a summary using the data obtained by speech recognition and the supplemental information. When generating the summary, the generation unit 140 supplements the cause of the delay on the "Tokaido Line" to generate the summary. This results in a more accurate summary.

[0062] Therefore, according to the third embodiment, the information processing device 100 can generate a more accurate summary.

[0063] The third embodiment can be combined with the first modification of the first embodiment. Specifically, in step S31, the generation unit 140 extracts keywords from the data obtained by speech recognition and the supplemental information. The processes from step S32 onward are the same as those in FIG. 7 .

[0064] The third embodiment can be combined with the second modification of the first embodiment. Specifically, in step S41, the generation unit 140 extracts keywords from the data obtained by speech recognition and the supplemental information. The processes from step S42 onward are the same as those in FIG. 8 .

[0065] The third embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary using data obtained by speech recognition, supplemental information, and a trained model. In particular, when the generation unit 140 inputs the data and supplemental information into the trained model, the trained model outputs a summary. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large amount of character strings and supplemental information are prepared as input data. Training data is prepared in which summary sentences corresponding to the character strings and supplemental information in the input data are used as correct labels. Then, the trained model is created by performing machine learning using the input data and training data so that the summary sentences, which are output data of the trained model, match the correct labels.

[0066] The third embodiment can be combined with the second embodiment. Specifically, the generating unit 140 generates a summary using the combined data and the supplemental information.

[0067] The third embodiment can be combined with the first embodiment or the first to third modifications of the first embodiment, and the second embodiment.

[0068] Embodiment 4 Next, embodiment 4 will be described. In embodiment 4, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 4, description of matters common to embodiment 1 will be omitted.

[0069] 14 is a block diagram showing the functions of an information processing device according to the fourth embodiment. The information processing device 100 further includes an analysis unit 180. Part or all of the analysis unit 180 may be realized by a processing circuit. Alternatively, part or all of the analysis unit 180 may be realized as a program module executed by the processor 101. The functions of the analysis unit 180 will be described later.

[0070] Next, the processing executed by the information processing device 100 will be described using a flowchart. Fig. 15 is a flowchart showing an example of processing executed by the information processing device of embodiment 4. The processing in Fig. 15 differs from the processing in Fig. 4 in that step S12f is executed. Therefore, step S12f will be described in Fig. 15. Further, description of the processing other than step S12f will be omitted.

[0071] (Step S12f) The analysis unit 180 uses the data obtained by speech recognition to analyze whether the content of the data is of high urgency. For example, the analysis unit 180 performs the analysis using a rule base. Examples of methods using a rule base include a dictionary method and a regular expression method. Furthermore, for example, the analysis unit 180 performs the analysis using a trained model. For example, high urgency content is content indicating the occurrence of an accident, fire, disaster, etc. For example, if the content of the data indicates the occurrence of an accident, fire, disaster, etc., the analysis unit 180 outputs information indicating a high urgency. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large number of keywords are prepared as input data. Training data is prepared in which the level of urgency corresponding to the keywords in the input data is used as a correct label. Then, the trained model is created by performing machine learning using the input data and the training data so that the level of urgency, which is the output data of the trained model, matches the correct label.

[0072] Furthermore, the analysis unit 180 may use data obtained by speech recognition to analyze the level of urgency corresponding to the content of the data. For example, if the content of the data includes "earthquake" or "disaster," the level of urgency is "highest." For example, if the content of the data includes "accident" or "failure," the level of urgency is "high." For example, if the content of the data includes "congestion" or "delay," the level of urgency is "medium." The level of urgency may also be expressed as a numerical value rather than as a degree such as "high."

[0073] The generation unit 140 generates a summary using information obtained by the analysis, which indicates whether the level of urgency is high. For example, if the information indicates that the level of urgency is high, the generation unit 140 generates a summary that takes into account the high level of urgency. On the other hand, if the information indicates that the level of urgency is low, the generation unit 140 generates a summary that takes into account the low level of urgency.

[0074] The generation process will be described in detail below. In step S22, the generation unit 140 detects a template to which the extracted keywords are to be applied. Here, two management tables are prepared. The two management tables are shown below.

[0075] 16 is a diagram showing an example of a management table according to the fourth embodiment. Templates to be used when the level of urgency is high are registered in the management table 111a. Templates to be used when the level of urgency is low are registered in the management table 111b. The management tables 111a and 111b are stored in the storage unit 110 or an external device.

[0076] The generation unit 140 selects a management table to refer to using information indicating whether the level of urgency is high. For example, if the information indicates a high level of urgency, the generation unit 140 selects the management table 111a. Then, the generation unit 140 detects a template to which the extracted keyword is to be applied, based on the management table 111a. On the other hand, if the information indicates a low level of urgency, the generation unit 140 selects the management table 111b. Then, the generation unit 140 detects a template to which the extracted keyword is to be applied, based on the management table 111b.

[0077] The generation unit 140 generates a summary by applying the keyword to a template. Specifically, for example, assume that the keyword is "personal injury accident." Since the information indicating whether the urgency is high indicates that the urgency is high, the generation unit 140 selects the management table 111a. Then, the generation unit 140 applies the keyword to the template to generate the summary "Service on the Tokaido Line has been suspended due to a personal injury accident." In this manner, the summary is generated.

[0078] Furthermore, when the urgency level is analyzed, a management table corresponding to the urgency level is prepared. The management table corresponding to the urgency level is stored in the storage unit 110 or an external device. The generation unit 140 generates a summary using the urgency level. Specifically, after keywords are extracted, the generation unit 140 obtains the management table corresponding to the urgency level and detects a template to which the extracted keywords should be applied based on the management table. The generation unit 140 generates a summary by applying the keywords to the template. In this manner, a summary is generated.

[0079] The generation unit 140 may not only match keywords to templates but also transform the summary. For example, if the keyword is "earthquake," the generation unit 140 may transform the summary "Service on the Tokaido Line has been suspended due to an earthquake." into the summary "Service on the Tokaido Line has been suspended due to an earthquake." For example, by moving the keyword "earthquake" to the beginning of the sentence, the user can quickly learn of information with a high level of urgency. The generation unit 140 may also omit part of the summary. For example, the omitted sentence is information that is less necessary to convey to the user. Specifically, for example, if the keyword is "earthquake," the generation unit 140 may omit part of the summary "Service on the Tokaido Line has been suspended due to an earthquake." into the summary "An earthquake has occurred, and service on the Tokaido Line has been suspended." In this way, the generation unit 140 may omit information that is less necessary to convey to the user. The output unit 150 outputs the transformed summary or the omitted summary.

[0080] According to the fourth embodiment, the information processing device 100 generates a summary that takes into account the level of urgency by using information indicating whether the level of urgency is high or low. Thus, the information processing device 100 can generate a more accurate summary.

[0081] The fourth embodiment can be combined with the first modification of the first embodiment. Specifically, in step S32, the generation unit 140 uses a keyword to search for past cases matching the keyword. Here, as shown in FIG. 16 , multiple past cases to be used when the level of urgency is high and multiple past cases to be used when the level of urgency is low are prepared. The multiple past cases to be used when the level of urgency is high and the multiple past cases to be used when the level of urgency is low are stored in the storage unit 110 or an external device. The generation unit 140 selects a past case to reference using information indicating whether the level of urgency is high. For example, if the information indicates a high level of urgency, the generation unit 140 selects multiple past cases to be used when the level of urgency is high. Then, the generation unit 140 searches for a past case matching the keyword from the selected multiple past cases. Furthermore, for example, if the information indicates a low level of urgency, the generation unit 140 selects multiple past cases to be used when the level of urgency is low. Then, the generation unit 140 searches for a past case matching the keyword from the selected multiple past cases. The generating unit 140 generates a summary of the retrieved past cases.

[0082] Furthermore, past cases corresponding to the urgency level may be stored in the storage unit 110 or an external device. The generation unit 140 may search for past cases matching the keyword from among the past cases corresponding to the urgency level. The generation unit 140 generates a summary of the searched past cases. In this way, the fourth embodiment can be combined with the first modification of the first embodiment.

[0083] The fourth embodiment can be combined with the second modification of the first embodiment. Specifically, in step S43, the generation unit 140 generates a summary using information indicating whether the urgency is high, keywords, and a trained model. Specifically, when the generation unit 140 inputs the information indicating whether the urgency is high and keywords to the trained model, the trained model outputs a summary that takes the urgency into consideration. In this way, the fourth embodiment can be combined with the second modification of the first embodiment. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large number of keywords and information indicating whether the urgency is high are prepared as input data. Training data is prepared in which the summary corresponding to the keywords in the input data is labeled as a correct answer. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary, which is output data of the trained model, matches the correct answer label.

[0084] The generation unit 140 may generate a summary using the urgency, the keywords, and the trained model. In particular, when the generation unit 140 inputs the urgency and the keywords into the trained model, the trained model outputs a summary that takes the urgency into consideration.

[0085] The fourth embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary using information indicating whether the urgency is high, data obtained by speech recognition, and a trained model. Specifically, when the generation unit 140 inputs the information indicating whether the urgency is high and the data to the trained model, the trained model outputs a summary that takes the urgency into consideration. In this way, the fourth embodiment can be combined with the third modification of the first embodiment. The trained model can be created by machine learning, such as DNN. For example, in machine learning, a large number of character strings and information indicating whether the urgency is high are prepared as input data. Training data is prepared in which a summary corresponding to the character string in the input data is used as a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary, which is output data of the trained model, matches the correct answer label.

[0086] The generation unit 140 generates a summary using the urgency, the data obtained by speech recognition, and the trained model. In detail, when the generation unit 140 inputs the urgency and the data into the trained model, the trained model outputs a summary that takes the urgency into consideration.

[0087] The fourth embodiment can be combined with the second embodiment. Specifically, the generation unit 140 generates a summary using information indicating whether the urgency is high or low and the combined data. In this way, the fourth embodiment can be combined with the second embodiment.

[0088] The fourth embodiment can be combined with the first embodiment or the first to third modifications of the first embodiment, and the second embodiment.

[0089] The features of the above-described embodiments can be combined with each other as appropriate.

[0090] The above embodiments describe the generation of summaries of train operation status by the information processing device of the present disclosure, but this is merely an example. For example, the information processing device of the present disclosure may generate summaries of business status of tourist spots or commercial facilities.

[0091] 10 Microphone, 20 Display, 30 Speaker, 100 Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 110 Storage unit, 111 Management table, 111a, 111b Management table, 120 Acquisition unit, 130 Speech recognition execution unit, 140 Generation unit, 150 Output unit, 160 Determination unit, 170 Combination unit, 180 Analysis unit, 200 Output control device, 300 Server.

Claims

1. An information processing device having an acquisition unit that acquires voice data; a voice recognition execution unit that performs voice recognition using the voice data; a generation unit that generates a summary of the voice data using keywords in the data obtained by the voice recognition or word vectors based on the data; and an output unit that outputs the summary.

2. The information processing device according to claim 1, wherein the generation unit extracts the keywords from the data, detects a template for applying the keywords, and generates the summary by applying the keywords to the template.

3. The information processing device of claim 1, wherein the generation unit extracts the keyword from the data, uses the keyword to search for past cases that match the keyword, and generates the past cases obtained by the search as the summary sentence, and the past cases are information that has been output as a summary sentence in the past.

4. The information processing device of claim 1, wherein the acquisition unit acquires a trained model, and the generation unit extracts the keywords from the data and generates the summary sentence using the keywords and the trained model.

5. The information processing device according to claim 1, wherein the acquisition unit acquires a trained model, and the generation unit generates the summary sentence using the data and the trained model.

6. An information processing device according to any one of claims 1 to 5, wherein the generation unit generates the summary sentence using combined data, which is data in which a plurality of pieces of voice data are combined and in which contents that are semantically connected are combined.

7. An information processing device as described in any one of claims 1 to 6, wherein the acquisition unit acquires supplementary information regarding the data obtained by speech recognition, and the generation unit generates the summary sentence using the data obtained by speech recognition and the supplementary information.

8. An information processing device as described in any one of claims 1 to 6, further comprising an analysis unit that uses the data obtained by voice recognition to analyze whether the content of the data is of high urgency or not, or the urgency corresponding to the content of the data, and the generation unit generates the summary sentence using information obtained by the analysis, which indicates whether the urgency is high or not, or the urgency.

9. The information processing device according to claim 8, wherein the generation unit modifies the summary sentence or omits a part of the summary sentence, and the output unit outputs the modified summary sentence or the omitted summary sentence.

10. An output method comprising: an information processing device acquiring voice data; performing voice recognition using the voice data; generating a summary of the voice data using keywords in the data obtained by the voice recognition or word vectors based on the data; and outputting the summary.

11. An output program that causes a computer to execute the following processes: acquire voice data, perform voice recognition using the voice data, generate a summary of the voice data using keywords in the data obtained by the voice recognition or word vectors based on the data, and output the summary.

Citation Information

Patent Citations

  • Discourse summary template creation system and discourse summary template creation program

    JP2013120547A