Information processing device, output method, and output program

The information processing device addresses the challenge of redundant voice recognition outputs by generating summary sentences from voice data, enhancing user understanding.

JP2024169378APending Publication Date: 2024-12-05MITSUBISHI ELECTRIC CORP

Patent Information

Application Number
JP2024083371
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-25
Filing Date
2024-05-22
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Voice recognition outputs often result in redundant, grammatically inappropriate, and repetitive content, making it difficult for users to understand the information.

Method used

An information processing device that acquires voice data, performs voice recognition, extracts keywords, generates a summary sentence using word vectors or trained models, and outputs this summary for easy user comprehension.

Benefits of technology

The device effectively transforms complex voice recognition outputs into concise and understandable summary sentences, improving user comprehension.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169378000001_ABST
    Figure 2024169378000001_ABST
Patent Text Reader

Abstract

To output information that users can easily understand.SOLUTION: An information processing device 100 comprises an acquisition unit 120, a speech recognition execution unit 130, a generation unit 140, and an output unit 150. The acquisition unit 120 acquires speech data. The speech recognition execution unit 130 executes speech recognition by using the speech data. The generation unit 140 generates a summary of the speech data using keywords in the data obtained by speech recognition or word vectors based on the data. The output unit 150 outputs the summary.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an output method, and an output program. [Background technology]

[0002] Speech recognition technology is known. For example, Patent Document 1 describes a technology related to the speech recognition technology. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 065266 Summary of the Invention [Problem to be solved by the invention]

[0004] An utterance may be speech-recognized, and the results of the speech recognition may be displayed or output as audio. Here, the content of the utterance may have the following characteristics: The content is redundant. The same words are repeated. The content is not grammatically correct spoken language. The content of the utterance may have such characteristics. Therefore, when an utterance having such characteristics is speech-recognized, and the results of the speech recognition are displayed or output as audio, it may be difficult for the user to understand the information.

[0005] An objective of the present disclosure is to output information that is easily understandable to a user. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided an information processing device, the information processing device having an acquisition unit that acquires voice data, a voice recognition execution unit that executes voice recognition using the voice data, a generation unit that generates a summary of the voice data using keywords in data obtained by the voice recognition or a word vector based on the data, and an output unit that outputs the summary. Effect of the Invention

[0007] According to the present disclosure, it is possible to output information that is easily understandable to a user. [Brief description of the drawings]

[0008] [Figure 1] 1A and 1B are diagrams illustrating a providing system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating hardware included in an information processing device according to the first embodiment. [Diagram 3] 1 is a block diagram showing functions of an information processing device according to a first embodiment; [Figure 4] 4 is a flowchart showing an example of processing executed by the information processing device according to the first embodiment. [Diagram 5] 1 is a flowchart showing an example of a summary generation process according to the first embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of a management table according to the first embodiment. [Figure 7] 13 is a flowchart showing an example of a summary generating process according to a first modification of the first embodiment. [Figure 8] 13 is a flowchart showing an example of a summary generating process according to a second modification of the first embodiment. [Figure 9] 13 is a flowchart showing an example of a summary generating process according to a third modification of the first embodiment. [Figure 10] FIG. 11 is a block diagram showing the functions of an information processing device according to a second embodiment. [Figure 11] 13 is a flowchart showing an example of processing executed by an information processing device according to the second embodiment. [Figure 12] FIG. 11 is a diagram showing a provision system according to a third embodiment. [Figure 13] 13 is a flowchart showing an example of processing executed by an information processing device according to a third embodiment. [Figure 14] FIG. 13 is a block diagram showing the functions of an information processing device according to a fourth embodiment. [Figure 15]13 is a flowchart showing an example of processing executed by an information processing device according to a fourth embodiment. [Figure 16] FIG. 13 is a diagram illustrating an example of a management table according to the fourth embodiment. [Figure 17] FIG. 13 is a block diagram showing the functions of an information processing device according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1 1(A) and (B) are diagrams showing a provision system according to a first embodiment. Fig. 1(A) shows the provision system. The provision system includes a microphone 10, a display 20, a speaker 30, a camera 40, and an information processing device 100. The microphone 10, the display 20, the speaker 30, the camera 40, and the information processing device 100 are connected via a network. The network is a wired network or a wireless network.

[0011] The information processing device 100 is a device that executes an output method. For example, the information processing device 100 is a server. The processing of the information processing device 100 will be briefly described. The information processing device 100 acquires voice data from a microphone 10. For example, the sound of the voice data is a voice announcement in a train or in a station, a voice announcement in a tourist spot or a commercial facility, or the like. The information processing device 100 generates a summary of the voice data based on the voice data. The information processing device 100 may output the summary to a display 20. The information processing device 100 may output an audio signal of the summary to a speaker 30. This provides the summary to a user. The information processing device 100 may also translate the summary. The information processing device 100 may output the translated summary to the display 20, or may output an audio signal of the translated summary to the speaker 30.

[0012] Here, the providing system may include an output control device 200. FIG. 1(B) shows that the output control device 200 is connected to the display 20, the speaker 30, and the information processing device 100. The information processing device 100 may output a summary or an audio signal of the summary to the display 20 and the speaker 30 via the output control device 200. The output control device 200 may also have a translation function. When the output control device 200 has the translation function, the output control device 200 translates the input summary. The output control device 200 outputs the translated summary to the display 20. The output control device 200 also outputs an audio signal of the translated summary to the speaker 30.

[0013] 1(A) shows a case where the microphone 10, the display 20, the speaker 30, and the camera 40 are present outside the information processing device 100. At least one of the microphone 10, the display 20, the speaker 30, and the camera 40 may be included in the information processing device 100. For example, the information processing device 100 having at least one of the microphone 10, the display 20, the speaker 30, and the camera 40 is a signage.

[0014] Next, the hardware of the information processing device 100 will be described. 2 is a diagram showing hardware included in the information processing device of embodiment 1. The information processing device 100 is a computer. The information processing device 100 includes a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0015] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0016] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The non-volatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the non-volatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0017] Next, functions of the information processing device 100 will be described. 3 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 100 includes a storage unit 110, an acquisition unit 120, a voice recognition execution unit 130, a generation unit 140, and an output unit 150.

[0018] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103 . Some or all of the acquisition unit 120, the voice recognition execution unit 130, the generation unit 140, and the output unit 150 may be realized by a processing circuit. Also, some or all of the acquisition unit 120, the voice recognition execution unit 130, the generation unit 140, and the output unit 150 may be realized as modules of a program executed by the processor 101. For example, the program executed by the processor 101 is also called an output program or an output program product. For example, the output program is recorded on a recording medium.

[0019] The storage unit 110 stores various information. The functions of the acquisition unit 120, the voice recognition execution unit 130, the generation unit 140, and the output unit 150 will be described in detail later.

[0020] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. FIG. 4 is a flowchart illustrating an example of processing executed by the information processing device according to the first embodiment. (Step S11) The acquisition unit 120 acquires voice data. For example, the acquisition unit 120 acquires the voice data from the microphone 10. The acquisition unit 120 may also acquire the voice data from the microphone 10 via an external device. The external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server. Note that a diagram of the external device is omitted. The voice data may also be called a voice signal.

[0021] (Step S12) The voice recognition execution unit 130 executes voice recognition using the voice data, thereby allowing the information processing device 100 to obtain data that is a character string. (Step S13) The information processing device 100 executes a summary generation process. For example, the generation unit 140 generates a summary of the speech data by using keywords in the data obtained by speech recognition or a word vector based on the data. (Step S14) The output unit 150 outputs the summary. For example, the output unit 150 outputs the summary to the display 20. Also, for example, the output unit 150 outputs the summary of the audio signal to the speaker 30. Also, for example, the output unit 150 outputs the summary to the output control device 200.

[0022] Next, an example of the summary generation process will be shown. 5 is a flowchart showing an example of the summary generation process according to Embodiment 1. For example, the process in FIG. 5 corresponds to step S13. (Step S21) The generation unit 140 extracts keywords from the data obtained by speech recognition. The extraction process will be described. For example, the generation unit 140 extracts keywords from the data using a rule base that indicates keywords. For example, a method that uses a rule base is a method that uses a dictionary, a method that uses regular expressions, etc. Also, for example, the generation unit 140 extracts keywords from the data using a trained model. In detail, when the generation unit 140 inputs the data to the trained model, the trained model outputs the keywords. The trained model can be created by machine learning such as DNN (Deep Neural Network). For example, in machine learning, a large amount of character strings are prepared as input data. Training data is prepared in which keywords corresponding to the character strings of the input data are used as correct answer labels. Then, the trained model is created by performing machine learning using the input data and the training data so that keywords, which are output data of the trained model, match the correct answer labels.

[0023] (Step S22) The generating unit 140 detects a template to which the extracted keywords are to be applied. The template detection process will be described with reference to the drawings.

[0024] 6 is a diagram showing an example of a management table according to the first embodiment. For example, the management table 111 is stored in the storage unit 110. The management table 111 has items of item number, keyword, and template. For example, if the keyword is “restarting operation”, the generating unit 140 detects the template of the record with the item number 1.

[0025] Furthermore, the management table 111 may be stored in an external device. When the management table 111 is stored in an external device, the generating unit 140 acquires the management table 111 from the external device via the acquiring unit 120.

[0026] Furthermore, the generation unit 140 may detect a template using a trained model. Specifically, when the generation unit 140 inputs a keyword to the trained model, the trained model outputs an item number in the management table 111. For example, the trained model outputs "item number 1." The generation unit 140 detects the template of the output item number as a template to which the keyword is to be applied. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of keywords are prepared as input data. Training data is prepared in which templates corresponding to the keywords of the input data are used as correct answer labels. Then, the trained model is created by performing machine learning using the input data and the training data so that the templates, which are output data of the trained model, match the correct answer labels.

[0027] (Step S23) The generation unit 140 generates a summary sentence by applying the keywords to a template. For example, the generation unit 140 generates the summary sentence "Train service is expected to resume."

[0028] Here, the content of an utterance may have the following characteristics: The content is redundant. The same words are repeated. The utterance is not grammatically correct spoken language. The content of an utterance may have such characteristics. Therefore, when an utterance having such characteristics is speech-recognized and the result of the speech recognition is displayed or output as voice, it may be difficult for the user to understand the information.

[0029] According to the first embodiment, the information processing device 100 generates a summary based on the content of an utterance and outputs the summary. This allows the information processing device 100 to output information that is easily understandable to the user even if the content of the utterance has the above-mentioned characteristics.

[0030] Variation 1 of the first embodiment In a first modification of the first embodiment, a case will be described in which a summary is generated by a different method. 7 is a flowchart showing an example of a summary generation process according to Modification 1 of Embodiment 1. For example, the process in FIG.

[0031] (Step S31) The generation unit 140 extracts keywords from data obtained by voice recognition.

[0032] (Step S32) The generation unit 140 uses the keyword to search for past cases matching the keyword. Note that a past case is information that has been output in the past as a summary sentence. Here, the memory unit 110 or the external device stores multiple past cases. For example, if the memory unit 110 stores multiple past cases, the generation unit 140 searches for a past case matching the keyword from among the multiple past cases stored in the memory unit 110. Also, for example, if the external device stores multiple past cases, the generation unit 140 accesses the external device and searches for a past case matching the keyword from among the multiple past cases stored in the external device.

[0033] (Step S33) The generating unit 140 generates a summary sentence from the past cases obtained by the search. This sentence may be expressed as follows: The generating unit 140 outputs the searched past cases as a summary sentence.

[0034] According to the first modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0035] Variation 2 of embodiment 1 In a second modification of the first embodiment, a case will be described in which a summary is generated by a different method. 8 is a flowchart showing an example of a summary generation process according to Modification 2 of Embodiment 1. For example, the process in FIG. 8 corresponds to step S13.

[0036] (Step S41) The generation unit 140 extracts keywords from data obtained by voice recognition. (Step S42) The acquisition unit 120 acquires the trained model from the storage unit 110 or an external device. (Step S43) The generation unit 140 generates a summary sentence by using the keyword and the trained model. In detail, when the generation unit 140 inputs the keyword to the trained model, the trained model outputs the summary sentence. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of keywords are prepared as input data. Training data is prepared in which summary sentences corresponding to the keywords in the input data are used as correct answer labels. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentences, which are output data of the trained model, match the correct answer labels.

[0037] According to the second modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0038] Variation 3 of embodiment 1 In a third modification of the first embodiment, a case will be described in which a summary is generated by a different method. 9 is a flowchart showing an example of a summary generation process according to Modification 3 of Embodiment 1. For example, the process in FIG.

[0039] (Step S51) The acquisition unit 120 acquires a trained model from the storage unit 110 or an external device. (Step S52) The generation unit 140 generates a summary sentence by using the data obtained by the speech recognition and the trained model. In detail, when the generation unit 140 inputs the data to the trained model, the trained model outputs the summary sentence. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of character strings are prepared as input data. Training data is prepared in which a summary sentence corresponding to the character string of the input data is used as a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentence, which is output data of the trained model, matches the correct answer label.

[0040] According to the third modification of the first embodiment, the information processing device 100 can output information that can be easily understood by the user.

[0041] Embodiment 2 Next, a description will be given of embodiment 2. In embodiment 2, differences from embodiment 1 will be mainly described. In embodiment 2, descriptions of matters common to embodiment 1 will be omitted.

[0042] First, a speaker may give a talk that is connected in meaning in multiple parts. Therefore, when a summary is generated based on the voice data of one talk, the accuracy of the generated summary may be low. Therefore, in the second embodiment, a case where a more accurate summary is generated will be described.

[0043] 10 is a block diagram showing functions of an information processing device according to the second embodiment. The information processing device 100 further includes a determination unit 160 and a combination unit 170. A part or all of the determination unit 160 and the combination unit 170 may be realized by a processing circuit. Also, a part or all of the determination unit 160 and the combination unit 170 may be realized as a module of a program executed by the processor 101. The functions of the determining unit 160 and the combining unit 170 will be explained later.

[0044] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. Fig. 11 is a flowchart showing an example of processing executed by the information processing device of embodiment 2. The processing in Fig. 11 differs from the processing in Fig. 4 in that steps S11a, 12a to 12d are executed. Therefore, steps S11a, 12a to 12d will be described in Fig. 11. Descriptions of the processing other than steps S11a, 12a to 12d will be omitted.

[0045] (Step S11a) The acquisition unit 120 acquires audio data for one episode. (Step S12a) The determination unit 160 determines whether or not the voice data acquired in step S11a is the first voice data of the same person. If the acquired voice data is the first voice data, the determination unit 160 stores the data obtained by voice recognition in the storage unit 110. Then, the process proceeds to step S11a. If the acquired voice data is voice data of the same person, the process proceeds to step S12b. Note that the determination unit 160 can determine whether or not the acquired voice data is voice data of the same person by analysis.

[0046] (Step S12b) The determination unit 160 determines whether there is a semantic connection between the data obtained by the speech recognition and the data stored in the storage unit 110. For example, the determination unit 160 determines whether there is a semantic connection by using a rule base. For example, a method using a rule base is a method using a dictionary or a method using regular expressions. In addition, for example, the determination unit 160 vectorizes the data obtained by the speech recognition and the data stored in the storage unit 110, and compares the two vectorized data to determine whether there is a semantic connection. If there is a semantic connection, the process proceeds to step S12c, otherwise the process proceeds to step S12d.

[0047] (Step S12c) The combining unit 170 combines the data obtained by the voice recognition with the data stored in the storage unit 110. The combining unit 170 stores the combined data in the storage unit 110. Then, the process proceeds to step S11a. (Step S12d) Acquiring unit 120 acquires the combined data from storage unit 110. The combined data is data in which a plurality of pieces of voice data are combined, and is data in which contents having a common meaning are combined. If the combined data is not stored in the storage unit 110, the generation unit 140 uses the voice data for one episode.

[0048] The generating unit 140 generates a summary sentence by using the combined data. For example, in step S21, the generating unit 140 extracts keywords from the combined data. The process from step S22 onward is the same as the process in FIG.

[0049] According to the second embodiment, the information processing device 100 generates a summary based on a plurality of utterances that are semantically related, and therefore the information processing device 100 can generate a more accurate summary.

[0050] Furthermore, there are cases where a speaker inverts a sentence and utters multiple utterances that are connected in meaning. Even when a speaker speaks in this way, the second embodiment is effective.

[0051] Here, the external device may execute steps S11a, S12, and S12a to S12c. When the external device executes the processes, in step S12d, the acquiring section 120 acquires the combined data from the external device.

[0052] The second embodiment can be combined with the first modification of the first embodiment. Specifically, in step S31, the generation unit 140 extracts keywords from the combined data. The processes from step S32 onward are the same as those in FIG.

[0053] The second embodiment can be combined with the second modification of the first embodiment. Specifically, in step S41, the generation unit 140 extracts keywords from the combined data. The processes from step S42 onward are the same as those in FIG.

[0054] The second embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary sentence by using the combined data and the trained model. In particular, when the generation unit 140 inputs the combined data to the trained model, the trained model outputs the summary sentence. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of combined data is prepared as input data. Training data is prepared in which a summary sentence corresponding to the combined data of the input data is used as a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentence, which is output data of the trained model, matches the correct answer label.

[0055] Embodiment 3 Next, a description will be given of embodiment 3. In embodiment 3, differences from embodiment 1 will be mainly described. In embodiment 3, descriptions of matters common to embodiment 1 will be omitted.

[0056] 12 is a diagram showing a provision system according to the third embodiment. The provision system further includes a server 300. The information processing device 100 and the server 300 communicate with each other via a network. The network may be a wired network or a wireless network.

[0057] The server 300 stores supplementary information when generating a summary. The supplementary information is useful information. For example, the supplementary information may include a character string indicating the content of the useful information. For example, information on a social networking service (SNS) is stored in the server 300. For example, when a summary about a train is generated, the information is information such as the reason for the train stopping and the reason for the delay. The server 300 performs filtering and extracts useful information from the information on the SNS. This allows the server 300 to store the supplementary information, which is useful information. The server 300 updates the supplementary information periodically. In addition, the server 300 may transmit the supplementary information to the information processing device 100 periodically. When the acquisition unit 120 of the information processing device 100 acquires the supplementary information, the acquisition unit 120 stores the supplementary information in the storage unit 110.

[0058] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. Fig. 13 is a flowchart showing an example of processing executed by the information processing device of the third embodiment. The processing in Fig. 13 differs from the processing in Fig. 4 in that step S12e is executed. Therefore, step S12e will be described in Fig. 13. Descriptions of the processing other than step S12e will be omitted.

[0059] (Step S12e) The acquisition unit 120 acquires supplementary information related to the data obtained by speech recognition from the storage unit 110 or the server 300. This sentence may be expressed as follows: The acquisition unit 120 acquires supplementary information related to words included in the data obtained by speech recognition from the storage unit 110 or the server 300. For example, if the data includes a line name, the acquisition unit 120 acquires supplementary information related to the line name.

[0060] The generating unit 140 generates a summary sentence by using the data obtained by the speech recognition and the supplementary information. For example, in step S21, the generating unit 140 extracts keywords from the data and the supplementary information. The process from step S22 onward is the same as the process in FIG. 5.

[0061] Here, for example, the content of the utterance may lack some information due to a speech recognition error of the voice announcement, an erroneous utterance, etc. Also, for example, the content of the utterance may lack a prerequisite due to an omission of the utterance, etc. In such a case, the accuracy of the summary generated based only on the content of the utterance may be low. Therefore, the information processing device 100 uses the supplemental information to supplement the content that is missing from the content of the utterance. As a result, a more accurate summary is generated.

[0062] A specific example is given. The data obtained by speech recognition includes "Tokaido Line." The content of the utterance is assumed to lack the cause of the delay of the "Tokaido Line." The supplementary information indicates the cause of the delay of the "Tokaido Line." The generation unit 140 generates a summary sentence using the data obtained by speech recognition and the supplementary information. When generating the summary sentence, the generation unit 140 supplements the cause of the delay of the "Tokaido Line" to generate the summary sentence. This results in a more accurate summary sentence being generated.

[0063] Therefore, according to the third embodiment, the information processing device 100 can generate a more accurate summary.

[0064] The third embodiment can be combined with the first modification of the first embodiment. Specifically, in step S31, the generating unit 140 extracts keywords from the data obtained by the speech recognition and the supplemental information. The processes from step S32 onward are the same as those in FIG. 7.

[0065] The third embodiment can be combined with the second modification of the first embodiment. Specifically, in step S41, the generation unit 140 extracts keywords from the data obtained by the speech recognition and the supplemental information. The processes from step S42 onward are the same as those in FIG. 8.

[0066] The third embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary sentence by using the data obtained by speech recognition, the supplementary information, and the trained model. In particular, when the generation unit 140 inputs the data and the supplementary information to the trained model, the trained model outputs the summary sentence. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of character strings and supplementary information are prepared as input data. Training data is prepared in which summary sentences corresponding to the character strings and supplementary information of the input data are used as correct answer labels. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentences, which are output data of the trained model, match the correct answer labels.

[0067] The third embodiment can be combined with the second embodiment. Specifically, the generating unit 140 generates a summary sentence by using the combined data and the supplemental information.

[0068] The third embodiment can be combined with the first embodiment or the first to third modifications of the first embodiment, and the second embodiment.

[0069] Embodiment 4 Next, a fourth embodiment will be described. In the fourth embodiment, differences from the first embodiment will be mainly described. In the fourth embodiment, descriptions of the same matters as the first embodiment will be omitted.

[0070] 14 is a block diagram showing functions of an information processing device according to the fourth embodiment. The information processing device 100 further includes an analysis unit 180. A part or all of the analysis unit 180 may be realized by a processing circuit. Also, a part or all of the analysis unit 180 may be realized as a module of a program executed by the processor 101. The function of the analysis unit 180 will be explained later.

[0071] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. Fig. 15 is a flowchart showing an example of processing executed by the information processing device of embodiment 4. The processing in Fig. 15 differs from the processing in Fig. 4 in that step S12f is executed. Therefore, step S12f will be described in Fig. 15. Descriptions of the processing other than step S12f will be omitted.

[0072] (Step S12f) The analysis unit 180 uses the data obtained by the voice recognition to analyze whether the content of the data is of high urgency. For example, the analysis unit 180 performs the analysis using a rule base. For example, a method using a rule base is a method using a dictionary, a method using regular expressions, etc. Also, for example, the analysis unit 180 performs the analysis using a trained model. For example, a content of high urgency is a content indicating that an accident, fire, disaster, etc. has occurred. For example, when the content of the data is a content indicating that an accident, fire, disaster, etc. has occurred, the analysis unit 180 outputs information indicating that the urgency is high. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of keywords are prepared as input data. Training data is prepared in which the level of urgency corresponding to the keyword of the input data is used as a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the level of urgency, which is output data of the trained model, matches the correct answer label.

[0073] Furthermore, the analysis unit 180 may use data obtained by speech recognition to analyze the urgency corresponding to the content of the data. For example, if the content of the data includes "earthquake" or "disaster", the urgency is "highest". For example, if the content of the data includes "accident" or "breakdown", the urgency is "high". For example, if the content of the data includes "congestion" or "delay", the urgency is "medium". The urgency may be expressed as a numerical value instead of a degree such as "high".

[0074] Furthermore, when analyzing the urgency corresponding to the content of the data, the analysis unit 180 may analyze the urgency corresponding to the content of the data by taking into account the paralanguage. The analysis process will be described in detail. The analysis unit 180 analyzes the paralanguage by using the voice data acquired by the acquisition unit 120. As a result, the information processing device 100 can obtain the paralanguage. For example, the paralanguage is a fast speech, a high-pitched speech, a length of a sound division (i.e., a pause), the number of pauses per a given time, and a loudness of a voice. The analysis unit 180 analyzes the urgency corresponding to the content of the data by taking into account the paralanguage obtained by analyzing the voice data. A specific example will be described. It is assumed that the content of the data includes "accident". The paralanguage is "fast speech". The analysis unit 180 analyzes that the urgency corresponding to the content of the data is "highest". In other words, when a speaker talks about the accident in a fast manner, it can be said that the content of the data is the most urgent. Therefore, the analysis unit 180 analyzes that the urgency corresponding to the content of the data is "highest". In this way, the information processing device 100 can improve the accuracy of analyzing the urgency by taking the paralanguage into consideration. The urgency may be expressed by a numerical value.

[0075] In the above, a case has been described in which the analysis unit 180 analyzes the paralanguage. The analysis of the paralanguage may be performed by a device external to the information processing device 100. For example, the information processing device 100 transmits voice data to the device. The device analyzes the paralanguage. The acquisition unit 120 acquires the paralanguage from the device. Then, the analysis unit 180 analyzes the urgency corresponding to the content of the data, taking the paralanguage into consideration.

[0076] The generating unit 140 generates a summary sentence using information obtained by the analysis, which indicates whether the urgency is high or not. For example, if the information indicates that the urgency is high, the generating unit 140 generates a summary sentence taking into consideration the high urgency. Also, if the information indicates that the urgency is low, the generating unit 140 generates a summary sentence taking into consideration the low urgency.

[0077] The generation process will be described in detail. In step S22, the generation unit 140 detects a template to which the extracted keywords are applied. Here, two management tables are prepared. The two management tables are shown below.

[0078] 16 is a diagram showing an example of a management table according to the fourth embodiment. Templates to be used when the level of urgency is high are registered in the management table 111a. Templates to be used when the level of urgency is low are registered in the management table 111b. The management tables 111a and 111b are stored in the storage unit 110 or an external device.

[0079] The generating unit 140 selects a management table to refer to by using information indicating whether the urgency is high or not. For example, when the information indicates that the urgency is high, the generating unit 140 selects the management table 111a. Then, the generating unit 140 detects a template to which the extracted keyword is applied based on the management table 111a. Also, for example, when the information indicates that the urgency is low, the generating unit 140 selects the management table 111b. Then, the generating unit 140 detects a template to which the extracted keyword is applied based on the management table 111b.

[0080] The generation unit 140 generates a summary by applying the keyword to a template. Specifically, for example, assume that the keyword is "accident resulting in injury or death." Since the information indicating whether the urgency is high or not indicates that the urgency is high, the generation unit 140 selects the management table 111a. Then, the generation unit 140 generates a summary "Service on the Tokaido Line has been suspended due to an accident resulting in injury or death" by applying the keyword to the template. In this manner, the summary is generated.

[0081] Furthermore, when the urgency level is analyzed, a management table corresponding to the urgency level is prepared. The management table corresponding to the urgency level is stored in the storage unit 110 or an external device. The generation unit 140 generates a summary sentence using the urgency level. Specifically, after a keyword is extracted, the generation unit 140 obtains a management table corresponding to the urgency level, and detects a template to which the extracted keyword is to be applied based on the management table. The generation unit 140 generates a summary sentence by applying the keyword to the template. In this manner, the summary sentence is generated.

[0082] The generating unit 140 may not only apply the keyword to the template, but also transform the summary sentence. For example, when the keyword is "earthquake", the generating unit 140 may transform the summary sentence "The Tokaido Line is not operating due to an earthquake." into the summary sentence "The Tokaido Line is not operating due to an earthquake." For example, by moving the keyword "earthquake" to the beginning of the sentence, the user can quickly learn information with a high degree of urgency. The generating unit 140 may also omit a part of the summary sentence. For example, the omitted sentence is information that is less necessary to be conveyed to the user. Specifically, for example, when the keyword is "earthquake", the generating unit 140 omits a part of the summary sentence "The Tokaido Line is not operating due to an earthquake." into the summary sentence "An earthquake has occurred, and the Tokaido Line is not operating." In this way, the generating unit 140 may omit information that is less necessary to be conveyed to the user. The output unit 150 outputs the transformed summary sentence or the omitted summary sentence.

[0083] According to the fourth embodiment, the information processing device 100 uses information indicating whether the urgency is high or not to generate a summary that takes the urgency into consideration, thereby enabling the information processing device 100 to generate a more accurate summary.

[0084] The fourth embodiment can be combined with the first modification of the first embodiment. Specifically, in step S32, the generating unit 140 uses a keyword to search for a past case matching the keyword. Here, as shown in FIG. 16, a plurality of past cases used when the urgency is high and a plurality of past cases used when the urgency is low are prepared. The plurality of past cases used when the urgency is high and the plurality of past cases used when the urgency is low are stored in the storage unit 110 or an external device. The generating unit 140 selects a past case to be referred to using information indicating whether the urgency is high or not. For example, when the information indicates that the urgency is high, the generating unit 140 selects a plurality of past cases used when the urgency is high. Then, the generating unit 140 searches for a past case matching the keyword from among the selected plurality of past cases. Also, for example, when the information indicates that the urgency is low, the generating unit 140 selects a plurality of past cases used when the urgency is low. Then, the generating unit 140 searches for a past case matching the keyword from among the selected plurality of past cases. The generating unit 140 generates a summary of the retrieved past cases.

[0085] Furthermore, past cases corresponding to the urgency may be stored in the storage unit 110 or an external device. The generating unit 140 may search for past cases matching the keyword from among the past cases corresponding to the urgency. The generating unit 140 generates a summary of the searched past case. In this way, the fourth embodiment can be combined with the first modification of the first embodiment.

[0086] The fourth embodiment can be combined with the second modification of the first embodiment. Specifically, in step S43, the generation unit 140 generates a summary sentence using information indicating whether the urgency is high or not, the keywords, and the trained model. In particular, when the generation unit 140 inputs the information indicating whether the urgency is high or not and the keywords to the trained model, the trained model outputs a summary sentence in which the urgency is taken into consideration. In this way, the fourth embodiment can be combined with the second modification of the first embodiment. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large number of keywords and information indicating whether the urgency is high are prepared as input data. Training data is prepared in which a summary sentence corresponding to the keyword of the input data is a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentence, which is output data of the trained model, matches the correct answer label.

[0087] The generation unit 140 may generate a summary sentence by using the urgency, the keyword, and the trained model. In detail, when the generation unit 140 inputs the urgency and the keyword to the trained model, the trained model outputs a summary sentence in which the urgency is taken into consideration.

[0088] The fourth embodiment can be combined with the third modification of the first embodiment. Specifically, in step S52, the generation unit 140 generates a summary sentence using information indicating whether the urgency is high or not, data obtained by speech recognition, and the trained model. In detail, when the generation unit 140 inputs the information indicating whether the urgency is high or not and the data to the trained model, the trained model outputs a summary sentence in which the urgency is taken into consideration. In this way, the fourth embodiment can be combined with the third modification of the first embodiment. The trained model can be created by machine learning such as DNN. For example, in machine learning, a large amount of character strings and information indicating whether the urgency is high are prepared as input data. Training data is prepared in which a summary sentence corresponding to the character string of the input data is used as a correct answer label. Then, the trained model is created by performing machine learning using the input data and the training data so that the summary sentence, which is output data of the trained model, matches the correct answer label.

[0089] The generation unit 140 generates a summary sentence by using the urgency, the data obtained by the speech recognition, and the trained model. In detail, when the generation unit 140 inputs the urgency and the data to the trained model, the trained model outputs a summary sentence in which the urgency is taken into consideration.

[0090] The fourth embodiment can be combined with the second embodiment. Specifically, the generating unit 140 generates a summary sentence by using the information indicating whether the urgency is high or not and the combined data. In this way, the fourth embodiment can be combined with the second embodiment.

[0091] The fourth embodiment can be combined with the first embodiment or the first to third modifications of the first embodiment, and the second embodiment.

[0092] Embodiment 5. Next, a fifth embodiment will be described. In the fifth embodiment, differences from the first to fourth embodiments will be mainly described. In the fifth embodiment, descriptions of commonalities with the first to fourth embodiments will be omitted. For example, the information processing device 100 in the fifth embodiment is a signage.

[0093] 17 is a block diagram showing the functions of an information processing device according to the fifth embodiment. The information processing device 100 further includes an omission control unit 190. A part or all of the omission control unit 190 may be realized by a processing circuit. Also, a part or all of the omission control unit 190 may be realized as a module of a program executed by the processor 101.

[0094] The camera 40 photographs a user who is in front of the information processing device 100. The acquisition unit 120 acquires an image obtained by photographing the user from the camera 40. The omitted control unit 190 performs user authentication by face authentication using the image. In this way, the omitted control unit 190 can identify the user. Note that the method of identifying the user is not limited to face authentication. The information processing device 100 can identify the user by an authentication code displayed on the screen of the user's terminal. The authentication code is distributed individually to the user. For example, the authentication code is a two-dimensional barcode or the like. The acquisition unit 120 acquires an image obtained by photographing the authentication code displayed on the screen of the user's terminal (for example, a smartphone) from the camera 40. The omitted control unit 190 performs code authentication of the authentication code indicated by the image and performs user authentication. The omitted control unit 190 acquires information that the user already knows from the storage unit 110 or an external device. For example, if the user has asked a question in the past and has received an answer to the question, the information that the user already knows is the answer. Also, for example, when certain information is provided to a user's terminal (e.g., a smartphone), the information that the user already knows is the provided information. Furthermore, for example, when the user acquires information via another medium, the information that the user already knows is the acquired information. The omission control unit 190 omits information that the user already knows from the summary generated by the generation unit 140. The output unit 150 outputs the summary from which the information that the user already knows is omitted to the display 20.

[0095] According to the fifth embodiment, the information processing device 100 does not need to provide the user with information that the user already knows. In other words, the information processing device 100 can efficiently convey information to the user.

[0096] The features of each of the embodiments and the modified examples of each of the embodiments described above can be combined with each other as appropriate.

[0097] The above embodiments describe the generation of a summary of train operation status by the information processing device of the present disclosure, but this is merely an example. For example, the information processing device of the present disclosure may generate a summary of the business status of a tourist spot or a commercial facility.

[0098] Furthermore, the following supplementary notes are disclosed with respect to each embodiment. (Appendix 1) An acquisition unit that acquires voice data; a voice recognition execution unit that executes voice recognition using the voice data; a generation unit that generates a summary of the speech data by using keywords in the data obtained by the speech recognition or word vectors based on the data; an output unit for outputting the summary; An information processing device having the above configuration. (Appendix 2) the generation unit extracts the keywords from the data, detects a template to which the keywords are to be applied, and generates the summary by applying the keywords to the template. 2. An information processing device according to claim 1. (Appendix 3) the generation unit extracts the keyword from the data, searches for past cases matching the keyword using the keyword, and generates the summary sentence from the past cases obtained by the search; The past case is information that has been output as a summary sentence in the past. 2. An information processing device according to claim 1. (Appendix 4) The acquisition unit acquires a trained model, The generation unit extracts the keywords from the data, and generates the summary sentence by using the keywords and the trained model. 2. An information processing device according to claim 1. (Appendix 5) The acquisition unit acquires a trained model, The generation unit generates the summary sentence by using the data and the trained model. 2. An information processing device according to claim 1. (Appendix 6) The generation unit generates the summary sentence using combined data, which is data in which a plurality of pieces of voice data are combined and data in which contents having a semantic connection are combined. 6. An information processing device according to any one of claims 1 to 5. (Appendix 7) The acquisition unit acquires supplementary information related to the data acquired by speech recognition, the generation unit generates the summary sentence by using the data obtained by the speech recognition and the supplemental information. 7. An information processing device according to any one of claims 1 to 6. (Appendix 8) An analysis unit is further provided for analyzing whether the content of the data is of high urgency or not, or the urgency corresponding to the content of the data, using the data obtained by the voice recognition, The generation unit generates the summary sentence using information obtained by the analysis, the information indicating whether the urgency is high or the urgency. 7. An information processing device according to any one of claims 1 to 6. (Appendix 9) When analyzing the urgency corresponding to the content of the data, the analysis unit analyzes the urgency corresponding to the content of the data in consideration of the paralanguage obtained by analyzing the voice data. 9. The information processing device according to claim 8. (Appendix 10) The generation unit modifies the summary or omits a part of the summary, The output unit outputs the transformed summary sentence or the abbreviated summary sentence. 9. The information processing device according to claim 8. (Appendix 11) An omission control unit; A camera and A display and and The camera captures an image of a user in front of the information processing device or a screen of a terminal held by the user, The acquisition unit acquires an image obtained by photographing the user or the screen from the camera, the omission control unit performs user authentication using the image to identify the user, acquires information already known by the user, and omits the information already known by the user from the summary; The output unit outputs a summary sentence from which information already known by the user is omitted to the display. 11. An information processing device according to any one of claims 1 to 10. (Appendix 12) An information processing device, Acquire audio data, performing speech recognition using said speech data; generating a summary of the speech data using keywords in the data obtained by the speech recognition or word vectors based on the data; outputting the summary sentence; Output method. (Appendix 13) On the computer, Acquire audio data, performing speech recognition using said speech data; generating a summary of the speech data using keywords in the data obtained by the speech recognition or word vectors based on the data; outputting the summary sentence; The output program that causes the processing to occur. [Explanation of symbols]

[0099] 10 microphone, 20 display, 30 speaker, 40 camera, 100 information processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 storage unit, 111 management table, 111a, 111b management tables, 120 acquisition unit, 130 voice recognition execution unit, 140 generation unit, 150 output unit, 160 judgment unit, 170 combination unit, 180 analysis unit, 190 omission control unit, 200 output control device, 300 server.

Claims

1. An acquisition unit that acquires voice data; a voice recognition execution unit that executes voice recognition using the voice data; a generation unit that generates a summary of the speech data by using keywords in the data obtained by the speech recognition or word vectors based on the data; an output unit for outputting the summary; An information processing device having the above configuration.

2. the generation unit extracts the keywords from the data, detects a template to which the keywords are to be applied, and generates the summary by applying the keywords to the template. The information processing device according to claim 1 .

3. the generation unit extracts the keyword from the data, searches for past cases matching the keyword using the keyword, and generates the summary sentence from the past cases obtained by the search; The past case is information that has been output as a summary sentence in the past. The information processing device according to claim 1 .

4. The acquisition unit acquires a trained model, The generation unit extracts the keywords from the data, and generates the summary sentence by using the keywords and the trained model. The information processing device according to claim 1 .

5. The acquisition unit acquires a trained model, The generation unit generates the summary sentence by using the data and the trained model. The information processing device according to claim 1 .

6. The generation unit generates the summary sentence using combined data, which is data in which a plurality of pieces of voice data are combined and data in which contents having a semantic connection are combined.

3. The information processing device according to claim 1 or 2.

7. The acquisition unit acquires supplementary information related to the data acquired by speech recognition, the generation unit generates the summary sentence by using the data obtained by the speech recognition and the supplemental information.

3. The information processing device according to claim 1 or 2.

8. An analysis unit is further provided for analyzing whether the content of the data is of high urgency or not, or the urgency corresponding to the content of the data, using the data obtained by the voice recognition, The generation unit generates the summary sentence using information obtained by the analysis, the information indicating whether the urgency is high or the urgency.

3. The information processing device according to claim 1 or 2.

9. When analyzing the urgency corresponding to the content of the data, the analysis unit analyzes the urgency corresponding to the content of the data in consideration of the paralanguage obtained by analyzing the voice data. The information processing device according to claim 8.

10. The generation unit modifies the summary or omits a part of the summary, The output unit outputs the transformed summary sentence or the abbreviated summary sentence. The information processing device according to claim 8.

11. An omission control section; A camera and A display and and The camera captures an image of a user in front of the information processing device or a screen of a terminal held by the user, The acquisition unit acquires an image obtained by photographing the user or the screen from the camera, the omission control unit performs user authentication using the image to identify the user, acquires information already known by the user, and omits the information already known by the user from the summary; The output unit outputs a summary sentence from which information already known by the user is omitted to the display.

3. The information processing device according to claim 1 or 2.

12. An information processing device, Acquire audio data, performing speech recognition using said speech data; generating a summary of the speech data using keywords in the data obtained by the speech recognition or word vectors based on the data; outputting the summary sentence; Output method.

13. On the computer, Acquire audio data, performing speech recognition using said speech data; generating a summary of the speech data using keywords in the data obtained by the speech recognition or word vectors based on the data; outputting the summary sentence; The output program that causes the processing to occur.

Citation Information

Patent Citations

  • Information management system and information management method

    WO2017065266A1

Cited By

  • Chain

    CN121739059A