Information processing apparatus, information processing method, and storage medium

Abstract

To properly transmit necessary information to a driver of a moving body.SOLUTION: The information processing apparatus includes an acquisition unit that acquires, from an application, intent information including character string information indicating a character string constituting a notification statement to be output by voice to a driver of a moving body and application priority level information indicating an application priority level set for each of the applications, and an output suspension timing indicating a timing at which notification of the notification statement is to be completed, a calculation unit that calculates an output suspension time that is a remaining time until the output suspension timing is reached, and a generation unit that generates a notification statement to be output by voice to the driver on the basis of the character string information. The generation unit generates a notification sentence including a character string, and changes an expression of the notification sentence such that a timing at which voice output of the generated notification sentence is completed is earlier than the output deferment timing.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, an information processing program, and a storage medium. [Background technology]

[0002] Conventionally, there is known a technique for controlling multiple types of audio output when multiple types of audio output are simultaneously present. For example, a technique is known in which a decision is made as to whether or not to put the output of subsequent audio information on hold based on a time relationship between a scheduled time for completing transmission of the significant content of the earlier audio information and an allowable delay time from the time of issuance of an output request for the later audio information until the output starts. Then, a technique is known in which, if it is determined that the output of the later audio information can be put on hold, the output of the later audio information is put on hold, and the output of the earlier audio information is given priority before the output of the later audio information is output. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6020189 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional technology, the output of the earlier voice information is given priority, and then the output of the later voice information is simply performed, so it is not always possible to output both the earlier voice information and the later voice information within the desired time. In other words, in the above-mentioned conventional technology, it is not always possible to convey all the necessary information to the user within the desired time.

[0005] Furthermore, the above-described conventional technology has problems such as voice interruptions making it difficult to understand the content, and does not take into consideration cases where a user's response is required.

[0006] The present invention has been made in consideration of the above, and aims to provide an information processing device, an information processing method, an information processing program, and a storage medium that can appropriately convey necessary information to the driver of a mobile body. [Means for solving the problem]

[0007] The information processing device described in claim 1 includes an acquisition unit that acquires, from an application, string information indicating a string of characters constituting a notification message to be output audibly to a driver of a moving body, intent information including application priority information indicating an application priority set for each application, and an output deferred timing indicating a timing at which the notification of the notification message should be completed, a calculation unit that calculates an output deferred time that is the remaining time until the output deferred timing is reached, and a generation unit that generates a notification message to be output audibly to the driver based on the string information, wherein the generation unit generates a provisional notification message including a plurality of application notification messages having different application priorities, and when a total playback time of the provisional notification messages exceeds the output deferred time, shortens the lengths of the application notification messages from the plurality of application notification messages included in the provisional notification message in order from the application notification message having the lowest application priority, thereby generating notification messages that can be played within the output deferred time.

[0008] An information processing method described in claim 18 includes an acquisition step of acquiring, from an application, character string information indicating a character string constituting a notification message to be output aloud to a driver of a mobile body, intent information including application priority information indicating an application priority set for each application, and an output deferred timing indicating a timing at which the notification of the notification message should be completed; a calculation step of calculating an output deferred time which is the remaining time until the output deferred timing is reached; and a generation step of generating a notification message to be output aloud to the driver based on the character string information, wherein the generation step generates a provisional notification message including a plurality of application notification messages having different application priorities, and if a total playback time of the provisional notification messages exceeds the output deferred time, shortens lengths of the application notification messages from the plurality of application notification messages included in the provisional notification message in order from the application notification message having the lowest application priority, thereby generating notification messages that can be played within the output deferred time.

[0009] and a generating step of generating a notification message to be output by voice to the driver of a moving object, based on the character string information. The generating step generates a provisional notification message including a plurality of application notification messages having different application priorities, and, if a total playback time of the provisional notification messages exceeds the output grace period, shortens the lengths of the application notification messages, starting with the application notification message having the lowest application priority, from the plurality of application notification messages included in the provisional notification message, to generate notification messages that can be played back within the output grace period.

[0010] The storage medium described in claim 20 is an information processing program for causing a computer to execute the following steps: acquiring, from an application, character string information indicating a character string constituting a notification message to be output audibly to a driver of a moving body; intent information including application priority information indicating an application priority set for each application; and an output deferred timing indicating when the notification of the notification message should be completed; calculating an output deferred time which is the remaining time until the output deferred timing is reached; and generating, based on the character string information, a notification message to be output audibly to the driver. The generating step generates a provisional notification message including a plurality of application notification messages having different application priorities, and, if the total playback time of the provisional notification messages exceeds the output deferred time, shortens the lengths of the application notification messages from the plurality of application notification messages included in the provisional notification message in order from the application notification message with the lowest application priority, thereby generating notification messages that can be played back within the output deferred time. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an application device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an overview of information processing according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of intent information according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of intent information according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of intent information according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing an information processing procedure according to the embodiment. [Figure 9]FIG. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of an application device or an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, a mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

[0013] (Embodiment) [1. Information Processing System Configuration] First, the configuration of an information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment. As shown in Fig. 1, the information processing system 1 includes an application device 10 (hereinafter also referred to as an application device 10) and an information processing device 100. The application device 10 and the information processing device 100 are connected to each other via a predetermined network N so as to be able to communicate with each other via a wired or wireless connection. Note that the information processing system 1 shown in Fig. 1 may include a plurality of application devices 10 and a plurality of information processing devices 100.

[0014] The application device 10 is an information processing device that executes an application that provides information to a driver of a vehicle (an example of a mobile object), and is realized by, for example, a server device or a cloud system. FIG. 1 illustrates a case where the application device 10 is realized by a cloud system. Note that, in the following description, an example will be described in which the mobile object is a vehicle, but the mobile object is not limited to a vehicle. The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile object, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0015] In the following description, application devices 10 will be referred to as application devices 10-1 and 10-2 depending on the type of application executed by application device 10. For example, application device 10-1 is an application device 10 that executes application #1 (hereinafter also referred to as application #1). In addition, for example, application device 10-2 is an application device 10 that executes application #2 (hereinafter also referred to as application #2). In the following description, application devices 10-1 and 10-2 will be referred to as application devices 10 when there is no particular distinction between them.

[0016] The information processing device 100 is, for example, a stationary navigation device or a drive recorder installed in a vehicle. Note that the information processing device 100 is not limited to a navigation device or a drive recorder, and may also be a portable terminal such as a smartphone used by a vehicle occupant.

[0017] [2. Application Device Configuration] Next, the configuration of the application device according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the application device according to the embodiment. As shown in Fig. 2, the application device 10 has a communication unit 11, a storage unit 12, and a control unit 13.

[0018] (Communications Department 11) The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is also connected to a network N (see FIG. 1) by wire or wirelessly.

[0019] (Storage unit 12) The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 12 stores information about a machine learning model used to generate intent information. The storage unit 12 also stores application priority information indicating application priority set for each application.

[0020] In addition, the memory unit 12 stores status information including sensor information such as vehicle location information, guidance route information including searched routes and destinations, user interest information, user schedule information, etc., linked to the ID of the information processing device 100 (an example of identification information that identifies the information processing device 100 installed in each vehicle).

[0021] (Control unit 13) The control unit 13 is a controller, and is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like, executing various programs (corresponding to examples of information processing programs) stored in a storage device inside the application device 10 using a storage area such as a RAM as a working area. In the example shown in FIG. 2, the control unit 13 includes an acquisition unit 131, an intent generation unit 132, and a transmission unit 133.

[0022] (Acquisition part 131) The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires a machine learning model that receives an application notification message as input and outputs intent information. For example, the acquisition unit 131 acquires a machine learning model that has been trained based on training data including a combination of the application notification message and the intent information. Upon acquiring the machine learning model, the acquisition unit 131 stores information about the acquired machine learning model in the storage unit 12.

[0023] Furthermore, the acquisition unit 131 acquires sensor information including location information of the information processing device 100 and the like, and the results of voice recognition of the user in the vehicle, from the information processing device 100 via the communication unit 11.

[0024] (Intent generation unit 132) The intent generation unit 132 generates intent information, which is information that has been converted from metadata about information about a notification text that is to be output as voice to the driver of a moving object. Here, the intent information according to the embodiment will be described with reference to Figs. 5 to 7. Figs. 5 to 7 are diagrams showing an example of intent information according to the embodiment.

[0025] The intent generation unit 132 generates an application notification sentence related to a notification. In the example shown in Fig. 5, the intent generation unit 132 generates application notification sentence #13 such as "Turn right at the upcoming intersection. Convenience store A is the landmark. Please drive in the innermost lane" based on the guide route information and location information of the information processing device 100. After generating application notification sentence #13, the intent generation unit 132 generates intent information T1 based on the generated application notification sentence #13.

[0026] For example, the intent generation unit 132 extracts a character string from the application notification #13. Subsequently, the intent generation unit 132 generates intent information T1 including the extracted character string. In the example shown in FIG. 5, the intent generation unit 132 refers to the storage unit 12 and acquires a machine learning model #1 that receives the application notification #1 as input and outputs a character string indicating the notification content. For example, the intent generation unit 132 acquires a machine learning model #1 that has been trained based on training data including a combination of the application notification #1 and a character string indicating the notification content. After acquiring the machine learning model #1, the intent generation unit 132 inputs the application notification #13 into the acquired machine learning model #1 and acquires the character string indicating the notification content, such as "next intersection," "turn right," "convenience store," and "innermost," output from the machine learning model #1. The intent generation unit 132 generates intent information T1 including the acquired character string.

[0027] Alternatively, instead of using a machine learning model, the intent generation unit 132 may extract character strings from the application notification sentence using a natural language processing technique such as morphological analysis. For example, the intent generation unit 132 performs morphological analysis on the application notification sentence ##13 to extract words such as "up next," "intersection," "turn right," "convenience store A," "first," and "inside" from the application notification sentence #1. Next, the intent generation unit 132 compares the extracted words with predetermined dictionary information to determine the word's part of speech, conjugation type, synonyms, and the like. Next, based on the result of comparing the extracted words with the predetermined dictionary information, the intent generation unit 132 generates intent information T1 including character strings indicating the notification content, such as "next intersection," "turn right," "convenience store," and "first,"

[0028] Furthermore, the intent generation unit 132 generates intent information including intention information indicating the type of notification set for each character string. For example, the intent generation unit 132 refers to the storage unit 12 and acquires a machine learning model #2 that receives an application notification sentence as input and outputs intention information indicating the type of notification for each character string. For example, the intent generation unit 132 acquires a machine learning model #2 that has been trained based on training data including a combination of the application notification sentence and the intention information set for each character string. For example, the intent generation unit 132 acquires a machine learning model #3 that has been trained to output, as output information, a character string indicating the type of notification for each character string when an application notification sentence is input.

[0029] When the intent generation unit 132 acquires the machine learning model #2, it inputs the app notification sentence #1 into the acquired machine learning model #2 and acquires intent information for each character string output from the machine learning model #2. In FIG. 5, the intent generation unit 132 acquires intent information, which is the character string "route" indicating the type of notification, for the character strings "next intersection" and "right turn." Furthermore, the intent generation unit 132 acquires intent information, which is the character string "landmark" indicating the type of notification, for the character string "convenience store." Furthermore, the intent generation unit 132 acquires intent information, which is the character string "driving lane" indicating the type of notification, for the character string "innermost." The intent generation unit 132 generates intent information T1 including the intent information for each acquired character string.

[0030] Furthermore, the intent generation unit 132 generates intent information including content priority information indicating the content priority set for each character string. For example, the intent generation unit 132 refers to the storage unit 12 and acquires a machine learning model #3 that receives an application notification as input and outputs content priority information indicating the content priority for each character string. For example, the intent generation unit 132 acquires a machine learning model #3 that has been trained based on training data including a combination of the application notification and the content priority information set for each character string. For example, the intent generation unit 132 acquires a machine learning model #3 that has been trained to output, as output information, a number indicating the content priority for each character string (e.g., numbers such as 1, 2, 3, ... in descending order of priority) when an application notification is input.

[0031] When the intent generation unit 132 acquires the machine learning model #3, it inputs the app notification text #1 into the acquired machine learning model #3 and acquires content priority information for each character string output from the machine learning model #3. In FIG. 5, the intent generation unit 132 acquires content priority information, which is the number "1" indicating content priority, for the character strings "next intersection" and "turn right." The intent generation unit 132 also acquires content priority information, which is the number "3" indicating content priority, for the character string "convenience store." The intent generation unit 132 also acquires content priority information, which is the number "2" indicating content priority, for the character string "innermost." The intent generation unit 132 generates intent information T1 including content priority information for each acquired character string.

[0032] Furthermore, the intent generation unit 132 generates intent information that further includes notification priority information indicating the notification priority set for each piece of intent information. For example, the intent generation unit 132 refers to the storage unit 12 and acquires a machine learning model #4 that receives an application notification as input and outputs notification priority information. For example, the intent generation unit 132 acquires a machine learning model #4 that has been trained based on training data including a combination of an application notification and notification priority information. For example, the intent generation unit 132 acquires a machine learning model #4 that has been trained to output, as output information, a number indicating the notification priority of a notification related to the application notification (e.g., numbers 1, 2, 3, ... in descending order of priority) when an application notification is input.

[0033] When the intent generation unit 132 acquires the machine learning model #4, it inputs the application notification #1 into the acquired machine learning model #4 and acquires the notification priority information output from the machine learning model #4. In FIG. 5, the intent generation unit 132 acquires notification priority information (In priority) that is the number "1" indicating the notification priority for the notification related to the application notification #1. The intent generation unit 132 generates intent information T1 that includes the acquired notification priority information.

[0034] Furthermore, the intent generation unit 132 generates intent information that further includes application priority information indicating the application priority set for each application for which the intent information is generated. Although not shown in the drawings, in FIG. 5, the intent generation unit 132 refers to the storage unit 12 and acquires application priority information indicating the application priority set for application #1. For example, the intent generation unit 132 acquires a number indicating the application priority set for each application (for example, numbers such as 1, 2, 3, ... in descending order of priority). In FIG. 5, the intent generation unit 132 acquires application priority information that is the number "1" indicating the application priority. The intent generation unit 132 generates intent information T1 that includes the acquired application priority information.

[0035] Furthermore, the intent generation unit 132 may determine overall priority information indicating the overall priority of the intent information based on the notification priority information and the application priority information. For example, the intent generation unit 132 may calculate an average value by adding together a number indicating the notification priority and a number indicating the application priority, and use the calculated average value as the overall priority information indicating the overall priority of the intent information. Although not shown in the figure, in FIG. 5, the intent generation unit 132 calculates an average value "1" by adding together a number "1" indicating the notification priority and a number "1" indicating the application priority, and uses the average value "1" as the overall priority information indicating the overall priority of the intent information T1.

[0036] In the example shown in FIG. 6, the intent generation unit 132 generates an application notification sentence #24 based on the location information of the information processing device 100, saying, "Turn right at the upcoming intersection, and 400 meters ahead you will find the cake shop "XXX" that has been talked about recently on TV. We recommend their Mont Blanc with a light sweetness." Next, the intent generation unit 132 generates intent information T2 based on the application notification sentence #24 in the same manner as in the case of FIG. 5 described above. Specifically, the intent generation unit 132 generates intent information T2 including character strings indicating notification content such as "next intersection," "turn right," "400 meters ahead," "cake shop," "store name "XXX," "popular on TV," "Mont Blanc," and "lightly sweet."

[0037] 6, the intent generation unit 132 acquires intent information, which is a character string "route," indicating a type of notification, for the character strings "next intersection," "turn right," and "400 m ahead." The intent generation unit 132 also acquires intent information, which is a character string "recommended," indicating a type of notification, for the character strings "cake shop," "store name "XXX," and "popular on TV." The intent generation unit 132 also acquires intent information, which is a character string "recommended," indicating a type of notification, for the character strings "Mont Blanc" and "low-sweet." The intent generation unit 132 generates intent information T2 including intent information for each acquired character string.

[0038] 6, the intent generation unit 132 acquires content priority information, which is the number "1" indicating the content priority, for the character strings "next intersection," "turn right," "400 meters ahead," and "cake shop." The intent generation unit 132 also acquires content priority information, which is the number "3" indicating the content priority, for the character string "store name 'XXX'." The intent generation unit 132 also acquires content priority information, which is the number "4" indicating the content priority, for the character strings "popular on TV" and "low-sweet." The intent generation unit 132 also acquires content priority information, which is the number "2" indicating the content priority, for the character string "Mont Blanc." The intent generation unit 132 generates intent information T2 including content priority information for each of the acquired character strings.

[0039] 6, the intent generation unit 132 acquires notification priority information (In priority) that is the number "2" indicating the notification priority for the notification related to application notification #2. The intent generation unit 132 generates intent information T2 that includes the acquired notification priority information.

[0040] 6, the intent generation unit 132 refers to the storage unit 12 and acquires application priority information, which is the number "2" indicating the application priority set for application #2. The intent generation unit 132 generates intent information T2 including the acquired application priority information.

[0041] Also, although not shown in the figure, in Figure 6, the intent generation unit 132 adds and averages the number "2" indicating the notification priority and the number "2" indicating the application priority, and sets the average value "2" as the overall priority information indicating the overall priority of the intent information T2.

[0042] In the example shown in FIG. 7, the intent generation unit 132 generates an application notification sentence #34 that reads, "A passing rain is forecast near your home in one hour. Is it okay to bring in the laundry? (If the user's response is "No,") a video phone call will be connected to your home. / (If the user's response is "Yes," end)." Next, the intent generation unit 132 generates intent information T3 based on the application notification sentence #34 in the same manner as in the case of FIG. 5 described above. Specifically, the intent generation unit 132 generates intent information T3 that includes character strings indicating notification content, such as "passing rain," "near your home," "one hour later," "laundry," "Yes / No," and "Yes: end, No: video phone call."

[0043] 7, the intent generation unit 132 acquires intent information, which is a character string "weather forecast," indicating the type of notification, for the character strings "passing shower," "near home," and "one hour later." The intent generation unit 132 also acquires intent information, which is a character string "information," indicating the type of notification, for the character string "laundry." The intent generation unit 132 also acquires intent information, which is a character string "user response," indicating the type of notification, for the character strings "Yes / No" and "Yes: finished, No: video phone." The intent generation unit 132 generates intent information T3 including intent information for each of the acquired character strings.

[0044] 7, the intent generation unit 132 acquires content priority information, which is the number "1" indicating content priority, for the character strings "passing shower" and "near home." The intent generation unit 132 also acquires content priority information, which is the number "2" indicating content priority, for the character string "one hour later." The intent generation unit 132 also acquires content priority information, which is the number "3" indicating content priority, for the character string "laundry." The intent generation unit 132 also acquires content priority information, which is the number "4" indicating content priority, for the character strings "Yes / No" and "Yes: Finish, No: Video phone." The intent generation unit 132 generates intent information T3 including content priority information for each acquired character string.

[0045] 7, the intent generation unit 132 acquires notification priority information (In priority) that is the number "3" indicating the notification priority for the notification related to application notification #3. The intent generation unit 132 generates intent information T3 that includes the acquired notification priority information.

[0046] 7, the intent generation unit 132 refers to the storage unit 12 and acquires application priority information, which is the number "3" indicating the application priority set for application #3. The intent generation unit 132 generates intent information T3 including the acquired application priority information.

[0047] Also, although not shown in the figure, in Figure 7, the intent generation unit 132 adds and averages the number "3" indicating the notification priority and the number "3" indicating the application priority, and sets the average value "3" as the overall priority information indicating the overall priority of the intent information T3.

[0048] (Transmitter 133) The transmission unit 133 transmits the intent information to the information processing device 100, which generates a notification sentence based on the intent information. For example, when the intent generation unit 132 generates the intent information, the transmission unit 133 transmits the intent information to the information processing device 100. For example, when the intent generation unit 132 generates pieces of intent information T1 to T3, the transmission unit 133 transmits the pieces of intent information T1 to T3 to the information processing device 100.

[0049] 3. Configuration of Information Processing Device Next, the configuration of the information processing device according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device according to the embodiment. As shown in Fig. 3, the information processing device 100 has a communication unit 110, a storage unit 120, a sensor unit 130, a voice output unit 140, a voice recognition unit 150, and a control unit 160.

[0050] (Communication unit 110) The communication unit 110 is realized by, for example, a NIC, a modem chip, an antenna module, etc. The communication unit 110 is also connected to a network N (see FIG. 1) by wire or wirelessly.

[0051] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

[0052] (Sensor unit 130) The sensor unit 130 includes various sensors. For example, the sensor unit 130 includes a GNSS (Global Navigation Satellite System). The GNSS sensor receives radio waves including positioning data transmitted from navigation satellites using the GNSS. The positioning data is used to detect the absolute position of the vehicle from latitude and longitude information, etc. The GNSS used may be, for example, a GPS (Global Positioning System), or another system. The sensor unit 130 outputs the positioning data generated by the GNSS sensor to the control unit 160.

[0053] The sensor unit 130 also includes a vehicle speed sensor. The vehicle speed sensor detects the traveling speed of the vehicle and generates vehicle speed data corresponding to the traveling speed. The sensor unit 130 also outputs the vehicle speed data generated by the vehicle speed sensor to the control unit 160.

[0054] The sensor unit 130 also includes an acceleration sensor. The acceleration sensor detects the acceleration of the vehicle and generates acceleration data corresponding to the running acceleration. The sensor unit 130 outputs the acceleration data generated by the acceleration sensor to the control unit 160. The sensor unit 130 may also calculate the speed based on the acceleration data.

[0055] The sensor unit 130 also includes a camera. The camera captures images of the surroundings of the vehicle and generates captured images under the control of the control unit 160. The sensor unit 130 also outputs the captured images generated by the camera to the control unit 160.

[0056] (Audio output unit 140) The audio output unit 140 includes a speaker, converts the digital audio signal input from the control unit 160 into an analog audio signal by D / A (Digital / Analog) conversion, and outputs audio corresponding to the analog audio signal from the speaker.

[0057] Furthermore, the audio output unit 140 outputs to the driver by voice the notification sentences generated by the generation unit 163. Specifically, the audio output unit 140 outputs by voice the plurality of application notification sentences in turn in the order determined by the generation unit 163.

[0058] (Speech recognition unit 150) The voice recognition unit 150 is realized by the control unit 160 executing a voice recognition application stored in the storage unit 120. The voice recognition unit 150 also recognizes the voice of the driver. The voice recognition unit 150 also performs a process of converting the driver's voice received by a microphone that is part of a voice input unit (not shown) into text data. The process of converting into text data may be performed by a dedicated server (not shown).

[0059] Furthermore, the speech recognition unit 150 has a barge-in function. When the decision unit 164 determines to enable the barge-in function, the speech recognition unit 150 enables the barge-in function. On the other hand, when the decision unit 164 determines not to enable the barge-in function, the speech recognition unit 150 disables the barge-in function.

[0060] (control unit 160) The control unit 160 is a controller, and is realized by, for example, a CPU, an MPU, an ASIC, an FPGA, or the like, executing various programs (corresponding to examples of information processing programs) stored in a storage device inside the information processing device 100 using a storage area such as a RAM as a work area. In the example shown in FIG. 3, the control unit 160 has an acquisition unit 161, a calculation unit 162, a generation unit 163, and a determination unit 164.

[0061] (Acquisition part 161) The acquisition unit 161 acquires various types of information. Specifically, the acquisition unit 161 acquires situation information including driving information related to the driving situation of the mobile object and driving information related to the driving situation of the driver. For example, the acquisition unit 161 acquires information related to the driving speed of the mobile object, the density of traveling vehicles, traffic congestion information, road type (highway, urban street, residential road, suburban road, mountain road, straight road, intersection, curve, etc.), time, and weather (daytime, nighttime, sunny, rain, snow, snow cover, icy, etc.) from the sensor unit 130 as examples of driving information. Note that the acquisition unit 161 may also acquire information related to traffic congestion information, road type (highway, urban street, residential road, suburban road, mountain road, straight road, intersection, curve, etc.), time, and weather (daytime, nighttime, sunny, rain, snow, snow cover, icy, etc.) from an external information providing device.

[0062] Furthermore, the acquisition unit 161 acquires, as examples of driving information, attribute information of the driver, whether or not there is a passenger, and attribute information of the passenger. For example, the acquisition unit 161 acquires attribute information of the driver and attribute information of the passenger that are registered in advance in a predetermined database. Furthermore, the acquisition unit 161 may acquire information on whether or not there is a passenger from the driver through self-declaration by the driver.

[0063] Furthermore, the acquisition unit 161 acquires intent information from each of the multiple applications. Here, an overview of information processing by the information processing device 100 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an overview of information processing according to an embodiment. In the example shown in Fig. 4, the acquisition unit 161 of the information processing device 100 simultaneously acquires intent information T1 to T3 from three applications #1 to #3, respectively.

[0064] For example, the acquisition unit 161 acquires, from an application, intent information including character string information indicating character strings constituting a notification sentence to be output as voice to the driver of the mobile body and intent information indicating a type of notification set for each character string. The acquisition unit 161 also acquires intent information further including content priority information indicating a content priority set for each character string. The acquisition unit 161 also acquires intent information further including notification priority information indicating a notification priority set for each intent information. The acquisition unit 161 also acquires intent information further including application priority information indicating an application priority set for each application.

[0065] The acquisition unit 161 also acquires an output deferral timing indicating the timing at which the notification of the notification message should be completed. Here, the output deferral timing may be the time at which the notification of the notification message should be completed or the location at which the notification of the notification message should be completed. For example, the acquisition unit 161 acquires the output deferral timing based on intent information acquired from an application. For example, the acquisition unit 161 acquires information "next intersection" as the output deferral timing of the notification message related to the intent information T1 based on the intent information T1. Furthermore, the acquisition unit 161 acquires information "next intersection" and information "cake shop" as the output deferral timing of the notification message related to the intent information T2 based on the intent information T2. ​​Furthermore, the acquisition unit 161 acquires information "one hour later" as the output deferral timing of the notification message related to the intent information T3 based on the intent information T3.

[0066] (Calculation unit 162) The calculation unit 162 calculates an output grace period, which is the remaining time until the output grace period is reached. Specifically, the calculation unit 162 calculates the output grace period based on the output grace period acquired by the acquisition unit 161. For example, when the output grace period is a predetermined time, the calculation unit 162 calculates the remaining time from the current time until the predetermined time, which is the output grace period, as the output grace period. Furthermore, when the output grace period is a predetermined position, the calculation unit 162 calculates the estimated arrival time from the current position to the predetermined position, which is the output grace period, as the output grace period. For example, the calculation unit 162 calculates the estimated arrival time based on the route to the predetermined position and the expected average speed of the moving object.

[0067] Furthermore, the calculation unit 162 calculates a safety level indicating the degree of safety of the traveling situation of the mobile object or the driving situation of the driver, based on the situation information acquired by the acquisition unit 161. For example, the calculation unit 162 calculates a higher safety level as the vehicle speed is lower. Furthermore, for example, the calculation unit 162 calculates a higher safety level when traveling on suburban roads than when traveling on urban roads.

[0068] Furthermore, the calculation unit 162 calculates a margin of safety that indicates the degree of safety that the driver has to pay attention to things other than driving the moving object, based on the situation information acquired by the acquisition unit 161. For example, the calculation unit 162 calculates the margin of safety to be higher when the vehicle is traveling on a straight road than when it is not. Furthermore, for example, the calculation unit 162 calculates the margin of safety to be lower when the weather is rainy than when it is sunny.

[0069] Next, after calculating the safety level and the margin level, the calculation unit 162 calculates an output margin time based on the calculated safety level and margin level. Specifically, the higher the safety level, the longer the output margin time calculated by the calculation unit 162. For example, the higher the safety level, the longer the output margin time calculated by the calculation unit 162 than the estimated arrival time. Furthermore, the higher the margin level, the longer the output margin time calculated by the calculation unit 162. For example, the higher the safety level, the longer the output margin time calculated by the calculation unit 162 than the estimated arrival time.

[0070] (Generation unit 163) The generation unit 163 generates a notification sentence to be output aloud to the driver based on the intent information. For example, the generation unit 163 generates a notification sentence to be output aloud to the driver based on character string information and intention information included in the intent information. More specifically, the generation unit 163 generates a notification sentence including a character string based on the character string information and the intention information, and changes the expression of the notification sentence so that the timing at which the audio output of the generated notification sentence is completed is before the output deferral timing. The generation unit 163 changes the expression of the notification sentence to generate a notification sentence that can be played back within the output deferral time.

[0071] For example, the generation unit 163 generates a provisional notification message including a character string, and if the playback time of the provisional notification message does not exceed the output grace period, changes the wording of the notification message to a more polite wording than the provisional notification message. For example, the generation unit 163 generates a provisional notification message, "Turn right at the next intersection," with a playback time of 2 seconds, based on the character strings "next intersection" and "turn right" included in the intent information T1 shown in FIG. 5 and the character string "route" that is the intention information corresponding to the character strings "next intersection" and "turn right." In this case, if the output grace period is 3 seconds, the generation unit 163 changes the wording of the notification message to "Turn right at the next intersection" (playback time 2.5 seconds), which is a more polite wording than the provisional notification message, because the playback time (2 seconds) of the provisional notification message does not exceed the output grace period (3 seconds).

[0072] Furthermore, the generation unit 163 generates a provisional notification sentence including a character string, and if the playback time of the provisional notification sentence exceeds the output grace period, changes the expression of the notification sentence to a style that does not include an auxiliary verb at the end of the sentence. For example, the generation unit 163 generates a provisional notification sentence, "Turn right at the next intersection," with a playback time of 2.5 seconds, based on the character strings "next intersection" and "turn right" included in the intent information T1 shown in FIG. 5 and the character string "route," which is the intention information corresponding to the character strings "next intersection" and "turn right." In this case, if the output grace period is 2 seconds, the generation unit 163 changes the expression of the notification sentence to "Turn right at the next intersection" (playback time 2 seconds), which is a style that does not include an auxiliary verb at the end of the sentence. Note that if the playback time of the provisional notification sentence exceeds the output grace period, the generation unit 163 may change the expression of the notification sentence to a noun-end style.

[0073] Furthermore, the generation unit 163 generates a notification to be output aloud to the driver based on the character string information, the intention information, and the content priority information. Specifically, the generation unit 163 generates a notification including a character string based on the content priority so that the audio output of the notification is completed before the output deferral timing. More specifically, the generation unit 163 generates a provisional notification including a plurality of character strings with different content priorities, and if the playback time of the provisional notification exceeds the output deferral time, the generation unit 163 deletes the character strings included in the provisional notification, starting with the character string with the lowest content priority, to generate a notification that can be played within the output deferral time.

[0074] For example, the generation unit 163 generates a provisional notification message that reads, "Turn right at the upcoming intersection. Convenience store A is the landmark. Please drive in the innermost lane," which includes all of the character strings "next intersection" and "turn right" with content priority of "1," the character string "innermost" with content priority of "2," and the character string "convenience store" with content priority of "3" that are included in the intent information T1 shown in Fig. 5. Next, the generation unit 163 estimates the playback time of the provisional notification message to be 8 seconds.

[0075] In this case, if the output grace period is 7 seconds, the generation unit 163 deletes the character string "convenience store" with a low content priority of "3" from among the multiple character strings included in the provisional notification, because the playback time of the provisional notification (8 seconds) exceeds the output grace period (7 seconds). Next, the generation unit 163 generates a provisional notification saying "Turn right at the next intersection. Please drive in the innermost lane.", which includes the character strings "next intersection" and "turn right" with a content priority of "1" and the character string "innermost lane" with a content priority of "2". Next, the generation unit 163 estimates the playback time of the provisional notification to be 6 seconds. Since the playback time of the provisional notification (6 seconds) does not exceed the output grace period (7 seconds), the generation unit 163 adopts the generated provisional notification as the notification.

[0076] Furthermore, the generation unit 163 generates a plurality of different app notification sentences based on a plurality of different intent information acquired from a plurality of different applications, and determines, based on the notification priority information acquired by the acquisition unit 161, to output the app notification sentences in audio in order of the app notification sentences generated based on the intent information with the highest notification priority.

[0077] For example, the generation unit 163 generates three different application notification messages #1' to #3' based on the intent information #1 to #3 acquired from three different applications #1 to #3, respectively. Next, the generation unit 163 acquires information that the notification priority of application #1 is "1," the notification priority of application #2 is "2," and the notification priority of application #3 is "3" from the notification priority information included in each of the intent information #1 to #3. Next, when outputting notification messages including three application notification messages #1' to #3', the generation unit 163 determines that the application notification message #1' generated based on the intent information #1 with the highest notification priority will be output first, the application notification message #2' generated based on the intent information #2 with the next highest notification priority will be output second, and the application notification message #3' generated based on the intent information #3 with the lowest notification priority will be output last.

[0078] Furthermore, the generation unit 163 may determine the order in which to output a plurality of application notification messages by voice based on the application priority instead of the notification priority. Specifically, the generation unit 163 generates a plurality of different application notification messages based on a plurality of different intent information acquired from a plurality of different applications, and determines, based on the application priority information acquired by the acquisition unit 161, to output the application notification messages of the applications with the highest application priority as voice messages.

[0079] For example, from the application priority information included in each of the intent information #1 to #3, the generation unit 163 obtains information that the application priority of application #1 is "1," the application priority of application #2 is "2," and the application priority of application #3 is "3." Next, when outputting a notification message including three application notifications #1' to #3', the generation unit 163 determines that application notification #1' generated based on intent information #1 with the highest application priority should be output aloud first, application notification #2' generated based on intent information #2 with the next highest application priority should be output aloud second, and application notification #3' generated based on intent information #3 with the lowest application priority should be output aloud last.

[0080] Furthermore, the generation unit 163 may determine the order in which to output a plurality of application notification messages by voice, based on an overall priority indicating an overall priority, instead of the notification priority or the application priority. Specifically, the generation unit 163 generates a plurality of different application notification messages based on a plurality of different intent information acquired from a plurality of different applications, and determines, based on the overall priority information acquired by the acquisition unit 161, to output a plurality of application notification messages by voice in order starting from the application notification message with the highest overall priority.

[0081] For example, the generation unit 163 obtains information that the overall priority of application #1 is "1," the overall priority of application #2 is "2," and the overall priority of application #3 is "3" from the overall priority information included in each of the intent information #1 to #3. Next, when outputting a notification message including three application notifications #1' to #3', the generation unit 163 determines that application notification #1' generated based on intent information #1 with the highest overall priority should be output aloud first, application notification #2' generated based on intent information #2 with the next highest overall priority should be output aloud second, and application notification #3' generated based on intent information #3 with the lowest overall priority should be output aloud last.

[0082] The generation unit 163 also generates all app notification sentences that can be generated based on each piece of intent information acquired from each application. For example, based on intent information T1 shown in FIG. 5 acquired from application #1, the generation unit 163 generates two app notification sentences #11′ and #11, “Turn right at the next intersection” (playback time: 2 seconds) and “Turn right at the next intersection” (playback time: 2.5 seconds), which contain only character strings with a content priority of “1.” The generation unit 163 also generates app notification sentence #12, “Turn right at the next intersection. Please drive in the innermost lane.” (playback time: 6 seconds), which contains character strings with a content priority of “2” or lower. The generation unit 163 also generates app notification sentence #13, “Turn right at the upcoming intersection. Convenience store A is the landmark. Please drive in the innermost lane.”, which contains character strings with all notification priorities. In this way, the generation unit 163 generates all app notification sentences #11′, #11, #12, and #13 that can be generated based on intent information T1.

[0083] Furthermore, based on the intent information T2 shown in FIG. 6 acquired from application #2, the generation unit 163 generates two application notification sentences #21' and #21, "Turn right at the next intersection, and there is a cake shop 400 meters ahead" and "Turn right at the next intersection, and there is a cake shop 400 meters ahead," which contain only character strings with a content priority of "1." The generation unit 163 also generates application notification sentence #22, "Turn right at the next intersection, and there is a cake shop 400 meters ahead. We recommend the Mont Blanc," which contains a character string with a content priority of "2" or lower. The generation unit 163 also generates application notification sentence #23, "Turn right at the next intersection, and there is a cake shop 'XXX' 400 meters ahead. We recommend the Mont Blanc," which contains a character string with a content priority of "3" or lower. Furthermore, the generation unit 163 generates application notification sentence #24, which includes character strings of all notification priorities and reads, "Turn right at the upcoming intersection, and 400 meters ahead you will find the cake shop "XXX," which has been a hot topic on TV recently. We recommend the Mont Blanc, which has a mild sweetness." In this way, the generation unit 163 generates all of the application notification sentences #21', #21, #22, #23, and #24 that can be generated based on the intent information T2.

[0084] Furthermore, based on the intent information T3 shown in FIG. 7 acquired from the application #3, the generation unit 163 generates an application notification sentence #31 saying "A passing rain is forecast to be near your home." which includes only a character string with a content priority of "1." The generation unit 163 also generates an application notification sentence #32 saying "A passing rain is forecast to be near your home in one hour." which includes a character string with a content priority of "2" or lower. The generation unit 163 also generates an application notification sentence #33 saying "A passing rain is forecast to be near your home in one hour. Is it OK to take in the laundry?" which includes a character string with a content priority of "3" or lower. The generation unit 163 also generates an application notification sentence #34 saying "A passing rain is forecast to be near your home in one hour. Is it OK to take in the laundry? (If the user's response is "No,") a video phone call will be connected to your home. / (If the user's response is "Yes," the process ends.)" which includes character strings with all notification priorities. In this way, the generation unit 163 generates all of the application notification messages #31, #32, #33, and #34 that can be generated based on the intent information T3.

[0085] Next, after generating all of the app notification messages that can be generated, generation unit 163 estimates the playback time of each app notification message. For example, generation unit 163 estimates the playback time of each of app notification messages #11', #11, #12, #13, #21', #21, #22, #23, #24, #31, #32, #33, and #34 that can be generated.

[0086] Next, generation unit 163 estimates the playback time of each application notification message and selects the application notification message with the longest playback time from among the application notification messages. For example, generation unit 163 selects application notification message #13, which is the longest notification message for application #1, application notification message #24, which is the longest notification message for application #2, and application notification message #34, which is the longest notification message for application #3.

[0087] Next, generation unit 163 selects the application notification message with the longest playback time from among the application notifications, and estimates the total playback time of notifications that include all of the selected application notifications. For example, generation unit 163 estimates the total playback time of notifications that include all of the selected application notifications #13, #24, and #34. Next, generation unit 163 determines whether the total playback time of the notifications is less than or equal to the output grace time.

[0088] When determining that the total playback time of the notification messages is equal to or less than the output grace time, generation unit 163 decides to output the selected application notification messages. When determining to output the selected application notification messages, generation unit 163 generates notification messages that include all of the selected application notification messages.

[0089] On the other hand, if generation unit 163 determines that the total playback time of the notification messages is not equal to or less than the output grace period, it selects the next shortest application notification message in ascending order of application priority (or may be the notification priority or the overall priority). For example, if generation unit 163 determines that the total playback time of all of the selected notification messages including application notification message #13, application notification message #24, and application notification message #34 is not equal to or less than the output grace period, it selects the next shortest application notification message #33 in place of application notification message #34, in ascending order of application priority, starting with application #3. Next, when generation unit 163 selects the next shortest application notification message in ascending order of application priority, it estimates the total playback time of all of the selected notification messages including all of the selected application notification messages.

[0090] In this way, the generation unit 163 generates a provisional notification message including multiple app notification messages with different app priorities (or, it may be, notification priorities or overall priorities), and if the total playback time of the provisional notification message exceeds the output grace period, the length of the app notification message is shortened in order from the app notification message with the lowest app priority (or, it may be notification priorities or overall priorities) among the multiple app notification messages included in the provisional notification message, and a notification message that can be played within the output grace period is generated.

[0091] (Decision unit 164) The determination unit 164 determines an acceptance time for accepting a response from the driver to a notification message when a notification message regarding notification content requiring a response from the driver is to be output to the driver by voice, based on the situation information acquired by the acquisition unit 161. Specifically, the determination unit 164 determines the acceptance time based on a comparison between the playback time of the notification message generated by the generation unit 163 and the output grace period calculated by the calculation unit 162. More specifically, the determination unit 164 determines that the acceptance time is short when the total playback time of the provisional notification message including the character string generated by the generation unit 163 is longer than the output grace period by less than a predetermined time.

[0092] When the determination unit 164 determines that the reception time is short, the generation unit 163 generates a notification message including only the numbers of the options to be selected by the driver and the items of the options. For example, the generation unit 163 generates a notification message such as "Up ahead, we have recommended restaurants 1. Italian, 2. Chinese, and 3. Japanese. Which would you like?" that includes only the numbers of the options "1," "2," and "3" and the items of the options "Italian," "Chinese," and "Japanese."

[0093] Furthermore, when the total playback time of the provisional notification messages is longer than the output grace time by a predetermined time or more, the determination unit 164 determines that there is no reception time. When the determination unit 164 determines that there is no reception time, the generation unit 163 generates a notification message that does not include notification content that requires a response from the driver. For example, when the determination unit 164 determines that there is no reception time, instead of the app notification message #34 that includes the user's response and reads, "A passing rain is forecast to be near your home in one hour. Is it OK to bring in the laundry? (If the user's response is "No,") we will connect you to your home via video phone. / (If the user's response is "Yes," end)," the generation unit 163 generates an app notification message #34' that deletes the part below the user's response and reads, "A passing rain is forecast to be near your home in one hour. Is it OK to bring in the laundry?"

[0094] Furthermore, the determination unit 164 determines the reception time based on the safety level and margin calculated by the calculation unit 162. For example, the higher the safety level calculated by the calculation unit 162, the longer the reception time determined by the determination unit 164. Furthermore, the higher the margin calculated by the calculation unit 162, the longer the reception time determined by the determination unit 164.

[0095] Furthermore, the decision unit 164 determines whether to enable the barge-in function of the speech recognition unit 150 based on the situation information acquired by the acquisition unit 161. Specifically, the decision unit 164 determines whether to enable the barge-in function of the speech recognition unit 150 based on the safety level and margin calculated by the calculation unit 162. For example, the decision unit 164 determines to enable the barge-in function when the safety level calculated by the calculation unit 162 exceeds a first threshold. Furthermore, the decision unit 164 determines to enable the barge-in function when the margin calculated by the calculation unit 162 exceeds a second threshold.

[0096] (Transmitter 165) The transmitting unit 165 transmits the information acquired by the acquiring unit 161 to the application device 10. Specifically, the transmitting unit 165 transmits situation information including travel information related to the travel situation of the moving object and driving information related to the driving situation of the driver acquired by the acquiring unit 161 to the application device 10. For example, the transmitting unit 165 transmits the situation information to the application device 10 in real time. Furthermore, for example, the transmitting unit 165 may transmit the situation information to the application device 10 at predetermined time intervals (for example, every 30 seconds or every minute).

[0097] [4. Information processing flow] Next, a procedure of information processing according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of information processing according to the embodiment. In the example shown in Fig. 8, the calculation unit 162 of the information processing device 100 calculates an output grace period, which is the remaining time until the output grace period is reached (step S101).

[0098] The generation unit 163 of the information processing device 100 generates all app notification messages that can be generated based on each piece of intent information acquired from each application (step S102). After generating all app notification messages that can be generated, the generation unit 163 estimates the playback time of each app notification message (step S103).

[0099] After estimating the playback time of each application notification message, generation unit 163 selects the application notification message with the longest playback time from among the application notification messages (step S104). After selecting the application notification message with the longest playback time from among the application notification messages, generation unit 163 estimates the total playback time of notifications including all of the selected application notification messages (step S105). Next, generation unit 163 determines whether the total playback time of the notification messages is equal to or less than the output grace time (step S106).

[0100] When determining that the total playback time of the notification messages is equal to or less than the output grace time (step S106; Yes), the generation unit 163 decides to output the selected application notification messages (step S107). When determining to output the selected application notification messages, the generation unit 163 generates notification messages that include all of the selected application notification messages.

[0101] On the other hand, if the generation unit 163 determines that the total playback time of the notification messages is not equal to or less than the output grace time (step S106; No), the generation unit 163 selects the next shortest application notification message in ascending order of application priority (step S108). After selecting the next shortest application notification message in ascending order of application priority, the generation unit 163 estimates the total playback time of the notification messages including all of the selected application notification messages (step S105).

[0102] [5. Modifications] [5-1. Adding information based on priority] In the above-described embodiment, the generation unit 163 generates a temporary notification message including multiple character strings with different content priorities, and if the playback time of the temporary notification message exceeds the output grace period, the generation unit 163 deletes the multiple character strings included in the temporary notification message in order, starting with the character string with the lowest content priority, to generate a notification message that can be played within the output grace period. However, the method of generating a notification message is not limited to this.

[0103] Specifically, the generation unit 163 generates a provisional notification message that includes, preferentially, a character string with a higher content priority among multiple character strings with different content priorities, and if the total playback time of the provisional notification message does not exceed the output grace period, adds a character string with a lower content priority than the character string included in the provisional notification message to generate a notification message that can be played within the output grace period.

[0104] For example, based on the intent information T1 shown in FIG. 5 acquired from application #1, the generation unit 163 generates two application notifications #11' and #11, "Turn right at the next intersection" (playback time: 2 seconds) and "Turn right at the next intersection" (playback time: 2.5 seconds), which contain only a character string with a content priority of "1." If the output grace period is 7 seconds, the generation unit 163 generates a notification that can be played within the output grace period by adding a character string with a content priority of "2," which has a lower content priority than the character string included in the provisional notification, since the total playback time of the provisional notification does not exceed the output grace period. For example, the generation unit 163 adds a character string with a content priority of "2," which has a lower content priority, to a character string with a content priority of "1," to generate a notification that can be played within the output grace period: "Turn right at the next intersection. Please stay in the innermost lane." (playback time: 6 seconds).

[0105] Furthermore, in the above-described embodiment, the generation unit 163 generates a provisional notification message including multiple app notification messages with different app priorities (or may be notification priorities or overall priorities), and if the total playback time of the provisional notification messages exceeds the output grace period, the length of the app notification messages included in the provisional notification message is shortened in order from the app notification message with the lowest app priority (or may be notification priority or overall priority) to generate notification messages that can be played within the output grace period. However, the method of generating notification messages is not limited to this.

[0106] Specifically, the generation unit 163 generates a provisional notification message that preferentially includes an app notification message with a higher app priority (or which may be a notification priority or an overall priority) from among multiple app notification messages with different app priorities (or which may be a notification priority or an overall priority), and if the total playback time of the provisional notification message does not exceed the output grace period, adds an app notification message with a lower app priority (or which may be a notification priority or an overall priority) than the app notification message included in the provisional notification message, thereby generating a notification message that can be played within the output grace period.

[0107] For example, the generation unit 163 generates a provisional notification message that preferentially includes application notification messages #1 to #3 with application priority levels of "1" to "3" from among application notification message #1 with application priority level of "1", application notification message #2 with application priority level of "2", application notification message #3 with application priority level of "3", application notification message #4 with application priority level of "4", etc. Next, if the total playback time of the provisional notification message that includes all of application notification messages #1 to #3 does not exceed the output grace period, the generation unit 163 adds application notification message #4 with application priority level of "4", which has a lower application priority level than the application notification messages included in the provisional notification message, to the provisional notification message, thereby generating a notification message that can be played within the output grace period.

[0108] [5-2. Deleting information based on priority] Furthermore, in the above-described embodiment, the generation unit 163 generates a provisional notification message including multiple app notification messages with different app priorities (or may be notification priorities or overall priorities), and if the total playback time of the provisional notification messages exceeds the output grace period, the length of the app notification messages included in the provisional notification message is shortened in order from the app notification message with the lowest app priority (or may be notification priority or overall priority) to generate notification messages that can be played within the output grace period. However, the method of generating notification messages is not limited to this.

[0109] Specifically, the generation unit 163 generates a provisional notification message including multiple app notification messages with different app priorities (or may be notification priorities or overall priorities), and if the total playback time of the provisional notification message exceeds the output grace period, deletes the app notification messages included in the provisional notification message in order, starting with the app notification message with the lower app priority (or may be notification priorities or overall priorities), and generates a notification message that can be played within the output grace period.

[0110] For example, the generation unit 163 generates a provisional notification message including an application notification message #1 with an application priority of "1," an application notification message #2 with an application priority of "2," an application notification message #3 with an application priority of "3," and an application notification message #4 with an application priority of "4." If the total playback time of the provisional notification messages exceeds the output grace period, the generation unit 163 subsequently deletes the application notification messages included in the provisional notification message in order, starting with application notification message #4 with the lowest application priority of "4," and generates a notification message that can be played within the output grace period.

[0111] [5-3. Intent information generation process] In the above-described embodiment, the intent generation unit 132 generates an application notification sentence and generates intent information based on the generated application notification sentence. However, the method of generating intent information is not limited to this. Specifically, the intent generation unit 132 may generate intent information based on situation information acquired from the information processing device 100. For example, the intent generation unit 132 generates intent information T1 to T3 shown in FIGS. 5 to 7 based on situation information including guide route information, position information, and the like acquired from the information processing device 100. Then, the transmission unit 133 transmits the intent information generated by the intent generation unit 132 to the information processing device 100.

[0112] [5-4. Priority information change processing] In the above-described embodiment, an example has been described in which the information processing device 100 generates a notification message based on content priority information, notification priority information, application priority information, or overall priority information acquired from the application device 10. However, this is not limiting. Specifically, when generating a notification message, the generation unit 163 of the information processing device 100 may change the content priority, notification priority, application priority, or overall priority based on past notification history, user attribute information, interest information, passenger information, and situation information. For example, the generation unit 163 may weight information (e.g., character strings) that is likely to be of interest to the user so that it has a higher priority based on the past notification history, user attribute information, interest information, passenger information, and situation information. Furthermore, for example, the generation unit 163 may weight notifications that have not received a response in the past based on a history of no response to notifications (no visit to a facility, no response within the response time), so that the notification priority of notifications that have not received a response in the past is lowered. The generation unit 163 generates a notification message based on the weighted content priority, notification priority, application priority, or overall priority.

[0113] [6. Effects] As described above, the information processing device 100 according to the embodiment includes an acquisition unit 161 and a generation unit 163. The acquisition unit 161 acquires, from an application, intent information including character string information indicating character strings constituting a notification sentence to be output by voice to the driver of a mobile object, intention information indicating a type of notification set for each character string, and an output deferral timing indicating a timing at which the notification of the notification sentence should be completed. The generation unit 163 generates a notification sentence to be output by voice to the driver based on the character string information and the intention information. The generation unit 163 generates a notification sentence including the character string, and modifies the expression of the notification sentence so that the timing at which the voice output of the generated notification sentence is completed is before the output deferral timing.

[0114] As a result, the information processing device 100 does not receive a fixed notification message from the application, but receives intent information, which is information in which information related to the notification message has been converted into metadata. This allows the information processing device 100 to generate a notification message that can convey all necessary information to the driver of the mobile object within a desired time period, depending on the situation information. Therefore, the information processing device 100 can appropriately convey necessary information to the driver of the mobile object.

[0115] The information processing device 100 further includes a calculation unit 162. The calculation unit 162 calculates an output grace period, which is the remaining time until the output grace period is reached. The acquisition unit 161 acquires situation information including travel information related to the travel status of the mobile object and driving information related to the driving status of the driver. The calculation unit 162 calculates the output grace period based on the situation information acquired by the acquisition unit 161. The generation unit 163 generates a notification message that can be played back within the output grace period.

[0116] This allows the information processing device 100 to appropriately convey necessary information to the driver of the moving object within the output grace period.

[0117] In addition, the calculation unit 162 calculates a safety level indicating the degree of safety of the traveling situation of the mobile body or the driving situation of the driver, and a margin indicating the degree of leeway the driver has to pay attention to things other than driving the mobile body, based on the situation information acquired by the acquisition unit 161, and calculates an output grace period based on the calculated safety level and margin.

[0118] This allows the information processing device 100 to appropriately convey necessary information to the driver of the mobile object within the output grace period that takes into consideration the traveling conditions of the mobile object or the safety of the driving conditions of the driver.

[0119] Furthermore, the higher the safety level, the longer the output grace period that the calculation unit 162 calculates.

[0120] As a result, when the safety level is high, the information processing device 100 can appropriately convey more necessary information to the driver of the moving object.

[0121] Furthermore, the calculation unit 162 calculates a longer output margin time as the margin is higher.

[0122] As a result, when the margin of safety is high, the information processing device 100 can appropriately convey more necessary information to the driver of the moving object.

[0123] Furthermore, the generation unit 163 generates a temporary notification message including a character string, and if the total playback time of the temporary notification message does not exceed the output grace time, changes the wording of the notification message to a more polite wording than the temporary notification message.

[0124] This allows the information processing device 100 to convey necessary information to the driver of the moving object in more polite terms within the output grace period.

[0125] Furthermore, the generation unit 163 generates a temporary notification message including a character string, and if the total playback time of the temporary notification message exceeds the output grace time, changes the expression of the notification message to a style that does not include an auxiliary verb at the end of the sentence.

[0126] This allows the information processing device 100 to convey necessary information to the driver of the moving object in simple terms within the output grace period.

[0127] Furthermore, the generating unit 163 generates a temporary notification message including a character string, and if the total playback time of the temporary notification message exceeds the output grace time, changes the expression of the notification message to a noun-ending sentence style.

[0128] This allows the information processing device 100 to convey necessary information to the driver of the moving object in simple terms within the output grace period.

[0129] Furthermore, the acquiring unit 161 acquires intent information that further includes content priority information indicating content priority set for each character string. The generating unit 163 generates a provisional notification message that includes a plurality of character strings with different content priorities, and if the total playback time of the provisional notification message exceeds the output grace time, deletes the character strings included in the provisional notification message in order, starting with the character string with the lowest content priority, to generate a notification message that can be played back within the output grace time.

[0130] As a result, the information processing device 100 can transmit necessary information to the driver of the moving object within the output grace period, preferentially in descending order of content priority.

[0131] Furthermore, the acquisition unit 161 acquires intent information that further includes notification priority information indicating a notification priority set for each piece of intent information. The generation unit 163 generates a plurality of different application notification sentences based on the plurality of different intent information acquired from the plurality of different applications, and determines, based on the notification priority information acquired by the acquisition unit 161, to output the application notification sentences from among the plurality of application notification sentences in order of the application notification sentences generated based on the intent information with the highest notification priority.

[0132] This allows the information processing device 100 to transmit necessary information to the driver of the moving object within the output grace period, preferentially in descending order of notification priority.

[0133] Furthermore, the generation unit 163 generates a provisional notification message including a plurality of application notification messages with different notification priorities, and if the total playback time of the provisional notification message exceeds the output grace time, the length of the application notification message is shortened in order from the application notification message with the lowest notification priority among the plurality of application notification messages included in the provisional notification message, thereby generating a notification message that can be played within the output grace time.

[0134] This allows the information processing device 100 to transmit necessary information to the driver of the moving object within the output grace period, preferentially in descending order of notification priority.

[0135] Furthermore, the acquisition unit 161 acquires intent information further including application priority information indicating application priority set for each application. The generation unit 163 generates a plurality of different application notification sentences based on the plurality of different intent information acquired from each of the plurality of different applications, and determines, based on the application priority information acquired by the acquisition unit 161, to output the application notification sentences of the applications having the highest application priority as audio, starting with the application notification sentences having the highest application priority.

[0136] As a result, the information processing device 100 can transmit necessary information to the driver of the moving object within the output grace period, preferentially in descending order of application priority.

[0137] Furthermore, the generation unit 163 generates a provisional notification message including a plurality of application notification messages with different application priorities, and if the total playback time of the provisional notification message exceeds the output grace time, the length of the application notification message is shortened in order from the application notification message with the lowest application priority among the plurality of application notification messages included in the provisional notification message, thereby generating a notification message that can be played within the output grace time.

[0138] As a result, the information processing device 100 can transmit necessary information to the driver of the moving object within the output grace period, preferentially in descending order of application priority.

[0139] The information processing device 100 further includes a determination unit 164. When a notification message regarding notification content requiring a response from the driver is to be output to the driver by voice, the determination unit 164 determines an acceptance time for accepting a response from the driver to the notification message, based on the situation information acquired by the acquisition unit 161. Specifically, the determination unit 164 determines that the acceptance time is short when the total playback time of the provisional notification message including the character string generated by the generation unit 163 is longer than the output grace time by less than a predetermined time. When the determination unit 164 determines that the acceptance time is short, the generation unit 163 generates a notification message including only the number of an option to be selected by the driver and the item of the option.

[0140] This allows the information processing device 100 to convey notification content requiring a response from the driver of the moving object within the output grace period.

[0141] Furthermore, if the total playback time of the provisional notification message is longer than the output grace time by a predetermined time or more, the determination unit 164 determines that there is no reception time. If the determination unit 164 determines that there is no reception time, the generation unit 163 generates a notification message that does not include notification content that requires a response from the driver.

[0142] This allows the information processing device 100 to convey notification content requiring a response from the driver of the moving object within the output grace period.

[0143] The information processing device 100 further includes an audio output unit 140. The audio output unit 140 outputs the notification text generated by the generation unit 163 to the driver by voice.

[0144] This allows the information processing device 100 to output necessary information to the driver of the moving object by voice.

[0145] Furthermore, the audio output unit 140 outputs the plurality of application notification messages in turn in the order determined by the generation unit 163.

[0146] This allows the information processing device 100 to output necessary information to the driver of the moving body by voice in order of priority.

[0147] [7. Hardware Configuration] Moreover, the application device 10 or the information processing device 100 according to the above-described embodiments is realized by, for example, a computer 1000 configured as shown in Fig. 9. Fig. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the application device 10 or the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0148] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0149] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0150] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via an input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600. Note that instead of the CPU 1100, an MPU (Micro Processing Unit) or, since a GPU requires a large amount of computing power, a GPU (Graphics Processing Unit) may also be used.

[0151] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0152] For example, when the computer 1000 functions as the application device 10 or the information processing device 100, the CPU 1100 of the computer 1000 executes a program loaded onto the RAM 1200 to realize the functions of the control unit 13 or the control unit 160. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0153] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have undergone various modifications and improvements based on the knowledge of those skilled in the art.

[0154] [8. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0155] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0156] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0157] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, an acquisition unit can be read as an acquisition means or an acquisition circuit. [Explanation of symbols]

[0158] 1. Information Processing Systems 10 Application Device 11 Communications Department 12 Storage section 13 Control Unit 131 Acquisition Department 132 Intent Generation Unit 133 Transmitter 100 Information processing device 110 Communications Department 120 Storage section 130 Sensor unit 140 Audio output section 150 Voice Recognition Unit 160 control section 161 Acquisition Department 162 Calculation Unit 163 Generation part 164 Decision Section 165 Transmitter

Claims

1. an acquisition unit that acquires, from an application, intent information including character string information indicating a character string constituting a notification sentence to be output as voice to a driver of a mobile body, application priority information indicating an application priority set for each of the applications, and an output deferral timing indicating a timing at which the notification of the notification sentence should be completed; a calculation unit that calculates an output grace time, which is the remaining time until the output grace time is reached; a generating unit that generates a notification sentence to be output as voice to the driver based on the character string information; Equipped with the generation unit generates a provisional notification message including a plurality of application notification messages having different application priorities, and when a total playback time of the provisional notification messages exceeds the output grace time, shortens lengths of the application notification messages among the plurality of application notification messages included in the provisional notification message in order from the application notification message having the lowest application priority, thereby generating notification messages that can be played within the output grace time.

1. An information processing device comprising:

2. The acquisition unit acquiring situation information including travel information relating to a travel situation of the moving body and driving information relating to a driving situation of the driver; The calculation unit calculating the output grace period based on the status information acquired by the acquisition unit; The generation unit generating the notification message that can be reproduced within the output grace period; 2. The information processing apparatus according to claim 1, wherein:

3. The calculation unit calculating a safety level indicating the degree of safety of the traveling situation of the mobile body or the driving situation of the driver and a margin indicating the degree of margin that the driver can pay attention to things other than driving the mobile body based on the situation information acquired by the acquisition unit, and calculating the output grace time based on the calculated safety level and margin; 3. The information processing apparatus according to claim 2, wherein:

4. The calculation unit The higher the safety level, the longer the calculated output grace time.

4. The information processing apparatus according to claim 3,

5. The calculation unit The higher the margin, the longer the calculated output grace time.

5. The information processing apparatus according to claim 3, wherein the information processing apparatus is a computer.

6. The generation unit generating a provisional notification message including the character string, and if the total playback time of the provisional notification message does not exceed the output grace time, changing the wording of the notification message to a more polite wording than the provisional notification message; 6. The information processing device according to claim 2, wherein:

7. The generation unit generating a provisional notification sentence including the character string, and if the total playback time of the provisional notification sentence exceeds the output grace time, changing the expression of the notification sentence to a style that does not include an auxiliary verb at the end of the sentence; 7. The information processing device according to claim 2, wherein:

8. The generation unit generating a provisional notification sentence including the character string, and if the total playback time of the provisional notification sentence exceeds the output grace time, changing the expression of the notification sentence to a noun-ending sentence style; 8. The information processing device according to claim 2, wherein:

9. The acquisition unit acquiring the intent information further including content priority information indicating a content priority set for each of the character strings; The generation unit generating a provisional notification message including a plurality of character strings with different content priorities, and if the total playback time of the provisional notification message exceeds the output grace time, deleting the character strings with the lowest content priorities from the plurality of character strings included in the provisional notification message in order, thereby generating a notification message that can be played back within the output grace time; 9. The information processing device according to claim 2, wherein the information processing device is a computer.

10. The acquisition unit acquiring intent information further including notification priority information indicating a notification priority set for each of the intent information; The generation unit generating a plurality of different application notification sentences based on a plurality of different pieces of intent information acquired from a plurality of different applications, and determining, based on the notification priority information acquired by the acquisition unit, to output the application notification sentences of the plurality of application notification sentences in order of the application notification sentences generated based on the intent information having the highest notification priority; 10. The information processing device according to claim 2, wherein:

11. The generation unit generating a provisional notification message including a plurality of application notification messages with different notification priorities, and if a total playback time of the provisional notification messages exceeds the output grace time, shortening the lengths of the application notification messages in order from the application notification message with the lowest notification priority among the plurality of application notification messages included in the provisional notification message, thereby generating notification messages that can be played within the output grace time; 11. The information processing apparatus according to claim 10,

12. The generation unit generating a plurality of different application notification messages based on a plurality of different pieces of intent information acquired from a plurality of different applications, and determining, based on the application priority information acquired by the acquisition unit, to output the application notification messages of the plurality of application notification messages in order of application priority, starting with the application notification message having the highest application priority; 12. The information processing device according to claim 2, wherein the information processing device is a computer.

13. a determination unit that determines, when a notification message regarding notification content requiring a response from the driver is to be output to the driver by voice based on the situation information acquired by the acquisition unit, a time period during which a response from the driver to the notification message is to be accepted; The determination unit determining that the reception time is short when a total playback time of the provisional notification message including the character string generated by the generation unit is longer than the output grace time by less than a predetermined time; The generation unit If the determination unit determines that the acceptance time is short, the notification message is generated including only the number of an option to be selected by the driver and an item of the option.

13. The information processing device according to claim 2, wherein:

14. The determination unit If the total playback time of the provisional notification message is longer than the output grace time by a predetermined time or more, it is determined that the reception time does not exist; The generation unit If the determination unit determines that the reception time does not exist, a notification message is generated that does not include notification content that requires a response from the driver. The information processing device according to claim 13.

15. The notification message generated by the generation unit is output to the driver by voice.

15. The information processing device according to claim 1, wherein:

16. The audio output unit outputting a plurality of application notification sentences in turn in voice in the order determined by the generation unit; 16. The information processing apparatus according to claim 15,

17. The acquisition unit acquiring the intent information further including intent information indicating a type of notification set for each of the character strings; The generation unit generating a notification to be output as voice to the driver based on the character string information and the intention information; 17. The information processing device according to claim 1, wherein:

18. An information processing method executed by an information processing device, an acquisition step of acquiring, from an application, intent information including character string information indicating a character string constituting a notification sentence to be output as voice to a driver of a mobile body, application priority information indicating an application priority set for each of the applications, and an output deferral timing indicating a timing at which the notification of the notification sentence should be completed; a calculation step of calculating an output grace time, which is a remaining time until the output grace time is reached; a generating step of generating a notification message to be output as voice to the driver based on the character string information; Including, The generating step generates a provisional notification message including a plurality of application notification messages having different application priorities, and if a total playback time of the provisional notification messages exceeds the output grace time, the lengths of the application notification messages are shortened in order from the application notification message having the lowest application priority among the plurality of application notification messages included in the provisional notification message, thereby generating notification messages that can be played within the output grace time.

1. An information processing method comprising:

19. an acquisition step of acquiring, from an application, intent information including character string information indicating a character string constituting a notification sentence to be output as voice to a driver of a mobile body, application priority information indicating an application priority set for each of the applications, and an output deferral timing indicating a timing at which the notification of the notification sentence should be completed; a calculation step of calculating an output grace time, which is a remaining time until the output grace time is reached; a generating step of generating a notification message to be output as voice to the driver based on the character string information; An information processing program for causing a computer to execute the above, The generating step generates a provisional notification message including a plurality of application notification messages having different application priorities, and if a total playback time of the provisional notification messages exceeds the output grace time, the lengths of the application notification messages are shortened in order from the application notification message having the lowest application priority among the plurality of application notification messages included in the provisional notification message, thereby generating notification messages that can be played within the output grace time. Information processing program.

20. an acquisition step of acquiring, from an application, intent information including character string information indicating a character string constituting a notification sentence to be output as voice to a driver of a mobile body, application priority information indicating an application priority set for each of the applications, and an output deferral timing indicating a timing at which the notification of the notification sentence should be completed; a calculation step of calculating an output grace time, which is a remaining time until the output grace time is reached; a generating step of generating a notification message to be output as voice to the driver based on the character string information; An information processing program for causing a computer to execute the above, The generating step generates a provisional notification message including a plurality of application notification messages having different application priorities, and if a total playback time of the provisional notification messages exceeds the output grace time, the lengths of the application notification messages are shortened in order from the application notification message having the lowest application priority among the plurality of application notification messages included in the provisional notification message, thereby generating notification messages that can be played within the output grace time. A storage medium storing an information processing program.

Images