Information processing device, information processing method, information processing program, and storage medium

Abstract

The present invention provides an information processing device, an information processing method, an information processing program, and a storage medium that can effectively assist the driver's auditory perception of the external conditions of a moving object. [Solution] The first information processing device 110 is characterized by comprising: an information acquisition unit 111 that acquires information from a storage unit 130 regarding sounds that the driver of the mobile vehicle 1 is likely to miss; a sound acquisition unit 113 that acquires sounds from outside the mobile vehicle 1; and a determination unit 114 that determines whether or not the external sounds include sounds that the driver is likely to miss, corresponding to the information acquired by the information acquisition unit 111.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, an information processing program, and a storage medium storing the information processing program for performing information processing on a moving object.

Background Art

[0002] Conventionally, as an example of information processing for a moving object, information processing for determining whether there is an object that is difficult for a driver to visually recognize outside the moving object has been proposed (see, for example, Patent Document 1). In the information processing described in this Patent Document 1, based on an image or the like outside the moving object, it is determined whether there is an object that is difficult to visually recognize. According to such information processing, by giving various notifications to the driver according to the determination result, it is possible to appropriately prompt the driver to pay attention.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, when the driver recognizes the situation outside the moving object, it is also frequently done to recognize the external situation aurally. However, regarding aural recognition, at present, it is left to the individual hearing ability and concentration of the driver, and at present, information processing that effectively assists the driver's aural recognition of the situation outside the moving object has not been done yet.

[0005] Therefore, an example of the problem of the present invention is to provide an information processing apparatus, an information processing method, an information processing program, and a storage medium that can effectively assist the driver's aural recognition of the situation outside the moving object.

Means for Solving the Problems

[0006] To solve the aforementioned problems and achieve the objective, the information processing device of the present invention is characterized by comprising: an information acquisition unit that acquires information on sounds that are likely to be missed by the driver of the mobile body from a storage unit that stores information on sounds that are likely to be missed by each individual driver of the mobile body, based on at least one of the behavior of the mobile body and the behavior of the driver of the mobile body; a sound acquisition unit that acquires sounds from outside the mobile body; a determination unit that determines whether or not the external sounds include sounds corresponding to the information; and a playback unit that plays the external sounds inside the mobile body in a playback mode according to the determination result of the determination unit.

[0007] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the information processing method of the present invention is characterized by comprising: an information acquisition step of acquiring information on sounds that are likely to be missed by the driver of the mobile body from a storage unit that stores information on sounds that are likely to be missed by each individual driver of the mobile body, based on at least one of the behavior of the mobile body and the behavior of the driver of the mobile body; a sound acquisition step of acquiring sounds from outside the mobile body; a determination step of determining whether or not the external sounds include sounds corresponding to the information; and a playback step of playing back the external sounds inside the mobile body in a playback mode corresponding to the determination result in the determination step.

[0008] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the information processing program of the present invention is characterized by causing a computer to execute the information processing method of the present invention described above.

[0009] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the storage medium of the present invention is characterized by storing the information processing program of the present invention described above. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic block diagram showing an information processing device according to the first embodiment, mounted on a mobile device. [Figure 2]Figure 1 is a schematic diagram showing the information stored in the memory unit. [Figure 3] Figure 1 is a schematic diagram showing the information table used by the determination unit for determination. [Figure 4] Figure 1 is a schematic diagram showing the information table used by the memory processing unit to determine operating information. [Figure 5] Figure 1 is a flowchart illustrating the processing flow in the information processing method executed by the first information processing device. [Figure 6] Figure 1 is a flowchart illustrating the processing flow in the information processing method executed by the second information processing device. [Figure 7] This is a schematic diagram showing the mobile unit and server in the second embodiment. [Figure 8] Figure 7 is a schematic block diagram showing the information processing devices in the mobile unit and the server, respectively. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below. An information processing device according to an embodiment of the present invention comprises an information acquisition unit, a sound acquisition unit, and a determination unit. The information acquisition unit acquires information from a storage unit regarding sounds that are likely to be missed by the driver of a moving vehicle. The sound acquisition unit acquires sounds from outside the moving vehicle. The determination unit then determines whether or not the external sounds include sounds corresponding to the information acquired from the storage unit.

[0012] According to the information processing device of this embodiment, it is determined whether or not sounds corresponding to the information acquired from the storage unit, that is, sounds that the driver of the moving vehicle is likely to miss, are included in the sounds outside the moving vehicle. Based on this determination result, it is possible to take appropriate actions such as prompting the driver to pay attention to sounds that are likely to be missed. In this way, the information processing device of this embodiment can effectively assist the driver's auditory perception of the external situation of the moving vehicle.

[0013] Here, the information processing apparatus of the present embodiment includes a playback unit that plays back external sounds in a playback mode according to the determination result of the determination unit inside the moving body.

[0014] According to the present embodiment, by playing back external sounds in a playback mode according to the determination result of the determination unit, it is possible to prompt the driver to pay attention in a mode that is easy to hear for sounds that are easy to miss.

[0015] Also, in the present embodiment, when the determination unit determines that the sound corresponding to the information acquired from the storage unit is included, the playback unit plays back the external sound in a playback mode that emphasizes the sound with respect to other sounds.

[0016] According to the present embodiment, it is possible to prompt the driver to pay attention by a playback mode with a small processing load that emphasizes sounds that are easy to miss.

[0017] Also, the information processing apparatus of the present embodiment includes a notification unit that notifies the driver of the presence of the sound when the determination unit determines that the sound corresponding to the information acquired from the storage unit is included.

[0018] According to the present embodiment, by the notification by the above notification unit, it is possible to directly prompt the driver to pay attention to sounds that are easy to miss.

[0019] Also, in the present embodiment, the storage unit stores information on sounds that are easy for the individual driver to miss.

[0020] According to the present embodiment, it is possible to finely determine for each individual driver based on the stored content of the storage unit regarding sounds that are easy to miss, which may vary depending on the person.

[0021] Also, in the present embodiment, the storage unit stores information on sounds that are easy for an unspecified number of people in the age group to which the driver belongs to miss.

[0022] According to this embodiment, sounds that are easily missed, which may differ depending on the age group, can be precisely determined for each age group of the driver based on the contents of the memory unit.

[0023] Furthermore, the information processing method according to an embodiment of the present invention comprises an information acquisition step, a sound acquisition step, and a determination step. The information acquisition step is a step of acquiring information from a storage unit about sounds that the driver of a moving vehicle is likely to miss hearing. The sound acquisition step is a step of acquiring sounds from outside the moving vehicle. The determination step is a step of determining whether or not the external sounds include sounds corresponding to the information acquired from the storage unit.

[0024] According to the information processing method of this embodiment, it is determined whether or not a sound corresponding to the information acquired from the memory unit, that is, a sound that the driver of the moving vehicle is likely to miss, is included in the external sounds of the moving vehicle. Based on this determination result, it is possible to take appropriate actions such as prompting the driver to pay attention to sounds that are likely to be missed. In this way, the information processing method of this embodiment can effectively assist the driver's auditory perception of the external situation of the moving vehicle.

[0025] Furthermore, the information processing program according to the embodiment of the present invention causes a computer to execute the information processing method described above.

[0026] According to the information processing program of this embodiment, by having a computer execute the above-described information processing method, it is possible to effectively assist the driver's auditory perception of the external conditions of a moving object.

[0027] Furthermore, the storage medium according to the embodiment of the present invention stores the above-described information processing program.

[0028] According to the storage medium of this embodiment, the stored information processing program can cause a computer to execute the information processing method described above. This effectively assists the driver's auditory perception of the external conditions of the moving object. [Examples]

[0029] The following describes specific examples of embodiments that effectively assist the driver's auditory perception of the external environment of a moving object, with reference to the figures. First, the first embodiment will be described.

[0030] Figure 1 is a schematic block diagram showing an information processing device according to the first embodiment, mounted on a mobile device.

[0031] The mobile device 1 shown in Figure 1 is a passenger car and comprises a first information processing device 110, a second information processing device 120, and a storage unit 130.

[0032] The first information processing device 110 is a device that, based on the contents stored in the memory unit 130, detects the presence of sounds outside the mobile vehicle 1 that the driver may easily miss and notifies the driver. The second information processing device 120 is a device that detects sounds that the driver of the mobile vehicle 1 may easily miss and stores information about such sounds in the memory unit 130. The information stored in the memory unit 130 by the second information processing device 120 is used for processing by the first information processing device 110.

[0033] In this embodiment, the components of the first information processing device 110 and the second information processing device 120 are, with some exceptions, built into a car navigation system mounted on the mobile device 1. Furthermore, the storage unit 130 utilizes a portion of the storage unit provided by the car navigation system.

[0034] Before describing the components of the first information processing device 110 and the second information processing device 120, we will first describe the information stored in the storage unit 130.

[0035] Figure 2 is a schematic diagram showing the information stored in the memory unit shown in Figure 1.

[0036] As shown in Figure 2, the memory unit 130 stores information about sounds that individual drivers are likely to miss. Specifically, it stores an information table 131 in which the personal names of people assumed to be drivers of the mobile vehicle 1 and information about sounds that each person is likely to miss are associated with each other.

[0037] In the example in Figure 2, driver "Mr. A" is likely to miss "children's voices" and "skateboarding sounds," and "children's voices" and "skateboarding sounds" are associated with the personal name "Mr. A" as names that include the sources of those sounds, as well as information about sounds that are easily missed. In this example, "skateboarding sounds" are sounds that are easily missed by a large number of people in the same age group as "Mr. A."

[0038] Because driver "Mr. B" is in a different age group than "Mr. A," the "sound of a skateboard" is not associated with him. Instead, the "sound of a bicycle" is associated with the individual "Mr. B" as a sound that "Mr. B" might easily miss.

[0039] Furthermore, while driver "C," who belongs to the same age group as "A," does not personally have any sounds that he is likely to miss, the sound of a skateboard is associated with the individual name "C" as a sound that is likely to be missed by a large number of people in C's age group.

[0040] Here, "children's voices," "bicycle sounds," and "skateboard sounds" are given as examples of sounds that are easily overlooked, and "skateboard sounds" are given as an example of a sound that is easily overlooked by an unspecified number of people in the driver's age group. However, these are merely examples, and the sounds that are easily overlooked, and the sounds that are easily overlooked by an unspecified number of people in the driver's age group, can be appropriately set according to the actual situation.

[0041] The first information processing device 110 shown in Figure 1 detects the presence of easily overlooked sounds outside the mobile vehicle 1 based on the contents of the memory unit 130 and notifies the driver. The first information processing device 110 includes an information acquisition unit 111, a personal identification unit 112, a sound acquisition unit 113, a determination unit 114, and a corresponding processing unit 115. The corresponding processing unit 115 also includes a playback unit 115a and a notification unit 115b.

[0042] The information acquisition unit 111 acquires information from the storage unit 130 regarding sounds that the driver of the mobile vehicle 1 is likely to miss. In this embodiment, as will be described later, sounds that an individual driver identified by the personal identification unit 112 is likely to miss are read from the information table 131 of the storage unit 130 and acquired.

[0043] The personal identification unit 112 identifies the individual driver of the mobile vehicle 1. Specific identification methods, though not specified here, include methods such as the driver entering their name via key input, identification through fingerprint authentication, or reading personal authentication information from the driver's smartphone or other device.

[0044] The sound acquisition unit 113 has microphones installed at various locations on the mobile body 1, and acquires external sounds from the mobile body 1 through these microphones.

[0045] The determination unit 114 determines whether the external sound acquired by the sound acquisition unit 113 contains a sound corresponding to the information acquired by the information acquisition unit 111, that is, a sound that the driver of the mobile vehicle 1 is likely to miss. The determination unit 114 stores the information table described below and uses this information table to make the determination.

[0046] Figure 3 is a schematic diagram showing the information table used for determination by the determination unit shown in Figure 1.

[0047] The information table 114a shown in Figure 3 is a table that associates the names of predetermined attention sounds that drivers should pay attention to with the extraction conditions for those attention sounds. In the example in Figure 3, the names of the attention sounds are listed as "children's voices," "bicycle sounds," and "skateboard sounds." The extraction conditions for each attention sound are defined by the frequency range and sound level range specific to that sound.

[0048] Furthermore, the sounds that require attention are not limited to the aforementioned "children's voices," "bicycle sounds," and "skateboard sounds," but can be set as appropriate. Also, the extraction conditions are not limited to frequency range and sound level range; any conditions that are sufficient for extracting sounds that require attention are not required.

[0049] The determination unit 114 shown in Figure 1 first performs frequency analysis on the external sound acquired by the sound acquisition unit 113, and based on the analysis results, extracts attention sounds that meet the extraction conditions defined in the information table 114a. Then, it determines whether any attention sounds have been extracted from the external sound, and whether the extracted attention sounds match the sounds acquired by the information acquisition unit 111 that are likely to be missed by the driver of the mobile vehicle 1. If the two match, the determination unit 114 determines that the external sound contains sounds that are likely to be missed by the driver.

[0050] The playback unit 115a of the response processing unit 115 has a speaker installed inside the mobile body 1 and plays back external sounds acquired by the sound acquisition unit 113 through that speaker. At this time, the playback unit 115a plays back the external sound in a playback mode according to the determination result of the determination unit 114. In this embodiment, if the determination unit 114 determines that there is a sound that the driver is likely to miss, the playback unit 115a plays back the external sound in a playback mode that emphasizes that sound compared to other sounds.

[0051] The playback modes implemented in the playback unit 115a, which emphasize sounds that are easily missed compared to other sounds, are not specified here, but examples include modes that increase the volume of sounds that are easily missed during playback, or modes that add echo to sounds that are easily missed during playback.

[0052] The notification unit 115b of the response processing unit 115 notifies the driver of the presence of a sound if the determination unit 114 determines that the sound contains a sound that the driver is likely to miss. The notification method used by the notification unit 115b is not specified here, but examples include notifying via voice message, displaying a message text or message image on a display screen, or lighting up an LED indicator.

[0053] Next, the second information processing device 120 shown in Figure 1 will be described. In this embodiment, the second information processing device 120 utilizes the personal identification unit 112 and the sound acquisition unit 113 provided in the first information processing device 110 as components of the second information processing device 120. In addition to the personal identification unit 112 and the sound acquisition unit 113, the second information processing device 120 also includes an operation information acquisition unit 121 and a storage processing unit 122.

[0054] The driving information acquisition unit 121 acquires driving information relating to both the behavior of the mobile body 1 and the behavior of the driver of the mobile body 1. Regarding the behavior of the mobile body 1, it acquires operational information of various driving devices such as the steering wheel and brakes, and regarding the driver's behavior, it acquires information about the driver's gaze using an eye tracker mounted on the mobile body 1.

[0055] Furthermore, the acquisition of driving information is not limited to the acquisition method described above; any method that acquires information regarding the behavior of the mobile body 1 or the driver is acceptable. Also, unlike this embodiment, it is possible to acquire driving information regarding only one of the behaviors of the mobile body 1 or the driver.

[0056] The memory processing unit 122 stores information about sounds that the driver is likely to miss in the memory unit 130, based on external sounds acquired by the sound acquisition unit 113 and driving information acquired by the driving information acquisition unit 121.

[0057] The memory processing unit 122 first attempts to extract the aforementioned warning sounds from the external sounds acquired by the sound acquisition unit 113. The memory processing unit 122 stores an information table similar to the information table 114a (Figure 3) stored by the determination unit 114 of the first information processing device 110, and attempts to extract warning sounds using this information table. If any warning sounds are extracted, the memory processing unit 122 determines whether or not driving information related to dangerous driving has been acquired by the driving information acquisition unit 121. Another information table, which will be explained below, is used to determine whether or not driving information related to dangerous driving has been acquired.

[0058] Figure 4 is a schematic diagram showing the information table used by the memory processing unit shown in Figure 1 to determine operating information.

[0059] The information table 122a shown in Figure 4 is a table that associates driving names, which are information related to various types of dangerous driving, with the extraction conditions for each type of dangerous driving. In the example in Figure 4, the driving names related to dangerous driving are "sudden braking," "distraction," and "equipment operation." The extraction condition for "sudden braking" is that brake operation information related to sudden braking was obtained from the brake system. The extraction condition for "distraction" is that the direction of the driver's gaze detected by the eye tracker is directed in a direction unrelated to the movement of the mobile body 1. The extraction condition for "equipment operation" is that equipment operation information was obtained for equipment mounted on the mobile body 1, such as a car navigation system or audio system.

[0060] Furthermore, dangerous driving is not limited to the aforementioned "sudden braking," "distraction," and "operation of equipment," but can be defined as appropriate. Also, the extraction conditions are not limited to those mentioned above; any condition that can serve as a criterion for identifying dangerous driving is acceptable, and the specific content of the conditions is not required.

[0061] The memory processing unit 122 shown in Figure 1, when it detects a sound of concern from external sounds acquired by the sound acquisition unit 113, determines in the driving information acquisition unit 121 whether or not driving information related to dangerous driving that matches the extraction conditions in the information table 122a in Figure 4 has been obtained. If driving information related to dangerous driving has been obtained, the information regarding the sound of concern extracted from external sounds is stored in the memory unit 130 as information about sounds that drivers are likely to miss. This storage is done by adding an item to the information table 131 shown in Figure 2. That is, if a sound that drivers are likely to miss is obtained for the first time for driver "Mr. A", a column for the personal name "Mr. A" is added to the information table 131, and the information obtained by the memory processing unit 122 is recorded in the column for sounds that drivers are likely to miss, which corresponds to the personal name "Mr. A". If a column for the personal name "Mr. A" already exists and new information about sounds that drivers are likely to miss is obtained, the newly obtained information is added to the column for sounds that drivers are likely to miss, which corresponds to the personal name "Mr. A".

[0062] Next, we will describe the information processing methods performed in the first information processing device 110 and the second information processing device 120, respectively, as described above. First, we will describe the information processing methods performed in the first information processing device 110.

[0063] Figure 5 is a flowchart showing the processing flow in the information processing method executed by the first information processing device shown in Figure 1.

[0064] The process shown in this flowchart starts when power is supplied to the navigation device of mobile unit 1 and is executed in parallel with the normal navigation process.

[0065] Once processing starts, the personal identification process is first performed by the personal identification unit 112 (step S111). In the personal identification process, as described above, the driver is identified by methods such as name key input or fingerprint impression.

[0066] Next, for the identified individual driver, the information acquisition unit 111 performs an information acquisition process to obtain information about easily missed sounds from the storage unit 130 (step S112). In the information acquisition process, names including the sources of easily missed sounds, which are associated with the individual name in the information table 131 shown in Figure 2, are read from the storage unit 130, for example, for "Mr. A," these names include "children's voices" and "skateboard sounds."

[0067] Next, the sound acquisition process is performed by the sound acquisition unit 113 (step S113). In the sound acquisition process, external sounds from the mobile body 1 are acquired through microphones located at various points on the mobile body 1, which are provided by the sound acquisition unit 113.

[0068] Then, the determination unit 114 determines whether the acquired external sound contains any of the attention sounds defined in the information table 114a shown in Figure 3 (step S114). The determination is made by an extraction process using the extraction conditions associated with each attention sound in the information table 114a. If an attention sound is successfully extracted, it is determined that the external sound contains that attention sound. If none of the attention sounds in the information table 114a are extracted, it is determined that no attention sounds are included.

[0069] If the external sound contains a warning sound (a Yes determination in step S114), the determination unit 114 determines whether that warning sound matches a sound that is likely to be missed by an individually identified driver, corresponding to the information acquired in the information acquisition process (step S115).

[0070] The two determination processes in steps S114 and S115 correspond to determination processes that determine whether or not the external sound contains sounds that are easily missed, corresponding to the information acquired in the information acquisition process.

[0071] If the extracted warning sound matches a sound that the driver is likely to miss (a Yes determination in step S115), the playback unit 115a of the response processing unit 115 plays the external sound in a playback mode that emphasizes the sound that is likely to be missed compared to other sounds (step S116). In addition, the notification unit 115b of the response processing unit 115 notifies the driver of the presence of the sound that is likely to be missed (step S117).

[0072] On the other hand, if the external sound does not contain any sounds that require attention (No determination in step S114), the process returns to the sound acquisition process in step S113, and the subsequent processes are repeated. Also, if the sounds that require attention included in the external sound do not match sounds that are easily missed (No determination in step S115), the playback unit 115a plays the external sound as is (step S118).

[0073] Furthermore, the processing when the external sound does not contain a warning sound is not limited to the process of returning to the sound acquisition process in step S113 and omitting playback, as shown in the flowchart of this embodiment in Figure 5. Unlike this embodiment, even if the external sound does not contain a warning sound (No determination in step S114), the process may proceed to step S118 and the playback unit 115a may play the external sound as is.

[0074] Once the notification in step S117 or playback in step S118 is complete, the process returns to the sound acquisition step in step S113, and the subsequent processes are repeated.

[0075] The information processing method shown in the flowchart of Figure 5 continues to run until the navigation device's power is turned off.

[0076] The information processing program that causes a computer to execute the information processing method shown in the flowchart of Figure 5 is stored in the storage medium of the navigation device or in the storage medium of an in-vehicle computer device separate from the navigation device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0077] Next, the information processing method executed in the second information processing device 120 will be described.

[0078] Figure 6 is a flowchart showing the processing flow in the information processing method executed by the second information processing device shown in Figure 1.

[0079] The process shown in this flowchart also starts when power is supplied to the navigation device of mobile unit 1 and is executed in parallel with the normal navigation process.

[0080] When processing starts, the personal identification process is performed in the personal identification unit 112 (step S121), similar to the process in the flowchart of Figure 5, and the sound acquisition process is performed in the sound acquisition unit 113 (step S122), similar to the process in the flowchart of Figure 5. In this embodiment, the process in the flowchart of Figure 6 is performed concurrently with the process in the flowchart of Figure 5. Therefore, in this embodiment, the personal identification process (step S111) and sound acquisition process (step S113) performed in the flowchart of Figure 5 also serve as the personal identification process (step S121) and sound acquisition process (step S122) in the flowchart of Figure 6.

[0081] In the flowchart of Figure 6, the driving information acquisition unit 121 then performs a driving information acquisition step (step S123) in which driving information is acquired regarding both the behavior of the mobile body 1 and the behavior of the driver of the mobile body 1. In the driving information acquisition step, information on the operation of various devices on the mobile body 1 is acquired regarding the behavior of the mobile body 1, and information on the driver's gaze is acquired by the eye tracker regarding the driver's behavior.

[0082] Next, the memory processing unit 122 determines whether or not the external sounds acquired in the sound acquisition process (step S122) contain any sounds that require attention (step S124). This determination is made by extracting sounds that require attention from the external sounds based on an information table similar to the information table 114a in Figure 3 used in step S114 of the flowchart in Figure 5.

[0083] If the external sound contains a warning sound (a Yes determination in step S124), the memory processing unit 122 determines whether or not the driving information acquisition unit 121 has acquired driving information related to dangerous driving (step S125). In this determination, the information table 122a shown in Figure 4 is referenced, and the memory processing unit 122 determines whether or not the information related to dangerous driving specified in this information table 122a has been acquired by the driving information acquisition unit 121.

[0084] If information regarding dangerous driving has been obtained (Yes determination in step S125), the memory processing unit 122 performs the following memory processing step (step 126). In the memory processing step, the name including the source of the warning sound extracted in the processing of step S124 is recorded in the information table 131 of the memory unit 130 as information about sounds that the identified driver is likely to miss. At this time, if the extracted name including the source of the warning sound is already recorded in the information table 131 in association with that driver, the entry is omitted. Also, if the personal name of the identified driver is not recorded in the information table 131, a new entry field for that personal name is created, and the name including the source of the warning sound is recorded in association with that personal name.

[0085] Once this memory processing step (step S126) is completed, the process returns to step S122 and the subsequent processing is repeated. Also, if the external sounds do not contain any warning sounds (No determination in step S124) or if no information regarding dangerous driving has been acquired (No determination in step S125), the process returns to step S122 and the subsequent processing is repeated.

[0086] The information processing method shown in the flowchart of Figure 6, along with the information processing method shown in the flowchart of Figure 5, continues to run until the power to the navigation device is turned off.

[0087] The information processing program that causes the computer to execute the information processing method shown in the flowchart of Figure 6 is stored in the storage medium of the navigation device or in the storage medium of a separate in-vehicle computer device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0088] The first embodiment described above provides the following benefits. First, according to the first information processing device 110 shown in Figure 1, the information processing method, information processing program, and storage medium shown in the flowchart of Figure 5, the following determination is made. That is, it is determined whether or not a sound corresponding to the information acquired from the storage unit 130, i.e., a sound that the driver of the mobile body 1 is likely to miss, is included in the sounds outside the mobile body 1. Based on this determination result, operations such as appropriately alerting the driver to sounds that are likely to be missed can be carried out. In this way, this embodiment can effectively assist the driver's auditory perception of the external situation of the mobile body 1.

[0089] In this embodiment, a playback unit 115a is provided that plays back external sounds in a playback manner corresponding to the above determination result. This allows external sounds that are easily missed to be played back inside the mobile body 1 in a manner that is easy for the driver to hear, thereby alerting the driver.

[0090] Furthermore, in this embodiment, the playback unit 115a plays external sounds in a playback mode that emphasizes sounds that the driver is likely to miss compared to other sounds. This allows the driver to be alerted by a playback mode that has a low processing load, by emphasizing sounds that are easily missed.

[0091] Furthermore, this embodiment includes a notification unit 115b that informs the driver of the presence of sounds that are easily overlooked by the driver. This notification allows the driver to be directly reminded of sounds that are easily missed.

[0092] Furthermore, in this embodiment, the memory unit 130 stores information about sounds that individual drivers are likely to miss. This allows for detailed determination of sounds that are easily missed, which may vary from person to person, based on the contents of the memory unit 130.

[0093] Furthermore, in this embodiment, the memory unit 130 stores information about sounds that are easily missed by an unspecified number of people in the driver's age group. This allows for detailed determination of sounds that are easily missed, which may differ depending on age group, based on the contents of the memory unit 130, for each driver's age group.

[0094] Furthermore, according to the second information processing device 120 shown in Figure 1, the information processing method, information processing program, and storage medium represented by the flowchart in Figure 6, information regarding sounds that the driver is likely to miss is stored in the storage unit 130 based on external sounds and driving information. This allows for operations such as appropriately alerting the driver to the presence of easily missed sounds outside the mobile vehicle 1 based on the information in the storage unit 130. In terms of storing information in the storage unit 130, this embodiment can effectively assist the driver's auditory perception of the external situation of the mobile vehicle 1.

[0095] In this embodiment, when a warning sound is extracted from external noise and driving information related to dangerous driving is acquired, the information regarding the extracted warning sound is stored as information about sounds that drivers are likely to miss. By linking warning sounds with dangerous driving in this way, it is possible to identify sounds that drivers are likely to miss with high accuracy and store information about such sounds.

[0096] Furthermore, in this embodiment, the dangerous driving described above is defined as sudden braking as a behavior of the moving object 1, or as behavior indicating that the driver is not paying attention to a warning sound. According to this, it is possible to identify warning sounds that the driver almost missed and noticed at the last moment, or warning sounds that were completely missed, with a high degree of accuracy, and to store information about such sounds.

[0097] Next, a second embodiment will be described.

[0098] In the first embodiment described above, the detection and memory of easily missed sounds were entirely performed by the onboard equipment of the mobile unit 1. In contrast, in the second embodiment described below, part of the processing is performed by the onboard equipment of the mobile unit, and another part is performed by the equipment of a server wirelessly connected to the mobile unit.

[0099] Figure 7 is a schematic diagram showing the mobile unit and server in the second embodiment, and Figure 8 is a schematic block diagram showing the information processing devices in the mobile unit and server shown in Figure 7, respectively.

[0100] In this embodiment, as shown in Figure 7, multiple mobile units 2, each a passenger car, are wirelessly connected to a single server 3. Information regarding sounds that drivers are likely to miss is stored in a storage unit 320 provided in the server 3. This storage unit 320 stores information regarding sounds that drivers are likely to miss in a table format equivalent to the information table 131 shown in Figure 2. However, in this embodiment, the storage unit 320 stores an information table in which the personal names of the drivers of each of the multiple mobile units 2 are associated with names that include the sources of sounds that each driver is likely to miss.

[0101] The process for determining sounds that are easily missed is performed by the mobile information processing device 210 mounted on each mobile unit 2, based on information acquired wirelessly from the storage unit 320. This determination is made using the information table 114a of sounds requiring attention, shown in Figure 3, which is commonly shared by the mobile information processing devices 210 of multiple mobile units 2.

[0102] Furthermore, the mobile information processing device 210 identifies the driver of the mobile vehicle 2, acquires external sounds, and acquires driving information. The identified driver's name, the acquired information on external sounds, and the driving information are sent to the server 3 via wireless communication.

[0103] On server 3, the server information processing device 310 determines sounds that are likely to be missed by the driver of each mobile unit 2 based on various information sent from each mobile unit 2, and stores information about those sounds in the storage unit 320. The process of determining sounds that are likely to be missed is performed using the dangerous driving information table 122a shown in Figure 4, which is provided by the server information processing device 310.

[0104] As shown in Figure 8, the mobile information processing device 210 includes a first information acquisition unit 211, a first personal identification unit 212, a first sound acquisition unit 213, a determination unit 214, a correspondence processing unit 215, a first operation information acquisition unit 216, and a first communication unit 217. The correspondence processing unit 215 includes a playback unit 215a and a notification unit 215b.

[0105] The first information acquisition unit 211 acquires information from the server 3 via the first communication unit 217 regarding sounds that the driver of the mobile vehicle 2 is likely to miss. The first personal identification unit 212 identifies the driver of the mobile vehicle 2 and sends the identified driver's name to the server 3 via the first communication unit 217. The first sound acquisition unit 213 acquires external sounds from the mobile vehicle 2 and sends information regarding the acquired external sounds to the server 3 via the first communication unit 217. The determination unit 214 uses the attention sound information table 114a to determine whether the external sounds acquired by the first sound acquisition unit 213 include sounds that the driver is likely to miss, corresponding to the information acquired by the first information acquisition unit 211. The playback unit 215a of the correspondence processing unit 215 plays the external sounds in a playback mode that emphasizes the sounds that are likely to miss compared to other sounds if the external sounds include sounds that are likely to miss. The notification unit 215b notifies the driver of the presence of the sounds that are likely to miss. The first driving information acquisition unit 216 acquires driving information relating to both the behavior of the mobile unit 2 and the behavior of the driver. The first communication unit 217 communicates wirelessly with the server 3. This wireless communication by the first communication unit 217 is performed by attaching an identifier, such as a vehicle number, to the various information being communicated to identify the mobile unit 2 on which the first communication unit 217 is installed.

[0106] Furthermore, the server information processing device 310 includes a second sound acquisition unit 311, a second personal identification unit 312, a second operation information acquisition unit 313, a storage processing unit 314, a second information acquisition unit 315, and a second communication unit 316.

[0107] The second sound acquisition unit 311 receives information about external sounds of the mobile body 2, acquired and sent by the first sound acquisition unit 213 of the mobile body 2, via the second communication unit 316. The second personal identification unit 312 receives the personal name of the driver, identified and sent by the first personal identification unit 212 of the mobile body 2, via the second communication unit 316. The second driving information acquisition unit 313 receives driving information acquired and sent by the first driving information acquisition unit 216 of the mobile body 2, via the second communication unit 316. Based on the external sounds received as information by the second sound acquisition unit 311 and the driving information received by the second driving information acquisition unit 313, the memory processing unit 314 retrieves information about sounds that the driver of the personal name received by the first personal identification unit 212 is likely to miss. The dangerous driving information table 122a is used in this processing by the memory processing unit 314. The memory processing unit 314 stores the retrieved information about easily missed sounds in the memory unit 320. The second information acquisition unit 315, in response to a request from the first information acquisition unit 211 in the mobile unit 2, acquires information from the storage unit 320 regarding easily missed sounds corresponding to the personal name of the driver of the mobile unit 2, and sends it to the mobile unit 2 via the second communication unit 316. The second communication unit 316 communicates wirelessly with each mobile unit 2. When the second communication unit 316 transmits information to each mobile unit 2, it also attaches an identifier to the information to identify the recipient mobile unit 2 before transmitting it.

[0108] Next, we will describe the processing of the information processing method executed by the mobile information processing device 210 and the server information processing device 310.

[0109] Here, the information processing method performed by the mobile information processing device 210 is substantially the same as the process shown in the flowchart of Figure 5, except that the elements that execute each step of the process are different. Similarly, the information processing method performed by the server information processing device 310 is substantially the same as the process shown in the flowchart of Figure 6, except that the elements that execute each step of the process are different. Therefore, the process of the information processing method performed by the mobile information processing device 210 will be explained below with reference to the flowchart of Figure 5, and the process of the information processing method performed by the server information processing device 310 will be explained with reference to the flowchart of Figure 6.

[0110] First, we will explain the information processing method performed by the mobile information processing device 210.

[0111] This process starts when power is turned on to the navigation device of the mobile unit 2. First, in step S111, the first personal identification unit 212 identifies the driver and sends the identified driver's name to the server 3 via the first communication unit 217. The server 3 receives the sent name via the second communication unit 316 to the second personal identification unit 312.

[0112] In step S112, the first information acquisition unit 211 requests the server 3 to acquire information from the storage unit 320 via the first communication unit 217. Upon receiving this request, the server 3's second information acquisition unit 315 acquires information from the storage unit 320 regarding sounds that drivers may easily miss, based on the personal names received by the second personal identification unit 312, and sends it to the requesting mobile device 2 via the second communication unit 316. The mobile device 2 receives the received information via the first information acquisition unit 211 through the first communication unit 217.

[0113] In step S113, the first sound acquisition unit 213 acquires external sounds from the mobile body 2 and sends information about those sounds to the server 3 via the first communication unit 217. The server 3 receives the sent information via the second communication unit 316 to the second sound acquisition unit 311.

[0114] In step S114, the determination unit 214 uses the information table 114a of warning sounds shown in Figure 3 to determine whether or not a warning sound is included in the acquired external sound. If a warning sound is included (a Yes determination in step S114), in step S115, the determination unit 214 determines whether or not that warning sound matches a sound that an identified driver is likely to miss.

[0115] If the warning sound matches a sound that is easily missed (a Yes judgment in step S115), in step S116, the playback unit 215a plays the external sound with the easily missed sound emphasized. Also, in step S117, the notification unit 215b notifies the driver of the presence of the easily missed sound. If the external sound does not include a warning sound (a No judgment in step S114) or if the warning sound does not match a sound that is easily missed (a No judgment in step S115), in step S118, the playback unit 215a plays the external sound as is. After the processing in step S117 or step S118, the process returns to step S113 and the subsequent processing is repeated.

[0116] In this embodiment, the process shown in the flowchart of Figure 5 continues to run until the navigation device of the mobile unit 2 is turned off.

[0117] In this embodiment, an information processing program that causes a computer to execute the information processing method shown in the flowchart of Figure 5 is stored in the storage medium of the navigation device of each mobile unit 2, or in the storage medium of an in-vehicle computer device separate from the navigation device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0118] Next, the processing of the information processing method executed by the server information processing device 310 will be explained with reference to the flowchart in Figure 6.

[0119] This process starts when power is turned on to server 3. First, in step S121, the second personal identification unit 312 waits for the driver's personal name to be sent from the first personal identification unit 212 of mobile body 2. When this personal name is sent, the second personal identification unit 312 receives it via the second communication unit 316. Once the second personal identification unit 312 receives the driver's personal name from mobile body 2, it is assumed that the acquisition of various information has started in the mobile body 2 indicated by the identifier attached to that personal name, and from this point onward, the processing corresponding to that mobile body 2 begins.

[0120] In step S122, the second sound acquisition unit 311 receives information about external sounds of the mobile body 2, sent from the first sound acquisition unit 213 of the mobile body 2, via the second communication unit 316. Also, in step S123, the second operation information acquisition unit 313 receives operation information, sent from the first operation information acquisition unit 216 of the mobile body 2, via the second communication unit 316.

[0121] In step S124, the memory processing unit 314 determines whether the sound information received by the second sound acquisition unit 311 includes a warning sound, using the warning sound information table 114a in Figure 3. If a warning sound is included (Yes determination in step S124), in step 125, the memory processing unit 314 determines whether the second driving information acquisition unit 313 has acquired driving information related to dangerous driving, using the dangerous driving information table 122a in Figure 4.

[0122] If dangerous driving information has been acquired (a Yes determination in step S125), in step S126, the memory processing unit 314 stores the name including the source of the extracted warning sound in the memory unit 130 as information about sounds that are easily missed, as it is associated with the driver's personal name.

[0123] Once the memory processing step S126 is completed, the process returns to step S122 and the subsequent processing is repeated. Also, if the sound information received by the second sound acquisition unit 311 does not include a warning sound (No determination in step S124), or if no information regarding dangerous driving is acquired (No determination in step S125), the process returns to step S122 and the subsequent processing is repeated.

[0124] In this embodiment, the information processing method shown in the flowchart of Figure 6 continues to run until the server is powered off.

[0125] In this embodiment, the information processing program that causes a computer to execute the information processing method shown in the flowchart of Figure 6 is stored in a storage medium that forms part of the storage unit 320 provided in the server 3, or in a storage medium separate from the storage unit 320. Furthermore, the storage medium that stores this information processing program is not limited to a storage medium installed in the server 3, but may be a known portable storage medium or a storage medium installed in another server connected to the server 3 via a network.

[0126] The mobile information processing device 210, server information processing device 310, information processing method, information processing program, and storage medium of the second embodiment described above can also be used to obtain the same effects as in the first embodiment. That is, in this embodiment as well, it is possible to determine whether sounds that the driver of the mobile body 2 is likely to miss are included in the external sounds and to provide appropriate warnings. In other words, it goes without saying that in this embodiment as well as in the first embodiment, it is possible to effectively assist the driver's auditory perception of the external situation of the mobile body 2.

[0127] Furthermore, in this embodiment as well, similar to the first embodiment, information regarding sounds that the driver is likely to miss is stored in the memory unit 320 based on external sounds and driving information of the mobile body 2. Based on the information in this memory unit 320, operations such as appropriately prompting attention can be performed in each mobile body 2. In other words, in this embodiment as well as in the first embodiment, it goes without saying that the storage of information in this memory unit 320 can effectively assist the driver's auditory perception of the external situation of the mobile body 2.

[0128] Furthermore, the present invention is not limited to the embodiments described above, and includes other configurations that can achieve the objectives of the present invention, as well as the following modifications.

[0129] For example, in the first and second embodiments described above, the mobile bodies 1 and 2 are exemplified as passenger cars. However, the mobile body is not limited to passenger cars; it may also be a motorcycle, a truck, a bus or other large vehicle, etc., and the specific type of mobile body is not limited.

[0130] Furthermore, in the first and second embodiments described above, the first information processing device 110, the second information processing device 120, and the mobile information processing device 210 are exemplified, each comprising components built into a navigation device mounted on a mobile body. However, the information processing device that performs processing on the mobile body is not limited to these, and may also comprise components built into a computer provided separately from the navigation device, etc. [Explanation of symbols]

[0131] 1,2 Mobile Unit 3 servers 110 First Information Processing Device 111 Information Acquisition Department 112 Personal identification section 113 Sound acquisition section 114,214 Judgment section 114a,122a Information Table 115,215 Corresponding processing unit 115a,215a Playback section 115b, 215b Notification section 120 Second Information Processing Device 121 Driving Information Acquisition Unit 122,314 Memory Processing Unit 130,320 storage section 131 Information Table 210 Mobile Information Processing Device 310 Server Information Processing Device 211 1st Information Acquisition Department 212 1st Individual Identification Department 213 1st note acquisition part 216 First Operation Information Acquisition Unit 217 1st Communications Department 311 2nd tone acquisition part 312 Second Individual Identification Department 313 Second Operation Information Acquisition Unit 315 2nd Information Acquisition Department 316 Second Communications Department

Claims

1. An information acquisition unit acquires information about sounds that the driver of the mobile vehicle is likely to miss from a storage unit that stores information about sounds that the driver of the mobile vehicle is likely to miss, based on at least one of the behavior of the mobile vehicle and the behavior of the driver of the mobile vehicle. A sound acquisition unit that acquires external sounds from the aforementioned moving object, A determination unit that determines whether or not the external sound contains a sound corresponding to the information, Inside the moving body, there is a playback unit that plays the external sound in a playback mode corresponding to the determination result of the determination unit, An information processing device characterized by having the following features.

2. The mobile body further comprises a personal identification unit that identifies the individual driver of the mobile body, The information processing apparatus according to claim 1, characterized in that the information acquisition unit acquires information from the storage unit regarding sounds that the driver of the mobile vehicle is likely to miss, based on the information of the personal identification unit.

3. The information processing apparatus according to claim 1 or 2, characterized in that, if the determination unit determines that the information contains a sound corresponding to the information, the playback unit plays the external sound in a playback manner that emphasizes the sound compared to other sounds.

4. The information processing device according to any one of claims 1 to 3, characterized in that, if the determination unit determines that a sound corresponding to the information is included, it is further provided with a notification unit that notifies the driver of the presence of the sound.

5. The information processing device according to any one of claims 1 to 4, characterized in that the memory unit stores information about sounds that are likely to be missed by an unspecified number of people in the age group to which the driver belongs.

6. Information acquisition step: From a storage unit that stores information on sounds that are likely to be missed by each individual driver of the mobile vehicle, based on at least one of the behavior of the mobile vehicle and the behavior of the driver of the mobile vehicle, information on sounds that are likely to be missed by the driver of the mobile vehicle is acquired. A sound acquisition step for acquiring external sounds from the moving object, A determination step of determining whether the external sound contains a sound corresponding to the information, A playback step in which, inside the moving body, the external sound is reproduced in a playback mode corresponding to the determination result in the determination step, An information processing method characterized by comprising:

7. An information processing program characterized by causing a computer to execute the information processing method described in claim 6.

8. A storage medium characterized by storing the information processing program described in claim 7.

Images