Information processing device, information processing method, information processing program, and storage medium

Abstract

The present invention provides an information processing device, an information processing method, an information processing program, and a storage medium that can effectively assist the driver's auditory perception of the external conditions of a moving object. [Solution] The warning processing device 110 is characterized by comprising: a data acquisition unit 111 that acquires data from a storage unit 220 in which information relating to sounds that the driver of the mobile body 1 is likely to miss is associated with a location or area; a sound acquisition unit 112 that acquires sounds from outside the mobile body 1; a location acquisition unit 113 that acquires the location of the mobile body 1; and a warning determination unit 114 that determines whether the location of the mobile body 1 is in the location or area in the above data and whether the sound corresponding to the information associated with that location or area is included in the external sounds.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, an information processing program, and a storage medium storing the information processing program for performing information processing on a moving object.

Background Art

[0002] Conventionally, as an example of information processing for a moving object, information processing for determining whether there is an object that is difficult for a driver to visually recognize outside the moving object has been proposed (see, for example, Patent Document 1). In the information processing described in this Patent Document 1, based on an image or the like outside the moving object, it is determined whether there is an object that is difficult to visually recognize. According to such information processing, by giving various notifications to the driver according to the determination result, the driver can be appropriately alerted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, when the driver recognizes the situation outside the moving object, it is also frequently done to recognize the situation outside auditorily. However, for auditory recognition, at present, it is left to the individual auditory ability and concentration of the driver, and at present, information processing that effectively assists the driver's auditory recognition of the situation outside the moving object has not been done yet.

[0005] Therefore, an example of the problem of the present invention is to provide an information processing apparatus, an information processing method, an information processing program, and a storage medium that can effectively assist the driver's auditory recognition of the situation outside the moving object.

Means for Solving the Problems

[0006] To solve the aforementioned problems and achieve the objective, the information processing apparatus of the present invention is characterized by comprising: an acquisition unit that acquires data from a storage unit in which information relating to sounds that the driver of the mobile body is likely to miss based on at least one of the behavior of the mobile body and the behavior of the driver of the mobile body is associated with a location or region; a sound acquisition unit that acquires sounds from outside the mobile body; a location acquisition unit that acquires the location of the mobile body; a determination unit that determines whether the location of the mobile body is in the location or region and whether the sound corresponding to the information is included in the external sound; and a playback unit that plays the external sound inside the mobile body in a playback mode according to the determination result of the determination unit.

[0007] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the information processing method of the present invention is characterized by comprising: an acquisition step of acquiring data from a storage unit in which information relating to sounds that the driver of the mobile body is likely to miss based on at least one of the behavior of the mobile body and the behavior of the driver of the mobile body is associated with a location or region; a sound acquisition step of acquiring sounds from outside the mobile body; a location acquisition step of acquiring the location of the mobile body; a determination step of determining whether the location of the mobile body exists in the location or region and whether the sound corresponding to the information is included in the external sound; and a playback step of playing back the external sound inside the mobile body in a playback mode corresponding to the determination result in the determination step.

[0008] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the information processing program of the present invention is characterized by causing a computer to execute the information processing method of the present invention described above.

[0009] Furthermore, in order to solve the aforementioned problems and achieve the objectives, the storage medium of the present invention is characterized by storing the information processing program of the present invention described above. [Brief explanation of the drawing]

[0010] [Figure 1]This is a schematic diagram showing the mobile unit and server in the first embodiment. [Figure 2] Figure 1 is a schematic block diagram showing the device configuration for both the mobile unit and the server. [Figure 3] Figure 2 is a schematic diagram showing the data structure of the data stored in the memory unit. [Figure 4] Figure 3 shows the data structure in table format. [Figure 5] Figure 2 is a schematic diagram showing the information table used by the warning determination unit for making its determination. [Figure 6] Figure 2 is a schematic diagram showing the information table used by the information acquisition and determination unit to determine driving information. [Figure 7] Figure 2 is a schematic diagram illustrating the statistical processing of individualized information performed in the data creation unit. [Figure 8] Figure 2 is a flowchart illustrating the processing flow of the information processing method performed by the warning device shown. [Figure 9] Figure 2 is a flowchart illustrating the processing flow of the information processing method executed by the information acquisition processing device shown. [Figure 10] Figure 2 is a flowchart illustrating the processing flow of the data creation method performed by the data creation device shown. [Figure 11] This is a schematic block diagram showing the device configuration for the mobile unit and the server in the second embodiment. [Figure 12] Figure 11 is a flowchart illustrating the processing flow of the information processing method executed by the information acquisition processing device shown. [Figure 13] Figure 11 is a flowchart illustrating the processing flow of the data creation method performed by the data creation device shown. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present invention will be described. The information processing apparatus according to an embodiment of the present invention includes an acquisition unit, a sound acquisition unit, a position acquisition unit, and a determination unit. The acquisition unit acquires, from a storage unit, data in which information on sounds that are likely to be missed by a driver of a moving body is associated with a position or a region. The sound acquisition unit acquires sounds outside the moving body. The position acquisition unit acquires the position of the moving body. The determination unit determines whether the position of the moving body exists in the position or region in the data acquired by the acquisition unit and whether a sound corresponding to the information in the data acquired by the acquisition unit is included in the external sounds.

[0012] According to the information processing apparatus of the present embodiment, it is determined whether a sound corresponding to the information in the data acquired from the storage unit, that is, a sound that is likely to be missed by the driver of the moving body, is included in the external sounds of the moving body. Based on this determination result, operations such as appropriately prompting the driver to pay attention to sounds that are likely to be missed can be performed. Thus, according to the information processing apparatus of the present embodiment, the driver's auditory recognition of the external situation of the moving body can be effectively assisted.

[0013] Here, the information processing apparatus of the present embodiment includes a reproduction unit that reproduces external sounds in a reproduction mode according to the determination result of the determination unit inside the moving body.

[0014] According to the present embodiment, by reproducing external sounds in a reproduction mode according to the determination result of the determination unit, it is possible to prompt the driver to pay attention to sounds that are likely to be missed in a manner that is easy to hear.

[0015] Also, in the present embodiment, when the determination unit determines that a sound corresponding to the information in the data acquired from the storage unit is included, the reproduction unit reproduces the external sounds in a reproduction mode in which the sound is emphasized with respect to other sounds.

[0016] According to the present embodiment, it is possible to prompt the driver to pay attention by a reproduction mode with a small processing load that emphasizes sounds that are likely to be missed.

[0017] In addition, when the determination unit of the information processing apparatus according to the present embodiment determines that the sound corresponding to the information in the data acquired from the storage unit is included, the information processing apparatus includes a notification unit that notifies the driver of the presence of the sound.

[0018] According to the present embodiment, by the notification by the above notification unit, it is possible to directly prompt the driver's attention regarding a sound that is easy to miss.

[0019] In addition, the information processing method according to the embodiment of the present invention includes an acquisition step, a sound acquisition step, a position acquisition step, and a determination step. The acquisition step is a step of acquiring, from the storage unit, data in which information regarding a sound that is easy for the driver of the moving body to miss is associated with a position or region. The sound acquisition step is a step of acquiring a sound outside the moving body. The position acquisition step is a step of acquiring the position of the moving body. The determination step is a step of determining whether the position of the moving body exists in the position or region in the data acquired in the acquisition step and whether the sound corresponding to the information in the data acquired in the acquisition step is included in the external sound.

[0020] According to the information processing method of the present embodiment, it is determined whether a sound corresponding to the information in the data acquired from the storage unit, that is, a sound that is easy for the driver of the moving body to miss, is included in the external sound of the moving body. Thereby, based on the determination result, operations such as appropriately prompting the driver's attention regarding a sound that is easy to miss can be performed. Thus, according to the information processing method of the present embodiment, the driver's auditory recognition of the external situation of the moving body can be effectively assisted.

[0021] In addition, the information processing program according to the embodiment of the present invention causes a computer to execute the above-described information processing method.

[0022] According to the information processing program according to the present embodiment, by causing a computer to execute the above-described information processing method, the driver's auditory recognition of the external situation of the moving body can be effectively assisted.

[0023] Furthermore, the storage medium according to the embodiment of the present invention stores the above-described information processing program.

[0024] According to the storage medium of this embodiment, the stored information processing program can cause a computer to execute the information processing method described above. This effectively assists the driver's auditory perception of the external conditions of the moving object. [Examples]

[0025] The following describes specific examples of embodiments that effectively assist the driver's auditory perception of the external environment of a moving object, with reference to the figures. First, the first embodiment will be described.

[0026] Figure 1 is a schematic diagram showing the mobile unit and server in the first embodiment, and Figure 2 is a schematic block diagram showing the device configuration of the mobile unit and server shown in Figure 1.

[0027] In this embodiment, as shown in Figure 1, multiple mobile units 1, each a passenger car, are wirelessly connected to a single server 2. The server 2 is equipped with a storage unit 220 that stores data representing locations and areas where sounds that drivers are likely to miss frequently occur. The server 2 is also equipped with a data creation device 210 that creates the data to be stored in the storage unit 220.

[0028] On the other hand, each mobile unit 1 is equipped with a warning processing device 110 and an information acquisition processing device 120. The warning processing device 110 acquires data stored in the storage unit 220 of the server 2 via wireless communication and, based on that data, warns the driver about sounds that are easy to miss. The information acquisition processing device 120 acquires information for creating data to be stored in the storage unit 220 of the server 2.

[0029] In this embodiment, the components of the warning processing device 110 and the information acquisition processing device 120 are built into a navigation device mounted on the mobile body 1, with the exception of some of them.

[0030] Before describing the components of the mobile device 1's attention-raising processing device 110, the information acquisition processing device 120, and the server 2's data creation device 210, we will first describe the data structure of the data stored in the storage unit 220.

[0031] Figure 3 is a schematic diagram showing the data structure of the data stored in the memory unit shown in Figure 2, and Figure 4 is a diagram showing the data structure shown in Figure 3 in table format.

[0032] The data structure 221 shown in Figures 3 and 4 is a data structure that includes statistical correspondence information 221b as follows: The statistical correspondence information 221b is information that associates a location or area on the road network 221a with information about sounds that are statistically likely to be missed by the drivers of an unspecified number of mobile bodies 1 when they travel through these locations or areas. The location on the road network 221a refers to the node 221c that constitutes the road network 221a, or the point 221e on the road link 221d that constitutes the road network 221a. The area on the road network 221a refers to the road link 221d that constitutes the road network 221a. The information about sounds that drivers are likely to miss refers to the source of that sound.

[0033] In the examples in Figures 3 and 4, five statistical correspondence information 221b are illustrated. In the first statistical correspondence information 221b-1, "children" and "bicycles" are associated with road link L1 as sources of easily missed sounds. In the second statistical correspondence information 221b-2, "bicycles" are associated with road link L2. In the third statistical correspondence information 221b-3, "bicycles" are associated with point P1 on road link L3. In the fourth statistical correspondence information 221b-4, "children" are associated with node N1. In the fifth statistical correspondence information 221b-5, "skateboards" are associated with node N2.

[0034] These statistical correspondences 221b refer to situations such as when schools, parks, or alleys where skateboarders, cyclists, or children are likely to be found are in close proximity to busy roads or large shopping malls where the sounds they make tend to be drowned out by noise. In such locations or areas, statistically, many drivers are more likely to miss the sounds of skateboards, bicycles, or children's voices.

[0035] In this embodiment, the data having the data structure 221 described above is stored in a storage unit 220 provided in the server 2. This storage unit 220 is a hard disk drive, which is a storage device provided in the server 2. However, the storage device for the above data is not limited to a hard disk drive; it may be any other storage device that can write to and read from, or a known portable storage medium. When the data is stored in a portable storage medium, the storage unit 220 becomes a reader / writer device that can install and remove the portable storage medium.

[0036] Then, the warning processing device 110 of each mobile unit 1 issues a warning to the driver based on the data stored in the memory unit 220.

[0037] Furthermore, in this embodiment, the data creation device 210 of the server 2 generates statistical correspondence information 221b, that is, creates data having a data structure 221 that includes the statistical correspondence information 221b.

[0038] Then, the information acquisition processing device 120 of each mobile unit 1 acquires various types of information that serve as the source for generating statistical correspondence information 221b in the data creation device 210 of the server 2.

[0039] As shown in Figure 2, the mobile device 1's warning processing device 110 includes a data acquisition unit 111, a sound acquisition unit 112, a position acquisition unit 113, a warning determination unit 114, a response processing unit 115, and a first communication unit 116. The response processing unit 115 also includes a playback unit 115a and a notification unit 115b.

[0040] The data acquisition unit 111 acquires data in the data structure 221 shown in Figures 3 and 4 from the storage unit 220 of the server 2 via the first communication unit 116. This data associates information about sounds that are easily missed by the driver of a moving vehicle with location or region.

[0041] The sound acquisition unit 112 has microphones installed at various locations on the mobile body 1, and acquires external sounds from the mobile body 1 through these microphones.

[0042] The position acquisition unit 113 acquires the position of the moving object 1 using a positioning system such as GNSS (Global Navigation Satellite System).

[0043] First, the warning determination unit 114 determines whether the position of the moving object 1, acquired by the position acquisition unit 113, is located in the position or region specified in the statistical correspondence information 221b of the data acquired by the data acquisition unit 111. Furthermore, the warning determination unit 114 determines whether the sound that the driver is likely to miss, which is associated with that position or region in the statistical correspondence information 221b, is included in the external sound acquired by the sound acquisition unit 112.

[0044] Here, the warning determination unit 114 stores the information table described below and uses this information table to determine which sounds the driver is likely to miss.

[0045] Figure 5 is a schematic diagram showing the information table used by the warning determination unit shown in Figure 2 for making determinations.

[0046] The information table 114a shown in Figure 5 is a table that associates the names of predefined attention sounds that drivers should pay attention to with the extraction conditions for those attention sounds. In the example in Figure 5, the names of the attention sounds are listed as "children's voices," "bicycle sounds," and "skateboard sounds." The extraction conditions for each attention sound are defined by the frequency range and sound level range specific to that sound.

[0047] Furthermore, the sounds that require attention are not limited to the aforementioned "children's voices," "bicycle sounds," and "skateboard sounds," but can be set as appropriate. Also, the extraction conditions are not limited to frequency range and sound level range; any conditions that are sufficient for extracting sounds that require attention are not required.

[0048] The warning determination unit 114, shown in Figure 2, first performs frequency analysis on external sounds acquired by the sound acquisition unit 112 when determining sounds that drivers are likely to miss. Based on the analysis results, it extracts warning sounds that meet the extraction conditions specified in the information table 114a. It then determines whether any warning sounds are extracted from the external sounds and whether the extracted warning sounds match the sounds that drivers are likely to miss in the statistical correspondence information 221b. If the two match, the warning determination unit 114 determines that the external sounds contain sounds that drivers are likely to miss.

[0049] The playback unit 115a of the correspondence processing unit 115 has a speaker installed inside the mobile body 1 and plays back external sounds acquired by the sound acquisition unit 112 through that speaker. At this time, the playback unit 115a plays back the external sounds in a playback manner corresponding to the judgment result of the attention alert determination unit 114. In this embodiment, if the position of the mobile body 1 is in the position or region of the statistical correspondence information 221b, and the attention alert determination unit 114 determines that the external sounds include sounds that are easy to miss in the statistical correspondence information 221b, the playback unit 115a performs the following playback. That is, in such a case, the playback unit 115a plays back the external sounds in a playback manner that emphasizes the sounds that are easy to miss, which have been extracted as attention-grabbing sounds by the attention alert determination unit 114, compared to other sounds.

[0050] The playback modes implemented in the playback unit 115a, which emphasize sounds that are easily missed compared to other sounds, are not specified here, but examples include modes that increase the volume of sounds that are easily missed during playback, or modes that add echo to sounds that are easily missed during playback.

[0051] The notification unit 115b of the response processing unit 115 notifies the driver of the presence of an easily overlooked sound if the attention determination unit 114 determines that the position of the moving object 1 is within the position or region specified in the statistical response information 221b and that an easily overlooked sound is included in the external sound. The notification method used by the notification unit 115b is not specified here, but examples include notifying via voice message, displaying a message text or message image on the display screen, or lighting an LED indicator.

[0052] The first communication unit 116 communicates wirelessly with the server 2. This wireless communication by the first communication unit 116 is performed by attaching an identifier, such as a vehicle number, to the various information being communicated, in order to identify the mobile unit 1 on which the first communication unit 116 is installed.

[0053] Next, the information acquisition processing device 120 shown in Figure 2 will be described. In this embodiment, the information acquisition processing device 120 utilizes the sound acquisition unit 112, position acquisition unit 113, and first communication unit 116 of the warning processing device 110 as components of the information acquisition processing device 120. In addition to the sound acquisition unit 112, position acquisition unit 113, and first communication unit 116, the information acquisition processing device 120 also includes an operation information acquisition unit 121, an information acquisition determination unit 122, a recognition unit 123, and an information output unit 124.

[0054] The driving information acquisition unit 121 acquires driving information relating to both the behavior of the mobile unit 1 and the behavior of the driver. Regarding the behavior of the mobile unit 1, it acquires operational information of various driving devices such as the steering wheel and brakes, and regarding the driver's behavior, it acquires information about the driver's gaze using an eye tracker mounted on the mobile unit 1.

[0055] Furthermore, the acquisition of driving information is not limited to the acquisition method described above; any method that acquires information regarding the behavior of the mobile body 1 or the driver is acceptable. Also, unlike this embodiment, it is possible to acquire driving information regarding only one of the behaviors of the mobile body 1 or the driver.

[0056] The information acquisition and determination unit 122 determines whether the external sounds acquired by the sound acquisition unit 112 and the driving information acquisition unit 121 contain sounds that the driver is likely to miss. First, the information acquisition and determination unit 122 attempts to extract the aforementioned attention-grabbing sounds from the external sounds acquired by the sound acquisition unit 112. The information acquisition and determination unit 122 stores an information table similar to the information table 114a (Figure 5) stored by the attention-grabbing determination unit 114 of the attention-grabbing processing device 110, and attempts to extract attention-grabbing sounds using this information table. If any attention-grabbing sounds are extracted, the information acquisition and determination unit 122 determines whether driving information related to dangerous driving has been acquired by the driving information acquisition unit 121. Another information table, which will be explained below, is used to determine this driving information related to dangerous driving.

[0057] Figure 6 is a schematic diagram showing the information table used by the information acquisition and determination unit shown in Figure 2 to determine driving information.

[0058] The information table 122a shown in Figure 6 is a table that associates driving names, which are information related to various types of dangerous driving, with the extraction conditions for each type of dangerous driving. In the example in Figure 4, the driving names related to dangerous driving are "sudden braking," "distraction," and "equipment operation." The extraction condition for "sudden braking" is that brake operation information related to sudden braking was obtained from the brake system. The extraction condition for "distraction" is that the direction of the driver's gaze detected by the eye tracker is directed in a direction unrelated to the movement of the mobile body 1. The extraction condition for "equipment operation" is that equipment operation information was obtained for equipment mounted on the mobile body 1, such as a navigation system or audio system.

[0059] Furthermore, dangerous driving is not limited to the aforementioned "sudden braking," "distraction," and "operation of equipment," but can be defined as appropriate. Also, the extraction conditions are not limited to those mentioned above; any condition that can serve as a criterion for identifying dangerous driving is acceptable, and the specific content of the conditions is not required.

[0060] The information acquisition and determination unit 122 shown in Figure 2, when it detects a warning sound from external sounds acquired by the sound acquisition unit 112, determines whether the driving information acquisition unit 121 has obtained driving information related to dangerous driving that matches the extraction conditions in the information table 122a in Figure 6. If a warning sound is extracted from the external sounds and driving information related to dangerous driving is obtained, the information acquisition and determination unit 122 determines that the warning sound is included in the external sounds as a sound that is easily overlooked.

[0061] The recognition unit 123 recognizes sources of sounds that drivers are likely to miss, based on external sounds and driving information. Specifically, the recognition unit 123 recognizes sources of sounds that drivers are likely to miss, such as "children" or "bicycles," which the information acquisition and determination unit 122 has determined to be included in the external sounds as sounds that drivers are likely to miss, based on external sounds and driving information.

[0062] When the information acquisition and determination unit 122 determines that the external sound contains a sound that the driver is likely to miss, the information output unit 124 generates the following individualized information. This individualized information associates the information regarding the sound that the driver is likely to miss with the position of the moving object 1 acquired by the position acquisition unit 113 when the information acquisition and determination unit 122 determines that the external sound contains a sound that the driver is likely to miss. In this embodiment, the source of the sound recognized by the recognition unit 123 is used as the information regarding the sound that the driver is likely to miss.

[0063] Furthermore, in the individualized information, the information associated with easily missed sounds is not limited to the location of the mobile body 1, but may also be an area that includes its location. In this case, an example of an area that includes the location of the mobile body 1 is the road link 221d to which the location of the mobile body 1 obtained by GNSS belongs in the road network 221a shown in Figure 3. Also, the information regarding easily missed sounds in the individualized information is not limited to the source as described above, but may also be recorded data representing the sound.

[0064] When the information output unit 124 generates individual correspondence information in this manner, it outputs that individual correspondence information to the server 2, which is an information collection server, via the first communication unit 116.

[0065] Server 2 receives individual correspondence information sent from each of the multiple mobile units 1 in the data creation device 210. The data creation device 210 comprises an information collection unit 211, a data creation unit 212, a provision unit 213, and a second communication unit 214.

[0066] The information gathering unit 211 collects the above-mentioned individual correspondence information from multiple mobile bodies 1, which associates information about sounds that the driver of each mobile body 1 is likely to miss with the location where the sound originated.

[0067] The data creation unit 212 performs statistical processing on the individual correspondence information collected by the information collection unit 211 to create data in the data structure 221 shown in Figures 3 and 4, in which information on sounds that drivers are statistically likely to miss is associated with location or area.

[0068] Figure 7 is a schematic diagram illustrating the statistical processing performed on individual correspondence information in the data creation unit shown in Figure 2.

[0069] Figure 7 shows an example in which the first to eleventh individual correspondence information 211a-1, 211a-2, ... 211a-11 is collected, and statistical processing generates the first statistical correspondence information 221b-1 and the third statistical correspondence information 221b-3, which are also shown in Figures 3 and 4.

[0070] The first individual correspondence information 211a-1 is information that associates "children" with location G1 on road link L4 as a source of sound that is easily missed, and the second individual correspondence information 211a-2 is information that associates "bicycles" with location G2 on road link L4. The two locations G1 and G2 on road link L4 are dispersed to each other by a predetermined amount or more.

[0071] The third individual correspondence information 211a-3 is information that associates "child" with position G3 on road link L1, the fourth individual correspondence information 211a-4 is information that associates "child" with position G4 on road link L1, the fifth individual correspondence information 211a-5 is information that associates "bicycle" with position G5 on road link L1, the sixth individual correspondence information 211a-6 is information that associates "bicycle" with position G6 on road link L1, the seventh individual correspondence information 211a-7 is information that associates "bicycle" with position G7 on road link L1, and the eighth individual correspondence information 211a-8 is information that associates "child" with position G8 on road link L1. The six positions G3 to G8 on road link L1 are dispersed to a predetermined degree or more away from each other.

[0072] The ninth individual correspondence information 211a-9 is information that associates "bicycle" with position G9 on road link L3, and the tenth individual correspondence information 211a-10 is information that associates "bicycle" with position G10 on road link L3. The eleventh individual correspondence information 211a-11 is also information that associates "bicycle" with position G11 on road link L3. The three positions G9 to G11 on road link L3 are concentrated within a predetermined range near point P3 on road link L3.

[0073] In this embodiment, in the statistical processing of the individual correspondence information 211a, first, the number of individual correspondence information 211a is tallied for each of the following: within a predetermined range near node 221c, on road link 221d, and within a predetermined range near point 221e on road link 221d. Then, statistical correspondence information 221b is generated by associating the source of easily missed sounds in each individual correspondence information 211a with the locations where three or more individual correspondence information 211a exist.

[0074] In the example shown in Figure 7, the terms "child" and "bicycle" are associated with road link L1, where six individual correspondence information items 211a exist, generating the first statistical correspondence information 221b-1, which is also shown in Figures 3 and 4. Additionally, the term "bicycle" is associated with point P3 on road link L3, where three individual correspondence information items 211a are concentrated in the vicinity, generating the third statistical correspondence information 221b-3.

[0075] Furthermore, the statistical processing is not limited to finding locations where there are more than a threshold number of individual corresponding information 211a, as described above. Any processing that finds locations where sounds that drivers are likely to miss occur statistically frequently is acceptable, regardless of the specific processing content. Also, even when employing processing to find locations where there are more than a threshold number of individual corresponding information 211a, the threshold is not limited to three as described above, and the threshold can be set arbitrarily.

[0076] The data creation unit 212 shown in Figure 2 generates statistical correspondence information 221b by performing the statistical processing described above whenever the number of individual correspondence information 211a collected by the information collection unit 211 reaches a predetermined number. The data creation unit 212 sequentially stores the generated statistical correspondence information 221b in the storage unit 220. By accumulating statistical correspondence information 221b in the storage unit 220 in this way, the data creation unit 212 creates the data for the data structure 221 shown in Figures 3 and 4.

[0077] Furthermore, the providing unit 213 shown in Figure 2 reads the data created by the data creation unit 212 from the storage unit 220 in response to a request from the data acquisition unit 111 in the mobile device 1's alert processing device 110, and provides it to the requesting mobile device 1 via the second communication unit 214.

[0078] In this embodiment, the provisioning unit 213 is shown as an example of providing data to the requesting mobile body 1 in response to a request from the data acquisition unit 111. However, the method of provision may also be such that the provisioning unit provides data to the mobile body at regular intervals, rather than responding to a request from the mobile body. In this case, the mobile body would store the provided data in its own storage unit and read it out as needed for use.

[0079] Furthermore, in this embodiment, we have illustrated a configuration in which the mobile device 1 receiving the data is the source of the individual correspondence information 211a described above, but the data recipient is not limited to such a mobile device. The data recipient can be any mobile device that has the function of receiving the above data wirelessly from the server 2 and uses this data for some kind of information processing.

[0080] The second communication unit 214 shown in Figure 2 performs wireless communication with each mobile unit 1. When the second communication unit 214 transmits information to each mobile unit 1, it attaches an identifier to the information to identify the recipient mobile unit 1 before transmitting it.

[0081] Next, we will explain the information processing methods performed by the warning processing device 110, the information acquisition processing device 120, and the data creation device 210, as shown in Figure 2. First, we will explain the information processing method performed by the warning processing device 110.

[0082] Figure 8 is a flowchart showing the processing flow of the information processing method performed by the warning processing device shown in Figure 2.

[0083] The information processing method shown in the flowchart of Figure 8 is a warning process that alerts the driver of the mobile vehicle 1 to sounds that are easily missed, and starts when power is turned on to the navigation device of the mobile vehicle 1. When the process starts, the data acquisition process is executed by the data acquisition unit 111 (step S111). In the data acquisition process (step S111), first, the data acquisition unit 111 requests the server 2 to transmit data from the storage unit 220 via the first communication unit 116. Upon receiving this request, the provision unit 213 of the server 2 reads the data from the storage unit 220 and sends the data to the requesting mobile vehicle 1 via the second communication unit 214. Then, the data acquisition unit 111 acquires the received data via the first communication unit 116.

[0084] Next, the sound acquisition process is performed by the sound acquisition unit 112 (step S112), and then the position acquisition process is performed by the position acquisition unit 113 (step S113).

[0085] Next, the alert determination unit 114 extracts warning sounds from the external sounds acquired in the sound acquisition step (step S112) as defined in the information table 114a shown in Figure 6 (step S114). If no warning sounds are extracted (No determination in step S114), the process returns and the sound acquisition step (step S112) and the position acquisition step (step S113) are repeated.

[0086] On the other hand, if a sound requiring attention is detected (a Yes determination in step S114), the alert determination unit 114 determines whether the position acquired in the position acquisition step (step S113) is located in a position or region in the statistical correspondence information 221b of the data acquired in the data acquisition step (step S111) (step S115). As described above, the positions or regions in the statistical correspondence information 221b are positions or regions where sounds that drivers are likely to miss occur statistically frequently. Therefore, in step S115, it is determined whether the position acquired in the position acquisition step (step S113), i.e., the position of the moving object 1, is located in such a position or region.

[0087] If the location of the moving object 1 is in a position or area where sounds that are easily missed statistically occur frequently (Yes determination in step S115), the warning determination unit 114 makes the following determination. That is, the warning determination unit 114 determines whether the attention-grabbing sound extracted in step S114 matches the sounds that drivers are likely to miss in the statistical correspondence information 221b (step S116).

[0088] If the attention-grabbing sound extracted from the external sound matches a sound that is easily missed (a Yes determination in step S116), the warning determination unit 114 determines that the external sound contains a sound that is easily missed. In this case, the playback unit 115a of the response processing unit 115 plays the external sound in a playback mode that emphasizes the sound that is easily missed compared to other sounds (step S117). In addition, the notification unit 115b of the response processing unit 115 notifies the driver of the presence of the sound that is easily missed (step S118).

[0089] On the other hand, if the mobile body 1 is not located in a position or region where sounds that are easily missed statistically occur frequently (No determination in step S115), the playback unit 115a plays the external sound as is (step S119). Also, if the attention-grabbing sound extracted from the external sound does not match the sound that is easily missed (No determination in step S116), the playback unit 115a plays the external sound as is in step S119.

[0090] Once the notification in step S118 or playback in step S119 is complete, the process returns to step S112, and the subsequent processes are repeated.

[0091] The warning process shown in the flowchart of Figure 8 will continue to run until the navigation system is powered off.

[0092] The information processing program that causes a computer to execute the information processing method for the warning process shown in the flowchart of Figure 8 is stored in the storage medium of the navigation device or in the storage medium of an in-vehicle computer device separate from the navigation device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0093] Next, the information processing method executed in the information acquisition processing device 120 will be described.

[0094] Figure 9 is a flowchart showing the processing flow of the information processing method executed by the information acquisition processing device shown in Figure 2.

[0095] The information processing method shown in the flowchart of Figure 9 is an information acquisition process that acquires information to create data to be stored in the storage unit 220 of server 2. This information acquisition process starts together with the warning process described above when power is turned on to the navigation device of mobile unit 1, and is executed in parallel with the warning process.

[0096] When processing starts, the sound acquisition process is performed by the sound acquisition unit 112 (step S121), and then the position acquisition process is performed by the position acquisition unit 113 (step S122). In this embodiment, the sound acquisition process (step S112) and position acquisition process (step S113) in the warning process shown in Figure 8 also serve as the sound acquisition process (step S121) and position acquisition process (step S122) in the information acquisition process.

[0097] In the flowchart of Figure 9, the next step is to acquire driving information by the driving information acquisition unit 121 (step S123), which acquires driving information regarding both the behavior of the mobile body 1 and the behavior of the driver. In the driving information acquisition step (step S123), information on the operation of various devices on the mobile body 1 is acquired regarding the behavior of the mobile body 1, and information on the driver's gaze is acquired by the eye tracker regarding the driver's behavior.

[0098] Next, the information acquisition and determination unit 122 determines whether or not the external sounds acquired in the sound acquisition process (step S121) contain any sounds that require attention (step S124). This determination is made by extracting sounds that require attention from the external sounds based on an information table similar to the information table 114a in Figure 5 used in step S115 of the flowchart in Figure 8.

[0099] If the external sounds include a warning sound (a Yes determination in step S124), the information acquisition determination unit 122 determines whether or not the driving information acquisition unit 121 has acquired driving information related to dangerous driving (step S125). In this determination, the information table 122a shown in Figure 6 is referenced, and the information acquisition determination unit 122 determines whether or not the information related to dangerous driving specified in this information table 122a has been acquired by the driving information acquisition unit 121.

[0100] If information regarding dangerous driving has been obtained (Yes determination in step S125), the following information output process is performed (step 126). In the information output process (step 126), first, the recognition unit 123 recognizes the source of the warning sound extracted in the processing of step 124 as a source of sound that the driver is likely to miss. Then, the information output unit 124 associates the position of the moving object 1 obtained in the position acquisition process (step S122) with that source and generates the individual correspondence information 211a described above. The information output unit 124 outputs the individual correspondence information 211a to the server 2 via the first communication unit 116.

[0101] Once this information output process (step S126) is completed, the process returns to step S121 and the subsequent processes are repeated. Furthermore, if the external sound does not contain any warning sounds (No determination in step S124) or if no information regarding dangerous driving has been acquired (No determination in step S125), the process returns to step S121 and the subsequent processes are repeated.

[0102] The information acquisition process shown in the flowchart of Figure 9, along with the warning process shown in the flowchart of Figure 8, continues to run until the navigation device's power is turned off.

[0103] The information processing program that causes a computer to execute the information acquisition process shown in the flowchart of Figure 9 is also stored in the storage medium of the navigation device or in the storage medium of an in-vehicle computer device separate from the navigation device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0104] Next, we will describe the information processing method performed in the data creation device 210 of server 2.

[0105] Figure 10 is a flowchart showing the processing flow of the data creation method performed by the data creation device shown in Figure 2.

[0106] The data creation process shown in the flowchart of Figure 10 involves collecting individual correspondence information 211a from multiple mobile units 1 and creating data to be stored in the storage unit 220. This process starts when power is turned on to the server 2.

[0107] When processing starts, the information collection process is executed by the information collection unit 211 (step S211). In the information collection process (step S211), the information collection unit 211 collects individual corresponding information 211a from multiple mobile bodies 1, and when the number of collected items reaches a predetermined number, the process moves on to the next step.

[0108] In the next step, the data creation process is performed by the data creation unit 212 (step S212). In the data creation process (step S212), statistical processing is performed on the collected individual correspondence information 211a, as explained with reference to Figure 7, to generate statistical correspondence information 221b, in which information about sounds that drivers are statistically likely to miss is associated with location or area. Furthermore, in the data creation process (step S212), the generated statistical correspondence information 221b is sequentially stored in the storage unit 220. As the statistical correspondence information 221b is accumulated in the storage unit 220 in this way, the data of the data structure 221 shown in Figures 3 and 4 is created in the data creation process (step S212).

[0109] Once the data creation process (step S212) is completed, the process returns to step S211 and the subsequent processes are repeated.

[0110] In this embodiment, the data creation process shown in the flowchart of Figure 10 continues to run until the server is powered off.

[0111] In this embodiment, a data creation program that causes a computer to execute the data creation method shown in the flowchart of Figure 10 is stored in a storage medium that forms part of the storage unit 220 provided in the server 2, or in a storage medium separate from the storage unit 220. Furthermore, the storage medium that stores this data creation program is not limited to a storage medium installed in the server 2, but may be a known portable storage medium or a storage medium installed in another server connected to the server 2 via a network.

[0112] The following effects can be obtained with the various information processing devices, various information processing methods, various information processing programs, various storage media, and the data structure 221 in Figures 3 and 4, as well as the storage media and storage devices that store this data structure 221, as described above in the first embodiment.

[0113] First, we will explain the effects of the warning processing device 110 shown in Figure 2, the information processing method and information processing program performed in the warning processing shown in Figure 8, and the various storage media.

[0114] According to the warning processing device 110 and information processing method of this embodiment, it is determined whether or not the easily missed sounds in the statistical correspondence information 221b of the data acquired from the storage unit 220 are included in the external sounds of the mobile body 1. Based on this determination result, operations such as appropriately alerting the driver to easily missed sounds can be carried out. In this way, this embodiment can effectively assist the driver's auditory perception of the external situation of the mobile body 1.

[0115] In this embodiment, the warning processing device 110 includes a playback unit 115a that plays back external sounds in a playback manner corresponding to the judgment result of the warning determination unit 114 inside the mobile body 1. With this, by playing back external sounds in a playback manner corresponding to the judgment result of the warning determination unit 114, it is possible to warn the driver of sounds that are easily missed in a manner that is easy for the driver to hear.

[0116] Furthermore, in this embodiment, if the warning determination unit 114 determines that an easily overlooked sound is included in the external sound, the playback unit 115a plays the external sound in a playback mode that emphasizes that sound compared to other sounds. This allows the driver to be alerted by a playback mode that emphasizes easily overlooked sounds, which has a low processing burden.

[0117] Furthermore, the warning processing device 110 of this embodiment includes a notification unit 115b that notifies the driver of the presence of a sound if the warning determination unit 114 determines that a sound corresponding to information in the data acquired from the storage unit 220 is included. This makes it possible to directly alert the driver to sounds that are easily overlooked.

[0118] Furthermore, according to the information processing program of this embodiment, by having the above-mentioned warning processing performed by a computer, the driver's auditory recognition of the external conditions of the mobile body 1 can be effectively assisted.

[0119] Furthermore, according to the storage medium of this embodiment, the stored information processing program can cause the computer to execute the above-mentioned warning process. This effectively assists the driver's auditory perception of the external conditions of the mobile body 1.

[0120] Next, we will explain the information acquisition processing device 120 shown in Figure 2, the information processing method, information processing program, and the effects of various storage media performed in the information acquisition process shown in Figure 9.

[0121] According to the information acquisition processing device 120 and information processing method of this embodiment, when a sound that is easily missed by the driver occurs, individual corresponding information 211a that associates the source of the sound that is easily missed by the driver with the position of the mobile body 1 is output to the server 2. This allows the server 2 to use the individual corresponding information 211a for operations such as creating data to appropriately alert the driver to sounds that are easily missed. In this way, according to this embodiment, the driver's auditory perception of the external situation of the mobile body 1 can be effectively assisted.

[0122] In this embodiment, the information acquisition processing device 120 includes a recognition unit 123 that recognizes the source of sounds that the driver is likely to miss based on external sounds and driving information. The source recognized by the recognition unit 123 is then used as information regarding sounds that the driver is likely to miss to generate the individual response information 211a described above. As a result, since the source of the sound is used as information regarding sounds that the driver is likely to miss, it is possible to more effectively assist the driver's auditory perception by informing the driver of the source of the sound that the driver is likely to miss and prompting them to pay close attention.

[0123] Furthermore, in this embodiment, the information acquisition and determination unit 122 determines that a predetermined attention sound, which is a sound that the driver should pay attention to, is included in the external sounds as a sound that the driver is likely to overlook, when the following conditions are met. Specifically, the information acquisition and determination unit 122 makes the above determination when a attention sound is extracted from the external sounds acquired by the sound acquisition unit 112, and driving information related to dangerous driving is acquired by the driving information acquisition unit 121. This allows for a highly accurate determination of sounds that the driver is likely to overlook by linking predetermined attention sounds with dangerous driving.

[0124] Furthermore, in this embodiment, dangerous driving behaviors include "sudden braking" as a movement of the moving object 1, and "looking away" or "operating equipment" as behaviors indicating that the driver is not paying attention to the warning sound. This allows for a high degree of accuracy in identifying warning sounds that the driver almost missed and noticed at the last moment, or warning sounds that were completely missed, as sounds that are easily overlooked by the driver and are included in the external noise.

[0125] Furthermore, according to the information processing program of this embodiment, by having a computer perform the information acquisition process described above, the driver's auditory perception of the external conditions of the mobile body 1 can be effectively assisted.

[0126] Furthermore, according to the storage medium of this embodiment, the stored information processing program allows the computer to execute the information acquisition process described above. This effectively assists the driver's auditory perception of the external conditions of the mobile body 1.

[0127] Next, we will explain the data creation device 210 shown in Figure 2, the data creation method performed in the data creation process shown in Figure 10, the data creation program, and the effects of various storage media.

[0128] According to the data creation device 210 and data creation method of this embodiment, statistical correspondence information 221b is generated in which the source of sounds that drivers are statistically likely to miss, i.e., sounds that a large number of drivers are likely to miss, is associated with a location or area. Then, by accumulating this statistical correspondence information 221b in the storage unit 220, the data of the data structure 221 shown in Figures 3 and 4 is created. Based on this data, operations such as appropriately alerting the driver when the moving object 1 is present in a location or area associated with a sound that is likely to be missed can be performed. Thus, according to this embodiment, it is possible to create data that effectively assists the driver's auditory perception of the external situation of the moving object 1.

[0129] In this embodiment, the data created by the data creation device 210 is data relating to the location or area of ​​the road network 221a where the source of a sound that the driver is likely to miss is located. Based on the created data, it is possible to obtain a location or area corresponding to the road network 221a on which the mobile body 1 is traveling, which serves as a location or area to draw attention to sounds that are easily missed. In other words, it is possible to create data that enables more effective assistance that matches the movement of the mobile body 1.

[0130] Furthermore, the data creation device 210 in this embodiment includes a providing unit 213 that provides the data created by the data creation unit 212 to a predetermined mobile body 1. As a result, the created data is provided to the mobile body 1, which can more effectively assist the driver of the mobile body 1.

[0131] Furthermore, according to the data creation program of this embodiment, by having a computer perform the data creation process described above, it is possible to effectively assist the driver's auditory perception of the external conditions of the mobile body 1.

[0132] Furthermore, with the storage medium of this embodiment, the data creation program stored in it can cause the computer to execute the data creation process described above. This effectively assists the driver's auditory perception of the external conditions of the mobile body 1.

[0133] Next, we will explain the effects of the data structure 221 shown in Figures 3 and 4, the storage medium that stores the data in this data structure 221, and the storage device.

[0134] According to the data structure 221 of this embodiment, by comparing the data of this data structure 221 with the statistical correspondence information 221b described above and the position of a specific mobile object 1 and external sounds, the following operations can be performed. That is, the statistical correspondence information 221b can be used to appropriately alert the driver to the location or area corresponding to the source of a sound that is easily overlooked when the specific mobile object 1 is traveling. In this way, the data structure 221 of this embodiment can effectively assist the driver's auditory perception of the external situation of the mobile object 1.

[0135] In this embodiment, the location or region in the statistical correspondence information 221b corresponds to the road network 221a. Based on the configured data, a location or region corresponding to the road network 221a on which the mobile body 1 travels can be obtained as a location or region to draw attention to sounds that are easily missed. In other words, more effective assistance can be provided that matches the movement of the mobile body 1.

[0136] Furthermore, in this embodiment, the location or region in the statistical correspondence information 221b corresponds to the road link 221d or node 221c that constitute a road in the road network 221a. Based on the configured data, a detailed location or region in the road network 221a can be obtained as a location or region to draw attention to sounds that are easily missed. In other words, detailed assistance can be provided that matches the movement of the mobile body 1.

[0137] Furthermore, according to the storage medium and storage device of this embodiment, the data stored in each can be used for operation in the attention-raising processing device 110 in the specific mobile body 1 described above. This effectively assists the driver's auditory perception of the external conditions of the mobile body 1.

[0138] This concludes the explanation of the first embodiment. Next, we will describe the second embodiment.

[0139] The second embodiment differs from the first embodiment described above in that the individualized information 211a is generated by the server's data creation device rather than the mobile information acquisition processing device. Below, the second embodiment will be explained, focusing on the differences from the first embodiment. On the other hand, the mobile body's warning device, the warning processing performed by the warning device, and the data structure of the warning data are the same as in the first embodiment, so their explanation will be omitted.

[0140] Figure 11 is a schematic block diagram showing the device configurations for the mobile unit and server in the second embodiment. In Figure 11, components equivalent to those in the first embodiment shown in Figure 2 are indicated with the same reference numerals as in Figure 2, and therefore, redundant explanations of these equivalent components will be omitted below.

[0141] In the second embodiment, the information acquisition processing device 320, which is provided on the mobile body 3 together with the warning processing device 110, does not have a component equivalent to the information acquisition determination unit 122 provided in the information acquisition processing device 120 in the first embodiment. In this embodiment, the information output unit 324 of the information acquisition processing device 320 outputs external sound information related to sounds outside the mobile body 3, driving information related to both the behavior of the mobile body 3 and the behavior of the driver, and position information related to the position of the mobile body 3 to the server 4. This output is repeated at regular intervals.

[0142] Furthermore, regarding the driving information output by the information output unit 324, in this embodiment as well, it may be information relating to at least one of the behaviors of the mobile body 3 and the driver, or it may be information relating to only one of the behaviors of the mobile body 3 and the driver.

[0143] In the data creation device 410 of server 4, the information collection unit 411 collects external sound information, driving information, and location information sent from each mobile unit 3.

[0144] In this embodiment, the data creation device 410 of the server 4 is provided with an information generation unit 412 that generates the individual correspondence information 211a shown in Figure 7. In the first embodiment, the information generation unit 412 includes an information acquisition determination unit 412a and a recognition unit 412b, which are equivalent to the information acquisition determination unit 122 and recognition unit 123 that were provided in the information acquisition processing device 120 of the mobile body 1. Furthermore, the information generation unit 412 includes a generation unit 412c that generates the individual correspondence information 211a based on the determination result of the information acquisition determination unit 412a and the recognition content of the recognition unit 412b.

[0145] In this embodiment, the information acquisition and determination unit 412a first determines whether the external sound information collected by the information collection unit 411 includes a sound requiring attention. For this determination, an information table equivalent to the information table 114a of sounds requiring attention shown in Figure 5 is used.

[0146] If the external sound contains a warning sound, the information acquisition determination unit 412a determines whether the information collection unit 411 has collected driving information related to dangerous driving, which is defined in an information table equivalent to the information table 122a shown in Figure 6.

[0147] If an external sound contains a warning sound and driving information related to dangerous driving has been collected, the information acquisition and determination unit 412a determines that the external sound contains a warning sound that is likely to be overlooked by the driver. The recognition unit 412b then recognizes the source of the warning sound as a source of sound that is likely to be overlooked by the driver.

[0148] The generation unit 412c adopts the source recognized by the recognition unit 412b as information about sounds that the driver is likely to miss, and associates the location represented by the location information collected together with the external sound information by the information collection unit 411 with this source to generate individual correspondence information 211a.

[0149] In this embodiment, whenever external sound information, location information, and driving information are sent from each of the multiple mobile bodies 3 at regular intervals, the information generation unit 412 repeatedly generates individual corresponding information 211a for each mobile body 3.

[0150] Then, the data creation unit 212, which is equivalent to that of the first embodiment, performs the statistical processing described above to generate statistical correspondence information 221b whenever the number of individual correspondence information 211a generated by the information generation unit 412 reaches a predetermined number. This statistical correspondence information 221b is stored in the storage unit 220, thereby creating the data for the data structure 221 shown in Figures 3 and 4.

[0151] Next, the information processing method executed in the information acquisition processing device 320 will be described.

[0152] Figure 12 is a flowchart showing the processing flow of the information processing method executed by the information acquisition processing device shown in Figure 11.

[0153] The information processing method shown in the flowchart of Figure 11 is an information acquisition process that acquires information to create data to be stored in the storage unit 220 of the server 4. This information acquisition process starts when power is turned on to the navigation device of the mobile unit 3.

[0154] When processing starts, the sound acquisition process is performed by the sound acquisition unit 112 (step S311), the position acquisition process is performed by the position acquisition unit 113 (step S312), and then the operation information acquisition process is performed by the operation information acquisition unit 121 (step S313). These processes in this embodiment are the sound acquisition process (step S121), the position acquisition process (step S122), and the operation information acquisition process (step S123) of the first embodiment shown in Figure 9. This is an equivalent process.

[0155] In this embodiment, following these steps, an information output step is performed by the information output unit 324 (step S314). In the information output step (step S314), external sound information related to external sounds acquired in the sound acquisition step (step S311) and position information related to positions acquired in the position acquisition step (step S312) are output to the server 2. Furthermore, in the information output step (step S314), the operation information acquired in the operation information acquisition step (step S313) is also output to the server 2 together with the external sound information and position information mentioned above.

[0156] After the information output process (step S314) is completed, the process returns to step S311, and the subsequent processes are repeated.

[0157] The information acquisition process shown in the flowchart in Figure 12 continues to run until the navigation device is powered off.

[0158] The information processing program that causes a computer to execute the information acquisition process represented by the flowchart in Figure 12 is stored in the storage medium of the navigation device or in the storage medium of an in-vehicle computer device separate from the navigation device. Furthermore, the storage medium that stores this information processing program is not limited to the storage medium of these devices, but may also be a known portable storage medium or a storage medium installed on a server connected to these devices via a network.

[0159] Next, we will describe the data creation method performed in the data creation device 410 of server 4.

[0160] Figure 13 is a flowchart showing the processing flow of the data creation method performed by the data creation device shown in Figure 11.

[0161] The data creation process shown in the flowchart of Figure 13 involves collecting external sound information, location information, and driving information from multiple mobile units 3, and creating data to be stored in the storage unit 220. This process starts when power is turned on to the server 4.

[0162] When processing starts, the information collection process is executed by the information collection unit 411 (step S411). In the information collection process (step S411), the information collection unit 411 collects external sound information, location information, and driving information from each mobile body 1.

[0163] Next, the information acquisition and determination unit 412a determines whether or not the external sounds represented by the external sound information collected in the information acquisition process (step S411) include any sounds that require attention (step S412). For this determination, an information table similar to the information table 114a for sounds requiring attention shown in Figure 5 is used.

[0164] If the external sounds include a warning sound (a Yes determination in step S412), the information acquisition determination unit 412a determines whether or not driving information related to dangerous driving has been acquired in the information acquisition process (step S411) (step S413). For this determination, an information table similar to the dangerous driving information table 122a shown in Figure 6 is referenced.

[0165] If driving information related to dangerous driving has been acquired (Yes determination in step S413), the following information generation process is executed (step 414). In the information generation process (step 414), the recognition unit 412b recognizes the source of the warning sound extracted in the processing of step 412 as a source of sound that the driver is likely to miss. Then, the generation unit 412c associates the position of the moving object 1 represented by the position information collected in the information collection process (step S411) with that source and generates individual correspondence information 211a.

[0166] In the next step, the data creation process is performed by the data creation unit 212 (step S415). In the data creation process (step S415), statistical processing, as explained with reference to Figure 7, is performed on the generated individual correspondence information 211a to generate statistical correspondence information 221b, in which information about sounds that drivers are statistically likely to miss is associated with location or area. Furthermore, in the data creation process (step S415), the generated statistical correspondence information 221b is sequentially stored in the storage unit 220. In this way, the data of the data structure 221 shown in Figures 3 and 4 is created in the data creation process (step S415).

[0167] Once the data creation process (step S415) is completed, the process returns to step S411 and the subsequent processes are repeated.

[0168] In this embodiment, the data creation process shown in the flowchart of Figure 13 continues to run until the power to server 4 is turned off.

[0169] In this embodiment, the data creation program that causes a computer to execute the data creation process shown in the flowchart of Figure 13 is stored in a storage medium that forms part of the storage unit 220 provided in the server 4, or in a storage medium separate from the storage unit 220. Furthermore, the storage medium that stores this data creation program is not limited to a storage medium installed in the server 4, but may be a known portable storage medium or a storage medium installed in another server connected to the server 4 via a network.

[0170] In the second embodiment described above, we will explain the differences from the first embodiment, specifically the information acquisition processing device 320 shown in Figure 11, the information processing method for the information acquisition process shown in Figure 12, the information processing program, and the effects of various storage media.

[0171] According to the information acquisition processing device 320 and the information processing method of the information acquisition process in this example, external sound information regarding sounds outside the mobile body 3, driving information regarding both the behavior of the mobile body 3 and the behavior of the driver, and position information regarding the position of the mobile body are output to the server 4. This allows the server 4 to create data to appropriately alert the driver to sounds that are easily missed, and to use the above-mentioned information for such operations. In this way, according to this embodiment, the driver's auditory perception of the external situation of the mobile body 3 can be effectively assisted.

[0172] Furthermore, according to the information processing program of this embodiment, by having a computer execute the information processing method for the information acquisition process described above, it is possible to effectively assist the driver's auditory perception of the external conditions of the mobile body 3.

[0173] Furthermore, according to the storage medium of this embodiment, the stored information processing program can be used to execute the above-described information processing method on a computer. This effectively assists the driver's auditory perception of the external conditions of the mobile body 3.

[0174] Next, we will explain the differences between the second embodiment and the first embodiment, namely the data creation device 410 shown in Figure 11, the data creation method for the data creation process shown in Figure 13, the data creation program, and the effects of various storage media.

[0175] According to the data creation device 410 and data creation method of this embodiment, for each driver of a mobile body 3, individual correspondence information 211a is generated, which associates information about sounds that are likely to be missed with the location or area where the sound occurred. Then, by statistical processing of the individual correspondence information 211a, data is created that has statistical correspondence information 221b, which associates information about sounds that are statistically likely to be missed by drivers, i.e., sounds that are likely to be missed by an unspecified number of drivers, with location or area. As a result, based on this data, operations such as appropriately alerting the driver when the mobile body 3 is in a location or area associated with a sound that is likely to be missed can be carried out. In this way, according to this embodiment, data can be created to effectively assist the driver's auditory perception of the external situation of the mobile body 3.

[0176] Furthermore, according to the data creation program of this embodiment, by having a computer perform the data creation process described above, it is possible to create data that effectively assists the driver's auditory perception of the external conditions of the mobile body 3.

[0177] Furthermore, according to the storage medium of this embodiment, the stored data creation program can cause the above-described data creation process to be executed by a computer. This makes it possible to create data that effectively assists the driver's auditory perception of the external conditions of the mobile body 3.

[0178] Furthermore, the present invention is not limited to the embodiments described above, and includes other configurations that can achieve the objectives of the present invention, as well as the following modifications.

[0179] For example, in the first and second embodiments described above, the mobile bodies 1 and 3 are exemplified as passenger cars. However, the mobile body is not limited to passenger cars; it may also be a motorcycle, a truck, a bus or other large vehicle, etc., and the specific type of mobile body is not limited.

[0180] Furthermore, in the first and second embodiments described above, examples of information processing devices for mobile bodies include a warning processing device 110 and an information acquisition processing device 120, 320, which are equipped with components built into a navigation device mounted on the mobile body. However, the information processing device for mobile bodies is not limited to these, and may also include a control unit built into a computer provided separately from the navigation device. [Explanation of Symbols]

[0181] 1,3 Mobile Unit 2,4 Servers 110 Warning / Warning Device 111 Data Acquisition Unit 112 Sound acquisition section 113 Position acquisition part 114 Alert Judgment Department 114a,122a Information Table 115 Corresponding Processing Unit 115a Playback section 115b Notification Department 116 First Communications Department 120,320 Information Acquisition Processing Device 121 Driving Information Acquisition Unit 122 Information acquisition judgment unit 123 Recognition part 124,324 Information output section 210,410 Data creation device 211,411 Information Gathering Department 211a Individual Support Information 211a-1 First Individual Response Information 211a-2 Second Individual Response Information 211a-3 Third Individual Response Information 211a-4 Fourth Individual Response Information 211a-5 Fifth Individual Response Information 211a-6 Information on the 6th Individual Response 211a-7 Information on the 7th Individual Response 211a-8 Information on the 8th Individual Response 211a-9 Individual Response Information for Section 9 211a-10 10th Individual Response Information 211a-11 Individual Response Information No. 11 212 Data Creation Department 213 Provision Department 214 Second Communications Department 220 Storage section 221 Data Structures 221a Road Network 221b Statistical correspondence information 221b-1 First statistical correspondence information 221b-2 Second statistical correspondence information 221b-3 Third statistical correspondence information 221b-4 Fourth statistical correspondence information 221b-5 Fifth statistical correspondence information 221c node 221d Road Link 221e points 412 Information generation section 412a Information acquisition judgment unit 412b Recognition part 412c Generator

Claims

1. An acquisition unit acquires data from a storage unit that associates location or region with information regarding sounds that the driver of the mobile object is likely to miss, based on at least one of the behavior of the mobile object and the behavior of the driver of the mobile object. A sound acquisition unit that acquires external sounds from the aforementioned moving object, A position acquisition unit that acquires the position of the moving body, A determination unit that determines whether the position of the moving body is within the position or region and whether the sound corresponding to the information is included in the external sound, Inside the moving body, there is a playback unit that plays the external sound in a playback mode corresponding to the determination result of the determination unit, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, wherein the determination unit performs an extraction process to extract a cautionary sound predetermined as a sound that the driver should pay attention to from the external sound acquired by the sound acquisition unit, and determines whether the cautionary sound matches the sound corresponding to the information associated with the position or region when the cautionary sound is extracted from the external sound and the position of the moving body is within the position or region.

3. The information processing apparatus according to claim 1 or 2, characterized in that, if the determination unit determines that the information contains a sound corresponding to the information, the playback unit plays the external sound in a playback manner that emphasizes the sound compared to other sounds.

4. The information processing device according to any one of claims 1 to 3, characterized in that, if the determination unit determines that a sound corresponding to the information is included, it is further provided with a notification unit that notifies the driver of the presence of the sound.

5. The acquisition step involves acquiring data from a storage unit that associates location or region with information regarding sounds that the driver of the mobile object is likely to miss, based on at least one of the behavior of the mobile object and the behavior of the driver of the mobile object. A sound acquisition step for acquiring external sounds from the moving object, A position acquisition step for acquiring the position of the moving body, A determination step of determining whether the position of the moving body is within the position or region and whether the sound corresponding to the information is included in the external sound, A playback step in which, inside the moving body, the external sound is reproduced in a playback mode corresponding to the determination result in the determination step, An information processing method characterized by comprising:

6. An information processing program characterized by causing a computer to execute the information processing method described in claim 5.

7. A storage medium characterized by storing the information processing program described in claim 6.

Images