Remote monitoring device, remote monitoring method, remote monitoring program, remote monitoring system, and remote monitoring device

Abstract

A remote monitoring device (10) is provided with: an acquisition unit (1) that acquires a plurality of pieces of position information indicating the position of each of a plurality of vehicles, and a plurality of pieces of sound data indicating sounds around each of the plurality of vehicles; a detection unit (3) that detects the siren sound of the emergency vehicle from each of the plurality of acquired sound data; an estimation unit (4) that estimates the position of the emergency vehicle on the basis of the plurality of pieces of position information of the plurality of vehicles and the plurality of pieces of sound data when a siren sound is detected from each of the plurality of pieces of sound data; a specifying unit (5) that specifies, from among the plurality of vehicles, a vehicle to which the emergency vehicle is approaching, on the basis of the estimated change in the position of the emergency vehicle over time; and a notification unit (6) that notifies the vehicle in which the emergency vehicle is approaching when the vehicle in which the emergency vehicle is approaching is identified from among the plurality of vehicles.

Description

Technical Field

[0001] The present disclosure relates to a technology for remotely monitoring a plurality of vehicles that travel autonomously and by remote control. Background Art

[0002] For example, the monitoring device of Patent Document 1 includes: an acquisition unit that acquires a plurality of voice information detected in a moving body that is remotely monitored by a monitor and moves by automatic driving; and a notification unit that notifies the monitor of target voice information, which is voice information that satisfies a given condition for confirming the required action status among the plurality of voice information acquired by the acquisition unit. Furthermore, the acquisition unit acquires a plurality of image information and a plurality of voice information detected in the moving body that is remotely monitored by a monitor and moves by automatic driving. The notification unit displays a plurality of image information on the monitoring device in a display mode that can identify the situation in which the image information corresponds to the target voice information, which is voice information that satisfies a given condition among the plurality of voice information acquired by the acquisition unit.

[0003] However, in the above-mentioned conventional technology, it is difficult to quickly and appropriately remotely control a vehicle that is approaching an emergency vehicle, and further improvement is needed.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 6971069 Summary of the invention

[0007] The present disclosure has been made to solve the above-mentioned problems, and an object of the present disclosure is to provide a technology capable of quickly and appropriately remotely controlling a vehicle that an emergency vehicle is approaching.

[0008] The remote monitoring device involved in the present disclosure remotely monitors a plurality of vehicles that travel autonomously and by remote control, and the remote monitoring device comprises: an acquisition unit, which acquires a plurality of position information indicating the positions of the respective vehicles, and a plurality of sound data indicating the surrounding sounds of the respective vehicles; a detection unit, which detects the sirens of emergency vehicles from the acquired plurality of sound data; an estimation unit, which estimates the position of the emergency vehicle based on the plurality of position information and the plurality of sound data of the plurality of vehicles when the sirens are detected from the plurality of sound data; a determination unit, which determines a vehicle that the emergency vehicle is approaching from among the plurality of vehicles based on a temporal change of the estimated position of the emergency vehicle; and a notification unit, which notifies the vehicle that the emergency vehicle is approaching when the vehicle that the emergency vehicle is approaching is determined from among the plurality of vehicles.

[0009] According to the present disclosure, it is possible to quickly and appropriately remotely control a vehicle to which an emergency vehicle is approaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a diagram showing the overall configuration of a remote monitoring system in an embodiment of the present disclosure.

[0011] Figure 2 It is a diagram showing the structure of a remote monitoring device in an embodiment of the present disclosure.

[0012] Figure 3 It is a block diagram showing a detailed structure of a detection unit in an embodiment of the present disclosure.

[0013] Figure 4 The figure shows an example of a speech waveform and a spectrogram of an ambulance siren.

[0014] Figure 5 The figure shows an example of a speech waveform and a spectrogram of a police car siren.

[0015] Figure 6 It is a block diagram showing a detailed configuration of an estimating unit in the embodiment of the present disclosure.

[0016] Figure 7 This is a schematic diagram for explaining a process of estimating the position of an emergency vehicle.

[0017] Figure 8 It is a block diagram showing a detailed configuration of an estimating unit in a modification of the embodiment of the present disclosure.

[0018] Fig. 9 It is a block diagram showing a detailed structure of a determination unit in an embodiment of the present disclosure.

[0019] Fig.10 This is a flowchart for explaining the operation of the remote monitoring process of the remote monitoring device in the embodiment of the present disclosure.

[0020] Fig.11 This is a diagram showing an example of a display screen displayed on the display unit in a normal mode when no emergency vehicle is approaching.

[0021] Fig.12 FIG. 1 is a diagram showing an example of a display screen displayed on the display unit in an emergency vehicle approaching mode in which an emergency vehicle is approaching.

[0022] Fig.13 This is a diagram showing another example of the display screen displayed on the display unit in the normal mode when no emergency vehicle is approaching.

[0023] Fig.14 It is a diagram showing another example of the display screen displayed on the display unit in the emergency vehicle approaching mode in which an emergency vehicle is approaching. DETAILED DESCRIPTION

[0024] (Knowledge that forms the basis of this disclosure)

[0025] Patent document 1 discloses the following scheme: a target voice message is notified to a monitor, and the target voice message is a voice message that satisfies a given condition requiring confirmation of an action status among a plurality of voice messages detected in a moving body. However, the monitoring device of patent document 1 cannot detect the positional relationship between the moving body being remotely monitored and the emergency vehicle, and cannot detect to what extent the emergency vehicle is approaching a moving body among a plurality of moving bodies. Therefore, there is the following problem: when an emergency vehicle is approaching a moving body, it is difficult to make the moving body appropriately avoid the emergency vehicle through remote control so as not to hinder the passage of the emergency vehicle.

[0026] In order to solve the above problems, the following technology is disclosed.

[0027] (1) A remote monitoring device according to one embodiment of the present disclosure remotely monitors a plurality of vehicles that are autonomously traveling and are traveling by remote control, the remote monitoring device comprising: an acquisition unit that acquires a plurality of position information indicating the positions of the plurality of vehicles and a plurality of sound data indicating the surrounding sounds of the plurality of vehicles; a detection unit that detects sirens of emergency vehicles from the acquired plurality of sound data; an estimation unit that estimates the position of the emergency vehicle based on the plurality of position information and the plurality of sound data of the plurality of vehicles when the sirens are detected from the plurality of sound data; a determination unit that determines a vehicle that the emergency vehicle is approaching from among the plurality of vehicles based on a temporal change in the estimated position of the emergency vehicle; and a notification unit that notifies the vehicle that the emergency vehicle is approaching when the emergency vehicle is determined from among the plurality of vehicles.

[0028] According to this configuration, since the vehicle to which the emergency vehicle is approaching is identified from among a plurality of vehicles and the approaching vehicle is notified, the approaching vehicle of the emergency vehicle can be remotely controlled promptly and appropriately.

[0029] (2) In the remote monitoring device described in (1) above, the remote monitoring device may also include: a frequency domain conversion unit for converting the multiple sound data in the time domain into multiple sound data in the frequency domain, and the detection unit includes: a peak frequency detection unit for detecting peak frequencies from the frequency spectra of each of the multiple sound data in the frequency domain; an emergency vehicle frequency pattern storage unit for pre-storing frequency patterns of peak frequencies of multiple siren sounds that differ according to the type of emergency vehicle; and a determination unit for comparing each of the frequency patterns of the detected multiple peak frequencies with each of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit, and when the frequency pattern of the detected peak frequency is consistent with any one of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit, determining that the siren sound is included in the sound data in which the peak frequency is detected.

[0030] According to this configuration, when the frequency pattern of the peak frequency detected from the sound data matches any of the plurality of frequency patterns pre-stored in the emergency vehicle frequency pattern storage unit, it is determined that the sound data contains a siren sound. Therefore, it is possible to easily detect whether the sound data contains a siren sound.

[0031] (3) In the remote monitoring device described in (1) or (2) above, the remote monitoring device may also include: a frequency domain conversion unit for converting the plurality of sound data in the time domain into a plurality of sound data in the frequency domain; the estimation unit includes: a peak frequency level detection unit for detecting the level of the peak frequency from the frequency spectrum of each of the plurality of sound data in the frequency domain; and a position estimation unit for estimating the position of the emergency vehicle using the plurality of position information of the plurality of vehicles, the detection results of the siren sounds of each of the plurality of sound data, and the levels of the plurality of peak frequencies detected from the plurality of sound data respectively.

[0032] There is a correlation between the distances between the multiple vehicles and the emergency vehicle and the magnitudes of the sound data including the siren sounds picked up by the multiple vehicles. Therefore, the position of the emergency vehicle can be estimated using the multiple position information of the multiple vehicles, the detection results of the siren sounds of the multiple sound data, and the levels of the multiple peak frequencies detected from the multiple sound data.

[0033] (4) In the remote monitoring device described in (1) or (2) above, the remote monitoring device may also include: a frequency domain conversion unit for converting the plurality of sound data in the time domain into a plurality of sound data in the frequency domain, the estimation unit including: a delay time calculation unit for calculating the delay time of two different sound data among the plurality of sound data in the frequency domain according to each of a plurality of combinations of the two different sound data; and a position estimation unit for estimating the position of the emergency vehicle using the plurality of position information of the plurality of vehicles, the detection results of the siren sounds of the respective plurality of sound data, and the plurality of delay times calculated according to each of the plurality of combinations.

[0034] The distances between the two different vehicles and the emergency vehicle can be used to estimate the delay time of the two different sound data picked up by the two different vehicles. Therefore, the estimated delay time of the two different sound data and the acquired delay time of the two different sound data can be used to estimate the position of the emergency vehicle.

[0035] (5) In the remote monitoring device described in any one of (1) to (4) above, the determination unit may also include: an emergency vehicle position storage unit for storing the estimated position of the emergency vehicle; a moving direction estimating unit for estimating the moving direction of the emergency vehicle based on the position of the emergency vehicle estimated last time and the position of the emergency vehicle estimated this time stored in the emergency vehicle position storage unit; and a vehicle determination unit for determining the vehicle that the emergency vehicle is approaching based on the estimated moving direction of the emergency vehicle and the multiple position information of the multiple vehicles.

[0036] According to this configuration, by estimating the moving direction of the emergency vehicle, it is possible to identify a vehicle existing on an extension line of the moving direction of the emergency vehicle as a vehicle approaching the emergency vehicle.

[0037] (6) In the remote monitoring device described in any one of (1) to (5) above, the notification unit may notify a vehicle located within a predetermined range from the estimated position of the emergency vehicle from among the plurality of vehicles.

[0038] According to this configuration, since vehicles located within a predetermined range from the estimated position of the emergency vehicle are notified from among a plurality of vehicles, the remote monitoring person can know the vehicles existing near the emergency vehicle and monitor the vehicles.

[0039] (7) In the remote monitoring device described in any one of (1) to (6) above, the acquisition unit may also acquire image data captured by cameras respectively provided to the plurality of vehicles, and when the vehicle to which the emergency vehicle is approaching is identified from among the plurality of vehicles, the notification unit outputs the image data captured by the cameras provided to the vehicle to which the emergency vehicle is approaching to the display unit.

[0040] According to this configuration, since the video data captured by the camera of the vehicle approaching the emergency vehicle is output to the display unit, the remote observer can remotely control the vehicle approaching the emergency vehicle while checking the video data displayed on the display unit.

[0041] (8) In the remote monitoring device described in any one of (1) to (7) above, it may be that, when the vehicle that the emergency vehicle is approaching is determined from among the multiple vehicles, the notification unit outputs a notification image to the display unit, the notification image including a map, a plurality of first icons indicating the positions of the multiple vehicles on the map, and a second icon indicating the estimated position of the emergency vehicle on the map.

[0042] According to this structure, when a vehicle on which an emergency vehicle is approaching is identified from among multiple vehicles, a notification image is output to the display unit, the notification image including a map, multiple first icons indicating the positions of the multiple vehicles on the map, and a second icon indicating the position of the estimated emergency vehicle on the map.

[0043] Therefore, the remote observer can know the position of the emergency vehicle and the positions of the plurality of vehicles by checking the notification image displayed on the display unit, and can remotely control the vehicle to which the emergency vehicle is approaching.

[0044] (9) In the remote monitoring device described in (8) above, the notification unit may display the first icon indicating the position on the map of the vehicle to which the emergency vehicle is approaching in a manner different from other first icons indicating the positions on the map of other vehicles to which the emergency vehicle is not approaching.

[0045] According to this structure, since the first icon indicating the position on the map of the vehicle to which an emergency vehicle is approaching is displayed differently from other first icons indicating the positions on the map of other vehicles to which no emergency vehicle is approaching, the remote monitor can easily know the position of the vehicle to which the emergency vehicle is approaching.

[0046] (10) In the remote monitoring device described in any one of (1) to (9) above, when the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles, the notification unit may output a notification sound to a speaker, wherein the notification sound is used to notify the vehicle that the emergency vehicle is approaching.

[0047] According to this structure, when an emergency vehicle is approaching from among a plurality of vehicles, a notification sound for notifying the approaching emergency vehicle is output to the speaker. Therefore, the remote monitor can easily know the presence of the approaching emergency vehicle by hearing the notification sound.

[0048] In addition, the present disclosure can be implemented not only as a remote monitoring device having the above-mentioned characteristic structure, but also as a remote monitoring method that executes characteristic processing corresponding to the characteristic structure of the remote monitoring device. In addition, it can also be implemented as a computer program that causes a computer to execute the characteristic processing included in such a remote monitoring method. Therefore, the following another method can also achieve the same effect as the above-mentioned remote monitoring device.

[0049] (11) Another aspect of the present disclosure relates to a remote monitoring method in a remote monitoring device for remotely monitoring a plurality of vehicles that are autonomously driven and driven by remote control. In the remote monitoring method, a plurality of position information indicating the positions of the plurality of vehicles and a plurality of sound data indicating the surrounding sounds of the plurality of vehicles are acquired, and sirens of emergency vehicles are respectively detected from the acquired plurality of sound data. When the sirens are respectively detected from the plurality of sound data, the position of the emergency vehicle is estimated based on the plurality of position information and the plurality of sound data of the plurality of vehicles. Based on the time change of the estimated position of the emergency vehicle, a vehicle that the emergency vehicle is approaching is determined from among the plurality of vehicles. When the vehicle that the emergency vehicle is approaching is determined from among the plurality of vehicles, the vehicle that the emergency vehicle is approaching is notified.

[0050] (12) Another embodiment of the present disclosure relates to a remote monitoring program that remotely monitors a plurality of vehicles that are autonomously traveling and are traveling by remote control, and the remote monitoring program causes a computer to function as the following parts: an acquisition unit that acquires a plurality of position information indicating the positions of the plurality of vehicles and a plurality of sound data indicating the surrounding sounds of the plurality of vehicles; a detection unit that detects the sirens of emergency vehicles from the plurality of sound data acquired; an estimation unit that estimates the position of the emergency vehicle based on the plurality of position information and the plurality of sound data of the plurality of vehicles when the sirens are detected from the plurality of sound data; a determination unit that determines a vehicle that the emergency vehicle is approaching from among the plurality of vehicles based on a time change of the estimated position of the emergency vehicle; and a notification unit that notifies the vehicle that the emergency vehicle is approaching when the emergency vehicle is determined from among the plurality of vehicles.

[0051] (13) A remote monitoring system according to another embodiment of the present disclosure comprises: a plurality of vehicles that travel autonomously and by remote control; and a remote monitoring device that remotely monitors the plurality of vehicles, wherein the plurality of vehicles respectively comprise: a position information acquisition unit that acquires position information indicating the position of the vehicle; a microphone that acquires sound data indicating the surrounding sound of the vehicle; and a communication unit that transmits the position information and the sound data to the remote monitoring device, wherein the remote monitoring device comprises: an acquisition unit that acquires a plurality of position information indicating the positions of the plurality of vehicles and a plurality of sound data indicating the surrounding sound of the plurality of vehicles. a plurality of sound data; a detection unit, which detects the sirens of emergency vehicles respectively from the acquired plurality of sound data; an estimating unit, which estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data when the sirens are respectively detected from the plurality of sound data; a determining unit, which determines the approaching vehicle of the emergency vehicle from among the plurality of vehicles based on the time change of the estimated position of the emergency vehicle; and a notifying unit, which notifies the approaching vehicle of the emergency vehicle when the approaching vehicle of the emergency vehicle is determined from among the plurality of vehicles.

[0052] According to this configuration, since the vehicle to which the emergency vehicle is approaching is identified from among a plurality of vehicles and the approaching vehicle is notified, the approaching vehicle of the emergency vehicle can be remotely controlled promptly and appropriately.

[0053] (14) Another embodiment of the present invention relates to a device comprising: an acquisition unit for acquiring a plurality of position information indicating the positions of a plurality of vehicles, and a plurality of sound data indicating the surrounding sounds of the plurality of vehicles; a detection unit for detecting the sirens of emergency vehicles from the plurality of sound data acquired; an estimation unit for estimating the position of the emergency vehicle based on the plurality of position information and the plurality of sound data of the plurality of vehicles when the sirens are detected from the plurality of sound data; a determination unit for determining the approaching vehicle of the emergency vehicle from among the plurality of vehicles based on the temporal change of the estimated position of the emergency vehicle; and a notification unit for notifying the approaching vehicle of the emergency vehicle when the approaching vehicle of the emergency vehicle is determined from among the plurality of vehicles.

[0054] (15) Another embodiment of the present disclosure relates to a non-temporary computer-readable recording medium having a remote monitoring program recorded thereon, the remote monitoring program remotely monitoring a plurality of vehicles that are autonomously driven and driven by remote control, the remote monitoring program causing a computer to function as: an acquisition unit that acquires a plurality of position information indicating the positions of the plurality of vehicles and a plurality of sound data indicating the surrounding sounds of the plurality of vehicles; a detection unit that detects the sirens of emergency vehicles from the plurality of sound data acquired; an estimation unit that estimates the position of the emergency vehicle based on the plurality of position information and the plurality of sound data of the plurality of vehicles when the sirens are detected from the plurality of sound data; a determination unit that determines a vehicle that the emergency vehicle is approaching from among the plurality of vehicles based on a time change of the estimated position of the emergency vehicle; and a notification unit that notifies the vehicle that the emergency vehicle is approaching when the emergency vehicle is determined from among the plurality of vehicles.

[0055] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments are examples of specific implementations of the present disclosure and do not limit the technical scope of the present disclosure.

[0056] (Implementation Method)

[0057] Figure 1 It is a diagram showing the overall configuration of a remote monitoring system in an embodiment of the present disclosure.

[0058] Figure 1 The remote monitoring system shown includes a remote monitoring device 10 , a first vehicle 11A, a second vehicle 11B, and a third vehicle 11C.

[0059] The remote monitoring device 10 is, for example, a personal computer, and can be connected to the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C via a network 12 so as to communicate with each other. The network 12 is, for example, the Internet. Figure 2 To be described later.

[0060] The first vehicle 11A, the second vehicle 11B, and the third vehicle 11C are, for example, electric robots, electric cars, electric trucks, or electric drones. For example, the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C move within a given area to carry the user's goods. The first vehicle 11A, the second vehicle 11B, and the third vehicle 11C are monitored by a remote monitor. The first vehicle 11A, the second vehicle 11B, and the third vehicle 11C drive autonomously and by remote control. The first vehicle 11A, the second vehicle 11B, and the third vehicle 11C normally drive autonomously, but in an emergency when an emergency vehicle is approaching, they drive by remote control performed by a remote monitor using a remote monitoring device 10.

[0061] In the present embodiment, the remote monitoring system includes three vehicles (the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C), but the present disclosure is not particularly limited thereto, and the remote monitoring system may include two vehicles or four or more vehicles.

[0062] The first vehicle 11A, the second vehicle 11B, and the third vehicle 11C all have the same structure. Therefore, only the structure of the first vehicle 11A will be described below.

[0063] The first vehicle 11A includes a position information acquisition unit 111 , a microphone 112 , a camera 113 , and a communication unit 114 .

[0064] The position information acquisition unit 111 is, for example, a GPS (Global Positioning System) receiver, and acquires position information indicating the position of the first vehicle 11A.

[0065] The microphone 112 acquires sound data indicating sounds around the first vehicle 11A.

[0066] The camera 113 captures the surroundings of the first vehicle 11A. The camera 113 captures the front, rear, or entire surroundings of the first vehicle 11A. The first vehicle 11A may include not one camera 113 but multiple cameras that capture the front and rear of the first vehicle 11A.

[0067] The communication unit 114 transmits the position information acquired by the position information acquisition unit 111, the sound data acquired by the microphone 112, and the image data acquired by the camera 113 to the remote monitoring device 10. The communication unit 114 periodically transmits the position information, the sound data, and the image data to the remote monitoring device 10. For example, the communication unit 114 may transmit the position information, the sound data, and the image data to the remote monitoring device 10 every 5 seconds. In this case, the communication unit 114 may transmit 5 seconds of sound data and 5 seconds of image data to the remote monitoring device 10.

[0068] Figure 2 1 is a diagram showing a configuration of a remote monitoring device 10 in an embodiment of the present disclosure.

[0069] The remote monitoring device 10 remotely monitors a plurality of vehicles (a first vehicle 11A, a second vehicle 11B, and a third vehicle 11C). Figure 2 The remote monitoring device 10 shown includes an acquisition unit 1 , a frequency domain conversion unit 2 , a detection unit 3 , an estimation unit 4 , a determination unit 5 , a notification unit 6 , a display unit 7 , a speaker 8 , and a communication unit 9 .

[0070] The communication unit 9 receives position information, audio data, and video data transmitted from a plurality of vehicles (the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C) under remote monitoring.

[0071] The acquisition unit 1 acquires a plurality of position information indicating the positions of each of the plurality of vehicles, and a plurality of sound data indicating the sounds around each of the plurality of vehicles. The acquisition unit 1 acquires the plurality of position information and the plurality of sound data from the communication unit 9. The acquisition unit 1 further acquires a plurality of image data captured by the cameras respectively provided in the plurality of vehicles. The acquisition unit 1 acquires the plurality of image data from the communication unit 9. The acquisition unit 1 outputs the position information to the estimation unit 4 and the determination unit 5, outputs the sound data to the frequency domain conversion unit 2, and outputs the image data to the notification unit 6.

[0072] The acquisition unit 1 acquires position information indicating the position of the first vehicle 11A and first sound data indicating the surrounding sound of the first vehicle 11A. In addition, the acquisition unit 1 acquires position information indicating the position of the second vehicle 11B and second sound data indicating the surrounding sound of the second vehicle 11B. In addition, the acquisition unit 1 acquires position information indicating the position of the third vehicle 11C and third sound data indicating the surrounding sound of the third vehicle 11C.

[0073] The frequency domain conversion unit 2 converts the sound data of each of the multiple vehicles acquired by the acquisition unit 1 into sound data in the frequency domain. The frequency domain conversion unit 2 converts the multiple sound data in the time domain acquired by the acquisition unit 1 into multiple sound data in the frequency domain. The frequency domain conversion unit 2 converts the sound data in the time domain into sound data in the frequency domain by fast Fourier transform.

[0074] The frequency domain conversion unit 2 converts the first sound data in the time domain acquired from the first vehicle 11A into the first sound data in the frequency domain. In addition, the frequency domain conversion unit 2 converts the second sound data in the time domain acquired from the second vehicle 11B into the second sound data in the frequency domain. In addition, the frequency domain conversion unit 2 converts the third sound data in the time domain acquired from the third vehicle 11C into the third sound data in the frequency domain.

[0075] The detection unit 3 detects the siren sound of the emergency vehicle according to each of the multiple sound data in the frequency domain transformed by the frequency domain transformation unit 2. The detection unit 3 detects the siren sound of the emergency vehicle from the multiple sound data respectively. The emergency vehicle is, for example, an ambulance, a fire truck, or a police car. The detection unit 3 outputs a siren sound detection result indicating whether the siren sound is detected from the multiple sound data respectively. In addition, the detailed structure of the detection unit 3 will be described later.

[0076] When the siren sound is detected from the plurality of sound data, the estimation unit 4 estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data. When the siren sound of the emergency vehicle is detected from the plurality of sound data by the detection unit 3, the estimation unit 4 estimates the position of the emergency vehicle based on the sound data in the frequency domain transformed by the frequency domain transformation unit 2 and the position information of each vehicle acquired by the acquisition unit 1. The estimation unit 4 outputs the position information indicating the estimated position of the emergency vehicle. In addition, the detailed structure of the estimation unit 4 will be described later.

[0077] The determination unit 5 determines the vehicle that the emergency vehicle is approaching from among the plurality of vehicles based on the time change of the position of the emergency vehicle estimated by the estimation unit 4. The determination unit 5 estimates the moving direction of the emergency vehicle based on the time change of the position of the emergency vehicle estimated by the estimation unit 4, and determines the vehicle that the emergency vehicle is approaching using the estimated moving direction of the emergency vehicle. The determination unit 5 outputs information for identifying the vehicle that the determined emergency vehicle is approaching. In addition, the detailed structure of the determination unit 5 will be described later.

[0078] When the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles by the identification unit 5, the notification unit 6 notifies the vehicle that the emergency vehicle is approaching. The notification unit 6 notifies the remote monitor 13 of the vehicle that the emergency vehicle is approaching by using the identification result of the vehicle that the emergency vehicle is approaching based on the identification unit 5, the plurality of image data acquired by the acquisition unit 1, and the position information of the plurality of vehicles acquired by the acquisition unit 1.

[0079] When a vehicle that an emergency vehicle is approaching is identified from among a plurality of vehicles, the notification unit 6 outputs a notification image to the display unit 7, the notification image including a map, a plurality of first icons indicating the positions on the map of a plurality of vehicles, and a second icon indicating the position on the map of the estimated emergency vehicle. The notification unit 6 displays the first icon indicating the position on the map of the vehicle to which the emergency vehicle is approaching in a manner different from other first icons indicating the positions on the map of other vehicles to which no emergency vehicle is approaching. For example, the notification unit 6 emphasizes the first icon indicating the position on the map of the vehicle to which the emergency vehicle is approaching. In addition, when a vehicle that an emergency vehicle is approaching is identified from among a plurality of vehicles, the notification unit 6 outputs image data captured by a camera of the vehicle to which the emergency vehicle is approaching to the display unit 7.

[0080] Furthermore, when the notification unit 6 identifies a vehicle that an emergency vehicle is approaching from among a plurality of vehicles, the notification unit 6 outputs a notification sound to the speaker 8 for notifying the vehicle that the emergency vehicle is approaching.

[0081] The display unit 7 is, for example, a liquid crystal display device, and displays the notification image and video data output by the notification unit 6 .

[0082] The speaker 8 outputs the notification sound output by the notification unit 6 to the outside.

[0083] The remote monitor 13 monitors the notification image displayed on the display unit 7 and the notification sound output from the speaker 8 , and remotely controls the vehicle as necessary.

[0084] In addition, in the present embodiment, the remote monitoring device 10 includes the display unit 7 and the speaker 8, but the present disclosure is not particularly limited thereto. The remote monitoring device 10 may not include the speaker 8 but may only include the display unit 7. In addition, the remote monitoring device 10 may not include the display unit 7 but may only include the speaker 8. In addition, the remote monitoring device 10 may not include the display unit 7 and the speaker 8, and may be connected to the display unit 7 and the speaker 8 provided outside.

[0085] In addition, the remote monitoring device 10 may also include an operation unit for receiving remote control of a vehicle by the remote monitor 13. The operation unit receives selection of a vehicle to be remotely controlled from among a plurality of vehicles by the remote monitor 13, and receives remote control of the selected vehicle. For example, the remote monitor 13 remotely controls a vehicle that an emergency vehicle is approaching, and moves the vehicle to a place where the emergency vehicle will not be obstructed from passing.

[0086] Next, Figure 2 The detailed structure of the detection unit 3 shown in FIG.

[0087] Figure 3 It is a block diagram showing a detailed structure of the detection unit 3 in the embodiment of the present disclosure.

[0088] The detector 3 includes a peak frequency detection unit 31 , an emergency vehicle frequency pattern storage unit 32 , and a siren sound determination unit 33 .

[0089] The peak frequency detection unit 31 detects a peak frequency from the frequency spectrum of each of the plurality of audio data in the frequency domain. The peak frequency detection unit 31 includes a first peak frequency detection unit 31A, a second peak frequency detection unit 31B, and a third peak frequency detection unit 31C.

[0090] The first peak frequency detection unit 31A detects a peak frequency from the frequency spectrum of the first sound data in the frequency domain collected by the first vehicle 11A.

[0091] The second peak frequency detection unit 31B detects a peak frequency from the frequency spectrum of the second sound data in the frequency domain collected by the second vehicle 11B.

[0092] The third peak frequency detection unit 31C detects a peak frequency from the frequency spectrum of the third sound data in the frequency domain collected by the third vehicle 11C.

[0093] The emergency vehicle frequency pattern storage unit 32 stores in advance frequency patterns of peak frequencies of a plurality of siren sounds that differ according to the types of emergency vehicles such as ambulances, fire trucks, and police cars. Siren sounds differ according to the types of emergency vehicles.

[0094] Figure 4 is a diagram showing an example of a speech waveform and a spectrogram of an emergency vehicle siren. Figure 5 The figure shows an example of a speech waveform and a spectrogram of a police car siren.

[0095] like Figure 4As shown, in the siren of an emergency vehicle, after the first peak frequency of about 960 Hz is detected for about 0.6 seconds, the second peak frequency of about 770 Hz is detected for about 0.6 seconds. The siren of an emergency vehicle is a frequency pattern that repeats the first peak frequency and the second peak frequency that is lower than the first peak frequency in every given period.

[0096] In addition, if Figure 5 As shown, in the police car siren, the frequency pattern increases from the first peak frequency of about 400 Hz to the second peak frequency of about 870 Hz and then decreases from the second peak frequency to the first peak frequency within a period of about 6 seconds. The police car siren has a frequency pattern that increases from the first peak frequency to the second peak frequency and then decreases from the second peak frequency to the first peak frequency within a given period.

[0097] The siren sound determination unit 33 compares the frequency pattern of the peak frequency of each of the multiple sound data detected by the peak frequency detection unit 31 with the frequency pattern of the peak frequency of each type of emergency vehicle stored in the emergency vehicle frequency pattern storage unit 32, and determines whether each of the multiple sound data contains the siren sound of an emergency vehicle.

[0098] The siren sound determination unit 33 analyzes the frequency patterns of the multiple peak frequencies detected from the multiple sound data by the peak frequency detection unit 31. The siren sound determination unit 33 compares the frequency patterns of the multiple peak frequencies detected from the multiple sound data by the peak frequency detection unit 31 with the frequency patterns of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32. And, when the frequency pattern of the detected peak frequency matches any one of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32, the siren sound determination unit 33 determines that the sound data in which the peak frequency is detected contains a siren sound.

[0099] More specifically, the siren sound determination unit 33 analyzes the frequency pattern of the peak frequency detected from the first sound data by the first peak frequency detection unit 31A. The siren sound determination unit 33 compares the frequency pattern of the peak frequency detected from the first sound data with the plurality of frequency patterns stored in the emergency vehicle frequency pattern storage unit 32. And, when the frequency pattern of the peak frequency detected from the first sound data matches any one of the plurality of frequency patterns stored in the emergency vehicle frequency pattern storage unit 32, the siren sound determination unit 33 determines that the first sound data in which the peak frequency is detected contains a siren sound.

[0100] In addition, the siren sound determination unit 33 analyzes the frequency pattern of the peak frequency detected from the second sound data by the second peak frequency detection unit 31B. The siren sound determination unit 33 compares the frequency pattern of the peak frequency detected from the second sound data with the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32. And, when the frequency pattern of the peak frequency detected from the second sound data matches any one of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32, the siren sound determination unit 33 determines that the second sound data in which the peak frequency is detected contains a siren sound.

[0101] In addition, the siren sound determination unit 33 analyzes the frequency pattern of the peak frequency detected from the third sound data by the third peak frequency detection unit 31C. The siren sound determination unit 33 compares the frequency pattern of the peak frequency detected from the third sound data with the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32. And, when the frequency pattern of the peak frequency detected from the third sound data matches any one of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit 32, the siren sound determination unit 33 determines that the third sound data in which the peak frequency is detected contains a siren sound.

[0102] The siren sound determination unit 33 outputs a siren sound detection result indicating whether or not a siren sound is included in each of the plurality of sound data. The siren sound determination unit 33 outputs a siren sound detection result for the first sound data, a siren sound detection result for the second sound data, and a siren sound detection result for the third sound data.

[0103] Furthermore, the siren sound determination unit 33 may include the type of emergency vehicle that matches the frequency pattern of the peak frequencies detected from the plurality of sound data in the siren sound detection result.

[0104] In addition, the siren sound determination unit 33 may output the type of emergency vehicle that matches the frequency pattern of the peak frequencies detected from the plurality of sound data to the notification unit 6. Furthermore, the notification unit 6 may also notify the type of the emergency vehicle. For example, the notification unit 6 may output information indicating which of the ambulance, fire truck, or police car is approaching to the display unit 7.

[0105] Next, Figure 2 The detailed structure of the estimating unit 4 shown in FIG.

[0106] Figure 6 is a block diagram showing a detailed structure of the estimation unit 4 in the embodiment of the present disclosure. Figure 6 2 shows the estimating unit 4 in a case where three vehicles are remotely monitored, but the number of vehicles to be remotely monitored is not limited to three.

[0107] The estimating unit 4 includes a peak frequency level detecting unit 41 and a position estimating unit 42 .

[0108] The peak frequency level detection unit 41 detects the level of the peak frequency from the frequency spectrum of each of the plurality of audio data in the frequency domain. The peak frequency level detection unit 41 includes a first peak frequency level detection unit 41A, a second peak frequency level detection unit 41B, and a third peak frequency level detection unit 41C.

[0109] The first peak frequency level detection unit 41A detects the level of the peak frequency from the frequency spectrum of the first sound data in the frequency domain collected by the first vehicle 11A.

[0110] The second peak frequency level detection unit 41B detects the level of the peak frequency from the frequency spectrum of the second sound data in the frequency domain collected by the second vehicle 11B.

[0111] The third peak frequency level detection unit 41C detects the level of the peak frequency from the frequency spectrum of the third sound data in the frequency domain collected by the third vehicle 11C.

[0112] The position estimating unit 42 estimates the position of the emergency vehicle using a plurality of position information of a plurality of vehicles, the detection results of the siren sounds of each of a plurality of sound data, and the levels of a plurality of peak frequencies detected from each of the plurality of sound data.

[0113] More specifically, the position estimating unit 42 estimates the position of the emergency vehicle using the position information of the first vehicle 11A, the position information of the second vehicle 11B, the position information of the third vehicle 11C, the siren detection result of the first sound data, the siren detection result of the second sound data, the siren detection result of the third sound data, the level of the peak frequency detected from the first sound data, the level of the peak frequency detected from the second sound data, and the level of the peak frequency detected from the third sound data.

[0114] Furthermore, the position estimating unit 42 identifies the plurality of sound data in which the siren sound is detected based on the detection results of the siren sound of each of the plurality of sound data, and estimates the position of the emergency vehicle using the plurality of position information of the plurality of vehicles that picked up the identified plurality of sound data and the levels of the plurality of peak frequencies detected from the identified plurality of sound data.

[0115] Figure 7 This is a schematic diagram for explaining a process of estimating the position of an emergency vehicle.

[0116] exist Figure 7In the example, the emergency vehicle 201 is traveling near the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C. The first sound data, the second sound data, and the third sound data picked up by the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C include the siren sound of the emergency vehicle 201. Assuming the position of the first vehicle 11A is (xa, ya), the position of the second vehicle 11B is (xb, yb), the position of the third vehicle 11C is (xc, yc), and the position of the emergency vehicle 201 is (x0, y0), the distances da, db, and dc between the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C and the emergency vehicle 201 are expressed by the following mathematical formulas (1) to (3). The position information of the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C is known and acquired by the acquisition unit 1.

[0117]

[0118]

[0119]

[0120] If the levels of the peak frequencies detected from the first sound data, the second sound data, and the third sound data collected by the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C are Pa, Pb, and Pc, since the magnitude of the sound is proportional to the inverse of the distance, Pa, Pb, Pc, da, db, and dc are expressed as Pa: Pb = 1 / da: 1 / db, Pb: Pc = 1 / db: 1 / dc, and Pc: ​​Pa = 1 / dc: 1 / da. By transforming these mathematical expressions, the relationship between the distances da, db, and dc is expressed by the following mathematical expressions (4) to (6) using Pa, Pb, and Pc.

[0121]

[0122]

[0123]

[0124] The position estimation unit 42 sequentially scans the positions (x, y) on a given map, substitutes x into x0 of the above-mentioned mathematical formulas (1) to (3), and substitutes y into y0, and calculates da, db, and dc. In addition, the size of the given map, that is, the range of the position (x, y) to be scanned is set based on the positions of the plurality of vehicles (the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C) from which the sound data of the siren sound detected by the emergency vehicle is acquired. For example, a given range from the plurality of vehicles (the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C) may be set as the scanning range. The position estimation unit 42 calculates da / db, db / dc, and dc / da using the calculated da, db, and dc. The position estimation unit 42 calculates the error between da / db and Pb / Pa, the error between db / dc and Pb / Pc, and the error between dc / da and Pc / Pa for all the position coordinates on the given map. The position estimating unit 42 calculates the position (x, y) where the errors between the calculated da / db, db / dc, and dc / da and Pb / Pa, Pb / Pc, and Pc / Pa are minimized as the position (x0, y0) of the emergency vehicle.

[0125] That is, the position estimation unit 42 sequentially calculates da / db, db / dc, and dc / da from the coordinate position at the upper left end to the coordinate position at the lower right end on a given map. Furthermore, the position estimation unit 42 calculates the coordinate position where the difference between da / db and Pb / Pa, the difference between db / dc and Pb / Pc, and the difference between dc / da and Pc / Pa are closest to 0 as the position (x0, y0) of the emergency vehicle.

[0126] Furthermore, the position of the emergency vehicle is not limited to be estimated based on three vehicles, and the position of the emergency vehicle can be estimated similarly even if it is estimated based on four or more vehicles.

[0127] Next, the detailed configuration of the estimating unit in a modified example of the present embodiment will be described.

[0128] Figure 8 1 is a block diagram showing a detailed configuration of an estimating unit 4A in a modified example of the embodiment of the present disclosure. The estimating unit 4A in the modified example of the present embodiment estimates the position of the emergency vehicle based on the difference in arrival time of sounds from the emergency vehicle to each vehicle.

[0129] The estimating unit 4A includes a delay time calculating unit 43 and a position estimating unit 42A.

[0130] The delay time calculation unit 43 calculates the delay time of two different sound data among the plurality of sound data in the frequency domain for each of the plurality of combinations of two different sound data. The delay time calculation unit 43 includes a first delay time calculation unit 43A, a second delay time calculation unit 43B, and a third delay time calculation unit 43C.

[0131] The first delay time calculation unit 43A calculates the delay time of the siren sound of the emergency vehicle reaching the second vehicle 11B relative to the siren sound of the emergency vehicle reaching the first vehicle 11A based on the first sound data in the frequency domain picked up by the first vehicle 11A and the second sound data in the frequency domain picked up by the second vehicle 11B. The first delay time calculation unit 43A calculates the delay time between the first sound data and the second sound data. The first delay time calculation unit 43A divides the first sound data in the frequency domain picked up by the first vehicle 11A by the second sound data in the frequency domain picked up by the second vehicle 11B, converts the result of the division into sound data in the time domain, and obtains the impulse response of the converted sound data in the time domain, thereby calculating the delay time of the siren sound of the second vehicle 11B relative to the first vehicle 11A.

[0132] The second delay time calculation unit 43B calculates the delay time of the siren sound of the emergency vehicle arriving at the third vehicle 11C relative to the siren sound of the emergency vehicle arriving at the second vehicle 11B. The second delay time calculation unit 43B calculates the delay time between the second sound data and the third sound data. The calculation method of the delay time by the second delay time calculation unit 43B is the same as the calculation method of the delay time by the first delay time calculation unit 43A.

[0133] The third delay time calculation unit 43C calculates the delay time of the siren sound of the emergency vehicle arriving at the first vehicle 11A relative to the siren sound of the emergency vehicle arriving at the third vehicle 11C. The third delay time calculation unit 43C calculates the delay time between the third sound data and the first sound data. The calculation method of the delay time by the third delay time calculation unit 43C is the same as the calculation method of the delay time by the first delay time calculation unit 43A.

[0134] The position estimating unit 42A estimates the position of the emergency vehicle using the plurality of position information of the plurality of vehicles, the detection results of the siren sounds of the plurality of sound data, and the plurality of delay times calculated for each of the plurality of combinations.

[0135] More specifically, the position estimation unit 42A estimates the position of the emergency vehicle using the position information of the first vehicle 11A, the position information of the second vehicle 11B, the position information of the third vehicle 11C, the siren detection result of the first sound data, the siren detection result of the second sound data, the siren detection result of the third sound data, the delay time of the second sound data relative to the first sound data, the delay time of the third sound data relative to the second sound data, and the delay time of the first sound data relative to the third sound data.

[0136] Furthermore, the position estimating unit 42A identifies the plurality of sound data in which the siren sound is detected based on the detection results of the siren sound of each of the plurality of sound data, and estimates the position of the emergency vehicle using the plurality of position information of the plurality of vehicles that picked up the identified plurality of sound data and the delay time calculated based on the identified plurality of sound data.

[0137] The distances da, db, and dc between the first vehicle 11A, the second vehicle 11B, and the third vehicle 11C and the emergency vehicle 201 are represented by the above-mentioned mathematical expressions (1) to (3).

[0138] Assuming that the delay time of the sound data of the second vehicle 11B relative to the first vehicle 11A, the delay time of the sound data of the third vehicle 11C relative to the second vehicle 11B, and the delay time of the sound data of the first vehicle 11A relative to the third vehicle 11C are LT1, LT2, and LT3, the relationship between the distances da, db, and dc is expressed by the following mathematical formulas (7) to (9) using LT1, LT2, and LT3. In addition, c represents the speed of sound (340 m / sec).

[0139]

[0140]

[0141]

[0142] The position estimation unit 42A sequentially scans the positions (x, y) on the given map, substitutes x into x0 of the above mathematical formulas (1) to (3), and substitutes y into y0 to calculate da, db, and dc. The position estimation unit 42A uses the calculated da, db, and dc to calculate (db-da) / c, (dc-db) / c, and (da-dc) / c. The position estimation unit 42A calculates the error between the value of (db-da) / c and the delay time LT1, the error between the value of (dc-db) / c and the delay time LT2, and the error between the value of (da-dc) / c and the delay time LT3 for all position coordinates on the given map. The position estimating unit 42A calculates the position (x0, y0) of the emergency vehicle as the position (x, y) where the errors between the calculated values ​​of (db-da) / c, (dc-db) / c and (da-dc) / c and the delay times LT1, LT2 and LT3 calculated by the first delay time calculating unit 43A, the second delay time calculating unit 43B and the third delay time calculating unit 43C are minimized.

[0143] That is, the position estimation unit 42A sequentially calculates (db-da) / c, (dc-db) / c, and (da-dc) / c from the coordinate position at the upper left end to the coordinate position at the lower right end on a given map. The position estimation unit 42A then calculates the coordinate position closest to 0 between the difference between (db-da) / c and LT1, the difference between (dc-db) / c and LT2, and the difference between (da-dc) / c and LT3 as the position (x0, y0) of the emergency vehicle.

[0144] Furthermore, the position of the emergency vehicle is not limited to be estimated based on three vehicles, and the position of the emergency vehicle can be estimated similarly even if it is estimated based on four or more vehicles.

[0145] In addition, the delay time calculation unit 43 may limit the frequency spectrum to the frequency band of the emergency vehicle siren sound (e.g., 300 Hz to 1200 Hz), divide the frequency spectrum of the sound data with the frequencies other than the siren sound attenuated, and convert it into the sound data in the time domain. In this way, the noise components other than the emergency vehicle siren sound are reduced, and the delay time can be calculated more accurately.

[0146] Next, Figure 2 The detailed structure of the determination unit 5 shown in FIG.

[0147] Fig. 9 It is a block diagram showing a detailed structure of the determination unit 5 in the embodiment of the present disclosure.

[0148] The identification unit 5 includes an emergency vehicle position storage unit 51 , a movement direction estimation unit 52 , and a vehicle identification unit 53 .

[0149] The emergency vehicle position storage unit 51 stores the position of the emergency vehicle estimated by the estimating unit 4 .

[0150] The moving direction estimating unit 52 estimates the moving direction of the emergency vehicle based on the temporal change in the position of the emergency vehicle estimated by the estimating unit 4. The moving direction estimating unit 52 estimates the moving direction of the emergency vehicle based on the position of the emergency vehicle estimated last time and the position of the emergency vehicle estimated this time stored in the emergency vehicle position storage unit 51.

[0151] The vehicle identification unit 53 identifies the vehicle to which the emergency vehicle is approaching based on the moving direction of the emergency vehicle estimated by the moving direction estimation unit 52 and a plurality of position information of a plurality of vehicles.

[0152] The vehicle identification unit 53 identifies a vehicle located on an extension line of the moving direction of the emergency vehicle as a vehicle that the emergency vehicle is approaching. In addition, the vehicle identification unit 53 may also identify a vehicle located on an extension line of the moving direction of the emergency vehicle and on the same road (road section) as the emergency vehicle as a vehicle that the emergency vehicle is approaching.

[0153] In addition, the determination unit 5 may also include a vehicle position storage unit that stores position information of multiple vehicles, and a vehicle moving direction estimating unit that estimates the moving directions of multiple vehicles. The vehicle moving direction estimating unit may also estimate the moving directions of each of the multiple vehicles based on the positions of the multiple vehicles acquired last time and stored in the vehicle position storage unit, and the positions of the multiple vehicles acquired this time. The vehicle determination unit 53 may also determine whether the extension line of the moving direction of the emergency vehicle intersects with the extension line of the moving directions of each of the multiple vehicles. The vehicle determination unit 53 may also determine the vehicle whose extension line of the moving direction of the emergency vehicle intersects with the extension line of the moving direction of the vehicle as the vehicle that the emergency vehicle is approaching.

[0154] In addition, in the present embodiment, the acquisition unit 1, the frequency domain conversion unit 2, the peak frequency detection unit 31, the siren sound determination unit 33, the peak frequency level detection unit 41, the position estimation units 42 and 42A, the delay time calculation unit 43, the movement direction estimation unit 52, and the vehicle identification unit 53 are implemented by a processor. The processor is composed of, for example, a central processing unit (CPU).

[0155] In the present embodiment, the emergency vehicle frequency pattern storage unit 32 and the emergency vehicle position storage unit 51 are implemented by a memory, such as a ROM (Read Only Memory) or an EEPROM (Electrically Erasable Programmable Read Only Memory).

[0156] Next, the operation of the remote monitoring process of the remote monitoring device 10 in the embodiment of the present disclosure will be described.

[0157] Fig.10 This is a flowchart for explaining the operation of the remote monitoring process of the remote monitoring device 10 in the embodiment of the present disclosure.

[0158] First, in step S1 , the acquisition unit 1 acquires position information and voice data of each of the plurality of vehicles. At this time, the acquisition unit 1 acquires position information and voice data of each of the plurality of vehicles received by the communication unit 9 .

[0159] Next, in step S2 , the frequency domain conversion unit 2 converts the plurality of sound data in the time domain acquired by the acquisition unit 1 into a plurality of sound data in the frequency domain.

[0160] Next, in step S3 , the detector 3 detects the sirens of the emergency vehicles from the plurality of sound data in the frequency domain.

[0161] Next, in step S4, the estimation unit 4 determines whether the siren sound is detected from each of the plurality of sound data. At this time, the estimation unit 4 may also determine whether the siren sound is detected from all of the plurality of sound data. In addition, the estimation unit 4 may also determine whether the siren sound is detected from a given number or more of the plurality of sound data. The given number is, for example, 2 or 3. If there are two sound data, the position of the emergency vehicle can be estimated, and if there are three or more sound data, the position of the emergency vehicle can be reliably estimated.

[0162] Here, when it is determined that the siren sound is not detected from each of the plurality of sound data (NO in step S4 ), the process returns to step S1 .

[0163] On the other hand, when it is determined that the siren sound is detected from each of the plurality of sound data (YES in step S4 ), in step S5 , the estimating unit 4 estimates the position of the emergency vehicle based on the position information of each of the plurality of vehicles and the plurality of sound data.

[0164] Next, in step S6 , the identification unit 5 identifies a vehicle to which the emergency vehicle is approaching from among the plurality of vehicles based on the temporal change in the position of the emergency vehicle estimated by the estimation unit 4 .

[0165] Next, in step S7, the notification unit 6 determines whether the approaching emergency vehicle has been identified from among the plurality of vehicles by the identification unit 5. Even if the plurality of sound data collected by the plurality of vehicles includes a siren sound, there is a case where there is no approaching emergency vehicle among the plurality of vehicles. In such a case, it is not necessary to notify the remote monitor. Therefore, the notification unit 6 determines whether the approaching emergency vehicle has been identified from among the plurality of vehicles.

[0166] Here, when it is determined that the approaching vehicle of the emergency vehicle has not been identified (NO in step S7 ), the process returns to step S1 .

[0167] On the other hand, when it is determined that the approaching vehicle of the emergency vehicle has been identified (YES in step S7 ), in step S8 , the notification unit 6 notifies the vehicle of the approaching emergency vehicle.

[0168] In this way, since the vehicle to which the emergency vehicle is approaching is identified from among a plurality of vehicles and the vehicle to which the emergency vehicle is approaching is notified, it is possible to remotely control the vehicle to which the emergency vehicle is approaching quickly and appropriately.

[0169] Fig.11 2 is a diagram showing an example of a display screen displayed on the display unit 7 in a normal mode when no emergency vehicle is approaching. Fig.12 It is a diagram showing an example of a display screen displayed on the display unit 7 in an emergency vehicle approaching mode in which an emergency vehicle is approaching.

[0170] exist Fig.11 In the normal mode, the display unit 7 displays a map image 81 showing the positions of a plurality of vehicles on a map, and images 82, 83, 84 obtained by photographing the front (travel direction) of each vehicle. On the map image 81, first icons 811, 812, 813 indicating the positions of a plurality of vehicles (the first vehicle, the second vehicle, and the third vehicle) are displayed. When a vehicle that an emergency vehicle is approaching is identified based on a plurality of sound data, the display screen changes from Fig.11 The normal mode automatically switches to Fig.12 Emergency vehicle approach mode.

[0171] Furthermore, in the normal mode, the display unit 7 may further display arrows indicating the movement directions of each of the plurality of vehicles.

[0172] exist Fig.12 In the emergency vehicle approaching mode, the display unit 7 displays a map image 81, a vehicle ( Fig.12On the map image 81, there are displayed first icons 811, 812, 813 indicating the positions of a plurality of vehicles (the first vehicle, the second vehicle, and the third vehicle), and a second icon 814 indicating the position of the emergency vehicle. In addition, the first icon 813 indicating the approaching emergency vehicle is highlighted. For example, the first icon 813 is decorated so that it is prominently displayed on the map, displayed in a flashing manner, or displayed in a color different from the other first icons 811, 812.

[0173] Furthermore, in the emergency vehicle approaching mode, the display unit 7 may further display an arrow indicating the moving direction of the emergency vehicle. Furthermore, in the emergency vehicle approaching mode, the display unit 7 may further display arrows indicating the moving directions of each of the plurality of vehicles.

[0174] In addition, in Fig.12 In such a case where there is an approaching emergency vehicle, the speaker 8 may also notify the remote observer of the approaching emergency vehicle by sound.

[0175] Fig.13 2 is a diagram showing another example of a display screen displayed on the display unit 7 in a normal mode when no emergency vehicle is approaching. Fig.14 1 is a diagram showing another example of the display screen displayed on the display unit 7 in the emergency vehicle approaching mode in which an emergency vehicle is approaching.

[0176] exist Fig.13 In the normal mode, the display unit 7 displays a map image 91 showing the positions of multiple vehicles on the map, an image 92 obtained by photographing the front (travel direction) of the vehicle selected by the remote monitor, and an image 93 obtained by photographing the front (travel direction) of other vehicles not selected by the remote monitor. Fig.13 The map image 91 shown is Fig.11 The input unit (not shown) may also receive a selection of one image from a plurality of images captured by a plurality of vehicles by a remote observer. The image 92 selected by the remote observer is displayed larger than the image 93 not selected by the remote observer. When an emergency vehicle is identified as approaching based on a plurality of sound data, the display screen is changed from Fig.13 The normal mode automatically switches to Fig.14 Emergency vehicle approach mode.

[0177] exist Fig.14In the emergency vehicle approaching mode, the display unit 7 displays a map image 91, an image 94 obtained by photographing the direction of the vehicle approaching the emergency vehicle, and an image 95 obtained by photographing the direction of the vehicle approaching the emergency vehicle different from the direction of the emergency vehicle. In addition, the speaker 8 notifies the remote observer of the approaching emergency vehicle by sound. Fig.14 The map image 91 shown is Fig.12 The image 94 obtained by photographing the direction of the approaching emergency vehicle is displayed larger than the image 95 obtained by photographing the direction of the approaching emergency vehicle from another direction different from the direction of the emergency vehicle. The remote monitor refers to the images 94 and 95 photographed by the approaching emergency vehicle and the map image 91. In addition, the speaker 8 can also output the sound data transmitted by the approaching emergency vehicle.

[0178] In the normal mode, the image 92 selected by the remote monitor is displayed at the largest size. In contrast, in the emergency vehicle approaching mode, the image 94 captured in the direction of the approaching emergency vehicle is displayed at the largest size.

[0179] In the emergency vehicle approaching mode, the remote monitor can remotely control the vehicle to which the emergency vehicle is approaching as needed so as not to obstruct the passage of the emergency vehicle.

[0180] In addition, in the present embodiment, the notification unit 6 may notify a vehicle within a given range from the position of the emergency vehicle estimated by the estimating unit 4 from among the plurality of vehicles. The notification unit 6 may also calculate the distance between each of the plurality of vehicles and the emergency vehicle based on the plurality of position information of the plurality of vehicles and the position information of the emergency vehicle estimated by the estimating unit 4, and notify the vehicle whose calculated distance is less than the given distance.

[0181] In addition, in the present embodiment, the determination unit 5 may further include: a moving speed estimating unit that estimates the moving speed of the emergency vehicle based on the time change of the position of the emergency vehicle estimated by the estimating unit 4. The moving speed estimating unit may also estimate the moving speed of the emergency vehicle based on the position of the emergency vehicle estimated last time, the position of the emergency vehicle estimated this time, and the time from the last estimation to the current estimation stored in the emergency vehicle position storage unit 51. The notification unit 6 may also notify the estimated moving speed of the emergency vehicle.

[0182] In addition, even if the position of the emergency vehicle is estimated, if the emergency vehicle is stopped, the determination unit 5 cannot estimate the moving direction of the emergency vehicle and cannot determine the vehicle that the emergency vehicle is approaching. Therefore, even if the position of the emergency vehicle is estimated, if the vehicle that the emergency vehicle is approaching is not determined, the determination unit 5 can determine that the vehicle is within a given range from the position of the emergency vehicle estimated by the estimation unit 4 as the vehicle that the emergency vehicle is approaching.

[0183] In the present embodiment, the microphone 112 for acquiring sound data representing sounds around the first vehicle 11A is installed in the first vehicle 11A, but the present disclosure is not particularly limited thereto. For example, the microphone 112 may be installed on the road.

[0184] In addition, the present disclosure can be applied not only to a remote monitoring device that remotely controls an autonomously driven vehicle, but also to a server device that notifies a vehicle with a communication function that an emergency vehicle is approaching. The server device can also obtain multiple position information representing the respective positions of multiple vehicles with a communication function, and obtain multiple sound data representing the surrounding sounds of each of the multiple vehicles. In addition, when the server device determines that an emergency vehicle is approaching a vehicle, it can also notify the vehicle that the emergency vehicle is approaching.

[0185] In addition, in each of the above-mentioned embodiments, each structural element may also be implemented by being constituted by dedicated hardware or executing a software program suitable for each structural element. Each structural element may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory. In addition, the program may also be implemented by another independent computer system by recording the program in a recording medium and transferring it, or by transferring the program via a network.

[0186] Part or all of the functions of the device involved in the embodiments of the present disclosure are typically implemented as LSI (Large Scale Integration) as an integrated circuit. They can be either individually single-chip or single-chip to include part or all. In addition, integrated circuitization is not limited to LSI, and can also be implemented by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connection and settings of the circuit unit inside the LSI.

[0187] Furthermore, part or all of the functions of the apparatus according to the embodiments of the present disclosure may be implemented by a processor such as a CPU executing a program.

[0188] In addition, all the numbers used in the above description are exemplified in order to specifically describe the present disclosure, and the present disclosure is not limited to the exemplified numbers.

[0189] In addition, the order of executing each step shown in the above flowchart is exemplified for the purpose of specifically explaining the present disclosure, and other orders may be used within the scope of obtaining the same effect. In addition, part of the above steps may be executed simultaneously (in parallel) with other steps.

[0190] Industrial Applicability

[0191] The technology according to the present disclosure can quickly and appropriately remotely control a vehicle approaching an emergency vehicle, and is therefore useful as a technology for remotely monitoring a plurality of vehicles that are autonomously traveling and traveling by remote control.

Claims

1. A remote monitoring device for remotely monitoring a plurality of vehicles that are autonomously traveling and are traveling by remote control, the remote monitoring device comprising: an acquisition unit that acquires a plurality of pieces of position information indicating the positions of the respective vehicles and a plurality of pieces of sound data indicating surrounding sounds of the respective vehicles; a detection unit, detecting the sirens of emergency vehicles respectively from the acquired plurality of sound data; an estimating unit that estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data when the siren sound is detected from each of the plurality of sound data; a determination unit that determines, from among the plurality of vehicles, a vehicle to which the emergency vehicle is approaching based on the estimated temporal change in the position of the emergency vehicle; and The notification unit notifies the vehicle that the emergency vehicle is approaching when the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles.

2. The remote monitoring device according to claim 1, wherein: The remote monitoring device further comprises: a frequency domain conversion unit that converts the plurality of sound data in the time domain into a plurality of sound data in the frequency domain; The detection unit comprises: a peak frequency detection unit that detects a peak frequency from a frequency spectrum of each of the plurality of sound data in the frequency domain; An emergency vehicle frequency pattern storage unit pre-stores frequency patterns of peak frequencies of a plurality of siren sounds that differ according to the type of emergency vehicle; and The determination unit compares each of the frequency patterns of the detected multiple peak frequencies with each of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit, and determines that the siren sound is included in the sound data in which the peak frequency is detected, when the frequency pattern of the detected peak frequency is consistent with any one of the multiple frequency patterns stored in the emergency vehicle frequency pattern storage unit.

3. The remote monitoring device according to claim 1, wherein: The remote monitoring device further comprises: a frequency domain conversion unit that converts the plurality of sound data in the time domain into a plurality of sound data in the frequency domain; The estimation unit comprises: a peak frequency level detection unit that detects a level of a peak frequency from a frequency spectrum of each of the plurality of audio data in the frequency domain; and The position estimating unit estimates the position of the emergency vehicle using the plurality of position information of the plurality of vehicles, the detection results of the siren sound of each of the plurality of sound data, and the levels of a plurality of peak frequencies detected from the plurality of sound data.

4. The remote monitoring device according to claim 1, wherein: The remote monitoring device further comprises: a frequency domain conversion unit that converts the plurality of sound data in the time domain into a plurality of sound data in the frequency domain; The estimation unit comprises: a delay time calculation unit that calculates a delay time of two different sound data among the plurality of sound data in the frequency domain for each of a plurality of combinations of the two different sound data; and The position estimating unit estimates the position of the emergency vehicle using the plurality of position information of the plurality of vehicles, the detection results of the siren sounds of the respective plurality of sound data, and the plurality of delay times calculated for each of the plurality of combinations.

5. The remote monitoring device according to claim 1, wherein: The determining unit comprises: an emergency vehicle position storage unit for storing the estimated position of the emergency vehicle; a moving direction estimating unit that estimates the moving direction of the emergency vehicle based on the position of the emergency vehicle estimated last time and the position of the emergency vehicle estimated this time stored in the emergency vehicle position storage unit; and The vehicle identification unit identifies the vehicle that the emergency vehicle is approaching based on the estimated moving direction of the emergency vehicle and the plurality of position information of the plurality of vehicles.

6. The remote monitoring device according to claim 1, wherein: The notification unit notifies a vehicle located within a given range from the estimated position of the emergency vehicle from among the plurality of vehicles.

7. The remote monitoring device according to claim 1, wherein: The acquisition unit further acquires image data captured by cameras respectively provided in the plurality of vehicles. When the vehicle to which the emergency vehicle is approaching is identified from among the plurality of vehicles, the notification unit outputs the image data captured by the camera included in the vehicle to which the emergency vehicle is approaching to a display unit.

8. The remote monitoring device according to claim 1, wherein: When the vehicle that the emergency vehicle is approaching is determined from among the multiple vehicles, the notification unit outputs a notification image to the display unit, the notification image including a map, multiple first icons indicating the positions of the multiple vehicles on the map, and a second icon indicating the estimated position of the emergency vehicle on the map.

9. The remote monitoring device according to claim 8, wherein: The notification unit displays a first icon indicating a position on the map of the vehicle to which the emergency vehicle is approaching differently from other first icons indicating positions on the map of other vehicles to which the emergency vehicle is not approaching.

10. The remote monitoring device according to claim 1, wherein: When the vehicle to which the emergency vehicle is approaching is identified from among the plurality of vehicles, the notification unit outputs a notification sound to a speaker, the notification sound being used to notify the vehicle that the emergency vehicle is approaching.

11. A remote monitoring method, in a remote monitoring device for remotely monitoring a plurality of vehicles that are autonomously traveling and are traveling by remote control, In the remote monitoring method, acquiring a plurality of pieces of position information indicating the positions of the respective vehicles and a plurality of pieces of sound data indicating surrounding sounds of the respective vehicles, Detecting the sirens of emergency vehicles from the acquired plurality of sound data respectively, When the siren sound is detected from each of the plurality of sound data, the position of the emergency vehicle is estimated based on the plurality of position information of the plurality of vehicles and the plurality of sound data, determining a vehicle to which the emergency vehicle is approaching from among the plurality of vehicles based on the estimated temporal change in the position of the emergency vehicle, When the vehicle to which the emergency vehicle is approaching is identified from among the plurality of vehicles, the vehicle is notified of the approaching emergency vehicle.

12. A remote monitoring program for remotely monitoring a plurality of vehicles that are autonomously traveling and are traveling by remote control, the remote monitoring program causing a computer to function as: an acquisition unit that acquires a plurality of pieces of position information indicating the positions of the respective vehicles and a plurality of pieces of sound data indicating surrounding sounds of the respective vehicles; a detection unit, detecting the sirens of emergency vehicles respectively from the acquired plurality of sound data; an estimating unit that estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data when the siren sound is detected from each of the plurality of sound data; a determination unit that determines, from among the plurality of vehicles, a vehicle to which the emergency vehicle is approaching based on the estimated temporal change in the position of the emergency vehicle; and The notification unit notifies the vehicle that the emergency vehicle is approaching when the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles.

13. A remote monitoring system comprising: Multiple vehicles, operating both autonomously and by remote control; and A remote monitoring device for remotely monitoring the plurality of vehicles, The plurality of vehicles respectively include: a position information acquisition unit that acquires position information indicating the position of the vehicle; a microphone that acquires sound data representing sounds surrounding the vehicle; and a communication unit, which transmits the position information and the sound data to the remote monitoring device, The remote monitoring device comprises: an acquisition unit that acquires a plurality of pieces of position information indicating the positions of the respective vehicles and a plurality of pieces of sound data indicating surrounding sounds of the respective vehicles; a detection unit, detecting the sirens of emergency vehicles respectively from the acquired plurality of sound data; an estimating unit that estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data when the siren sound is detected from each of the plurality of sound data; a determination unit that determines, from among the plurality of vehicles, a vehicle to which the emergency vehicle is approaching based on the estimated temporal change in the position of the emergency vehicle; and The notification unit notifies the vehicle that the emergency vehicle is approaching when the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles.

14. A device comprising: an acquisition unit that acquires a plurality of position information indicating the positions of each of the plurality of vehicles and a plurality of sound data indicating surrounding sounds of each of the plurality of vehicles; a detection unit, detecting the sirens of emergency vehicles respectively from the acquired plurality of sound data; an estimating unit that estimates the position of the emergency vehicle based on the plurality of position information of the plurality of vehicles and the plurality of sound data when the siren sound is detected from each of the plurality of sound data; a determination unit that determines, from among the plurality of vehicles, a vehicle to which the emergency vehicle is approaching based on the estimated temporal change in the position of the emergency vehicle; and The notification unit notifies the vehicle that the emergency vehicle is approaching when the vehicle that the emergency vehicle is approaching is identified from among the plurality of vehicles.

Images