Information processor, siren sound determination method and siren sound determination program

Abstract

To provide an information processor capable of determining whether siren sound is included in collected sound or not.SOLUTION: An information processor includes: a storage part for storing a learning model that has learned image data in which sound is converted into an image and teacher data in which information indicating whether a siren sound is included in the sound is annotated; a reception part for receiving input of sound; a conversion part for converting the sound into an image indicating the sound volume by lightness, converting the sound into an image of lightness corresponding to a center value of the sound volume, and generating image data; a pre-processing part for performing pre-processing on image data converted by the conversion part; a determination part for determining whether a siren sound is included in the sound received by the reception part, based on the image data after processing by the pre-processing part and the learning model; and a notification part for notifying an alert when the determination part determines that a siren sound is included.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, a siren sound determination method, and a siren sound determination program that determine whether or not a sound includes a siren sound. [Background technology]

[0002] The Road Traffic Act of Japan requires that when an emergency vehicle is passing, ordinary vehicles must move to the side of the road and stop temporarily so as not to impede the emergency vehicle. However, in some cases, the driver of the vehicle may miss the sound of the emergency vehicle's siren. Therefore, Patent Document 1 discloses an emergency vehicle safe passage device that identifies the siren sound of an emergency vehicle from sounds received by a microphone and has a display that displays a message notifying the approach of an emergency vehicle when the siren sound is detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-67587 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 has a problem in that, due to the configuration in which a siren sound is identified from the sound picked up by a microphone, if a sound similar to a siren is heard, it may be mistaken for a siren sound.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide an information processing device, a siren sound determination method, and a siren sound determination program that can provide other methods for detecting siren sounds from collected sounds. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present invention includes a memory unit that stores image data in which sound is converted into an image and a learning model that has learned teacher data annotated with information indicating whether the sound contains a siren sound, a reception unit that receives sound input, a conversion unit that converts the sound into an image that represents the volume of the sound in terms of brightness and converts the sound into an image with a brightness corresponding to the median volume to generate image data, a pre-processing unit that performs pre-processing on the image data converted by the conversion unit, a determination unit that determines whether the sound received by the reception unit contains a siren sound based on the image data after processing by the pre-processing unit and the learning model, and an alarm unit that issues an alert when the determination unit determines that the sound contains a siren sound.

[0007] In the information processing device according to one embodiment of the present invention, the conversion unit may correct the sound so that the median volume of the sound becomes a predetermined reference value, and then convert the sound into an image.

[0008] In the information processing device according to an embodiment of the present invention, the conversion section may perform a Fourier transform on a sound of a predetermined length of time, and convert the sound into image data indicating frequency components on a time axis.

[0009] In the information processing device according to an embodiment of the present invention, the preprocessing unit may execute, as the preprocessing, noise removal for cutting a predetermined frequency band from the image data.

[0010] In the information processing device according to an embodiment of the present invention, the preprocessing unit may remove frequencies other than the frequency of the siren sound as the preprocessing.

[0011] In an information processing device according to one embodiment of the present invention, the learning model is a learning model that has further learned the type of siren sound, and when the judgment unit determines that the sound includes a siren sound, it judges the type of the siren sound, and the notification unit may notify information indicating the type of siren sound together with an alert.

[0012] A method for determining whether a siren sound is included in an information processing device according to one embodiment of the present invention includes a reception step in which a computer that can access a memory unit that stores image data in which sound is converted into an image and a learning model that has learned teacher data annotated with information indicating whether the sound contains a siren sound, and the computer executes the following steps: a reception step in which the input of a sound is received; a conversion step in which the sound is converted into an image to generate image data; a preprocessing step in which preprocessing is performed on the image data converted by the conversion step; a determination step in which the sound received in the reception step determines whether the sound contains a siren sound based on the image data after processing by the preprocessing step and the learning model; and an alarm step in which an alert is issued if the determination step determines that the sound contains a siren sound.

[0013] A siren sound determination program for an information processing device according to one embodiment of the present invention provides a computer that can access a memory unit that stores image data in which sound has been converted into an image and a learning model that has learned teacher data annotated with information indicating whether the sound contains a siren sound, and the computer has the following functions: a reception function that accepts sound input, a conversion function that converts sound into an image and generates image data, a preprocessing function that performs preprocessing on the image data converted by the conversion function, a determination function that determines whether the sound received by the reception function contains a siren sound based on the image data after processing by the preprocessing function and the learning model, and an alarm function that issues an alert when the determination function determines that the sound contains a siren sound. Effect of the Invention

[0014] According to one embodiment of the present invention, it is possible to determine whether or not a collected sound includes a siren sound. [Brief description of the drawings]

[0015] [Figure 1] 1 is a schematic diagram of a siren sound determination system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention; [Diagram 3]1 is a flowchart illustrating an example of the operation of an information processing device according to an embodiment of the present invention, showing a method for generating a learning model. [Figure 4] 10 is a flowchart showing an example of operation of an information processing device according to an embodiment of the present invention, illustrating a determination process for determining whether a sound includes a siren sound. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an information processing device according to an embodiment of the invention according to the present disclosure (also referred to as the present invention) will be described with reference to the drawings.

[0017] <Summary> FIG. 1 is a diagram showing an overview of a system for determining the presence or absence of a siren sound according to this embodiment.

[0018] The information processing device 100 shown in FIG. 1 is a computer system that can determine whether or not a collected sound includes a siren sound and notify the result of the determination. As shown in FIG. 1, the information processing device 100 acquires sound data collected by a microphone 11 mounted on a vehicle 10. The microphone 11 may be provided outside the vehicle 10 or may be provided inside the vehicle 10. At this time, the collected sound data may include a siren sound emitted by an emergency vehicle 20 (for example, an ambulance, but is not limited to this). The information processing device 100 converts the acquired sound data into an image and determines whether or not the sound data includes a siren sound using the image. The information processing device 100 determines whether or not the sound data includes a siren sound using a learning model 141 that has learned image data 31 in which a sound including a siren sound has been converted into an image and image data 30 in which a sound not including a siren sound has been converted into an image. Then, when the information processing device 100 determines that the sound data includes a siren sound, it notifies the vehicle 10 of that effect. Upon receiving the notification, the vehicle 10 can move to the side of the road so as not to impede the passage of the emergency vehicle 20. In the case of the vehicle 10, if the windows are closed and music is being played, there is a possibility that the siren sound of the emergency vehicle 20 outside the vehicle 10 may be missed, but the information processing device 100 according to this embodiment can detect the siren sound and issue an alert to that effect. The information processing device 100 may be mounted on the vehicle 10, or may be a device outside the vehicle 10 and connected to the vehicle 10 via a network so as to be able to communicate with the vehicle 10.

[0019] The network between the vehicle 10 and the information processing device 100 includes a wireless network and a wired network. Specifically, for example, the network may be a wireless LAN (WLAN), a wide area network (WAN), a long term evolution (LTE), a mobile communication system such as a fourth generation communication (4G), a fifth generation communication (5G), or a sixth generation communication (6G) or later. The network is not limited to these examples, and may be, for example, a public switched telephone network (PSTN), Bluetooth (Bluetooth (registered trademark)), an optical line, an asymmetric digital subscriber line (ADSL), a satellite communication network, or the like. Furthermore, the network may include a wireless communication method for IoT, such as a narrow band IoT (NB-IoT) or an enhanced machine type communication (eMTC). The network may also include a communication method used in road-to-vehicle communication or vehicle-to-vehicle communication. The network may be a combination of these.

[0020] <Configuration> Fig. 2 is a block diagram showing a configuration example of the information processing device 100. As shown in Fig. 2, the information processing device 100 includes a communication unit 110, an input unit 120, a control unit 130, a storage unit 140, and an output unit 150. The information processing device 100 may be a computer including a processor and a memory, and may be realized by a server device, a PC, a tablet terminal, a notebook PC, a smartphone, or the like, but is not limited to these.

[0021] The communication unit 110 has a function of transmitting and receiving information to and from other devices through wired or wireless communication. The communication unit 110 may receive sound data from an external device (e.g., a traveling vehicle) and transmit the sound data to the control unit 130. The communication unit 110 may also transmit designated information to a designated destination in accordance with an instruction from the control unit 130. As an example, the communication unit 110 may transmit information indicating that a siren sound is included.

[0022] The input unit 120 has a function of receiving input from a user of the information processing device 100 and transmitting the input to the control unit 130. The input unit 120 can be realized by, for example, hardware keys provided in the information processing device 100, or soft keys such as a touch panel or touch keys. Note that the input to the input unit 120 may be a voice input, in which case the input unit 120 is realized by a microphone. The input unit 120 may be, for example, a microphone, and may collect surrounding sounds and transmit the collected sounds to the control unit 130. The collected surrounding sounds may include a siren sound.

[0023] The storage unit 140 is a recording medium that stores various programs and various data required for the operation of the information processing device 100. The storage unit 140 is realized by, for example, a hard disc drive (HDD), a solid state drive (SSD), a flash memory, etc., but is not limited to these. The storage unit 140 may be, for example, a cloud storage that the information processing device 100 can access via the communication unit 110.

[0024] The memory unit 140 may store a learning model 141.

[0025] The learning model 141 is a learning model that learns whether or not a siren sound is included in the image data obtained by converting sound into an image and the sound that is the source of the image data. In other words, the learning model 141 is a model that learns the relationship between the image data obtained by converting sound into an image and the presence or absence of a siren sound. The learning model 141 may be a model that is generated by learning using the image data obtained by converting sound into an image and data annotated with information indicating whether or not the sound that is the source of the image data contains a siren sound as teacher data. As an example, the learning model 141 may be a model that is generated by performing learning using a neural network, a support vector machine, or the like, but is not limited to these. Here, the image data obtained by converting sound into an image may be an image in which the sound is expressed by frequency components in the time axis direction, and the volume of the sound may be indicated by the brightness of the image. The image data obtained by converting the sound may be image data obtained by Fourier transforming the sound. In this case, it is preferable that the image data is image data obtained by converting the sound that is corrected based on the median value of the volume of the original sound into an image. In other words, the image data may be image data in which the median volume of the sound is calculated, the median value is corrected to become a predetermined reference value (the entire sound is shifted so that its median volume becomes equal to the reference value), and the corrected sound is Fourier transformed.

[0026] The learning model 141 may be created by the information processing device 100 or may be created by a device other than the information processing device 100.

[0027] The output unit 150 has a function of outputting instructed data according to an instruction from the control unit 130. The output unit 150 may output information indicating that a siren sound was included in the sound, for example. The output of information by the output unit 150 may be output by characters or images on a monitor (display device) attached to or connected to the information processing device 100, or may be output by sound from a speaker attached to or connected to the information processing device 100, or may be output by communication to an external device (for example, the vehicle 10) via the communication unit 110. The output unit 150 may output an alert indicating that a siren sound has been detected, for example, according to an instruction from the control unit 130.

[0028] The control unit 130 is a processor having a function of controlling each unit of the information processing device 100. The control unit 130 executes a program stored in the storage unit 140 to realize the functions to be performed by the information processing device 100. The control unit 130 realizes a reception unit 131, a conversion unit 132, a preprocessing unit 133, a determination unit 134, and a notification unit 135. The control unit 130 may also function as a learning unit 136.

[0029] The reception unit 131 receives sound data indicating a sound that is likely to include a siren sound. The reception unit 131 may receive sound data obtained by collecting sound from a device (e.g., the vehicle 10) external to the information processing device 100 via the communication unit 101, for example. Alternatively, the reception unit 131 may receive sound data via the input unit 102 serving as a microphone. The reception unit 131 transmits the received sound data to the conversion unit 132. The reception unit 131 may receive sound data of a predetermined length of time, or may receive sound data sequentially by streaming.

[0030] The conversion unit 132 converts the sound data received by the reception unit 131 into image data. As an example, the conversion unit 132 converts the sound data of a predetermined length of time into image data by applying a Fourier transform (or a fast Fourier transform). That is, the conversion unit 132 converts the sound data into image data with the time axis on the horizontal axis and the frequency axis on the vertical axis. The conversion unit 132 also converts the sound into image data that expresses the volume of the sound as the brightness of the image. The conversion unit 132 may generate an image that is brighter as the volume increases and darker as the volume decreases, but this may be reversed. Here, the brightness of the image may refer to the luminance of the image. Therefore, the conversion unit 132 converts the sound into image data in which data is plotted at frequencies where the sound exists at each timing in the time axis direction and the luminance of the plot is determined by the volume. Before converting the sound into image data, the conversion unit 132 may correct the sound according to the median value of the volume of the sound and then convert it into image data. That is, the conversion unit 132 specifies a median value from the volume of each time (for example, it may be every 0.1 seconds, but is not limited to this) for the transmitted sound data of a predetermined length of time. Then, the conversion unit 132 corrects the volume of the entire sound data so that the median value becomes the same as a predetermined reference value (volume) and then converts it into image data. That is, the conversion unit 132 corrects (shifts) the volume of the entire sound by the volume shifted so that the median value becomes the predetermined reference value. This allows the information processing device 100 to make the luminance value of the generated image data fall within a certain range as a whole, and can more accurately detect whether or not the sound contains a siren sound compared to a case where the sound is not corrected based on the median value. That is, when a sound collected in a noisy place and a sound collected in a quiet place are converted into image data using the same volume standard, there is a possibility that there will be a large difference in brightness (luminance) as the image. In this case, the brightness of the image may make it difficult to accurately determine whether or not a siren is present, but by correcting the sound in advance based on the median volume of the sound and then converting it into an image, it is possible to detect the presence or absence of a siren whether the sound is collected in a noisy place or a quiet place. Note that the image data may have a time axis on the vertical axis and a frequency axis on the horizontal axis.The predetermined time length may be, for example, 1 second, 5 seconds, or 0.5 seconds, and is arbitrary. This predetermined time length may be determined depending on what the information processing device 100 uses the detection of the siren sound for. In other words, if real-time performance is required, the predetermined time length is better to be short, and if not, it does not have to be short. However, the predetermined time length must be long enough to determine whether or not the sound data contains a siren sound. If the sound data accepted by the accepting unit 131 is longer than the predetermined time length, the converting unit 132 may divide the sound data in the time axis direction so as to have the predetermined time length, and then convert it into image data. By converting the sound data into image data by the converting unit 132, it becomes possible to determine the presence or absence of a siren sound, which is difficult to recognize in a sound state, as an image.

[0031] The preprocessing unit 133 performs preprocessing of the image data into which the conversion unit 132 converts the sound data. The preprocessing here refers to processing performed on the image data before determining whether or not the sound data includes a siren sound, and is to delete unnecessary data, which may generally be called noise removal. By performing noise removal, the information processing device 100 can make a more accurate determination. For example, the preprocessing unit 133 converts frequencies equal to or higher than a first threshold value and frequencies equal to or lower than a second threshold value into image data so that a predetermined frequency band can be extracted. Here, the predetermined frequency band may be the frequency band of a siren sound. For example, the law stipulates that the frequency of the siren sound of an ambulance is 770 Hz to 960 Hz. Also, for example, the frequency of the siren sound of a fire engine is stipulated to be 300 Hz to 850 Hz. Therefore, the preprocessing unit 133 may perform filtering processing of the image data according to the frequency of the siren sound to be detected. Specifically, if it is sufficient to detect only the siren sound of an ambulance as the siren sound, the preprocessing unit 133 may cut off frequencies below 760 Hz and above 970 Hz. If it is sufficient to detect both a fire engine and an ambulance as the siren sound, the preprocessing unit 133 may cut off frequencies below 300 Hz and above 960 Hz. The amount of error to be set for the frequency of the siren sound may be appropriate and may be set in advance by the user. This allows, for example, a portion corresponding to a sound that becomes noise in a frequency range higher than the siren sound to be cut from the image data, so that the information processing device 100 can more accurately determine the presence or absence of a siren sound. This allows, for example, a portion corresponding to a sound that becomes noise in a frequency range lower than the siren sound to be cut from the image data, so that the information processing device 100 can more accurately determine the presence or absence of a siren sound. The preprocessing unit 133 transmits the image data after the preprocessing to the determination unit 134.

[0032] Note that the pre-processing unit 133 may perform processing other than frequency cutting as necessary.

[0033] The determination unit 134 determines whether or not a siren sound is included in the image data preprocessed by the preprocessing unit 133. The determination unit 134 uses the transmitted image data and the learning model 141 to determine whether or not a siren sound is included in the image data. That is, the determination unit 134 determines whether or not a siren sound is included in the sound that is the basis of the image data. The determination unit 134 may input the transmitted image data to the learning model 141 and determine whether or not a siren sound is included in the image data. This determination may be made by outputting a percentage indicating the possibility that a siren sound is included, and in this case, if the percentage is equal to or greater than a predetermined value (for example, equal to or greater than 70%), it may be determined that a siren sound is included. The determination unit 134 transmits the determination result (presence or absence of a siren sound) to the notification unit 135. The determination unit 134 may transmit the above-mentioned percentage to the notification unit 135 as a determination result.

[0034] The notification unit 135 may notify the result of the determination by the determination unit 134. That is, when the determination unit 134 determines that the image data includes a siren sound, an alert indicating that fact may be issued. As an example, the notification unit 135 may cause the output unit 150 to output (display) information indicating that the sound received by the reception unit 131 includes a siren sound on a monitor connected to the information processing device 100 by using characters or images. Alternatively, the notification unit 135 may cause the output unit 150 to output information indicating that the siren sound was included by using voice to a speaker connected to the information processing device 100. Alternatively, the notification unit 135 may cause the output unit 150 to output a stop instruction to the vehicle as information indicating that the siren sound was included to the output unit 150. When the sound data received by the reception unit 131 is a sound collected by the vehicle 10 and the vehicle 10 is an autonomous vehicle (which may be a driving assistance vehicle (a vehicle with partial automatic control)), a stop instruction is output to the vehicle 10 when an approaching emergency vehicle is detected, thereby stopping the autonomous vehicle and allowing the emergency vehicle to easily pass. The stop instruction may be an instruction to prompt the autonomous vehicle to stop by the side of the road on which the autonomous vehicle is traveling. The stop instruction may be a command capable of directly controlling the autonomous vehicle, or may be an instruction to have the autonomous vehicle interpret the stop instruction received and perform autonomous driving.

[0035] In addition, if it is determined that the image data does not contain a siren sound, the notification unit 135 may not notify anything, or may notify information indicating that the siren sound was not included.

[0036] The learning unit 136 generates the learning model 141. The learning unit 136 learns, as teacher data, image data obtained by converting sound data into an image and information annotated with information indicating whether or not the sound data contains a siren sound, to generate the learning model 141. The learning unit 136 can generate the learning model 141 capable of estimating whether or not the image data obtained by converting sound data into an image contains a siren sound by learning a plurality of teacher data. As an example, the learning unit 136 may perform learning using a neural network or a support vector machine, but is not limited to these. The learning unit 136 may store the generated learning model 141 in the storage unit 140. The learning unit 136 may also have a function of accepting input of new teacher data, performing re-learning, and updating the learning model 141.

[0037] The above is the description of the configuration of the information processing device 100.

[0038] <Operation> Now, the operation of the information processing device 100 will be described.

[0039] First, the operation for generating the learning model 141 will be described with reference to FIG.

[0040] As shown in FIG. 3, the learning unit 136 of the control unit 130 receives image data obtained by converting sound into an image and teacher data annotated with information indicating whether or not the sound includes a siren sound. This teacher data may be data received from an external device via the communication unit 110, or may be data acquired from an external memory or the like connected to the information processing device 100 via the input unit 120, or may be teacher data generated by the control unit 130 after converting a sound received by the information processing device 100 into image data by the conversion unit 132 and receiving information from the user of the information processing device 100 indicating whether or not a siren sound is included based on whether or not a siren sound is being sounded. The image data obtained by converting a sound included in the teacher data into an image may be image data in which the actual sound is converted into frequency components in the time axis direction and the volume is expressed by brightness (luminance). The sound may or may not include a siren sound. In addition, the image data may be converted from sound that has been corrected based on the median volume of the sound, or may be converted from sound that has been corrected by shifting the overall volume of the sound so that the median volume of the sound is the same as a reference value.

[0041] The learning unit 136 receives multiple pieces of teacher data and learns the relationship between image data in which sound is converted into an image and a siren sound. That is, the learning unit 136 learns whether or not the image data contains a siren sound. The learning unit 136 learns the multiple pieces of teacher data and generates a learning model 141.

[0042] The learning unit 136 stores the generated learning model 141 in the storage unit 140 (step S303), and ends the process.

[0043] Next, the process of determining whether or not a sound includes a siren sound will be described with reference to the flowchart shown in FIG.

[0044] The reception unit 131 of the information processing device 100 receives a sound input (step S401). The sound may or may not include a siren sound. The reception unit 131 may receive the sound input by receiving sound data from an external device via the communication unit 110, or may receive the sound input from the microphone of the input unit 120. The reception unit 131 transmits the received sound to the conversion unit 132.

[0045] When the conversion unit 132 receives the sound data from the reception unit 131, the conversion unit 132 samples the volume of the sound data every unit time (for example, every 0.1 seconds) and identifies the median value. The conversion unit 132 performs a correction to shift the overall volume of the sound so that the identified median value becomes a predetermined reference value. In other words, the conversion unit 132 performs a correction to shift the overall volume of the sound by the amount of the shift so that the median value becomes equal to the reference value. Thereafter, the conversion unit 132 applies FFT (Fast Fourier Transfer) processing to the sound data to convert it into image data having a time axis and a frequency axis (step S402). The conversion unit 132 transmits the image data obtained by converting the sound data to the pre-processing unit 133.

[0046] When the image data is transmitted, the pre-processing unit 133 performs pre-processing on the image data (step S403). That is, the pre-processing unit 133 cuts out data of frequencies other than a predetermined frequency band. That is, the pre-processing unit 133 removes noise, which becomes unnecessary data, from the transmitted image data. The unnecessary data may be sound data of frequencies other than the frequency band of the siren sound. The pre-processing unit 133 transmits the image data after noise removal to the determination unit 134.

[0047] When the determination unit 134 receives the image data after noise removal, it determines whether or not the image data contains a siren sound (step S404). That is, the determination unit 134 inputs the transmitted image data to the learning model 141 and determines whether or not the image data contains a siren sound. Strictly speaking, whether or not the image data contains a siren sound means whether or not the sound data that is the source of the image data contains a siren sound.

[0048] When the determination unit 134 determines that the image data includes a siren sound (YES in step S404), the notification unit 135 notifies the user that the image data includes a siren sound (step S405) and ends the process. The notification may be realized by at least one of displaying information on the monitor of the information processing device 100, outputting audio from the speaker of the information processing device 100, and transmitting information to an external device via the communication unit 110.

[0049] On the other hand, if the determination unit 134 determines that the image data does not include a siren sound (NO in step S404), the information processing device 100 ends the process. At this time, the notification unit 135 may notify information indicating that the sound does not include a siren sound.

[0050] The operation of the information processing device 100 has been described above.

[0051] <Summary> As described above, the information processing device 100 can convert sound into an image and determine whether or not a sound contains a siren sound by using the learning model 141 that has learned the relationship between image data and a siren sound. By converting into an image instead of sound, the presence or absence of a siren sound can be determined by a different approach from the conventional approach. The information processing device 100 determines the presence or absence of a siren sound by converting the sound into image data in which the presence of the sound is expressed by frequency components in the time axis direction and the volume of the sound is expressed by the brightness of the image (pixel brightness). At this time, the information processing device 100 corrects the sound so that the median value of the sound matches a reference value before converting the sound into image data according to the volume of the collected sound. Therefore, the brightness of the image data after conversion can be made uniformly the same, so that the information processing device 100 can accurately determine the presence or absence of a siren sound regardless of the place where the sound is collected (regardless of whether it is a quiet place or a noisy place). By being able to determine the presence or absence of a siren sound, for example, the information processing device 100 can instruct an autonomous vehicle to pull over to the side of the road and stop temporarily when a siren sound is detected.

[0052] <Supplementary Information> It goes without saying that the information processing device 100 according to the above embodiment is not limited to the above embodiment, and may be realized by other methods. Various modified examples will be described below.

[0053] (1) In the above embodiment, the learning model 141 is a model that has learned image data including a siren sound and image data not including a siren sound, but the siren sound may include various types of emergency vehicles, or may include only the siren sound of one type of emergency vehicle. Also, the learning model 141 may be prepared as a learning model corresponding to each emergency vehicle. That is, a learning model corresponding to the type of emergency vehicle may be prepared individually.

[0054] In addition, when the learning model 141 is a model that has learned image data including siren sounds of various emergency vehicles, the learning model 141 may be a model that can estimate which emergency vehicle the siren sound belongs to. In this case, the learning model 141 is a model that has learned image data in which sounds including a siren sound are converted into images, and information in which information indicating which emergency vehicle the siren sound belongs to is annotated as teacher data. As a result, when the learning model 141 inputs image data in which sounds are converted into images, if a siren sound is included, the learning model 141 can estimate and output information indicating that a siren sound is included and which emergency vehicle the siren sound belongs to. Therefore, the information processing device 100 can prompt a user, a vehicle, or the like to take action according to the type of the estimated siren sound.

[0055] (2) In the above embodiment, the siren sound is the siren sound of an emergency vehicle, but this is not limited to the siren sound of an emergency vehicle. The siren sound may be a siren sound of a vehicle other than an emergency vehicle, such as an emergency earthquake warning alarm.

[0056] (3) In the above embodiment, the information processing device 100 may be configured not to issue a notification when a siren sound is detected once and the same siren sound is detected again. For example, if a siren sound is detected from another image data within a predetermined time (for example, 30 seconds, but not limited to this) after a siren sound is detected from the image data once, the information processing device 100 may be configured not to issue a notification. Alternatively, if a siren sound is detected once and the siren sound continues to be detected in temporally consecutive image data, the information processing device 100 may be configured not to issue a notification based on the image data after the image data in which the siren sound was detected. Furthermore, when the information processing device 100 detects a siren sound, the notification process of notifying the information processing device 100 may be performed with the permission of the user of the information processing device 100. Furthermore, when the information processing device 100 detects a siren sound and after making the notification, the information processing device 100 may receive an instruction to stop the notification based on an input from the user and make an instruction to stop the notification. That is, when a siren sound is detected continuously from a plurality of consecutive image data, the receiving side may be bothered if the presence of a siren sound is continuously notified, but this configuration can suppress this. In addition, when the siren sound notification has been stopped, if a new siren sound is detected from the image data after a predetermined time (for example, 15 seconds, but is not limited to this) has elapsed since the siren sound could no longer be detected from the image data, the information processing device 100 may make a new notification.

[0057] (4) In the above embodiment, information processing device 100 determines whether or not a siren sound is included using image data that has been preprocessed by preprocessing unit 133. However, determination unit 134 may be configured to determine whether or not a siren sound is included using image data that has not been preprocessed.

[0058] (5) In the above embodiment, the detection of the siren sound is described as being performed by the information processing device 100 mounted on the vehicle 10 or an information processing device 100 external to the vehicle 10, but this is not limited to the above. The detection of the siren sound may be realized by an application having the same function as the information processing device 100, and may be realized, for example, by the smartphone of the driver of the vehicle 10 executing the application.

[0059] (6) Each functional unit of the information processing device 100 may be realized by a logic circuit (hardware) or a dedicated circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), or may be realized by software using a CPU (Central Processing Unit). Each functional unit may be realized by one or more integrated circuits, and the functions of multiple functional units may be realized by one integrated circuit. Furthermore, the above-mentioned information processing device 100 may be realized by multiple information processing device computers, and depending on the function, it may be realized by calling an external platform, etc., using an API (Application Programming Interface), etc.

[0060] When each functional unit of the information processing device 100 is realized by software, the information processing device 100 includes a CPU that executes instructions of a program that is software that realizes each function, a ROM (Read Only Memory) or a storage device (these are referred to as "recording media") in which the program and various data are recorded so as to be readable by a computer (or CPU), and a RAM (Random Access Memory) in which the program is expanded. The object of the present invention is achieved by the computer (or CPU) reading and executing the program from the recording medium. That is, the information processing device 100 according to the present invention functions as each of the above-mentioned components by the CPU executing the program loaded on the RAM. As the recording medium, a "non-transient tangible medium", for example, a semiconductor memory, a programmable logic circuit, etc. can be used. The program may also be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) capable of transmitting the program. The present invention may also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[0061] The above program may be implemented using, for example, a scripting language such as ActionScript, JavaScript (registered trademark), Python, or Ruby, an object-oriented programming language such as C language, C++, C#, Objective-C, Swift, or Java (registered trademark), or a markup language such as HTML5. Furthermore, the term "section, module, or unit" in the claims may be read as "means" or "circuit." For example, a communication section may be read as a communication means or a communication circuit.

[0062] Furthermore, the program of the present disclosure may be provided to the information processing device 100 via any transmission medium (such as a communication network or broadcast waves) capable of transmitting the program.

[0063] (7) As mentioned above, the present invention has been described with respect to a specific embodiment, but it is to be noted that the present invention is not limited to the disclosed embodiment. Also, the scope of the claims may include combinations of the claims, and may include modifications and similar configurations of the claims.

[0064] According to each aspect of the present disclosure described above, safe and comfortable operation of autonomous vehicles will be possible, which will contribute to achieving Goal 11 of the Sustainable Development Goals (SDGs), "Sustainable cities and communities." [Explanation of symbols]

[0065] 100 Information processing device 110 Communications Department 120 Input section 130 Control section 131 Reception 132 Conversion unit 133 Pretreatment section 134 Judgment section 135 Information Department 136 Learning Department 140 Storage section 150 Output section

Claims

1. A storage unit that stores image data obtained by converting a sound into an image and a learning model that has learned teacher data annotated with information indicating whether the sound includes a siren sound; A reception unit that receives an input of sound; a conversion unit that converts the sound into an image that represents the volume of the sound by brightness, the conversion unit converting the sound into an image with brightness corresponding to a median value of the volume to generate image data; a pre-processing unit that performs pre-processing on the image data converted by the conversion unit; a determination unit that determines whether or not the sound received by the reception unit includes a siren sound based on the image data processed by the preprocessing unit and the learning model; a notification unit that issues an alert when the determination unit determines that the sound includes a siren sound; An information processing device comprising:

2. The conversion unit corrects the sound so that a median value of the volume of the sound becomes a predetermined reference value, and then converts the sound into an image.

2. The information processing apparatus according to claim 1,

3. The conversion unit converts a sound of a predetermined length of time into image data indicating frequency components on a time axis by performing a Fourier transform.

3. The information processing apparatus according to claim 2.

4. The preprocessing unit performs noise removal by cutting a predetermined frequency band from the image data as the preprocessing.

4. The information processing apparatus according to claim 3.

5. The preprocessing unit removes frequencies other than the frequency of a siren sound as the preprocessing.

5. The information processing apparatus according to claim 4.

6. The learning model is a learning model that further learns a type of siren sound, When the determination unit determines that the siren sound is included, the determination unit further determines a type of the siren sound, The notification unit notifies information indicating a type of siren sound together with the alert.

2. The information processing apparatus according to claim 1,

7. A computer that can access a storage unit that stores a learning model that has learned image data obtained by converting a sound into an image and teacher data annotated with information indicating whether the sound includes a siren sound, a receiving step of receiving an input of sound; A conversion step of converting the sound into an image that represents the volume of the sound by brightness, and converting the sound into an image with brightness corresponding to a median value of the volume to generate image data; a pre-processing step of performing pre-processing on the image data converted by the conversion step; a determination step of determining whether or not the sound received in the reception step includes a siren sound based on the image data processed in the preprocessing step and the learning model; a notification step of notifying an alert when the determination step determines that the sound includes a siren sound; A method for determining whether a siren is heard.

8. A computer that can access a storage unit that stores a learning model that has learned image data obtained by converting a sound into an image and teacher data annotated with information indicating whether the sound includes a siren sound, A reception function for receiving sound input; A conversion function for converting the sound into an image that represents the volume of the sound by brightness, and converting the image into an image with brightness corresponding to the median value of the volume to generate image data; a pre-processing function for performing pre-processing on the image data converted by the conversion function; A determination function that determines whether or not the sound received by the reception function includes a siren sound based on the image data processed by the preprocessing function and the learning model; a notification function that issues an alert when the determination function determines that the sound includes a siren sound; A siren sound detection program that makes this possible.

Images