Sound environment management device, sound environment management method, and program

Abstract

When making sounds generated outside a mobile device audible inside the device, the comfort level of the person riding in the device is less likely to be reduced. [Solution] The sound environment management device comprises an external sound analysis unit and a setting adjustment unit. The external sound analysis unit analyzes external sound data indicating sounds outside the mobile body. The setting adjustment unit adjusts the settings related to the noise cancellation function inside the mobile body for each of several predetermined areas in the mobile body, in response to the detection of a predetermined sound through the analysis of the external sound data.

Description

[Technical Field]

[0001] The present invention relates to a sound environment management apparatus, a sound environment management method, and a program. [Background Art]

[0002] Various sounds are generated outside a mobile body such as a vehicle, and can reach the interior of the mobile body. An example of a technology for controlling how sound is heard inside a mobile body is disclosed in, for example, Patent Document 1 and Patent Document 2 below.

[0003] The following Patent Document 1 discloses a system in which, when a required sound is superimposed on noise outside a vehicle and arrives inside the vehicle, required sound components of predetermined types that a passenger should actively hear inside the vehicle are extracted from the sound outside the vehicle, and the extracted required outside sound is relatively emphasized against other noise and taken into the vehicle interior.

[0004] Further, the following Patent Document 2 discloses a system that determines whether to cancel noise from a noise source generated outside the vehicle, and controls a speaker inside the vehicle to cancel the noise at the timing when the vehicle reaches the noise source. [Prior Art Documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-23587 [Patent Document 2] Japanese Patent Application Laid-Open No. 2019-128490 [Summary of the Invention] [Problem to be Solved by the Invention]

[0006] In the technologies described in Patent Documents 1 and 2, when sounds generated outside a mobile vehicle are made audible inside the vehicle, they are audible to all persons riding in the vehicle. Here, even if the sounds generated outside the vehicle are important to the person operating the vehicle, they are often noise (unnecessary information) that reduces comfort for other persons. One example of the object of the present invention is to make it less likely to reduce the comfort of persons riding in a mobile vehicle when sounds generated outside the vehicle are made audible inside the vehicle. [Means for solving the problem]

[0007] The first invention in this disclosure is, An external sound analysis unit analyzes external sound data that indicates sounds from outside the moving object, A setting adjustment unit adjusts the settings related to the noise cancellation function inside the mobile body for each of a predetermined number of regions in the mobile body, in response to the detection of a predetermined sound by the analysis of the external sound data. This is a sound environment management device equipped with [specific features / features].

[0008] The second invention in this disclosure is, Computers External sound data indicating sounds from outside the moving object is analyzed, In response to the detection of a predetermined sound through the analysis of the external sound data, the settings related to the noise cancellation function inside the mobile body are adjusted for each of the predetermined multiple regions within the mobile body. This is a sound environment adjustment method that includes the following.

[0009] The third invention in this disclosure is, On the computer, External sound data indicating sounds from outside the moving object is analyzed, In response to the detection of a predetermined sound through the analysis of the external sound data, the settings related to the noise cancellation function inside the mobile body are adjusted for each of the predetermined multiple regions within the mobile body. This is a program that performs an action that includes [a certain action]. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the usage environment for the sound environment management device described in this disclosure. [Figure 2] This figure shows a first example of the functional configuration of the sound environment management device in this disclosure. [Figure 3] This figure shows an example of the computer hardware configuration for realizing the sound environment management device described in this disclosure. [Figure 4] This flowchart shows a first example of processing performed by the sound environment management device in this disclosure. [Figure 5] This figure shows a second example of the functional configuration of the sound environment management device in this disclosure. [Figure 6] This flowchart shows a second example of processing performed by the sound environment management device in this disclosure. [Modes for carrying out the invention]

[0011] The embodiments relating to this disclosure will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate. Unless otherwise specified, in each block diagram, each block represents a functional unit configuration, not a hardware unit configuration. Furthermore, when elements such as blocks in the diagrams are linked by arrows, the direction of the arrows is merely to make the flow of information easier to understand. In this case, unless otherwise specified, the direction of the arrows does not limit the direction of communication (one-way communication / two-way communication).

[0012] ·overview Figure 1 shows an example of the usage environment of the sound environment management device in this disclosure. The sound environment management device 10 illustrated in this figure is used together with a mobile body 20. The sound environment management device 10 may be located inside the mobile body 20 or outside the mobile body 20.

[0013] The moving object 20 is, for example, a vehicle, a ship, or an airplane. When the moving object 20 is a vehicle, the moving object 20 may be, for example, a private car, a taxi, a bus, or a railway car. A plurality of people can board the moving object 20. A plurality of regions 200 are set inside the moving object 20. Different people can be located in the regions 200. In the example shown in the figure, one person can be located in one region 200. However, a plurality of people may be located in at least one region 200.

[0014] At least one seat 210 and an output device 220 are provided in one region 200. When a plurality of seats 210 are provided in one region 200, output devices 220 respectively corresponding to the plurality of seats 210 may be provided, or one output device 220 may be provided for the plurality of seats 210 (that is, for one region 200).

[0015] The output device 220 outputs content. For example, the output device 220 includes a speaker and outputs audio. The output device 220 may further include a display. In this case, the output device 220 further outputs images and videos. Hereinafter, images and videos are referred to as "videos". Note that the speaker and the display may be arranged at different positions from each other. For example, the speaker may be provided on the headrest of the seat 210, and the display may be provided in front of the seat 210.

[0016] Further, the moving object 20 includes a sound collection device 230 for collecting sound generated externally. In the example of the figure, the moving object 20 includes one sound collection device 230 at each of four corners, for a total of four sound collection devices 230. However, the number and arrangement of the sound collection devices 230 are not limited to the example shown in this figure. The sound collection device 230 converts detected sound (air vibrations) into an analog or digital electrical signal (sound data) and outputs it. In the present disclosure, the sound data generated by the sound collection device 230 is also referred to as "external sound data".

[0017] The external sound data output from the sound collection device 230 is used to implement a noise canceling function inside the moving object 20. For example, external sound (i.e., noise) is canceled by combining data of opposite phase with the external sound data obtained from the sound collection device 230. As will be described later, the sound environment management device 10 according to the present disclosure has a function of adjusting settings related to the noise canceling function for each region based on an analysis result of the external sound data.

[0018] - First Embodiment FIG. 2 is a diagram showing a first example of the functional configuration of the sound environment management device according to the present disclosure. The sound environment management device 10 in this figure includes an external sound analysis unit 110 and a setting adjustment unit 120.

[0019] The external sound analysis unit 110 analyzes external sound data. As described above, the external sound data is data representing sound outside the moving object 20 generated by the sound collection device 230 provided in the moving object 20. The external sound analysis unit 110 detects that a predetermined sound is generated outside the moving object 20 by analyzing the external sound data.

[0020] In response to detection of a predetermined sound through analysis of the external sound data, the setting adjustment unit 120 adjusts settings related to the noise canceling function inside the moving object 20 for each predetermined region 200 predefined in the moving object 20.

[0021] As an example, the setting adjustment unit 120 adjusts the noise cancellation function settings in the first region (hereinafter also referred to as the "first region") so that a predetermined sound can be heard in the region 200 corresponding to the driver's seat of the mobile body 20, out of a plurality of regions 200. In the example in Figure 1, the region corresponding to the driver's seat is the upper right region 200. In the example in Figure 1, the setting adjustment unit 120 recognizes the upper right region 200 as the first region and adjusts the noise cancellation function settings in the first region so that a predetermined sound can be heard in the first region. For example, the setting adjustment unit 120 adjusts the settings so that the noise cancellation function is not applied in the first region (turns off the noise cancellation function in the first region). Alternatively, the setting adjustment unit 120 may generate noise cancellation data for the first region by subtracting components related to a predetermined sound from external sound data and then inverting the phase. Through such adjustments, the predetermined sound can be heard in the first region without being canceled.

[0022] Here, a person located in a seat not involved in driving the mobile vehicle 20 (for example, the seat 210 located at the rear in the figure) may have little need to hear a specific sound. Therefore, the setting adjustment unit 120 may adjust the noise cancellation function settings in at least one of the other areas other than the first area corresponding to the driver's seat so that the specific sound cannot be heard in that other area. For example, if the setting adjustment unit 120 detects a person in an area other than the driver's seat using the in-vehicle video or weight sensors provided on each seat 210, it turns on the noise cancellation function in the output device 220 corresponding to the area where the person was detected. In this case, the setting adjustment unit 120 generates noise cancellation data to cancel sounds (noise) occurring outside the mobile vehicle 20 by inverting the phase of the sound data collected by the sound collection device 230. By combining the generated noise cancellation data with the external sound data, external sounds, including the specific sound, are canceled in the target area.

[0023] Figure 3 shows an example of the computer hardware configuration for realizing the sound environment management device described in this disclosure.

[0024] The computer 1000 illustrated in this figure has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0025] The computer 1000 illustrated in this figure has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0026] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0027] Processor 1020 includes the Central Processing Unit (CPU) and the Graphics Processing Unit (GPU), among others.

[0028] Memory 1030 is a type of main memory implemented using Random Access Memory (RAM), etc.

[0029] The storage device 1040 is an auxiliary storage device implemented as a Hard Disk Drive (HDD), Solid State Drive (SSD), memory card, or Read Only Memory (ROM). The storage device 1040 stores at least a program module that implements the functions of the sound environment management device 10 described above (external sound analysis unit 110 and setting adjustment unit 120).

[0030] The processor 1020 realizes the functions corresponding to the program modules by loading the program modules read from the storage device 1040 onto the memory 1030 and executing them. For example, the processor 1020 realizes the functions of the external sound analysis unit 110 as described in this disclosure by reading the program module corresponding to the external sound analysis unit 110 onto the memory 1030 and executing it. Similarly, the processor 1020 realizes the functions of the setting adjustment unit 120 as described in this disclosure by reading the program module corresponding to the setting adjustment unit 120 onto the memory 1030 and executing it. This operation of the processor 1020 is common to all embodiments included in this disclosure.

[0031] The program module described above may be recorded on a recording medium other than the storage device 1040. The recording medium for recording the program module includes any medium usable by a non-temporary, tangible computer 1000. The recording medium for recording the program module may have program code embedded in it that can be read by the computer 1000 (processor 1020).

[0032] The input / output interface 1050 is an interface for connecting the computer 1000 (sound environment management device 10) to various devices. For example, the computer 1000 (sound environment management device 10) is connected to the output device 220 or sound collection device 230 described above via the input / output interface 1050.

[0033] The network interface 1060 is an interface for connecting the computer 1000 (sound environment management device 10) to a network. This network includes, for example, a Local Area Network (LAN) or a Wide Area Network (WAN). The network interface 1060 supports various wireless or wired communication standards. The computer 1000 (sound environment management device 10) establishes communication with other devices on the network via the network interface 1060. For example, the computer 1000 (sound environment management device 10) may be communicably connected to the output device 220 or sound collection device 230 described above via the network interface 1060.

[0034] An example of the operation of the sound environment management device described in this disclosure will be explained below with reference to Figure 4.

[0035] Figure 4 is a flowchart showing a first example of processing performed by the sound environment management device in this disclosure.

[0036] The external sound analysis unit 110 analyzes external sound data that indicates sounds from outside the mobile body 20 (step S102). For example, the external sound analysis unit 110 detects a predetermined sound from the external sound data using a machine learning model that has been trained to separate various sounds contained in a complex sound and detect a predetermined sound.

[0037] The setting adjustment unit 120 determines whether a predetermined sound was detected from the external sound data in step S102 (step S104). For example, the external sound analysis unit 110 analyzes the external sound data and notifies the setting adjustment unit 120 of information indicating whether or not a predetermined sound was separated and detected from the external sound data. Based on the information notified by the external sound analysis unit 110, the setting adjustment unit 120 can determine whether or not a predetermined sound was detected from the external sound data.

[0038] Here, the specified sound includes any sound that should be conveyed to a person inside the mobile body 20, in particular to a person capable of operating the mobile body 20. The specified sound is not particularly limited, but includes at least one of the following: the sound of an emergency vehicle siren, the sound of a railroad crossing warning, the sound of a pedestrian signal, voices related to traffic guidance (such as announcements in a parking lot, the voice of a traffic control officer at a construction site, etc.), and sounds originating from other mobile bodies present around the mobile body 20 (such as engine noise, turn signal sounds, road noise generated while driving, etc.).

[0039] If a predetermined sound is detected from the external sound data (step S104: YES), the setting adjustment unit 120 adjusts the noise cancellation function settings for each region 200 inside the mobile body 20 (step S106). For example, the setting adjustment unit 120 adjusts the noise cancellation function settings for the first region (the region corresponding to the driver's seat) so that a predetermined sound can be heard in that region. The setting adjustment unit 120 also adjusts the noise cancellation function settings for other regions other than the first region so that a predetermined sound cannot be heard in those other regions.

[0040] On the other hand, if no predetermined sound is detected from the external sound data (step S104: NO), the process in step S106 described above is not performed. In this case, the setting adjustment unit 120 maintains the settings of the noise-canceling function applied in each of the multiple regions 200 inside the mobile body 20 (step S108).

[0041] As described above, according to the sound environment management device in this disclosure, when a predetermined sound is generated outside the mobile body 20, adjustments are made for each of the multiple regions 200 set inside the mobile body 20. For example, the predetermined sound becomes audible in the region where a person who needs to hear the predetermined sound is located, such as the region corresponding to the driver's seat, and the predetermined sound becomes inaudible in other regions (regions where people who do not need to hear the predetermined sound are located). In this way, the comfort of people located in those other regions is prevented from being reduced by the predetermined sound. As a result, when a sound generated outside the mobile body is made audible inside the mobile body, it is possible to reduce the comfort of people riding in the mobile body.

[0042] <Variation> In some cases, it may not be desirable to notify only the first area (the area corresponding to the driver's seat) of a predetermined sound. For example, when the driver's workload is relatively high, such as when operating the mobile device 20 in a busy or unfamiliar area, it may be preferable for occupants other than the driver to assist with driving, such as by helping with safety checks.

[0043] To address the cases described above, the setting adjustment unit 120 may be configured to estimate the workload of a person in the first area (the area corresponding to the driver's seat) and adjust the noise cancellation function settings in each area according to that workload. For example, the setting adjustment unit 120 can estimate the workload of the driver based on the results of analyzing the driver's facial image and identifying eye movements and facial expressions. If the setting adjustment unit 120 determines that the estimated workload of a person in the first area is above a predetermined threshold, it may adjust the noise cancellation function settings in another area adjacent to the first area (hereinafter referred to as the "second area") so that a predetermined sound can be heard in that second area. Here, the second area is, for example, the area corresponding to the passenger seat in a private car. In this way, tasks such as identifying the source of a predetermined sound can be delegated to another person other than the driver (a person supporting the driver), preventing the driver's workload from becoming too heavy.

[0044] In addition, the setting adjustment unit 120 may identify the relative direction of a predetermined sound source and adjust the noise cancellation function settings for each region of the mobile body 20 so that the predetermined sound can be heard in the region corresponding to the identified direction. For example, if multiple sound-collecting devices 230 are arranged on the mobile body 20 as illustrated in Figure 1, the setting adjustment unit 120 can estimate the relative direction of a predetermined sound source by comparing the audio data output from each sound-collecting device 230. The setting adjustment unit 120 may then adjust the noise cancellation function settings so that the predetermined sound can be heard in the region 200 closest to the estimated direction. In this case, the setting adjustment unit 120 targets only the region 200 where a person has been detected by the in-vehicle camera footage or weight sensors provided for each seat. By doing this, when a predetermined sound is detected, the predetermined sound can be heard by a person who can easily identify the source of the predetermined sound. As a result, for example, even if the source of a predetermined sound is located in a direction that is difficult to see from the driver's seat, a person who can easily see that direction can identify the source and appropriately assist the driver.

[0045] • Second embodiment Figure 5 shows a second example of the functional configuration of the sound environment management device in this disclosure. The sound environment management device 10 illustrated in this figure further includes an attribute information acquisition unit 130 in addition to the configuration illustrated in Figure 2.

[0046] The attribute information acquisition unit 130 acquires information indicating the attributes of the person located in each of the multiple regions 200 inside the mobile body 20 (hereinafter also referred to as "attribute information"). For example, the attribute information acquisition unit 130 acquires and analyzes video footage from an in-vehicle camera (not shown) that films the inside of the vehicle, thereby acquiring attributes such as the presence or absence of a person and the age group of the person for each region 200 (or each seat 210).

[0047] In the example shown in this figure, the setting adjustment unit 120 further uses attribute information acquired for each of the multiple regions 200 (or each seat 210) to adjust the noise cancellation function settings for each of the multiple regions 200.

[0048] Figure 6 is a flowchart illustrating a second example of processing performed by the sound environment management device in this disclosure. The flowchart illustrated in this figure is the same as the flowchart illustrated in Figure 4, except that it further includes steps S202 to S206. The following will mainly describe the steps that differ from the flowchart illustrated in Figure 4.

[0049] If the external sound analysis unit 110 analyzes the external sound data and detects a predetermined sound from the external sound data (step S104: YES), the attribute information acquisition unit 130 identifies the area where the person is located based on the video inside the moving body 20 (step S202).

[0050] Then, the attribute information acquisition unit 130 acquires information indicating the attributes of the detected person based on the video footage of that person (step S204). As an example, the attribute information acquisition unit 130 acquires information indicating the age group of the person (child / adult) or the behavioral state of the person (sleeping / awake) as attribute information.

[0051] The setting adjustment unit 120 then further adjusts the noise cancellation function settings in each of the multiple areas 200 using the acquired attribute information (step S206). For example, based on attribute information about the people located in each area, the setting adjustment unit 120 identifies people who have little need to hear a predetermined sound (people who are not suitable for assisting the driver or for whom it is difficult to provide assistance), such as infants or people who are sleeping. The setting adjustment unit 120 then adjusts the noise cancellation function settings in the areas where the people identified as "people who have little need to hear a predetermined sound" are located so that the predetermined sound cannot be heard. On the other hand, the setting adjustment unit 120 adjusts the noise cancellation function settings in the areas where other people, i.e., the driver and / or people who can assist the driver are located, so that the predetermined sound can be heard.

[0052] As described above, according to the second example of the sound environment management device in this disclosure, the system takes into account the attributes of the people located in each of the multiple areas set up inside the mobile body, and adjusts the system so that external sounds, including the predetermined sound, are not audible in the area where a person who has little need to hear the predetermined sound is located. This prevents the comfort level from being unnecessarily reduced in the area where a person who has little need to hear the predetermined sound is located.

[0053] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0054] Furthermore, although the flowchart used in the above description shows multiple steps (processes) in order, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments can be combined to the extent that their contents do not conflict. [Explanation of symbols]

[0055] 10 Sound environment management device 1000 computers 1010 Bus 1020 processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces 110 External sound analysis unit 120 Setting adjustment section 130 Attribute information acquisition section 20 Mobile Units 200 areas 210 seats 220 Output Devices 230 Sound Acquisition Devices

Claims

1. An external sound analysis unit analyzes external sound data that indicates sounds from outside the moving object, A setting adjustment unit adjusts the settings related to the noise cancellation function inside the mobile body for each of a predetermined number of regions in the mobile body, in response to the detection of a predetermined sound by the analysis of the external sound data. A sound environment management device equipped with the following features.

2. The setting adjustment unit adjusts the settings in the first region so that the predetermined sound can be heard in the first region corresponding to the driver's seat of the mobile body among the plurality of regions. The sound environment management device according to claim 1.

3. The setting adjustment unit adjusts the settings so that the predetermined sound cannot be heard in at least one area other than the driver's seat. The sound environment management device according to claim 2.

4. The aforementioned setting adjustment unit is We estimate the workload of the person in the first region, If the size of the workload is determined to be above a predetermined threshold, the settings in the second region are adjusted so that the predetermined sound can be heard in the second region adjacent to the first region. The sound environment management device according to claim 2 or 3.

5. The system further includes an attribute information acquisition unit that acquires attribute information indicating the attributes of a person for each of the aforementioned multiple areas, The setting adjustment unit further uses the attribute information acquired for each of the multiple regions to adjust the settings in each of the multiple regions. The sound environment management device according to any one of claims 1 to 3.

6. The aforementioned predetermined sound includes at least one of the following: the sound of an emergency vehicle siren, the warning sound of a railway crossing, the sound of a pedestrian signal, a voice related to traffic guidance, and a sound originating from another moving object present around the moving object. The sound environment management device according to any one of claims 1 to 3.

7. The setting adjustment unit identifies the relative direction of the predetermined sound source and adjusts the settings so that the sound can be heard in the region of the moving body corresponding to the identified direction. The sound environment management device according to any one of claims 1 to 3.

8. Computers External sound data indicating sounds from outside the moving object is analyzed, In response to the detection of a predetermined sound through the analysis of the external sound data, the settings related to the noise cancellation function inside the mobile body are adjusted for each of the predetermined multiple regions within the mobile body. A method for adjusting the sound environment, including the following.

9. On the computer, External sound data indicating sounds from outside the moving object is analyzed, In response to the detection of a predetermined sound through the analysis of the external sound data, the settings related to the noise cancellation function inside the mobile body are adjusted for each of the predetermined multiple regions within the mobile body. A program that performs an action that includes the following.

Images