Control method, control system, and noise visualization system

Abstract

A control method according to the present invention is to be carried out by a control system (110), wherein: collected sound information, which indicates collected sounds that are collected in a first space where a user is present, is acquired (S101); a propagation sound, which propagates from the first space to a second space that is above, below, or adjacent to the first space, is inferred on the basis of the collected sounds (S102); and propagation sound information, which visually indicates the loudness of the propagation sound in a direction associated with the second space as viewed from the user, is presented with a presentation system (150) (S103).

Description

Control method, control system, and noise visualization system

[0001] The present disclosure relates to a control method performed by a control system.

[0002] Patent Documents 1 to 4 and Non-Patent Documents 1 to 3 disclose techniques related to the present disclosure.

[0003] In particular, Patent Document 1 shows a configuration in which multiple floor models that simulate the floor structure of a house are installed side by side, each with a different structure, so that the difference in floor impact sound generated from each floor model can be experienced. Also, Patent Document 2 shows a method of capturing floor impact sound on a lower floor and displaying the obtained insulation performance. Furthermore, Patent Document 3 shows a method of making people aware of the generation of noise by visualizing whether or not sound generated in a room is transmitted as noise to adjacent spaces.

[0004] Japanese Patent Application Laid-Open No. 11-153946 Japanese Patent Application Laid-Open No. 2006-113032 Japanese Patent Application Laid-Open No. 2012-21969 Japanese Patent No. 4907483

[0005] Yoshio Matsumoto et al., "Development of a Real-Time Face and Gaze Measurement System and Its Application to Intelligent Interfaces," Information Processing Society of Japan Journal: Computer Vision and Image Media, Vol. 47, No. SIG15 (CVIM16), pp. 10-21, October 2006. "Types of Noise and Soundproofing Measures / Soundproofing Methods - Unnoise," [online], [searched July 7, 2023], Internet <URL: https: / / www.annoise.com / sound / index.html>, "Types of Walls and Soundproofing Performance / Soundproofing Methods - Unnoise," [online], [searched July 7, 2023], Internet <URL: https: / / www.annoise.com / sound / index.html>. com / sound / walls-types-soundproof-performance. html>

[0006] However, it is not easy to present information that allows people to intuitively grasp where and how much sound is propagating.

[0007] Therefore, the present disclosure provides a control method and the like that can present information that allows a user to intuitively grasp where and how much sound is propagating.

[0008] A control method according to one aspect of the present disclosure is a control method performed by a control system, which acquires collected sound information indicating collected sound, which is sound collected in a first space, which is a space where a user is located, estimates propagated sound, which is sound propagating from the first space to a second space, which is a space above, below, or adjacent to the first space, based on the collected sound, and presents propagated sound information, which visually indicates the volume of the propagated sound, in a direction associated with the second space as seen from the user, using a presentation system.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0010] A control method according to one aspect of the present disclosure makes it possible to present information that allows a user to intuitively grasp where and how much sound is propagating.

[0011] 1 is a conceptual diagram showing an example of sound propagating from a first space to a second space. FIG. 2 is a block diagram showing an example configuration of a noise visualization system according to an embodiment. FIG. 3 is a block diagram showing an example implementation of a control system according to an embodiment. FIG. 4 is a flowchart showing an example operation of the control system according to an embodiment. FIG. 5 is a conceptual diagram showing an example usage scene of the noise visualization system according to an embodiment. FIG. 6 is a conceptual diagram showing another example usage scene of the noise visualization system according to an embodiment. FIG. 7 is a conceptual diagram showing another example of propagation sound information according to an embodiment. FIG. 8 is a conceptual diagram showing yet another example of propagation sound information according to an embodiment.

[0012] FIG. 1 is a conceptual diagram showing an example of sound propagation from a first space to a second space. In this example, the first space is a room on the 10th floor, and the second space is a room on the 9th floor. For example, sound is generated in the room on the 10th floor. In the room on the 10th floor, the sound may be generated by a person shown in FIG. 1, a family member such as a child's footsteps, or an object falling in the room on the 10th floor. Then, the sound generated in the room on the 10th floor may propagate to the room on the 9th floor.

[0013] It is not easy for a person in a room on the 10th floor to know how much sound is propagating from that room to the room on the 9th floor. Furthermore, sound may propagate not only to rooms on the floor below, but also to adjacent rooms on the same floor.

[0014] Patent Document 3 discloses a method for making people aware of the occurrence of noise by visualizing whether a sound generated in a room will be transmitted as noise to adjacent spaces. Specifically, a sound generated in a room is evaluated separately for transmission to rooms on the same floor and transmission to rooms on a floor below. If the sound is determined to be noise, a notification of the occurrence of noise is issued.

[0015] However, it is not easy to present information that allows a user to intuitively grasp where and how much sound is propagating. For example, it is difficult to intuitively grasp where and how much sound is propagating by using a notification of noise occurrence as in Patent Document 3.

[0016] Therefore, the control method of Example 1 according to one aspect of the present disclosure is a control method performed by a control system, which acquires collected sound information indicating collected sound, which is sound collected in a first space, which is a space where a user is located, estimates propagated sound, which is sound propagating from the first space to a second space, which is a space above, below, or adjacent to the first space, based on the collected sound, and presents propagated sound information visually indicating the volume of the propagated sound in a direction associated with the second space as seen from the user, using a presentation system.

[0017] This makes it possible to present propagation sound information visually indicating the volume of sound propagating to a second space above, below, or adjacent to the first space in a direction associated with the second space, thereby presenting information that allows users to intuitively grasp where and how much sound is propagating.

[0018] Furthermore, the control method of Example 2 according to one aspect of the present disclosure may be the control method of Example 1, further comprising estimating a sound source position of the collected sound and estimating the propagated sound using the sound source position.

[0019] This makes it possible to accurately estimate the sound propagating from the first space to the second space according to the sound source position of the sound collected in the first space, and therefore to present information that accurately indicates the extent of the sound propagation.

[0020] Furthermore, the control method of Example 3 according to one aspect of the present disclosure may be the control method of Example 1 or Example 2, in which the propagation sound information includes at least one of a number, a waveform, and a symbol that visually indicates the magnitude of the propagation sound.

[0021] This makes it possible to present the magnitude of the propagated sound using at least one of numbers, waveforms, and symbols, thereby making it possible to easily present the level of sound that is being propagated.

[0022] Furthermore, the control method of Example 4 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 3, further estimating the type of the collected sound or the propagated sound, and the propagated sound information further visually indicating the type.

[0023] This makes it possible to present the type of sound, and therefore the type of sound that is propagating.

[0024] Furthermore, a control method of Example 5 according to one aspect of the present disclosure may be the control method of Example 4, in which the type corresponds to whether the propagated sound is noise or not.

[0025] This makes it possible to indicate whether or not a propagated sound corresponds to noise, i.e., whether or not a sound that corresponds to noise is propagating.

[0026] Furthermore, the control method of Example 6 according to one aspect of the present disclosure may be the control method of Example 4 or Example 5, in which the type corresponds to the type of sound source of the collected sound.

[0027] This makes it possible to present the type of sound source corresponding to the propagating sound, and therefore to present the type of sound source from which the sound is propagating.

[0028] Furthermore, the control method of Example 7 according to one aspect of the present disclosure may be the control method of Example 6, in which the type of the sound source is one of a plurality of types including a person, an animal, and an artificial object.

[0029] This makes it possible to indicate whether the sound source is a person, an animal, an artificial object, etc. Therefore, it becomes possible to indicate which sound is propagating from among a person, an animal, an artificial object, etc.

[0030] Furthermore, the control method of Example 8 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 7, further acquiring configuration information indicating the configuration between the first space and the second space, and estimating the propagated sound using the configuration information.

[0031] This makes it possible to accurately estimate the sound propagating from the first space to the second space based on the configuration between the first space and the second space, and therefore to present information that accurately indicates the level of sound propagating.

[0032] Furthermore, the control method of Example 9 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 8, and further may acquire sealing status information indicating the sealing status of at least one of the first space and the second space, and estimate the propagated sound using the sealing status information.

[0033] This makes it possible to accurately estimate the sound propagating from the first space to the second space based on the sealing state of the first space or the second space, and therefore to present information that accurately indicates the level of sound propagating.

[0034] Furthermore, the control method of Example 10 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 9, further acquiring directional information indicating at least one of the user's facial orientation and line of sight, and causing the presentation system to present the propagation sound information when at least one of the facial orientation and line of sight is directed toward the direction associated with the second space as seen from the user.

[0035] This makes it possible to present the propagation sound information in the direction associated with the second space when the face direction or line of sight is directed in that direction, thereby making it possible to present the propagation sound information efficiently in accordance with the face direction or line of sight.

[0036] Furthermore, the control method of Example 11 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 10, wherein, when the second space is a space adjacent to the first space, the direction associated with the second space as seen from the user is the direction of the second space as seen from the user, and when the second space is a space above or below the first space, the direction associated with the second space as seen from the user is the direction opposite to the second space as seen from the user.

[0037] This makes it possible to present the volume of sound propagating upward in the downward direction, the volume of sound propagating downward in the upward direction, and the volume of sound propagating in an adjacent direction in the adjacent direction, thereby making it possible to visually simulate in the first space the situation of sound propagating from the first space to the second space.

[0038] Furthermore, the control method of Example 12 according to one aspect of the present disclosure may be any of the control methods of Examples 1 to 11, and may further include reproducing the propagated sound using a reproduction system.

[0039] This makes it possible to auditorily present the sound propagating from the first space to the second space, thereby making it possible to auditorily simulate in the first space the situation in which the sound propagates from the first space to the second space.

[0040] Furthermore, the program of Example 13 according to one aspect of the present disclosure may be a program for causing the control system to execute any one of the control methods of Examples 1 to 12.

[0041] This makes it possible to make the control system perform the above control method according to a program.

[0042] Furthermore, the control system of Example 14 according to one aspect of the present disclosure may be a control system including: an acquirer that acquires collected sound information indicating collected sound, which is sound collected in a first space that is a space where a user is located; an estimator that estimates, based on the collected sound, a propagated sound, which is sound that propagates from the first space to a second space that is above, below, or adjacent to the first space; and a controller that causes a presentation system to present propagated sound information that visually indicates the volume of the propagated sound in a direction associated with the second space as seen from the user.

[0043] This makes it possible to present propagation sound information visually indicating the volume of sound propagating to a second space above, below, or adjacent to the first space in a direction associated with the second space, thereby presenting information that allows users to intuitively grasp where and how much sound is propagating.

[0044] Furthermore, a noise visualization system of Example 15 according to one aspect of the present disclosure may be a noise visualization system including the control system of Example 14 and the presentation system.

[0045] This makes it possible to operate the control system and presentation system in the noise visualization system, thereby making it possible to present transmitted sound information without relying on other systems.

[0046] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0047] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, the order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the scope of the claims.

[0048] Fig. 2 is a block diagram showing an example configuration of a noise visualization system according to an embodiment. The noise visualization system 100 shown in Fig. 2 visualizes, in the first space, propagation sound that is sound propagating from a first space to a second space. For example, each of the first space and the second space is a room or a dwelling unit in a building. The first space is the space where the user is located. The second space is a space above, below, or adjacent to the first space. Here, "adjacent" means adjacent in the horizontal direction, that is, adjacent on the same floor.

[0049] For example, the noise visualization system 100 visually displays footsteps heard by a person downstairs on the ceiling of the user's room, and visually displays sounds heard by a person in a neighboring room on a wall in the direction of the neighboring room. Furthermore, the noise visualization system 100 may play back the visually displayed sounds. The noise visualization system 100 allows the user in the user's room to experience the situation of noise propagating from the user's room to surrounding rooms. This allows the user to experience the situation of noise propagating to the surroundings in the user's room.

[0050] 2 , the noise visualization system 100 includes a control system 110, a collection system 120, a detection system 130, a playback system 140, and a presentation system 150. The noise visualization system 100 may include only some of these systems. Specifically, the noise visualization system 100 may not include the detection system 130 or the playback system 140.

[0051] The control system 110 is, for example, a computer system, and performs control to visualize, in the first space, sound propagating from the first space to the second space. In Fig. 2, the control system 110 includes an acquirer 111, an estimator 112, and a controller 113.

[0052] The acquirer 111 is, for example, a circuit, and acquires collected sound information indicating collected sound, which is sound collected in the first space by the collection system 120. The acquirer 111 may acquire the collected sound information from the collection system 120 via a wired or wireless communication path.

[0053] The acquirer 111 may further acquire configuration information indicating the configuration between the first space and the second space. The configuration between the first space and the second space may be the structure, materials, or a combination thereof between the first space and the second space. The acquirer 111 may acquire the configuration information from a user's terminal device via a wired or wireless communication path. Alternatively, the acquirer 111 may acquire the configuration information input to the control system 110 via an input interface.

[0054] The acquirer 111 may further acquire sealed status information indicating the sealed status of at least one of the first space and the second space. In this case, the acquirer 111 may acquire the sealed status information from a user's terminal device via a wired or wireless communication path. Alternatively, the acquirer 111 may acquire sealed status information input to the control system 110 via an input interface. Alternatively, the acquirer 111 may acquire sealed status information indicating the sealed status detected by the detection system 130 from the detection system 130 via a wired or wireless communication path.

[0055] The acquirer 111 may further acquire directional information indicating at least one of the facial orientation and the line of sight of the user in the first space. In this case, the acquirer 111 may acquire the directional information indicating at least one of the facial orientation and the line of sight detected by the detection system 130 from the detection system 130 via a wired or wireless communication path.

[0056] The acquirer 111 may further acquire playback instruction information for playing the propagation sound. For example, the acquirer 111 may acquire the playback instruction information from the collection system 120, the detection system 130, the playback system 140, or the presentation system 150 via a wired or wireless communication path in response to a user action or the like.

[0057] The estimator 112 is, for example, a circuit that estimates sound propagating from the first space to the second space based on sound collected in the first space. For example, the estimator 112 may estimate the propagating sound by analyzing multiple characteristics of the collected sound, such as the loudness, frequency characteristics, and waveform of the collected sound, and simulating sound propagating from the first space to the second space based on the multiple characteristics of the collected sound.

[0058] Furthermore, the estimator 112 may estimate the source position of the collected sound, i.e., the source position of the sound collected in the first space, as one of the multiple characteristics of the collected sound. Then, the estimator 112 may estimate the propagated sound by simulating the sound propagating from the first space to the second space using the source position of the collected sound. Specifically, it is assumed that the sound source position affects the sound propagating from the first space to the second space. The estimator 112 may reflect this influence in the estimation of the propagated sound.

[0059] Furthermore, the estimator 112 may estimate the propagated sound by simulating the sound propagating from the first space to the second space using configuration information indicating the configuration between the first space and the second space. Specifically, the configuration, such as the structure and materials, between the first space and the second space is assumed to affect the sound propagating from the first space to the second space. More specifically, as described in Non-Patent Document 3, it is assumed that the soundproofing performance differs depending on the type of wall. The estimator 112 may reflect this influence in the estimation of the propagated sound.

[0060] Furthermore, the estimator 112 may estimate the propagated sound by simulating the sound propagating from the first space to the second space using sealing status information indicating the sealing status of at least one of the first space and the second space. Specifically, it is assumed that the sealing status of the first space and the second space affects the sound propagating from the first space to the second space. For example, if the first space and the second space are sealed, sound is less likely to propagate, and if the first space and the second space are open, sound is more likely to propagate. The estimator 112 may reflect this influence in the estimation of the propagated sound.

[0061] The estimator 112 may also estimate the type of collected sound or propagated sound. The type of collected sound or propagated sound may correspond to whether the propagated sound is noise or may correspond to the type of sound source of the collected sound. The type of sound source may be a person, an animal, an artificial object, or the like.

[0062] The controller 113 is, for example, a circuit, and causes the presentation system 150 to present propagation sound information that visually indicates the magnitude of the propagation sound in a direction associated with the second space as seen from the user in the first space. Specifically, the controller 113 may transmit presentation control information to the presentation system 150 to cause the presentation system 150 to present the propagation sound information.

[0063] For example, the direction of the second space or the direction opposite to the second space is associated with the second space as a direction relative to the user in the first space. Specifically, the direction of the second space, i.e., the direction toward the second space, is associated with the second space adjacent to the first space. Furthermore, the direction opposite to the second space, i.e., the direction opposite to the direction toward the second space, is associated with the second space above or below the first space.

[0064] Therefore, when the second space is adjacent to the first space, the presentation system 150 presents the propagation sound information in the direction of the second space as seen by the user of the first space. Also, when the second space is above or below the first space, the presentation system 150 presents the propagation sound information in the opposite direction to the second space as seen by the user of the first space.

[0065] The transmitted sound information may be a number, a waveform, a symbol, or a combination thereof. The transmitted sound information may further indicate the type of collected sound or transmitted sound. In other words, the transmitted sound information may indicate not only the volume of the transmitted sound but also the type of collected sound or transmitted sound.

[0066] Furthermore, the controller 113 may cause the presentation system 150 to present the propagation sound information in accordance with the direction information obtained by the acquirer 111. Specifically, the controller 113 may cause the presentation system 150 to present the propagation sound information when at least one of the facial orientation and the line of sight of the user in the first space is directed in a direction associated with the second space as seen from the user in the first space.

[0067] Furthermore, the controller 113 may cause the playback system 140 to play back the propagation sound estimated by the estimator 112 in accordance with the playback instruction information acquired by the acquirer 111. In this case, the controller 113 may transmit playback control information to the playback system 140 to cause the playback system 140 to play back the propagation sound.

[0068] The collection system 120 includes, for example, one or more microphones, collects sound in the first space, and transmits collected sound information indicating the sound collected in the first space to the control system 110. The collection system 120 may include one or more microphones installed in the first space, or may include a microphone included in a wearable device worn by a user in the first space.

[0069] The detection system 130 includes, for example, one or more sensors, detects at least one of the facial orientation and line of sight of the user in the first space, and transmits directional information indicating at least one of the facial orientation and line of sight of the user in the first space to the control system 110.

[0070] For example, the detection system 130 may include a camera that captures an image of the user in the first space and detect at least one of the user's facial orientation and line of sight from the image of the user captured by the camera. Alternatively, the detection system 130 may include a sensor such as an attitude sensor, a tilt sensor, or an orientation sensor and detect at least one of the user's facial orientation and line of sight from sensing results obtained by the sensor. The detection system 130 may include a sensor included in a wearable device worn by the user in the first space.

[0071] Furthermore, the detection system 130 may include a camera included in a wearable device worn by a user in the first space, and may detect at least one of the user's facial orientation and line of sight from an image of the first space captured by the camera.

[0072] Furthermore, the detection system 130 may detect at least one of the user's facial orientation and line of sight based on the method described in Patent Document 4 or Non-Patent Document 1.

[0073] The playback system 140 includes, for example, one or more sound output devices, and plays back and outputs the propagation sound. For example, the playback system 140 receives playback control information from the control system 110, and plays back and outputs the propagation sound in accordance with the playback control information. The playback system 140 may include a speaker, earphones, or headphones. The playback system 140 may include earphones or headphones included in a wearable device worn by a user in the first space.

[0074] The presentation system 150 includes, for example, one or more presenters, and presents propagation sound information in a direction associated with the second space as seen from the user in the first space. For example, the presentation system 150 receives presentation control information from the control system 110, and presents the propagation sound information in accordance with the presentation control information.

[0075] Specifically, the presentation system 150 may include smart glasses included in a wearable device worn by a user in the first space, and the presentation system 150 may present the propagation sound information on the smart glasses when the direction of the user's face or line of sight coincides with the direction associated with the second space, so that the propagation sound information is presented in the direction associated with the second space.

[0076] Alternatively, the presentation system 150 may include a plurality of displays arranged on a plurality of surfaces of the first space in which the user is located. That is, the presentation system 150 may include a plurality of displays arranged on a plurality of walls, floors, and ceilings of the room in which the user is located. Then, the presentation system 150 may present the propagation sound information on a display in a direction associated with the second space.

[0077] The presentation system 150 may be a display system that displays propagation sound information. That is, presentation may be interpreted as display.

[0078] The control system 110 may also include one or more of the collection system 120, the detection system 130, the playback system 140, and the presentation system 150. The term "system" may also be interpreted as "device." These systems or devices may also be composed of multiple devices.

[0079] The estimator 112 of the control system 110 may be divided into a propagation sound estimator that estimates propagation sound and a sound source position estimator that estimates the sound source position of the collected sound. The acquirer 111 of the control system 110 may be divided into a collected sound information acquirer that acquires collected sound information, a configuration information acquirer that acquires configuration information, a sealing condition information acquirer that acquires sealing condition information, and a direction information acquirer that acquires direction information.

[0080] Fig. 3 is a block diagram showing an example implementation of the control system 110 shown in Fig. 2. In Fig. 3, the control system 110 includes a processor 116 and a memory 117. Several components of the control system 110 shown in Fig. 2 may be implemented by the processor 116 and the memory 117 shown in Fig. 3.

[0081] The processor 116 is a dedicated or general-purpose processor for processing information in the control system 110 and is a processor that can access the memory 117. The processor 116 may be an electric circuit. The processor 116 may also be a processor such as a CPU. The processor 116 may also be a collection of multiple processors.

[0082] At least a portion of each of the obtainer 111, the estimator 112, and the controller 113 shown in FIG. 2 may be implemented by the processor 116.

[0083] The memory 117 is a dedicated or general-purpose memory for storing information in the control system 110. The memory 117 may be an electric circuit. The memory 117 may be connected to the processor 116 or may be included in the processor 116. The memory 117 may be a collection of multiple memories. The memory 117 may also be a magnetic disk, an optical disk, or the like, and may be expressed as a storage or a recording medium, or the like. The memory 117 may also be a non-volatile memory or a volatile memory.

[0084] For example, collected sound information, propagated sound information, configuration information, sealed state information, or directional information may be stored in the memory 117. Furthermore, the memory 117 may store information for the processor 116 to process information, or may store a program for the processor 116 to process information.

[0085] At least a portion of each of the obtainer 111, the estimator 112, and the controller 113 shown in FIG. 2 may be implemented by the memory 117.

[0086] 4 is a flowchart showing an example of the operation of the control system 110 shown in FIG. 2 and other figures. First, the acquirer 111 acquires collected sound information indicating collected sound, which is sound collected in a first space where a user is located (S101). The estimator 112 estimates transmitted sound, which is sound propagating from the first space to a second space, based on the collected sound (S102). Here, the second space is a space above, below, or adjacent to the first space. Then, the controller 113 causes the presentation system 150 to present transmitted sound information visually indicating the volume of the transmitted sound in a direction associated with the second space as seen from the user in the first space (S103).

[0087] This makes it possible to present propagation sound information visually indicating the volume of sound propagating to a second space above, below, or adjacent to the first space in a direction associated with the second space, thereby presenting information that allows users to intuitively grasp where and how much sound is propagating.

[0088] For example, the estimator 112 may estimate the sound source position of the collected sound. Then, the estimator 112 may estimate the propagated sound using the sound source position. This makes it possible to accurately estimate the sound propagating from the first space to the second space according to the sound source position of the sound collected in the first space. Therefore, it becomes possible to present information that accurately indicates the extent to which the sound is propagating.

[0089] Furthermore, for example, the transmitted sound information may include at least one of a number, a waveform, and a symbol that visually indicates the magnitude of the transmitted sound. This makes it possible to present the magnitude of the transmitted sound using at least one of a number, a waveform, and a symbol. Therefore, it is possible to present in an easy-to-understand manner how much sound is being transmitted.

[0090] Furthermore, for example, the estimator 112 may further estimate the type of collected sound or propagated sound. The propagated sound information may further visually indicate the type of collected sound or propagated sound. This makes it possible to present the type of sound. Therefore, it becomes possible to present what kind of sound is propagating.

[0091] Furthermore, for example, the type may correspond to whether or not the propagated sound is noise. This makes it possible to indicate whether or not the propagated sound corresponds to noise. In other words, it becomes possible to indicate whether or not a sound that corresponds to noise is propagating.

[0092] Furthermore, for example, the type may correspond to the type of sound source of the collected sound. This makes it possible to present the type of sound source corresponding to the propagated sound. Therefore, it becomes possible to present the type of sound source from which the sound is propagating.

[0093] Furthermore, for example, the type of sound source may be any one of a plurality of types including a person, an animal, and an artificial object. This makes it possible to indicate whether the sound source is a person, an animal, an artificial object, etc. Therefore, it becomes possible to indicate which sound is propagating from among a person, an animal, an artificial object, etc.

[0094] Furthermore, for example, the acquirer 111 may acquire configuration information indicating the configuration between the first space and the second space. Then, the estimator 112 may estimate the propagated sound using the configuration information. This makes it possible to accurately estimate the sound propagating from the first space to the second space based on the configuration between the first space and the second space. Therefore, it becomes possible to present information that accurately indicates the extent to which sound is propagating.

[0095] Furthermore, for example, the acquirer 111 may acquire sealing status information indicating the sealing status of at least one of the first space and the second space. Then, the estimator 112 may estimate the propagated sound using the sealing status information. This makes it possible to accurately estimate the sound propagating from the first space to the second space based on the sealing status of the first space or the second space. Therefore, it becomes possible to present information that accurately indicates the level of sound propagating.

[0096] Furthermore, for example, the acquirer 111 may further acquire direction information indicating at least one of the facial orientation and the line of sight of the user in the first space. Then, the controller 113 may cause the presentation system 150 to present the propagation sound information when at least one of the facial orientation and the line of sight is directed in a direction associated with the second space as seen from the user in the first space.

[0097] This makes it possible to present the propagation sound information in the direction associated with the second space when the face direction or line of sight is directed in that direction, thereby making it possible to present the propagation sound information efficiently in accordance with the face direction or line of sight.

[0098] Furthermore, for example, if the second space is adjacent to the first space, the direction associated with the second space as seen by the user of the first space may be the direction of the second space as seen by the user of the first space. And, if the second space is above or below the first space, the direction associated with the second space as seen by the user of the first space may be the opposite direction to the second space as seen by the user of the first space.

[0099] This makes it possible to present the volume of sound propagating upward in the downward direction, the volume of sound propagating downward in the upward direction, and the volume of sound propagating in an adjacent direction in the adjacent direction, thereby making it possible to visually simulate in the first space the situation of sound propagating from the first space to the second space.

[0100] Furthermore, for example, the controller 113 may cause the playback system 140 to play the propagation sound. This makes it possible to auditorily present the sound propagating from the first space to the second space. Therefore, it becomes possible to auditorily simulate in the first space the situation of sound propagating from the first space to the second space.

[0101] FIG. 5 is a conceptual diagram showing an example of a usage scene of the noise visualization system 100 shown in FIG. 2 etc. In this example, the second space is a space below the first space. Furthermore, multiple microphones 121 to 124 are arranged in the first space. The multiple microphones 121 to 124 are an example of the collection system 120. Furthermore, a user in the first space is wearing smart glasses 151. The smart glasses 151 are an example of the presentation system 150.

[0102] Collected sound information indicating sounds collected in the first space by the multiple microphones 121 to 124 is transmitted to the control system 110 via a wired or wireless communication path. The control system 110 estimates the sound propagating from the first space to the second space based on the sounds collected in the first space. Then, the control system 110 causes the smart glasses 151 to present the propagated sound information in a direction associated with the second space as seen from the user in the first space.

[0103] In this example, the second space below the first space is associated with the opposite direction to the second space. That is, the propagation sound information is presented in the upward direction, which is the opposite direction to the second space as seen by the user in the first space. More specifically, the propagation sound information is presented on the ceiling. Also, in this example, the propagation sound information is presented as a waveform that visually indicates the propagation sound. The amplitude of the waveform corresponds to the volume of the propagation sound.

[0104] As a result, the sound propagating downstairs is visualized on the ceiling. Therefore, the noise visualization system 100 can visually simulate the sound situation downstairs in the first space, allowing the user in the first space to intuitively recognize the sound propagating downstairs.

[0105] Fig. 6 is a conceptual diagram showing another example of a usage scene of the noise visualization system 100 shown in Fig. 2 etc. In this example, the second space is a space adjacent to the first space.

[0106] 5, a plurality of microphones 121 to 124 are arranged in the first space, and a user in the first space is wearing smart glasses 151.

[0107] Then, collected sound information indicating sounds collected in the first space by the multiple microphones 121 to 124 is transmitted to the control system 110. The control system 110 estimates the sound propagating from the first space to the second space based on the sounds collected in the first space. Then, the control system 110 causes the smart glasses 151 to present the propagated sound information in a direction associated with the second space as seen from the user in the first space.

[0108] In this example, the direction of the second space adjacent to the first space is associated with the second space. That is, the propagation sound information is presented in the direction of the second space as seen by the user in the first space. More specifically, the propagation sound information is presented on the wall between the first space and the second space.

[0109] 5, the transmitted sound information is presented as a waveform that visually indicates the transmitted sound. The amplitude of the waveform corresponds to the loudness of the transmitted sound.

[0110] As a result, the sound propagating to the adjacent second space is visualized on the wall. Therefore, the noise visualization system 100 can visually simulate the sound situation in the adjacent second space in the first space, allowing the user in the first space to intuitively recognize the sound propagating to the adjacent second space.

[0111] Here, the second space adjacent to the first space is associated with the direction of the second space, not the opposite direction of the second space. When the second space is below the first space, the situation downstairs is more accurately simulated by presenting propagation sound information on the ceiling. On the other hand, when the second space is adjacent to the first space, even if propagation sound information is not presented on the wall opposite the second space, the adjacent situation is accurately simulated by presenting propagation sound information on the wall between the first and second spaces. Sound propagating to the second space adjacent to the first space is intuitively recognized by the user of the first space.

[0112] Furthermore, there are cases where the equipment in the first space and the equipment in the second space have a plane-symmetrical relationship with respect to the wall between the first space and the second space. In particular, in such cases, the situation next to the first space can be accurately simulated by presenting propagation sound information on the wall between the first space and the second space.

[0113] However, the direction associated with the second space is not limited to the above example. A second space adjacent to a first space may be associated with a direction opposite to the second space, rather than the direction of the second space. Depending on the environment, it may be easier to understand if propagation sound information indicating sound propagating to a second space adjacent to a first space is presented in the direction opposite to the second space.

[0114] Furthermore, a second space above or below a first space may be associated with a direction of the second space, rather than an opposite direction to the second space. In some cases, it may be easier to understand if propagation sound information indicating sound propagating to a second space above or below a first space is presented in the direction of the second space.

[0115] 5 and 6, the control system 110 may estimate the sound source position of the collected sound based on the collected sound, which is sound collected in the first space by the multiple microphones 121 to 124. Specifically, the control system 110 may estimate the sound source position for each of the multiple microphones 121 to 124 by estimating that the louder the sound collected by that microphone, the closer the sound source position to that microphone.

[0116] The control system 110 may then estimate the sound propagating to the second space by simulating the attenuation of the sound from the sound source position, which makes it possible to more accurately estimate the sound propagating to the second space.

[0117] 5 and 6, the control system 110 may estimate the propagated sound based on the collected sound, regardless of the sound source position. In this case, only one microphone may be used instead of the multiple microphones 121 to 124. In this case, a microphone included in a wearable device including the smart glasses 151 may be used.

[0118] 5 and 6, the control system 110 may estimate the sound propagating from the first space to the second space based on the configuration between the first space and the second space. Specifically, in the example of Fig. 5, the control system 110 may estimate the sound propagating from the first space to the second space based on the structure and material of the ceiling and floor. In the example of Fig. 6, the control system 110 may estimate the sound propagating from the first space to the second space based on the structure and material of the wall.

[0119] 5 and 6, the control system 110 may estimate the sound propagating from the first space to the second space based on the sealing state of at least one of the first space and the second space. For example, if the first space and the second space are connected via a staircase or a passage without being blocked, it may be estimated that the propagated sound will be louder. Also, if a window is open, it may be estimated that the propagated sound will be louder because sound propagates through the window.

[0120] Furthermore, in the examples of Figures 5 and 6, smart glasses 151 are used, but the propagated sound information may also be presented on displays arranged on the walls, ceiling, and floor.

[0121] In addition, in the examples of Figures 5 and 6, when at least one of the user's facial direction and line of sight matches the direction associated with the second space as seen from the user, propagation sound information indicating the volume of sound propagating into the second space may be presented.

[0122] That is, which of the multiple second spaces should have its propagation sound information presented may be determined based on at least one of the user's facial orientation and gaze direction. A camera installed in the first space may be used to detect the user's facial orientation or gaze direction, or a camera mounted on a wearable device worn by the user may be used. Alternatively, various sensors mounted on the wearable device may be used. Furthermore, the method described in Patent Document 4 or Non-Patent Document 1 may be used to detect the user's facial orientation or gaze direction.

[0123] 5 and 6, the transmission sound may be reproduced by the user taking a certain action. For example, the user may use his / her index finger to designate a waveform displayed as transmission sound information, thereby reproducing the transmission sound corresponding to the waveform. Alternatively, the user may vocally instruct reproduction, thereby reproducing the transmission sound corresponding to the waveform the user is viewing. When the transmission sound is reproduced, the transmission sound may be output from headphones mounted on the wearable device worn by the user.

[0124] This allows the user to confirm by hearing how much sound the propagation sound information shown in waveform specifically corresponds to.

[0125] 5 and 6, four microphones 121 to 124 are used. As described above, only one microphone may be used. Alternatively, only two microphones may be used, only three microphones may be used, or five or more microphones may be used. For example, microphones may be installed on a total of six surfaces, including four walls, a ceiling, and a floor.

[0126] Furthermore, in the examples of FIGS. 5 and 6, the waveform of the transmitted sound is presented as the transmitted sound information, but the transmitted sound information is not limited to the waveform of the transmitted sound.

[0127] 7 is a conceptual diagram showing another example of the propagation sound information. In this example, the propagation sound information includes a number indicating the volume corresponding to the loudness of the propagation sound and a volume bar. As in this example, a number indicating the loudness of the propagation sound may be presented as the propagation sound information. This makes it possible to present the loudness of the propagation sound in an easy-to-understand manner.

[0128] Fig. 8 is a conceptual diagram showing yet another example of transmitted sound information. In this example, the transmitted sound information includes a symbol indicating the type of transmitted sound, and the type of transmitted sound corresponds to whether the transmitted sound is noise or not. Specifically, if the transmitted sound is not noise, the symbol on the left side of Fig. 8 is presented. If the transmitted sound is noise, the symbol on the right side of Fig. 8 is presented.

[0129] For example, whether the propagated sound corresponds to noise may be determined based on whether at least one of the magnitude, frequency, and waveform of the propagated sound conforms to the noise conditions. Furthermore, the estimator 112 of the control system 110 may estimate whether the propagated sound corresponds to noise based on whether at least one of the magnitude, frequency, and waveform of the propagated sound conforms to the noise conditions.

[0130] Furthermore, the symbols may indicate the magnitude of the propagated sound. For example, in the symbols on the left side of Fig. 8, the magnitude of the propagated sound may be represented by the number of arcs. Also, in the symbols on the right side of Fig. 8, the magnitude of the propagated sound may be represented by the number of lightning marks. These symbols indicate the magnitude of the propagated sound and whether or not the propagated sound is noise.

[0131] 9 is a conceptual diagram showing yet another example of propagation sound information. In this example, the propagation sound information includes a symbol indicating the type of sound collected in the first space, and the type of sound corresponds to the type of sound source. More specifically, as shown in FIG. 9, the symbol included in the propagation sound information indicates which of multiple types of sound sources the sound source is, including a person, an animal, and an artificial object.

[0132] The loudness of the propagated sound may also be expressed by the size or number of elements included in the symbol. For example, the loudness of the propagated sound may be expressed by the size or number of people, animals, or man-made objects. Alternatively, the loudness of the propagated sound may be expressed by adding an arc or lightning mark, as in the example of Figure 8. Alternatively, the loudness of the propagated sound may be expressed by combining a symbol such as that shown in Figure 9 with the waveforms shown in Figures 5 and 6 or the numbers shown in Figure 7.

[0133] Furthermore, when the type of sound is estimated more specifically, the type of sound may be expressed more specifically. As with the noises described in Non-Patent Document 2, various sounds are expected to be generated in the first space.

[0134] For example, if the type of sound is estimated to be any of music on an audio player, talking, the sound of a door opening and closing, the sound of water flowing from a water pipe, footsteps, etc., such type may be expressed. Furthermore, if a speaker such as a family member is identified as the sound source, the speaker may be expressed as the type of sound. Alternatively, if the type of pet that is the sound source is identified, the type of pet may be expressed as the type of sound.

[0135] Furthermore, the type of sound may be represented by a silhouette, an image, a video, or any combination thereof.

[0136] For example, if the sound collected in the first space is the sound of a child running around, a silhouette of the child or an image of a smiling child may be presented as the transmitted sound information together with the waveform of the transmitted sound.Furthermore, if the sound collected in the first space is the sound of a musical instrument, a symbol of a guitar representing the instrument and sweat representing effort may be presented as the transmitted sound information together with the waveform of the transmitted sound.

[0137] This makes it possible to provide information for evaluating whether the propagated sound is acceptable to people in the vicinity.

[0138] While this is primarily intended to cover sounds associated with daily life, it may also be applied to sounds associated with construction work such as renovations. For example, an icon indicating whether the sound is noise or not, and an icon indicating the speaker or tool that is the sound source, are displayed along with a waveform showing the propagating sound. This allows construction work to proceed smoothly within an acceptable range while checking and suppressing sounds propagating to the surrounding area.

[0139] Although the noise visualization system, control system, etc. have been described above according to the embodiments, the aspects of the noise visualization system, control system, etc. are not limited to the embodiments. Modifications conceivable by those skilled in the art may be made to the embodiments, and multiple components in the embodiments may be combined in any manner.

[0140] For example, a process performed by a specific component in an embodiment may be performed by another component instead of the specific component. Furthermore, the order of multiple processes may be changed, or multiple processes may be performed in parallel. Furthermore, ordinal numbers such as "first" and "second" used in the description may be changed, removed, or newly assigned as appropriate. These ordinal numbers do not necessarily correspond to a meaningful order, and may be used to identify elements.

[0141] Also, for example, a phrase "at least one of a first element, a second element, and a third element" corresponds to the first element, the second element, the third element, or any combination thereof.

[0142] Furthermore, a method including steps performed by each component of the noise visualization system, the control system, etc. may be performed by any system or device. In other words, the method may be performed by the noise visualization system or control system described above, or by another system or device.

[0143] For example, a part or all of the method may be executed by a computer including a processor, a memory, an input / output circuit, etc. In this case, the method may be executed by the computer executing a program for causing the computer to execute the method.

[0144] For example, the above program causes a computer to execute a control method in which collected sound information indicating collected sound, which is sound collected in a first space where a user is located, estimates propagated sound, which is sound propagating from the first space to a second space that is above, below, or adjacent to the first space, based on the collected sound, and presents propagated sound information that visually indicates the volume of the propagated sound in a direction associated with the second space as seen from the user, using a presentation system.

[0145] The above program may also be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0146] Furthermore, each component of the noise visualization system, the control system, etc. may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, etc., or a combination of these. The general-purpose hardware may be configured with a memory in which the program is recorded and a general-purpose processor that reads and executes the program from the memory, etc. Here, the memory may be a semiconductor memory or a hard disk, etc., and the general-purpose processor may be a CPU, etc.

[0147] Furthermore, the dedicated hardware may be configured with a memory and a dedicated processor, etc. For example, the dedicated processor may refer to the memory and execute the above-described method.

[0148] Furthermore, each component of the noise visualization system, the control system, etc. may be an electric circuit. These electric circuits may form a single electric circuit as a whole, or may be separate electric circuits. Furthermore, these electric circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc.

[0149] The present disclosure is applicable to a noise visualization system or the like that presents information about where and how much sound is propagating.

[0150] 100 Noise visualization system 110 Control system 111 Acquirer 112 Estimator 113 Controller 116 Processor 117 Memory 120 Acquisition system 121, 122, 123, 124 Microphone 130 Detection system 140 Playback system 150 Presentation system 151 Smart glasses

Claims

1. A control method performed by a control system, comprising: acquiring collected sound information indicating collected sound, which is sound collected in a first space which is a space in which a user is located; estimating a transmitted sound, which is sound that propagates from the first space to a second space which is above, below or adjacent to the first space, based on the collected sound; and presenting transmitted sound information which visually indicates the volume of the transmitted sound in a direction corresponding to the second space as seen from the user, by a presentation system.

2. The control method according to claim 1, further comprising estimating a source position of the collected sound, and estimating the propagated sound using the source position.

3. The control method according to claim 1 or 2, wherein the transmitted sound information includes at least one of a number, a waveform, and a symbol that visually indicates the loudness of the transmitted sound.

4. The control method according to claim 1 or 2, further comprising estimating a type of the collected sound or the propagated sound, and the propagated sound information further visually indicating the type.

5. The control method according to claim 4, wherein the type corresponds to whether the propagated sound is noise or not.

6. The control method according to claim 4, wherein the type corresponds to a type of sound source of the collected sound.

7. The control method according to claim 6, wherein the type of the sound source is one of a plurality of types including a person, an animal, and an artificial object.

8. A control method as described in claim 1 or 2, further comprising obtaining configuration information indicating a configuration between the first space and the second space, and estimating the propagated sound using the configuration information.

9. A control method as described in claim 1 or 2, further comprising acquiring sealing condition information indicating a sealing condition of at least one of the first space and the second space, and estimating the propagated sound using the sealing condition information.

10. A control method as described in claim 1 or 2, further comprising acquiring directional information indicating at least one of the user's facial direction and line of sight, and causing the presentation system to present the propagated sound information when at least one of the facial direction and line of sight is directed toward the direction corresponding to the second space as seen from the user.

11. A control method as described in claim 1 or 2, wherein, when the second space is a space adjacent to the first space, the direction associated with the second space as seen by the user is the direction of the second space as seen by the user, and, when the second space is a space above or below the first space, the direction associated with the second space as seen by the user is the direction opposite to the second space as seen by the user.

12. The control method according to claim 1 or 2, further comprising reproducing the propagated sound by a reproduction system.

13. A program for causing the control system to execute the control method according to claim 1 or 2.

14. A control system comprising: an acquirer that acquires collected sound information indicating collected sound, which is sound collected in a first space that is a space in which a user is located; an estimator that estimates a propagated sound, which is sound that propagates from the first space to a second space that is above, below, or adjacent to the first space, based on the collected sound; and a controller that causes a presentation system to present propagated sound information that visually indicates the volume of the propagated sound in a direction corresponding to the second space as seen from the user.

15. A noise visualization system comprising: the control system according to claim 14; and the presentation system.

Images