Voice adjustment method, voice adjustment device, voice adjustment system, and program

Abstract

A sound adjustment method comprises: a condition estimation step (S102) for estimating the condition of a user using a first space among the first space and a second space that are adjacent to each other; a use condition determination step (S104) for determining, on the basis of the estimated condition of the user using the first space, whether or not the first space is used for the work of a plurality of operators including the user using the first space; and a sound propagation adjustment step (step S105 and step S107) for switching, on the basis of the determination result in the use condition determination step (S104), whether or not noise cancellation is applied in which the listening level of the sound generated in one of the first space and the second space is reduced when the sound is listened in the other of the first space and the second space.

Description

Technical Field

[0001] The present disclosure relates to a sound adjustment method, a sound adjustment device, a sound adjustment system and a program. Background Art

[0002] Patent Document 1 discloses a technique related to a vehicle sound insulating structure capable of switching between a sound insulating state and a non-sound insulating state.

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-137358 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Furthermore, the soundproofing state and the non-soundproofing state described in Patent Document 1 are usually switched according to the state of the vehicle. However, depending on the conditions of the space to which the device is applied, appropriate switching may not be performed simply by switching according to the state.

[0008] Therefore, the present disclosure provides a sound adjustment method and the like that can more appropriately switch whether or not to allow sound to pass between two spaces.

[0009] Means for solving problems

[0010] A sound adjustment method involved in one embodiment of the present invention includes: a condition estimation step of estimating the condition of users using the first space between mutually adjacent first and second spaces; a utilization condition determination step of determining whether the first space is used for operations of multiple operators including users using the first space based on the estimated condition of users using the first space; and a sound propagation adjustment step of switching whether to apply noise elimination based on the determination result in the utilization condition determination step, in which the listening level of the sound generated in one of the first and second spaces when heard in the other of the first and second spaces is reduced.

[0011] In addition, a sound adjustment device involved in one embodiment of the present invention comprises: a situation estimation unit, which estimates the situation of users using the first space between the mutually adjacent first space and second space; a utilization situation determination unit, which determines whether the first space is used for operations of multiple operators including users using the first space based on the estimated situation of users using the first space; and a sound propagation adjustment unit, which switches whether to apply noise cancellation based on the determination result of the utilization situation determination unit, in which the listening level of the sound generated in one of the first space and the second space when being listened to in the other of the first space and the second space is reduced.

[0012] In addition, a sound adjustment system involved in one embodiment of the present disclosure includes: a first microphone for collecting sounds in a first space, a second microphone for collecting sounds in a second space adjacent to the first space, a first speaker for outputting sounds in the first space, a second speaker for outputting sounds in the second space, a sound adjustment device, and a control device, wherein the sound adjustment device has: a situation estimation unit for estimating the situation of a user using the first space; a utilization situation determination unit for determining whether the first space is used for operations of a plurality of operators including the user using the first space based on the estimated situation of the user using the first space; and a sound propagation adjustment unit for switching whether to apply noise cancellation based on a determination result in the utilization situation determination unit, in which a listening level of sounds generated in one of the first space and the second space when heard in the other of the first space and the second space is reduced, and the control device controls the first microphone, the second microphone, the first speaker, and the second speaker according to whether the noise cancellation switched by the sound propagation adjustment unit is applied.

[0013] In addition, these overall or specific methods can also be implemented by devices, integrated circuits, computer programs or non-volatile recording media such as computer-readable CD-ROMs, and can also be implemented by any combination of devices, integrated circuits, computer programs and non-volatile recording media.

[0014] Effects of the Invention

[0015] According to the sound adjustment method and the like in the present disclosure, it is possible to more appropriately switch whether or not to allow sound to pass between two spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a diagram for explaining a space to which the sound adjustment system according to the embodiment is applied.

[0017] Figure 2It is a diagram showing an example of use of the sound adjustment system involved in the embodiment.

[0018] Figure 3 This is a block diagram showing an example of the hardware configuration of the sound adjustment system according to the embodiment.

[0019] Figure 4 This is a flowchart showing an example of the operation of the sound adjustment system involved in the embodiment.

[0020] Figure 5 It is a diagram showing a first example of sound adjustment implemented by the sound adjustment system involved in the embodiment.

[0021] Figure 6 It is a diagram showing a second example of sound adjustment implemented by the sound adjustment system involved in the embodiment.

[0022] Figure 7 It is a diagram showing a third example of sound adjustment implemented by the sound adjustment system involved in the embodiment.

[0023] Figure 8 It is a diagram showing a fourth example of sound adjustment implemented by the sound adjustment system involved in the embodiment.

[0024] Fig. 9 It is a diagram showing a fifth example of sound adjustment implemented by the sound adjustment system involved in the embodiment.

[0025] Fig.10 It is a diagram showing a sixth example of sound adjustment implemented by the sound adjustment system involved in the embodiment. DETAILED DESCRIPTION

[0026] (Knowledge that is the basis of this disclosure)

[0027] In recent years, the idea of ​​free address has become popular, and there are more and more cases of people choosing an area in the office where they can work easily and perform tasks such as business. Furthermore, not limited to the office, the so-called remote office, which is connected to the office through a network such as the Internet, so that they do not go to the office, and perform work in any place such as home, mobile destination, resort, public space, etc., and the hybrid office method that combines attendance office and remote office has gradually been accepted in recent years. Based on such a situation, in the home or office, it is necessary to configure a space for performing business or a shared space in a limited space. Therefore, the space for business and the space for non-business are sometimes adjacent. In such a case, there is a problem of leakage of voice from each space to the adjacent space. In other words, there is a problem that the sound generated in one space crosses the boundary part (such as a wall) and propagates to the space of the other side, and is heard by the user who uses the space of the other side. In addition, there is also a situation where a space changes from a business space to a non-business space, or from a non-business space to a business space over time, and its use changes.

[0028] For example, when a meeting is taking place in one party's space, if noise is transmitted from the other party's space, the sound of the meeting is mixed with the transmitted noise, which may hinder the progress of the meeting. The sound transmitted from the other party's space can be reduced in hearing level by applying general noise cancellation. In addition, there is a case where the space of one party is used as a space for working alone on other dates and times. In particular, there is a case where the space of one party is used as a remote meeting from home via the Internet on a certain day, and is used as a single workroom at home on another day. In a remote meeting via the Internet, the problem described above occurs, so it is preferable to reduce the sound transmitted from the other party's space, but in a case where, for example, there are children playing in the other party's space, there is a need to transmit the sound from the other party's space as it is in order to determine whether the children need to be taken care of.

[0029] Therefore, the present disclosure describes the following sound adjustment method, etc.: In order to more appropriately switch whether to allow sounds represented by speech to pass between two spaces, it is determined whether to reduce the listening level of the sound propagated between the space of the other party, that is, whether to apply noise cancellation, based on the utilization status of the space of one party. In addition, in the present disclosure, not allowing sounds represented by speech to pass means reducing the listening level of the propagated sound. Therefore, whether to allow sound to pass is a concept that means whether to propagate the sound without applying noise cancellation, or whether the propagation of the sound is made zero in a strict sense or the listening level is relatively reduced compared to the original state.

[0030] (Public Summary)

[0031] The summary of the present disclosure is as follows.

[0032] The sound adjustment method involved in the first embodiment of the present invention includes: a situation estimation step, estimating the situation of users using the first space between the mutually adjacent first space and second space; a utilization situation determination step, determining whether the first space is used for operations of multiple operators including the users using the first space based on the estimated situation of the users using the first space; and a sound propagation adjustment step, switching whether to apply noise elimination based on the determination result in the utilization situation determination step, in which the listening level of the sound generated in one of the first space and the second space when being listened to in the other of the first space and the second space is reduced.

[0033] Thus, it is possible to switch whether to apply noise cancellation based on the result of the determination of whether the first space is used for work by multiple workers. For example, when the first space is used for work with multiple workers such as a meeting, noise cancellation is applied. Thus, it is possible to more appropriately switch whether to allow sound to pass between the two spaces.

[0034] In the sound adjustment method according to the second aspect of the present disclosure, in the sound adjustment method according to the first aspect, the noise cancellation includes a first noise cancellation in which a listening level of the sound generated in the second space when heard in the first space is reduced.

[0035] Thus, by applying the noise cancellation, it is possible to reduce the listening level of the sound generated in the second space when it is heard in the first space.

[0036] In the sound adjustment method according to the third aspect of the present disclosure, in the sound adjustment method described in the first or second aspect, noise cancellation includes second noise cancellation in which a listening level of the sound generated in the first space when listened to in the second space is reduced.

[0037] Thus, by applying the noise cancellation, it is possible to reduce the listening level of the sound generated in the first space when it is heard in the second space.

[0038] In addition, the sound adjustment method involved in the 4th mode of the present disclosure includes a situation confidentiality determination step in the sound adjustment method described in any one of the 1st to 3rd modes. In the situation confidentiality determination step, based on the estimated situation of the user using the first space, it is determined whether the situation confidentiality of the user using the first space exists, and in the sound propagation adjustment step, whether to apply noise elimination is switched based on whether the situation confidentiality of the user using the first space exists.

[0039] This makes it possible to switch whether or not to apply noise cancellation based on the result of determination as to whether or not confidentiality exists in the situation of the user utilizing the first space.

[0040] In addition, in the sound adjustment method involved in the 5th mode of the present disclosure, in the sound adjustment method described in the 4th mode, in the situation estimation step, it is determined whether the user is performing a task as the situation of the user using the first space, and in the situation confidentiality determination step, when the estimated situation of the user using the first space indicates that the user is performing a task, it is determined that the situation confidentiality of the user using the first space exists.

[0041] Thus, when the user utilizing the first space is performing a task, it is possible to determine that the situation confidentiality exists, and whether or not to apply noise cancellation can be switched based on the determination result.

[0042] In addition, the sound adjustment method involved in the 6th mode of the present disclosure is in the sound adjustment method described in any one of the 1st to 5th modes, and further includes: an attribute estimation step, estimating the attributes of the user using the first space; and an attribute confidentiality determination step, based on the estimated attributes of the user using the first space, determining whether the attribute confidentiality of the user using the first space exists, and in the sound propagation adjustment step, switching whether noise elimination is applicable is also based on whether the attribute confidentiality of the user using the first space exists.

[0043] This makes it possible to switch whether or not to apply noise cancellation based on the result of determination whether or not the attribute confidentiality of the user using the first space exists.

[0044] In addition, in the sound adjustment method involved in the 7th embodiment of the present invention, in the sound adjustment method described in the 6th embodiment, in the attribute estimation step, information related to the affiliation of all operators involved in the operation using the first space is estimated as the attribute of the user using the first space, and in the attribute confidentiality determination step, when the information related to the affiliation of all operators is not consistent, it is determined that the attribute confidentiality of the user using the first space exists.

[0045] Thus, when the information on the affiliation of all workers involved in the work using the first space is not consistent, it can be determined that the attribute confidentiality of the users using the first space exists, and whether to apply noise cancellation can be switched based on the determination result.

[0046] In addition, in the sound adjustment method involved in the 8th embodiment of the present disclosure, in the sound adjustment method described in the 6th embodiment, in the attribute estimation step, it is estimated whether the user using the first space is a child as the attribute of the user using the first space, and in the attribute confidentiality determination step, when it is estimated that the user using the first space is a child, it is determined that the attribute confidentiality of the user using the first space does not exist.

[0047] Thus, when the user using the first space is a child, it is possible to determine that the attribute confidentiality of the user using the first space exists, and whether to apply noise cancellation can be switched based on the determination result.

[0048] In addition, the sound adjustment method involved in the 9th mode of the present disclosure includes a situation confidentiality determination step in the sound adjustment method described in any one of the 6th to 8th modes, in which it is determined whether the situation confidentiality of the user who is estimated to use the first space exists, and in the sound propagation adjustment step, when the determination result in the utilization situation determination step indicates that the first space is used for work, noise elimination is applied, and when the determination result in the utilization situation determination step indicates that the first space is not used for work, based on whether the situation confidentiality of the first space exists and whether the attribute confidentiality of the user who uses the first space exists, it is switched whether to apply noise elimination.

[0049] Thus, when the first space is used for operations involving multiple operators, noise elimination can be applied. When the first space is not used for operations involving multiple operators, whether to apply noise elimination can be switched based on the determination results of whether situation confidentiality exists and whether attribute confidentiality exists.

[0050] In addition, in the sound adjustment method involved in the tenth embodiment of the present disclosure, in the sound adjustment method described in the ninth embodiment, noise elimination includes: a first noise elimination that reduces the listening level of the sound generated in the second space when being listened to in the first space, and a second noise elimination that reduces the listening level of the sound generated in the first space when being listened to in the second space. In the sound propagation adjustment step, when the determination result in the utilization status determination step indicates that the first space is used for work, the first noise elimination is applied, and when the determination result in the utilization status determination step indicates that the first space is not used for work, whether to apply the second noise elimination is switched based on whether the confidentiality of the status of the first space exists and whether the confidentiality of the attributes of the user using the first space exists.

[0051] Thus, when the first space is used for operations accompanied by multiple operators, the first noise elimination can be applied, and when the first space is not used for operations accompanied by multiple operators, whether to apply the second noise elimination can be switched based on the judgment results of whether situation confidentiality exists and whether attribute confidentiality exists.

[0052] Furthermore, in the sound adjustment method according to an eleventh aspect of the present disclosure, in the sound adjustment method according to any one of the first to tenth aspects, in the utilization status determination step, when the work is interrupted, it is determined that the first space is not being used for the work.

[0053] Thus, when the work is interrupted, it is possible to determine that the first space is not being used for the work, and switch whether to apply the noise cancellation based on the determination result.

[0054] In addition, in the sound adjustment method involved in the 12th embodiment of the present invention, in the sound adjustment method described in the 11th embodiment, the operation is remote sound output, and the remote sound output is performed by collecting sounds emitted by one or more operators in a first space including a user among multiple operators, and outputting the collected sounds in a space different from the first space via a network, and in the remote sound output, the output sounds are heard by one or more operators in different spaces among the multiple operators, and the interruption of the operation is that the collection of sounds in the first space is muted.

[0055] Thus, when the remote sound output is interrupted due to the muting of the sound collection in the first space, it is possible to determine that the first space is not used for the operation, and switch whether to apply noise cancellation based on the determination result. The remote sound output is performed by collecting sounds emitted by one or more operators in the first space including the user among multiple operators, and outputting the collected sounds in a space different from the first space via a network, and in the remote sound output, the output sounds are heard by one or more operators in different spaces among the multiple operators.

[0056] In addition, the sound adjustment method involved in the 13th mode of the present disclosure is in the sound adjustment method described in any one of the 1st to 12th modes, and further includes: an alternative prompt step, when the sound propagation adjustment step is switched to apply noise elimination, based on the sound after the listening level is reduced, the content expressed by the sound is output as an image.

[0057] This makes it possible to suppress problems in the case of allowing sound to pass through, and recognize the content represented by the sound by observing the output image.

[0058] Moreover, the program according to the 14th aspect of the present disclosure is a program for causing a computer to execute the sound adjustment method according to any one of the 1st to 13th aspects.

[0059] In this way, the same effect as the above-mentioned sound adjustment method is achieved through the computer.

[0060] In addition, the sound adjustment device involved in the 15th mode of the present disclosure includes: a situation estimation unit, which estimates the situation of users using the first space between the mutually adjacent first space and second space; a utilization situation determination unit, which determines whether the first space is used for operations of multiple operators including the users using the first space based on the estimated situation of the users using the first space; and a sound propagation adjustment unit, which switches whether to apply noise cancellation based on the determination result in the utilization situation determination unit, in which the listening level of the sound generated in one of the first space and the second space when being listened to in the other of the first space and the second space is reduced.

[0061] This achieves the same effect as the above-mentioned sound adjustment method.

[0062] In addition, the sound adjustment system involved in the 16th mode of the present disclosure comprises: a first microphone for collecting sounds in a first space, a second microphone for collecting sounds in a second space adjacent to the first space, a first speaker for outputting sounds in the first space, a second speaker for outputting sounds in the second space, a sound adjustment device, and a control device, the sound adjustment device comprising: a situation estimation unit for estimating the situation of a user using the first space; a utilization situation determination unit for determining whether the first space is used for operations of multiple operators including the user using the first space based on the estimated situation of the user using the first space; and a sound propagation adjustment unit for switching whether noise cancellation is applied based on a determination result in the utilization situation determination unit, in which the listening level of sounds generated in one of the first space and the second space when being listened to in the other of the first space and the second space is reduced, and the control device controls the first microphone, the second microphone, the first speaker, and the second speaker according to whether the noise cancellation switched by the sound propagation adjustment unit is applied.

[0063] This achieves the same effect as the above-mentioned sound adjustment method.

[0064] In addition, these overall or specific methods can also be implemented by devices, integrated circuits, computer programs or non-volatile recording media such as computer-readable CD-ROMs, and can also be implemented by any combination of devices, integrated circuits, computer programs and non-volatile recording media.

[0065] Hereinafter, the embodiments are described in detail with reference to the accompanying drawings as appropriate. In which, an overly detailed description is sometimes omitted. For example, a detailed description of a known matter or a repeated description of a substantially identical structure is sometimes omitted. This is to avoid the following description from becoming too lengthy so that it is easy for those skilled in the art to understand.

[0066] In addition, the inventor provides the drawings and the following description in order to enable those skilled in the art to fully understand the present disclosure, and it is not intended that the subject matter described in the claims be limited thereby.

[0067] (Implementation Method)

[0068] [constitute]

[0069] Figure 1 This is a diagram for explaining a space to which the sound adjustment system according to the embodiment is applied. Figure 2 : is a diagram showing an example of using the sound adjustment system according to the embodiment. Figure 2 The sound adjustment system 1 is viewed from above (see the following description). Figure 3 ) is applied to the graph of the space.

[0070] Figure 1 and Figure 2 Indicates the first space 201 and the second space 202 which are separated by the partition 13 and are adjacent to each other. The partition 13 is an example of a boundary separating the first space 201 and the second space 202. In addition to being separated by a movable partition 13, the first space 201 and the second space 202 may also be separated by a fixed wall. In addition, the first space 201 and the second space 202 may not be separated physically, as long as they are divided conceptually. In other words, there may be no physical separation, and a certain boundary of the continuous space may be used as a boundary, with one side being the first space 201 and the other side being the second space.

[0071] On the partition 13 arranged at the boundary between the first space 201 and the second space 202, the first speaker 12 is arranged toward the first space 201, the second speaker 12a is arranged toward the second space 202, the first microphone 11a is arranged toward the first space 201, and the second microphone 11 is arranged toward the second space 202. The first space 201 is, for example, a space for people to work (work space) and can be used by more than one person. For example, the first space 201 is used for the user of the first space 201 to perform a task. Sometimes the task is the user's personal business (working alone), and sometimes it is a meeting conducted by multiple participants (or also called operators) including the user. The second space 202 is, for example, a space shared by an unspecified number of people, such as a lobby, for people to come and go or take a break.

[0072] Figure 2 1 shows a part of the sound adjustment system 1 including the control device 100, the first microphone 11a, the second microphone 11, the first speaker 12, and the second speaker 12a. Figure 3The sound adjustment system 1 further includes a sound adjustment device 300 (see the following description) connected to the control device 100 and the like via a network. Figure 3 ), and several other devices.

[0073] The first microphone 11a is disposed on the side of the partition 13 close to the first space 201. The first microphone 11a detects the sound generated in the first space 201. In addition, the first microphone 11a is not limited to being disposed on the partition 13 as long as it can detect the sound generated in the first space 201, and may be disposed on the ceiling of the first space 201, on the wall disposed in the first space 201, on the floor of the first space 201, or on a table, chair, etc. disposed in the first space 201.

[0074] The second microphone 11 is disposed on the side of the partition 13 close to the second space 202. The second microphone 11 detects the sound generated in the second space 202. In addition, the second microphone 11 is not limited to being disposed on the partition 13 as long as it can detect the sound generated in the second space 202, and may be disposed on the ceiling of the second space 202, on the wall disposed in the second space 202, on the floor of the second space 202, or on a table, chair, etc. disposed in the second space 202.

[0075] The first speaker 12 is arranged on the side of the partition 13 close to the first space 201. The first speaker 12 is used to achieve noise cancellation (first noise cancellation) by outputting a sound for making the sound generated in the second space 202 difficult to be heard by people in the first space 201. Here, the sound generated in the second space 202 is, for example, a conversation of users using the second space 202, a sound caused by actions, an operating sound of a device configured in the second space 202, etc. The conversation of users using the second space 202 can be a conversation between multiple users using the second space 202, or a conversation between a user using the second space 202 and a user outside the second space 202 (i.e., a user in a different space) through a voice call using a communication terminal. In addition, the first speaker 12 can also have a speaker array composed of a plurality of speaker units arranged in a specific direction, and can also be capable of controlling directivity in a specific direction.

[0076] The second speaker 12a is arranged on the side of the partition 13 close to the second space 202. The second speaker 12a is used to achieve noise elimination (second noise elimination) by outputting a sound for making the sound generated in the first space 201 difficult to be heard by people in the second space 202. Here, the sound generated in the first space 201 is, for example, a conversation between users using the first space 201, a sound caused by an action, an operating sound of a device configured in the first space 201, etc. The conversation between users using the first space 201 can be a conversation between multiple users using the first space 201, or a conversation between a user using the first space 201 and a user outside the first space 201 (i.e., a user in a different space) through a voice call using a communication terminal. In addition, the second speaker 12a can also have a speaker array composed of a plurality of speaker units arranged in a specific direction, and can also be capable of controlling directivity in a specific direction.

[0077] Furthermore, as the second microphone 11 and the first speaker 12, the positions or directions of the second microphone 11 and the first speaker 12 may be adjusted and arranged so that the second microphone 11 is not likely to pick up the sound output by the first speaker 12. Similarly, as the first microphone 11a and the second speaker 12a, the positions or directions of the first microphone 11a and the second speaker 12a may be adjusted and arranged so that the first microphone 11a is not likely to pick up the sound output by the second speaker 12a.

[0078] In addition, as the second microphone 11 and the first speaker 12, the volume of the first speaker 12 or the sensitivity of the second microphone 11 may be adjusted so that the second microphone 11 is not likely to pick up the sound output by the first speaker 12. Similarly, as the first microphone 11a and the second speaker 12a, the volume of the second speaker 12a or the sensitivity of the first microphone 11a may be adjusted so that the first microphone 11a is not likely to pick up the sound output by the second speaker 12a.

[0079] The first microphone 11a, the second microphone 11a, the first speaker 12, and the second speaker 12a may not be arranged on the partition plate 13 as long as they are arranged so that the second microphone 11 does not easily pick up the sound output by the first speaker 12 and the first microphone 11a does not easily pick up the sound output by the second speaker 12a.

[0080] The control device 100 obtains the detection result from the second microphone 11, and generates a sound for the first noise cancellation such as a phase inversion sound output from the first speaker 12 based on the obtained detection result. Similarly, the control device 100 obtains the detection result from the first microphone 11a, and generates a sound for the second noise cancellation such as a phase inversion sound output from the second speaker 12a based on the obtained detection result.

[0081] Next, the specific configuration of the sound adjustment system 1 will be described.

[0082] Figure 3 This is a block diagram showing an example of the hardware configuration of the sound adjustment system according to the embodiment.

[0083] like Figure 3 As shown, the sound adjustment system 1 includes a camera device 401, a PC 402, a microphone 403, the sound adjustment device 300, a control device 100, a first microphone 11a, a second microphone 11, a first speaker 12, and a second speaker 12a as a hardware configuration.

[0084] The sound adjustment device 300 is implemented by, for example, a cloud server constructed on the network or an edge server installed in a building to which the first space 201 and the second space 202 belong. The sound adjustment device 300 is implemented by, for example, a processor, a main memory, a storage device, and a communication IF (Interface) (not shown).

[0085] The processor is a processor that executes a program stored in a storage device or the like.

[0086] The main memory is a volatile storage area used to temporarily store data generated during processing by the processor, used as a work area when the processor executes a program, or used to temporarily store data received by the communication IF.

[0087] The storage device is a nonvolatile storage device that stores various data such as programs, etc. The storage device stores various data generated as a result of processing performed by the processor, for example.

[0088] The communication IF is a communication interface for transferring data with an external device via a communication network.

[0089] As shown in the figure, the sound adjustment device 300 includes an acquisition unit 301, a state estimation unit 302, an attribute estimation unit 303, a use state determination unit 304, a state confidentiality determination unit 305, an attribute confidentiality determination unit 306, a sound propagation adjustment unit 307, and an output unit 308, as a functional configuration that functions by executing various programs by the above-mentioned processor, main memory, storage device, and communication IF (Interface). In other words, the sound adjustment device 300 has programs for realizing the above-mentioned various functional configurations in addition to the processor, main memory, storage device, and communication IF (Interface).

[0090] The acquisition unit 301 is connected to the camera device 401, the PC 402, and the microphone 403, and acquires information used for state estimation and attribute estimation from these devices. The information used for state estimation and attribute estimation is, for example, image information in the first space 201 from the camera device 401 installed in the first space 201, a work log of the PC 402 brought into the first space 201, and sound information in the first space 201 from the microphone 403 installed in the first space 201. The microphone 403 may also be the first microphone 11a. In addition, the camera device 401, the PC 402, and the microphone 403 may not all be provided.

[0091] Information used for state estimation and attribute estimation is acquired by the acquisition unit 301 , and is output to the state estimation unit 302 and the attribute estimation unit 303 .

[0092] The status estimation unit 302 estimates the status of the user using the first space 201. The status of the user using the first space 201 is information indicating whether the user is performing a task such as a business. In the status estimation, for example, a work log from the PC 402 is used. In the status estimation unit 302, if the user performs a task (operates an application, etc.) using the PC 402, it is estimated that the user is performing the task. The estimation result of the status estimation unit 302 is output to the usage status determination unit 304 and the status confidentiality determination unit 305.

[0093] The attribute estimation unit 303 estimates the attributes of the user using the first space 201. The attributes of the user using the first space 201 are information such as the organizational attributes (attributes related to affiliation) and social attributes of the user. For example, an image from the camera device 401 is used in the attribute estimation. When the user (e.g., face) matches the registered user, the attribute estimation unit 303 estimates that the user belongs to one organization. When the user does not match the registered user, the attribute estimation unit 303 estimates that the user belongs to another organization. The estimation result of the attribute estimation unit 303 is output to the attribute confidentiality determination unit 306.

[0094] In attribute estimation, for example, the sound from the microphone 403 may be used. The attribute estimation unit 303 estimates that the user is a child if the user (for example, the sound and content) meets a predetermined condition, and estimates that the user is not a child otherwise.

[0095] The usage status determination unit 304 determines whether the first space 201 is used for work by multiple operators. The determination result is output to the sound propagation adjustment unit 307. The work by multiple operators is, for example, a conversation completed in the first space 201, or a remote conversation accompanied by two-way remote sound output (so-called online conference).

[0096] Remote sound output refers to the process of sending sounds collected (recorded) in one space via a network and outputting (reproducing) them in another space, so that speech in one space can be heard in another space.

[0097] For example, by collecting (recording) sounds emitted by one or more operators in the first space 201 including users of the first space 201 among multiple operators, and outputting (reproducing) the collected sounds in a space different from the first space 201 via a network, and by collecting (recording) sounds emitted by one or more operators in other spaces (not shown) among multiple operators, and outputting (reproducing) the collected sounds in the first space 201 via a network, a remote conversation is conducted between the first space 201 and other spaces.

[0098] In such a remote conversation, the output sound is heard in the other space by remote sound output from the first space 201, but when, for example, a noise from the second space 202 is transmitted to the first space 201, the noise is also heard in the other space by remote sound output. In this way, the transmission of sound from the second space 202 to the first space 201 may hinder the work of multiple operators.

[0099] The situation confidentiality determination unit 305 determines whether situation confidentiality exists based on the estimation result of the situation estimation unit 302, and the situation confidentiality indicates that the situation of the user using the first space 201 should be kept confidential. The determination result is output to the sound propagation adjustment unit 307. For example, if the user using the first space 201 is performing a task, the situation confidentiality determination unit 305 determines that situation confidentiality exists. When situation confidentiality exists, a sound that should be kept confidential (should not be leaked) may be generated in the first space 201, and therefore should not be heard by the user using the second space 202. In other words, the propagation of sound from the first space 201 to the second space 202 is not preferable from the viewpoint of confidentiality.

[0100] The attribute confidentiality determination unit 306 determines whether attribute confidentiality exists, which indicates that the attribute of the user using the first space 201 is an attribute that should be kept confidential, based on the estimation result of the attribute estimation unit 303. The determination result is output to the sound propagation adjustment unit 307. The attribute confidentiality determination unit 306 determines that attribute confidentiality exists, for example, when the users using the first space 201 include users of different affiliations, that is, when the information related to the affiliations of all users using the first space 201 is not consistent. When attribute confidentiality exists, a conversation that should be kept confidential (should not be leaked) may occur in the first space 201. As an example, the first space 201 and the second space 202 are spaces under the jurisdiction of Company A, and in the first space 201, employees belonging to Company A and employees belonging to Company B, which is different from Company A, are having a meeting. In this case, it is not preferable that the speech of the employee belonging to Company B is heard by another employee belonging to Company A who happens to be in the second space 202. The same is true for a case where an employee belonging to Company A is having a meeting with an employee belonging to Company A who is in a different department from the employee in the first space 201. In this way, when attribute confidentiality exists, the user using the second space 202 should not hear the sound. In other words, the propagation of sound from the first space 201 to the second space 202 is not preferable from the viewpoint of confidentiality.

[0101] On the other hand, the attribute confidentiality determination unit 306 determines that attribute confidentiality does not exist, for example, when the user using the first space 201 is a child. As an example, a case where two spaces in a family are the first space 201 and the second space 202 is described. In this family, it is assumed that a child lives in the first space 201 and the guardian of the child works remotely in the second space 202. In this case, the sound generated in the first space 201 is a sound generated as a result of the child's actions including vocalization. The child's actions sometimes include actions that require emergency response from the guardian, so there is a possibility that sounds that should be heard by the user using the second space 202 are generated. In other words, sometimes it is necessary for sounds to be transmitted from the first space 201 to the second space 202.

[0102] The sound propagation adjustment unit 307 switches whether to apply noise cancellation, in which the listening level of the sound generated in one of the first space 201 and the second space 202 is reduced when it is heard in the other of the first space 201 and the second space 202, based on the determination results of the use situation determination unit 304, the situation confidentiality determination unit 305, and the attribute confidentiality determination unit 306. More specifically, the sound propagation adjustment unit 307 switches whether to apply the first noise cancellation, in which the listening level of the sound generated in the second space 202 is reduced when it is heard in the first space 201, based on the determination results of the use situation determination unit 304. In addition, the sound propagation adjustment unit 307 switches whether to apply the second noise cancellation, in which the listening level of the sound generated in the first space 201 is reduced when it is heard in the second space 202, based on the determination results of the situation confidentiality determination unit 305 and the attribute confidentiality determination unit 306. When switching whether to apply noise cancellation, the sound propagation adjustment unit 307 outputs a control signal for performing the control to the output unit 308 .

[0103] The output unit 308 is connected to the control device 100 , and transmits the control signal output from the sound propagation adjustment unit 307 to the control device 100 .

[0104] The control device 100 controls the first microphone 11a, the second microphone 11, the first speaker 12, and the second speaker 12a according to the control signal, and selectively controls whether to apply or not apply the first noise canceling and whether to apply or not apply the second noise canceling.

[0105] [action]

[0106] Next, the operation of the sound adjustment system 1 according to the embodiment will be described.

[0107] Figure 4 This is a flowchart showing an example of the operation of the sound adjustment system involved in the embodiment.

[0108] First, the acquisition unit 301 acquires information for state estimation and attribute estimation from the camera device 401 , the PC 402 , the microphone 403 , and the like ( S101 ).

[0109] Next, the situation estimation unit 302 estimates the situation of the user using the first space 201 (situation estimation step S102 ). In addition, the attribute estimation unit 303 estimates the attribute of the user using the first space 201 (attribute estimation step S103 ).

[0110] Next, the usage status determination unit 304 determines whether the first space 201 is used for operations (here, remote conversations) performed by multiple operators (usage status determination step S104). Specifically, the usage status determination unit 304 determines whether the first space 201 is used for remote conversations performed by multiple operators based on whether there are more than one PC 402 working in the first space 201 and connected to more than one PC in other spaces via a predetermined remote conversation application. When the usage status determination unit 304 determines that the first space 201 is used for remote conversations performed by multiple operators (S104: Yes), the process proceeds to step S105. When it is determined that the first space 201 is not used for remote conversations performed by multiple operators (S104: No), the process proceeds to step S106.

[0111] In step S105, the sound propagation adjustment unit 307 outputs a control signal for switching to the first noise canceling, and the output unit 308 transmits it to the control device 100. Then, in the control device 100, the reverse phase sound is generated based on the sound collected by the second microphone 11, and the first speaker 12 reproduces it, thereby applying the first noise canceling. In addition, if the first noise canceling is already applied, the control device 100 maintains this state and controls the second microphone 11 and the first speaker 12.

[0112] In step S106, the situation confidentiality determination unit 305 determines whether the situation confidentiality exists (situation confidentiality determination step), and the attribute confidentiality determination unit 306 determines whether the attribute confidentiality exists (attribute confidentiality determination step), thereby determining whether at least one of the situation confidentiality and the attribute confidentiality exists. If it is determined that at least one of the situation confidentiality and the attribute confidentiality exists (S106: Yes), the process proceeds to step S107. If it is determined that neither the situation confidentiality nor the attribute confidentiality exists (S106: No), the process ends.

[0113] In step S107, the sound propagation adjustment unit 307 outputs a control signal for switching to the second noise cancellation, and the output unit 308 sends it to the control device 100. Then, in the control device 100, the reverse phase sound is generated based on the sound collected by the first microphone 11a, and the second speaker 12a is reproduced, so that the second noise cancellation is applied. In addition, if the second noise cancellation is already applied, the control device 100 maintains this state and controls the first microphone 11a and the second speaker 12a. Step S105 and step S107 are also collectively referred to as a sound propagation adjustment step.

[0114] Below, refer to Figures 5 to 10 A specific example of a result of the above-mentioned operation will be described. Figures 5 to 10The figures respectively show the first to sixth examples of the sound propagation adjustment implemented by the sound adjustment system involved in the embodiment. In addition, in each figure, the first space and the second space are shown, and when there is sound propagating across the boundary separating the first space and the second space, the arrow in that direction is indicated by a solid line. In addition, when there is no sound propagating across the boundary (when noise cancellation is applied), the arrow in that direction is indicated by a dotted line and a cross is added.

[0115] exist Figure 5 In the example shown, the user in the first space is having a remote conversation with the user in the other space, so the answer in the use status determination step S104 is yes. In addition, the user in the other space belongs to a different group than the user in the first space. In the figure, the different group is indicated by dotted shadows. Figure 6 to Figure 10 The same is true in Figure 5 In the example of , it is determined that the attribute confidentiality exists, so the answer is yes in step S106. Figure 5 In the example, the propagation of sound from the second space to the first space is reduced at a listening level by the first noise cancellation, and the propagation of sound from the first space to the second space is reduced at a listening level by the second noise cancellation.

[0116] In addition, here, in the case where the propagation of sound reduces the listening level by noise cancellation, that is, in the above example, when it is switched to apply noise cancellation in the sound propagation adjustment step, a situation may occur in which it is desired to convey only the content expressed by the sound without the propagation of sound. For example, if a task with a higher priority occurs, or an emergency contact is required between the first space and the second space, and noise cancellation is applied and it is difficult to communicate by sound, a problem sometimes occurs. Therefore, as an example, it is assumed that there is a sound whose listening level is reduced due to the application of noise cancellation in the transmission of sound from the second space to the first space, and the following description is made. For example, based on the sound whose listening level is reduced, the content expressed by the sound can be converted into text information by a speech recognition model, etc., and an image representing the text information can be generated and output, so that the generated image is displayed (prompted) on the PC of the user using the first space, etc. In this way, the influence of the propagation of the sound itself on the remote conversation can be suppressed, and only the content expressed by the sound can be conveyed, which effectively plays a role in the above situation.

[0117] exist Figure 6In the example shown, the user in the first space is having a remote conversation with the user in the other space, but the sound collection in the first space is muted. When the sound collection in the first space is muted, it is considered that the operation (remote conversation) is interrupted. When the remote conversation is interrupted, the first space is not used for the remote conversation, so the result is No in the utilization status determination step S104. In addition, the users in the other spaces belong to different groups from the users in the first space. Therefore, in Figure 6 In the example of , it is determined that the attribute confidentiality exists, so the answer is yes in step S106. Figure 6 In the example, the propagation of sound from the second space to the first space is propagated as it is without applying the first noise cancellation, and the propagation of sound from the first space to the second space is reduced in listening level by the second noise cancellation.

[0118] exist Figure 7 In the example shown, a plurality of users are working in the first space (here, not remote conversations), so the answer is yes in the usage status determination step S104. In addition, the users in the first space include users who belong to a different group from other users in the first space. Figure 7 In the example of , it is determined that the attribute confidentiality exists, so the answer is yes in step S106. Figure 7 In the example, the propagation of sound from the second space to the first space is reduced in listening level by the first noise cancellation, and the propagation of sound from the first space to the second space is reduced in listening level by the second noise cancellation.

[0119] exist Figure 8 In the example shown, a plurality of users are having a conversation in the first space, so the answer is yes in the usage status determination step S104. In addition, among the users in the first space, there are no users belonging to different groups. Figure 8 In the example of , it is determined that the attribute confidentiality does not exist, but since the user is performing a task in the first space, the answer is yes in step S106. Figure 8 In the example, the propagation of sound from the second space to the first space is reduced in listening level by the first noise cancellation, and the propagation of sound from the first space to the second space is reduced in listening level by the second noise cancellation. In addition, for example, the situation confidentiality determination unit 305 may not be provided, and if it is determined in step S106 whether only attribute confidentiality exists, then the result is no in step S106, and therefore, the second noise cancellation is not applied to the propagation of sound from the first space to the second space, and the sound is propagated as it is.

[0120] exist Fig. 9 In the example shown, in the first space Figure 5 Similarly, the user is having a remote conversation with a user in another space, so the answer is yes in the usage status determination step S104. Figure 5 Likewise, users are having remote conversations with users in other spaces. Fig. 9 In the example, two sound adjustment systems are used: a sound adjustment system for the first space and a sound adjustment system for the second space (hereinafter referred to as other sound adjustment systems). That is, from the perspective of other sound adjustment systems, the first space in the figure is equivalent to the adjacent second space. Fig. 9 In the example of , it is assumed that the situation confidentiality determination unit 305 is not provided, and only the existence of attribute confidentiality is determined in step S106. Fig. 9 In the example, the users of the other space belong to the same group as the users of the first space, so it is determined that the attribute confidentiality does not exist, and thus the answer is yes in step S106. However, in the other sound adjustment system, the answer is yes in the utilization determination step S104, so as a result, the propagation of sound from the second space to the first space is reduced in listening level by the first noise elimination, and the propagation of sound from the first space to the second space is reduced in listening level by the first noise elimination of the other sound adjustment system.

[0121] exist Fig.10 In the example shown, a child who is a user lives in the first space, and the user is performing a task in the second space. Fig.10 In the example shown, it is also related to Fig. 9 Similarly, it is assumed that two groups of sound adjustment systems, namely the sound adjustment system for the first space and the other sound adjustment systems, are used. In the first space, the user is not performing any tasks, etc., and there are no operations being performed by multiple operators. Therefore, the result is No in the utilization status determination step S104, and since it is determined that the attribute confidentiality does not exist, the result is No in step S106. On the other hand, in the other sound adjustment systems, the user is performing tasks, but there are no operations being performed by multiple operators. Therefore, the result is No in the utilization status determination step S104, and since it is determined that the status confidentiality exists, the result is Yes in step S106. As a result, in Fig.10 In the example shown, the propagation of sound from the second space to the first space is reduced in listening level by the second noise elimination of other sound adjustment systems, while the propagation of sound from the first space to the second space is neither affected by the second noise elimination nor by the first noise elimination of other sound adjustment systems, but is propagated as is.

[0122] (Other embodiments)

[0123] The sound adjustment device according to the embodiment shows an example implemented by a cloud server or an edge server. However, for example, it can also be implemented by a dedicated information processing device placed in the first space or the second space. The dedicated information processing device can also be built into the partition. If an information processing device serving as the sound adjustment device is built into the partition, it is also possible to include the first and second microphones and the first and second speakers, and implement a sound adjustment system as one partition-type device.

[0124] In addition, in the above embodiment, each component can also be constituted by dedicated hardware, or can be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0125] In addition, each component can also be a circuit (or an integrated circuit). These circuits can either form one circuit as a whole or be individual circuits respectively. In addition, each of these circuits can be a general-purpose circuit or a dedicated circuit respectively.

[0126] In addition, the whole or a specific mode of the present disclosure can also be implemented by a system, a device, a method, an integrated circuit, a computer program, or a non-volatile recording medium such as a computer-readable CD-ROM. In addition, it can also be implemented by any combination of a system, a device, a method, an integrated circuit, a computer program, and a computer-readable non-volatile recording medium.

[0127] For example, the present disclosure can be implemented as a sound adjustment method executed by a sound adjustment device (a computer or a DSP), or can be implemented as a program for causing a computer or a DSP to execute the above sound adjustment method.

[0128] In addition, in the above embodiment, the processing executed by a specific processing unit can also be executed by another processing unit. In addition, the order of multiple processes in the operation of the sound adjustment system described in the above embodiment can also be changed, and multiple processes can also be executed in parallel.

[0129] In addition, modes obtained by applying various modifications conceivable by those skilled in the art to each embodiment, or modes implemented by arbitrarily combining the components and functions in each embodiment without departing from the gist of the present disclosure, are all included in the present disclosure.

[0130] Industrial Applicability

[0131] The present disclosure is useful as a sound adjustment system or the like for providing an appropriate sound environment to users of a utilization space.

[0132] Explanation of Reference Numerals:

[0133] 1Sound adjustment system

[0134] 11. Second Microphone

[0135] 11a1st Microphone

[0136] 121st speaker

[0137] 12a 2nd speaker

[0138] 13 Partition

[0139] 100 control devices

[0140] 201 No.1 Space

[0141] 202 The Second Space

[0142] 300 sound adjustment device

[0143] 301 Acquisition Department

[0144] 302 Situation Assessment Department

[0145] 303 Attribute Estimation Department

[0146] 304 Utilization Status Determination Unit

[0147] 305 Confidentiality Assessment Department

[0148] 306 Attribute Confidentiality Determination Unit

[0149] 307 Sound Propagation Adjustment Department

[0150] 308 Output

[0151] 401 Camera Device

[0152] 402PC

[0153] 403 Microphone

Claims

1. A sound adjustment method, include: a state estimating step of estimating a state of a user using the first space of the first space and the second space adjacent to each other; a utilization status determination step of determining whether the first space is used for work by a plurality of workers including the user who uses the first space, based on an estimated status of the user who uses the first space; as well as The sound propagation adjustment step switches whether to apply noise cancellation based on the determination result in the utilization status determination step, in which the listening level of the sound generated in one of the first space and the second space when being heard in the other of the first space and the second space is reduced.

2. The sound adjustment method according to claim 1, The noise cancellation includes a first noise cancellation in which a listening level of the sound generated in the second space when the sound is listened to in the first space is reduced.

3. The sound adjustment method according to claim 1, The noise cancellation includes a second noise cancellation in which a listening level of the sound generated in the first space when heard in the second space is reduced.

4. The sound adjustment method according to claim 1, include: a situation confidentiality determination step of determining whether confidentiality of the situation of the user using the first space exists based on the estimated situation of the user using the first space, In the sound propagation adjustment step, whether to apply the noise cancellation is switched based on whether confidentiality of the situation of the user using the first space exists.

5. The sound adjustment method according to claim 4, In the state estimation step, as the state of the user using the first space, it is determined whether the user is performing a task, In the situation confidentiality determination step, when the estimated situation of the user using the first space indicates that the user is performing a task, it is determined that confidentiality of the situation of the user using the first space exists.

6. The sound adjustment method according to claim 1, further comprising: include: an attribute estimating step of estimating attributes of users utilizing the first space; as well as an attribute confidentiality determination step of determining whether confidentiality of attributes of users using the first space exists based on the estimated attributes of the users using the first space, In the sound propagation adjustment step, whether to apply the noise cancellation is switched based on whether the attribute confidentiality of the user using the first space exists.

7. The sound adjustment method according to claim 6, In the attribute estimation step, information on the affiliation of each of all workers involved in work using the first space is estimated as an attribute of the user using the first space, In the attribute confidentiality determination step, when the information on the affiliations of all the operators is not consistent, it is determined that the attribute confidentiality of the user using the first space exists.

8. The sound adjustment method according to claim 6, In the attribute estimation step, whether the user using the first space is a child is estimated as an attribute of the user using the first space, In the attribute confidentiality determination step, when it is estimated that the user using the first space is a child, it is determined that the attribute confidentiality of the user using the first space does not exist.

9. The sound adjustment method according to any one of claims 6 to 8, include: a status confidentiality determination step of determining whether the estimated status confidentiality of the user utilizing the first space exists; In the sound propagation adjustment step, When the determination result in the utilization status determination step indicates that the first space is utilized for the operation, the noise elimination is applied, When the determination result in the utilization status determination step indicates that the first space is not used for the task, whether the noise cancellation is applied is switched based on whether the confidentiality of the status of the first space exists and whether the confidentiality of the attributes of the user using the first space exists.

10. The sound adjustment method according to claim 9, The noise cancellation include: a first noise cancellation for reducing a listening level of the sound generated in the second space when the sound is listened to in the first space; and a second noise canceller that reduces the listening level of the sound generated in the first space when it is listened to in the second space, In the sound propagation adjustment step, When the determination result in the utilization status determination step indicates that the first space is utilized for the operation, the first noise elimination is applied, When the determination result in the utilization status determination step indicates that the first space is not used for the operation, whether the second noise cancellation is applied is switched based on whether the confidentiality of the status of the first space exists and whether the confidentiality of the attributes of the user using the first space exists.

11. The sound adjustment method according to claim 1, In the utilization status determination step, when the work is interrupted, it is determined that the first space is not being used for the work.

12. The sound adjustment method according to claim 11, The operation is remote sound output, and the remote sound output is performed by collecting sounds emitted by one or more operators in the first space including the user among the plurality of operators, and outputting the collected sounds in a space different from the first space via a network, and in the remote sound output, the output sounds are heard by one or more operators in the different space among the plurality of operators, The interruption of the operation means that the collection of sound in the first space is muted.

13. The sound adjustment method according to claim 1, further comprising: include: In the alternative presentation step, when the application of the noise canceling is switched in the sound propagation adjustment step, the contents expressed by the sound are output as an image based on the sound whose listening level is reduced.

14. A program for causing a computer to execute the sound adjustment method according to claim 1.

15. A sound adjustment device, comprising: a state estimation unit that estimates a state of a user who uses the first space, of the first space and the second space that are adjacent to each other; a usage status determination unit that determines whether the first space is used for work by a plurality of workers including the user who uses the first space, based on an estimated status of the user who uses the first space; as well as The sound propagation adjustment unit switches whether to apply noise cancellation based on the determination result in the utilization status determination unit, in which the listening level of the sound generated in one of the first space and the second space when being heard in the other of the first space and the second space is reduced.

16. A sound adjustment system comprising: The first microphone collects the sound in the first space; a second microphone for collecting sounds in a second space adjacent to the first space; a first speaker for outputting sound in the first space; a second speaker for outputting sound in the second space; Sound adjustment device; as well as Control device, The sound adjustment device comprises: a state estimation unit that estimates a state of a user using the first space; a usage status determination unit that determines whether the first space is used for work by a plurality of workers including the user who uses the first space, based on an estimated status of the user who uses the first space; as well as a sound propagation adjustment unit that switches whether to apply noise cancellation based on the determination result of the utilization status determination unit, in which a hearing level of the sound generated in one of the first space and the second space when heard in the other of the first space and the second space is reduced, The control device controls the first microphone, the second microphone, the first speaker, and the second speaker according to whether the noise canceling switched by the sound propagation adjustment unit is applied.

Images