Information processing device, information processing method, and program

Abstract

The present disclosure pertains to an information processing device, an information processing method, and a program that make it possible to improve the efficiency of adjusting an impulse response. The level control unit changes the levels of the first sound component and the second sound component in response to adjusting parameters related to the first sound component and the second sound component included in the first impulse response. The technology according to the present disclosure can be applied, for example, to a remote concert system.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program that can make an impulse response adjustment work efficient. Background Art

[0002] There is known a remote ensemble system that enables a plurality of performers to perform an ensemble in a state where the plurality of performers are respectively located at remote locations.

[0003] For example, Patent Document 1 discloses a remote ensemble system that realizes high-level ensemble of multiple performers at a remote location by convolving acoustic signals of multiple co-performing users with a head-related transfer function that matches the positional relationship between users in a virtual space.

[0004] There is a case where, in such a remote ensemble system, a performer (user) listens to an acoustic signal obtained by convolving an impulse response of a sound field reproduction into a microphone (mike) input.

[0005] [Citation List]

[0006] [Patent Document]

[0007] [PTL 1]

[0008] WO 2022 / 196073 Summary of the invention

[0009] [Technical issues]

[0010] When the user adjusts the transfer characteristics from the sound source to the user's ears, for example, the absolute volume is a characteristic that changes as the impulse response is adjusted, and therefore, it is necessary to repeat fine adjustments until these adjustment items converge to optimal values.

[0011] Considering this situation, the present disclosure can make the impulse response adjustment work efficiently.

[0012] [Technical solutions to technical problems]

[0013] An information processing device according to the present disclosure is an information processing device including a level control unit that changes the level of a first sound component and the level of a second sound component in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response.

[0014] The information processing method according to the present disclosure is an information processing method including: at an information processing device, in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response, changing the level of the first sound component and the level of the second sound component.

[0015] A program according to the present disclosure is a program that causes a computer to execute a process of changing a level of a first sound component and a level of a second sound component in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response.

[0016] According to the present disclosure, a level of the first sound component and a level of the second sound component are changed in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 : is a diagram showing a configuration example of a remote ensemble system according to an embodiment of the present technology.

[0018] Figure 2 is a diagram showing an example of a device provided in a compartment.

[0019] Figure 3 is a diagram showing an example of a sound component of an impulse response.

[0020] Figure 4 is a diagram showing an example of a UI for adjustment of a conventional assumable impulse response.

[0021] Figure 5 is a diagram illustrating an example of a UI for adjustment of an impulse response according to the present disclosure.

[0022] Figure 6 is a diagram illustrating an example of a UI for adjustment of an impulse response according to the present disclosure.

[0023] Figure 7 is a diagram illustrating an example of a UI for adjustment of an impulse response according to the present disclosure.

[0024] Figure 8 is a block diagram showing an example of a functional configuration of an information processing apparatus.

[0025] Fig. 9 This is a flowchart for explaining the flow of impulse response adjustment processing.

[0026] Fig.10 This is a flowchart for explaining the flow of level control matching an adjustment item.

[0027] Fig.11This is a flowchart for explaining the flow of level control matching an adjustment item.

[0028] Fig.12 is a block diagram showing a configuration example of hardware of a computer. DETAILED DESCRIPTION

[0029] Hereinafter, a mode for carrying out the present disclosure (hereinafter referred to as an embodiment) will be described. Note that the description will be given in the following order.

[0030] 1. Configuration and issues of remote ensemble system

[0031] 2. Example of UI for impulse response adjustment

[0032] 3. Configuration of information processing device and impulse response adjustment process

[0033] 4. Application Examples

[0034] 5. Computer configuration examples

[0035] <1. Remote Ensemble System Configuration and Issues>

[0036] Figure 1 : is a diagram showing a configuration example of a remote ensemble system according to an embodiment of the present technology.

[0037] Figure 1 The remote ensemble system 1 shown in FIG. 1 is a system for a so-called remote ensemble performed by performers at a remote site.

[0038] Figure 1 The example in shows players P1 to P4 as players of an orchestra. The instruments played by players P1 and P2 are violins, and the instrument played by player P3 is a cello. The instrument played by player P4 is a trumpet.

[0039] Note that the number of performers is not limited to four, and in practice the remote ensemble is performed by more performers using more types of instruments. The number of performers varies depending on the band formation.

[0040] Figure 1The remote ensemble system 1 is configured by connecting a plurality of information processing devices used by performers P1 to P4 to a transmission control device 10. The transmission control device 10 and each information processing device may be connected by wired communication, or may be connected by wireless communication. In the case of connecting the transmission control device 10 and the plurality of information processing devices used by performers P1 to P4 by wired communication, a predetermined standard such as a universal serial bus (USB) cable may be used. In addition, in the case of connecting the transmission control device 10 and the plurality of information processing devices used by performers P1 to P4 by wireless communication, any communication protocol such as Wi-Fi (registered trademark) or Bluetooth (registered trademark) may be used.

[0041] Performers P1 to P4 perform in a remote space. For example, different compartments prepared in a studio are used as performance spaces. Figure 1 , the dotted rectangles surrounding the performers P1 to P4 indicate that the performers P1 to P4 perform in respective different booths.

[0042] Figure 2 is a diagram showing an example of a device provided in a compartment.

[0043] like Figure 2 As shown, the booth of the performer P1 is provided with a headset 110, a microphone (mike) 120, and an information processing device 130. The headset 110 and the mike 120 are connected to the information processing device 130 configured as a personal computer (PC), a smart phone, a tablet terminal, etc. The mike 120 can be directly connected to the transmission control device 10 as appropriate. The headset 110, the mike 120, and the information processing device 130 can each be connected by wire or wirelessly.

[0044] The headphone 110 is an output device equipped by the head of the player P1. The headphone 110 outputs the performance sound of the player P1 and the performance sound of the co-player under the control of the information processing device 130. In-ear headphones (earphones in the ear) may be used as the output device instead of the headphone.

[0045] The microphone 120 collects the performance sound of the performer P1 (the sound from the instrument performed by the performer P1).

[0046] The booths of the players P2 to P4 are each provided with three devices of a headphone 110, a microphone 120, and an information processing device 130, similarly to the booth of the player P1.

[0047] As described above, in the remote ensemble system 1 , each performer is equipped with the headphone 110 , and performs facing the microphone 120 while listening to the performance sound output from the headphone 110 .

[0048] This remote ensemble system 1 enables performers to immerse themselves in their own performance by virtually reproducing the actual performance environment. For this virtual reproduction, not only the impulse response of the sound field but also the absolute volume needs to be accurately reproduced, which is different from the sound field reproduction for conventional appreciation of content such as movies and music.

[0049] Furthermore, at the time of performance during sound field reproduction, the distance between the microphone 120 that collects the sound from the musical instrument and the headphone 110 equipped by the performer's head becomes close. Here, in the case where an open-type headphone is used as the headphone 110, howling or the like may occur due to acoustic feedback between the headphone 110 and the microphone 120. In order to avoid this problem, in the case of using a closed-type headphone or an in-ear headphone, the performer cannot directly listen to the sound from his or her own musical instrument, and therefore listens to an acoustic signal obtained by convolving an impulse response of the sound field reproduction including the direct sound into the microphone input via the headphone.

[0050] However, regarding the transfer function (transfer characteristics) from a musical instrument as a sound source to both ears of a player, it is known that, for example,

[0051] The propagation distance is very short, and the instrument itself has a complex radiation pattern

[0052] · Components transmitted through bone conduction etc. also affect hearing.

[0053] In view of this, the transfer function obtained by the measurement or calculation performed in advance already causes the performer to have a subjective feeling of discomfort.

[0054] Furthermore, reproduction of absolute volume requires calibration of sensitivity within a single playback environment and is already difficult in practice in many cases.

[0055] Therefore, performers need to carefully adjust the

[0056] "Absolute volume"

[0057] · “Level of direct sound from one’s own instrument” (hereinafter, referred to simply as “level of direct sound”)

[0058] • “Level of indirect sound from own instrument” (hereinafter, simply referred to as “level of indirect sound”).

[0059] Figure 3 1 is a diagram illustrating an example of sound components of an impulse response for a playback sound collected from a musical instrument played by a player through a microphone 120 and played back through a headphone 110 mounted on the player's head.

[0060] like Figure 3 As shown, the sound components of the impulse response can be roughly classified into "direct sound" and "indirect sound". In addition, the "indirect sound" can be classified into "early reflection sound" and "late reverberation sound". When finely adjusting the impulse response, the player adjusts the level of each of these adjustment items (parameters).

[0061] On the other hand, the “absolute volume” is determined based on the sum of the “energy of direct sound” and the “energy of indirect sound”, and therefore, the “level of direct sound” as an adjustment item is not completely unrelated to the “absolute volume”. Therefore, in the case of adjusting the “level of direct sound”, it becomes necessary to readjust the “absolute volume”, and conversely, in the case of adjusting the “absolute volume”, it becomes necessary to readjust the “level of direct sound”.

[0062] Due to the correlation between these adjustment items (parameters), the performer needs to repeat fine adjustments until all of these adjustment items converge to the optimal value.

[0063] Here, the “level of direct sound” adopts a value such as the amplitude of the direct sound, and the “energy of direct sound” adopts a value obtained by, for example, cumulatively adding the squares of the sample values ​​of the impulse response used for signal processing over the time segment of the direct sound. The “level of indirect sound” and the “energy of indirect sound” can be defined similarly to the direct sound. In addition, the “ratio of direct sound to indirect sound” described later is, for example, the ratio of the energy of the direct sound to the energy of the indirect sound. The “level” and “energy” of each of the direct sound and the indirect sound are not limited to these, and can be defined based on other physical quantities related to each sound.

[0064] <2. Example of UI for Adjustment of Impulse Response>

[0065] Hereinafter, an example of a user interface (UI) used for adjustment of the above-mentioned impulse response will be described.

[0066] Figure 4 is a diagram showing an example of a UI for adjustment of a conventional assumable impulse response.

[0067] Figure 4 An adjustment screen Ad1 presented as a UI for adjustment of an impulse response is shown.

[0068] The adjustment screen Ad1 is provided with sliders indicating respective values ​​of “absolute volume”, “level of direct sound”, and “level of indirect sound” as adjustment items (parameters), and accepts adjustment of these parameters.

[0069] Figure 4An example of the processing of the adjustment work in the case of adjusting the level of the direct sound in the adjustment screen Ad1 is shown. More specifically, the "level of the direct sound" is adjusted by the user's (player's) operation. In this case, the "absolute volume" related to the level of the direct sound changes, and therefore, the "absolute volume" is adjusted by the user's operation. Then, the "level of the direct sound" changes, and therefore, the "level of the direct sound" is adjusted again by the user's operation. In this case, the "absolute volume" changes again, and therefore, the "absolute volume" is adjusted again by the user's operation.

[0070] As described above, “absolute volume” and “level of direct sound” are not independent of each other, and therefore, it is necessary for the user to finely adjust “absolute volume” and “level of direct sound” alternately until each of them converges to an optimal value.

[0071] In contrast, the technology according to the present disclosure is based on the following two viewpoint conversion and aggregation adjustment items (parameters) to make the impulse response adjustment work efficient.

[0072] (1) Subjective ease of adjustment for the performer

[0073] Adjustment items such as "level of direct sound" and "level of indirect sound" are classified for the convenience of description in the case of synthesized impulse response, and are not directly associated with the sound that the performer (user) finally listens to. That is, the performer cannot separate and listen to only the "direct sound" from the sound finally output from the headphone, and cannot separate and listen to only the "indirect sound".

[0074] This is a factor that makes it difficult for the player to subjectively understand how the adjustment operation has been reflected in the adjustment result even when the player performs the adjustment operation based on the classification such as “direct sound” and “indirect sound”.

[0075] In contrast, the adjustment items (parameters) are converted into “items that allow the player to easily and subjectively grasp how the adjustment operation has been reflected in the adjustment result” to make the impulse response adjustment work efficient.

[0076] (2) Reduction of the extra dimensions of adjustment items

[0077] "Absolute volume", "level of direct sound" and "level of indirect sound" as adjustment items are not independent of each other, and therefore, the number of dimensions of parameters becomes too large. As the number of adjustment items becomes larger, the adjustment work becomes more complicated.

[0078] Therefore, the extra dimensions of the adjustment terms are reduced, and the number of adjustment terms is reduced to make the impulse response adjustment work efficiently.

[0079] Figure 5 and Figure 6 is a diagram illustrating an example of a UI for adjustment of an impulse response according to the present disclosure.

[0080] Figure 5 and Figure 6 An adjustment screen Ad11 presented as a UI for adjustment of the impulse response is shown.

[0081] In addition to Figure 4 In addition to the configuration similar to that of the adjustment screen Ad1 in FIG. 1 , the adjustment screen Ad11 is provided with a slider indicating the value of the "ratio of direct sound to indirect sound" as an adjustment item (parameter), and accepts adjustment of the parameter. Note that as a UI for adjustment of the parameter, a UI other than a slider (e.g., a radio button or an audio input) may be used.

[0082] Figure 5 An example of the processing of the adjustment work in the case of adjusting the absolute volume in the adjustment screen Ad11 is shown. More specifically, when the "absolute volume" is adjusted by the user's (player's) operation, the "level of direct sound" and the "level of indirect sound" are automatically changed based on the relationship expression expressing each of the "level of direct sound" and the "level of indirect sound" using the "absolute volume". In this case, the "ratio of direct sound to indirect sound" does not change and remains constant.

[0083] also, Figure 6 An example of the processing of the adjustment work in the case of adjusting the ratio of direct sound to indirect sound in the adjustment screen Ad11 is shown. More specifically, when the "ratio of direct sound to indirect sound" is adjusted by the user's (player's) operation, the "level of direct sound" and the "level of indirect sound" are automatically changed based on the relationship expression expressing each of the "level of direct sound" and the "level of indirect sound" using the "ratio of direct sound to indirect sound". In this case, the "absolute volume" does not change and remains constant.

[0084] As described, by adjusting the "absolute volume" and "ratio of direct sound to indirect sound" in the adjustment screen Ad11, which allows the user to subjectively and easily grasp how the adjustment operation has been reflected in the adjustment result, the "level of direct sound" and "level of indirect sound" are automatically adjusted.

[0085] Note that the adjustment screen Ad11 may be provided with indicators indicating only the corresponding values ​​of the “level of direct sound” and the “level of indirect sound” in response to the adjustment of the “absolute volume” and the “ratio of direct sound to indirect sound”, instead of the corresponding sliders of the “level of direct sound” and the “level of indirect sound”. In this case, these indicators are configured not to accept the adjustment of the “level of direct sound” and the “level of indirect sound”.

[0086] Here, when adjusting the "level of direct sound" or the "level of indirect sound", the adjustment may be not accepted by fixing the user-specified values ​​in the "absolute volume" and the "ratio of direct sound to indirect sound", or the non-fixed values ​​may be automatically changed. For example, when the "level of direct sound" is changed in a state where the "absolute volume" is fixed, the "level of indirect sound" and the "ratio of direct sound to indirect sound" are automatically adjusted without changing the absolute volume (i.e., the sum of the level of direct sound and the level of indirect sound). Note that the parameters to be fixed may be arbitrarily specified by the user as described above, or may be determined based on environmental information (e.g., the size or material of the performance environment) when measuring the impulse response, or may be determined by a method other than the above method.

[0087] In addition, as in Figure 7 In the adjustment screen Ad12 shown in , only a slider for adjusting each of the “absolute volume” and the “ratio of direct sound to indirect sound” may be presented, without presenting sliders or indicators for the “level of direct sound” and the “level of indirect sound”. Note that the user can appropriately set the items presented on the adjustment screen among the “absolute volume”, “level of direct sound”, “level of indirect sound”, and “ratio of direct sound to indirect sound” as items to be presented on the adjustment screen. In addition, although the above four parameters have been cited as examples of items to be presented on the adjustment screen, parameters other than these parameters may be additionally added and presented.

[0088] <3. Configuration of Information Processing Device and Impulse Response Adjustment Processing>

[0089] Hereinafter, a configuration of an information processing device to which the technology according to the present disclosure is applied and an impulse response adjustment process of the information processing device will be described.

[0090] (Configuration of Information Processing Device)

[0091] Figure 8 : is a block diagram showing a functional configuration example of the information processing device 130 to which the technology according to the present disclosure is applied. Figure 8 At least part of the functional blocks shown in is realized by executing a program by a central processing unit (CPU) installed on a PC or the like configuring the information processing apparatus 130 .

[0092] Figure 8 The information processing device 130 shown in includes an acoustic signal acquisition unit 151, an impulse response holding unit 152, a convolution processing unit 153, an output control unit 154, a UI control unit 155, a UI presentation unit 156, and a horizontal control unit 157.

[0093] The acoustic signal acquisition unit 151 acquires an acoustic signal of a performance sound collected by the microphone 120. The acoustic signal acquired by the acoustic signal acquisition unit 151 is supplied to the convolution processing unit 153.

[0094] The impulse response holding unit 152 holds the impulse response of the sound field reproduction measured or calculated in advance in the performance environment in which the user (performer) plays the musical instrument. The impulse response held in the impulse response holding unit 152 is acquired by the convolution processing unit 153, the UI control unit 155, and the level control unit 157 as needed. In addition to the measured impulse response, the impulse response holding unit 152 can hold (store) the environmental information when the impulse response was measured (for example: the type, size (such as volume), and shape of the performance environment such as a concert hall or a stadium, and the material of the wall surface or floor surface to be used for the performance environment).

[0095] The convolution processing unit 153 performs a convolution process of convolving the impulse response acquired from the impulse response holding unit 152 into the acoustic signal supplied from the acoustic signal acquiring unit 151. The acoustic signal subjected to the convolution process is supplied to the output control unit 154.

[0096] The output control unit 154 causes the headphone 110 to output a playback sound based on the acoustic signal supplied from the convolution processing unit 153 .

[0097] The UI control unit 155 controls the UI for referring to the Figure 5 and Figure 6 The UI control unit 155 controls the presentation of a UI (adjustment screen) for adjusting the impulse response described above. More specifically, the UI control unit 155 controls the presentation of a UI indicating the value of a parameter (adjustment item) related to the sound component included in the impulse response held in the impulse response holding unit 152, and includes a GUI portion (e.g., a slider or a button) that accepts adjustment of the parameter. In addition, the UI is not limited to the above-mentioned slider or the like, and UIs such as knobs, buttons, and voices may be used.

[0098] Furthermore, the UI control unit 155 supplies the level control unit 157 with the setting value of each adjustment item set in the UI presented by the UI presenting unit 156 , and operation information indicating the user's operation on the UI.

[0099] The UI presenting unit 156 includes a display unit capable of displaying a UI, such as a liquid crystal display, a light emitting diode (LED) display, or an electroluminescent (EL) display, and an operating unit capable of accepting user operations, such as a keyboard or a mouse. The UI presenting unit 156 may include a touch panel monitor having corresponding functions of a display unit and an operating unit. The UI presenting unit 156 may be configured integrally with the information processing device 130 configured as a PC or the like, or may be configured separately from the information processing device 130.

[0100] The level control unit 157 changes the levels of the first sound component and the second sound component included in the impulse response held in the impulse response holding unit 152 in response to the user operation indicated by the operation information from the UI control unit 155. The user's operation indicated by the operation information from the UI control unit 155 is, for example, an operation for adjusting parameters related to the first sound component and the second sound component included in the impulse response.

[0101] More specifically, the level control unit 157 changes the level of the first sound component and the level of the second sound component based on a relational expression expressing each of the level of the first sound component and the level of the second sound component using the parameter adjusted in response to the user's operation. Note that the following description will be given assuming that the first sound component included in the impulse response is a "direct sound" and the second sound component is an "indirect sound".

[0102] (Impulse response adjustment processing)

[0103] Will refer to Fig. 9 The flowchart shown in Figure 8 Flow of impulse response adjustment processing performed by the level control unit 157 of the information processing device 130 in FIG.

[0104] In step S11 , the level control unit 157 acquires setting values ​​of the pre-adjustment absolute volume Ga, the pre-adjustment level L1a of the direct sound, and the pre-adjustment level L2a of the indirect sound set in the UI presented by the UI presenting unit 156 via the UI control unit 155 .

[0105] In step S12 , the level control unit 157 calculates the pre-adjustment energy E1a of the direct sound, the pre-adjustment energy E2a of the indirect sound, and the pre-adjustment overall energy Ea based on the pre-adjustment impulse response supplied to the convolution processing unit 153 .

[0106] Each of the energy E1a of the direct sound and the energy E2a of the indirect sound is obtained by cumulatively adding the squares of the sample values ​​of each of the level L1a of the direct sound and the level L2a of the indirect sound. In addition, the overall energy Ea is obtained as the sum of the energy E1a of the direct sound and the energy E2a of the indirect sound.

[0107] In step S13 , the level control unit 157 changes the level of the direct sound and the level of the indirect sound of the impulse response held in the impulse response holding unit 152 in response to the adjustment of the adjustment item in the UI presented by the UI presenting unit 156 .

[0108] Here, one of the “absolute volume” and the “ratio of direct sound to indirect sound” is adjusted as an adjustment item in the UI.

[0109] (a) Level control of direct sound and indirect sound while adjusting the absolute volume

[0110] First, refer to Fig.10 The flowchart in the description is in Fig. 9 The process of controlling the levels of direct sound and indirect sound when the "absolute volume" is adjusted as an adjustment item in step S13.

[0111] In step S111 , the level control unit 157 acquires, via the UI control unit 155 , the setting value of the adjusted absolute volume Gb in the UI presented by the UI presenting unit 156 .

[0112] In step S112, the level control unit 157 calculates the adjusted level L1b of the direct sound indicated using the adjusted absolute volume Gb. The adjusted level L1b of the direct sound is expressed by the following relational expression using the adjusted absolute volume Gb, the pre-adjustment absolute volume Ga, and the pre-adjustment level L1a of the direct sound.

[0113] [Mathematical formula 1]

[0114] L1b = L1a × Gb ÷ Ga … (1)

[0115] In step S113, the level control unit 157 calculates the adjusted level L2b of the indirect sound indicated using the adjusted absolute volume Gb. The adjusted level L2b of the indirect sound is expressed by the following relational expression using the adjusted absolute volume Gb, the pre-adjustment absolute volume Ga, and the pre-adjustment level L2a of the indirect sound.

[0116] [Mathematical formula 2]

[0117] L2b = L2a×Gb÷Ga … (2)

[0118] As described above, when adjusting the absolute volume, the level control unit 157 changes each of the "level of direct sound" and the "level of indirect sound" according to the rate of change of the absolute volume (Gb / Ga). That is, the adjusted level L1b of the direct sound and the adjusted level L2b of the indirect sound can be updated according to the ratio of the absolute volume Ga before adjustment to the absolute volume Gb after adjustment.

[0119] (b) Level control of direct sound and indirect sound in case of adjusting the ratio of direct sound to indirect sound

[0120] Next, we will refer to Fig.11 The flowchart in the description is in Fig. 9 The flow of controlling the levels of direct sound and indirect sound when the “ratio of direct sound to indirect sound” is adjusted as an adjustment item in step S13.

[0121] In step S121 , the level control unit 157 acquires, via the UI control unit 155 , a setting value of the adjusted ratio Rb of the direct sound to the indirect sound in the UI presented by the UI presenting unit 156 .

[0122] In step S122 , the level control unit 157 calculates the adjusted level L1 b of the direct sound indicated using the adjusted ratio Rb of the direct sound to the indirect sound.

[0123] In step S123 , the level control unit 157 calculates the adjusted level L2 b of the indirect sound indicated using the adjusted ratio Rb of the direct sound to the indirect sound.

[0124] Hereinafter, a description will be given of deriving a relational expression expressing each of the adjusted level L1b of the direct sound and the adjusted level L2b of the indirect sound using the adjusted ratio Rb of the direct sound to the indirect sound.

[0125] First, the pre-adjustment energy E1a of the direct sound before adjustment, the pre-adjustment energy E2a of the indirect sound before adjustment, and the pre-adjustment overall energy Ea are expressed by the following relational expressions.

[0126] [Mathematical formula 3]

[0127] Ea=E1a+E2a … (3)

[0128] On the other hand, since the ratio of the adjusted energy E1b of the direct sound to the adjusted energy E2b of the indirect sound is the same as the adjusted ratio Rb of the direct sound to the indirect sound as the adjustment target, the following equation holds.

[0129] [Formula 4]

[0130] Rb = E1b÷E2b … (4)

[0131] Furthermore, as a restriction condition for restricting the change in the overall energy (absolute volume) before and after the adjustment of the ratio of the direct sound to the indirect sound, the following equation holds.

[0132] [Formula 5]

[0133] Ea=E1b+E2b … (5)

[0134] By substituting equation (5) into equation (4), the following equation can be obtained.

[0135] [Mathematical formula 6]

[0136] Rb=E1b÷(Ea-E1b)…(6)

[0137] Furthermore, for the adjusted energy E1b of the direct sound, equation (6) is transformed as follows.

[0138] [Formula 7]

[0139] E1b=Rb×(Ea-E1b)=Rb×Ea-Rb×E1b

[0140] E1b+Rb×E1b=Rb×Ea

[0141] E1b(1+Rb)=Rb×Ea

[0142] E1b=Rb×Ea÷(1+Rb)…(7)

[0143] By substituting equation (7) into equation (4), the following equation can be obtained for the adjusted energy E2b of the indirect sound.

[0144] [Mathematical formula 8]

[0145] E2b=E1b÷Rb

[0146] ={Rb×Ea÷(1+Rb)}÷Rb

[0147] = Ea÷(1+Rb) … (8)

[0148] Using the above-mentioned equations (7) and (8), the control target value of the ratio of each energy of the direct sound and the indirect sound can be obtained as follows according to the adjusted ratio Rb of the direct sound to the indirect sound.

[0149] That is, since the amount of change in the level of the direct sound before and after the adjustment and the amount of change in the energy of the direct sound before and after the adjustment have a proportional relationship, the following relational expression holds.

[0150] [Mathematical formula 9]

[0151] E1b = E1a × L1b ÷ L1a … (9)

[0152] By substituting equation (9) into equation (7), the following equation can be obtained for the adjusted level L1b of the direct sound.

[0153] [Formula 10]

[0154] E1a×L1b÷L1a=

[0155] L1b=L1a×Rb×Ea÷(1+Rb)÷E1a… (10)

[0156] Similarly, since the amount of change in the level of the indirect sound before and after the adjustment and the amount of change in the energy of the indirect sound before and after the adjustment have a proportional relationship, the following relational expression holds.

[0157] [Mathematical formula 11]

[0158] E2b=E2a×L2b÷L2a… (11)

[0159] By substituting equation (11) into equation (8), the following equation can be obtained for the adjusted level L2b of the indirect sound.

[0160] [Mathematical formula 12]

[0161] E2a×L2b÷L2a=Ea÷(1+Rb)

[0162] L2b=L2a×Ea÷(1+Rb)÷E2a… (12)

[0163] As described above, when adjusting the ratio of direct sound to indirect sound, the level control unit 157 changes each of the "level of direct sound" and the "level of indirect sound" according to the adjusted ratio Rb of direct sound to indirect sound. That is, the adjusted level L1b of the direct sound and the adjusted level L2b of the indirect sound can be updated based on the above equations (10) and (12).

[0164] According to the above processing, by using one of "absolute volume" and "ratio of direct sound to indirect sound" as an adjustment item of impulse response, it is possible to make it easy for the performer to subjectively grasp how the adjustment operation has been reflected in the adjustment result. In addition, by reducing the number of adjustment items such as "level of direct sound" and "level of indirect sound" that are difficult for the performer to subjectively understand, it is possible to prevent the adjustment work from becoming complicated. Therefore, the impulse response adjustment work can be made efficient.

[0165] <4. Application Examples>

[0166] Hereinafter, another application example of the adjustment of the impulse response according to the present disclosure will be described.

[0167] (Application of the sound component of the impulse response to other classifications)

[0168] An example in which the impulse response is classified into “direct sound” and “indirect sound” and adjusted has been described above. The technology according to the present disclosure is not limited thereto, and the impulse response classified into any “first sound component” and “second sound component” may be finely adjusted.

[0169] For example, the impulse response may be classified into "direct sound + early reflected sound" and "late reverberation sound" to finely adjust the impulse response. In addition, the impulse response may be a reflected sound from a specific direction and a reflected sound from a direction other than the specific direction, such as "reflected sound from the front" and "reflected sound from directions other than the front", to finely adjust the impulse response. These classifications may be arbitrarily set by the user, or may be automatically set by associating with environmental information indicating the performance environment or the like in which the impulse response stored in the impulse response holding unit 152 has been measured.

[0170] (Application to other purposes)

[0171] The technology according to the present disclosure is not limited to the remote ensemble system, and can be applied to the purpose of making the adjustment of all impulse responses indicating the transfer characteristics from the sound source to the user's ears efficient. For example, the technology according to the present disclosure can be applied to the use exemplified below.

[0172] (1) Stereo mixing / mastering

[0173] In relation to stereo mixing / mastering work, it is possible to imagine a situation where the impulse response is adjusted for the purpose of optimizing the sound field effect. In this case, the overall volume (absolute volume) affects the "equal loudness curve" that connects the perceived expansion of the sound or the perceived equal sound pressure levels of the tones, and therefore needs to be kept constant during work.

[0174] Generally, when the levels of the direct sound, early reflection sound, late reverberation sound, etc. of the impulse response are adjusted, the overall volume also changes, and therefore, the absolute volume needs to be readjusted after the adjustment.

[0175] In contrast, by applying the technology according to the present disclosure, even when, for example, the ratio of "direct sound + early reflected sound" to "late reverberation sound" is adjusted, readjustment of the absolute volume can be made unnecessary, and stereo mixing / mastering work can be made efficient.

[0176] (2) User game sound adjustment

[0177] For game sound effects, it is assumed that the sound field such as the direction of direct sound, indirect sound and reflected sound is finely adjusted according to the user's preference. In this case, when these sound fields are adjusted, the overall volume also changes, and therefore, the absolute volume needs to be readjusted after the adjustment.

[0178] In contrast, by applying the technology according to the present disclosure, even when, for example, the ratio of “direct sound” to “indirect sound” is adjusted, readjustment of the absolute volume is made unnecessary, and the adjustment work of the game sound effect is made efficient and the UI is simplified.

[0179] Note that the technology according to the present disclosure is applicable to uses such as live concerts, lessons, and conferences in a virtual space other than the above, and applicable uses are not limited to these.

[0180] <5. Computer Configuration Example>

[0181] The above-described series of processing may be executed by hardware or may be executed by software. When the series of processing is executed by software, a program configuring the software is installed from a program recording medium to a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0182] Fig.12 1 is a block diagram showing an example of the configuration of the hardware of a computer that executes the above-mentioned series of processes by a program. Fig.12 The configuration shown in the figure is the same as the configuration of the PC to be configured.

[0183] The CPU 301 , a read only memory (ROM) 302 , and a random access memory (RAM) 303 are connected to one another via a bus 304 .

[0184] The bus 304 is also connected to an input / output interface 305. The input / output interface 305 is connected to an input unit 306 including a keyboard, a mouse, etc. and an output unit 307 including a display, a speaker, etc. In addition, the input / output interface 305 is connected to a storage unit 308 including a hard disk, a nonvolatile memory, etc., a communication unit 309 including a network interface, etc., and a drive 310 that drives a removable medium 311.

[0185] In the computer configured as described above, for example, the CPU 301 performs the above-described series of processing by loading a program stored in the storage unit 308 to the RAM 303 via the input / output interface 305 and the bus 304 and executing the program, for example.

[0186] For example, the program executed by the CPU 301 is recorded on the removable medium 311 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting to be installed in the storage unit 308 .

[0187] The program executed by the computer may be a program that executes a plurality of processes in time series in the order described herein, or may be a program that executes processes in parallel or at a necessary point such as when called.

[0188] Meanwhile, as used herein, a system means a collection of multiple components (such as devices, modules (components), etc.), and all components may or may not be located in the same housing. Therefore, multiple devices stored in separate housings and connected via a network and a single device including multiple modules housed in a single housing are both systems.

[0189] The benefits described herein are merely examples and are not intended to be limiting, and other benefits may be obtained.

[0190] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present disclosure.

[0191] For example, an embodiment of the present disclosure adopts a configuration of cloud computing in which a plurality of devices share and collaboratively process a function through a network.

[0192] Furthermore, each step described in the above flowchart may be performed by a single device or by a plurality of devices in a shared manner.

[0193] Furthermore, when one step includes a plurality of processes, the plurality of processes included in the one step may be performed by a single device, or may be performed by a plurality of devices in a distributed manner.

[0194] The beneficial effects described herein are merely exemplary and not limiting, and other beneficial effects may be obtained.

[0195] Furthermore, the technology according to the present disclosure may be configured as follows. (1)

[0197] An information processing device includes: a level control unit that changes a level of a first sound component and a level of a second sound component in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response. (2)

[0199] In the information processing device described in (1), the impulse response is measured or calculated in advance. (3)

[0201] In the information processing device described in (1), the level control unit changes the level of the first sound component and the level of the second sound component based on a relational expression expressing each of the level of the first sound component and the level of the second sound component using a parameter. (4)

[0203] In the information processing device according to any one of (1) to (3), the parameter is an absolute volume. (5)

[0205] In the information processing device described in (4), the level control unit changes the level of the first sound component and the level of the second sound component according to the adjusted rate of change of the absolute volume. (6)

[0207] In the information processing device according to any one of (1) to (3), the parameter is a ratio of the first sound component to the second sound component. (7)

[0209] In the information processing device described in (6), before and after adjusting the ratio, the level control unit changes the level of the first sound component and the level of the second sound component without changing the sum of the energy of the first sound component and the energy of the second sound component. (8)

[0211] In the information processing device according to any one of (1) to (7),

[0212] The first sound component includes direct sound, and

[0213] The second sound component includes indirect sound. (9)

[0215] In the information processing device described in (8), the indirect sound includes an early reflection sound or a late reverberation sound. (10)

[0217] In the information processing device according to any one of (1) to (7),

[0218] The first sound component includes direct sound and early reflected sound, and

[0219] The second sound component includes a late reverberation sound. (11)

[0221] In the information processing device according to any one of (1) to (7),

[0222] The first sound component includes reflected sound from a specific direction, and

[0223] The second sound component includes reflected sound from outside the specific direction. (12)

[0225] The information processing device according to any one of (1) to (11) further includes a UI control unit that controls presentation of a user interface (UI) that indicates a value of the parameter and accepts adjustment of the parameter. (13)

[0227] In the information processing device described in (12),

[0228] The UI also includes an indicator indicating values ​​of a level of the first sound component and a level of the second sound component that change in response to adjustment of the parameter, and

[0229] The indicator does not accept adjustments of the level of the first sound component and the level of the second sound component. (14)

[0231] In the information processing device described in (12), the level control unit acquires at least one of the parameter adjusted through the UI, the level of the first sound component, and the level of the second sound component via the UI control unit. (15)

[0233] In the information processing device according to any one of (1) to (14), the impulse response indicates a transfer characteristic from the sound source to both ears of the user. (16)

[0235] The information processing device described in (15) includes an output control unit that causes an output device used by a user to output a playback sound, the playback sound being based on a convolution process performed on the impulse response for which a level of a first sound component and a level of a second sound component have been adjusted with respect to an acoustic signal from a sound source. (17)

[0237] An information processing method includes: at an information processing device, in response to adjustment of parameters related to the first sound component and the second sound component included in an impulse response, changing a level of a first sound component and a level of a second sound component. (18)

[0239] A program causing a computer to execute a process of changing a level of a first sound component and a level of a second sound component in response to adjustment of a parameter related to the first sound component and the second sound component included in an impulse response.

[0240] [Reference Numbers List]

[0241] 1 Remote Ensemble System

[0242] 10 Transmission control device

[0243] 110 Headphones

[0244] 120 Microphones

[0245] 130 Information processing device

[0246] 151 Acoustic Signal Acquisition Unit

[0247] 152 Impulse Response Hold Unit

[0248] 153 Convolution Processing Unit

[0249] 154 Output control unit

[0250] 155 UI control unit

[0251] 156 UI Presentation Unit

[0252] 157 Horizontal control unit.

Claims

1. An information processing device, comprising: A level control unit changes a level of the first sound component and a level of the second sound component in response to adjustment of parameters related to the first sound component and the second sound component included in the impulse response.

2. The information processing device according to claim 1, wherein: The impulse response is either measured or calculated in advance.

3. The information processing device according to claim 1, wherein: The level control unit changes the level of the first sound component and the level of the second sound component based on a relational expression expressing each of the level of the first sound component and the level of the second sound component using the parameter.

4. The information processing device according to claim 1, wherein: The parameter is the absolute volume.

5. The information processing device according to claim 4, wherein: The level control unit changes the level of the first sound component and the level of the second sound component according to the adjusted rate of change of the absolute volume.

6. The information processing device according to claim 1, wherein: The parameter is a ratio of the first sound component to the second sound component.

7. The information processing device according to claim 6, wherein: Before and after adjusting the ratio, the level control unit changes the level of the first sound component and the level of the second sound component without changing the sum of the energy of the first sound component and the energy of the second sound component.

8. The information processing device according to claim 1, wherein: The first sound component comprises direct sound, and The second sound component includes indirect sound.

9. The information processing device according to claim 8, wherein: The indirect sound includes early reflection sound or late reverberation sound.

10. The information processing device according to claim 1, wherein: The first sound component includes direct sound and early reflected sound, and The second sound component includes a late reverberation sound.

11. The information processing device according to claim 1, wherein: The first sound component includes reflected sound from a specific direction, and The second sound component includes reflected sound coming from outside the specific direction.

12. The information processing device according to claim 1, further comprising: A UI control unit controls presentation of a user interface (UI) indicating a value of the parameter and accepting adjustment of the parameter.

13. The information processing device according to claim 12, wherein: The UI further includes an indicator indicating values ​​of the level of the first sound component and the level of the second sound component changed in response to adjustment of the parameter, and the indicator does not accept adjustment of the level of the first sound component and the level of the second sound component.

14. The information processing device according to claim 12, wherein: The level control unit acquires at least one of the parameter adjusted through the UI, the level of the first sound component, and the level of the second sound component via the UI control unit.

15. The information processing device according to claim 1, wherein: The impulse response indicates the transfer characteristics from the sound source to the user's ears.

16. The information processing device according to claim 15 includes an output control unit, which enables an output device used by a user to output a playback sound, wherein the playback sound is based on a convolution process performed on the impulse response, for which the level of the first sound component and the level of the second sound component have been adjusted for the acoustic signal from the sound source.

17. An information processing method, comprising: At the information processing apparatus, a level of the first sound component and a level of the second sound component are changed in response to adjustment of a parameter related to the first sound component and the second sound component included in the impulse response.

18. A program causing a computer to execute the following process: in response to adjustment of a parameter related to the first sound component and the second sound component included in an impulse response, changing a level of a first sound component and a level of a second sound component.

Images