Information processing methods, information processing systems, and programs

Abstract

To reproduce a variety of sound fields.SOLUTION: An information processing system acquires a first sound field parameter representing a feature of a first sound field and a second sound field parameter representing a feature of a second sound field having acoustic characteristics different from those of the first sound field, and generates a third sound field parameter that represents the characteristics of a third sound field having different acoustic characteristics from the first sound field and the second sound field by using the first sound field parameter and the second sound field parameter.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to a technique for controlling a sound field.

Background Art

[0002] Various techniques for controlling the sound field perceived by a listener have been proposed conventionally. For example, Patent Document 1 discloses a technique for calculating a sound field information parameter group from signals recorded by a plurality of microphones and generating a new sound field information parameter group by moving the origin of the sound field information parameter group by a movement operator.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, against the background of the spread of acoustic reproduction technologies such as higher-order ambisonics technology, highly diverse sound fields are required to be reproduced. In view of the above circumstances, one aspect of the present disclosure aims to reproduce diverse sound fields.

Means for Solving the Problems

[0005] In order to solve the above problems, an information processing method according to one aspect of the present disclosure acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and generates a third sound field parameter representing the characteristics of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field by using the first sound field parameter and the second sound field parameter.

[0006] An information processing system according to one aspect of the present disclosure comprises an acquisition unit that acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and a generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter representing the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields.

[0007] A program according to one aspect of the present disclosure causes a computer system to function as an acquisition unit that acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and a generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter representing the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram illustrating the configuration of the information system in the first embodiment. [Figure 2] This is a block diagram illustrating the configuration of an information provision system. [Figure 3] This is a diagram illustrating a spherical microphone array. [Figure 4] This is a flowchart of the analysis process. [Figure 5] This is an explanatory diagram of the discrete wavelet transform. [Figure 6] This is an explanatory diagram of spherical harmonics. [Figure 7] This is a block diagram illustrating the configuration of an information processing system. [Figure 8] This is a flowchart of the playback process. [Figure 9] This is a flowchart of the synthesis process. [Figure 10] This is an explanatory diagram of the synthesis process. [Figure 11] This is an explanatory diagram of the synthesis process in the second embodiment. [Figure 12] This is a block diagram illustrating the configuration of an electronic musical instrument in the third embodiment. [Modes for carrying out the invention]

[0009] A: First Embodiment Figure 1 is a block diagram illustrating the configuration of the information system 100 in the first embodiment. The information system 100 in the first embodiment comprises an information provision system 10 and an information processing system 20. The information processing system 20 can communicate with the information provision system 10 via a communication network 200, such as the Internet. The information provision system 10 is implemented, for example, by a server system, and the information processing system 20 is implemented by an information device such as a smartphone, tablet terminal, or personal computer.

[0010] The information provision system 10 is a computer system that generates sound field parameters Z(Z1,Z2) that represent the characteristics of a specific sound field. The characteristics of a sound field include, for example, the distribution of sound pressure of an incoming sound relative to the listening point. Specifically, the distribution of acoustic energy on a spherical surface centered on the listening point is represented by the sound field parameters Z. The sound field parameters Z of a specific sound field can also be described as parameters for reproducing that sound field in any space.

[0011] The information provision system 10 of the first embodiment generates a sound field parameter Z1 representing the characteristics of the first sound field and a sound field parameter Z2 representing the characteristics of the second sound field. The first sound field and the second sound field are sound fields formed in acoustic spaces with different acoustic characteristics. For example, the first sound field and the second sound field are sound fields corresponding to different acoustic halls. Specifically, there are differences between the first sound field and the second sound field in various conditions related to the propagation of sound waves, such as the shape, size, or sound absorption coefficient of the acoustic space. The first sound field or the second sound field may be an acoustic hall designed with specific acoustic characteristics, or it may be an anechoic chamber where reflection from the inner wall surface hardly occurs. In the following description, when it is not necessary to distinguish between the first sound field and the second sound field, both will be collectively referred to as the "observed sound field". The sound field parameter Z1 and the sound field parameter Z2 are provided to the information processing system 20 via the communication network 200. Note that sound field parameter Z1 is an example of a "first sound field parameter," and sound field parameter Z2 is an example of a "second sound field parameter."

[0012] The information processing system 20 is installed in the acoustic space R where user U is located. The information processing system 20 generates sound field parameter Z3 using sound field parameter Z1 and sound field parameter Z2. Sound field parameter Z3 is a parameter that represents the characteristics of a third sound field, which has different acoustic characteristics from the first and second sound fields. Specifically, the third sound field is a sound field with acoustic characteristics intermediate between the first and second sound fields. That is, the third sound field is a sound field that reflects both the acoustic characteristics of the first sound field and the acoustic characteristics of the second sound field. Note that the third sound field is not limited to a sound field in which the acoustic characteristics of the first and second sound fields are equally reflected. For example, a sound field in which the first sound field is predominantly reflected compared to the second sound field, or a sound field in which the second sound field is predominantly reflected compared to the first sound field, are also included in the "third sound field". The information processing system 20 reproduces the third sound field in the acoustic space R using the sound field parameter Z3. In other words, the information processing system 20 controls sound reproduction so that the user U in the acoustic space R perceives the third sound field. Note that the sound field parameter Z3 is an example of a "third sound field parameter".

[0013] [Information Provision System 10] FIG. 2 is a block diagram illustrating the configuration of the information providing system 10. The information providing system 10 includes a control device 11, a storage device 12, and a communication device 13. Note that the information providing system 10 can be realized as a single device or as a plurality of devices separately configured from each other.

[0014] The control device 11 is one or more processors that control each element of the information providing system 10. Specifically, for example, the control device 11 is configured by one or more types of processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0015] The communication device 13 communicates with the information processing system 20 via the communication network 200. For example, the communication device 13 transmits the sound field parameter Z1 and the sound field parameter Z2 to the information processing system 20. Note that the communication between the communication device 13 and the communication network 200 may be either wired communication or wireless communication.

[0016] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. For example, known recording media such as semiconductor recording media and magnetic recording media, or a combination of multiple types of recording media are used as the storage device 12. Note that, for example, a portable recording medium detachable from the information providing system 10, or a recording medium (e.g., cloud storage) accessible by the control device 11 via the communication network 200 may be used as the storage device 12.

[0017] The memory device 12 stores Q observation signals X(1) to X(Q) for each of the first sound field and the second sound field. The Q observation signals X(1) to X(Q) corresponding to the first sound field are signals representing the waveforms of sound waves picked up in parallel with each other in the first sound field. Similarly, the Q observation signals X(1) to X(Q) corresponding to the second sound field are signals representing the waveforms of sound waves picked up in parallel with each other in the second sound field.

[0018] For generating the Q observation signals X(1) to X(Q) in the observed sound field (first sound field / second sound field), the spherical microphone array 30 in FIG. 3 is used. The spherical microphone array 30 is a sound pickup device in which Q microphones 32 are dispersedly installed on the surface of a spherical housing 31 with a radius r. Any one observation signal X(q) is a signal representing the waveform of the sound picked up by the q-th (q = 1 to Q) microphone 32 among the Q microphones 32 in the observed sound field. The observation signal X(q) of the first sound field is recorded with the spherical microphone array 30 installed in the first sound field, and the observation signal X(q) of the second sound field is recorded with the spherical microphone array 30 installed in the second sound field. Therefore, the acoustic characteristics of the observed sound field are reflected in each observation signal X(q).

[0019] FIG. 4 is a flowchart of a process (hereinafter referred to as "analysis process") in which the control device 11 generates the sound field parameter Z. The analysis process is executed for each of the first sound field and the second sound field. That is, the sound field parameter Z1 is generated by the analysis process regarding the first sound field, and the sound field parameter Z2 is generated by the analysis process regarding the second sound field.

[0020] The control device 11 acquires the Q observation signals X(1) to X(Q) recorded in the observed sound field (first sound field / second sound field) from the memory device 12 (Sa1). The control device 11 decomposes each observation signal X(q) into K acoustic components C(q,1) to C(q,K) (Sa2). The K acoustic components C(q,1) to C(q,K) are components corresponding to different frequency bands in the observation signal X(q).

[0021] The control device 11 of the first embodiment generates acoustic components C(q,1) to C(q,K) from the observed signal X(q) using discrete wavelet transform. Specifically, the Haar wavelet exemplified in Figure 5 is used to generate each acoustic component C(q,k). In Figure 5, the symbol "H" represents a high-pass filter (HPF), and the symbol "L" represents a low-pass filter (LPF). The symbol "DS" represents downsampling, which reduces the sampling frequency by half.

[0022] As illustrated in Figure 5, the discrete wavelet transform generates one approximate component c(q,0) and K detailed components c(q,1) to c(q,K). The control device 11 generates the K acoustic components C(q,1) to C(q,K) by performing the calculation in the following equation (1).

number

[0023] As illustrated in Figure 4, the control device 11 generates the sound field parameter Z of the observed sound field (Sa3). The sound field parameter Z is a set of expansion coefficients Bnm(k) corresponding to the acoustic components C(q,k) (C(1,k) to C(Q,k)) in each of the Q observed signals X(1) to X(Q). The expansion coefficients Bnm(k) are weighted values ​​(spherical harmonic coefficients) corresponding to the spherical harmonic function Ynm, which corresponds to a combination of degree n and order m, as illustrated in Figure 6. Specifically, the control device 11 calculates the expansion coefficients Bnm(k) by the following equation (2).

number

[0024] As can be understood from the above description, the sound field parameter Z of the first embodiment includes a plurality of expansion coefficients Bnm(k) corresponding to different spherical harmonics Ynm. Specifically, the sound field parameter Z includes a plurality of expansion coefficients Bnm(k) corresponding to different combinations of order n and order m for each of the K frequency bands. As can be understood from the above description, the control device 11 functions as a HOA (higher order Ambisonics) encoder. The information provision system 10 transmits the sound field parameter Z1 of the first sound field and the sound field parameter Z2 of the second sound field generated by the above procedure to the information processing system 20.

[0025] [Information Processing System 20] Figure 7 is a block diagram illustrating the configuration of the information processing system 20. The information processing system 20 comprises a control device 21, a storage device 22, a communication device 23, an operating device 24, and a playback system 25. The information processing system 20 can be implemented as a single device or as multiple devices configured separately from each other.

[0026] The control device 21 is one or more processors that control each element of the information processing system 20. Specifically, the control device 21 is composed of one or more types of processors, such as a CPU, GPU, SPU, DSP, FPGA, or ASIC.

[0027] The communication device 23 communicates with the information provision system 10 via the communication network 200. For example, the communication device 23 receives sound field parameter Z1 and sound field parameter Z2 from the information provision system 10. The communication between the communication device 23 and the communication network 200 may be either wired or wireless.

[0028] The storage device 22 is one or more memories that store the program executed by the control device 21 and various data used by the control device 21. For example, known recording media such as semiconductor recording media and magnetic recording media, or a combination of multiple types of recording media, can be used as the storage device 22. Alternatively, for example, a portable recording media that can be attached to and detached from the information provision system 10, or a recording media accessible by the control device 21 via the communication network 200 (e.g., cloud storage), may be used as the storage device 22.

[0029] The storage device 22 of the first embodiment stores sound field parameters Z1 and Z2 received by the communication device 23. The storage device 22 also stores acoustic signal A. Acoustic signal A is a multi-channel signal representing the waveform of sound, such as performance sound or singing sound. The data format of acoustic signal A is arbitrary.

[0030] The operating device 24 is an input device that accepts operations from user U. For example, an operator operated by user U, or a touch panel that detects contact by user U, can be used as the operating device 24. Alternatively, the operating device 24 may be separate from the information processing system 20 and connected to the information processing system 20 by wire or wireless connection.

[0031] User U can instruct the information processing system 20 to set an instruction value W by operating the control device 24. The instruction value W is a variable that represents the degree to which each of the sound field parameters Z1 and Z2 is reflected in the sound field parameter Z3. The instruction value W changes continuously in response to the operation of the control device 24 by user U. For example, the control device 24 includes a rotary operator such as a rotatable knob, or a reciprocating operator such as a slider that can move back and forth linearly. The instruction value W changes continuously depending on the angle of the rotary operator or the position of the reciprocating operator. Specifically, the instruction value W is set within the range of 0 or more and 1 or less in response to the instruction from user U.

[0032] The playback system 25 is an audio system consisting of multiple speakers 251 corresponding to different channels. For example, a surround sound system consisting of three or more speakers 251, or a stereo system consisting of two speakers 251, are exemplified as the playback system 25.

[0033] Multiple speakers 251 are installed at different positions within the acoustic space. For example, multiple speakers 251 are arranged around the user U. Each speaker 251 is a sound-emitting device that emits sound represented by the acoustic signal A. Specifically, sound waves are emitted when the acoustic signal A of each channel is supplied to the speaker 251 of that channel. Note that the D / A converter that converts the acoustic signal A of each channel from digital to analog, and the amplifier that amplifies the acoustic signal A, are omitted from the illustration for convenience. In addition, a playback system 25 separate from the information processing system 20 may be connected to the information processing system 20 by wire or wireless.

[0034] Figure 8 is a flowchart of the process executed by the control device 21 (hereinafter referred to as "playback process"). For example, the playback process is started when user U makes an operation on the operation device 24. When the playback process starts, the control device 21 acquires sound field parameter Z1 and sound field parameter Z2 from the storage device 22 (Sb1). As described above, the control device 21 functions as an element (acquisition unit) that acquires sound field parameter Z1 and sound field parameter Z2. Note that the acquisition destination of sound field parameter Z1 and sound field parameter Z2 is not limited to the storage device 22. For example, the control device 21 may receive sound field parameter Z1 and sound field parameter Z2 transmitted from the information provision system 10 via the communication network 200 using the communication device 23.

[0035] The control device 21 determines whether or not it has received an instruction from user U to change the instruction value W (Sb2). If it has received an instruction to change the instruction value W (Sb2: YES), the control device 21 updates the instruction value W to the value instructed by user U (Sb3).

[0036] The control device 21 performs a synthesis process (Sb4). The synthesis process generates the sound field parameter Z3 using the sound field parameter Z1 and the sound field parameter Z2. The updated instruction value W is applied to the synthesis process. On the other hand, if no change in the instruction value W is instructed (Sb2: NO), the update of the instruction value W (Sb3) and the synthesis process (Sb4) are not performed. As can be understood from the above explanation, the control device 21 functions as an element (generation unit) that generates the sound field parameter Z3 from the sound field parameter Z1 and the sound field parameter Z2.

[0037] Figure 9 is a flowchart of the synthesis process. When the synthesis process is started, the control device 21 generates the acoustic energy distribution D1 of the first sound field from the sound field parameter Z1 (Sb41). The acoustic energy distribution D1 is the distribution of acoustic energy (sound pressure) on a sphere of radius r. Specifically, the control device 21 calculates the acoustic energy distribution D1 as the weighted sum of multiple spherical harmonics Ynm, to which multiple expansion coefficients Bnm(k) included in the sound field parameter Z1 are applied as weighted values.

[0038] Similarly, the control device 21 generates the acoustic energy distribution D2 of the second sound field from the sound field parameter Z2 (Sb42). Specifically, the control device 21 calculates the acoustic energy distribution D2 as the weighted sum of multiple spherical harmonics Ynm, to which multiple expansion coefficients Bnm(k) included in the sound field parameter Z2 are applied as weighted values. Note that the order of generating the acoustic energy distribution D1 (Sb41) and the acoustic energy distribution D2 (Sb42) may be reversed.

[0039] Figure 10 is an explanatory diagram of the synthesis process. Figure 10 schematically illustrates the acoustic energy distribution D1 of the first sound field and the acoustic energy distribution D2 of the second sound field. Acoustic energy distribution D1 includes an acoustic energy peak P1, and acoustic energy distribution D2 includes an acoustic energy peak P2.

[0040] The control device 21 generates the acoustic energy distribution D3 of the third sound field such that the peak P3 of acoustic energy is located midway between the peak P1 of acoustic energy in the acoustic energy distribution D1 and the peak P2 of acoustic energy in the acoustic energy distribution D2 (Sb43).

[0041] Specifically, the position of peak P3 in acoustic energy distribution D3 is set to a position obtained by dividing the position of peak P1 in acoustic energy distribution D1 and the position of peak P2 in acoustic energy distribution D2 according to the indicated value W. Specifically, the closer the indicated value W is to the minimum value of 0, the closer peak P3 in acoustic energy distribution D3 is to peak P1 in acoustic energy distribution D1, and the closer the indicated value W is to the maximum value of 1, the closer peak P3 in acoustic energy distribution D3 is to peak P2 in acoustic energy distribution D2. When the indicated value W is set to the minimum value of 0, acoustic energy distribution D1 is applied as acoustic energy distribution D3, and when the indicated value W is set to the maximum value of 1, acoustic energy distribution D2 is applied as acoustic energy distribution D3. For comparison between peak P1 in acoustic energy distribution D1 and peak P2 in acoustic energy distribution D2, for example, the Wasserstein distance is used.

[0042] The control device 21 generates sound field parameters Z3 corresponding to the acoustic energy distribution D3 generated by the above process (Sb44). The sound field parameters Z3 include multiple expansion coefficients Bnm(k) corresponding to different spherical harmonics Ynm. As can be understood from the above explanation, the control device 21 generates sound field parameters Z3 in which each of the sound field parameters Z1 and Z2 is reflected to the extent in accordance with the instructions from the user U. The specific procedure for the synthesis process is as described above.

[0043] As illustrated in Figure 8, the control device 21 weights each of the multiple acoustic signals A stored in the memory device 22 according to the sound field parameter Z3 (Sb5). That is, the control device 21 controls the volume and phase of the acoustic signals A of each channel so that a third sound field represented by the sound field parameter Z3 is formed within the acoustic space R. In other words, the control device 21 functions as an HOA decoder. Known techniques can be arbitrarily employed to control the sound field by applying the sound field parameter Z3.

[0044] The control device 21 supplies each controlled acoustic signal A according to the sound field parameter Z3 to the speaker 251 corresponding to the acoustic signal A in the playback system 25 (Sb6). As sound waves corresponding to the acoustic signal A are radiated from each speaker 251, a third sound field is formed in the acoustic space R. That is, the user U can perceive a third sound field that is different from the first and second sound fields.

[0045] The control device 21 determines whether a predetermined termination condition has been met (Sb7). The termination condition is, for example, that the user U has instructed the system to terminate, or that the entire playback of the sound signal A has been completed. If the termination condition is not met (Sb7: NO), the control device 21 proceeds to step Sb2. That is, each time the user U instructs a change in the instruction value W, the system updates the instruction value W (Sb3) and performs a synthesis process (Sb4) applying the updated instruction value W. If the termination condition is met (Sb7: YES), the control device 21 terminates the playback process.

[0046] As described above, in the first embodiment, the sound field parameter Z3 of the third sound field is generated using the sound field parameter Z1 of the first sound field and the sound field parameter Z2 of the second sound field. Therefore, a variety of sound fields between the first and second sound fields can be reproduced. In particular, in the first embodiment, the degree of influence of sound field parameters Z1 and Z2 on sound field parameter Z3 is controlled according to instructions (instruction value W) from the user U. Therefore, a third sound field can be reproduced according to the intentions of the user U. Furthermore, in the first embodiment, the instruction value W applied to the generation of sound field parameter Z3 changes continuously according to instructions from the user U. Therefore, sound field parameter Z3 can be generated for any third sound field that is in the process of continuously changing from one of the first and second sound fields to the other. In other words, it is possible to morph between the first and second sound fields.

[0047] B: Second Embodiment A second embodiment will now be described. For elements whose function is the same as in the first embodiment in each of the embodiments described below, the same reference numerals as in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0048] Figure 11 is an explanatory diagram of the synthesis process in the second embodiment. As described above with reference to Figure 10, in the synthesis process of the first embodiment, the acoustic energy distribution D3 of the third sound field is generated such that the acoustic energy peak P3 is located midway between the acoustic energy peak P1 in the first sound field and the acoustic energy peak P2 in the second sound field. In the second embodiment, the control device 21 generates the sound field parameter Z3 of the third sound field by a weighted sum of the sound field parameter Z1 and the sound field parameter Z2 during the synthesis process.

[0049] Specifically, the control device 21 generates the acoustic energy distribution D3 of the third sound field by a weighted sum of the acoustic energy distribution D1 and the acoustic energy distribution D2. As illustrated in Figure 11, the acoustic energy distribution D3 includes a peak P31 corresponding to the acoustic energy distribution D1 and a peak P32 corresponding to the acoustic energy distribution D2. The position of peak P31 is the same as the position of peak P1 in the acoustic energy distribution D1, and the position of peak P32 is the same as the position of peak P2 in the acoustic energy distribution D2.

[0050] The value of peak P31 in acoustic energy distribution D3 is set to the value of peak P1 in acoustic energy distribution D1 multiplied by a weighted value (1-W) corresponding to the indicated value W. The value of peak P32 in acoustic energy distribution D3 is set to the value of peak P2 in acoustic energy distribution D2 multiplied by the indicated value W. Therefore, as the indicated value W approaches the minimum value of 0, the value of peak P31 increases and the value of peak P32 decreases. On the other hand, as the indicated value W approaches the maximum value of 1, the value of peak P31 decreases and the value of peak P32 increases. When the indicated value W is set to the minimum value of 0, acoustic energy distribution D1 is applied as acoustic energy distribution D3, and when the indicated value W is set to the maximum value of 1, acoustic energy distribution D2 is applied as acoustic energy distribution D3.

[0051] The operation other than the synthesis process is the same as in the first embodiment. Therefore, the same effects as in the first embodiment are achieved in the second embodiment as well. In addition, in the second embodiment, the sound field parameter Z3 of the third sound field is generated by a weighted sum (e.g., average) of sound field parameter Z1 and sound field parameter Z2. Therefore, the processing load required for the synthesis process can be reduced compared to the first embodiment.

[0052] On the other hand, in the first embodiment, the sound field parameter Z3 of the third sound field is generated such that the peak P3 is located midway between the peak P1 of the acoustic energy distribution D1 in the first sound field and the peak P2 of the acoustic energy distribution D2 in the second sound field. Therefore, compared to the second embodiment, there is an advantage in that the sound field parameter Z3 of the third sound field can be generated in which the user U can clearly perceive the acoustic characteristics intermediate between the first and second sound fields.

[0053] C: Third Embodiment Figure 12 is a block diagram illustrating the configuration of the electronic musical instrument 40 in the third embodiment. The electronic musical instrument 40 is an information processing system that reproduces sounds in response to performance operations by the user U. The electronic musical instrument 40 illustrated in Figure 12 is an electronic keyboard musical instrument that, in addition to the same elements as the information processing system 20 of the first embodiment (control device 21, storage device 22, communication device 23, operation device 24, playback system 25), is equipped with a keyboard 26 and a sound source device 27.

[0054] The keyboard 26 is composed of multiple keys corresponding to multiple different pitches. The sound source device 27 generates an acoustic signal A representing a musical tone of the pitch corresponding to the key operated by the user U. The function of the sound source device 27 may also be realized by the control device 21 executing a program. In other words, the element that generates the acoustic signal A in response to the performance operation by the user U (sound source unit) may be realized by either a software sound source realized by the control device 21, or a hardware sound source (sound source device 27) dedicated to generating the acoustic signal A.

[0055] In the first embodiment, the sound field parameter Z3 was applied to the acoustic signal A stored in the memory device 22 (Sb5). In the third embodiment, the control device 21 applies the sound field parameter Z3 to the acoustic signal A generated by the sound source device 27. Except for the method of acquiring the acoustic signal A, it is the same as in the first embodiment. Therefore, the same effects as in the first embodiment are achieved in the third embodiment as well. In the above description, the third embodiment has been described based on the first embodiment, but the second embodiment may also be applied to the electronic musical instrument 40.

[0056] In the third embodiment, as in the first embodiment, the instruction value W corresponding to the user U's instruction is applied to the generation of the sound field parameter Z3 of the third sound field. Therefore, the user U can play the electronic instrument 40 in the third sound field having the desired acoustic characteristics. For example, the user U can practice playing the electronic instrument 40 in a sound field equivalent to that of the acoustic hall where a concert in which they are scheduled to perform will be held. Alternatively, the user U can play the electronic instrument 40 in a sound field equivalent to that of, for example, a world-renowned acoustic hall.

[0057] D: Variant Specific modifications added to each of the embodiments exemplified above are shown below. Multiple embodiments arbitrarily selected from the embodiments described above and the modifications exemplified below may be combined as appropriate, within the bounds of mutual consistency.

[0058] (1) In the above-described forms, the sound field parameters Z(Z1~Z3) are exemplified as having a configuration that includes multiple expansion coefficients Bnm(k) corresponding to different spherical harmonics Ynm. However, the content of the sound field parameters Z is not limited to the above examples. It is also conceivable that the sound field parameters Z represent acoustic energy distributions D(D1~D3) on a single sphere. For example, sound field parameter Z1 represents the acoustic energy distribution D1 of the first sound field, and sound field parameter Z2 represents the acoustic energy distribution D2 of the second sound field. Similarly, a sound field parameter Z3 representing the acoustic energy distribution D3 of the third sound field may be used. Furthermore, sound field parameters Z1, Z2, and Z3 may be of different types or forms. For example, it is conceivable that sound field parameters Z1 and Z2 are composed of multiple expansion coefficients Bnm(k), and sound field parameter Z3 represents the acoustic energy distribution D3 of the third sound field.

[0059] (2) In the first embodiment, the acoustic energy distribution D3 of the third sound field was generated by placing peak P3 midway between peak P1 of the acoustic energy distribution D1 and peak P2 of the acoustic energy distribution D2. However, the specific procedure of the synthesis process is not limited to the above examples. For example, a configuration in which multiple sound field parameters Z corresponding to different numerical values ​​of the instruction value W are pre-stored in the storage device 22 as selection candidates is also conceivable. Each selection candidate's sound field parameter Z may consist of multiple expansion coefficients Bnm(k), or it may be a parameter that represents the acoustic energy distribution D. The position of peak P3 differs for each selection candidate. In the synthesis process, the control device 21 selects from the multiple selection candidates stored in the storage device 22 the selection candidate corresponding to the instruction value W instructed by the user U as the acoustic energy distribution D3 of the third sound field. Similarly in the second embodiment, a configuration in which multiple sound field parameters Z corresponding to different numerical values ​​of the instruction value W are pre-stored in the storage device 22 as selection candidates is also conceivable.

[0060] (3) In each of the above-described embodiments, K acoustic components C(q,1)~C(q,K) were generated from the observed signal X(q) by discrete wavelet transform. However, the method for generating K acoustic components C(q,1)~C(q,K) is not limited to the above examples. For example, K acoustic components C(q,1)~C(q,K) corresponding to different frequency bands may be generated by discrete Fourier transform on the observed signal X(q). However, the above-described embodiments that utilize discrete wavelet transform have the advantage of being able to achieve both frequency resolution and time resolution compared to embodiments that utilize discrete Fourier transform. Furthermore, K acoustic components C(q,1)~C(q,K) may be generated from the observed signal X(q) by using a filter bank composed of multiple band-pass filters with different passbands. As can be understood from the above explanation, the generation of acoustic components C(q,1)~C(q,K) is not limited to calculations in the frequency domain, but may also be realized by calculations in the time domain.

[0061] (4) In the above-described embodiments, an example was given in which the playback system 25 is equipped with a plurality of speakers 251 arranged around the user U. However, the playback system 25 may also be headphones worn on the user U's head. Headphones include earphones worn on the user U's ears.

[0062] In the case where the playback system 25 is headphones, the head-related transfer function (head-related impulse response) is combined with the sound field parameter Z3, and the sound field of the acoustic signal A is controlled using the combined sound field parameter Z3. In other words, binaural playback is realized in which the user U perceives a third sound field. It is also conceivable that the head-related transfer function is convolved into the acoustic signal A, which reflects the sound field parameter Z3.

[0063] (5) In each of the above-described embodiments, the control device 11 of the information provision system 10 performed an analysis process to generate sound field parameters Z using Q observation signals X(1) to X(Q), but the control device 21 of the information processing system 20 may also perform the analysis process.

[0064] (6) For example, the information processing system 20 in each of the above forms may be realized by a server device that communicates with an information device such as a smartphone or tablet terminal. The control device 21 of the information processing system 20 receives an instruction value W from the information device via the communication device 23 in response to an instruction from the user U of the information device. The control device 21 generates the sound field parameter Z3 of the third sound field by the above synthesis process to which the instruction value W is applied. The control device 21 transmits the sound field parameter Z3 from the communication device 23 to the information device. The information device performs playback of the acoustic signal A to which the sound field parameter Z3 is applied. Alternatively, the acoustic signal A to which the sound field parameter Z3 is applied may be transmitted from the information processing system 20 to the information device.

[0065] (7) The functions of the information processing system 20 in each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 21 and the program stored in the storage device 22, as described above. The programs exemplified above can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for transient propagation signals (transitory, propagating signals), and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the recording medium in which the program is stored in the distribution device corresponds to the non-transitory recording medium described above.

[0066] E: Addendum From the forms exemplified above, the following configuration can be understood, for example.

[0067] An information processing method according to one aspect of the present disclosure (Aspect 1) acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter representing the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. In the above aspect, the third sound field parameter of the third sound field is generated using the first sound field parameter of the first sound field and the second sound field parameter of the second sound field. Therefore, a variety of sound fields between the first and second sound fields can be reproduced.

[0068] "Sound field parameters" are parameters that represent the acoustic characteristics of a sound field formed within an acoustic space. Specifically, sound field parameters are a set of multiple expansion coefficients Bnm corresponding to different spherical harmonics Ynm, or parameters of the distribution of acoustic energy on a single sphere (ΣnΣm{BnmYnm}).

[0069] The "first sound field parameter" is calculated, for example, using the results of sound collection in the acoustic space where the first sound field is formed. The "second sound field parameter" is calculated, for example, using the results of sound collection in the acoustic space where the second sound field is formed. However, the first and second sound field parameters are not limited to parameters obtained by actual sound collection in the acoustic space, but may also be parameters calculated by computational processing, for example. For example, by using the sound field parameter generated by the information processing method of this disclosure as one of the first and second sound field parameters, sound field parameters corresponding to further separate sound fields may be generated.

[0070] A "third sound field" is a sound field that has different characteristics from, for example, the first and second sound fields. For example, a third sound field is a sound field that is in the process of bringing one of the first and second sound fields closer to the other. In other words, a third sound field can also be described as a sound field that has intermediate characteristics between the first and second sound fields. Note that "intermediate" means that the characteristics of both the first and second sound fields are reflected, and is not limited to a sound field in which the first and second sound fields are reflected equally. For example, a sound field in which one of the first and second sound fields is predominantly reflected compared to the other is also included in the "third sound field". Specifically, a form in which the third sound field parameters are generated by a weighted sum of the first sound field parameters and the second sound field parameters, or a form in which the third sound field parameters are generated such that the peak of acoustic energy is located midway between the peak of acoustic energy in the first sound field and the peak of acoustic energy in the second sound field, is envisioned.

[0071] In a specific example of Embodiment 1 (Embodiment 2), in the generation of the third sound field parameter, the third sound field parameter is generated in which each of the first sound field parameter and the second sound field parameter is reflected to a degree corresponding to the user's instructions. In the above embodiment, the degree of influence of the first sound field parameter and the second sound field parameter on the third sound field parameter is controlled according to the user's instructions. Therefore, a third sound field that corresponds to the user's intention can be reproduced.

[0072] In a specific example of Embodiment 2 (Embodiment 3), the generation of the third sound field parameter is performed by a calculation that applies an instruction value that changes continuously in response to the user's operation, thereby generating the third sound field parameter from the first sound field parameter and the second sound field parameter. In the above embodiment, the instruction value applied to the generation of the third sound field parameter changes continuously in response to the user's instruction. Therefore, the third sound field parameter can be generated for any third sound field that is in the process of continuously changing from one of the first and second sound fields to the other. The instruction value is a numerical value that changes continuously in response to an operation on an operator such as a rotatable knob or a linearly movable slider.

[0073] In any specific example of Embodiments 1 to 3 (Embodiment 4), each of the multiple acoustic signals supplied to different speakers is further weighted according to the third sound field parameter. In the above embodiment, each of the multiple acoustic signals weighted according to the third sound field parameter is supplied to different speakers. Therefore, a user listening to the sound radiated from the multiple speakers can perceive a third sound field different from the first and second sound fields.

[0074] In any specific example of Embodiments 1 to 4 (Embodiment 5), each of the first sound field parameter and the second sound field parameter includes multiple expansion coefficients corresponding to different spherical harmonics. In any specific example of Embodiments 1 to 4 (Embodiment 6), each of the first sound field parameter and the second sound field parameter is a parameter representing the distribution of acoustic energy on a single sphere. The "distribution of acoustic energy" is, for example, the weighted sum (ΣnΣm{BnmYnm}) of multiple spherical harmonics Ynm to which multiple expansion coefficients Bnm are applied as weighted values.

[0075] In any specific example of Embodiments 1 to 5 (Embodiment 7), the third sound field parameter is generated by a weighted sum of the first sound field parameter and the second sound field parameter. In the above embodiment, the third sound field parameter is generated by a weighted sum (e.g., average) of the first sound field parameter and the second sound field parameter. Therefore, the processing load required for generating the third sound field parameter can be reduced.

[0076] In any specific example (Aspect 8) of Embodiments 1 to 5, the generation of the third sound field parameter is performed such that the acoustic energy in the third sound field is located midway between the peak of acoustic energy in the first sound field and the peak of acoustic energy in the second sound field. In the above embodiment, the third sound field parameter is generated such that the peak of acoustic energy in the third sound field is located midway between the peak of acoustic energy in the first sound field and the peak of acoustic energy in the second sound field. Therefore, it is possible to generate a third sound field parameter for a third sound field in which the acoustic characteristics intermediate between the first and second sound fields can be clearly perceived by the listener.

[0077] An information processing system according to one aspect of the present disclosure (Aspect 9) comprises an acquisition unit that acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and a generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter representing the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. The information processing system includes not only general-purpose computer systems but also computer systems for playing electronic musical instruments and the like.

[0078] A program according to one aspect of the present disclosure (Aspect 10) causes a computer system to function as an acquisition unit that acquires a first sound field parameter representing the characteristics of a first sound field and a second sound field parameter representing the characteristics of a second sound field having different acoustic characteristics from the first sound field, and a generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter representing the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. [Explanation of symbols]

[0079] 100... Information systems, 200... Communication networks, 10... Information provision systems, 11, 21... Control devices, 12, 22... Memory devices, 13, 23... Communication devices, 20... Information processing systems, 24... Operating devices, 25... Playback systems, 251... Speakers, 26... Keyboards, 27... Sound source devices, 30... Spherical microphone arrays, 31... Enclosures, 32... Microphones, 40... Electronic musical instruments.

Claims

1. First sound field parameters representing the characteristics of the first sound field and second sound field parameters representing the characteristics of the second sound field, which has different acoustic characteristics from the first sound field, are obtained. Using the first sound field parameter and the second sound field parameter, a third sound field parameter is generated that represents the characteristics of a third sound field having different acoustic properties from the first and second sound fields. Each of the first and second sound field parameters includes a plurality of expansion coefficients corresponding to different spherical harmonics. Information processing methods implemented by computer systems.

2. First sound field parameters representing the characteristics of the first sound field and second sound field parameters representing the characteristics of the second sound field, which has different acoustic characteristics from the first sound field, are obtained. Using the first sound field parameter and the second sound field parameter, a third sound field parameter is generated that represents the characteristics of a third sound field having different acoustic properties from the first and second sound fields. Each of the first and second sound field parameters is a parameter that represents the distribution of acoustic energy on a sphere. Information processing methods implemented by computer systems.

3. In generating the third sound field parameter, the third sound field parameter is generated in which each of the first and second sound field parameters is reflected to the extent indicated by the user's instructions. The information processing method according to claim 1 or claim 2.

4. In generating the third sound field parameter, the third sound field parameter is generated from the first sound field parameter and the second sound field parameter by performing a calculation that applies an instruction value that changes continuously in response to the user's operation. The information processing method of claim 3.

5. Furthermore, each of the multiple acoustic signals supplied to different speakers is weighted according to the third sound field parameter. The information processing method according to claim 1 or claim 2.

6. In generating the third sound field parameter, the third sound field parameter is generated by a weighted sum of the first sound field parameter and the second sound field parameter. The information processing method according to claim 1 or claim 2.

7. In generating the third sound field parameters, the parameters are generated such that the acoustic energy in the third sound field is located midway between the peak of the acoustic energy in the first sound field and the peak of the acoustic energy in the second sound field. The information processing method according to claim 1 or claim 2.

8. An acquisition unit that acquires a first sound field parameter representing the characteristics of the first sound field and a second sound field parameter representing the characteristics of the second sound field which has different acoustic characteristics from the first sound field. A generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter that represents the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. It is equipped with, Each of the first and second sound field parameters includes a plurality of expansion coefficients corresponding to different spherical harmonics. Information processing system.

9. An acquisition unit that acquires a first sound field parameter representing the characteristics of the first sound field and a second sound field parameter representing the characteristics of the second sound field which has different acoustic characteristics from the first sound field. A generation unit that uses the first sound field parameter and the second sound field parameter to generate a third sound field parameter that represents the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields. It is equipped with, Each of the first and second sound field parameters is a parameter that represents the distribution of acoustic energy on a sphere. Information processing system.

10. An acquisition unit that acquires a first sound field parameter representing the characteristics of the first sound field, and a second sound field parameter representing the characteristics of the second sound field which has different acoustic characteristics from the first sound field, and A generation unit that generates a third sound field parameter that represents the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields, using the first and second sound field parameters. It is a program that makes a computer system function as follows: Each of the first and second sound field parameters includes a plurality of expansion coefficients corresponding to different spherical harmonics. program.

11. An acquisition unit that acquires a first sound field parameter representing the characteristics of the first sound field, and a second sound field parameter representing the characteristics of the second sound field which has different acoustic characteristics from the first sound field, and A generation unit that generates a third sound field parameter that represents the characteristics of a third sound field having different acoustic characteristics from the first and second sound fields, using the first and second sound field parameters. It is a program that makes a computer system function as follows: Each of the first and second sound field parameters is a parameter that represents the distribution of acoustic energy on a sphere. program.

Images