Information processing method, information processing system, and program
By obtaining the characteristic parameters of different audio spaces and using spherical microphone arrays and discrete wavelet conversion technology, new sound field parameters with intermediate audio characteristics are generated, which solves the problem of reproducing sound field in the existing technology, and realizes the audio space diversified control and perceived effects.
Patent Information
- Application Number
- CN202280094650.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-05-23
- Filing Date
- 2022-10-31
- Publication Date
- 2024-11-19
AI Technical Summary
The prior art is difficult to effectively reproduce the sound field characteristics of different audio spaces, resulting in the inability to accurately control and simulate multiple sound fields for audio playback.
By obtaining characteristic parameters representing different audio spaces, using spherical microphone arrays and discrete wavelet conversion technology, new sound field parameters with intermediate audio characteristics are generated to realize the reproduction and control of the audio space.
The sound field features of different sound spaces are reproduced and controlled, and new sound fields can be generated different from the original sound space features, allowing the listener to perceive the transformation and mixing effects of multiple sound fields.
Smart Images

Figure CN118985141A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a technology for controlling a sound field. Background Art
[0002] Various techniques for controlling the sound field perceived by the listener have been proposed in the past. For example, Patent Document 1 discloses a technique in which a sound field information parameter group is calculated based on signals recorded by a plurality of microphones, and the origin of the sound field information parameter group is moved by a moving operator to generate a new sound field information parameter group.
[0003] Patent Document 1: Japanese Patent Application Publication No. 2017-191980 Summary of the invention
[0004] With the popularization of audio playback technologies such as advanced ambisonics technology, there is a high demand for reproducing various sound fields. In view of the above situation, one embodiment of the present invention aims to reproduce various sound fields.
[0005] In order to solve the above problems, an information processing method according to one embodiment of the present invention obtains a first sound field parameter representing a characteristic of a first sound field, and a second sound field parameter representing a characteristic of a second sound field having acoustic characteristics different from those of the first sound field, and generates a third sound field parameter using the first sound field parameter and the second sound field parameter, the third sound field parameter representing a characteristic of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field.
[0006] An information processing system according to one embodiment of the present invention comprises: an acquisition unit that acquires a first sound field parameter indicating a feature of a first sound field and a second sound field parameter indicating a feature of a second sound field having acoustic characteristics different from those of the first sound field; and a generation unit that generates a third sound field parameter indicating a feature of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field using the first sound field parameter and the second sound field parameter.
[0007] A program according to one embodiment of the present invention causes a computer system to function as: an acquisition unit that acquires first sound field parameters representing characteristics of a first sound field and second sound field parameters representing characteristics of a second sound field having acoustic characteristics different from those of the first sound field; and a generation unit that generates third sound field parameters representing characteristics of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field using the first sound field parameters and the second sound field parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 This is a block diagram illustrating the configuration of the information system according to the first embodiment.
[0009] Figure 2 This is a block diagram illustrating the configuration of an information providing system.
[0010] Figure 3 This is an illustration of a spherical microphone array.
[0011] Figure 4 This is a flowchart of the analysis process.
[0012] Figure 5 This is an illustration of discrete wavelet transform.
[0013] Figure 6 This is a diagram illustrating spherical harmonic functions.
[0014] Figure 7 This is a block diagram illustrating the configuration of an information processing system.
[0015] Figure 8 This is a flowchart of the playback process.
[0016] Fig. 9 is a flowchart of the synthesis process.
[0017] Fig.10 This is an explanatory diagram of the synthesis process.
[0018] Fig.11 It is an explanatory diagram of the synthesis process according to the second embodiment.
[0019] Fig.12 This is a block diagram illustrating the configuration of an electronic musical instrument according to a third embodiment. DETAILED DESCRIPTION
[0020] A: First embodiment
[0021] Figure 1 1 is a block diagram illustrating the configuration of an information system 100 according to the first embodiment. The information system 100 according to the first embodiment includes an information providing system 10 and an information processing system 20. The information processing system 20 can communicate with the information providing system 10 via a communication network 200 such as the Internet. The information providing system 10 is implemented, for example, by a server system, and the information processing system 20 is implemented, for example, by an information device such as a smartphone, a tablet terminal, or a personal computer.
[0022] The information providing system 10 is a computer system that generates sound field parameters Z (Z1, Z2) representing the characteristics of a specific sound field. The characteristics of the sound field are, for example, the distribution of the sound pressure of the incoming sound relative to the listening point. Specifically, the distribution of the acoustic energy on the sphere with the listening point as the center is represented by the sound field parameters Z. The sound field parameters Z of a specific sound field are also represented as parameters for reproducing the sound field in an arbitrary space.
[0023] The information providing system 10 of the first embodiment generates a sound field parameter Z1 representing the characteristics of the first sound field and a sound field parameter Z2 representing the characteristics of the second sound field. The first sound field and the second sound field are sound fields formed in acoustic spaces having different acoustic characteristics. For example, the first sound field and the second sound field are sound fields corresponding to different sound halls. Specifically, various conditions related to the propagation of sound waves, such as the shape, size, or sound absorption rate of the acoustic space, are different between the first sound field and the second sound field. In addition, the first sound field or the second sound field may be an anechoic room in which reflection at the inner wall surface hardly occurs, in addition to being an acoustic hall designed for specific acoustic characteristics. In the following description, when there is no need to distinguish between the first sound field and the second sound field, both are collectively referred to as "observation sound field". The sound field parameter Z1 and the sound field parameter Z2 are provided to the information processing system 20 via the communication network 200. In addition, the sound field parameter Z1 is an example of the "first sound field parameter", and the sound field parameter Z2 is an example of the "second sound field parameter".
[0024] The information processing system 20 is set in the sound space R where the user U is located. The information processing system 20 generates a sound field parameter Z3 using the sound field parameter Z1 and the sound field parameter Z2. The sound field parameter Z3 is a parameter that represents the characteristics of the third sound field having acoustic characteristics different from the first sound field and the second sound field. Specifically, the third sound field is a sound field having acoustic characteristics intermediate between the first sound field and the second sound field. That is, the third sound field is a sound field that reflects both the acoustic characteristics of the first sound field and the acoustic characteristics of the second sound field. In addition, the third sound field is not limited to a sound field that equally reflects the acoustic characteristics of the first sound field and the acoustic characteristics of the second sound field. For example, a sound field that reflects the first sound field more advantageously than the second sound field, or a sound field that reflects the second sound field more advantageously than the first sound field is also included in the "third sound field". The information processing system 20 reproduces the third sound field in the sound space R using the sound field parameter Z3. That is, the information processing system 20 controls the sound reproduction so that the user U in the sound space R perceives the third sound field. In addition, the sound field parameter Z3 is an example of the “third sound field parameter”.
[0025] [Information provision system 10]
[0026] Figure 2 1 is a block diagram illustrating the configuration of the information providing system 10. The information providing system 10 includes a control device 11, a storage device 12, and a communication device 13. The information providing system 10 may be implemented as a single device or may be implemented by a plurality of devices configured separately from each other.
[0027] The control device 11 is a single or multiple processors that control various elements of the information providing system 10. Specifically, the control device 11 is composed of one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).
[0028] The communication device 13 communicates with the information processing system 20 via the communication network 200. For example, the communication device 13 sends the sound field parameter Z1 and the sound field parameter Z2 to the information processing system 20. The communication between the communication device 13 and the communication network 200 may be either wired communication or wireless communication.
[0029] The storage device 12 is a single memory or multiple memories that store programs executed by the control device 11 and various data used by the control device 11. As the storage device 12, for example, a well-known recording medium such as a semiconductor recording medium and a magnetic recording medium, or a combination of multiple recording media is used. In addition, as the storage device 12, for example, a removable recording medium that can be attached to and detached from the information providing system 10, or a recording medium that the control device 11 can access via the communication network 200 (for example, cloud storage) can also be used.
[0030] The storage device 12 stores Q observation signals X(1) to X(Q) for each of the first sound field and the second sound field. The Q observation signals X(1) to X(Q) corresponding to the first sound field are signals representing the waveforms of sound waves picked up in parallel in the first sound field. Similarly, the Q observation signals X(1) to X(Q) corresponding to the second sound field are signals representing the waveforms of sound waves picked up in parallel in the second sound field.
[0031] For the generation of Q observation signals X(1) to X(Q) of the observation sound field (first sound field / second sound field), use Figure 3The spherical microphone array 30 is a sound collection device in which Q microphones 32 are dispersedly arranged on the surface of a spherical frame 31 with a radius r. An arbitrary observation signal X(q) is a signal representing the waveform of the sound picked up by the qth (q=1~Q) microphone 32 among the Q microphones 32 in the observation sound field. The observation signal X(q) of the first sound field is recorded when the spherical microphone array 30 is set in the first sound field, and the observation signal X(q) of the second sound field is recorded when the spherical microphone array 30 is set in the second sound field. Therefore, the acoustic characteristics of the observation sound field are reflected in each observation signal X(q).
[0032] Figure 4 1 is a flowchart of a process (hereinafter referred to as "analysis process") for generating sound field parameters Z by the control device 11. The analysis process is performed for each of the first sound field and the second sound field. That is, the sound field parameter Z1 is generated by the analysis process related to the first sound field, and the sound field parameter Z2 is generated by the analysis process related to the second sound field.
[0033] The control device 11 obtains Q observation signals X(1) to X(Q)(Sa1) recorded in the observation sound field (first sound field / second sound field) from the storage device 12. The control device 11 decomposes each observation signal X(q) into K sound components C(q,1) to C(q,K)(Sa2). The K sound components C(q,1) to C(q,K) are components corresponding to different frequency bands in the observation signal X(q).
[0034] The control device 11 of the first embodiment generates sound components C(q,1) to C(q,K) from the observation signal X(q) by discrete wavelet transform. Specifically, each sound component C(q,k) is generated by using Figure 5 The Haar wavelet is illustrated. Figure 5 The symbol "H" refers to a high-pass filter (HPF), and the symbol "L" refers to a low-pass filter (LPF). In addition, the symbol "DS" refers to downsampling, which is to reduce the sampling frequency by half.
[0035] like Figure 5 As shown in the example, discrete wavelet transform generates one approximate component c(q,0) and K detailed components c(q,1) to c(q,K). The control device 11 generates K acoustic components C(q,1) to C(q,K) by calculating the following equation (1).
[0036] [Formula 1]
[0037]
[0038] The symbol λ in equation (1) is a predetermined positive number, which is set according to the SN (signal-to-noise) ratio of the observation signal X(q). The acoustic component C(q,k) corresponds to the sound pressure of the kth component of the incoming sound to the qth microphone 32 of the spherical microphone array 30 .
[0039] like Figure 4 As shown in the example, the control device 11 generates a sound field parameter Z (Sa3) of the observed sound field. The sound field parameter Z is a set of expansion coefficients Bnm(k) corresponding to the acoustic components C(q, k) (C(1, k) to C(Q, k)) of the Q observed signals X(1) to X(Q). The expansion coefficient Bnm(k) is as follows: Figure 6 As shown in the example, it is a weighted value (spherical harmonic coefficient) corresponding to the spherical harmonic function Ynm corresponding to the combination of degree n and order m. Specifically, the control device 11 calculates the expansion coefficient Bnm(k) by the operation of the following formula (2).
[0040] [Formula 2]
[0041]
[0042] The function Jn in equation (2) is a spherical Bessel function of order n, and the symbol * is a conjugate complex number. The symbol dΩ is a surface integral. The symbol r is the radius of the spherical surface (ie, the frame 31 ) where sound waves are collected, as described above.
[0043] As understood from the above description, the sound field parameter Z of the first embodiment includes a plurality of expansion coefficients Bnm(k) corresponding to different spherical harmonic functions Ynm. Specifically, the sound field parameter Z includes a plurality of expansion coefficients Bnm(k) corresponding to different combinations of the order n and the number of bits m for each of the K frequency bands. As understood from the above description, the control device 11 functions as an HOA (higher order Ambisonics) encoder. The information providing system 10 sends the sound field parameter Z1 of the first sound field and the sound field parameter Z2 of the second sound field generated according to the above process to the information processing system 20.
[0044] [Information processing system 20]
[0045] Figure 7 2 is a block diagram illustrating the configuration of the information processing system 20. The information processing system 20 includes a control device 21, a storage device 22, a communication device 23, an operation device 24, and a playback system 25. In addition, the information processing system 20 may be implemented as a single device or may be implemented by a plurality of devices configured separately from each other.
[0046] The control device 21 is a single or multiple processors that controls each element of the information processing system 20. Specifically, the control device 21 is configured by one or more processors such as a CPU, a GPU, an SPU, a DSP, an FPGA, or an ASIC.
[0047] The communication device 23 communicates with the information providing system 10 via the communication network 200. For example, the communication device 23 receives the sound field parameter Z1 and the sound field parameter Z2 from the information providing system 10. The communication between the communication device 23 and the communication network 200 may be either wired communication or wireless communication.
[0048] The storage device 22 is a single memory or multiple memories that store programs executed by the control device 21 and various data used by the control device 21. As the storage device 22, for example, a well-known recording medium such as a semiconductor recording medium and a magnetic recording medium, or a combination of multiple recording media is used. In addition, as the storage device 22, for example, a removable recording medium that can be attached to and detached from the information providing system 10, or a recording medium that the control device 21 can access via the communication network 200 (for example, cloud storage) can also be used.
[0049] The storage device 22 of the first embodiment stores the sound field parameters Z1 and Z2 received by the communication device 23. In addition, the storage device 22 stores the acoustic signal A. The acoustic signal A is a signal of multiple channels representing the waveform of sound such as a musical performance or singing sound. The data format of the acoustic signal A is arbitrary.
[0050] The operating device 24 is an input device that receives an operation by the user U. For example, the operating device 24 may include an operating element operated by the user U or a touch panel that detects a contact by the user U. The operating device 24 that is separate from the information processing system 20 may be connected to the information processing system 20 by wire or wirelessly.
[0051] The user U can indicate the indication value W to the information processing system 20 by operating the operating device 24. The indication value W is a variable indicating the extent to which the sound field parameter Z1 and the sound field parameter Z2 are reflected in the sound field parameter Z3. The indication value W changes continuously in response to the operation of the operating device 24 by the user U. For example, the operating device 24 includes a rotary operating member such as a rotatable knob, or a reciprocating operating member such as a slider that can reciprocate linearly. The indication value W changes continuously in response to the angle of the rotary operating member or the position of the reciprocating operating member. Specifically, the indication value W is set in response to the instruction from the user U within a range of greater than 0 and less than 1.
[0052] The broadcasting system 25 is an audio system composed of a plurality of speakers 251 corresponding to different channels. As the broadcasting system 25, for example, a surround sound system composed of three or more speakers 251 or a stereo sound system composed of two speakers 251 is exemplified.
[0053] A plurality of speakers 251 are provided at different positions in the sound space. For example, a plurality of speakers 251 are arranged around the user U. Each speaker 251 is a sound-reproducing device that radiates the sound represented by the sound signal A. Specifically, the sound signal A of each channel is supplied to the speaker 251 of the channel and the sound wave is radiated. In addition, for the sake of convenience, the D / A converter that converts the sound signal A of each channel from digital to analog and the amplifier that amplifies the sound signal A are omitted from the illustration. In addition, the playback system 25 that is separate from the information processing system 20 is connected to the information processing system 20 by wire or wirelessly.
[0054] Figure 8 This is a flowchart of the process (hereinafter referred to as "playback process") performed by the control device 21. For example, the playback process is started by the operation of the user U on the operating device 24. If the playback process is started, the control device 21 obtains the sound field parameter Z1 and the sound field parameter Z2 (Sb1) from the storage device 22. As described above, the control device 21 functions as an element (acquisition unit) for acquiring the sound field parameter Z1 and the sound field parameter Z2. In addition, the acquisition target of the sound field parameter Z1 and the sound field parameter Z2 is not limited to the storage device 22. For example, the control device 21 can receive the sound field parameter Z1 and the sound field parameter Z2 sent from the information providing system 10 through the communication device 23 via the communication network 200.
[0055] The control device 21 determines whether or not an instruction to change the instruction value W is received from the user U (Sb2). If an instruction to change the instruction value W is received (Sb2: YES), the control device 21 updates the instruction value W to the value instructed by the user U (Sb3).
[0056] The control device 21 performs a synthesis process (Sb4). The synthesis process is a process of generating a sound field parameter Z3 using the sound field parameter Z1 and the sound field parameter Z2. For the synthesis process, the updated instruction value W is applied. On the other hand, when the change of the instruction value W is not instructed (Sb2: NO), the update (Sb3) of the instruction value W and the synthesis process (Sb4) are not performed. As understood from the above description, the control device 21 functions as an element (generating unit) that generates the sound field parameter Z3 based on the sound field parameter Z1 and the sound field parameter Z2.
[0057] Fig. 9Flowchart of the synthesis process. When the synthesis process is started, the control device 21 generates the acoustic energy distribution D1 of the first sound field according to the sound field parameter Z1 (Sb41). The acoustic energy distribution D1 is the distribution of the acoustic energy (sound pressure) on the sphere of radius r. Specifically, the control device 21 calculates the weighted sum of multiple spherical harmonic functions Ynm applied with multiple expansion coefficients Bnm(k) included in the sound field parameter Z1 as weighted values as the acoustic energy distribution D1.
[0058] Similarly, the control device 21 generates the acoustic energy distribution D2 of the second sound field based on the sound field parameter Z2 (Sb42). Specifically, the control device 21 calculates the weighted sum of the plurality of spherical harmonic functions Ynm applied using the plurality of expansion coefficients Bnm(k) included in the sound field parameter Z2 as weighted values as the acoustic energy distribution D2. In addition, the order of generating the acoustic energy distribution D1 (Sb41) and generating the acoustic energy distribution D2 (Sb42) may be reversed.
[0059] Fig.10 is an illustration of the synthesis process. Fig.10 The diagram schematically shows an acoustic energy distribution D1 of a first sound field and an acoustic energy distribution D2 of a second sound field. The acoustic energy distribution D1 includes a peak value P1 of acoustic energy, and the acoustic energy distribution D2 includes a peak value P2 of acoustic energy.
[0060] The control device 21 generates the acoustic energy distribution D3 of the third sound field such that the peak value P3 of the acoustic energy is located between the peak value P1 of the acoustic energy distribution D1 and the peak value P2 of the acoustic energy distribution D2 ( Sb43 ).
[0061] Specifically, the position of the peak P3 of the acoustic energy distribution D3 is set to a position obtained by dividing the position of the peak P1 of the acoustic energy distribution D1 and the position of the peak P2 of the acoustic energy distribution D2 in accordance with the indication value W. Specifically, the closer the indication value W is to the minimum value 0, the closer the peak P3 of the acoustic energy distribution D3 is to the peak P1 of the acoustic energy distribution D1, and the closer the indication value W is to the maximum value 1, the closer the peak P3 of the acoustic energy distribution D3 is to the peak P2 of the acoustic energy distribution D2. When the indication value W is set to the minimum value 0, the acoustic energy distribution D1 is applied as the acoustic energy distribution D3, and when the indication value W is set to the maximum value 1, the acoustic energy distribution D2 is applied as the acoustic energy distribution D3. In addition, for the comparison between the peak P1 of the acoustic energy distribution D1 and the peak P2 of the acoustic energy distribution D2, for example, the Wasserstein distance is used.
[0062] The control device 21 generates a sound field parameter Z3 corresponding to the acoustic energy distribution D3 generated in the above process (Sb44). The sound field parameter Z3 includes a plurality of expansion coefficients Bnm(k) corresponding to different spherical harmonic functions Ynm. As can be understood from the above description, the control device 21 generates the sound field parameter Z3 reflecting each of the sound field parameter Z1 and the sound field parameter Z2 to a degree corresponding to the instruction from the user U. The specific flow of the synthesis process is as described above.
[0063] like Figure 8 As illustrated, the control device 21 weights each of the plurality of sound signals A stored in the storage device 22 according to the sound field parameter Z3 (Sb5). That is, the control device 21 controls the volume and phase of the sound signal A of each channel so that the third sound field represented by the sound field parameter Z3 is formed in the sound space R. That is, the control device 21 functions as an HOA decoder. For the control of the sound field using the sound field parameter Z3, any known technique can be adopted.
[0064] The control device 21 provides each sound signal A controlled according to the sound field parameter Z3 to the speaker 251 corresponding to the sound signal A in the playback system 25 (Sb6). Sound waves corresponding to the sound signal A are radiated from the speakers 251, thereby forming a third sound field in the sound space R. That is, the user U can perceive the third sound field different from the first sound field and the second sound field.
[0065] The control device 21 determines whether a predetermined end condition is satisfied (Sb7). The end condition is, for example, that the end is instructed by the user U, or that all playback of the acoustic signal A has been completed. If the end condition is not satisfied (Sb7: NO), the control device 21 causes the process to proceed to step Sb2. That is, each time the user U instructs the change of the indication value W, the updating of the indication value W (Sb3) and the synthesis process (Sb4) to which the updated indication value W is applied are executed. If the end condition is satisfied (Sb7: YES), the control device 21 ends the playback process.
[0066] As described above, in the first embodiment, the sound field parameter Z3 of the third sound field is generated using the sound field parameter Z1 of the first sound field and the sound field parameter Z2 of the second sound field. Therefore, various sound fields between the first sound field and the second sound field can be reproduced. In the first embodiment, in particular, the degree of influence of the sound field parameter Z1 and the sound field parameter Z2 on the sound field parameter Z3 is controlled in accordance with the instruction (instruction value W) from the user U. Therefore, the third sound field corresponding to the intention of the user U can be reproduced. In addition, in the first embodiment, the instruction value W used for generating the sound field parameter Z3 changes continuously in accordance with the instruction from the user U. Therefore, the sound field parameter Z3 can be generated for any third sound field in the process of continuously changing from one of the first sound field and the second sound field to the other. That is, the first sound field and the second sound field can be deformed (morphing).
[0067] B: Second embodiment
[0068] In each embodiment described below, elements having the same functions as those of the first embodiment are denoted by the same reference numerals as those in the first embodiment, and detailed descriptions thereof are appropriately omitted.
[0069] Fig.11 This is an explanatory diagram of the synthesis process of the second embodiment. Fig.10 As described above, in the synthesis process of the first embodiment, the acoustic energy distribution D3 of the third sound field is generated in such a way that the peak value P3 of the acoustic energy is located between the peak value P1 of the acoustic energy of the first sound field and the peak value P2 of the acoustic energy of the second sound field. In the synthesis process, the control device 21 of the second embodiment generates the sound field parameter Z3 of the third sound field by the weighted sum of the sound field parameter Z1 and the sound field parameter Z2.
[0070] Specifically, the control device 21 generates the acoustic energy distribution D3 of the third sound field by taking the weighted sum of the acoustic energy distribution D1 and the acoustic energy distribution D2. Fig.11 As shown in the example, the acoustic energy distribution D3 includes a peak P31 corresponding to the acoustic energy distribution D1 and a peak P32 corresponding to the acoustic energy distribution D2. The peak P31 is located at the same position as the peak P1 of the acoustic energy distribution D1, and the peak P32 is located at the same position as the peak P2 of the acoustic energy distribution D2.
[0071] The value of the peak value P31 of the acoustic energy distribution D3 is set to a value obtained by multiplying the value of the peak value P1 of the acoustic energy distribution D1 by the weight value (1-W) corresponding to the indication value W. The value of the peak value P32 of the acoustic energy distribution D3 is set to a value obtained by multiplying the value of the peak value P2 of the acoustic energy distribution D2 by the indication value W. Therefore, the closer the indication value W is to the minimum value 0, the greater the value of the peak value P31, and the smaller the value of the peak value P32. On the other hand, the closer the indication value W is to the maximum value 1, the smaller the value of the peak value P31, and the larger the value of the peak value P32. When the indication value W is set to the minimum value 0, the acoustic energy distribution D1 is applied as the acoustic energy distribution D3, and when the indication value W is set to the maximum value 1, the acoustic energy distribution D2 is applied as the acoustic energy distribution D3.
[0072] The operations other than the synthesis process are the same as those of the first embodiment. Therefore, in the second embodiment, the same effects as those of the first embodiment are achieved. In addition, in the second embodiment, the sound field parameter Z3 of the third sound field is generated by the weighted sum (for example, average) of the sound field parameter Z1 and the sound field parameter Z2. Therefore, the processing load required for the synthesis process can be reduced compared to the first embodiment.
[0073] On the other hand, in the first embodiment, the sound field parameter Z3 of the third sound field is generated so that the peak value P3 is located at a position intermediate between the peak value P1 of the acoustic energy distribution D1 of the first sound field and the peak value P2 of the acoustic energy distribution D2 of the second sound field. Therefore, compared with the second embodiment, there is an advantage that the sound field parameter Z3 of the third sound field can be generated so that the user U can clearly perceive the acoustic characteristics intermediate between the first sound field and the second sound field.
[0074] C: Third embodiment
[0075] Fig.12 1 is a block diagram illustrating the configuration of an electronic musical instrument 40 according to the third embodiment. The electronic musical instrument 40 is an information processing system that reproduces sounds corresponding to a musical performance operation of a user U. Fig.12 The illustrated electronic musical instrument 40 is an electronic keyboard musical instrument having a keyboard 26 and a sound source device 27 in addition to the same elements as the information processing system 20 of the first embodiment (control device 21, storage device 22, communication device 23, operation device 24, playback system 25).
[0076] The keyboard 26 is composed of a plurality of keys corresponding to different pitches. The sound source device 27 generates an acoustic signal A, which indicates a musical sound of a pitch corresponding to a key operated by the user U. In addition, the function of the sound source device 27 can be realized by executing a program by the control device 21. That is, the element (sound source unit) that generates the acoustic signal A corresponding to the performance operation of the user U can be realized by any of a software sound source realized by the control device 21 and a hardware sound source (sound source device 27) dedicated to the generation of the acoustic signal A.
[0077] In the first embodiment, the sound field parameter Z3 (Sb5) is applied to the sound signal A stored in the storage device 22. The control device 21 of the third embodiment applies the sound field parameter Z3 to the sound signal A generated by the sound source device 27. Except for the method of obtaining the sound signal A, it is the same as the first embodiment. Therefore, in the third embodiment, the same effect as the first embodiment is achieved. In the above description, the third embodiment is described based on the first embodiment, but the second embodiment can also be applied to the electronic musical instrument 40.
[0078] In the third embodiment, as in the first embodiment, the instruction value W corresponding to the instruction of the user U is applied to the generation of the sound field parameter Z3 of the third sound field. Therefore, the user U can play the electronic musical instrument 40 in the third sound field having the desired acoustic characteristics. For example, the user U can practice playing the electronic musical instrument 40 in a sound field equivalent to a predetermined sound hall where a concert in which the user U participates is held. In addition, the electronic musical instrument 40 can be played in a sound field equivalent to, for example, a world-famous sound hall.
[0079] D: Deformation
[0080] Specific modified forms added to the above-exemplified forms are exemplified below. A plurality of forms arbitrarily selected from the above-described embodiment and the modified examples exemplified below can be appropriately combined within a range that does not contradict each other.
[0081] (1) In the above-mentioned embodiments, the sound field parameter Z (Z1 to Z3) includes a plurality of expansion coefficients Bnm(k) corresponding to different spherical harmonic functions Ynm, but the content of the sound field parameter Z is not limited to the above examples. It is also conceivable that the sound field parameter Z represents the acoustic energy distribution D (D1 to D3) on a sphere. For example, the sound field parameter Z1 represents the acoustic energy distribution D1 of the first sound field, and the sound field parameter Z2 represents the acoustic energy distribution D2 of the second sound field. Similarly, the sound field parameter Z3 representing the acoustic energy distribution D3 of the third sound field can be used. In addition, the sound field parameter Z1, the sound field parameter Z2, and the sound field parameter Z3 can be parameters of different types or forms. For example, it is also conceivable that the sound field parameter Z1 and the sound field parameter Z2 are composed of a plurality of expansion coefficients Bnm(k), and the sound field parameter Z3 represents the acoustic energy distribution D3 of the third sound field.
[0082] (2) In the first embodiment, the acoustic energy distribution D3 of the third sound field is generated by arranging the peak P3 in the middle between the peak P1 of the acoustic energy distribution D1 and the peak P2 of the acoustic energy distribution D2, but the specific flow of the synthesis process is not limited to the above example. For example, it is also conceivable that a plurality of sound field parameters Z corresponding to different numerical values of the instruction value W are stored in advance in the storage device 22 as selection candidates. The sound field parameter Z of each selection candidate can be composed of a plurality of expansion coefficients Bnm(k) or a parameter representing the acoustic energy distribution D. The position of the peak P3 is different for each selection candidate. In the synthesis process, the control device 21 selects the selection candidate corresponding to the instruction value W indicated by the user U from among the plurality of selection candidates stored in the storage device 22 as the acoustic energy distribution D3 of the third sound field. In the second embodiment, it is also conceivable that a plurality of sound field parameters Z corresponding to different numerical values of the instruction value W are stored in advance in the storage device 22 as selection candidates.
[0083] (3) In each of the above-mentioned methods, K sound components C(q, 1) to C(q, K) are generated from the observation signal X(q) by discrete wavelet transform, but the method of generating the K sound components C(q, 1) to C(q, K) is not limited to the above examples. For example, K sound components C(q, 1) to C(q, K) corresponding to different frequency bands can be generated by discrete Fourier transform of the observation signal X(q). However, the above-mentioned method using discrete wavelet transform has the advantage of being able to achieve both frequency resolution and time resolution compared to the method using discrete Fourier transform. In addition, K sound components C(q, 1) to C(q, K) can be generated from the observation signal X(q) by using a filter bank composed of a plurality of bandpass filters having different passbands. As understood from the above description, the generation of the sound components C(q, 1) to C(q, K) is not limited to the operation in the frequency domain, but can also be realized by the operation in the time domain.
[0084] (4) In the above-mentioned embodiments, the broadcasting system 25 includes a plurality of speakers 251 arranged around the user U. However, the broadcasting system 25 may be a headset worn on the head of the user U. The headset includes earphones worn on the ears of the user U.
[0085] In the case where the playback system 25 is a headphone, a head transfer function (head impulse response) is synthesized with respect to the sound field parameter Z3, and the synthesized sound field parameter Z3 is used to control the sound field of the acoustic signal A. That is, the stereo playback perceived by the user U realizes the third sound field. In addition, a method is also conceivable in which the head transfer function is convoluted with the acoustic signal A reflecting the sound field parameter Z3.
[0086] (5) In the above-mentioned embodiments, the control device 11 of the information providing system 10 performs the analysis process of generating the sound field parameter Z using the Q observation signals X(1) to X(Q). Alternatively, the control device 21 of the information processing system 20 may perform the analysis process.
[0087] (6) For example, the information processing system 20 of each of the above-mentioned modes can be realized by a server device that communicates between information devices such as smartphones and tablet terminals. The control device 21 of the information processing system 20 receives an instruction value W corresponding to an instruction from a user U of the information device from the information device through the communication device 23. The control device 21 generates a sound field parameter Z3 of the third sound field by the above-mentioned synthesis processing applying the instruction value W. The control device 21 sends the sound field parameter Z3 from the communication device 23 to the information device. The information device performs playback of the sound signal A to which the sound field parameter Z3 is applied. In addition, the sound signal A to which the sound field parameter Z3 is applied can be sent from the information processing system 20 to the information device.
[0088] (7) The functions of the information processing system 20 involved in each of the aforementioned methods are implemented as described above by the coordinated action of a single or multiple processors constituting the control device 21 and the program stored in the storage device 22. The program illustrated above is provided in the form of a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, preferably an optical recording medium (optical disc) such as a CD-ROM, and also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. In addition, as a non-transitory recording medium, it includes any recording medium other than a temporary transmission signal (transitory, propagating signal), and volatile recording media may not be excluded. In a structure in which a transmission device transmits a program via a communication network, the recording medium that stores the program in the transmission device is equivalent to the aforementioned non-transitory recording medium.
[0089] E: Appendix
[0090] According to the above-exemplified embodiment, for example, the following configuration can be understood.
[0091] An information processing method according to one embodiment of the present invention (embodiment 1) obtains a first sound field parameter representing a feature of a first sound field and a second sound field parameter representing a feature of a second sound field having acoustic characteristics different from those of the first sound field, and generates a third sound field parameter representing a feature of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field using the first sound field parameter and the second sound field parameter. In the above embodiment, the third sound field parameter of the third sound field is generated using the first sound field parameter of the first sound field and the second sound field parameter of the second sound field. Therefore, various sound fields between the first sound field and the second sound field can be reproduced.
[0092] The "sound field parameters" are parameters that represent the acoustic characteristics of the sound field formed in the acoustic space. Specifically, the sound field parameters are a set of a plurality of expansion coefficients Bnm corresponding to different spherical harmonic functions Ynm, or a parameter (ΣnΣm{BnmYnm}) of the distribution of acoustic energy on a sphere.
[0093] The "first sound field parameter" is calculated, for example, using the result of sound pickup in the acoustic space forming the first sound field. The "second sound field parameter" is calculated, for example, using the result of sound pickup in the acoustic space forming the second sound field. However, the first sound field parameter and the second sound field parameter are not limited to parameters obtained by actual sound pickup in the acoustic space, and may be parameters calculated by arithmetic processing, for example. For example, the sound field parameter generated by the information processing method of the present invention may be used as one of the first sound field parameter and the second sound field parameter, thereby further generating sound field parameters corresponding to individual sound fields.
[0094] The "third sound field" is, for example, a sound field having characteristics different from those of the first sound field and the second sound field. For example, a sound field that is in the process of gradually approaching one of the first sound field and the second sound field to the other is the third sound field. That is, the third sound field is also expressed as a sound field having characteristics intermediate between the first sound field and the second sound field. In addition, "intermediate" refers to a sound field that reflects the characteristics of both the first sound field and the second sound field, and is not limited to a sound field that equally reflects the first sound field and the second sound field. For example, a sound field in which one of the first sound field and the second sound field is reflected more advantageously than the other is also included in the "third sound field". Specifically, it is envisaged that the third sound field parameters are generated by a weighted sum of the first sound field parameters and the second sound field parameters, or the third sound field parameters are generated in a manner such that the peak of the acoustic energy exists at an intermediate position between the peak of the acoustic energy of the first sound field and the peak of the acoustic energy of the second sound field.
[0095] In a specific example (method 2) of method 1, in generating the third sound field parameter, the third sound field parameter is generated to reflect each of the first sound field parameter and the second sound field parameter to a degree corresponding to an instruction from a user. In the above method, the degree of influence of the first sound field parameter and the second sound field parameter on the third sound field parameter is controlled in accordance with an instruction from a user. Therefore, a third sound field corresponding to the intention of the user can be reproduced.
[0096] In a specific example of method 2 (method 3), in the generation of the third sound field parameter, the third sound field parameter is generated based on the first sound field parameter and the second sound field parameter by applying an operation of an indication value that changes continuously according to an operation from the user. In the above method, the indication value applied to the generation of the third sound field parameter changes continuously corresponding to the indication from the user. Therefore, the third sound field parameter can be generated for any third sound field that is in the process of continuously changing from one of the first sound field and the second sound field to the other. In addition, the indication value is a numerical value that changes continuously corresponding to the operation of an operating member such as a rotatable knob or an operating member such as a slider that can reciprocate linearly.
[0097] In a specific example (method 4) of any one of the methods 1 to 3, further, the plurality of sound signals supplied to different speakers are weighted according to the third sound field parameter. In the above method, the plurality of sound signals weighted according to the third sound field parameter are supplied to different speakers. Therefore, a user who listens to the radiated sound from the plurality of speakers can perceive the third sound field different from the first sound field and the second sound field.
[0098] In a specific example (method 5) of any one of the methods 1 to 4, the first sound field parameter and the second sound field parameter each include a plurality of expansion coefficients corresponding to different spherical harmonic functions. In addition, in a specific example (method 6) of any one of the methods 1 to 4, the first sound field parameter and the second sound field parameter are each a parameter representing the distribution of acoustic energy on a sphere. In addition, the “distribution of acoustic energy” is, for example, a weighted sum (ΣnΣm{BnmYnm}) of a plurality of spherical harmonic functions Ynm using a plurality of expansion coefficients Bnm as weighted values.
[0099] In a specific example (method 7) of any one of the methods 1 to 5, in generating the third sound field parameter, the third sound field parameter is generated by a weighted sum of the first sound field parameter and the second sound field parameter. In the above method, the third sound field parameter is generated by a weighted sum (e.g., average) of the first sound field parameter and the second sound field parameter. Therefore, the processing load required for generating the third sound field parameter can be reduced.
[0100] In a specific example (method 8) of any one of methods 1 to 5, in generating the third sound field parameters, the third sound field parameters are generated in such a manner that the acoustic energy in the third sound field is located between the peak of the acoustic energy of the first sound field and the peak of the acoustic energy of the second sound field. In the above method, the third sound field parameters are generated in such a manner that the peak of the acoustic energy of the third sound field is located between the peak of the acoustic energy of the first sound field and the peak of the acoustic energy of the second sound field. Therefore, the third sound field parameters of the third sound field can be generated in which the listener can clearly perceive the acoustic characteristics between the first sound field and the second sound field.
[0101] An information processing system according to one embodiment of the present invention (Aspect 9) comprises: an acquisition unit that acquires a first sound field parameter indicating a feature of a first sound field and a second sound field parameter indicating a feature of a second sound field having acoustic characteristics different from those of the first sound field; and a generation unit that generates a third sound field parameter indicating a feature of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field using the first sound field parameter and the second sound field parameter. The information processing system includes a computer system for playing electronic musical instruments, etc., in addition to a general-purpose computer system.
[0102] A program according to one embodiment of the present invention (embodiment 10) causes a computer system to function as: an acquisition unit that acquires first sound field parameters representing characteristics of a first sound field and second sound field parameters representing characteristics of a second sound field having acoustic characteristics different from those of the first sound field; and a generation unit that generates third sound field parameters using the first sound field parameters and the second sound field parameters, the third sound field parameters representing characteristics of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field.
[0103] Description of the label
[0104] 100…information system, 200…communication network, 10…information providing system, 11, 21…control device, 12, 22…storage device, 13, 23…communication device, 20…information processing system, 24…operating device, 25…playback system, 251…speaker, 26…keyboard, 27…sound source device, 30…spherical microphone array, 31…frame, 32…microphone, 40…electronic musical instrument.
Claims
1. An information processing method, which is implemented by a computer system, wherein: acquiring a first sound field parameter representing a characteristic of a first sound field and a second sound field parameter representing a characteristic of a second sound field having acoustic characteristics different from those of the first sound field, A third sound field parameter is generated using the first sound field parameter and the second sound field parameter, the third sound field parameter indicating a feature of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field.
2. The information processing method according to claim 1, wherein: In the generation of the third sound field parameter, the third sound field parameter is generated by reflecting each of the first sound field parameter and the second sound field parameter to a degree corresponding to an instruction from a user.
3. The information processing method according to claim 2, wherein: In generating the third sound field parameter, the third sound field parameter is generated from the first sound field parameter and the second sound field parameter by a calculation using an instruction value that changes continuously according to an operation from the user.
4. The information processing method according to any one of claims 1 to 3, wherein: Furthermore, each of the plurality of sound signals supplied to different speakers is weighted according to the third sound field parameter.
5. The information processing method according to any one of claims 1 to 3, wherein: The first sound field parameter and the second sound field parameter each include a plurality of expansion coefficients corresponding to different spherical harmonic functions.
6. The information processing method according to any one of claims 1 to 3, wherein: Each of the first sound field parameter and the second sound field parameter is a parameter indicating the distribution of acoustic energy on a spherical surface.
7. The information processing method according to any one of claims 1 to 3, wherein: In generating the third sound field parameter, the third sound field parameter is generated by a weighted sum of the first sound field parameter and the second sound field parameter.
8. The information processing method according to any one of claims 1 to 3, wherein: In generating the third sound field parameters, the third sound field parameters are generated so that the acoustic energy in the third sound field is located between the peak values of the acoustic energy in the first sound field and the peak values of the acoustic energy in the second sound field.
9. An information processing system comprising: an acquisition unit that acquires a first sound field parameter indicating a feature of a first sound field and a second sound field parameter indicating a feature of a second sound field having an acoustic characteristic different from that of the first sound field; and A generating unit generates a third sound field parameter representing a feature of a third sound field having acoustic characteristics different from those of the first sound field and the second sound field, using the first sound field parameter and the second sound field parameter.
10. A program that causes a computer system to function as the following functional unit: an acquisition unit that acquires a first sound field parameter indicating a feature of a first sound field and a second sound field parameter indicating a feature of a second sound field having an acoustic characteristic different from that of the first sound field; and A generating unit generates a third sound field parameter indicating a feature of a third sound field having an acoustic characteristic different from that of the first sound field and the second sound field, using the first sound field parameter and the second sound field parameter.
Citation Information
Patent Citations
Sound field information parameter group generation device, method and program
JP2017191980A
