Information processing methods, information processing systems, and programs

Abstract

It generates output data regarding the status of speaker devices reliably and with high accuracy. [Solution] The information processing system comprises: a first generation unit 621 that generates first data Y1 relating to the state of a speaker device by performing a first process, including processing by a physical model M1 that simulates the state of a speaker device, on input data X relating to the state of a speaker device; a second generation unit 622 that generates second data Y2 relating to the state of a speaker device by performing a second process, including processing by a machine learning-trained estimation model M2, on input data X; and a third generation unit 623 that generates output data Z relating to the state of a speaker device based on the first data Y1 and the second data Y2.

Description

Technical Field

[0001] The present disclosure relates to a technique for controlling reproduced sound by a speaker device.

Background Art

[0002] Various techniques for reducing the non-linear distortion of reproduced sound by a speaker device have been proposed conventionally. For example, Patent Document 1 discloses a configuration for reducing non-linear distortion using a lumped model that simulates the state of a speaker device. Further, Patent Document 2 discloses a configuration for calculating an input voltage for reducing non-linear distortion by inputting the displacement of a diaphragm of a speaker device into a deep neural network.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it is limited to faithfully reproduce the behavior of an actual speaker device by the lumped model of Patent Document 1. Therefore, it is actually difficult to generate output data capable of reducing non-linear distortion with high accuracy. Also, even in a configuration using a deep neural network as in Patent Document 2, it is difficult to stably generate high-precision output data. In the above description, attention was focused on reducing non-linear distortion, but the same problems occur in any scenario for estimating the state of a speaker device. In view of the above circumstances, one aspect of the present disclosure aims to stably and accurately generate output data regarding the state of a speaker device.

Means for Solving the Problems

[0005] To solve the above problems, an information processing method according to one aspect of the present disclosure generates first data relating to the state of a speaker device by performing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; generates second data relating to the state of the speaker device by performing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and generates output data relating to the state of the speaker device based on the first data and the second data.

[0006] An information processing system according to one aspect of the present disclosure comprises: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0007] A program according to one aspect of the present disclosure causes a computer system to function as follows: a first generation unit that generates first data relating to the state of a speaker device by executing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by executing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data. [Brief explanation of the drawing]

[0008] [Figure 1]This is a block diagram illustrating the configuration of the speaker system in the first embodiment. [Figure 2] This is a cross-section of the speaker unit. [Figure 3] This is a block diagram illustrating the configuration of an information processing system. [Figure 4] This is a block diagram illustrating the functional configuration of an information processing system. [Figure 5] This is a block diagram illustrating the specific configuration of the second analysis unit. [Figure 6] This is a flowchart of the output data generation process. [Figure 7] This is an explanatory diagram regarding machine learning for estimation models. [Figure 8] This is a flowchart of the learning processing unit. [Figure 9] This is a block diagram illustrating the specific configuration of the second analysis unit in the second embodiment. [Figure 10] This is a block diagram illustrating the specific configuration of the second analysis unit in the third embodiment. [Modes for carrying out the invention]

[0009] A: First Embodiment Figure 1 is a schematic diagram illustrating the configuration of the speaker system 100 in the first embodiment. The speaker system 100 of the first embodiment comprises an information processing system 10 and a speaker device 20. The information processing system 10 is supplied with an acoustic signal A from a signal supply device 30. The acoustic signal A is a signal that represents the waveform of sound. For example, the acoustic signal A is a sample sequence that represents the waveform of the vocal or instrumental sound of a musical piece.

[0010] The signal supply device 30 is, for example, a playback device that supplies an acoustic signal A recorded on a recording medium to the information processing system 10. A communication device that supplies an acoustic signal A received from a distribution device (not shown) via a communication network to the information processing system 10, or a sound collection device that generates an acoustic signal A by collecting ambient sounds, can also be used as the signal supply device 30. The signal supply device 30 may also be understood as an element of the speaker system 100.

[0011] The information processing system 10 generates an acoustic signal B by signal processing of acoustic signal A and supplies acoustic signal B to the speaker device 20. The speaker device 20 is a sound emitting device that radiates sound waves of the waveform represented by acoustic signal B. The speaker device 20 of the first embodiment comprises the speaker unit 50 shown in Figure 2. The speaker unit 50 comprises a frame 51, a magnet 52, a voice coil 53, a diaphragm 54, an edge 55, and a damper 56.

[0012] The frame 51 is a structural element that constitutes the exterior of the speaker unit 50. The magnet 52 is an annular permanent magnet. The voice coil 53 is a coil that can be displaced axially within the magnetic field generated by the magnet 52. The diaphragm 54 is a frustoconical structure. The inner edge of the diaphragm 54 is fixed to the voice coil 53. The outer edge of the diaphragm 54 is connected to the frame 51 via an edge 55. The diaphragm 54 and the frame 51 are also connected via a damper 56. The edge 55 and the damper 56 are annular elastic bodies. Sound waves are radiated as the diaphragm 54 reciprocates axially.

[0013] The high-frequency components of the sound reproduced by the speaker device 20 are accompanied by nonlinear distortion due to the nonlinearity of each element constituting the speaker device 20. The information processing system 10 generates acoustic signal B from acoustic signal A in such a way that the nonlinear distortion caused by the nonlinearity of the speaker device 20 is reduced.

[0014] FIG. 3 is a block diagram illustrating the configuration of the information processing system 10. The information processing system 10 includes a control device 11, a storage device 12, an input device 13, and an output device 14. The information processing system 10 is realized by an information device such as a smartphone, a tablet terminal, or a personal computer. Note that the information processing system 10 can be realized not only as a single device but also as a plurality of devices separately configured from each other.

[0015] The control device 11 is composed of one or more processors that control each element of the information processing system 10. For example, the control device 11 is composed of one or more types of processors such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0016] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is composed of a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 12 may be composed of a combination of multiple types of recording media. Also, a portable recording medium that can be attached to and detached from the information processing system 10, or a recording medium (e.g., cloud storage) that can be written to or read from by the control device 11 via a communication network may be used as the storage device 12.

[0017] The input device 13 receives the acoustic signal A from the signal supply device 30. For example, an input interface such as a USB (Universal Serial Bus) terminal, an HDMI (High-Definition Multimedia Interface / registered trademark) terminal, a MIDI (Musical Instrument Digital Interface) terminal, or a phone terminal is used as the input device 13. In the form where an analog acoustic signal A is supplied from the signal supply device 30, an A / D converter for converting the acoustic signal A from analog to digital is mounted on the input device 13.

[0018] The output device 14 supplies the acoustic signal B to the speaker device 20. Specifically, the output device 14 includes a D / A converter for converting the acoustic signal B from digital to analog and an amplifier for amplifying the acoustic signal B. Note that an output device 14 separate from the information processing system 10 may be connected to the information processing system 10 by wire or wirelessly.

[0019] FIG. 4 is a block diagram illustrating a functional configuration of the information processing system 10. The control device 11 realizes a plurality of functions (first analysis unit 61, second analysis unit 62) for processing the acoustic signal A by executing a program stored in the storage device 12.

[0020] The first analysis unit 61 generates input data X by processing the acoustic signal A. The input data X is data related to the state of the speaker device 20 when the acoustic signal A is supplied to the speaker device 20. Specifically, the input data X represents the state of the diaphragm 54 in the speaker device 20. For example, the first analysis unit 61 generates input data X including the displacement of the diaphragm 54, the velocity, and the acceleration of the diaphragm 54 when the acoustic signal A is supplied to the speaker device 20.

[0021] The generation of input data X by the first analysis unit 61 is repeated sequentially at a predetermined period. That is, a time series of multiple input data X corresponding to different sections of the acoustic signal A is generated. The first analysis unit 61 is composed of a linear model that simulates the behavior of each element of the speaker device 20. Any known mathematical model can be arbitrarily adopted as the linear model that constitutes the first analysis unit 61.

[0022] The second analysis unit 62 generates output data Z by processing the input data X. The output data Z is data representing the voltage (input voltage) to be supplied to the speaker device 20. The generation of output data Z by the second analysis unit 62 is repeated sequentially at a predetermined period. That is, a time series of multiple output data Z corresponding to different sections of the acoustic signal A is generated. The output device 14 generates an acoustic signal B whose signal value is the voltage represented by the output data Z. As described above, the second analysis unit 62 generates output data Z in such a way that nonlinear distortion caused by the nonlinearity of the speaker device 20 is reduced from the reproduced sound. As described above, according to the first embodiment, the voltage (acoustic signal B) input to the speaker device 20 can be determined from the state of the diaphragm 54 in the speaker device 20.

[0023] Figure 5 is a block diagram illustrating the specific configuration of the second analysis unit 62. The second analysis unit 62 comprises a first generation unit 621, a second generation unit 622, and a third generation unit 623. The input data X generated by the first analysis unit 61 is supplied in parallel to the first generation unit 621 and the second generation unit 622.

[0024] The first generation unit 621 generates first data Y1 from input data X. First data Y1 is data relating to the state of the speaker device 20. The state of the speaker device 20 represented by input data X is different from the state of the speaker device 20 represented by first data Y1. Specifically, first data Y1 is data representing the voltage that should be supplied to the speaker device 20. The generation of first data Y1 by the first generation unit 621 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple first data Y1 corresponding to different sections of the acoustic signal A is generated.

[0025] The first generation unit 621 generates first data Y1 by performing a first process on input data X. The first process is signal processing including processing by a physical model M1. The physical model M1 is a mathematical model that simulates the state of the speaker device 20. Specifically, the physical model M1 includes a plurality of characteristic parameters P related to the speaker device 20. Each characteristic parameter P is a TS (Thiele Small) parameter that represents the electrical or mechanical characteristics of each element of the speaker device 20 (e.g., magnet 52, voice coil 53, diaphragm 54, edge 55, or damper 56). For example, a known lumped parameter model that approximates the behavior of each element of the speaker device 20 using lumped parameters is used as the physical model M1. The first generation unit 621 repeatedly performs the first process of generating first data Y1 by inputting input data X to the physical model M1.

[0026] The first generation unit 621 generates first data Y1 such that nonlinear distortion is reduced in the sound reproduced from the speaker device 20 in accordance with the acoustic signal B. Therefore, if the first generation unit 621 operates ideally, it is possible to generate an acoustic signal B with sufficiently reduced nonlinear distortion from the first data Y1 alone. However, in reality, there is an error between the characteristics of the actual speaker device 20 and the characteristics represented by the physical model M1, making it difficult to generate an ideal acoustic signal B with sufficiently reduced nonlinear distortion from the first data Y1 alone. Considering these circumstances, in the first embodiment, second data Y2 is generated to correct the first data Y1. That is, the second data Y2 is data to compensate for the voltage error caused by the difference between the characteristics of the actual speaker device 20 and the characteristics represented by the physical model M1. In other words, the second data Y2 corresponds to the difference between the ideal voltage that should be supplied to the speaker device 20 in order to sufficiently reduce nonlinear distortion from the reproduced sound under the characteristics of the actual speaker device 20, and the voltage represented by the first data Y1.

[0027] The second generation unit 622 generates second data Y2 from the input data X. The second data Y2 comprehensively represents data relating to the state of the speaker device 20. The state of the speaker device 20 represented by the input data X is different from the state of the speaker device 20 represented by the second data Y2. The generation of second data Y2 by the second generation unit 622 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple second data Y2 corresponding to different sections of the acoustic signal A is generated. The first processing by the first generation unit 621 (generation of first data Y1) and the second processing by the second generation unit 622 (generation of second data Y2) are executed in parallel or sequentially.

[0028] The second generation unit 622 generates second data Y2 by performing a second process on the input data X. The second process is signal processing, including processing by the estimation model M2. The estimation model M2 is a machine learning-based statistical model. The estimation model M2 is implemented by a combination of a program that causes the control device 11 to perform an operation to generate second data Y2 from the input data X, and a number of variables (e.g., bias and weight values) applied to the operation. The numerical values ​​of each of the variables are set in advance by machine learning.

[0029] The estimation model M2 of the first embodiment is composed of a deep neural network such as a recurrent neural network (RNN). For example, estimation model M2 includes reservoir computing, which is an example of a recurrent neural network. Reservoir computing is a trained model in which a reservoir layer, in which a large number of interconnected nodes are randomly arranged, is interposed between the input layer and the output layer.

[0030] As described above, in the first embodiment, since the estimation model M2 includes reservoir computing, the processing load required for machine learning of the estimation model M2 can be reduced compared to a configuration in which the estimation model M2 is constructed using a general recurrent neural network. However, the configuration of the estimation model M2 is not limited to the above examples and may be changed as desired. For example, a recurrent neural network using various architectures such as LSTM (Long Short Term Memory) or GRU (Gated Recurrent Unit) may be adopted as the estimation model M2.

[0031] The third generation unit 623 generates output data Z based on the first data Y1 and the second data Y2. As mentioned above, the output data Z represents the voltage that should be supplied to the speaker device 20 in order to reduce nonlinear distortion caused by nonlinearity from the reproduced sound. Specifically, the third generation unit 623 generates output data Z by adding the first data Y1 and the second data Y2 (Z = Y1 + Y2). The generation of output data Z by the third generation unit 623 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple output data Z corresponding to different sections of the acoustic signal A is generated.

[0032] Figure 6 is a flowchart of the process by which the control device 11 (second analysis unit 62) generates output data Z from acoustic signal A (hereinafter referred to as the "output data generation process"). For example, the output data generation process is executed each time input data X is generated by the first analysis unit 61.

[0033] When the output data generation process begins, the control device 11 acquires the input data X generated by the first analysis unit 61 (Sa1). The control device 11 (first generation unit 621) generates the first data Y1 by performing a first process using the physical model M1 on the input data X (Sa2). The control device 11 (second generation unit 622) generates the second data Y2 by performing a second process using the machine learning-prepared estimation model M2 on the input data X (Sa3). The order of the first process (Sa2) and the second process (Sa3) may be reversed. The first process (Sa2) and the second process (Sa3) may also be executed in parallel with each other. The control device 11 (third generation unit 623) generates the output data Z by adding the first data Y1 and the second data Y2 (Sa4).

[0034] Here, we assume a configuration in which output data Z is generated solely by processing input data X by the physical model M1 (first processing) (hereinafter referred to as "proportionality 1"). In other words, proportionality 1 is a form in which the second generation unit 622 is omitted from the first embodiment. There are limitations to faithfully reproducing the behavior of the actual speaker device 20 using only the physical model M1. Therefore, in proportionality 1, it is actually difficult to generate output data Z that can reduce nonlinear distortion with high accuracy.

[0035] As described above, in the first embodiment, output data Z relating to the state of the speaker device 20 is generated based on first data Y1 generated in a first process including processing by a physical model M1, and second data Y2 generated in a second process including processing by a machine learning-prepared estimation model M2. Therefore, according to the first embodiment, output data Z can be generated with higher accuracy compared to proportionality 1.

[0036] Furthermore, we consider a configuration in which output data Z is generated solely by processing input data X by the estimation model M2 (second processing) (hereinafter referred to as "proportionality 2"). In other words, proportionality 2 is a form in which the first generation unit 621 is omitted from the first embodiment. In proportionality 2, it is difficult to stably generate highly accurate output data Z. As described above, in the first embodiment, output data Z relating to the state of the speaker device 20 is generated based on first data Y1 generated in the first processing which includes processing by the physical model M1, and second data Y2 generated in the second processing which includes processing by the machine learning-trained estimation model M2. Therefore, according to the first embodiment, output data Z can be generated stably and with high accuracy compared to proportionality 2.

[0037] Furthermore, in the first embodiment, output data Z is generated by adding first data Y1 and second data Y2. That is, second data Y2 corresponds to the difference between output data Z and first data Y1 (i.e., the error of first data Y1 with respect to the ideal output data Z). Therefore, second data Y2 can be generated by the second process so as to compensate for the error of first data Y1 with respect to output data Z.

[0038] [Machine Learning of Estimation Model M2] Figure 7 is an explanatory diagram regarding the machine learning of the estimation model M2. Multiple training data T are used for the machine learning of the estimation model M2. The multiple training data T are stored in the memory device 12. Each of the multiple training data T consists of a pair of training input data Xt and training output data Zt. The output data Zt of each training data T is the output data Z (i.e., the correct value) that should be generated for the input data Xt of that training data T.

[0039] The control device 11 functions as a learning processing unit 70. The learning processing unit 70 establishes an estimated model M2 using machine learning with multiple training data T. Figure 8 is a flowchart of the process (hereinafter referred to as "learning process") in which the control device 11 (learning processing unit 70) establishes an estimated model M2 using machine learning with multiple training data T.

[0040] When the learning process begins, the control device 11 (learning processing unit 70) selects one of the multiple training data T stored in the memory device 12 (hereinafter referred to as "selected training data T") (Sb1). As illustrated in Figure 7, the control device 11 generates output data Z by processing the input data Xt of the selected training data T with the second analysis unit 62, which includes a provisional estimation model M2 (Sb2). Specifically, the control device 11 performs the following processes: generating first data Y1 from input data Xt using a default physical model M1 (Sa2), generating second data Y2 from input data Xt using a provisional estimation model M2 (Sa3), and generating output data Z from the first data Y1 and the second data Y2 (Sa4).

[0041] The control device 11 calculates a loss function that represents the error between the output data Z generated by the above procedure and the output data Zt of the selected training data T (Sb3). The control device 11 updates several variables of the provisional estimated model M2 so that the loss function is reduced (ideally minimized) (Sb4).

[0042] The control device 11 determines whether a predetermined termination condition has been met (Sb5). The termination condition is, for example, that the loss function falls below a predetermined threshold, or that the amount of change in the loss function falls below a predetermined threshold. If the termination condition is not met (Sb5: NO), the control device 11 selects the unselected training data T stored in the memory device 12 as the new selected training data T (Sb1). That is, the process of updating multiple variables of the estimated model M2 (Sb1 to Sb4) is repeated until the termination condition is met (Sb5: YES).

[0043] If the termination condition is met (Sb5:YES), the control device 11 terminates the learning process. The estimated model M2 at the time the termination condition is met is determined to be the machine-learned estimated model M2.

[0044] As can be understood from the above explanation, the estimation model M2 learns the relationship between the input data Xt in multiple training data T and the second data Y2 that the second analysis unit 62 uses to generate appropriate output data Z for the input data Xt. Therefore, the second generation unit 622, which uses the machine-learned estimation model M2, outputs statistically valid second data Y2 for unknown input data X, based on the latent relationship between the input data Xt in multiple training data T and the appropriate second data Y2 for the input data Xt. In other words, in the first embodiment, machine learning of the estimation model M2 is performed so that the error between the provisional first data Y1 generated in the first processing of the input data Xt for learning and the output data Zt for learning is output as second data Y2 from the estimation model M2.

[0045] B: Second Embodiment A second embodiment of this disclosure will now be described. For elements whose function is the same as in the first embodiment in each of the embodiments described below, the same reference numerals as in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0046] Figure 9 is a block diagram illustrating the specific configuration of the second analysis unit 62 in the second embodiment. The second analysis unit 62 in the second embodiment is composed of the same elements as in the first embodiment (first generation unit 621, second generation unit 622, third generation unit 623).

[0047] The first data Y1 generated by the first generation unit 621 is supplied to the third generation unit 623, as in the first embodiment, and also to the second generation unit 622. The second generation unit 622 generates the second data Y2 by performing a second process on data D, which includes the input data X and the first data Y1. That is, the second generation unit 622 generates the second data Y2 by processing data D, which includes the input data X and the first data Y1, using the estimation model M2.

[0048] For example, input data X corresponding to a specific section of acoustic signal A and first data Y1 generated from the input data X for that section are input to the estimation model M2. However, the temporal relationship between the input data X and the first data Y1 input to the estimation model M2 is arbitrary. For example, input data X corresponding to a specific section of acoustic signal A and first data Y1 generated by the first generation unit 621 for the section immediately preceding that section may be input to the estimation model M2.

[0049] The training data T used for machine learning of the estimation model M2 is data in which the correct answer output data Z is associated with a pair of training input data Xt and first data Y1 generated by the first generation unit 621 from the input data Xt (i.e., training data D). The procedure for the learning process is the same as in the first embodiment.

[0050] The same effects as in the first embodiment are achieved in the second embodiment. In addition, in the second embodiment, the second process using the estimation model M2 is performed on the pair of input data X and the first data Y1 generated by the first process. Therefore, compared to the form in which the second process is performed only on the input data X (i.e., the state of the first process is not reflected in the second process), output data can be generated stably and with high accuracy.

[0051] C: Third Embodiment Figure 10 is a block diagram illustrating the specific configuration of the second analysis unit 62 in the third embodiment. The second analysis unit 62 in the third embodiment is composed of the same elements as in the first embodiment (first generation unit 621, second generation unit 622, third generation unit 623).

[0052] In the third embodiment, the second generation unit 622 is supplied with input data X, similar to the first embodiment, as well as a plurality of characteristic parameters P. As mentioned above, the plurality of characteristic parameters P are variables (e.g., TS parameters) that define the physical model M1 used in the first processing, and are stored in the storage device 12 in advance. Note that not all of the characteristic parameters P that define the physical model M1 need to be included in the data D input to the estimation model M2.

[0053] As described above, the second generation unit 622 generates second data Y2 by performing a second process on data D, which includes input data X and multiple characteristic parameters P of the physical model M1. In other words, the second generation unit 622 generates second data Y2 by processing data D, which includes input data X and multiple characteristic parameters P, using the estimation model M2.

[0054] The training data T used for machine learning of the estimation model M2 is data in which the correct output data Z is associated with the training input data Xt and the set of multiple characteristic parameters P of the physical model M1 (i.e., training data D). The procedure for the learning process is the same as in the first embodiment.

[0055] The same effects as in the first embodiment are achieved in the third embodiment. In addition, in the third embodiment, a second processing using the estimation model M2 is performed on data D which includes input data and multiple characteristic parameters P of the physical model. That is, the characteristics of the physical model M1 used to generate the first data Y1 are also reflected in the second processing which includes processing by the estimation model M2. Therefore, compared to a form in which the data D processed by the estimation model M2 does not include the characteristic parameters P of the physical model M1 (i.e., a form in which the characteristics of the physical model M1 are not reflected in the second processing), output data Z can be generated stably and with high accuracy.

[0056] The configuration of the second embodiment is also applicable to the third embodiment. That is, the second processing may be performed on data D which includes input data X, first data Y1, and a plurality of characteristic parameters P.

[0057] D: Fourth Embodiment In the first embodiment, an example was given in which the input data X represents the state of the diaphragm 54 in the speaker device 20, and the output data Z represents the voltage to be supplied to the speaker device 20. In the fourth embodiment, the contents of the input data X and output data Z differ from those of the first embodiment.

[0058] In the fourth embodiment, the input data X represents the voltage (input voltage) to be supplied to the speaker device 20. That is, the first analysis unit 61 generates input data X that represents the voltage corresponding to the signal value of the acoustic signal A. Since the signal value of the acoustic signal A corresponds to sound pressure, the voltage represented by the input data X can also be expressed as sound pressure.

[0059] Furthermore, the output data Z of the fourth embodiment represents the state of the diaphragm 54 in the speaker device 20. Specifically, the output data Z includes the displacement, velocity, and acceleration of the diaphragm 54. In other words, the second analysis unit 62 of the fourth embodiment estimates the state of the diaphragm 54 (output data Z) when the voltage represented by the input data X is supplied to the speaker device 20.

[0060] Furthermore, the configuration of the second embodiment, in which the first data Y1 generated by the first generation unit 621 is input to the second generation unit 622 together with the input data X, and the configuration of the third embodiment, in which multiple characteristic parameters P of the physical model M1 are input to the second generation unit 622, may also be applied similarly to the fourth embodiment.

[0061] E: Variation The following are examples of specific modifications that may be added to each of the embodiments exemplified above. Two or more embodiments may be arbitrarily selected from the following examples and merged as appropriate, provided they do not contradict each other.

[0062] (1) In the first embodiment, the output data Z is shown as representing the voltage to be supplied to the speaker device 20, but the output data Z may also be data representing the current to be supplied to the speaker device 20. Also, in the fourth embodiment, the input data X is shown as representing the voltage to be supplied to the speaker device 20, but the input data X may also be data representing the current to be supplied to the speaker device 20.

[0063] (2) In each of the above embodiments, the generation of first data Y1 using the physical model M1 was exemplified as the first process executed by the first generation unit 621, but the first process may include other processes. For example, the first process may include a process for processing the input data X input to the physical model M1, or a process for processing the first data Y1 generated by the physical model M1.

[0064] (3) In each of the above embodiments, the generation of second data Y2 using the estimation model M2 was exemplified as a second process performed by the second generation unit 622, but the second process may include other processes. For example, the second process may include a process to process the input data X that is input to the estimation model M2, or a process to process the second data Y2 generated by the estimation model M2.

[0065] (4) In the above-described embodiments, the third generation unit 623 is shown as adding the first data Y1 and the second data Y2, but the content of the process for generating output data Z from the first data Y1 and the second data Y2 is not limited to the above examples. For example, the third generation unit 623 may generate output data Z by a weighted sum of the first data Y1 and the second data Y2. The addition of the first data Y1 and the second data Y2 also includes a weighted sum of the first data Y1 and the second data Y2. Alternatively, the third generation unit 623 may generate output data Z by a predetermined operation applied to the first data Y1 and the second data Y2. For example, the third generation unit 623 may generate output data Z by processing the first data Y1 and the second data Y2 using various mathematical models such as regression models (linear regression models, nonlinear regression models) or deep neural networks.

[0066] (5) In each of the above-described embodiments, an embodiment in which the information processing system 10 is equipped with a learning processing unit 70 has been conveniently illustrated, but the learning processing unit 70 may be installed in a separate system (machine learning system) from the information processing system 10. The machine learning system is implemented by a server device such as a web server, and constructs the estimation model M2 through the learning process described above. The estimation model M2 constructed by the machine learning system is transferred to the information processing system 10.

[0067] (6) In each of the above-described embodiments, a speaker system 100 in which the information processing system 10 and the speaker device 20 are configured separately was given as an example. However, the above-described embodiments also apply to speaker systems (active speakers) in which the information processing system 10 is installed inside the speaker device 20. Furthermore, the speaker system 100 can be realized as a stationary system, or as a portable information device such as a smartphone, tablet terminal, or personal computer.

[0068] (7) The functions of the information processing system 10 exemplified above are realized through the cooperation of one or more processors constituting the control device 11 and a program stored in the storage device 12, as described above. The program according to this disclosure can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. A non-transitory recording medium includes any recording medium except for transient propagation signals, and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the storage medium in which the distribution device stores the program corresponds to the non-transitory recording medium described above.

[0069] (8) The notation "the nth" (where n is a natural number) in this application is used solely as a formal and convenient label to distinguish each element in notation and has no substantive meaning whatsoever. Therefore, there is no room for restrictive interpretation of the position or order of each element based on the notation "the nth".

[0070] F: Note From the forms exemplified above, the following configuration can be understood, for example.

[0071] An information processing method according to one aspect of the present disclosure (Aspect 1) generates first data relating to the state of a speaker device by performing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; generates second data relating to the state of the speaker device by performing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and generates output data relating to the state of the speaker device based on the first data and the second data.

[0072] In the above embodiment, output data regarding the state of the speaker device is generated based on first data generated in a first process including processing by a physical model and second data generated in a second process including processing by a machine learning-prepared estimation model. Therefore, compared to a configuration in which output data is generated by processing input data with a physical model, output data can be generated with higher accuracy. Furthermore, compared to a configuration in which output data is generated by processing input data with a machine learning-prepared estimation model, output data can be generated stably and with higher accuracy.

[0073] "The state of the speaker device" includes the behavior or characteristics of the speaker device. The behavior of the speaker device is any action that contributes to sound emission by the speaker device, such as the generation of magnetic force due to the supply of current to the voice coil, or the displacement of the diaphragm. The characteristics of the speaker device are any physical characteristics that contribute to sound emission by the speaker device, such as the elastic modulus of the damper supporting the diaphragm or the force coefficient (electromagnetic conversion coefficient) of the voice coil.

[0074] The data relating to the state of the speaker device (input data, first data, second data, output data) are arbitrary variables involved in the sound emission by the speaker device. For example, any data that affects the sound emission operation, such as physical quantities representing the behavior of the diaphragm (e.g., displacement, velocity, or acceleration), or voltage or current input to or output to the speaker device, is included in "data relating to the state of the speaker device."

[0075] "First process" is any process that includes processing by a physical model. That is, "first process" may consist only of processing by a physical model, or it may consist of processing by a physical model and other processes that are executed before and after said processing. Similarly, "second process" is any process that includes processing by a machine learning-prepared estimation model. That is, "second process" may consist only of processing by an estimation model, or it may consist of processing by an estimation model and other processes that are executed before and after said processing.

[0076] A "physical model" is any form of mathematical model that simulates the state of a speaker device. For example, a lumped-parameter system model is an example of a "physical model." An "estimation model," on the other hand, is a statistical model whose relationship between training control data and second training data has been acquired through prior machine learning. For example, a trained model such as a deep neural network is an example of an "estimation model."

[0077] In a specific example of Embodiment 1 (Embodiment 2), the generation of the second data is performed by executing the second process on data including the input data and the first data. In the above embodiment, the second process using the estimation model is performed on data including the input data and the generated first data. Therefore, compared to a form in which the second process is performed only on the input data (i.e., a form in which the status of the first process is not reflected in the second process), output data can be generated stably and with high accuracy.

[0078] In an embodiment 1 or a specific example of embodiment 2 (embodiment 3), the physical model includes a plurality of characteristic parameters relating to the speaker device, and in the generation of the second data, the second data is generated by performing the second processing on data including the input data and the plurality of characteristic parameters. In the above embodiments, the second processing using the estimation model is performed on data including the input data and the plurality of characteristic parameters of the physical model. That is, the characteristics of the physical model used to generate the first data are also reflected in the second processing, which includes processing by the estimation model. Therefore, compared to a form in which the data processed by the estimation model does not include the characteristic parameters of the physical model (i.e., a form in which the characteristics of the physical model are not reflected in the second processing), output data can be generated stably and with high accuracy.

[0079] In any specific example of Embodiments 1 to 3 (Embodiment 4), the input data is data relating to the state of the diaphragm in the speaker device, and the output data is data representing the voltage or current to be supplied to the speaker device. According to the above embodiments, the voltage or current (acoustic signal) to be input to the speaker device can be determined from the state of the diaphragm in the speaker device.

[0080] In any specific example of Embodiments 1 to 3 (Embodiment 5), the input data is data representing the voltage or current to be supplied to the speaker device, and the output data is data relating to the state of the diaphragm in the speaker device. According to the above embodiments, the displacement of the diaphragm in the speaker device can be estimated from the voltage or current (acoustic signal) input to the speaker device.

[0081] In any specific example (6) of embodiments 1 to 5, the output data is generated by adding the first data and the second data. In the embodiments described above, the output data is generated by adding the first data and the second data. That is, the second data corresponds to the difference between the output data and the first data (i.e., the error of the first data relative to the ideal output data). Therefore, the second data can be generated by the second processing so as to compensate for the error of the first data relative to the output data. In other words, machine learning of the estimation model is performed so that the error between the provisional first data generated in the first processing of the input data for training and the output data for training is output as the second data from the estimation model.

[0082] In any specific example of Embodiments 1 to 6 (Embodiment 7), the estimation model includes reservoir computing. According to the above embodiments, since the estimation model includes reservoir computing, the processing load required for machine learning of the estimation model can be reduced compared to the form in which the estimation model is constructed using a general recurrent neural network.

[0083] An information processing system according to one aspect of the present disclosure (Aspect 8) comprises: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0084] A program according to one aspect of the present disclosure (Aspect 9) causes a computer system to function as follows: a first generation unit that generates first data relating to the state of a speaker device by executing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by executing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data. [Explanation of Symbols]

[0085] 100...Speaker system, 10...Information processing system, 11...Control device, 12...Storage device, 13...Input device, 14...Output device, 20...Speaker, 30...Signal supply device, 61...First analysis unit, 62...Second analysis unit, 621...First generation unit, 622...Second generation unit, 623...Third generation unit, 70...Learning processing unit.

Claims

1. A first process, which includes processing using a physical model that simulates the state of the speaker device, is performed on input data relating to the state of the speaker device, thereby generating first data relating to the state of the speaker device. A second process, including processing by a machine learning-based estimation model, is performed on the input data to generate second data regarding the state of the speaker device. Based on the first data and the second data, output data relating to the state of the speaker device is generated. Information processing methods implemented by computer systems.

2. In generating the second data, the second data is generated by performing the second process on data including the input data and the first data. The information processing method of claim 1.

3. The physical model includes a plurality of characteristic parameters relating to the speaker device, In generating the second data, the second processing is performed on the data including the input data and the plurality of characteristic parameters to generate the second data. The information processing method of claim 1.

4. The input data is data relating to the state of the diaphragm in the speaker device. The output data is data representing the voltage or current to be supplied to the speaker device. The information processing method of claim 1.

5. The aforementioned input data represents the voltage or current to be supplied to the speaker device. The output data is data relating to the state of the diaphragm in the speaker device. The information processing method of claim 1.

6. In generating the aforementioned output data, The output data is generated by adding the first data and the second data. The information processing method of claim 1.

7. The aforementioned estimation model includes reservoir computing. The information processing method of claim 1.

8. A first generation unit generates first data relating to the state of the speaker device by performing a first process, which includes processing using a physical model that simulates the state of the speaker device, on input data relating to the state of the speaker device. A second generation unit generates second data relating to the state of the speaker device by performing a second process on the input data, which includes processing by a machine learning-based estimation model. A third generation unit generates output data relating to the state of the speaker device based on the first data and the second data. An information processing system equipped with the following features.

9. A first generation unit generates first data relating to the state of a speaker device by performing a first process, which includes processing using a physical model that simulates the state of the speaker device, on input data relating to the state of the speaker device. A second generation unit generates second data relating to the state of the speaker device by performing a second process on the input data, which includes processing by a machine learning-prepared estimation model, and A third generation unit generates output data relating to the state of the speaker device based on the first data and the second data. A program that makes a computer system function.

Images