Information processing method, information processing system, and program

Abstract

This information processing system comprises: a first generation unit 621 that generates first data Y1 pertaining to the state of a speaker device by executing, on input data X pertaining to the state of the speaker device, a first process including processing by a physical model M1 that simulates the state of the speaker device; a second generation unit 622 that generates second data Y2 pertaining to the state of the speaker device by executing, on the input data X, a second process including processing by an estimation model M2 that has undergone machine learning; and a third generation unit 623 that generates output data Z pertaining to the state of the speaker device on the basis of the first data Y1 and the second data Y2.

Description

Information Processing Method, Information Processing System, and Program

[0001] The present disclosure relates to a technique for controlling reproduced sound by a speaker device.

[0002] Various techniques for reducing non-linear distortion of reproduced sound by a speaker device have been proposed conventionally. For example, Patent Document 1 discloses a configuration for reducing non-linear distortion using a lumped model that simulates the state of a speaker device. Further, Patent Document 2 discloses a configuration for calculating an input voltage for reducing non-linear distortion by inputting the displacement of a diaphragm of a speaker device into a deep neural network.

[0003] U.S. Patent No. 10,547,942, U.S. Patent No. 11,356,773

[0004] However, it is limited to faithfully reproduce the behavior of an actual speaker device by the lumped model of Patent Document 1. Therefore, it is actually difficult to generate output data capable of reducing non-linear distortion with high accuracy. Also, even in a configuration using a deep neural network as in Patent Document 2, it is difficult to stably generate high-precision output data. In the above description, attention was paid to reducing non-linear distortion, but the same problem occurs in any scenario for estimating the state of a speaker device. In consideration of the above circumstances, one aspect of the present disclosure aims to stably and accurately generate output data regarding the state of a speaker device.

[0005] To solve the above problems, an information processing method according to one aspect of the present disclosure executes a first process including processing by a physical model that simulates the state of a speaker device on input data regarding the state of the speaker device to generate first data regarding the state of the speaker device, executes a second process including processing by a trained estimation model on the input data to generate second data regarding the state of the speaker device, and generates output data regarding the state of the speaker device based on the first data and the second data.

[0006] An information processing system according to one aspect of the present disclosure comprises: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0007] A program according to one aspect of the present disclosure causes a computer system to function as follows: a first generation unit that generates first data relating to the state of a speaker device by executing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by executing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0008] This is a block diagram illustrating the configuration of the speaker system in the first embodiment. This is a cross-sectional view of the speaker unit. This is a block diagram illustrating the configuration of the information processing system. This is a block diagram illustrating the functional configuration of the information processing system. This is a block diagram illustrating the specific configuration of the second analysis unit. This is a flowchart of the output data generation process. This is an explanatory diagram regarding machine learning of the estimation model. This is a flowchart of the learning processing unit. This is a block diagram illustrating the specific configuration of the second analysis unit in the second embodiment. This is a block diagram illustrating the specific configuration of the second analysis unit in the third embodiment.

[0009] A: Figure 1 of the first embodiment is a schematic diagram illustrating the configuration of the speaker system 100 in the first embodiment. The speaker system 100 of the first embodiment comprises an information processing system 10 and a speaker device 20. The information processing system 10 is supplied with an acoustic signal A from a signal supply device 30. The acoustic signal A is a signal that represents the waveform of sound. For example, the acoustic signal A is a sample sequence that represents the waveform of the vocal or instrumental sound of a musical piece.

[0010] The signal supply device 30 is, for example, a playback device that supplies an acoustic signal A recorded on a recording medium to the information processing system 10. A communication device that supplies an acoustic signal A received from a distribution device (not shown) via a communication network to the information processing system 10, or a sound collection device that generates an acoustic signal A by collecting ambient sounds, can also be used as the signal supply device 30. The signal supply device 30 may also be understood as an element of the speaker system 100.

[0011] The information processing system 10 generates an acoustic signal B by signal processing on an acoustic signal A and supplies the acoustic signal B to the speaker device 20. The speaker device 20 is a sound emitting device that radiates sound waves of the waveform represented by the acoustic signal B. The speaker device 20 of the first embodiment comprises the speaker unit 50 shown in Figure 2. The speaker unit 50 comprises a frame 51, a magnet 52, a voice coil 53, a diaphragm 54, an edge 55, and a damper 56.

[0012] The frame 51 is a structure that constitutes the exterior of the speaker unit 50. The magnet 52 is an annular permanent magnet. The voice coil 53 is a coil that can be displaced axially within the magnetic field generated by the magnet 52. The diaphragm 54 is a frustoconical structure. The inner edge of the diaphragm 54 is fixed to the voice coil 53. The outer edge of the diaphragm 54 is connected to the frame 51 via an edge 55. The diaphragm 54 and the frame 51 are also connected via a damper 56. The edge 55 and the damper 56 are annular elastic bodies. Sound waves are radiated as the diaphragm 54 reciprocates axially.

[0013] The high-frequency components of the sound reproduced by the speaker device 20 are accompanied by nonlinear distortion due to the nonlinearity of each element constituting the speaker device 20. The information processing system 10 generates an acoustic signal B from acoustic signal A in such a way that the nonlinear distortion caused by the nonlinearity of the speaker device 20 is reduced.

[0014] Figure 3 is a block diagram illustrating the configuration of the information processing system 10. The information processing system 10 comprises a control device 11, a storage device 12, an input device 13, and an output device 14. The information processing system 10 can be implemented as an information device such as a smartphone, tablet terminal, or personal computer. The information processing system 10 can be implemented as a single device or as a group of devices configured separately from each other.

[0015] The control device 11 is composed of one or more processors that control each element of the information processing system 10. For example, the control device 11 is composed of one or more types of processors such as a CPU (Central Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0016] The storage device 12 is one or more memories that store programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is composed of known recording media such as magnetic recording media or semiconductor recording media. The storage device 12 may be composed of a combination of multiple types of recording media. Alternatively, a portable recording media that can be attached to and detached from the information processing system 10, or a recording media that can be written to or read by the control device 11 via a communication network (e.g., cloud storage), may be used as the storage device 12.

[0017] The input device 13 receives the acoustic signal A from the signal supply device 30. For example, an input interface such as a USB (Universal Serial Bus) terminal, an HDMI (High-Definition Multimedia Interface / registered trademark) terminal, a MIDI (Musical Instrument Digital Interface) terminal, or a phone terminal can be used as the input device 13. In the case where an analog acoustic signal A is supplied from the signal supply device 30, an A / D converter that converts the acoustic signal A from analog to digital is installed in the input device 13.

[0018] The output device 14 supplies the acoustic signal B to the speaker device 20. Specifically, the output device 14 comprises a D / A converter that converts the acoustic signal B from digital to analog, and an amplifier that amplifies the acoustic signal B. Alternatively, the output device 14, separate from the information processing system 10, may be connected to the information processing system 10 by wire or wireless connection.

[0019] Figure 4 is a block diagram illustrating the functional configuration of the information processing system 10. The control device 11 executes a program stored in the storage device 12 to realize multiple functions (first analysis unit 61, second analysis unit 62) for processing the acoustic signal A.

[0020] The first analysis unit 61 generates input data X by processing the acoustic signal A. The input data X is data relating to the state of the speaker device 20 when the acoustic signal A is supplied to the speaker device 20. Specifically, the input data X represents the state of the diaphragm 54 in the speaker device 20. For example, the first analysis unit 61 generates input data X that includes the displacement of the diaphragm 54, the velocity and acceleration of the diaphragm 54, when the acoustic signal A is supplied to the speaker device 20.

[0021] The generation of input data X by the first analysis unit 61 is repeated sequentially at a predetermined period. That is, a time series of multiple input data X corresponding to different sections of the acoustic signal A is generated. The first analysis unit 61 is composed of a linear model that simulates the behavior of each element of the speaker device 20. Any known mathematical model can be arbitrarily adopted as the linear model that constitutes the first analysis unit 61.

[0022] The second analysis unit 62 generates output data Z by processing the input data X. The output data Z is data representing the voltage (input voltage) to be supplied to the speaker device 20. The generation of output data Z by the second analysis unit 62 is repeated sequentially at a predetermined period. That is, a time series of multiple output data Z corresponding to different sections of the acoustic signal A is generated. The output device 14 generates an acoustic signal B whose signal value is the voltage represented by the output data Z. As described above, the second analysis unit 62 generates output data Z in such a way that nonlinear distortion caused by the nonlinearity of the speaker device 20 is reduced from the reproduced sound. As described above, according to the first embodiment, the voltage (acoustic signal B) input to the speaker device 20 can be determined from the state of the diaphragm 54 in the speaker device 20.

[0023] Figure 5 is a block diagram illustrating the specific configuration of the second analysis unit 62. The second analysis unit 62 comprises a first generation unit 621, a second generation unit 622, and a third generation unit 623. The input data X generated by the first analysis unit 61 is supplied in parallel to the first generation unit 621 and the second generation unit 622.

[0024] The first generation unit 621 generates first data Y1 from input data X. First data Y1 is data relating to the state of the speaker device 20. The state of the speaker device 20 represented by input data X is different from the state of the speaker device 20 represented by first data Y1. Specifically, first data Y1 is data representing the voltage that should be supplied to the speaker device 20. The generation of first data Y1 by the first generation unit 621 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple first data Y1 corresponding to different sections of the acoustic signal A is generated.

[0025] The first generation unit 621 generates first data Y1 by performing a first process on input data X. The first process is signal processing including processing by a physical model M1. The physical model M1 is a mathematical model that simulates the state of the speaker device 20. Specifically, the physical model M1 includes a plurality of characteristic parameters P related to the speaker device 20. Each characteristic parameter P is a TS (Thiele Small) parameter that represents the electrical or mechanical characteristics of each element of the speaker device 20 (e.g., magnet 52, voice coil 53, diaphragm 54, edge 55, or damper 56). For example, a known lumped parameter model that approximates the behavior of each element of the speaker device 20 using lumped parameters is used as the physical model M1. The first generation unit 621 repeatedly performs the first process of generating first data Y1 by inputting input data X to the physical model M1.

[0026] The first generation unit 621 generates first data Y1 such that nonlinear distortion is reduced in the sound reproduced from the speaker device 20 in accordance with the acoustic signal B. Therefore, if the first generation unit 621 operates ideally, it is possible to generate an acoustic signal B with sufficiently reduced nonlinear distortion from the first data Y1 alone. However, in reality, there is an error between the characteristics of the actual speaker device 20 and the characteristics represented by the physical model M1, making it difficult to generate an ideal acoustic signal B with sufficiently reduced nonlinear distortion from the first data Y1 alone. Considering these circumstances, in the first embodiment, second data Y2 is generated to correct the first data Y1. That is, the second data Y2 is data to compensate for the voltage error caused by the difference between the characteristics of the actual speaker device 20 and the characteristics represented by the physical model M1. In other words, the second data Y2 corresponds to the difference between the ideal voltage that should be supplied to the speaker device 20 in order to sufficiently reduce nonlinear distortion from the reproduced sound under the characteristics of the actual speaker device 20, and the voltage represented by the first data Y1.

[0027] The second generation unit 622 generates second data Y2 from the input data X. The second data Y2 comprehensively represents data relating to the state of the speaker device 20. The state of the speaker device 20 represented by the input data X is different from the state of the speaker device 20 represented by the second data Y2. The generation of second data Y2 by the second generation unit 622 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple second data Y2 corresponding to different sections of the acoustic signal A is generated. The first processing by the first generation unit 621 (generation of first data Y1) and the second processing by the second generation unit 622 (generation of second data Y2) are executed in parallel or sequentially.

[0028] The second generation unit 622 generates second data Y2 by performing a second process on the input data X. The second process is signal processing including processing by the estimation model M2. The estimation model M2 is a machine learning-based statistical model. The estimation model M2 is realized by a combination of a program that causes the control device 11 to perform an operation to generate second data Y2 from the input data X, and a plurality of variables (e.g., bias and weight values) applied to the operation. The numerical values ​​of each of the plurality of variables are set in advance by machine learning.

[0029] The estimation model M2 of the first embodiment is composed of a deep neural network such as a recurrent neural network (RNN). For example, the estimation model M2 includes reservoir computing, which is an example of a recurrent neural network. Reservoir computing is a trained model in which a reservoir layer, in which a large number of interconnected nodes are randomly arranged, is interposed between the input layer and the output layer.

[0030] As described above, in the first embodiment, since the estimation model M2 includes reservoir computing, the processing load required for machine learning of the estimation model M2 can be reduced compared to a configuration in which the estimation model M2 is constructed using a general recurrent neural network. However, the configuration of the estimation model M2 is not limited to the above examples and may be changed as desired. For example, a recurrent neural network utilizing various architectures such as LSTM (Long Short Term Memory) or GRU (Gated Recurrent Unit) may be adopted as the estimation model M2.

[0031] The third generation unit 623 generates output data Z based on the first data Y1 and the second data Y2. As described above, the output data Z represents the voltage that should be supplied to the speaker device 20 in order to reduce nonlinear distortion caused by nonlinearity from the reproduced sound. Specifically, the third generation unit 623 generates output data Z by adding the first data Y1 and the second data Y2 (Z = Y1 + Y2). The generation of output data Z by the third generation unit 623 is repeated at a predetermined period each time the input data X is generated by the first analysis unit 61. That is, a time series of multiple output data Z corresponding to different sections of the acoustic signal A is generated.

[0032] Figure 6 is a flowchart of the process by which the control device 11 (second analysis unit 62) generates output data Z from the acoustic signal A (hereinafter referred to as the "output data generation process"). For example, the output data generation process is executed each time the first analysis unit 61 generates input data X.

[0033] When the output data generation process begins, the control device 11 acquires the input data X generated by the first analysis unit 61 (Sa1). The control device 11 (first generation unit 621) generates the first data Y1 by executing a first process using the physical model M1 on the input data X (Sa2). The control device 11 (second generation unit 622) generates the second data Y2 by executing a second process using the machine learning-prepared estimation model M2 on the input data X (Sa3). The order of the first process (Sa2) and the second process (Sa3) may be reversed. The first process (Sa2) and the second process (Sa3) may also be executed in parallel with each other. The control device 11 (third generation unit 623) generates the output data Z by adding the first data Y1 and the second data Y2 (Sa4).

[0034] Here, we assume a configuration in which output data Z is generated solely by processing input data X by the physical model M1 (first processing) (hereinafter referred to as "proportionality 1"). In other words, proportionality 1 is a configuration in which the second generation unit 622 is omitted from the first embodiment. There are limitations to faithfully reproducing the behavior of the actual speaker device 20 using only the physical model M1. Therefore, in proportionality 1, it is actually difficult to generate output data Z that can reduce nonlinear distortion with high accuracy.

[0035] As described above, in the first embodiment, output data Z relating to the state of the speaker device 20 is generated based on first data Y1 generated in a first process including processing by a physical model M1, and second data Y2 generated in a second process including processing by a machine learning-trained estimation model M2. Therefore, according to the first embodiment, output data Z can be generated with higher accuracy compared to proportionality 1.

[0036] Furthermore, we consider a configuration in which output data Z is generated solely by processing input data X by the estimation model M2 (second processing) (hereinafter referred to as "proportionality 2"). In other words, proportionality 2 is a configuration in which the first generation unit 621 is omitted from the first embodiment. In proportionality 2, it is difficult to stably generate highly accurate output data Z. As described above, in the first embodiment, output data Z relating to the state of the speaker device 20 is generated based on first data Y1 generated in the first processing which includes processing by the physical model M1, and second data Y2 generated in the second processing which includes processing by the machine learning-trained estimation model M2. Therefore, according to the first embodiment, output data Z can be generated stably and with high accuracy compared to proportionality 2.

[0037] Furthermore, in the first embodiment, output data Z is generated by adding first data Y1 and second data Y2. That is, second data Y2 corresponds to the difference between output data Z and first data Y1 (i.e., the error of first data Y1 with respect to the ideal output data Z). Therefore, second data Y2 can be generated by the second process so as to compensate for the error of first data Y1 with respect to output data Z.

[0038] [Machine Learning of Estimated Model M2] Figure 7 is an explanatory diagram regarding the machine learning of the estimated model M2. Multiple training data T are used for the machine learning of the estimated model M2. The multiple training data T are stored in the memory device 12. Each of the multiple training data T consists of a pair of training input data Xt and training output data Zt. The output data Zt of each training data T is the output data Z (i.e., the correct value) that should be generated for the input data Xt of that training data T.

[0039] The control device 11 functions as a learning processing unit 70. The learning processing unit 70 establishes an estimated model M2 using machine learning with multiple training data T. Figure 8 is a flowchart of the process (hereinafter referred to as "learning process") in which the control device 11 (learning processing unit 70) establishes an estimated model M2 using machine learning with multiple training data T.

[0040] When the learning process is started, the control device 11 (learning processing unit 70) selects one of the multiple training data T stored in the storage device 12 (hereinafter referred to as "selected training data T") (Sb1). As illustrated in Figure 7, the control device 11 generates output data Z by processing the input data Xt of the selected training data T with the second analysis unit 62, which includes a provisional estimation model M2 (Sb2). Specifically, the control device 11 performs the following processes: generating first data Y1 from input data Xt using a predetermined physical model M1 (Sa2), generating second data Y2 from input data Xt using a provisional estimation model M2 (Sa3), and generating output data Z from the first data Y1 and the second data Y2 (Sa4).

[0041] The control device 11 calculates a loss function that represents the error between the output data Z generated by the above procedure and the output data Zt of the selected training data T (Sb3). The control device 11 updates several variables of the provisional estimation model M2 so that the loss function is reduced (ideally minimized) (Sb4).

[0042] The control device 11 determines whether a predetermined termination condition has been met (Sb5). The termination condition is, for example, that the loss function falls below a predetermined threshold, or that the amount of change in the loss function falls below a predetermined threshold. If the termination condition is not met (Sb5: NO), the control device 11 selects the unselected training data T stored in the storage device 12 as new selected training data T (Sb1). That is, the process of updating multiple variables of the estimated model M2 (Sb1 to Sb4) is repeated until the termination condition is met (Sb5: YES).

[0043] If the termination condition is met (Sb5: YES), the control device 11 terminates the learning process. The estimated model M2 at the time the termination condition is met is determined to be the machine-learned estimated model M2.

[0044] As understood from the above description, the estimation model M2 learns the relationship between the input data Xt in a plurality of training data T and the second data Y2 for the second analysis unit 62 to generate appropriate output data Z for the input data Xt. Therefore, the second generation unit 622 using the machine-learned estimation model M2 outputs statistically valid second data Y2 for unknown input data X based on the relationship latent between the input data Xt in the plurality of training data T and the appropriate second data Y2 for the input data Xt. That is, in the first embodiment, machine learning of the estimation model M2 is executed such that the error between the provisional first data Y1 generated by the first process for the learning input data Xt and the learning output data Zt is output as the second data Y2 from the estimation model M2.

[0045] B: Second Embodiment The second embodiment of the present disclosure will be described. In each aspect exemplified below, for elements whose functions are the same as those in the first embodiment, the same reference numerals as in the description of the first embodiment are used, and detailed descriptions thereof are omitted as appropriate.

[0046] FIG. 9 is a block diagram illustrating a specific configuration of the second analysis unit 62 in the second embodiment. The second analysis unit 62 in the second embodiment is composed of the same elements as in the first embodiment (the first generation unit 621, the second generation unit 622, and the third generation unit 623).

[0047] The first data Y1 generated by the first generation unit 621 is supplied not only to the third generation unit 623 as in the first embodiment but also to the second generation unit 622. The second generation unit 622 generates the second data Y2 by executing a second process on the data D including the input data X and the first data Y1. That is, the second generation unit 622 generates the second data Y2 by processing the data D including the input data X and the first data Y1 with the estimation model M2.

[0048] For example, input data X corresponding to a specific section of an acoustic signal A and first data Y1 generated from the input data X of the section are input to an estimation model M2. However, the temporal relationship between the input data X and the first data Y1 input to the estimation model M2 is arbitrary. For example, the input data X corresponding to a specific section of the acoustic signal A and the first data Y1 generated by the first generation unit 621 for the section immediately preceding the section may be input to the estimation model M2.

[0049] Training data T used for machine learning of the estimation model M2 is data in which output data Z, which is a correct value, is associated with a set (i.e., learning data D) of training input data Xt and first data Y1 generated by the first generation unit 621 from the input data Xt. The procedure of the learning process is the same as that of the first embodiment.

[0050] In the second embodiment as well, the same effects as those of the first embodiment are achieved. Also, in the second embodiment, second processing using the estimation model M2 is executed for a set of the input data X and the first data Y1 that has been generated by the first processing . Therefore, compared with a form in which the second processing is executed only for the input data X (i.e., a form in which the situation of the first processing is not reflected in the second processing), output data can be generated stably and with high accuracy.

[0051] C: Third Embodiment FIG. 10 is a block diagram illustrating a specific configuration of the second analysis unit 62 in the third embodiment. The second analysis unit 62 in the third embodiment is composed of the same elements (first generation unit 621, second generation unit 622, third generation unit 623) as those in the first embodiment.

[0052] In addition to the input data X being supplied to the second generation unit 622 in the third embodiment in the same manner as in the first embodiment, a plurality of characteristic parameters P are also supplied. As described above, the plurality of characteristic parameters P are variables (for example, TS parameters) that define the physical model M1 used for the first processing, and are stored in advance in the storage device 12. Note that not all the characteristic parameters P that define the physical model M1 need to be included in the data D input to the estimation model M2.

[0053] As described above, the second generation unit 622 generates second data Y2 by performing a second process on data D, which includes input data X and a plurality of characteristic parameters P of the physical model M1. In other words, the second generation unit 622 generates second data Y2 by processing data D, which includes input data X and a plurality of characteristic parameters P, using the estimation model M2.

[0054] The training data T used for machine learning of the estimation model M2 is data in which the correct output data Z is associated with the training input data Xt and the set of multiple characteristic parameters P of the physical model M1 (i.e., the learning data D). The procedure for the learning process is the same as in the first embodiment.

[0055] The same effects as in the first embodiment are achieved in the third embodiment. In addition, in the third embodiment, a second processing using the estimation model M2 is performed on data D which includes input data and a plurality of characteristic parameters P of the physical model. That is, the characteristics of the physical model M1 used to generate the first data Y1 are also reflected in the second processing which includes processing by the estimation model M2. Therefore, compared to a form in which the data D processed by the estimation model M2 does not include the characteristic parameters P of the physical model M1 (i.e., a form in which the characteristics of the physical model M1 are not reflected in the second processing), output data Z can be generated stably and with high accuracy.

[0056] The configuration of the second embodiment is also applicable to the third embodiment. That is, the second processing may be performed on data D which includes input data X, first data Y1, and a plurality of characteristic parameters P.

[0057] D: Fourth Embodiment In the first embodiment, the input data X represents the state of the diaphragm 54 in the speaker device 20, and the output data Z represents the voltage to be supplied to the speaker device 20. In the fourth embodiment, the contents of the input data X and output data Z differ from those of the first embodiment.

[0058] In the fourth embodiment, the input data X represents the voltage (input voltage) to be supplied to the speaker device 20. That is, the first analysis unit 61 generates input data X that represents the voltage corresponding to the signal value of the acoustic signal A. Since the signal value of the acoustic signal A corresponds to sound pressure, the voltage represented by the input data X can also be expressed as sound pressure.

[0059] Furthermore, the output data Z of the fourth embodiment represents the state of the diaphragm 54 in the speaker device 20. Specifically, the output data Z includes the displacement, velocity, and acceleration of the diaphragm 54. In other words, the second analysis unit 62 of the fourth embodiment estimates the state of the diaphragm 54 (output data Z) when the voltage represented by the input data X is supplied to the speaker device 20.

[0060] Furthermore, the configuration of the second embodiment, in which the first data Y1 generated by the first generation unit 621 is input to the second generation unit 622 together with the input data X, and the configuration of the third embodiment, in which multiple characteristic parameters P of the physical model M1 are input to the second generation unit 622, may be similarly applied to the fourth embodiment.

[0061] E: The following are examples of specific modifications that can be added to each of the embodiments exemplified above. Two or more embodiments can be arbitrarily selected from the following examples and merged as appropriate, provided they do not contradict each other.

[0062] (1) In the first embodiment, the output data Z is shown as representing the voltage to be supplied to the speaker device 20, but the output data Z may also be data representing the current to be supplied to the speaker device 20. Also, in the fourth embodiment, the input data X is shown as representing the voltage to be supplied to the speaker device 20, but the input data X may also be data representing the current to be supplied to the speaker device 20.

[0063] (2) In each of the above-described embodiments, the generation of first data Y1 using the physical model M1 was exemplified as the first process executed by the first generation unit 621, but the first process may include other processes. For example, the first process may include a process for processing the input data X input to the physical model M1, or a process for processing the first data Y1 generated by the physical model M1.

[0064] (3) In each of the above-described embodiments, the generation of second data Y2 using the estimation model M2 was exemplified as a second process executed by the second generation unit 622, but the second process may include other processes. For example, the second process may include a process for processing the input data X input to the estimation model M2, or a process for processing the second data Y2 generated by the estimation model M2.

[0065] (4) In the above-described embodiments, the third generation unit 623 is shown as adding the first data Y1 and the second data Y2, but the content of the process for generating output data Z from the first data Y1 and the second data Y2 is not limited to the above embodiments. For example, the third generation unit 623 may generate output data Z by a weighted sum of the first data Y1 and the second data Y2. The addition of the first data Y1 and the second data Y2 also includes a weighted sum of the first data Y1 and the second data Y2. Alternatively, the third generation unit 623 may generate output data Z by a predetermined operation applied to the first data Y1 and the second data Y2. For example, the third generation unit 623 may generate output data Z by processing the first data Y1 and the second data Y2 using various mathematical models such as regression models (linear regression models, nonlinear regression models) or deep neural networks.

[0066] (5) In each of the above-described embodiments, an embodiment in which the information processing system 10 is equipped with a learning processing unit 70 has been conveniently illustrated, but the learning processing unit 70 may be installed in a separate system (machine learning system) from the information processing system 10. The machine learning system is implemented by a server device such as a web server, and constructs the estimation model M2 through the learning process described above. The estimation model M2 constructed by the machine learning system is transferred to the information processing system 10.

[0067] (6) In each of the above-described embodiments, a speaker system 100 in which the information processing system 10 and the speaker device 20 are configured separately was given as an example. However, each of the above-described embodiments also applies to a speaker system (active speaker) in which the information processing system 10 is installed inside the speaker device 20. Furthermore, the speaker system 100 can be realized not only as a stationary system, but also as a portable information device such as a smartphone, tablet terminal, or personal computer.

[0068] (7) The functions of the information processing system 10 as exemplified above are realized through the cooperation of one or more processors constituting the control device 11 and a program stored in the storage device 12, as described above. The program according to this disclosure can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, and optical recording media such as CD-ROMs (optical discs) are good examples, but any known form of recording medium such as semiconductor recording media or magnetic recording media is also included. A non-transitory recording medium includes any recording medium except transient propagation signals, and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the storage medium in which the distribution device stores the program corresponds to the non-transitory recording medium described above.

[0069] (8) The notation "the nth" (where n is a natural number) in this application is used solely as a formal and convenient label to distinguish each element in notation, and has no substantive meaning whatsoever. Therefore, there is no room for restrictive interpretation of the position or order of each element based on the notation "the nth".

[0070] F: From the forms exemplified above, the following configurations can be understood, for example.

[0071] An information processing method according to one aspect of the present disclosure (Aspect 1) generates first data relating to the state of a speaker device by performing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; generates second data relating to the state of the speaker device by performing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and generates output data relating to the state of the speaker device based on the first data and the second data.

[0072] In the above embodiment, output data relating to the state of the speaker device is generated based on first data generated in a first process including processing by a physical model and second data generated in a second process including processing by a machine learning-prepared estimation model. Therefore, compared to a configuration in which output data is generated by processing input data with a physical model, output data can be generated with higher accuracy. Furthermore, compared to a configuration in which output data is generated by processing input data with a machine learning-prepared estimation model, output data can be generated stably and with higher accuracy.

[0073] "The state of the speaker device" includes the behavior or characteristics of the speaker device. The behavior of the speaker device is any action that contributes to sound emission by the speaker device, such as the generation of magnetic force due to the supply of current to the voice coil, or the displacement of the diaphragm. The characteristics of the speaker device are any physical characteristics that contribute to sound emission by the speaker device, such as the elastic modulus of the damper supporting the diaphragm or the force coefficient (electromagnetic conversion coefficient) of the voice coil.

[0074] The data relating to the state of the speaker device (input data, first data, second data, output data) are arbitrary variables involved in the sound emission by the speaker device. For example, any data that affects the sound emission operation, such as physical quantities representing the behavior of the diaphragm (e.g., displacement, velocity, or acceleration), or voltage or current input to or output to the speaker device, is included in "data relating to the state of the speaker device."

[0075] "First process" is any process that includes processing by a physical model. That is, "first process" may consist only of processing by a physical model, or it may consist of processing by a physical model and other processes that are executed before and after said processing. Similarly, "second process" is any process that includes processing by a machine learning-prepared estimation model. That is, "second process" may consist only of processing by an estimation model, or it may consist of processing by an estimation model and other processes that are executed before and after said processing.

[0076] A "physical model" is any form of mathematical model that simulates the state of a speaker device. For example, a lumped-parameter system model is an example of a "physical model." An "estimation model," on the other hand, is a statistical model whose relationship between training control data and second training data has been acquired through prior machine learning. For example, a trained model such as a deep neural network is an example of an "estimation model."

[0077] In a specific example of Embodiment 1 (Embodiment 2), the generation of the second data is performed by executing the second process on data including the input data and the first data. In this embodiment, the second process using the estimation model is executed on data including the input data and the generated first data. Therefore, compared to a form in which the second process is executed only on the input data (i.e., a form in which the status of the first process is not reflected in the second process), output data can be generated stably and with high accuracy.

[0078] In a specific example of Embodiment 1 or Embodiment 2 (Embodiment 3), the physical model includes a plurality of characteristic parameters relating to the speaker device, and in the generation of the second data, the second data is generated by performing the second processing on data including the input data and the plurality of characteristic parameters. In the above embodiments, the second processing using the estimation model is performed on data including the input data and the plurality of characteristic parameters of the physical model. That is, the characteristics of the physical model used to generate the first data are also reflected in the second processing, which includes processing by the estimation model. Therefore, output data can be generated stably and with high accuracy compared to a form in which the data processed by the estimation model does not include the characteristic parameters of the physical model (i.e., a form in which the characteristics of the physical model are not reflected in the second processing).

[0079] In any specific example of Embodiments 1 to 3 (Embodiment 4), the input data is data relating to the state of the diaphragm in the speaker device, and the output data is data representing the voltage or current to be supplied to the speaker device. According to the above embodiments, the voltage or current (acoustic signal) to be input to the speaker device can be determined from the state of the diaphragm in the speaker device.

[0080] In any specific example of Embodiments 1 to 3 (Embodiment 5), the input data is data representing the voltage or current to be supplied to the speaker device, and the output data is data relating to the state of the diaphragm in the speaker device. According to the above embodiments, the displacement of the diaphragm in the speaker device can be estimated from the voltage or current (acoustic signal) input to the speaker device.

[0081] In any specific example of Embodiments 1 to 5 (Embodiment 6), the output data is generated by adding the first data and the second data. In the above embodiments, the output data is generated by adding the first data and the second data. That is, the second data corresponds to the difference between the output data and the first data (i.e., the error of the first data relative to the ideal output data). Therefore, the second data can be generated by the second processing so as to compensate for the error of the first data relative to the output data. In other words, machine learning of the estimation model is performed so that the error between the provisional first data generated in the first processing of the input data for training and the output data for training is output as the second data from the estimation model.

[0082] In any specific example of Embodiments 1 to 6 (Embodiment 7), the estimation model includes reservoir computing. According to the above embodiments, since the estimation model includes reservoir computing, the processing load required for machine learning of the estimation model can be reduced compared to the form in which the estimation model is constructed using a general recurrent neural network.

[0083] An information processing system according to one aspect of the present disclosure (Aspect 8) comprises: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0084] A program according to one aspect of the present disclosure (Aspect 9) causes a computer system to function as follows: a first generation unit that generates first data relating to the state of a speaker device by executing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by executing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

[0085] 100...Speaker system, 10...Information processing system, 11...Control device, 12...Storage device, 13...Input device, 14...Output device, 20...Speaker, 30...Signal supply device, 61...First analysis unit, 62...Second analysis unit, 621...First generation unit, 622...Second generation unit, 623...Third generation unit, 70...Learning processing unit.

Claims

1. An information processing method implemented by a computer system that generates first data relating to the state of a speaker device by performing a first process, which includes processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; generates second data relating to the state of the speaker device by performing a second process, which includes processing by a machine learning-trained estimation model, on the input data; and generates output data relating to the state of the speaker device based on the first data and the second data.

2. The information processing method of claim 1, wherein the generation of the second data is performed by executing the second process on data including the input data and the first data.

3. The information processing method according to claim 1, wherein the physical model includes a plurality of characteristic parameters relating to the speaker device, and in generating the second data, the second data is generated by performing the second processing on data including the input data and the plurality of characteristic parameters.

4. The information processing method according to claim 1, wherein the input data is data relating to the state of the diaphragm in the speaker device, and the output data is data representing the voltage or current to be supplied to the speaker device.

5. The information processing method according to claim 1, wherein the input data is data representing a voltage or current to be supplied to the speaker device, and the output data is data relating to the state of the diaphragm in the speaker device.

6. The information processing method of claim 1, wherein the output data is generated by adding the first data and the second data.

7. The information processing method of claim 1, which includes reservoir computing as the estimation model.

8. An information processing system comprising: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

9. A program that causes a computer system to function as follows: a first generation unit that generates first data relating to the state of a speaker device by performing a first process, including processing by a physical model that simulates the state of a speaker device, on input data relating to the state of the speaker device; a second generation unit that generates second data relating to the state of the speaker device by performing a second process, including processing by a machine learning-trained estimation model, on the input data; and a third generation unit that generates output data relating to the state of the speaker device based on the first data and the second data.

Images