Loudspeaker displacement model determination method, loudspeaker displacement determination method and device

Abstract

The invention provides a loudspeaker displacement and displacement model determination method and device, and electronic equipment, and the method comprises the steps: determining a first training set, the first training set comprises training data of a plurality of audio signals, and the plurality of audio signals are audio signals outputted by a loudspeaker with nonlinear characteristics; and training a loudspeaker displacement model by using the first training set, the loudspeaker displacement model being a neural network model. The method can achieve the precise prediction of the vibration displacement of the loudspeaker in a complex scene.

Description

Technical Field

[0001] The present disclosure relates to the field of loudspeaker protection, and in particular to a loudspeaker displacement model determination method, a loudspeaker displacement determination method and a device. Background Art

[0002] A speaker is a transducer device that converts electrical signals into acoustic signals. Its working principle is mainly to produce sound through the vibration of the diaphragm. In some extreme cases, if the diaphragm vibration exceeds the limit of the speaker itself, the speaker will exhibit nonlinear behavior, which will manifest as sound distortion and performance degradation. For this reason, it is necessary to monitor the vibration displacement of the speaker in advance so as to perform gain control on the playback signal. Summary of the invention

[0003] The present disclosure provides a loudspeaker displacement model determination method, a loudspeaker displacement determination method and device, and an electronic device to solve the problems in the related art.

[0004] A first aspect embodiment of the present disclosure proposes a method for determining a speaker displacement model, the method comprising: determining a first training set, the first training set comprising training data of a plurality of audio signals, the plurality of audio signals being audio signals output by a speaker having nonlinear characteristics; using the first training set to train a speaker displacement model, the speaker displacement model being a neural network model.

[0005] In some embodiments of the present disclosure, the training data of each audio signal includes: a voltage corresponding to the audio signal; a current corresponding to the audio signal; and a measured displacement of a speaker.

[0006] In some embodiments of the present disclosure, using a first training set, training a speaker displacement model includes: sampling each audio signal at a first sampling rate to determine a plurality of sampling samples, wherein each sampling sample includes a voltage and / or current corresponding to the audio signal; determining time domain features and frequency domain features corresponding to the plurality of sampling samples; obtaining a predicted displacement through a speaker displacement model based on the time domain features and the frequency domain features; and training the speaker displacement model based on the predicted displacement and the measured displacement.

[0007] In some embodiments of the present disclosure, based on time domain features and frequency domain features, obtaining predicted displacement through a speaker displacement model includes: fusing time domain features and frequency domain features to obtain fused features of multiple sampling samples; performing average pooling layer processing and fully connected layer processing on the fused features to obtain predicted displacement.

[0008] In some embodiments of the present disclosure, the method also includes: using a first verification set to verify the trained speaker displacement model to obtain a verification result, wherein the first verification set includes verification data of multiple audio signals, and the multiple audio signals are audio signals output by a speaker with nonlinear characteristics; when the verification result meets preset conditions, adjusting the parameters of the fully connected layer.

[0009] The second aspect of the present disclosure proposes a method for determining a speaker displacement, the method comprising: obtaining an input audio signal; using a speaker displacement model trained by the speaker displacement model training method according to any one of the first aspect of the embodiments of the present disclosure to determine the speaker displacement of the input audio signal.

[0010] The third aspect embodiment of the present disclosure proposes a device for determining a speaker displacement model, which includes: a determination module, used to determine a first training set, the first training set includes training data of multiple audio signals, and the multiple audio signals are audio signals output by a speaker with nonlinear characteristics; a training module, used to use the first training set to train a speaker displacement model, and the speaker displacement model is a neural network model.

[0011] The fourth aspect of the present disclosure proposes a speaker displacement determination device, which includes: an acquisition module for acquiring an input audio signal; a determination module for determining the speaker displacement of the input audio signal using a speaker displacement model trained by the speaker displacement model training method according to any one of the first aspect of the embodiments of the present disclosure.

[0012] The fifth aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect and the second aspect embodiments of the present disclosure.

[0013] The sixth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the methods described in the first and second aspects embodiments of the present disclosure.

[0014] The seventh aspect embodiment of the present disclosure proposes a computer program product, including a computer program, and the computer program executes the method described in the first aspect and the second aspect embodiment of the present disclosure when a processor is used.

[0015] The eighth aspect embodiment of the present disclosure proposes a chip, characterized in that it includes at least one processor and a communication interface; the communication interface is used to receive signals input into the chip or signals output from the chip, and the processor communicates with the communication interface and implements the methods described in the first and second aspects of the present disclosure through logic circuits or execution code instructions.

[0016] In summary, the speaker displacement and displacement model determination method proposed in the present invention obtains a displacement determination model by training a neural network model with data in complex scenarios, and monitors the speaker displacement based on the model. This can improve the accuracy of speaker displacement prediction in complex scenarios and achieve protection for speakers in complex scenarios or extreme situations.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0019] Figure 1 A schematic flow chart of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure;

[0020] Figure 2 A schematic flow chart of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure;

[0021] Figure 3 A schematic flow chart of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure;

[0022] Figure 4 A schematic flow chart of a method for determining loudspeaker displacement provided in an embodiment of the present disclosure;

[0023] Figure 5 An architecture diagram of a loudspeaker displacement model determination method system provided by an embodiment of the present disclosure;

[0024] Figure 6 An exemplary flow chart of a method for determining loudspeaker displacement provided in an embodiment of the present disclosure;

[0025] Figure 7 A schematic diagram of the structure of a device for determining a loudspeaker displacement model provided by an embodiment of the present disclosure;

[0026] Figure 8 A schematic diagram of the structure of a loudspeaker displacement determination device provided in an embodiment of the present disclosure;

[0027] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0028] Fig.10 A schematic diagram of the chip structure provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0030] The main working principle of the loudspeaker is to convert electrical signals into acoustic signals. The voice coil (which is required to be a conductor) is placed in a magnetic field and an alternating audio current is input into it. The voice coil is subjected to an alternating driving force to produce reciprocating motion, which drives the diaphragm to vibrate and repeatedly pushes the air (or propagation medium) to produce sound. However, when the diaphragm vibration exceeds the limit of the loudspeaker itself, the loudspeaker will exhibit nonlinear behavior, which will result in sound distortion and performance degradation.

[0031] Therefore, it is necessary to use a speaker protection algorithm to control the gain of the playback signal. In the speaker protection related modules, accurate modeling of the speaker's displacement response is crucial to optimizing the sound quality. For example, the speaker's equivalent circuit model can be used to protect the speaker to ensure that the diaphragm does not deviate too much when the speaker reaches the maximum loudness. The working principle is that the speaker protection algorithm can monitor the vibration of the speaker in advance and control the gain of the broadcast signal based on the monitoring data, so that the broadcast signal does not exceed the limit range of the speaker, thereby protecting the speaker.

[0032] The electrical continuous-time model of the loudspeaker equivalent circuit model can be expressed as:

[0033]

[0034] The continuous-time model of mechanics can be expressed as:

[0035]

[0036] In the above expression, R e is the DC resistance of the speaker, L E is the equivalent inductance of the speaker.

[0037] Mms is the vibration mass of the speaker, Rms is the mechanical damping of the speaker, Cms is the compliance of the speaker, and Bl is the electromechanical conversion factor. Based on these characteristic parameters of the speaker and the real-time input voltage, the voice coil displacement x(t) and the speed of the voice coil vibration can be obtained. and current.

[0038] However, when the speaker is in a complex scenario, such as high temperature and high pressure or when there is a problem with the speaker hardware itself, the speaker equivalent circuit model does not take into account the nonlinear characteristics of the speaker in complex scenarios, resulting in inaccurate prediction of displacement. Therefore, the speaker equivalent circuit model is not suitable for displacement prediction in complex scenarios. Based on the traditional speaker equivalent circuit model, the existing technology has conducted nonlinear theoretical research on some characteristic parameters, but its mathematical model is too complex and cannot bring out the maximum performance of the speaker in actual use, limiting the accuracy and efficiency of modeling.

[0039] In order to solve the problems existing in the related art, the present disclosure proposes a speaker displacement model determination method and a speaker displacement determination method. By training the model using training data in complex scenarios, and using the trained model to predict the speaker displacement, the accuracy of speaker vibration displacement prediction can be improved in complex scenarios.

[0040] For example, the speaker displacement and displacement model determination method disclosed in the present invention can use the following architecture to complete the prediction of speaker vibration displacement. The architecture can include a data acquisition module, a data preprocessing module, a neural network model design module, a model training module, and a model verification module. The data acquisition module can collect data in complex scenarios for model training and verification; the data preprocessing module can preprocess the collected audio signals; the neural network design module can be used to design a neural network structure adapted to the speaker displacement model; the model training module can use the data collected by data acquisition to train the model; the model verification module can verify the trained model and evaluate the performance of the model.

[0041] Figure 1 A schematic diagram of a flow chart of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure. Figure 1 As shown, the method can be performed by an electronic device, and the present disclosure does not limit the type of electronic device, which can perform model training. For example, the electronic device can be or the speaker displacement model determined by the electronic device can be used for a mobile terminal, a wearable device, a smart home device, or other device with a speaker assembly deployed therein. The method can include the following steps.

[0042] Step 101: determine a first training set.

[0043] In some embodiments, the first training set includes training data of a plurality of audio signals. For example, the plurality of audio signals may be audio signals output by a speaker having nonlinear characteristics.

[0044] In some embodiments, a speaker with nonlinear characteristics refers to a speaker that exhibits nonlinear characteristics in complex scenarios, such as high and low temperatures, high and low pressures, blocked holes, air leaks, speaker sound collisions, etc., which causes the speaker itself to change shape, such as the state of the speaker changes, thereby exhibiting nonlinear characteristics.

[0045] In some embodiments, the training data of each audio signal includes: a voltage corresponding to the audio signal; a current corresponding to the audio signal; and a measured displacement of a speaker.

[0046] In some embodiments, to obtain the first training set data, audio signals with different frequencies, amplitudes, and phases may be played through a speaker, and their voltage and current may be recorded, and displacement data may be obtained using a laser rangefinder.

[0047] In some embodiments, the data collected by the first training set may be data that has been preprocessed in preparation for input into a neural network. Data preprocessing may include data alignment, denoising, etc. In order to eliminate interference such as high-frequency noise and DC offset, the data may be filtered. At the same time, the voltage / current and displacement have separate clocks, and synchronous alignment sequence processing may be performed so that all data are in the same time dimension.

[0048] In some embodiments, the preprocessed data may be divided into a first training set, a first validation set, and a test set, wherein the first training set may be used to train the speaker training model.

[0049] Step 102: training a speaker displacement model.

[0050] In some embodiments, the first training set can be used to train the speaker displacement model. Since data with nonlinear characteristics in complex scenarios are used as training data for the model, the speaker displacement model can adapt to various complex scenarios and different speaker structures, thereby improving the accuracy of the speaker displacement model in predicting displacement in complex scenarios.

[0051] In some embodiments, the speaker displacement model may be a neural network model. The neural network has a strong learning ability and can learn the speaker vibration pattern from the input audio signal to improve the accuracy of the speaker displacement model in predicting the displacement.

[0052] In summary, the above-mentioned embodiments of the present application, by collecting data on speakers in complex scenes and with nonlinear characteristics, are used to train the speaker displacement model, so that the speaker displacement model can be adapted to different scenes and different speaker structures. Based on the powerful learning ability of the neural network, the accuracy of the speaker displacement model can be improved.

[0053] Figure 2 A schematic diagram of a flow chart of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure. Figure 2 As shown, based on Figure 1 In the embodiment shown, this embodiment specifically describes the above step 102. The method may include the following steps.

[0054] Step 201: Sample each audio signal.

[0055] In some embodiments, each audio signal may be an audio frame, and each audio signal in the first training set may be sampled at a first sampling rate. For example, each audio signal may be sampled at a sampling rate of 48 kHz, or other sampling rates may be used, which is not limited by the present disclosure.

[0056] In some embodiments, a plurality of sampled samples are determined through sampling, wherein each sampled sample may include a voltage and / or a current corresponding to the audio signal.

[0057] Step 202: determine the time domain features and frequency domain features corresponding to a plurality of sampling samples.

[0058] In some optional embodiments, a neural network may be used to design a neural network structure adapted to a speaker displacement model for acquiring time domain and frequency domain features in an audio signal. For example, a combination of a convolutional neural network (ConvNet) and a Fourier transform neural network (FFTNet) may be used.

[0059] In some embodiments, a plurality of sampling samples may be input into a speaker displacement model based on a neural network. Optionally, the speaker displacement model has two channels, corresponding to voltage and current respectively.

[0060] In some embodiments, one-dimensional convolution may be performed on multiple sampling samples of the input speaker displacement model to obtain time domain features corresponding to the multiple sampling samples, which may then be subjected to Fourier transform to extract frequency domain features, thereby determining time domain features and frequency domain features corresponding to the multiple sampling samples.

[0061] In some embodiments, the sampling samples are converted from time domain signals to frequency domain signals during Fourier transform, and the frequency domain signals are converted back to time domain before output. At this time, the output sampling samples have both time domain characteristics and frequency domain characteristics.

[0062] In some embodiments, the time domain features may be used to represent key change points of voltage and current, which may have a significant impact on the model.

[0063] Step 203: Based on the time domain features and the frequency domain features, a predicted displacement is obtained through a loudspeaker displacement model.

[0064] In some embodiments, the time domain features and the frequency domain features may be fused to obtain fused features of multiple sampled samples. For example, the features may be fused in a linear weighted manner, that is, different weight values ​​may be set for different features, and then an addition operation may be performed to achieve feature fusion.

[0065] In some embodiments, the fused features may be processed by an average pooling layer, and the dimension of the fused features may be reduced by the average pooling layer processing.

[0066] In some embodiments, the fusion features after dimension reduction may be subjected to a full connection layer (FC) processing, and a one-dimensional scalar may be obtained through the FC layer, and the scalar is the predicted displacement.

[0067] Step 204 : training a loudspeaker displacement model based on the predicted displacement and the measured displacement.

[0068] In some embodiments, the first training set includes multiple audio signals, and multiple predicted displacements corresponding to the multiple audio signals can be obtained.

[0069] In some embodiments, the speaker displacement model can be trained based on multiple predicted displacements and measured displacements. For example, for the same audio signal, the predicted displacement can be obtained after being processed by the speaker displacement model, the audio signal is played through a speaker, and the speaker displacement is measured using a laser rangefinder to obtain a measured displacement. The predicted displacement is compared with the measured displacement. When the numerical value difference is too large, the parameters of the model need to be adjusted. Optionally, the parameters of the fully connected layer or the network structure of the model can be adjusted, such as increasing or decreasing the number of network layers, increasing and / or decreasing the dimensions, etc.

[0070] In summary, the above-mentioned embodiments of the present application, by using a large amount of audio signal data to train the speaker displacement model based on the neural network, can make the model's prediction of the speaker displacement more accurate; training the model based on complex scene data can improve the adaptability of the model, so that the model can be used in various complex scenes; the trained model can improve the prediction speed, so that it can be suitable for real-time audio processing applications, and broaden the application scenarios of the model.

[0071] Figure 3A flowchart diagram of a method for determining a loudspeaker displacement model provided by an embodiment of the present disclosure is provided. Figure 1 or Figure 2 In the illustrated embodiment, the method may further include the following steps.

[0072] Step 301: verify the trained loudspeaker displacement model.

[0073] In some embodiments, the trained speaker displacement model may be validated using the first validation set.

[0074] In some embodiments, the data of the first validation set may be data that has not been processed by the speaker displacement model and may be used to determine the performance of the model when processing new data.

[0075] In some embodiments, the first verification set may include verification data of multiple audio signals, where the multiple audio signals are audio signals output by speakers having nonlinear characteristics, that is, the first verification set may include data in complex scenarios.

[0076] In some embodiments, the first validation set may also include data under normal scenarios.

[0077] In some embodiments, after the trained speaker displacement model is verified, adjustments can be made based on the verification results. For example, when the error between the predicted displacement and the actual measured displacement is large, the model can be adjusted, such as modifying the parameters of the fully connected layer of the model and / or adjusting the network structure of the embodiment model proposed in the present disclosure.

[0078] In summary, the embodiments proposed in the present disclosure can improve the accuracy of speaker displacement model prediction by verifying and adjusting the trained model.

[0079] Figure 4 The present invention provides a flowchart of a method for determining speaker displacement according to an embodiment of the present invention. The method can be performed by an electronic device. The present invention does not limit the type of electronic device, which can perform model reasoning. For example, the electronic device can be a mobile terminal, a wearable device, a smart home device, or other device equipped with a speaker assembly. Figure 4 As shown, this method can be used Figures 1 to 3 The loudspeaker displacement model obtained by any of the embodiments shown in the method may also include the following steps.

[0080] Step 401: Acquire an input audio signal.

[0081] In some embodiments, the audio signal may be collected and the voltage and / or current information corresponding to the audio signal may be recorded.

[0082] In some embodiments, the collected audio signals can be preprocessed, including data alignment, denoising, etc. When collecting audio signals, the data needs to be filtered to eliminate high-frequency noise and DC offset. At the same time, the voltage / current and displacement have separate clocks, and the sequences need to be synchronized so that the data are in the same time dimension.

[0083] Step 402: determine the speaker displacement of the input audio signal.

[0084] In some embodiments, the electronic device may use a speaker displacement determination model to determine the speaker displacement of the input audio signal. For example, the speaker displacement determination model may be Figures 1 to 3 The loudspeaker displacement model obtained by any of the embodiments shown in the figure is trained by an audio signal output by a loudspeaker with nonlinear characteristics, thereby improving the accuracy of loudspeaker displacement monitoring in complex scenes or extreme situations, which will not be described in detail here.

[0085] In some embodiments, voltage and / or current information corresponding to the preprocessed audio signal may be used as input data of the model.

[0086] In some embodiments, the speaker displacement model can perform one-dimensional convolution and Fourier transform processing on the input data to obtain its corresponding time domain features and frequency domain features.

[0087] In some embodiments, the time domain features and frequency domain features corresponding to the input data may be fused to obtain fused features corresponding to the audio signal. For example, weights may be set for different features through linear weighting to obtain the fused features of the audio signal.

[0088] In some embodiments, the fused features may be processed through an average pooling layer to reduce their dimension, and then convolved through a fully connected layer to obtain a one-dimensional scalar, which is the speaker displacement corresponding to the audio signal.

[0089] In some embodiments, a trained speaker displacement model may be used to determine the speaker displacement of the input audio signal.

[0090] In summary, the above embodiments of the present application obtain input data by acquiring an audio signal and preprocessing the audio signal. The speaker displacement model processes the input data to obtain the displacement of the speaker. By adopting the trained speaker displacement model, the accuracy of speaker displacement prediction can be improved. The trained model can achieve a faster inference speed and can be suitable for real-time audio processing applications.

[0091] The technical solution of the present disclosure is further described in detail below in conjunction with specific application examples.

[0092] Figure 5 An architectural diagram of a loudspeaker displacement model determination method system provided in an embodiment of the present disclosure.

[0093] The system includes data acquisition module, data preprocessing module, neural network model design module, model training module and model verification module.

[0094] The data acquisition module can collect data in complex scenarios for model training and verification; the data preprocessing module can preprocess the collected audio signals; the neural network design module can be used to design a neural network structure suitable for the speaker displacement model; the model training module can use the data collected by data acquisition to train the model; the model verification module can verify the trained model and evaluate the performance of the model.

[0095] Based on the above architecture, a method for determining the loudspeaker displacement is provided, such as Figure 6 As shown, the specific steps of the method are as follows.

[0096] Data acquisition module: It can collect data in complex scenarios, such as different environments (high and low temperature, high and low pressure), different states (blocked holes, air leakage) or other external forces that cause speaker defects (such as speaker deformation caused by external force collision). By using audio signals of different frequencies, amplitudes and phases, playing them through the speakers, recording their voltage and current, and using the laser rangefinder to obtain displacement data.

[0097] Data preprocessing module: The collected audio signals can be preprocessed, including data alignment, denoising, etc., to prepare data for input into the neural network. In data acquisition, in order to eliminate high-frequency noise and DC offset, the data needs to be filtered. At the same time, the voltage / current and displacement have separate clocks, and the sequence needs to be synchronized so that each data is in the same time dimension. The training input data of the neural network is finally obtained as the collected voltage and current [V(t), I(t)], and the label is the displacement data X(t) collected by the laser.

[0098] Neural Network Design Module: It can be used to design a neural network structure suitable for speaker displacement modeling. It can use a combination of ConvNet and FFTNet to capture the time domain and frequency domain features of the audio signal. Figure 6 As shown in the figure, 2*512 samples are input, and the two channels correspond to voltage and current respectively, with 10.7ms of data at a sampling rate of 48kHz; after one-dimensional convolution, the 64*512-dimensional time domain features after channel fusion are obtained, and then the frequency domain features are extracted through the Fourier transform structure model. After feature fusion, the average pooling layer and the FC layer are passed to output the final predicted displacement.

[0099] Model training module: Use the preprocessed data and divide it into training set, validation set and test set. Through a large number of iterative training of the neural network, the network weights and parameters are optimized so that it can accurately learn the mapping relationship between audio signals and speaker vibrations.

[0100] Model verification module: Use the verification set to verify the trained model and evaluate its performance on unseen data. Necessary adjustments can be made based on the verification results. For example, if the displacement obtained by the model through calculation is significantly different from the displacement obtained by the laser test, the parameters of the fully connected layer and / or the network structure of the model can be adjusted.

[0101] Figure 7 FIG. 7 is a schematic diagram of a structure of a loudspeaker displacement model determination device 700 provided in an embodiment of the present disclosure. Figure 7 As shown, the loudspeaker displacement model determination device includes:

[0102] The determination module 710 is used to determine a first training set, which includes training data of multiple audio signals, and the multiple audio signals are audio signals output by a speaker with nonlinear characteristics; the training module 720 is used to use the first training set to train a speaker displacement model, and the speaker displacement model is a neural network model.

[0103] In some embodiments, the training data of each audio signal includes: a voltage corresponding to the audio signal; a current corresponding to the audio signal; and a measured displacement of a speaker.

[0104] In some embodiments, the training module 720 can also be used to: sample each audio signal at a first sampling rate to determine multiple sampling samples, wherein each sampling sample includes a voltage and / or current corresponding to the audio signal; determine time domain features and frequency domain features corresponding to the multiple sampling samples; obtain a predicted displacement through a speaker displacement model based on the time domain features and the frequency domain features; and train the speaker displacement model based on the predicted displacement and the measured displacement.

[0105] In some embodiments, the training module 720 can also be used to: fuse the time domain features and the frequency domain features to obtain fused features of multiple sampling samples; perform average pooling layer processing and fully connected layer processing on the fused features to obtain predicted displacement.

[0106] In some embodiments, the device 700 also includes a verification module, which is used to: use a first verification set to verify the trained speaker displacement model to obtain a verification result, wherein the first verification set includes data of multiple audio signals, and the multiple audio signals are audio signals output by a speaker with nonlinear characteristics; when the verification result meets preset conditions, adjust the parameters of the fully connected layer.

[0107] In summary, the speaker displacement model determination device proposed in the present invention acquires and processes the first training set data, that is, acquires data in complex scenes or different speaker structures, and trains the speaker displacement model based on the data, which can improve the adaptability of the speaker displacement model. The speaker displacement model is designed based on a neural network, which can improve the accuracy of the speaker displacement model in predicting the displacement. The trained model can achieve a faster inference speed and can be suitable for real-time audio processing applications.

[0108] Figure 8 FIG. 8 is a schematic diagram of a structure of a speaker displacement determination device 800 provided in an embodiment of the present disclosure. Figure 8 As shown, the loudspeaker displacement determination device comprises:

[0109] The acquisition module 810 is used to acquire the input audio signal; the determination module 820 is used to determine the speaker displacement of the input audio signal using the speaker displacement model trained by the speaker displacement model training method according to any one of the methods of the first aspect of the embodiments of the present disclosure.

[0110] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0111] In summary, the speaker displacement determination device proposed in the present disclosure can improve the accuracy of displacement prediction and achieve a faster inference speed by using a trained speaker displacement model to predict displacement.

[0112] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0113] Fig. 9 The block diagram of an electronic device 900 for implementing the above-mentioned speaker displacement model determination method and speaker displacement determination method according to an exemplary embodiment is shown. For example, the electronic device 900 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0114] Reference Fig. 9, the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0115] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0116] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0117] The power supply component 906 provides power to the various components of the electronic device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 900.

[0118] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0119] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0120] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0121] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900, and the sensor assembly 914 can also detect the position change of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and the temperature change of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0122] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0123] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0124] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the electronic device 900 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the above embodiments of the present disclosure.

[0126] The embodiments of the present disclosure further provide a computer program product, including a computer program, which executes the method described in the above embodiments of the present disclosure when a processor is used.

[0127] Fig.10 FIG. 1 is a schematic diagram of a structure of a chip 1000 for implementing the above-mentioned loudspeaker displacement model determination method and loudspeaker displacement determination method according to an exemplary embodiment. Fig.10 The chip 1000 includes a communication interface 1001 and at least one processor 1002. The communication interface 1001 is used to receive a signal input to the chip 1000 or a signal output from the above chip 1000. The processor 1002 communicates with the communication interface 1001 and implements the speaker displacement model determination method and the speaker displacement determination method described in the above embodiments of the present disclosure through a logic circuit or executing code instructions.

[0128] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0129] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0130] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0132] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0133] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0135] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for determining a loudspeaker displacement model, characterized in that: The method comprises: Determine a first training set, wherein the first training set includes training data of a plurality of audio signals, wherein the plurality of audio signals are audio signals output by a speaker having nonlinear characteristics; The first training set is used to train a speaker displacement model, where the speaker displacement model is a neural network model.

2. The method according to claim 1, characterized in that The training data for each audio signal includes: The voltage corresponding to the audio signal; The current corresponding to the audio signal; The measured displacement of the loudspeaker is measured.

3. The method according to claim 2, characterized in that The using the first training set to train the speaker displacement model comprises: Sampling each audio signal at a first sampling rate to determine a plurality of sampled samples, wherein each sampled sample includes a voltage and / or a current corresponding to the audio signal; Determine the time domain features and frequency domain features corresponding to the multiple sampling samples; Based on the time domain features and the frequency domain features, obtaining predicted displacement through the loudspeaker displacement model; The loudspeaker displacement model is trained based on the predicted displacement and the measured displacement.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining the predicted displacement by the loudspeaker displacement model based on the time domain feature and the frequency domain feature comprises: Fusing the time domain features and the frequency domain features to obtain fused features of the multiple sampling samples; The fused features are processed by an average pooling layer and a fully connected layer to obtain the predicted displacement.

5. The method according to claim 4, characterized in that The method further comprises: Using a first validation set, validating the trained loudspeaker displacement model to obtain a validation result, wherein the first validation set includes validation data of a plurality of audio signals, and the plurality of audio signals are audio signals output by a loudspeaker having nonlinear characteristics; When the verification result meets the preset conditions, the parameters of the fully connected layer are adjusted.

6. A method for determining loudspeaker displacement, characterized in that: The method comprises: Get the input audio signal; The speaker displacement of the input audio signal is determined using the speaker displacement model trained by the speaker displacement model training method according to any one of claims 1 to 5.

7. A device for determining a loudspeaker displacement model, characterized in that: The device comprises: A determination module, configured to determine a first training set, wherein the first training set includes training data of a plurality of audio signals, wherein the plurality of audio signals are audio signals output by a speaker having nonlinear characteristics; A training module is used to train a speaker displacement model using the first training set, where the speaker displacement model is a neural network model.

8. A device for determining loudspeaker displacement, characterized in that: The device comprises: An acquisition module, used for acquiring an input audio signal; A determination module is used to determine the speaker displacement of the input audio signal using a speaker displacement model trained by the speaker displacement model training method according to any one of claims 1 to 5.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

11. A chip, characterized in that: It includes at least one processor and a communication interface; the communication interface is used to receive a signal input to the chip or a signal output from the chip, and the processor communicates with the communication interface and implements the method as described in any one of claims 1 to 6 through a logic circuit or executing code instructions.

Images