Non-contact Hall drum surface vibration sensing and intelligent pronunciation control method and system
By performing in-depth analysis and physical feature extraction on the signals collected by the Hall sensor array, a pitch and timbre mapping relationship is established, and an audio merging model is constructed. This solves the problem of unrealistic timbre changes in existing technologies and improves the naturalness and expressiveness of drum sounds.
Patent Information
- Application Number
- CN202511655934.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-03-20
AI Technical Summary
Existing non-contact Hall effect drum vibration sensing methods ignore the physical relationship between the physical behavior of the tympanic membrane and timbre parameters, resulting in unrealistic timbre changes, a lack of dynamic perception, and limited expressiveness.
The raw signal is acquired by a Hall sensor array, and the signal is preprocessed and physical feature parameters are extracted to obtain the tympanic membrane tension parameters, damping coefficient and impact position. A mapping relationship is established, and combined with timbre brightness and overtone enhancement factor, an audio merging model is constructed to achieve coordinated control of pitch and timbre characteristics.
It significantly improves the naturalness and expressiveness of the drum sound, meeting the needs of higher-performance smart electronic drum applications.
Smart Images
Figure CN121708879A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vocalization control technology, specifically to a non-contact Hall effect drumhead vibration sensing and intelligent vocalization control method and system. Background Technology
[0002] The non-contact Hall effect drumhead vibration sensing method involves attaching multiple small permanent magnets to the back of the drumhead. A Hall sensor array is fixedly installed inside the drum cavity, maintaining a non-contact gap with the permanent magnets. When the drumhead is struck and vibrates, the permanent magnets move accordingly, causing a change in the relative distance between them and the Hall sensor. This results in a change in the magnetic field strength passing through the Hall element. The Hall sensor outputs an analog voltage signal proportional to the change in magnetic field strength. This analog voltage signal directly reflects the displacement of the drumhead vibration. The signal is then amplified, purified, and converted into a digital signal. Nonlinear correction is performed to make the digital signal represent the actual displacement of the drumhead as linearly as possible. The processed digital audio signal is then converted back into an analog signal, amplified, and connected to an audio device to produce sound.
[0003] For non-contact Hall effect drumhead vibration sensing sound production methods, existing technologies only output analog voltage signals through Hall sensors, which are then directly converted into audio signals after simple amplification and filtering. The sound production control logic is simple, and traditional audio systems generally ignore the physical relationship between the physical behavior of the tympanic membrane caused by the impact and the timbre parameters, resulting in unrealistic timbre changes, a lack of dynamic perception, limited expressiveness, and affecting the performance effect. Summary of the Invention
[0004] To address the aforementioned problems, this invention provides a non-contact Hall effect drumhead vibration sensing and intelligent sound production control method and system.
[0005] This invention employs the following technical solution: a non-contact Hall effect drumhead vibration sensing and intelligent sound production control method, comprising:
[0006] The raw Hall sensor signal generated by the user striking the drumhead is acquired by a Hall sensor array inside the eardrum. The raw Hall sensor signal is then preprocessed to obtain a standard Hall sensor signal. ;
[0007] Standard Hall sensor electrical signal Physical feature parameters are extracted to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position;
[0008] The initial tympanic membrane tension parameter is obtained by using a tympanic membrane stress sensor. The first tympanic membrane tension parameter and the initial tympanic membrane tension parameter are compared and analyzed to obtain the tension decay value. Based on the pre-constructed mapping relationship between tension decay value and pitch, the pitch parameter is obtained.
[0009] A pre-constructed mapping relationship between damping coefficient and sound decay time, and a mapping relationship between impact position and timbre type, are used to obtain the corresponding sound decay time and timbre type based on the damping coefficient and impact position.
[0010] Standard Hall sensor electrical signal Extended timbre feature parameter extraction is performed to obtain timbre brightness and overtone enhancement factor;
[0011] The acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters are input into a pre-built audio merging model to obtain the audio output.
[0012] As a further description of the above technical solution: the method for obtaining the first tympanic membrane tension parameter includes:
[0013] The standard Hall sensor electrical signal is converted using a fast Fourier transform. Convert it into a frequency domain signal to obtain the intensity of each frequency component, where t is time;
[0014] Obtain the magnitude of the Fast Fourier Transform (FFT) to get the vibration amplitude intensity of each frequency component, and find the frequency point with the maximum vibration amplitude intensity. ;
[0015] Obtain the vibration radius and areal density of the tympanic membrane based on the frequency point. 、 The vibration radius and areal density of the tympanic membrane are used to calculate the first tympanic membrane tension parameter.
[0016] As a further description of the above technical solution: the method for obtaining the damping coefficient includes:
[0017] Extracting standard Hall sensor electrical signals The envelope of the vibrational displacement decaying over time;
[0018] The logarithmic decay rate is obtained based on the number of vibration cycles and the vibration amplitude, and the damping coefficient is calculated based on the logarithmic decay rate.
[0019] As a further description of the above technical solution: the method for obtaining the striking position includes:
[0020] Based on the center point of the drumhead, a coordinate system is constructed to obtain the position coordinates of multiple Hall sensor arrays installed inside the tympanic membrane, denoted as . ; Let x be the x-coordinate of the i-th Hall sensor. Let be the ordinate of the i-th Hall sensor;
[0021] Obtain the time when each Hall sensor receives the vibration wave, select the Hall sensor that receives the vibration wave earliest and mark it as the reference sensor, obtain the reception time of the reference sensor, and record it as the reference time. Obtain the time difference between other Hall sensors and the reference time, denoted as . ;in, , The time when the i-th Hall sensor receives the vibration wave;
[0022] Obtain the propagation speed of vibration waves on the tympanic membrane The difference between the actual distance from the impact position to the i-th Hall sensor and the distance from the impact position to the reference sensor is obtained.
[0023] The spatial coordinates of the striking position are calculated using triangulation.
[0024] As a further description of the above technical solution: the method for constructing the striking position-timbre type mapping relationship is as follows: the drum surface is divided into G striking position regions, where G is a positive integer greater than 1; striking signals from each striking position region are collected and corresponding timbre features are extracted; the timbre features are classified and labeled with timbre tags, and the timbre types include bright, soft, full, and hollow; through statistical analysis, a mapping relationship is established between the striking position and the corresponding timbre tag, thus constructing the striking position-timbre type mapping relationship.
[0025] As a further description of the above technical solution: the method for obtaining timbre brightness includes:
[0026] Standard Hall sensor electrical signal Perform p-level wavelet decomposition to obtain multiple high-frequency detail components. ; This represents the high-frequency detail components of the j-th layer of the i-th Hall sensor;
[0027] Based on multiple high-frequency detail components Calculate the energy of each layer ;
[0028] Based on the energy obtained from each layer The timbre brightness is calculated based on the preset energy weights for each layer.
[0029] As a further description of the above technical solution: the method for obtaining the overtone enhancement factor includes:
[0030] Standard Hall sensor electrical signal Perform a Fast Fourier Transform to obtain the spectrum. , For pitch frequency;
[0031] From the spectrum Find the main peak frequency The overtone enhancement factor is calculated and obtained.
[0032] As a further description of the above technical solution: A mapping relationship between tension decay value and pitch is established, and its mapping function form is as follows: In the formula, For pitch frequency, This is the tension decay value. This is a mapping function.
[0033] The mapping function is obtained by regression analysis of experimentally measured decay values and pitch frequency correspondence data.
[0034] As a further description of the above technical solution: the training method of the audio merging model includes:
[0035] X sets of training data are collected in advance, where X is a positive integer greater than 0. The training data includes standard Hall sensor electrical signals, pitch parameters, sound decay time, timbre type and extended timbre feature parameters, and corresponding audio.
[0036] A multimodal fusion generative network is adopted as the model architecture design, including an input layer, a fusion layer, and an output layer;
[0037] The collected X sets of training data are divided into training set, validation set and test set according to a preset ratio;
[0038] The model is trained using the training set, the optimizer and learning rate are set, the model parameters are updated using the backpropagation algorithm, the loss function is minimized, and the model is validated using the validation set to achieve the best performance on the validation set.
[0039] The trained model is evaluated using a test set, and the mean squared error index of the model is calculated to assess its performance. Once the model's performance evaluation meets the standards, the trained audio merging model is deployed and applied.
[0040] A non-contact Hall effect drumhead vibration sensing and intelligent sound production control system, used to implement the aforementioned non-contact Hall effect drumhead vibration sensing and intelligent sound production control method, the system comprising:
[0041] The raw data acquisition module collects the raw Hall sensor signals generated by the user striking the drumhead through a Hall sensor array inside the eardrum. It then performs signal preprocessing on these raw Hall sensor signals to obtain the standard Hall sensor signals. ;
[0042] The physical feature extraction module extracts standard Hall sensor electrical signals. Physical feature parameters are extracted to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position;
[0043] The first parameter processing module obtains the initial tympanic membrane tension parameter through the tympanic membrane stress sensor, compares and analyzes the first tympanic membrane tension parameter with the initial tympanic membrane tension parameter to obtain the tension decay value, and obtains the pitch parameter based on the pre-constructed mapping relationship between the tension decay value and the pitch.
[0044] The second parameter processing module pre-builds the mapping relationship between damping coefficient and sound decay time and the mapping relationship between impact position and timbre type, and obtains the corresponding sound decay time and timbre type based on the damping coefficient and impact position;
[0045] The extended feature extraction module is used for standard Hall sensor electrical signals. Extended timbre feature parameter extraction is performed to obtain timbre brightness and overtone enhancement factor;
[0046] The audio output module inputs the acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters into a pre-built audio merging model to obtain the audio output.
[0047] The beneficial effects of this application are as follows:
[0048] Compared with the existing technology that simply outputs an analog voltage signal from a Hall sensor and converts it directly into an audio signal after simple amplification and filtering, this application can perform a deeper structural analysis of the Hall sensor signal. Through signal preprocessing and physical feature parameter extraction, it systematically extracts key quantities reflecting the physical state of the tympanic membrane, such as tympanic membrane tension parameters, damping coefficient, and impact position, providing a real and accurate physical basis for sound generation.
[0049] Furthermore, this application introduces a tympanic membrane stress sensor to obtain an initial tension baseline, and combines the tension decay value, damping coefficient, and impact position to establish mapping relationships between pitch, sound decay time, and timbre type. In addition, it constructs an audio merging model by using timbre brightness and overtone enhancement factors extracted from standard Hall sensor electrical signals. This enables coordinated control of dynamic changes in pitch and timbre characteristics, thereby significantly improving the naturalness, expressiveness, and personalization of drum sound, and meeting the application requirements of higher-performance intelligent electronic drums. Attached Figure Description
[0050] The present invention will be further explained below with reference to the accompanying drawings and embodiments:
[0051] Figure 1 A flowchart of a non-contact Hall effect drumhead vibration sensing and intelligent sound control method provided in an embodiment of the present invention;
[0052] Figure 2 This is a flowchart of a method for obtaining a first tympanic membrane tension parameter provided in an embodiment of the present invention;
[0053] Figure 3 This is a flowchart of a method for obtaining the impact position provided in an embodiment of the present invention;
[0054] Figure 4 This is a module connection diagram of the non-contact Hall effect drumhead vibration sensing and intelligent sound control system provided in an embodiment of the present invention. Detailed Implementation
[0055] To make the technical means, creative features, objectives, and effects of this invention readily understandable, the invention is further described below with reference to specific illustrations. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0056] Example 1
[0057] Please see Figures 1-3 This invention provides a technical solution: a non-contact Hall effect drumhead vibration sensing and intelligent sound production control method.
[0058] The raw Hall sensor signal generated by the user striking the drumhead is acquired by a Hall sensor array inside the eardrum. The raw Hall sensor signal is then preprocessed to obtain a standard Hall sensor signal. ;
[0059] The signal preprocessing includes DC component removal, filtering and denoising, and normalization. DC component removal is used to eliminate offset, filtering and denoising is performed using wavelet thresholding or a low-pass filter, and normalization is used to adjust the maximum amplitude of the signal to 1 and unify the amplitude range.
[0060] It should be noted that, This represents the standard Hall sensor electrical signal of the i-th Hall sensor at time t;
[0061] Standard Hall sensor electrical signal Physical feature parameters are extracted to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position;
[0062] Methods for obtaining the first tympanic membrane tension parameter include:
[0063] The standard Hall sensor electrical signal is converted using a fast Fourier transform. Convert it into a frequency domain signal to obtain the intensity of each frequency component, where t is time;
[0064] The expression for the Fast Fourier Transform is: In the formula, For pitch frequency The corresponding amplitude, The duration of data collection. is a complex rotation factor that performs the transformation from the time domain to the frequency domain, where j is the imaginary unit;
[0065] Obtain the magnitude of the Fast Fourier Transform (FFT) to get the vibration amplitude intensity of each frequency component, and find the frequency point with the maximum vibration amplitude intensity. ;
[0066] The modulus The calculation method is as follows: In the formula, for The real part, for The imaginary part is the pitch frequency. The vibration amplitude and intensity at that location;
[0067] Obtain the vibration radius and areal density of the tympanic membrane based on the frequency point. 、 The vibration radius and areal density of the tympanic membrane are used to calculate the first tympanic membrane tension parameter;
[0068] The formula for calculating the first tympanic membrane tension parameter is: In the formula, This is the first tympanic membrane tension parameter, in Newtons. The areal density of the tympanic membrane material, expressed in kg / m². The effective vibration radius of the tympanic membrane, measured in meters.
[0069] It should be noted that the tympanic membrane tension parameter is used to control the pitch of the drum. The greater the tension, the tighter the drumhead is stretched, and the higher the vibration frequency. Therefore, the fundamental frequency rises and the pitch increases. That is, when the drum is struck hard, the tension increases instantaneously, the drumhead is stretched tighter, and the vibration frequency is higher. Therefore, the fundamental frequency rises and the pitch increases.
[0070] The initial tympanic membrane tension parameter is obtained by using a tympanic membrane stress sensor. The first tympanic membrane tension parameter and the initial tympanic membrane tension parameter are compared and analyzed to obtain the tension decay value. Based on the pre-constructed mapping relationship between tension decay value and pitch, the pitch parameter is obtained.
[0071] It should be noted that the mapping function for establishing the mapping relationship between tension decay value and pitch frequency is as follows: In the formula, For pitch frequency, This is the tension decay value. This is a mapping function.
[0072] The mapping function is obtained by regression analysis of experimentally measured decay values and pitch frequency correspondence data, or by derivation formula of theoretical frequency of tympanic membrane vibration, and is used to dynamically predict pitch changes based on real-time tympanic membrane tension decay.
[0073] Methods for obtaining the damping coefficient include:
[0074] Extracting standard Hall sensor electrical signals The envelope of the vibrational displacement decaying over time;
[0075] The logarithmic decay rate is obtained based on the number of vibration cycles and the vibration amplitude, and the damping coefficient is calculated based on the logarithmic decay rate.
[0076] It should be noted that the method for obtaining the envelope is as follows: by identifying all the peak points, these peak points are connected to form an envelope.
[0077] The formula for calculating the logarithmic decay rate is: In the formula, The logarithmic decay rate, The amplitude of the first wave peak. The amplitude of the (n+1)th wave peak. The number of vibration cycles spanned.
[0078] The formula for calculating the damping coefficient is as follows: In the formula, The logarithmic decay rate, is the damping coefficient.
[0079] It should be noted that the damping coefficient controls the decay time and high-frequency decay characteristics of the sound. The greater the damping, the faster the sound disappears, especially the high-frequency part decays faster, making the sound more muffled and shorter. The smaller the damping, the longer the sound lasts, the longer the reverberation, the more high frequencies are retained, and the brighter the sound. A high damping coefficient corresponds to a short and dry tone; a low damping coefficient corresponds to a long and resonant tone.
[0080] Methods for obtaining the hit location include:
[0081] Based on the center point of the drumhead, a coordinate system is constructed to obtain the position coordinates of multiple Hall sensor arrays installed inside the tympanic membrane, denoted as . ; Let x be the x-coordinate of the i-th Hall sensor. Let be the ordinate of the i-th Hall sensor;
[0082] Obtain the time when each Hall sensor receives the vibration wave, select the Hall sensor that receives the vibration wave earliest and mark it as the reference sensor, obtain the reception time of the reference sensor, and record it as the reference time. Obtain the time difference between other Hall sensors and the reference time, denoted as . ;in, , For the first The time it takes for a Hall sensor to receive a vibration wave;
[0083] Obtain the propagation speed of vibration waves on the tympanic membrane The difference between the actual distance from the impact position to the i-th Hall sensor and the distance from the impact position to the reference sensor is obtained.
[0084] The formula for calculating the distance difference is: In the formula, This represents the actual distance from the impact position to the i-th Hall sensor. This represents the propagation speed of the vibration wave on the tympanic membrane.
[0085] The spatial coordinates of the striking position were calculated using triangulation.
[0086] The calculation formula for the triangulation method is as follows:
[0087] In the formula, The coordinates of the hit position, The number of Hall sensors, For the position at (x, y) up to the th The distance between the sensors Let x, y be the geometric distance from the impact position (x, y) to the reference Hall sensor. This indicates the coordinates of the point of impact. This minimizes the sum of squared prediction errors for all measurement points;
[0088] It should be noted that the striking position determines which part of the drumhead is excited, thus affecting the drum's vibration mode. Different striking positions will excite different vibration modes, resulting in different timbres. For example, a center strike mainly excites the fundamental frequency and symmetrical modes, producing a full sound; while an edge strike excites more high-frequency asymmetrical modes, producing a sharper, metallic sound. By obtaining the striking position coordinates, the model will calculate the corresponding initial conditions based on the striking position, thereby generating sounds with different timbres.
[0089] A pre-constructed mapping relationship between damping coefficient and sound decay time, and a mapping relationship between impact position and timbre type, are used to obtain the corresponding sound decay time and timbre type based on the damping coefficient and impact position.
[0090] It should be noted that the mapping function for the damping coefficient-sound decay time relationship is as follows: In the formula, The decay time of the sound, in seconds. The damping coefficient is... For the fitting parameters, To fit the bias, the function is obtained from experimental measurement data through regression analysis.
[0091] The method for constructing the striking position-timbre type mapping relationship is as follows: the drumhead is divided into G striking position regions, where G is a positive integer greater than 1. The striking signals of each region are collected and the corresponding timbre features are extracted. The timbres are classified and labeled, including timbre types such as bright, soft, full, and hollow. Through statistical analysis, the striking position is mapped to the corresponding timbre label to construct the striking position-timbre type mapping relationship.
[0092] Standard Hall sensor electrical signal Extended timbre feature parameter extraction is performed to obtain timbre brightness and overtone enhancement factor;
[0093] Methods for obtaining timbre brightness include:
[0094] Standard Hall sensor electrical signal conduct Layered wavelet decomposition yields multiple high-frequency detail components. ; Indicates the first High-frequency detail components of the c-th layer of a Hall sensor;
[0095] Based on multiple high-frequency detail components Calculate the energy of each layer The calculation formula is: In the formula, For the first The energy of the c-th layer detail component of a Hall sensor.
[0096] Based on the energy obtained from each layer Based on the preset energy weights for each layer, the timbre brightness is calculated and obtained;
[0097] The formula for calculating the timbre brightness is: In the formula, For the first The timbre brightness of each Hall sensor For the weight of the j-th layer, optionally, =j, This represents the number of wavelet decomposition layers.
[0098] It should be noted that timbre brightness is used to characterize the proportion of "high frequency content" in a sound; the brighter the timbre, the higher the frequency content.
[0099] Methods for obtaining overtone enhancement factors include:
[0100] Standard Hall sensor electrical signal Perform a Fast Fourier Transform to obtain the spectrum. ;, For pitch frequency;
[0101] From the spectrum Find the main peak frequency The overtone enhancement factor is calculated and obtained.
[0102] The overtone enhancement factor The calculation formula is: In the formula, For the spectrum at frequency amplitude, For consideration before One overtone, optionally... ∈(5-10) For the first The overtone enhancement factor corresponding to each Hall sensor.
[0103] The acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters are input into a pre-built audio merging model to obtain the audio output.
[0104] Training methods for audio merging models include:
[0105] X sets of training data are collected in advance, where X is a positive integer greater than 0. The training data includes standard Hall sensor electrical signals, pitch parameters, sound decay time, timbre type and extended timbre feature parameters, as well as the audio corresponding to standard Hall sensor electrical signals, pitch parameters, sound decay time, timbre type and extended timbre feature parameters;
[0106] A multimodal fusion generative network is adopted as the model architecture design, including an input layer, a fusion layer, and an output layer;
[0107] In the input layer, the standard Hall sensor electrical signal (time series data) is extracted with local time series features through 1D convolution, and a 512-dimensional feature vector is output. The pitch parameter, sound decay time, timbre type and extended timbre feature parameters are embedded into a 512-dimensional vector through a fully connected layer and concatenated with the Hall feature.
[0108] The fusion layer uses a Transformer encoder to process the spliced features and capture cross-modal correlations, such as the interaction between Hall signal waveforms and timbre features;
[0109] The output layer uses a U-Net structure to generate a Mel spectrogram, that is, to recover a 256×128 spectrogram from 512-dimensional features through transposed convolution.
[0110] The collected X sets of training data are divided into training set, validation set and test set according to a preset ratio;
[0111] The model is trained using the training set, the optimizer and learning rate are set, the model parameters are updated using the backpropagation algorithm, the loss function is minimized, and the model is validated using the validation set to achieve the best performance on the validation set.
[0112] The trained model is evaluated using a test set, and the mean squared error index of the model is calculated to assess its performance. Once the model's performance meets the evaluation criteria, the trained audio is merged into the model and deployed for application.
[0113] Specifically, through the design of a multimodal fusion architecture, the Hall signal is a time-series signal, requiring 1D convolution to extract local features, while the parameters are static features that need to be processed by an embedding layer. Furthermore, by selecting a Transformer encoder, cross-modal correlations can be effectively captured, such as "sawtooth Hall signal + high overtone features" corresponding to a sharp timbre, ensuring that the model accurately learns the mapping from input parameters to audio, and finally generates the merged audio that meets expectations.
[0114] In this embodiment, compared with the prior art which only outputs analog voltage signals through Hall sensors and directly converts them into audio signals after simple amplification and filtering, this application can perform a deeper structural analysis of Hall sensor signals. Through signal preprocessing and physical feature parameter extraction, it systematically extracts key quantities reflecting the physical state of the tympanic membrane, such as tympanic membrane tension parameters, damping coefficients, and impact positions, providing a real and accurate physical basis for sound generation.
[0115] Furthermore, by introducing a tympanic membrane stress sensor to obtain the initial tension baseline, and combining the tension decay value, damping coefficient, and impact position, a mapping relationship between pitch, sound decay time, and timbre type is established. In addition, with the timbre brightness and overtone enhancement factor extracted from the standard Hall sensor electrical signal, an audio merging model is constructed, which can achieve coordinated control of the pitch, dynamic changes, and timbre characteristics of the sound, thereby significantly improving the naturalness, expressiveness, and personalization of the drum sound, and meeting the application requirements of higher-performance intelligent electronic drums.
[0116] Example 2
[0117] Please see Figure 4 This invention provides a technical solution: a non-contact Hall effect drumhead vibration sensing and intelligent sound production control system, which is used to implement the aforementioned non-contact Hall effect drumhead vibration sensing and intelligent sound production control method. The system includes:
[0118] The raw data acquisition module collects the raw Hall sensor signals generated by the user striking the drumhead through a Hall sensor array inside the eardrum. It then performs signal preprocessing on these raw Hall sensor signals to obtain the standard Hall sensor signals. ;
[0119] The physical feature extraction module extracts standard Hall sensor electrical signals. Physical feature parameters are extracted to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position;
[0120] The first parameter processing module obtains the initial tympanic membrane tension parameter through the tympanic membrane stress sensor, compares and analyzes the first tympanic membrane tension parameter with the initial tympanic membrane tension parameter to obtain the tension decay value, and obtains the pitch parameter based on the pre-constructed mapping relationship between the tension decay value and the pitch.
[0121] The second parameter processing module pre-builds the mapping relationship between damping coefficient and sound decay time and the mapping relationship between impact position and timbre type, and obtains the corresponding sound decay time and timbre type based on the damping coefficient and impact position;
[0122] The extended feature extraction module is used for standard Hall sensor electrical signals. Extended timbre feature parameter extraction is performed to obtain timbre brightness and overtone enhancement factor;
[0123] The audio output module inputs the acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters into a pre-built audio merging model to obtain the audio output.
[0124] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A non-contact Hall effect drumhead vibration sensing and intelligent sound production control method, characterized in that, include: The raw Hall sensor signal generated by the user striking the drumhead is collected by the Hall sensor array inside the eardrum. The raw Hall sensor signal is preprocessed to obtain the standard Hall sensor signal. Physical feature parameters are extracted from the standard Hall sensor electrical signal to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position; The initial tympanic membrane tension parameter is obtained by using a tympanic membrane stress sensor. The first tympanic membrane tension parameter and the initial tympanic membrane tension parameter are compared and analyzed to obtain the tension decay value. Based on the pre-constructed mapping relationship between tension decay value and pitch, the pitch parameter is obtained. A pre-constructed mapping relationship between damping coefficient and sound decay time, and a mapping relationship between impact position and timbre type, are used to obtain the corresponding sound decay time and timbre type based on the damping coefficient and impact position. Extended timbre feature parameters are extracted from the standard Hall sensor electrical signal to obtain timbre brightness and overtone enhancement factor; The acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters are input into a pre-built audio merging model to obtain the audio output.
2. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Methods for obtaining the first tympanic membrane tension parameter include: The standard Hall sensor electrical signal is converted using a fast Fourier transform. Convert it into a frequency domain signal to obtain the intensity of each frequency component, where t is time; Obtain the magnitude of the Fast Fourier Transform (FFT) to get the vibration amplitude intensity of each frequency component, and find the frequency point with the maximum vibration amplitude intensity. ; Obtain the vibration radius and areal density of the tympanic membrane based on the frequency point. 、 The vibration radius and areal density of the tympanic membrane are used to calculate the first tympanic membrane tension parameter.
3. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Methods for obtaining the damping coefficient include: Extracting standard Hall sensor electrical signals The envelope of the vibration displacement decaying over time; The logarithmic decay rate is obtained based on the number of vibration cycles and the vibration amplitude, and the damping coefficient is calculated based on the logarithmic decay rate.
4. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Methods for obtaining the hit location include: Based on the center point of the drumhead, a coordinate system is constructed to obtain the position coordinates of multiple Hall sensor arrays installed inside the tympanic membrane, denoted as . ; Let x be the x-coordinate of the i-th Hall sensor. Let be the ordinate of the i-th Hall sensor; Obtain the time when each Hall sensor receives the vibration wave, select the Hall sensor that receives the vibration wave earliest and mark it as the reference sensor, obtain the reception time of the reference sensor, and record it as the reference time. Obtain the time difference between other Hall sensors and the reference time, denoted as . ;in, , The time when the i-th Hall sensor receives the vibration wave; Obtain the propagation speed of vibration waves on the tympanic membrane The difference between the actual distance from the impact position to the i-th Hall sensor and the distance from the impact position to the reference sensor is obtained. The spatial coordinates of the striking position are calculated using triangulation.
5. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, The method for constructing the striking position-timbre type mapping relationship is as follows: the drum surface is divided into G striking position regions, where G is a positive integer greater than 1. The striking signals of each striking position region are collected and the corresponding timbre features are extracted. The timbre features are classified and labeled with timbre tags. The timbre types include bright, soft, full, and hollow. Through statistical analysis, a mapping relationship is established between the striking position and the corresponding timbre tag to construct the striking position-timbre type mapping relationship.
6. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Methods for obtaining timbre brightness include: Standard Hall sensor electrical signal Perform p-level wavelet decomposition to obtain multiple high-frequency detail components. ; This represents the high-frequency detail components of the j-th layer of the i-th Hall sensor; Based on multiple high-frequency detail components Calculate the energy of each layer ; Based on the energy obtained from each layer The timbre brightness is calculated based on the preset energy weights for each layer.
7. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Methods for obtaining overtone enhancement factors include: Standard Hall sensor electrical signal Perform a Fast Fourier Transform to obtain the spectrum. , For pitch frequency; From the spectrum Find the main peak frequency The overtone enhancement factor is calculated and obtained.
8. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, Establish the mapping relationship between tension decay value and pitch, with the mapping function taking the form: In the formula, For pitch frequency, This is the tension decay value. For mapping functions; The mapping function was obtained by regression analysis of experimentally measured decay values and pitch correspondence data.
9. The non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to claim 1, characterized in that, The training method for the audio merging model includes: X sets of training data are collected in advance, where X is a positive integer greater than 0. The training data includes standard Hall sensor electrical signals, pitch parameters, sound decay time, timbre type and extended timbre feature parameters, and corresponding audio. A multimodal fusion generative network is adopted as the model architecture design, including an input layer, a fusion layer, and an output layer; The collected X sets of training data are divided into training set, validation set and test set according to a preset ratio; The model is trained using the training set, the optimizer and learning rate are set, the model parameters are updated using the backpropagation algorithm, the loss function is minimized, and the model is validated using the validation set to achieve the best performance on the validation set. The trained model is evaluated using a test set, and the mean squared error index of the model is calculated to assess its performance. Once the model's performance evaluation meets the standards, the trained audio merging model is deployed and applied.
10. A non-contact Hall effect drumhead vibration sensing and intelligent sound production control system, used to implement the non-contact Hall effect drumhead vibration sensing and intelligent sound production control method according to any one of claims 1-9, characterized in that, The system includes: The raw data acquisition module acquires the raw Hall sensor signals generated by the user striking the drum surface through the Hall sensor array inside the eardrum, and performs signal preprocessing on the raw Hall sensor signals to obtain standard Hall sensor signals. The physical feature extraction module extracts physical feature parameters from the standard Hall sensor electrical signal to obtain the first tympanic membrane tension parameter, damping coefficient, and impact position. The first parameter processing module obtains the initial tympanic membrane tension parameter through the tympanic membrane stress sensor, compares and analyzes the first tympanic membrane tension parameter with the initial tympanic membrane tension parameter to obtain the tension decay value, and obtains the pitch parameter based on the pre-constructed mapping relationship between the tension decay value and the pitch. The second parameter processing module pre-builds the mapping relationship between damping coefficient and sound decay time and the mapping relationship between impact position and timbre type, and obtains the corresponding sound decay time and timbre type based on the damping coefficient and impact position; An extended feature extraction module is used to extract extended timbre feature parameters from the standard Hall sensor electrical signal to obtain timbre brightness and overtone enhancement factor. The audio output module inputs the acquired standard Hall sensor electrical signal, pitch parameters, sound decay time, timbre type, and extended timbre feature parameters into a pre-built audio merging model to obtain the audio output.