An electronic musical instrument with simulated plucking sensation and its interaction method

Abstract

This invention belongs to the field of electronic musical instruments, specifically an electronic musical instrument with simulated plucking tactile feedback and its interaction method. The electronic musical instrument includes: at least one simulated piano key assembly, comprising a key body and a base; one end of the key body is a fixed pivot end, and the other end is a suspended plucking end for the user to pluck; a mechanical tactile structure is disposed between the key body and the base to provide nonlinear damping and rebound feel during plucking, simulating the physical tactile feedback of a traditional plucked instrument; and a sensing and detection module is used to detect the plucking action, displacement, or speed of the key body and output corresponding electrical signals. This invention provides an intelligent music interaction system that retains the unique plucking feel of a traditional thumb piano, solves tuning and dead tone problems through digital means, and lowers the learning threshold by combining visual game guidance.

Description

Technical Field

[0001] This invention relates to the field of electronic musical instrument technology, and in particular to an electronic musical instrument (electronic thumb piano) with simulated plucking touch and its interaction method. Background Technology

[0002] The kalimba, as a therapeutic instrument, has become increasingly popular among young people in recent years. However, as an acoustic instrument, the traditional kalimba has significant physical limitations. Furthermore, most existing electronic music game devices use touchscreens or push-button controls, lacking the tactile feel of playing a real instrument.

[0003] Problems with traditional acoustic thumb pianos: Maintenance difficulties: It relies on the vibration of metal springs to produce sound, often goes out of tune, and requires repeated tuning with a tuning hammer.

[0004] Acoustic defects: The high-pitched keys (end keys) often produce "dead sounds" (muffled and silent) due to insufficient resonance, resulting in poor tonal consistency.

[0005] Unable to mute: The physical sound cannot be heard through headphones, and nighttime practice may disturb neighbors.

[0006] Problems with existing electronic musical instruments / gaming devices: Lack of tactile feedback: Most are keyboard-pressable or screen-touch tactile, lacking the unique "plucking" damping and rebound feedback of the thumper piano.

[0007] Too high a barrier to entry: Traditional electronic musical instruments still require learning music theory and sight-reading, have weak entertainment value, and cause a strong sense of frustration for beginners.

[0008] Therefore, we propose an electronic musical instrument with simulated plucking tactile feedback and its interaction method to solve the above problems. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing an intelligent music interaction system that retains the unique plucking feel of the traditional thumb piano, solves tuning and dead tone problems through digital means, and lowers the learning threshold by combining visual game guidance.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a digital timbre triggering and music interaction system based on the thumb piano playing structure, which mainly includes the following modules: 1. Hardware Architecture The plucked input module contains several simulated piano keys arranged in a fan shape or linearly. One end of each key is fixed, while the other end is a suspended plucked end for the user to pluck with their thumb.

[0011] Mechanical tactile structure module (key): Used to simulate the "elastic deformation" and "damped rebound" of the metal springs in a traditional thumb piano.

[0012] Preferred solution: Magnetic levitation structure. Magnets with opposite poles are placed below the keys at positions corresponding to the base 50. Magnetic repulsion provides nonlinear downward damping and rebound force, simulating the feeling of metal fatigue, without physical contact wear.

[0013] Alternative solution: Use a precision torsion spring or high-strength spring structure.

[0014] Silent design: It features a flexible buffer limit structure to absorb the sound of physical impacts, ensuring that "only music can be heard, and no other noises can be heard".

[0015] Sensing and detection module: Preferred solution: Hall effect. This method utilizes the magnetic field change caused by the displacement of piano keys, and the Hall sensor 33 detects the plucking action and its speed (force).

[0016] Alternative solutions: capacitive sensing, piezoelectric ceramic sensing, and light-blocking sensing.

[0017] Main control and output module: includes MCU main control chip, DSP audio processing unit, storage unit (storing high-fidelity MP3 samples), built-in speaker and screen driver circuit.

[0018] 2. Software Interaction Logic Non-physical sound generation mechanism: Plucking the keys does not directly produce musical sounds, but rather generates digital trigger signals. The main control module calls pre-stored digital timbres (such as piano, guitar, and synthesizer effects) based on the signals.

[0019] Multi-level velocity mapping: Based on the plucking speed / amplitude detected by the sensor, different levels of sampling are triggered (e.g., light plucking triggers a soft tone, heavy plucking triggers a bright tone).

[0020] Visualized waterfall flow guidance: The screen displays falling musical notes, which correspond one-to-one with the positions of the physical piano keys.

[0021] "Non-failure" error correction mechanism (Smart Correction): When in "game mode", if the user presses the wrong key at the wrong time, the system will not output noise, but will output preset harmonics (such as chord tones) or silence to ensure the music sounds coherent and reduce user frustration.

[0022] The tactile structure and interaction method of this invention are not only applicable to the Kalimba form, but also to any electronic musical instrument that is triggered by 'plucking', such as electronic guitar, electronic guqin, and electronic harp.

[0023] The beneficial effects of the electronic musical instrument with simulated plucking tactile sensation and its interaction method described in this invention are as follows: A perfect fusion of feel and digitalization: It solves the problem that electronic musical instruments feel like "toys," while using digital technology to completely solve the physical problems of "out of tune" and "dead notes" in traditional thumb pianos.

[0024] Zero learning curve: Combining waterfall-style game guidance and intelligent error correction algorithms, users can play complex pieces without music theory knowledge, relying solely on intuition and reaction.

[0025] Rich emotional expression: Based on the high-precision force detection of Hall / piezoelectric sensors, the electronic timbre can also express the musical emotions of strong and weak fluctuations.

[0026] Good structural mass production: Compared with traditional thumb pianos that require manual polishing and tuning, the modular electronic structure of this solution has high consistency and is suitable for large-scale industrial manufacturing. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the key cross-sectional structure of an electronic musical instrument with simulated plucking tactile sensation and its interaction method proposed in this invention. Figure 2 This is a system structure block diagram (plucking input / mechanical touch / sensor detection / main control and output) of an electronic musical instrument with simulated plucking tactile sensation and its interaction method proposed in this invention.

[0028] In the figure: 10, piano key body; 20, shaft end; 31, first permanent magnet; 32, second permanent magnet; 33, Hall sensor; 40, PCB circuit board; 50, base. Detailed Implementation

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0030] Example 1: Hardware Structure Based on Hall Magnetic Induction like Figure 1 , Figure 2 As shown, this embodiment includes: (1) Key assembly: The keys are made of PC material with a metal-like shape; one end of the key is a pivot end 20 (fixed end), and the other end is a plucking end (suspended end), which is used by the user to pluck with their thumb, forming a "pluck-rebound" feel similar to the metal spring of a traditional thumb piano. The keys can be arranged in a fan shape or a linear array, corresponding to different scales or different function keys.

[0031] (2) Magnetic repulsion rebound structure: A first permanent magnet 31 is embedded at the bottom of the key, and a second permanent magnet 32 is embedded at the corresponding position of the base 50. The two magnets are opposite with the same poles (N-N or S-S). In the initial state, the magnetic repulsion makes the key in a pre-loaded rebound state (it can be slightly lifted or close to the limiting position); when the user toggles the key, the relative distance between the first permanent magnet 31 and the second permanent magnet 32 changes, and the magnetic repulsion changes non-linearly with displacement, so as to provide a "first light then heavy / first damping then rebound" touch feeling during the toggling stroke, simulating the damping and rebound characteristics during the process of toggling a metal shrapnel. A flexible buffer limiting structure (such as a silica gel pad, an elastic limiting column, a buffer sheet, etc.) can be arranged in the base 50 to absorb the impact sound and limit the maximum stroke, realizing silent toggling and stroke consistency control.

[0032] (3) Hall detection module: A linear Hall sensor 33 is arranged on the PCB circuit board 40 of the base 50, and its position can be directly below or beside / diagonally below the second permanent magnet 32 to obtain a monotonous or approximately monotonous magnetic field change curve within the effective stroke, which is convenient for threshold determination and speed feature extraction. The Hall sensor 33 outputs an analog voltage signal, and the MCU samples the voltage signal through an internal or external ADC (for example, both 10~16bit resolutions are available), and obtains a discrete sequence V[k] (k is the sampling point number) at a sampling rate not lower than 1kHz. When the assembly direction causes the signal change direction to be opposite, equivalent under-threshold triggering or signal inversion processing can be adopted, which does not affect the implementation of this embodiment.

[0033] (4) Process of toggling event recognition and force (Velocity) calculation: To achieve reproducible force calculation, this embodiment gives a preferred calculation process (it is also allowed to use equivalent methods such as slope method / time method / energy method / machine learning regression to achieve the same effect): A. Static baseline calibration: In the state of no toggling, each key channel is continuously sampled N0 times (for example, N0 = 64 or 128), and the average value is taken to obtain the static baseline V0, which is used to offset the device offset and temperature drift: V0 = mean(V[k]). The real-time signal is denoted as . The V0 can be updated when powering on, during the idle period or when triggered regularly.

[0034] B. Filtering and debouncing: Filter to obtain , and the filtering can be first-order low-pass, moving average or median filtering, etc. (for example, the moving window W = 4~8). And set thresholds and hysteresis to avoid jitter: trigger threshold TH_on, release threshold TH_off (TH_off < TH_on), and continuous counting conditions can be set when necessary to suppress glitch triggering.

[0035] C. Detection of Note ON moment: When When a sample crosses a trigger threshold from a non-triggered state (e.g., crossing above TH_on, or an equivalent crossing below the threshold), the time corresponding to that sampling point is recorded as t_on, and the sample enters the "press / flick" state, generating a Note ON candidate event.

[0036] D. Velocity feature extraction (choose one or a combination): D1 slope method (preferred): Calculated within a short time window Tw after t_on. Maximum ascent slope Smax: ,in The sampling period is Tw; Tw can be represented by a fixed time or a fixed number of sampling points Nw. For example, Nw = 10~30 sampling points). The larger the Smax, the faster / greater the flicking.

[0037] D2 time method (optional): Set a higher amplitude threshold TH_hi (TH_hi > TH_on). Record the time interval from the first crossing of TH_on to the first crossing of TH_hi. . The smaller the value, the faster / stronger the flick.

[0038] E. Normalization and Mapping: Converting velocity characteristics into MIDI velocity values ​​(0-127) or equivalent velocity values ​​(Velocity). Using the slope method as an example: First, normalize: s=clamp((Smax-Smin) / (Smax_ref-Smin),0,1), where Smin is the lower limit of noise and Smax_ref is the reference value of the maximum effective slope; Smin and Smax_ref can be obtained through factory calibration or self-learning. For example, the slope distribution of the key can be statistically analyzed under multiple key strokes, and the high quantile value can be taken as Smax_ref and the noise environment statistical value can be taken as Smin.

[0039] Then perform curve mapping (optional): ( A value of 0.5-2.0 can be used to adjust the feel distribution between light and heavy flicks.

[0040] The final result is: Velocity = round(1 + s' * 126), and the upper and lower limits of Velocity are clipped to avoid extreme values.

[0041] If the time method is used, then: Then it is also mapped to 1~127; and It can also be obtained through calibration or self-learning.

[0042] F.Note OFF detection: When After the key falls back and crosses the release threshold (such as TH_off) for M consecutive times (e.g., M=4), it is determined that the key has returned to the release position, and Note OFF is output or the sound envelope ends. To avoid abnormal key jamming, a maximum duration T_hold can be set: if the release condition is not met after T_hold, Note OFF is forcibly output.

[0043] G. Multi-key consistency calibration: Different keys may have different amplitude / slope curves due to differences in magnet position, assembly tolerances, or structural design. Independent (V0, TH_on, TH_off, Smax_ref, or...) values ​​can be stored for each key. Parameters such as (etc.) can be used, or a unified model plus individual correction parameters can be adopted to ensure consistent dynamic mapping and performance experience.

[0044] Through the above steps, the calculation link of "pulling action - magnetic field change - voltage change - speed characteristics - force mapping" can be concretized, meeting the requirements of engineering feasibility and reproducibility.

[0045] Example 2: Music Interactive Control Method This embodiment provides a method for converting "piano key plucking signals" into "timbre triggering + gamified interactive output" (e.g., Figure 2 The software / hardware collaboration link shown includes the following steps: (1) Signal acquisition: For each key channel, the MCU samples the output of the Hall sensor at a frequency of not less than 1kHz to obtain the V[k] sequence; and performs static baseline calibration to obtain V0 when the device is powered on, during idle periods or timed triggers.

[0046] (2) Signal preprocessing: for Perform filtering (low-pass / moving average / median filtering, etc.) and debouncing, and set hysteresis thresholds TH_on and TH_off (add continuous counting conditions if necessary) to stably identify the toggle and bounce states.

[0047] (3) Event detection: When The Note ON event is triggered when the trigger threshold is crossed; The Note OFF event is triggered when the release threshold is exceeded and the continuous counting condition is met; and the maximum duration T_hold can be set to avoid abnormal key jamming.

[0048] (4) Force calculation: After Note ON is triggered, extract velocity features (maximum slope Smax or threshold arrival time) within the preset time window Tw. (etc.), and mapped to obtain Velocity∈[1,127] or equivalent power value (the calculation method can be found in Example 1, or the equivalent mapping method can be used).

[0049] (5) Instruction Encapsulation: The pitch NoteNumber is obtained by mapping the key number KeyID. The mapping can be implemented through a scale mapping table, a repertoire configuration table, or a modulation rule, supporting fixed modes, automatic modulation, or user-defined scales / modes. Subsequently, music event instructions are generated (compatible with MIDI or using custom protocols / other equivalent formats), such as: {Note ON,NoteNumber,Velocity,Timestamp} and {Note OFF,NoteNumber,Timestamp}.

[0050] (6) Pattern judgment and interaction logic: 6.1 Free Mode: Upon receiving Note ON, directly retrieve the corresponding sampling / synthesis parameters from the tone library, and output the selected tone (such as piano / guitar / synthesizer, etc.) by driving the volume, filtering, envelope or other tone parameters according to Velocity.

[0051] 6.2 Game Mode: Based on the timeline of the score and the current moment of the falling notes in the waterfall on the screen, determine whether the Timestamp falls into the judgment window (e.g., Perfect / Great / Good window; the window threshold is configurable, and the value is only an example).

[0052] a) If the timing matches the note: output the target note / chord timbre; display "Perfect / Great" on the screen; and drive the corresponding key lights to highlight or flash.

[0053] b) If the timing is mismatched or the wrong key is pressed: Execute "non-failure feedback" - do not output harsh wrong notes; calculate the consonant tone closest to the current key / chord interval relationship as the substitute tone through a preset scale mapping table or real-time harmony algorithm (e.g., a tone in the current chord, maintaining the sustained tone of the previous chord, outputting mute / weak volume, or aligning the output to the nearest legal note, etc., any or a combination thereof), and do not display penalty prompts to ensure the music flow continues and reduce frustration.

[0054] (7) Audio rendering: The DSP or audio synthesis module selects multi-level sampling or synthesis parameters according to NoteNumber and Velocity to complete mixing, envelope, dynamic range control and other processing to form an audio stream; this rendering process can be implemented in DSP, MCU, independent audio chip or external terminal.

[0055] (8) Output and feedback: The audio stream is output to the speaker or headphones via DAC / amplifier; the screen waterfall, score / prompt information and LED lighting effects are driven in sync to achieve a closed-loop interaction of "seeing - playing - hearing correctly / not being frustrated".

[0056] Figure 1 Supplementary explanation of working principle: Figure 1 The core of the structure shown lies in decoupling the "generation of tactile sensation" from "motion detection / digital sound generation": a) Tactile sensation generation: When the user plucks the key, the key rotates around the pivot end 20, causing the relative distance between the first permanent magnet 31 and the second permanent magnet 32 ​​to change; due to the repulsive force generated by the two magnets with the same poles, the resistance increases nonlinearly with the displacement during the plucking stroke, and the rebound force also changes with the displacement, thus more closely resembling the feel curve of a metal spring; when the user releases the key, it automatically rebounds to the initial position under the action of magnetic repulsion, realizing the simulation of rebound feel.

[0057] b) Silent limiting: A flexible buffer or limiting structure is set at the maximum travel position of the piano key to avoid mechanical noise caused by hard collision and to control the consistency of travel.

[0058] c) Motion detection: Hall sensor 33 is fixed on the PCB to detect changes in the magnetic field near the second permanent magnet 32 ​​and convert them into voltage signals; the MCU samples the voltage, identifies events and calculates force to obtain Note ON / OFF and Velocity.

[0059] d) Digital sound generation and interaction: The piano keys themselves do not rely on metal vibration to generate sound. Instead, the timbre is triggered and output by the DSP / sound source library based on NoteNumber and Velocity. At the same time, the screen waterfall and light feedback form a gamified guidance and training loop.

[0060] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An electronic musical instrument with simulated plucking touch, characterized in that, include: At least one simulated piano key assembly, the simulated piano key assembly including a key body (10) and a base (50), one end of the key body (10) being a fixed pivot end (20) and the other end being a suspended pivot end for the user to pluck; A mechanical tactile structure is provided between the key body (10) and the base (50) to provide nonlinear damping and rebound feel during plucking, simulating the physical tactile feel of plucking traditional plucked instruments; The sensing and detection module is used to detect the plucking action, displacement or speed of the piano key body (10) and output the corresponding electrical signal; The main control processing module is electrically connected to the sensing and detection module and is used to identify the dialing event and calculate the dialing force value based on the electrical signal. The interactive output module is electrically connected to the main control processing module and is used to output digital timbre, visual guidance and light feedback according to the toggle event and force value.

2. An electronic musical instrument with simulated plucking touch according to claim 1, characterized in that, The mechanical tactile structure is a magnetic levitation damping rebound structure, including: The first permanent magnet (31) is fixedly installed at the bottom of the piano key body (10); The second permanent magnet (32) is fixedly installed on the base (50) and is arranged opposite to the first permanent magnet (31) with the same pole. The magnetic repulsion between the first permanent magnet (31) and the second permanent magnet (32) changes nonlinearly with the displacement of the piano key being plucked, which is used to provide plucking damping and automatic rebound feel; The base (50) is also provided with a flexible buffer limiting structure, which is used to absorb physical impact sound and limit the maximum stroke of the piano keys.

3. An electronic musical instrument with simulated plucking touch according to claim 1, characterized in that, The sensing and detection module includes at least one of the following detection methods: Hall sensor detection: A linear Hall sensor (33) is set on the PCB circuit board (40) of the base (50). The Hall sensor (33) is located directly below, beside or diagonally below the second permanent magnet (32) and is used to detect the magnetic field change caused by the displacement of the piano key and output an analog voltage signal. Piezoelectric induction detection: A piezoelectric ceramic sheet is provided at the key body (10) or the shaft end (20) to detect the deformation stress generated by plucking and output an electrical signal; Photoelectric sensing detection: Detects the displacement of piano keys through a light interruptor or infrared diode, and outputs on / off or analog electrical signals; Mechanical contact detection: Detecting the plucking and rebounding action of piano keys through microswitches or elastic contacts.

4. An electronic musical instrument with simulated plucking touch according to claim 1, characterized in that, The main control processing module is configured to perform the following steps: Static baseline calibration: The static voltage baseline V0 is obtained by sampling under no-tampering conditions and is used to compensate for device bias and temperature drift; Signal preprocessing: The difference between the real-time sampled voltage signal and the baseline V0 is filtered and debouncing is performed, and the trigger threshold TH_on and release threshold TH_off are set; Twister event recognition: When the preprocessed signal crosses the trigger threshold, a Note ON event is output; when the signal falls back and crosses the release threshold, a Note OFF event is output. Velocity value calculation: Extract velocity features within a preset time window after Note ON is triggered, and map the velocity features to a MIDI velocity value of 0-127 or an equivalent velocity value (Velocity). Pitch mapping: Based on the key ID, the corresponding pitch NoteNumber is obtained, and music event commands compatible with MIDI or custom protocols are generated.

5. An electronic musical instrument with simulated plucking touch according to claim 1, characterized in that, The interactive output module includes a gamified interactive unit, which is configured to perform the following steps: Visual guidance: Displays a waterfall of falling notes on the display module, each corresponding to a specific piano key position; Non-failure feedback: When in game mode, if the user's pluck action does not match the timeline of the score or the wrong key is pressed, the system will not output a harsh wrong note, but will output a consonant tone, mute or reduce the volume that is closest to the current key / chord interval, to ensure the smooth listening experience of the music. Multiple modes: Supports free play mode and game-guided mode. In free play mode, the corresponding tone is triggered directly. In game mode, waterfall-style guidance and non-failure feedback are combined.

6. An electronic musical instrument with simulated plucking touch according to claim 1, characterized in that, The interactive output module also includes: Audio output unit: includes DAC decoding / amplifier circuitry and speaker / headphone interface, used to output audio signals according to NoteNumber and Velocity drive digital sound library; Lighting feedback unit: includes piano key backlight LEDs and ambient lights, used to highlight the corresponding piano keys or flash in rhythm with the music when Note ON is triggered; Storage and communication unit: Includes Flash / SD card storage unit and WiFi / Bluetooth wireless communication module, used to store digital sound library, sheet music data and synchronize data with external terminals.

7. An interactive method for an electronic musical instrument with simulated plucking tactile feedback, wherein the electronic musical instrument with simulated plucking tactile feedback is the electronic musical instrument with simulated plucking tactile feedback as described in any one of claims 1-6, characterized in that, Includes the following steps: S1. When the user plucks the suspended plucking end of the simulated piano key, the piano key rotates around the fixed rotating shaft end (20). The mechanical tactile structure provides nonlinear damping and rebound feel, simulating the physical tactile feel of plucking a traditional plucked instrument. S2, The sensor detection module detects the plucking action, displacement or speed of the piano keys and outputs the corresponding electrical signal to the main control processing module; S3, the main control processing module preprocesses the electrical signal, identifies events and calculates force, and generates Note ON / OFF events and force values ​​Velocity; S4. The main control processing module maps the piano key numbers to pitch NoteNumber and generates music event commands. S5. Execute the interaction logic according to the current mode: Free Mode: Directly triggers and outputs the corresponding digital tone based on NoteNumber and Velocity; Game Mode: Combining visual waterfall flow guidance, the target timbre is output for matching plucking actions, and non-failure feedback is performed for mismatched plucking actions, outputting a consonant tone or silence. S6. Output the audio signal to the speaker or headphones via DAC / amplifier, and simultaneously drive the visual interface and light feedback.

8. The interactive method for an electronic musical instrument with simulated plucking tactile feedback according to claim 7, characterized in that, In step S2, the sensing and detection module adopts Hall effect sensing detection, specifically including: Linear Hall sensor (33) detects the change in magnetic field caused by the displacement of piano keys and outputs an analog voltage signal; The main control processing module samples the voltage signal at a sampling rate of not less than 1 kHz to obtain a discrete voltage sequence V[k]. Calculate the static baseline V0 = mean(V[k]) and obtain the real-time difference signal. ; right Filtering and debouncing are performed to obtain the preprocessed signal. .

9. The interactive method for an electronic musical instrument with simulated plucking tactile feedback according to claim 7, characterized in that, In step S3, the force value is calculated using either the slope method or the time method: Slope method: Calculate within a preset time window Tw after Note ON is triggered. The maximum rising slope Smax is normalized and mapped to the Velocity value. Time-based method: Record the time interval from when the signal first crosses the trigger threshold TH_on to when it first crosses a higher amplitude threshold TH_hi. ,Will Mapped to Velocity values.

10. The interactive method for an electronic musical instrument with simulated plucking tactile feedback according to claim 7, characterized in that, In step S5, the non-failure feedback includes at least one of the following methods: Map the incorrect pluck to the nearest consonant in the current key / chord; Maintain the sustained tone of the previous chord or output silence; Align the incorrect pitch to a valid note in the current scale; No penalty prompts are displayed; only volume or light feedback is reduced.

Images