A breath rhythm-based keyboard instrument light guiding device system

Abstract

The application discloses a keyboard instrument light guiding device system based on breathing rhythm, and belongs to the technical field of intelligent musical instruments and music auxiliary teaching, and comprises a breathing sensor, a central controller, a key light guiding module and a mobile terminal APP. The mobile terminal APP runs a breathing rhythm analysis algorithm and a light and breathing synchronization algorithm. The application synchronously matches the breathing rhythm of a performer in real time with the phrase structure of a musical score, and guides the performer to perform correct breathing by means of dynamic key light changes (such as gradual brightening to guide inhalation at the beginning of a phrase, gradual dimming to guide exhalation at the end of a phrase, and breathing synchronization water ripple effect when pressing a key). The application effectively solves the problem of lack of physiological feedback and breathing guidance in the prior art, and improves the fluency and music expressiveness of keyboard instrument performance.

Description

Technical Field

[0001] This invention belongs to the field of intelligent musical instruments and music-assisted teaching technology. Specifically, it relates to a keyboard instrument device and its control method that matches the learner's breathing rhythm with the musical phrase structure of the score and provides synchronous guidance through lighting. Background Technology

[0002] Existing learning aids for keyboard instruments (such as pianos and electronic keyboards) mostly employ key light guidance technology, using LED lights placed under the keys to indicate the notes being played, to help beginners get started quickly. However, these systems primarily focus on the accuracy of notes and the synchronization of rhythm, neglecting the performer's physiological state during performance, especially their breathing rhythm.

[0003] In keyboard instrument playing, especially for long or complex passages, proper breathing is crucial for maintaining phrasing integrity, controlling musical expression, and avoiding fatigue. Traditional teaching methods rely primarily on verbal instruction from the teacher and the performer's self-awareness, lacking objective, real-time physiological feedback and guidance. Beginners often experience shallow, stiff breathing due to tension or focusing solely on fingering, resulting in phrasing that lacks coherence and musicality.

[0004] While some smart pianos exist in the current technology (such as CN210837142U), their core remains photoelectric sensors detecting keys and LED lights indicating notes, without involving the collection and synchronous guidance of physiological parameters. Other patents involve the regulation of biorhythms (such as CN105592777A), but their applications are mainly focused on sleep management or relaxation training, unrelated to the specific needs of musical instrument playing and musical phrase structure. No keyboard instrument teaching scheme has yet been found that analyzes the learner's real-time breathing rhythm (especially through wrist sensors) with the musical phrase structure of the score and synchronizes it with a light guidance system (such as gradual brightening / dimming of lights).

[0005] Therefore, how to combine the performer's real-time breathing rhythm with the musical phrase structure of the score, and how to assist the performer in breathing synchronization training through intelligent and dynamic lighting guidance, is a problem that existing technologies urgently need to solve. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] This invention aims to overcome the shortcomings of existing lighting guidance systems that only focus on note values ​​and neglect musical breathing and phrasing. This invention provides a keyboard instrument lighting guidance system based on breathing rhythm. It collects the performer's breathing rhythm in real time, matches it with the phrasing structure of the score, and provides synchronized guidance through customized lighting effects (such as gradual brightening, dimming, and water ripples). This helps learners master correct musical breathing techniques, improving the fluency of their performance and musical expressiveness.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention provides a keyboard instrument lighting guidance device system based on breathing rhythm, which is built on a unified "terminal-central control layer-cloud" architecture.

[0010] The cloud platform 10 is responsible for storing music score databases (such as XML, MusicXML, and MIDI formats), user profiles, adaptive learning algorithm models, and breathing training tutorials. The cloud platform is also responsible for compiling standard music scores into executable instruction sequences for the terminal.

[0011] The central control layer 20, typically a mobile app, serves as the "brain" of the system. It downloads instructions from the cloud, processes user input 201 and sensor data, runs core algorithms 203 (such as respiratory rhythm analysis algorithms and light-breath synchronization algorithms), and sends control commands to the hardware terminal.

[0012] The terminal 30 is a smart musical instrument hardware, serving as the "limbs" of the system. It is responsible for receiving instructions from the APP, driving actuators such as lights, and collecting performance data (such as button signals and sensor data) and sending it back to the APP.

[0013] Its core technical solution lies in the collaborative work between the terminal and the central control layer:

[0014] Terminal 30 (intelligent musical instrument hardware), i.e., the hardware system:

[0015] Breathing sensor 301: Used to collect physiological signals caused by the performer's breathing in real time. For example, a piezoelectric thin film sensor is used, which is encapsulated in an adjustable elastic wristband and worn on the performer's wrist to collect pulse fluctuations at the wrist and chest rise and fall pressure changes caused by breathing. Its sampling rate is 100Hz.

[0016] Central Controller 302: Employs a microprocessor (such as an ARM Cortex-M4), integrating a Bluetooth module (for communication with the app) and an ADC (analog-to-digital converter) module. It connects to the respiratory sensor to receive and process the analog signals from the sensor, converting them into digital signals and enabling wireless communication with the mobile app.

[0017] Key Light Guide Module 303: Designed for 88-key pianos or electronic keyboards, it is installed under each key and includes LED lights (such as SMD 5050) and a light guide plate to evenly project light. It receives commands from the central controller and guides the light with adjustable color, brightness, and dynamic effects (such as water ripples).

[0018] Central Control Layer (Mobile App) 20: As the central control layer of the system, it runs the core algorithm 203 (i.e., software algorithm 40) on smartphone / tablet terminal devices, and connects to the central controller via wireless communication for:

[0019] Music score parsing module 401: Parses target music scores (such as MIDI and MusicXML files) and identifies the musical phrase structure (including phrase beginnings, phrase transitions, and phrase endings).

[0020] Respiratory rhythm analysis algorithm 403: The APP receives raw data from the respiratory sensor from the central controller. First, a digital filtering algorithm (such as Kalman filtering) is used to reduce noise in the data, the key being to remove interference signals such as pulse fluctuations at the wrist. Next, a peak detection algorithm is used to identify the inspiratory peak and expiratory trough values, thereby calculating the real-time respiratory rate (breaths / minute) and respiratory depth (pressure change amplitude), with a processing delay ≤100ms.

[0021] Lighting and Breathing Synchronization Algorithm 402: This is the core algorithm of this invention. Based on the musical phrase structure and breathing rhythm analysis results (i.e., the user's current breathing state), lighting control rules are generated, and lighting control commands are sent to the central controller.

[0022] The aforementioned lighting control rules specifically include:

[0023] Phrase initiation guidance (inhalation): When the score enters the beginning of a new phrase, the APP instruction light slowly transitions from dark to bright, and its duration matches the learner's calibrated inhalation time (e.g., 2-3 seconds), thus prompting the learner to take a deep breath.

[0024] End-of-phrase guidance (exhalation): When the musical phrase ends, the APP instruction light slowly dims and goes out, with the duration matching the learner's calibrated exhalation time (e.g., 3-4 seconds), thus prompting the learner to exhale smoothly.

[0025] Interactive performance: When the learner presses a key, a specific lighting effect (such as water ripples) is triggered. Centered on the pressed key, the light spreads outwards to the surrounding keys, and the speed of diffusion is synchronized with the performer's breathing rhythm. This not only provides visual feedback but also organically combines the playing action with physiological rhythm.

[0026] In addition, the present invention includes a key personalization step:

[0027] Breathing Calibration Function: Upon first use, the app offers a "Breathing Calibration" function. Learners need to follow the prompts to breathe naturally (e.g., for 30 seconds). The app collects breathing data during this period, analyzes and determines the learner's baseline breathing rate and depth, and sets personalized light synchronization parameters accordingly.

[0028] (III) Beneficial Effects

[0029] Compared with the prior art, the present invention has the following significant innovations and beneficial effects:

[0030] This invention innovatively integrates breathing and musical phrases: For the first time, the light guidance system elevates itself from a simple "note prompt" to a level of "musical expression." It uses algorithms to analyze musical phrase structure and correlates it with physiological breathing rhythms, solving the problem that existing technologies cannot guide musical breathing.

[0031] A precise respiratory rhythm monitoring solution: This invention creatively employs a wristband-type piezoelectric thin-film sensor to collect respiratory signals (wrist pressure changes caused by chest rise and fall). Compared to chestband sensors, it is more convenient to wear; at the same time, through a specific Kalman filter algorithm, pulse interference is successfully filtered out from the wrist signal, achieving effective extraction of the main respiratory rhythm.

[0032] Personalized and Real-Time Closed-Loop Feedback: Through the "Breathing Calibration" function, the system establishes a personalized breathing model for each user. During practice, the system not only provides leading (guiding) lights but also monitors breathing rate in real time. When the learner's breathing rate deviates from the baseline value (e.g., ±3 breaths / minute), the app will remind them through a prompt sound, forming a complete "guidance-monitoring-feedback" closed loop.

[0033] Intuitive and rich lighting guidance logic: The lighting effects (gradual brightening / dimming) designed in this invention are matched with the start, end, and duration of breathing movements (inhalation / exhalation), conforming to human intuition and providing significant guidance. Simultaneously, the "ripple" button effect synchronized with the breathing rhythm enhances the immersive experience of practice.

[0034] Quantitative practice feedback: After practice, the system can generate a comprehensive report including indicators such as breathing stability score, key accuracy, and fluency of musical phrase completion, and supports data export (such as Excel format), making teaching and review based on evidence. Attached Figure Description

[0035] Figure 1 : A schematic diagram of the system architecture of this invention.

[0036] Figure 2 The software algorithm design and operation flowchart of this invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments of the present invention will be described in detail below.

[0038] Example 1: System Hardware Configuration

[0039] like Figure 1 As shown, the system in this embodiment includes a breathing sensor 301, a central controller 302, and a piano key light guide module 303.

[0040] 1. Breathing sensor 301:

[0041] Sensor type: Piezoelectric thin-film sensor;

[0042] Physical parameters: thickness 0.1mm, sensitivity 0.5V / kPa;

[0043] Packaging method: Encapsulated in an adjustable elastic wristband;

[0044] Wearing location: on the learner's wrist;

[0045] Acquisition principle: Real-time acquisition of pulse fluctuations at the wrist and pressure changes transmitted by chest rise and fall caused by respiration;

[0046] Performance specifications: Sampling rate 100Hz, data accuracy ≤±2%.

[0047] 2. Central Controller 302:

[0048] Processor: 32-bit ARM Cortex-M4 microprocessor (e.g., STM32F407);

[0049] Communication module: Integrated Bluetooth 5.0 module for wireless communication with the central control layer APP;

[0050] Signal conversion: An integrated 12-bit ADC (analog-to-digital converter) module is used to convert the analog pressure signal collected by the respiratory sensor into a digital signal with a conversion accuracy of ≤1LSB;

[0051] Power supply and dimensions: Operating voltage 5V (powered via USB interface or 12V power adapter), dimensions 50mm×60mm×15mm, fixed to the bottom of the keyboard instrument with screws.

[0052] 3. Key light guide module 303:

[0053] Compatible with 88-key standard pianos or electronic keyboards;

[0054] One SMD 5050 surface-mount LED is installed under each key;

[0055] The LED beads project light evenly onto the piano key surface through the light guide plate, ensuring clear visual guidance;

[0056] The color selection supports switching between at least eight basic colors, including red, green, blue, and yellow.

[0057] The brightness adjustment range is 50–200 cd / m²;

[0058] The LEDs are connected to the central controller via ribbon cables hidden in the gaps between the piano keys, without affecting the feel of pressing the keys.

[0059] Example 2: System Software Algorithm

[0060] like Figure 2 As shown, the core software algorithms of the system described in this embodiment all run in the central control layer (mobile APP), including the respiratory rhythm analysis algorithm 403 and the light and breathing synchronization algorithm 402, as well as cloud and APP functions.

[0061] Respiratory rhythm analysis algorithm 403:

[0062] Noise Reduction: After receiving the raw 100Hz sensor data S210, the APP first processes it using the embedded digital filtering algorithm (Kalman filtering in this embodiment) S220. The key to this step is to effectively filter out high-frequency pulse fluctuation signals and other noise interference, and extract low-frequency, smooth respiratory pressure change curves;

[0063] Feature extraction: The peak inspiratory pressure (maximum pressure / maximum change) and trough in expiration pressure (minimum pressure / minimum change) of the processed curve are identified by the Peak Detection S230 algorithm.

[0064] Parameter calculation S240: Calculate respiratory rate (breaths / minute) based on the time of peak and trough occurrences; calculate respiratory depth (pressure change amplitude) based on the pressure difference between peak and trough.

[0065] Performance: The algorithm has a processing latency of ≤100ms, ensuring the real-time performance of respiratory analysis.

[0066] Algorithm 402 for synchronizing light and breathing:

[0067] Phrasing Analysis 401: After the APP loads the sheet music (such as MusicXML), it automatically parses its phrasing structure (S160) and identifies the start point, transition point and end point of each phrasing;

[0068] Rule Formulation: Based on the musical phrase structure S160, and combined with the personalized breathing time obtained in the "breathing calibration" step S120 (see Example 3) (e.g., 2-3 seconds of inhalation and 3-4 seconds of exhalation), the algorithm generates the lighting control command S170:

[0069] Command example:

[0070] IF (Phrase Start) THEN (Light Instruction = Gradually Brighten, Duration = Calibrate Inhalation Time) S310;

[0071] IF (end of musical phrase) THEN (light instruction = dim, duration = calibrated exhalation time) S320;

[0072] IF (button trigger) THEN (light command = water ripple, diffusion speed = f (breathing rhythm) S330.

[0073] Cloud and App Functions:

[0074] Cloud database:

[0075] Contents: Stores sheet music for various keyboard instruments (covering classical, pop, jazz, and other styles) and breathing training tutorials.

[0076] Functions: Supports searching by track name, composer, and difficulty level, and is responsible for compiling standard scores (such as XML, MusicXML, MIDI) into a sequence of instructions that can be executed by the central controller.

[0077] APP interaction:

[0078] Mode Switching: Offers switching between "Practice Mode" and "Performance Mode". In Practice Mode, the light cues are more prominent, while in Performance Mode, the lighting effects are softer.

[0079] Breathing calibration: As described in Example 2, used to determine personalized breathing parameters;

[0080] Parameter fine-tuning: Learners can fine-tune the brightness of the lights and the intensity of the water ripple effect according to their own needs;

[0081] Data feedback: After practice, a comprehensive report is generated, including indicators such as breathing stability score (based on the range of breathing frequency fluctuations), key accuracy, and fluency of musical phrase completion, and data export is supported.

[0082] Example 3: Overall System Operation Flow

[0083] Just like Figure 2 As shown, this embodiment describes the complete process of a learner using the present invention:

[0084] 1. Device Wearing and Connection:

[0085] Learners wear the breathing sensor wristband on their wrists, adjusting the tightness to a comfortable level and ensuring close contact between the sensor and the skin.

[0086] The central controller is powered via USB, the indicator light flashes, and it enters Bluetooth pairing mode.

[0087] Learners open the mobile app, select "Keyboard Breathing Guidance Device" in "Device Management," and complete the Bluetooth pairing with the S110.

[0088] After successful pairing, the app's main interface displays the current breathing status, such as: "Respiratory rate: 16 breaths / minute, Breathing depth: moderate".

[0089] 2. Breathing Calibration and Parameter Settings:

[0090] For first-time use or when needed, learners click the "Breathing Calibration S120" button on the app.

[0091] Follow the app's prompts to take 30 seconds of natural breathing. The app collects and analyzes data in real time to determine the learner's baseline breathing rate and depth S240.

[0092] Learners select a target sheet music S130 in the APP (or download S140 from the cloud database).

[0093] The APP automatically loads the sheet music (S150), analyzes the musical phrases (S160), and generates a personalized lighting and breathing synchronization control scheme (S170) based on the newly calibrated breathing parameters.

[0094] Learners can then fine-tune the brightness of the lights or the intensity of the water ripple effect.

[0095] 3. Practice and guidance phase:

[0096] Learners click "Start Practice".

[0097] Musical phrase begins (inhalation) S310: The APP sends a command, the central controller drives the piano key light guide module, and the lights (e.g., the area lights related to the musical phrase) slowly illuminate, guiding the learner to take a deep breath. The time of the light gradually brightening matches the calibrated inhalation time.

[0098] Performance Tips: After inhalation, the key light corresponding to the first note of the musical phrase will turn constantly lit, prompting the learner to press the key.

[0099] Interactive performance: When a learner presses a key, the S330's water ripple lighting effect is triggered. Simultaneously, the app receives the key press signal, confirms it's correct, and then switches to the next key's lighting according to the musical phrase and the track's BPM.

[0100] End of musical phrase (exhale) S320: When the musical phrase ends, the lights (e.g., the light on the last note of the phrase or the area light) slowly turn off, guiding the learner to exhale smoothly.

[0101] Real-time feedback: Throughout the process, if the app detects that the learner's breathing rate deviates from the baseline value (e.g., ±3 breaths / minute), it will issue a prompt sound on the phone to remind the learner to adjust their breathing.

[0102] 4. Practice data feedback phase:

[0103] After the exercise is completed, the app automatically generates a comprehensive report.

[0104] The report includes indicators such as: breathing stability score (calculated based on the range of breathing frequency fluctuations throughout the practice), key press accuracy, and fluency of musical phrase completion.

[0105] It supports exporting breathing data and light guidance data from the report to Excel format, which facilitates in-depth analysis by learners or teachers to optimize subsequent practice plans.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. The core of the present invention lies in establishing a synchronous mapping relationship between real-time breathing rhythm, musical score and phrase structure, and lighting guidance effects, thereby achieving effective guidance of musical breathing in keyboard instrument teaching.

Claims

1. A breath rhythm based keyboard instrument light guiding device system, characterized in that, include: Breathing sensor: used to collect physiological signals caused by the performer's breathing in real time; Central controller: connected to the breathing sensor, used to receive and process the analog signals from the breathing sensor and convert them into digital signals; Key light guide module: Installed below the keys of a keyboard instrument, used to receive instructions from the central controller and perform light guidance; Mobile App: As the central control layer, it connects to the central controller via wireless communication. The mobile app runs a breathing rhythm analysis algorithm and a light and breathing synchronization algorithm. The mobile app also includes a music score parsing module, used to parse the phrase structure of the target music score; The mobile app processes the digital signal from the breathing sensor using the breathing rhythm analysis algorithm to calculate the performer's real-time breathing rhythm. Combined with the musical phrase structure analyzed by the score parsing module, it generates lighting control rules using the lighting and breathing synchronization algorithm and sends instructions to the central controller to drive the key light guide module to perform dynamic lighting guidance synchronized with the breathing rhythm.

2. The system of claim 1, wherein, The breathing sensor is a piezoelectric thin-film sensor, encapsulated in an adjustable elastic wristband and worn on the performer's wrist. It is used to collect the pressure changes at the wrist caused by the rise and fall of the chest caused by breathing.

3. The system of claim 1, wherein, The respiratory rhythm analysis algorithm includes: A digital filtering algorithm is used to denoise the raw data from the respiratory sensor in order to remove interference signals such as pulse fluctuations at the wrist. Peak detection algorithms are used to identify the peak inspiratory and trough expiratory values ​​in the processed data in order to calculate the real-time respiratory rate and respiratory depth.

4. The system of claim 1, wherein, The algorithm for synchronizing light and breathing includes: At the beginning of the musical phrase, the brightness of the light guide module of the piano keys is controlled to slowly transition from the preset minimum brightness to the maximum brightness, and the duration of the transition is matched with the inhalation time calibrated by the performer. At the end of the musical phrase, the brightness of the light guide module of the piano keys is controlled to slowly transition from the highest brightness to the lowest brightness, and the duration of the transition is matched with the exhalation time calibrated by the performer.

5. The system of claim 1 or 4, wherein, The lighting control rules generated by the lighting and breathing synchronization algorithm also include: When the player presses any key, a water ripple lighting effect is triggered, with the light spreading outwards from the pressed key to the surrounding keys. The diffusion speed of the water ripple lighting effect is inversely proportional to the real-time breathing frequency.

6. The system of claim 1, wherein, The mobile app also includes a breathing calibration function, which is used to collect the performer's natural breathing data within a preset time period to determine a personalized baseline breathing frequency and baseline breathing depth. The lighting and breathing synchronization algorithm generates lighting control rules based on the baseline breathing frequency and baseline breathing depth.

7. The system of claim 1, wherein, The system is built on a "terminal-central control layer-cloud" architecture: The terminal includes the breathing sensor, central controller, and piano key light guide module; The central control layer is the mobile APP; The cloud-based system is responsible for storing sheet music databases, user profiles, and breathing training tutorials.

8. A method for guiding the lighting of a keyboard instrument based on respiratory rhythm, characterized in that, Includes the following steps: Signal acquisition: Real-time acquisition of the performer's respiratory physiological signals via a respiratory sensor; Rhythm Analysis: The respiratory rhythm analysis algorithm running on the mobile APP is used to reduce noise and extract features from the physiological signals in order to calculate the performer's real-time respiratory rhythm. Music score analysis: The music score analysis module of the mobile APP is used to analyze the musical phrase structure of the target music score; Generation rules: The lighting and breathing synchronization algorithm running on the mobile APP synchronizes and matches the real-time breathing rhythm with the musical phrase structure to generate lighting control rules; Execution guidance: The mobile APP converts the lighting control rules into instructions and sends them to the central controller, which then drives the piano key light guide module to execute dynamic lighting guidance.

9. The method according to claim 1 or 8, characterized in that, The specific steps for executing the boot process include: At the beginning of the musical phrase, the brightness of the light is slowly increased, and the duration of the increase is matched with the performer's calibrated inhalation time to guide the inhalation; At the end of the musical phrase, the brightness of the lights is slowly reduced, and the duration of the reduction is matched with the exhalation time calibrated by the performer to guide the exhalation. When the performer presses a key, a water ripple lighting effect is triggered, and the diffusion speed of the water ripple lighting effect is inversely proportional to the real-time breathing frequency.

10. The system of claim 1, wherein, The key light guide module is suitable for 88-key keyboard instruments. Each key has a surface-mount LED light installed below it, and the LED light projects light onto the key surface through a light guide plate.

Images