Keyboard instrument intelligent multi-mode interaction music score recognition learning system and use method thereof
The intelligent multimodal interactive music reading learning system for keyboard instruments combines visual, auditory, and tactile modalities to solve the problems of memory difficulties and low efficiency in traditional piano music reading learning, and achieves an efficient and low-cost learning experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-20
- Publication Date
- 2026-03-13
AI Technical Summary
Traditional piano sight-reading methods suffer from problems such as difficulty in memorization, low learning efficiency, high cost, and limited learning environment, and cannot learn from multiple modalities simultaneously.
Design an intelligent multimodal interactive music reading and learning system for keyboard instruments. Combining visual, auditory, and tactile modalities, the system achieves multimodal synchronous learning through video display, audio modules, key light indicators, finger light indicators, and foot pedal control units, providing a multi-dimensional integrated learning environment.
It improves learning efficiency, reduces learning costs, enhances the flexibility and fun of learning, reduces reliance on live teachers, ensures the continuity and smoothness of the learning process, and enables users to quickly master notes and fingering.
Smart Images
Figure CN121661893A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of keyboard instrument learning technology, and to an intelligent multimodal interactive music reading learning system for keyboard instruments and its usage method. In particular, it relates to a piano music reading learning system based on multimodal interaction, which allows users to interact with the system through different modal channels (including visual, auditory and tactile) to achieve information transmission and task completion in a more efficient and natural way, thereby assisting learners in learning and improving their piano music reading ability. Background Technology
[0002] The main types of sheet music for piano are staff notation and numbered musical notation, used to record and disseminate music. Sheet music consists of a large number of musical notes and other music theory symbols. The advanced ability achieved by learning sheet music is called "sight-reading ability," which allows pianists to closely integrate the elements of musical notes, music theory symbols, corresponding piano keys, fingering, and sound effects, so that when they see the notes on the sheet music, their fingers can press the corresponding keys to produce the desired sound.
[0003] Traditional methods of learning piano music reading have several drawbacks that hinder memorization: 1) Students first learn to recognize different notes (staff symbols or numbered notes in simplified notation), then learn different variations of the same note's duration (e.g., the quarter note and the sixteenth note have different appearances), and learn specific music theory symbols in the score. Students must memorize the images of different symbols and their positional relationships with the five lines and blank spaces, as well as the correspondence between note positions and sound names. However, the recognition of graphic symbols cannot be closely linked to the actual sounds and piano keys; 2) In the practical stage, students need to translate the musical notes onto the piano keys. Because they only understand and recognize the notes without thoroughly memorizing all the note images and variations, they often resort to counting the squares. Relying on memorized principles and key note positions to deduce and identify other notes is time-consuming and laborious when faced with complex sheet music, making it difficult for students to quickly identify a large number of notes; 3) Traditional learning methods usually only identify individual notes, lacking training in recognizing overall units such as image blocks or note strings composed of several notes on the score, making it difficult for students to form a holistic understanding of the music and quick response; 4) In the traditional learning process, looking at the score and determining notes, keys, and fingering requires students to frequently move and switch between the score and the key area, consuming a lot of time and energy; 5) Traditional piano teaching relies on live teachers explaining music theory, which is costly and requires students to go to a special location to learn, which is quite restrictive in terms of time and space.
[0004] Therefore, developing a system that can assist users in learning to read piano music through multiple modalities, including vision, hearing, and touch, is of great practical significance. Summary of the Invention
[0005] Due to the aforementioned deficiencies in existing technologies, this invention provides a system that can assist users in completing piano music reading learning from multiple modalities, including visual, auditory, and tactile senses. Specifically, it is an intelligent multimodal interactive music reading learning system and method for keyboard instruments, which overcomes the shortcomings of current music reading learning methods, such as high cost, inability to learn from multiple modal channels simultaneously, and low learning efficiency.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A smart multimodal interactive music reading and learning system for keyboard musical instruments includes a central control module, a video display module, an audio module, a key light indicator module, a finger light indicator module, and a foot pedal control unit.
[0008] The central control module is connected to the video display module, audio module, piano key light indicator module, finger light indicator module and foot pedal control unit respectively.
[0009] The video display module is used to display musical scores (dynamic staff notation or simplified notation videos). Through the indication of the dynamic cursor, students can see the freeze-frame images of the notes that need to be learned in the score in real time and intuitively, forming real-time visual stimulation. A tablet can be used as the video display module, and even a tablet can be used as an input device to set the central control module.
[0010] The audio module is used to play audio;
[0011] The key light indicator module is used to indicate the key on the keyboard instrument corresponding to the current note. A row of key lights is set on the piano, which can indicate the key corresponding to the note according to the program. When the student is learning, the key lights will light up to accurately indicate the position of the key that the student should press.
[0012] The finger light indicator module is used to indicate the finger that presses the current note. Each finger is equipped with a light, and the program instructs the student to use which finger when playing the key. This finger light design can clearly indicate the finger movement, improve learning efficiency, and cultivate fingering habits.
[0013] The foot pedal control unit is used to send control signals to the central control module;
[0014] The central control module is used to acquire control signals from the foot pedal control unit, process the control signals to obtain a control scheme, and then send relevant control signals to the video display module, audio module, key light indicator module, and finger light indicator module according to the control scheme. Specifically, the control scheme is to control the key light indicator module and finger light indicator module according to a preset scheme corresponding to the current moment of the score, and to recall the off-screen audio information corresponding to the preset scheme and play it through the video display module and audio module. The off-screen audio information refers to the interpretation of the notes, phrases, or entire pages of the score in the form of video and / or voice.
[0015] The aforementioned intelligent multimodal interactive music reading and learning system for keyboard instruments can not only be used as an accessory for existing keyboard instruments, but also can be directly integrated with keyboard instruments, that is, the system can be directly installed on intelligent keyboard instruments as a component of them.
[0016] The aforementioned key light indicator module and finger light indicator module have been disclosed in the inventor's prior application "Intelligent Keyboard Musical Instrument Practice System" (application number: 2022101814853).
[0017] The control signals emitted by the aforementioned foot pedal control unit are not simply pause / play. The foot pedal control unit controls the teaching units corresponding to the sheet music. These teaching units are divided based on different parts of the sheet music, not simply on time points; they can be notes or musical phrases. The control of the foot pedal control unit is "synchronously linked" with the sheet music display, indicator lights, and video explanations.
[0018] The intelligent multimodal interactive music reading and learning system for keyboard instruments of this invention features a reasonable overall design. Compared with existing technologies, it also incorporates an audiovisual component. Through the cooperation of the audiovisual unit with the key light indicator module and the finger light indicator module, it can achieve multimodal interactive synchronous learning, allowing for interaction between piano sounds and auditory perception, and memorizing the sound effects corresponding to notes. Through the coordinated work of multiple human organs, a three-dimensional perception and memorization of notes is formed. The memorization process is direct and efficient. The system provides users with a multimodal information synchronous learning environment. The video / audio background sound, key light indicator module, and finger light indicator module can be triggered by the foot pedal control unit, creating a multi-dimensional integrated learning environment. This system allows users to "see, hear, press the keys under the light, and understand simultaneously." Compared to the traditional keyboard instrument learning process, which separates each learning step (reading the sheet music + teacher explanation + fingering practice), this not only lowers the learning threshold and cost for users but also significantly improves their learning efficiency. Using this system, users can conveniently and efficiently control the entire learning process via a foot pedal control unit (allowing for pause, short loop, or further control of the sheet music playback), recall the current teaching unit as needed, and control that unit, enabling "paused, replayable, and repeatable teaching unit practice," in conjunction with off-screen audio information (audiovisual). The unit demonstration further enhances learning effectiveness, enabling users to interact visually with musical note images, memorize note graphics, interact auditorily with off-screen sounds, understand symbol meanings, interact visually with piano key lights, memorize the corresponding key positions for notes, interact with keys using finger movements, and memorize spatial positions through haptic feedback. Users can repeatedly practice individual notes / chords until mastered. Compared to traditional one-time instruction, this allows for targeted practice and high learning efficiency. The system eliminates the need for teacher intervention, significantly reducing labor costs. Using a foot pedal control unit as the controller, it does not interfere with hand movements during piano playing, unlike traditional manual operation, enabling instant control upon pedaling. The system (which retrieves the current control scheme) ensures the continuity and smoothness of the learning process (score learning) without disrupting the learning rhythm. The key light indicator module, in conjunction with the finger light indicator module, provides fingering guidance that is more specific, operable, and less costly for users compared to traditional teaching methods, allowing for quick and easy learning. This system can directly teach users starting with a sample score, resulting in a shorter learning cycle and faster engagement compared to the current instrument teaching process (basic instruction → score practice). This prevents users from losing interest in learning an instrument due to lengthy basic lessons, demonstrating promising application prospects.
[0019] As a preferred technical solution:
[0020] As described above, this intelligent multimodal interactive music notation learning system for keyboard instruments allows for adjustments to the sheet music, fingering, and playback speed through settings on the central control module. The system provides optimized fingering for specific sheet music (i.e., a control scheme combining the key light indicator module and the finger light indicator module). To meet the needs of different users, the system can store optimized fingering for users with different hand lengths and body types (it can also be differentiated by user age and gender, such as toddlers, teenagers, adult men, adult women, etc., and can even include more types, such as adult little fingering, to meet the needs of more customers). When using the system, users can select the fingering type to suit different user needs. The playback speed of the sheet music (such as quarter notes, eighth notes) can be adjusted. For example, the playing instructions can be slowed down, allowing users to personally experience the process of playing each note. During the slowed-down process, combined with the teacher's voice-over explanation, users can more clearly feel the rhythm, dynamics, and timbre changes of the notes, establishing an intuitive correspondence between sound, notes, and staff symbols, deepening their memory and understanding of the notes.
[0021] The above-described intelligent multimodal interactive music reading and learning system for keyboard instruments sets a preset scheme corresponding to the current moment of the music score through the central control module settings;
[0022] The preset scheme corresponding to the current time of the score involves the key light indicator module and the finger light indicator module being controlled according to a segment control scheme that includes the current time of the score, and the off-screen audio information corresponding to the segment including the current time of the score. This preset scheme corresponding to the current time of the score can be set by defining the type (length) of the segment, such as setting a note mode (segment is a note, and the preset scheme controls it based on the note's time period); setting a note block mode (segment is a note block, and the preset scheme controls it based on the note block's time period); setting a note string mode (segment is a note string, and the preset scheme controls it based on the note string's time period); setting a musical phrase mode (segment is a musical phrase, and the preset scheme controls it based on the musical phrase's time period); or setting a full page mode (segment is a full page, and the preset scheme controls it based on the full page's time period).
[0023] As described above, this intelligent multimodal interactive music notation learning system for keyboard instruments allows users to enable or disable background audio information through settings on the central control module. The background audio function explains the names of fixed notes and their corresponding music theory knowledge, such as the meaning of the notes and key playing points. It also provides detailed explanations of music theory symbols in the score and demonstrates the playing effect of the notes for students to compare and experience. Furthermore, users can enable or disable background audio according to their needs and can set several sets of explanations with different levels of analytical depth.
[0024] As described above, this intelligent multimodal interactive music notation learning system for keyboard instruments treats each musical note, note block, note string, phrase, or entire page as a unit for specialized learning. This learning method breaks away from the traditional single-note learning model, enabling learners to better understand the combinations of notes and the overall structure of the music, facilitating the formation of large-scale visual memories and improving music notation recognition speed. By treating each musical note, note block, note string, phrase, or entire page as a unit for specialized learning, this system breaks away from the traditional single-note learning model, enabling users to better understand the combinations of notes and the overall structure of the music, facilitating the formation of large-scale visual memories and improving music notation recognition speed.
[0025] The creation of voice-over information involves dividing the learning content corresponding to the musical score into several different units and creating information for each unit individually. The smallest unit can be a single note; however, a unit can also be a note block, a sequence of notes, several syllables, a section of Yue opera, or even an entire musical score. The foot pedal control unit can be a step-forward or step-back switch, released after each press. When the foot pedal control unit is triggered, the learning content is limited to one unit. If this unit contains only one note, the control scheme may be a static display (with the key indicator module and finger indicator module constantly lit) or a dynamic demonstration. If a unit requires practicing multiple notes, the control scheme can be a dynamic, looping demonstration. The control logic of the foot pedal control unit can be to step to the next unit or to step back to the previous unit; those skilled in the art can configure it according to actual needs.
[0026] As described above, the intelligent multimodal interactive music reading and learning system for keyboard musical instruments includes a foot pedal control unit comprising multiple foot pedal controllers, each with a different control scheme upon activation. The presence of multiple foot pedal controllers provides the system with greater adjustability and facilitates user operation.
[0027] As described above, in a keyboard instrument intelligent multimodal interactive music reading and learning system, the foot pedal control unit is a foot pedal with two foot pedal controls. For example, if the foot pedal adopts a balanced design, the left and right or forward and backward are two foot pedal controls, each corresponding to a different control scheme. For example, left / forward is the control scheme for recalling the previous paragraph, and right / backward is the control scheme for recalling the next paragraph.
[0028] As described above, the intelligent multimodal interactive music reading and learning system for keyboard instruments includes a foot pedal control unit consisting of multiple foot pedals, each with a foot pedal control component. The system can be configured to have two foot pedals corresponding to the left and right feet, with each foot pedal corresponding to a different control scheme. For example, the left foot pedal is used to access the previous section, while the right foot pedal is used to access the next section.
[0029] Furthermore, the present invention also provides a method for using the intelligent multimodal interactive music reading and learning system for keyboard musical instruments as described above, comprising the following steps:
[0030] (1) Assemble and connect the components of the intelligent multimodal interactive music reading learning system for keyboard instruments, install the key light indicator module on the keyboard instrument, and install the finger light indicator module on the user's hand;
[0031] (2) Configure the central control module with the sheet music, fingering, playback speed, preset scheme corresponding to the current time of the sheet music, and opening / closing of voice-over information;
[0032] (3) The user begins to practice the sheet music with the assistance of the keyboard instrument intelligent multimodal interactive music reading and learning system;
[0033] (4) When the user needs to study a certain moment in detail, he / she can step on the foot pedal control unit. At this time, the key light indicator module and the finger light indicator module will operate according to the preset scheme corresponding to the current moment of the score, that is, in static (note mode) or dynamic loop of the paragraph. The video display module and the audio module will play or not play off-screen information according to the settings. The user can practice according to the prompts of the key light indicator module, the finger light indicator module, the video display module and the audio module. The user can repeatedly stimulate memory in the freeze state, freely control the learning progress by switching on and off, skip the content that has been mastered, and fully realize the goal of practicing more what you don't know and skipping what you have mastered.
[0034] (5) After the detailed learning, the user releases the foot pedal control unit, and the keyboard instrument intelligent multimodal interactive music reading learning system resumes from the detailed learning moment. If the user needs to perform detailed learning, step (4) is repeated until the music score is completed.
[0035] The method of using the system of this invention is not limited to this. The above is only one feasible technical solution. Those skilled in the art can customize the software in the central control module according to actual needs and give the foot pedal control unit different functions. The core of this invention is to introduce a foot pedal control unit that does not affect the user's hand operation to control the system. At the same time, the software divides the entire score into several sections and designs preset schemes for each section (including control schemes for the voice-over and key light indicator modules and finger light indicator modules). Through the foot pedal control unit, the preset scheme for the current section can be called up while the user is learning dynamically, so as to provide detailed guidance to the user. The user can carry out targeted and detailed learning. This dynamic learning method can greatly improve the user's learning efficiency.
[0036] The above technical solution is only one feasible technical solution of the present invention. The scope of protection of the present invention is not limited thereto. Those skilled in the art can reasonably adjust the specific design according to actual needs.
[0037] The above invention has the following advantages or beneficial effects:
[0038] (1) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention provides users with a brand-new auxiliary learning method. Users can learn keyboard instruments anytime and anywhere according to their own learning progress and needs, reducing reliance on the on-site guidance of real teachers. Whether at home, school or other places, as long as there are keyboard instruments and this system, students can learn independently, which greatly improves the flexibility and convenience of learning.
[0039] (2) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention saves users the time of counting squares and comparing with the textbook. Through intuitive video display, key light and finger light indication, and background voice explanation, users can quickly and accurately understand and master the meaning of notes. The synchronous multimodal interactive practice greatly improves learning efficiency. For example, in the past, users may have to spend time to recognize a complex note. With the help of this system, users can easily understand and remember the note in a shorter time and can practice the corresponding keys and fingering at the same time.
[0040] (3) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention adopts software teaching method, which partially replaces traditional real-person teaching. This not only saves users high tuition costs, but also avoids the problem of limited learning opportunities due to limited teacher resources. Users can listen to the background voice explanation repeatedly to deepen their understanding of music theory and improve learning effect.
[0041] (4) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention brings a brand-new learning experience to users through multimodal interactive learning. The combination of multiple modes such as video, light, foot pedal control unit and background voice allows users to fully mobilize multiple senses such as vision, hearing and touch during the learning process, enhance the fun and participation of learning, stimulate students' learning interest and enthusiasm, and provide a good user experience.
[0042] (5) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention uses a foot pedal control unit as a controller. Compared with traditional manual operation, it will not affect the operation of playing the piano with the hands. It can realize the ability to stop immediately when stepping on the pedal, without disrupting the learning rhythm, and ensuring the continuity and smoothness of the learning process (music score learning).
[0043] (6) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention, the key light indicator module and the finger light indicator module work together to provide fingering guidance to users. Compared with traditional teaching, it is more specific, more operable, and has a lower learning cost for users, making it easier for users to get started quickly.
[0044] (7) The keyboard instrument intelligent multimodal interactive music reading learning system of the present invention can directly teach and guide users with the demonstration score as the starting point. Compared with the current instrument teaching process (basic teaching → music practice), its cycle is shorter and it can quickly enter the user's learning excitement period, avoiding the user losing interest in learning the instrument due to the lengthy basic teaching. It has great application prospects. Attached Figure Description
[0045] The invention, its features, shape, and advantages will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Like reference numerals denote like parts throughout the drawings. The drawings are not drawn to scale; their focus is on illustrating the gist of the invention.
[0046] Figure 1 This is a schematic diagram of the intelligent multimodal interactive music reading and learning system for keyboard instruments according to the present invention. Detailed Implementation
[0047] The structure of the present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the present invention.
[0048] Example 1
[0049] A smart multimodal interactive music reading and learning system for keyboard musical instruments, such as Figure 1 As shown, it includes a central control module, a video display module, an audio module, a piano key light indicator module, a finger light indicator module, and a foot pedal control unit;
[0050] The central control module is connected to the video display module, audio module, piano key light indicator module, finger light indicator module and foot pedal control unit respectively;
[0051] The video display module is used to display the sheet music video;
[0052] The audio module is used to play audio;
[0053] The key light indicator module is used to indicate the key on the keyboard instrument corresponding to the current note;
[0054] The finger light indicator module is used to indicate the finger that presses the current note;
[0055] The foot pedal control unit is used to send control signals to the central control module. The foot pedal control unit is a foot pedal with two foot pedal controls, left and right. The left foot pedal control triggers the previous control scheme, and the right foot pedal control triggers the next control scheme.
[0056] The central control module is used to acquire control signals from the foot pedal control unit, process the control signals to obtain a control scheme, and then send relevant control signals to the video display module, audio module, key light indicator module, and finger light indicator module according to the control scheme. Specifically, the control scheme is to control the key light indicator module and finger light indicator module according to the preset scheme corresponding to the current moment of the music score, and recall the off-screen audio information corresponding to the preset scheme and play it through the video display module and audio module. The off-screen audio information refers to the interpretation of the notes, phrases, or entire pages of the music score in the form of video and / or voice.
[0057] The central control module allows for adjustments to the sheet music, fingering, playback speed, enabling or disabling voice-over information, and setting preset schemes corresponding to the current state of the sheet music.
[0058] The preset scheme corresponding to the current moment of the score is that the key light indicator module and the finger light indicator module are controlled according to the segment control scheme including the current moment of the score, and the off-screen audio information corresponds to the segment (note, note block, note string, musical phrase or whole page) including the current moment of the score.
[0059] The method of using the above-mentioned intelligent multimodal interactive music reading and learning system for keyboard instruments includes the following steps:
[0060] (1) Assemble and connect the components of the intelligent multimodal interactive music reading learning system for keyboard instruments, install the key light indicator module on the keyboard instrument, and install the finger light indicator module on the user's hand;
[0061] (2) Set the score, fingering, playback speed, preset scheme (segment length) corresponding to the current time of the score, and turn off the voice-over information of the central control module;
[0062] (3) The user begins to practice the sheet music with the assistance of the keyboard instrument intelligent multimodal interactive music reading and learning system;
[0063] (4) When the user needs to study a certain moment in detail, he / she can step on the foot pedal control unit. At this time, the key light indicator module and the finger light indicator module will operate according to the preset scheme corresponding to the current moment of the score, that is, in static or dynamic loop of the segment. The video display module and the audio module will play or not play off-screen information according to the settings. The user can practice according to the prompts of the key light indicator module, the finger light indicator module, the video display module and the audio module.
[0064] (5) After the detailed learning, the user releases the foot pedal control unit, and the keyboard instrument intelligent multimodal interactive music reading learning system resumes from the detailed learning moment. If the user needs to perform detailed learning, step (4) is repeated until the music score is completed.
[0065] Example 2
[0066] A smart multimodal interactive music reading and learning system for keyboard instruments is basically the same as that in Embodiment 1, except that the foot pedal control unit consists of a left foot pedal and a right foot pedal corresponding to the left and right feet respectively. Each foot pedal is equipped with a foot pedal control component. The left foot pedal triggers the control scheme for recalling the previous paragraph, and the right foot pedal triggers the control scheme for recalling the next paragraph.
[0067] Verification has shown that the intelligent multimodal interactive music reading and learning system for keyboard instruments of this invention is rationally designed. Compared with existing technologies, it also incorporates an audiovisual component. Through the cooperation of the audiovisual unit with the key light indicator module and the finger light indicator module, multimodal interactive synchronous learning can be achieved. The foot pedal control unit can pause, loop, or further control the music playback status. Combined with off-screen audio information (displayed by the audiovisual unit), it can further enhance the learning effect, enabling users to interact with the visual and note images to memorize note graphics, interact with the auditory and off-screen audio to understand the meaning of symbols, interact with the visual and key lights to memorize the corresponding key positions of notes, interact with the keys through finger movements to memorize spatial positions through tactile memory, and interact with the sound of the piano to memorize the sound effects corresponding to the notes. Through the coordinated work of the above multiple human organs, the system forms a comprehensive understanding of the musical notes. The system features three-dimensional perception and memory, making the learning process direct and efficient. Using a foot pedal control unit as the controller, it doesn't interfere with hand movements compared to traditional manual operation, allowing for instant stopping and starting without disrupting the learning rhythm and ensuring the continuity and smoothness of the learning process (music reading). The key light indicator module, in conjunction with the finger light indicator module, provides fingering guidance that is more specific and operable than traditional teaching methods, reducing the learning cost and facilitating quick mastery. This system allows for direct instruction starting with sample music scores, a shorter cycle than current instrument teaching processes (basic instruction → music practice) that quickly engages the user and prevents them from losing interest due to lengthy basic lessons, making it highly promising.
[0068] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention, and will not be elaborated here either.
[0069] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.
Claims
1. A smart multimodal interactive music reading and learning system for keyboard musical instruments, characterized in that: It includes a central control module, a video display module, an audio module, a piano key light indicator module, a finger light indicator module, and a foot pedal control unit; The central control module is connected to the video display module, audio module, piano key light indicator module, finger light indicator module and foot pedal control unit respectively. The video display module is used to display the music score video; The audio module is used to play audio; The key light indicator module is used to indicate the key on the keyboard instrument corresponding to the current note; The finger light indicator module is used to indicate the finger pressing the current note; The foot pedal control unit is used to send control signals to the central control module; The central control module is used to acquire control signals from the foot pedal control unit, process the control signals to obtain a control scheme, and then send relevant control signals to the video display module, audio module, key light indicator module, and finger light indicator module according to the control scheme. Specifically, the control scheme is to control the key light indicator module and finger light indicator module according to a preset scheme corresponding to the current moment of the score, and to recall the off-screen audio information corresponding to the preset scheme and play it through the video display module and audio module. The off-screen audio information refers to the interpretation of the notes, phrases, or entire pages of the score in the form of video and / or voice.
2. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 1, characterized in that, The central control module allows for adjustments to the sheet music, fingering, and playback speed.
3. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 1, characterized in that, The central control module is configured with preset settings corresponding to the current state of the musical score. The preset scheme corresponding to the current moment of the musical score is that the key light indicator module and the finger light indicator module are controlled according to the segment control scheme including the current moment of the musical score, and the off-screen audio information corresponds to the segment including the current moment of the musical score.
4. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 3, characterized in that, The voice-over information can be enabled or disabled by configuring the central control module.
5. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 3, characterized in that, The paragraph can be a musical note, a block of musical notes, a string of musical notes, a musical phrase, or an entire page.
6. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 1, characterized in that, The foot pedal control unit includes multiple foot pedal controllers, and each foot pedal controller has a different control scheme after being triggered.
7. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 6, characterized in that, The foot pedal control unit is a foot pedal, and the foot pedal is equipped with two foot pedal control components.
8. The intelligent multimodal interactive music reading and learning system for keyboard musical instruments according to claim 6, characterized in that, The foot pedal control unit consists of multiple foot pedals, with each foot pedal equipped with a foot pedal control element.
9. The method of using the intelligent multimodal interactive music reading and learning system for keyboard musical instruments as described in any one of claims 1 to 8, characterized in that, Includes the following steps: (1) Assemble and connect the components of the intelligent multimodal interactive music reading learning system for keyboard instruments, install the key light indicator module on the keyboard instrument, and install the finger light indicator module on the user's hand; (2) Configure the central control module with the sheet music, fingering, playback speed, preset scheme corresponding to the current time of the sheet music, and opening / closing of voice-over information; (3) The user begins to practice the sheet music with the assistance of the keyboard instrument intelligent multimodal interactive music reading and learning system; (4) When the user needs to study a certain moment in detail, he / she can step on the foot pedal control unit. At this time, the key light indicator module and the finger light indicator module will operate according to the preset scheme corresponding to the current moment of the score, that is, in static or dynamic loop of the segment. The video display module and the audio module will play or not play off-screen information according to the settings. The user can practice according to the prompts of the key light indicator module, the finger light indicator module, the video display module and the audio module. (5) After the detailed learning, the user releases the foot pedal control unit, and the keyboard instrument intelligent multimodal interactive music reading learning system resumes from the detailed learning moment. If the user needs to perform detailed learning, step (4) is repeated until the music score is completed.