Music performance education platform based on somatosensory interaction

Abstract

The invention discloses a music performance education platform based on somatosensory interaction, and belongs to the technical field of music education, and the platform specifically comprises the steps: collecting multi-dimensional data, and constructing a continuous time sequence action stream to restore a playing track; constructing and maintaining a standard action template and a style vector library, and training basic action expression and personalized style expression according to musical instrument and genre labels; meanwhile, the matching condition of the actions of the learner in the basic action expression and the personalized style expression is analyzed, and the actions of the learner are classified as wrong actions or reasonable personalized deviations by comparing the differences in the two analysis results; a new action evaluation standard is formed according to the basic performance and the learning progress of the learner, and the standard is provided for a principle and style discrimination module; translating the classification result into a technical correction instruction or an expression retention prompt; and receiving a style example and an overlay mark input by the teacher, and updating a style vector and adjusting the action analysis logic according to the mark after the teacher marks the action category.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of music education, in particular to a music performance education platform based on somatosensory interaction. BACKGROUND

[0002] In traditional music performance education, teachers need to guide learners through real-time observation of their body movements and playing postures, but due to the limitations of teaching scenes and teacher energy, it is difficult to achieve precise capture of action details and continuous feedback function. With the development of motion capture and somatosensory interaction technology, various music education auxiliary systems have emerged. Such systems mostly use inertial measurement units, optical sensors and other devices to collect somatosensory data such as joint angles, motion speed and acceleration of learners, compare them with preset standard action templates, and determine the action standardization through data deviation. Some systems also combine electromyographic signal feedback to assist in correcting playing posture. However, the core evaluation logic of existing somatosensory systems is built around a single or a small number of fixed standard templates, which does not adequately consider the differences in genre style of different musical instrument performances, and does not fully focus on the individualized action expression formed by the physiological characteristics and artistic perception differences of learners, which may misjudge reasonable stylistic deviations as errors and make it difficult to balance the dual needs of technical specifications and artistic expression.

[0003] In music performance education, the balance between standardized action training and individualized artistic expression is a core teaching difficulty, and the intervention of somatosensory interaction technology needs to break through the limitations of standard action theory. Existing somatosensory systems mostly match and determine based on fixed action templates, but the standard actions of different musical instruments have genre differences, and the physiological characteristics and artistic preferences of learners will form reasonable individualized action deviations. In view of the above teaching difficulties, how to identify incorrect actions that violate the principles of playing through multi-dimensional somatosensory data, distinguish between reasonable individualized deviations and standard errors, dynamically adjust the evaluation threshold according to the teaching stage, and at the same time convert the evaluation results into individualized improvement suggestions to avoid suppressing artistic expression, becomes a key technical problem to be solved. SUMMARY

[0004] The purpose of the present application is to provide a music performance education platform based on somatosensory interaction to solve the problems in the background art.

[0005] The purpose of the present application can be achieved by the following technical solutions: A music performance education platform based on somatosensory interaction, comprising: A multi-source acquisition module for acquiring multi-dimensional data and constructing continuous time series action flow to restore the playing track; A dual-channel training module for constructing and maintaining standard action templates and style vector library, and training basic action expression and individualized style expression according to musical instrument and genre labels; The principle and style discrimination module is used for simultaneously analyzing matching of the learner action in the basic action expression and the personalized style expression, and classifying the learner action as an error action or a reasonable personalized deviation by comparing differences between two types of analysis results; The stage adjustment module is used for forming a new action evaluation standard according to the basic performance and the learning progress of the learner, and providing the new action evaluation standard to the principle and style discrimination module, so that the analysis process presents stage differences; The evaluation module is used for receiving the classification result of the principle and style discrimination module, and translating the classification result into a technical correction instruction or an expression reservation prompt, while providing an action example and a decomposition exercise path for the learner to practice; The teacher interface module is used for receiving a style example and a review mark input by a teacher, and updating a style vector and adjusting an action analysis logic according to the mark after the teacher makes a mark on the action category.

[0006] As a further scheme of the present application, in the multi-source acquisition module, the multi-dimensional data includes joint angles, motion speeds, accelerations, electromyography and instrument touch signals.

[0007] As a further scheme of the present application, the process of constructing the continuous time sequence action flow to restore the performance track is: Inertial measurement units are arranged at joints of a performer, electromyography sensors are attached to muscle surfaces, and pressure sensors are installed at contact parts of an instrument, and all the sensors are started to collect data through a same hardware trigger signal; The joint angles, motion parameters, electromyography signals and touch signals collected by the sensors are transmitted to a processing unit, signal synchronization is realized through hardware time stamp alignment, and interference signals are removed through a filtering algorithm; Taking the synchronized signal time axis as a reference, the joint angles and the motion parameters are mapped into a limb motion track, and action details are supplemented by combining the electromyography signals and the touch signals, and a continuous time sequence action flow is formed in series.

[0008] As a further scheme of the present application, in the double-channel training module, the process of constructing and maintaining the standard action template and the style vector library, and training the basic action expression and the personalized style expression according to the instrument and the genre label is: Standard action data of professional performers of each instrument and genre are collected, classified according to the instrument and the genre label, action features are extracted to construct the standard action template, and style difference features are extracted to construct the style vector library; New professional action data and a style example input by the teacher interface module are received, compared with the constructed standard action template and the style vector library, deviation data is replaced, and contents corresponding to new labels are supplemented to complete maintenance; Take the standard action template as the benchmark, and train the basic action expression according to the instrument and genre label; take the style vector library as the reference, and train the personalized style expression under the same label.

[0009] As a further scheme of the present application: in the principle and style discrimination module, the process of classifying the learner's action as an error action or a reasonable personalized deviation by comparing the differences between the two types of analysis results is: Extract the feature data of the learner's action, compare it with the standard action template, and get the matching result of the basic action expression; at the same time, compare it with the style vector library, and get the matching result of the personalized style expression; Compare the matching results of the basic action expression and the personalized style expression, locate the difference points of the two in the action features, and determine the specific action links where the differences are; If the basic action expression does not match, regardless of the personalized style expression, the action is classified as an error action; if the basic action expression matches, the difference of the personalized style expression is within the range of the style vector library, and it is classified as a reasonable personalized deviation.

[0010] As a further scheme of the present application: in the stage adjustment module, the process of forming a new action evaluation standard according to the learner's basic performance and learning progress, and providing the standard to the principle and style discrimination module, so that the analysis process presents a staged difference is: Collect the initial action matching results of the learner as the basic performance, continuously record the action classification results and practice feedback in the learning process, and summarize the learning progress data; Determine the initial learning stage according to the basic performance, combine the trend of reducing error actions and the stability of reasonable personalized deviation in the learning progress, and divide new learning stages; For each learning stage, adjust the matching threshold of the basic action expression and the allowed difference range of the personalized style expression, and form the action evaluation standard of the corresponding stage; Transmit the action evaluation standards of each stage to the principle and style discrimination module, so that it calls the standards of the corresponding stage when analyzing, and forms a staged analysis difference.

[0011] As a further scheme of the present application: in the evaluation module, the process of receiving the classification results of the principle and style discrimination module, and translating the classification results into technical correction instructions or expression retention prompts is: The evaluation module receives the classification results output by the principle and style discrimination module, including the specific action link identification of error actions and reasonable personalized deviations; If it is an error action, the joint angle, motion speed, acceleration, electromyography and musical instrument touch signal deviation parameters corresponding to the action are extracted and translated into targeted technical correction instructions; if it is a reasonable individualized deviation, the corresponding elements of the style vector library are associated to generate an expression reservation prompt; For error actions, the dynamic images of the corresponding links in the standard action template are called as action examples; for reasonable deviations, similar style action images of players in the same genre are displayed as examples; Error actions are decomposed into progressive decomposition exercise steps in the order of joint motion to form a path; reasonable individualized deviations are designed into coherent exercise combinations according to style characteristics to form a path for learners to practice.

[0012] As a further scheme of the present application, in the teacher interface module, the style example and review annotation input by the teacher are received, and when the teacher makes an annotation on the action category, the process of updating the style vector and adjusting the action analysis logic according to the annotation is: The teacher interface module receives the style example action data input by the teacher and the review annotation of the action category of the learner, and stores the style examples and the annotation content classified according to the musical instrument and genre tags; When the teacher completes the action category annotation, the action features corresponding to the annotation are extracted, compared with the same tag vector in the style vector library, the conflicting vector is replaced, and the new features are supplemented to generate a new vector; According to the distinguishing basis of error actions and reasonable deviations reflected in the teacher's annotation, the matching logic and difference judgment parameters of the two types of expressions in the principle and style discrimination module are adjusted.

[0013] The present application has the following advantages: The present application effectively breaks through the limitations of the traditional somatosensory music education system, which is only based on standard actions, and realizes the dynamic balance between standardized action training and individualized artistic expression through the collaborative design of multiple modules. The multi-source acquisition module collects multi-dimensional somatosensory data and constructs continuous time sequence action flow, providing accurate data basis for action analysis; the double-channel training module constructs standard action templates and style vector libraries according to musical instrument and genre tags, fully considering the genre differences of different musical instruments to avoid the limitations of a single template; the principle and style discrimination module simultaneously analyzes the matching of basic actions and individualized styles, which can accurately identify error actions that violate the principles of playing, and at the same time, distinguish reasonable individualized deviations formed by physiological characteristics and artistic preferences of learners, solving the core teaching difficulty of balancing standardization and individualization, ensuring the standardization of playing actions, and reserving space for artistic expression.

[0014] In addition, the present application further improves teaching adaptability and learning experience through dynamic evaluation and personalized feedback mechanism. The stage adjustment module dynamically adjusts the evaluation threshold according to the basic performance and progress of the learner, so that the analysis process meets the needs of different learning stages, and avoids using a unified standard to require learners of different levels; the evaluation module translates the classification results into targeted technical correction instructions or expression reservation prompts, and provides action examples and decomposition exercise paths, so that the learner can clearly improve the direction and retain the enthusiasm of personalized expression; the teacher interface module introduces teacher experience feedback to continuously optimize the style vector library and analysis logic, ensuring that the system can adapt to diversified teaching scenarios and style requirements. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present application will be further described below in conjunction with the accompanying drawings.

[0016] Figure 1 is a module schematic diagram of a music performance education platform based on somatosensory interaction. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0018] Please refer to Figure 1 The present application is a music performance education platform based on somatosensory interaction, comprising the following steps: A multi-source acquisition module is used to acquire multi-dimensional data and construct continuous time series action flow to restore the performance track. A double-channel training module is used to construct and maintain standard action templates and style vector library, and train basic action expression and personalized style expression according to the instrument and genre label. A principle and style discrimination module is used to analyze the matching of learner actions in basic action expression and personalized style expression, and classify the learner actions as error actions or reasonable personalized deviation by comparing the differences between the two types of analysis results. A stage adjustment module is used to form a new action evaluation standard according to the basic performance and learning progress of the learner, and provide the standard to the principle and style discrimination module, so that the analysis process presents stage differences. An evaluation module is used to receive the classification results of the principle and style discrimination module, and translate the classification results into technical correction instructions or expression reservation prompts, while providing action examples and decomposition exercise paths for the learner to practice. A teacher interface module is configured to receive style examples and review annotations input by a teacher. After the teacher annotates the action category, the style vector is updated and the action analysis logic is adjusted based on the annotation.

[0019] In a preferred embodiment of the present application, in the multi-source acquisition module, the multi-dimensional data includes joint angles, motion speeds, accelerations, electromyography, and instrument touch signals; and the process of constructing a continuous time-series action stream to restore a performance track is as follows: Before collecting multi-dimensional data, the deployment and start trigger settings of various sensors need to be completed. Specifically, inertial measurement units are deployed at the key joints of the performer. These key joints are determined according to the type of instrument. For example, a piano performer needs to deploy at the finger, wrist, elbow, and shoulder joints, and a violin performer needs to focus on the left hand finger, wrist, elbow, and right hand bowing related joint deployment. The inertial measurement unit is directly fixed to the skin to accurately capture the joint motion state. Electromyography sensors are attached to the surface of the muscles related to the performance of the performer and the instrument. For example, for violin performance, electromyography sensors need to be attached to the surface of the muscle group controlling the fingers pressing the strings on the left forearm and the muscle group controlling the bowing on the right hand, to ensure that the muscle electrical signals during performance are captured. Pressure sensors are installed at the key positions of the instrument and the performer, such as the surface of the piano keys, the fingerboard and bow rod holding position of the violin, the string pressing fingerboard and plectrum contact position of the guitar, to sense the contact pressure signals during performance. After all the deployment is completed, the acquisition is started by the same hardware trigger signal, for example, by setting an independent start switch. At the moment of pressing the switch, the inertial measurement unit, electromyography sensor, and pressure sensor start working at the same time, ensuring the synchronous start of action acquisition.

[0020] After the sensors are started, various types of data will be continuously generated and transmitted to the preset processing unit. The inertial measurement unit collects joint angle, motion speed, and acceleration data, the electromyography sensor collects electromyography signal, and the pressure sensor collects instrument touch signal. These data are transmitted in real time to the processing unit through wired or wireless means. To ensure the time correspondence of different types of data, hardware timestamp alignment is used to realize signal synchronization. Each set of data collected by each sensor is marked with the same time information. For example, the finger joint angle, the electromyography signal of the corresponding muscle, and the touch signal of the finger pressing the string at a certain time are all marked with completely consistent time information, to ensure that the data can be accurately matched during subsequent processing. After synchronization is completed, interference signals are removed through a specific processing method. These interference signals may come from slight vibrations in the surrounding environment, slight shaking of the sensor connection line, and other factors unrelated to the performance action. After processing, only the valid data directly related to the performance action are retained.

[0021] After the data synchronization and de-interference processing is completed, the synchronized signal time axis is taken as a unified reference to develop the construction of continuous time sequence action flow. First, the motion parameters such as joint angle, motion speed and acceleration are integrated to map the trajectory of the performer's limb movement, for example, according to the change of finger joint angle, the speed of motion and the fluctuation of acceleration, the moving path of the fingers on the musical instrument can be clearly restored, and according to the related data of the elbow and shoulder joint, the swing trajectory of the arm can be restored. On this basis, combined with the muscle force details of the action supplemented by the electromyographic signal, the strength change of the electromyographic signal can reflect the degree of muscle contraction, for example, the intensity of the corresponding muscle electromyographic signal when playing forte is different from that when playing piano, and these differences can be accurately supplemented to the corresponding action link. At the same time, combined with the contact details of the action supplemented by the musical instrument touch signal, the change of the touch signal can reflect the degree of contact between the performer and the musical instrument, for example, the signal change of the pressure sensor when pressing the string can reflect the size of the pressing force, and these details and the trajectory of the limb movement are combined with each other, and are connected in the order of time axis to form a continuous time sequence action flow, which restores the whole performance trajectory of the performer from the beginning to the end.

[0022] In another preferred embodiment of the present application, in the double-channel training module, the process of constructing and maintaining the standard action template and the style vector library, and training the basic action expression and personalized style expression according to the musical instrument and genre label is as follows: When constructing the standard action template and the style vector library, the standard action data of professional performers of various musical instruments and various genres needs to be systematically collected. These musical instruments include common categories such as piano, violin, guitar and zither, and the genres include different style categories such as classical, jazz, folk and popular. The professional performers need to complete the core performance actions of the corresponding genres of various musical instruments, for example, the classical piano performer completes the basic actions such as scale, chord and tremolo, and the folk violin performer completes the characteristic actions such as changing the bow and rubbing the strings, and the multi-dimensional data such as joint angle, motion speed, acceleration, electromyography and musical instrument touch signal are collected synchronously. After the collection is completed, exclusive labels are set according to the combination of musical instruments and genres, for example, piano-classical, violin-folk, guitar-jazz, etc., and common action features are extracted from the data of the same label, these features include the joint motion range, muscle force rhythm and the change law of touch key or string pressing force during performance, etc., and these common features are integrated to form the standard action template. At the same time, unique style difference features are extracted from the data of different professional performers of the same musical instrument and the same genre, for example, the subtle emphasis of touch key force and the rhythm difference of finger movement of two classical piano performers, and these style difference features are sorted to form the style vector library.

[0023] After the construction of the standard action template and the style vector library, dynamic maintenance is needed through continuous reception of new data. The specific process is as follows: First, two parts of content are received, one part is the standard action data of new professional performers, and the other part is the style examples input by the teacher interface module. These style examples are representative genre style action data selected or recorded by the teacher according to teaching experience. Then, two steps of comparison work are carried out, the first step is for the standard action template, the newly received professional action data is matched to the existing template according to the instrument and genre label, the action features in the new data are extracted and compared with the common features in the corresponding template one by one, and the key dimensions such as joint motion trajectory, muscle force timing, and touch signal change are checked for consistency. If inconsistent bias data is found due to updated performance cognition or data collection bias, the accurate features in the new data are used to replace the biased part in the existing template after verification; the second step is for the style vector library, the style difference features in the newly received professional action data and the style example features input by the teacher are compared with the style vectors in the existing library under the same instrument and genre label, and it is judged whether the features conflict or have a supplement space. If new labels are encountered during maintenance, such as new instrument-genre combination labels or subdivided style labels under existing labels, the action features corresponding to the labels need to be supplemented to the standard action template, and the corresponding style difference features need to be supplemented to the style vector library to ensure that the coverage of the template and the library is continuously improved, and the overall maintenance work is completed.

[0024] The training of basic action expression and personalized style expression needs to rely on the constructed and maintained standard action template and style vector library, and is carried out according to the instrument and genre label classification. When training basic action expression, the standard action template under the same label is taken as the only reference to guide learners to imitate the common action features in the template, for example, under the piano-classical label, learners need to follow the finger joint bending angle, wrist motion trajectory, and touch key force change law in the template to ensure the standardization and normalization of the performance action, laying a foundation for subsequent style expression. When training personalized style expression, the style vector library under the same label is taken as a reference to guide learners to contact the style difference features of different professional performers in the library on the premise that they have mastered basic action expression, for example, under the same violin-folk label, learners can refer to the string rubbing speed and changing force emphasis of different performers in the library to form a unique expression form combining their own performance habits, which ensures that the action meets the core requirements of the genre and reflects individualized characteristics. The whole training process strictly follows the correspondence of the instrument and genre label to ensure the standardization of basic action expression and the adaptability of personalized style expression.

[0025] In another preferred embodiment of the present application, the principle is that the action matching of the learner is analyzed in the principle and style discrimination module, and the learner's action is classified as an error action or a reasonable personalized deviation by comparing the differences between the two types of analysis results. This embodiment takes the piano classical sonata performance learning scene as an example to illustrate in detail how the principle and style discrimination module analyzes the action matching of the learner and completes the action classification. Before the action matching analysis, the characteristic data of the learner's action needs to be extracted, and then compared with the standard action template and the style vector library respectively. Specifically, the learner performs the performance action according to the performance requirements of the piano classical sonata, and the multi-source acquisition module synchronously collects the angle data of the joints such as fingers, wrists, elbows, the movement speed and acceleration data of the fingers touching the keys, the electromyographic signals of the arms related muscles, and the touch signals of the fingers and the keys. From these collected data, the action characteristic data is extracted, which includes the bending angle range of the finger joints, the amplitude and speed of the wrist rotation, the force change when touching the keys, the timing law of muscle force, etc. The extracted characteristic data is compared with the standard action template under the piano-classical sonata label, and the standard action template clearly defines the standard action characteristics of this type of performance, such as the natural bending angle interval of the joints when the fingers touch the keys, the smooth rotation trajectory of the wrist when playing legato, etc. The matching result of the basic action expression is obtained by comparing each feature. At the same time, the extracted characteristic data is compared with the style vector library under the same label, and the style vector library contains the style difference characteristics of multiple classical piano performers in the performance of this sonata, such as some performers touching the keys lightly, some performers touching the keys heavily, some performers playing fingers slightly fast, and some performers playing fingers slightly slow, etc. The matching result of the personalized style expression is obtained by comparing.

[0026] After obtaining the two types of matching results, they need to be compared to locate the difference points and specific action links. In the comparison process, each action characteristic in the matching result of the basic action expression and the matching result of the personalized style expression is compared one by one to find the difference point. For example, in the matching result of the basic action expression, the joint angle of the learner's finger touching the keys meets the requirements of the standard action template, but in the matching result of the personalized style expression, the force characteristic of the finger touching the keys is different from most of the characteristics in the style vector library, and this force characteristic is the difference point. Further determine the specific action link where the difference is located, and judge it in combination with the musical phrase and action sequence, such as the above force difference appearing in the lyrical phrase of the first movement of the sonata, and the corresponding specific action link is the action of the right ring finger pressing the middle sound area key, so as to clearly define the actual performance action scene corresponding to the difference.

[0027] According to the analysis result of the difference point and the specific action link, the learner's action is classified according to the set rule. If the basic action expression does not match, regardless of the matching situation of the personalized style expression, the action is directly classified as an error action. For example, when the learner plays the jumping note paragraph of the classical sonata of the piano, the finger joint angle obviously exceeds the natural bending range required by the standard action template, even if the touch force and other style characteristics are consistent with a certain characteristic in the style vector library, the finger jumping note action is still classified as an error action. If the basic action expression matches, and the difference of the personalized style expression conforms to the style vector library range, the action is classified as reasonable individualization deviation. For example, when the learner plays the terminal sentence of the sonata, the joint angle of the finger touch key, the wrist rotation trajectory and other basic characteristics all conform to the standard action template, only the touch speed is slightly slower than the template, and such speed difference exists in the corresponding performer style characteristic record in the style vector library, the touch key action of the terminal sentence is classified as reasonable individualization deviation.

[0028] In another preferred embodiment of the application, a new action evaluation standard is formed according to the basic performance and learning progress of the learner, and the standard is provided to the principle and style discrimination module, so that the analysis process presents a process of stage difference: When collecting the basic performance and learning progress data of the learner, first, the initial action matching result of the learner's first core instrument performance action practice is obtained, which is generated by the principle and style discrimination module, including the specific type and number of error actions in the initial performance and the form of reasonable individualization deviation, which together constitute the basic performance of the learner. In the subsequent learning process, the action classification results after each practice of the learner are continuously recorded, including new error actions, corrected error actions, new reasonable individualization deviations and changes of original reasonable deviations. At the same time, the action execution difficulties and repeated action links in the practice process are collected. The practice feedback information is summarized, the classification results and the practice feedback system are summarized, and the complete learning progress data is formed, which fully reflects the change of the learner's action mastering situation.

[0029] According to the collected basic performance and learning progress data, the learning stage is divided. First, the initial learning stage is determined according to the basic performance. If the number of error actions in the basic performance of the learner is large, and the error actions are concentrated in the core basic actions such as holding posture and basic vocalization action, and there is little and unstable reasonable individualized deviation, the learner is divided into the entry stage. If the number of error actions in the core basic actions of the basic performance of the learner is small, and there is a small amount of stable reasonable individualized deviation, the learner is divided into the advanced stage. On this basis, the learning progress data is combined to further divide the new learning stage, and the trend of the reduction of error actions and the stability of reasonable individualized deviation are focused on. If the number of error actions of the learner in the entry stage is gradually reduced, the error actions in the core basic actions are basically eliminated, and the reasonable individualized deviation begins to show a fixed rule, the learner is divided into the advanced stage. If the error actions of the learner in the advanced stage only occur occasionally, the reasonable individualized deviation is stable and consistent with the characteristics of the corresponding genre, the learner is divided into the advanced stage.

[0030] For each learning stage divided, the matching threshold of basic action expression and the allowed difference range of individualized genre expression are adjusted to form the action evaluation standard of the corresponding stage. In the evaluation standard of the entry stage, the matching threshold of basic action expression is set to be consistent with the core characteristics of the standard action template, only a small range of deviation is allowed, and the allowed difference range of individualized genre expression is narrow, which focuses on the standardization of basic actions. In the evaluation standard of the advanced stage, the matching threshold of basic action expression is appropriately relaxed, which allows slight deviation on the premise that the core characteristics are consistent with the template, and the allowed difference range of individualized genre expression is expanded, which encourages the learner to form a preliminary genre tendency. In the evaluation standard of the advanced stage, the matching threshold of basic action expression is further optimized, which focuses on the standardization of action details, and the allowed difference range of individualized genre expression remains a moderate width, which supports the learner to develop unique expression on the basis of being consistent with the characteristics of the genre.

[0031] The action evaluation standards of each stage are completely transmitted to the principle and genre discrimination module, and the corresponding association between the stage and the standard is established in the module. When the learner practices, the stage adjustment module triggers the principle and genre discrimination module to call the corresponding action evaluation standard for analysis according to the learning stage currently in which the learner is located. In the entry stage analysis, whether the basic action is consistent with the core characteristics of the template is focused on, and the tolerance of individualized deviation is low. In the advanced stage analysis, both the slight deviation of the basic action and the genre expression within the range are concerned. In the advanced stage analysis, the accuracy of action details and the stability of genre expression are focused on, so that obvious differences in stage analysis are formed, which adapt to the learning needs of learners at different stages.

[0032] In another preferred embodiment of the present application, the process of the evaluation module receiving the classification result of the principle and genre discrimination module and translating the classification result into technical modification instructions or expression reservation prompts is: The evaluation module first receives the classification results output by the principle and style discrimination module, which clearly mark the specific action link identifiers in the learner's performance actions that belong to error actions and reasonable individualized deviations. The specific action link identifier refers to an independent action part with a clear function in the performance process, such as finger pressing action, wrist turning action, arm playing action, key touching or string touching action, etc. Each identifier corresponds to a distinguishable action stage in the performance process, ensuring that the evaluation module can accurately locate the action object that needs to be processed.

[0033] After obtaining the classification results, the evaluation module translates different information according to the action category. If it is determined to be an error action, the evaluation module extracts the five types of deviation parameters, including joint angle, motion speed, acceleration, electromyography, and instrument touch signal, from the historical data stored in the multi-source collection module when the error action occurs, and then converts these parameters into targeted technical correction instructions. For example, if the joint angle deviation is manifested as excessive bending of the finger joint, the correction instruction will clearly indicate that the bending degree of the finger joint needs to be adjusted; if the motion speed deviation is manifested as excessive motion speed, the correction instruction will indicate that the motion rate of the action needs to be slowed down; if the electromyography signal deviation is manifested as excessive muscle strength, the correction instruction will prompt to reduce the strength of the corresponding muscle group. If it is determined to be a reasonable individualized deviation, the evaluation module will associate the corresponding style elements in the style vector library with the same label according to the instrument and genre label of the action, and generate a retention prompt. The prompt content will clearly indicate that the deviation conforms to the genre style characteristics and the current action expression characteristics can be maintained.

[0034] After completing the translation of instructions and prompts, the evaluation module retrieves corresponding action examples for different categories of actions. For error actions, the evaluation module retrieves dynamic images consistent with the error action link identifier from the standard action templates constructed by the dual-channel training module as action examples. These dynamic images completely record the execution process of the standard action, including joint motion trajectory, muscle strength state, and instrument contact method, etc. for the learner to intuitively compare the differences between the error action and the standard action. For reasonable individualized deviations, the evaluation module selects action images with similar style characteristics from the stored action data of professional performers of the same genre as examples. These images demonstrate how different performers express the genre style through individualized action under the premise of conforming to the basic standard, helping learners understand the reasonableness and style value of the deviation.

[0035] Finally, the evaluation module forms the practice path for the two types of actions respectively. For the wrong actions, it is disassembled into progressive decomposition practice steps according to the order of joint movement, for example, a certain wrong action involves the movement deviation of two joints of fingers and wrists, the practice path will first set the practice step of adjusting the angle of the finger joint alone, and then set the practice step of the coordinated movement of the fingers and the wrist, and gradually correct the errors. For the reasonable individualized deviation, a coherent practice combination is designed according to its style characteristics, for example, a certain deviation reflects the unique change of the touch intensity, the practice path will select multiple phrases containing the intensity characteristics to form a coherent practice, so that the learner can consolidate and optimize the individualized expression in continuous practice, and the finally formed practice path will be directly provided to the learner to guide the targeted practice.

[0036] In another preferred embodiment of the application, in the teacher interface module, the receiving of the teacher's input style examples and the review annotation, when the teacher makes a label on the action category, the process of updating the style vector and adjusting the action analysis logic according to the label is: The teacher interface module builds a special information input and storage system, which receives two types of key teaching information. One is the style example action data, which is recorded or selected by the teacher according to the teaching goal, and contains complete joint angle, movement speed, acceleration, electromyography and instrument touch signal related to the performance action, which can accurately reflect the core style characteristics of a specific genre. The other is the review annotation of the learner's action category. The teacher will first check the preliminary classification result of the learner's action by the principle and style discrimination module, and confirm the accuracy of each action being labeled as a wrong action or a reasonable individualized deviation. If the preliminary classification does not match the actual teaching standard, the category label will be directly corrected to form the final review annotation. After the information is received, the module will strictly classify and store the style example action data and the review annotation content according to the preset instrument and genre tags, for example, the teacher's input of piano-classical style examples and corresponding review annotations will be classified into the "piano-classical" exclusive directory, and the relevant information of the Chinese zither-folk style will be classified into the "Chinese zither-folk" directory. The style examples and the annotation content under the same tag are related to each other, which ensures that they can be quickly matched when called later, avoiding information confusion.

[0037] When the teacher completes the review annotation of all learner action categories, the teacher interface module will automatically start the update process of the style vector library to ensure that the data in the library is consistent with the teaching standards. First, the module extracts the key action features directly related to the style expression from each action annotated by the teacher, including the rhythm of joint movement, the change of muscle force, the force fluctuation of instrument contact, and the core elements such as the transition mode of action. Taking the jazz piano split note action annotated by the teacher as an example, the wrist rotation rhythm features and the force change features of the fingers touching the keys are extracted. Then, the module compares these action features extracted with the existing vectors in the style vector library under the same instrument and genre label. If the extracted features conflict with an existing vector in the library, for example, the teacher's annotated classical violin bowing action features conflict with the force rhythm of the original bowing style vector under the same label in the library, and the teacher's annotated features are more consistent with the standard style expression in teaching, then the corresponding action features annotated by the teacher are used to replace the conflicting vector; if the extracted features are new style elements not included in the library under the same label.

[0038] While updating the style vector library, the teacher interface module will deeply mine the judgment criteria implied in the teacher's review annotation, and adjust the action analysis logic accordingly to make the subsequent action classification more in line with the actual teaching needs. The teacher's review annotation process will naturally reflect the specific basis for distinguishing between incorrect actions and reasonable individualized deviations, for example, the teacher annotates a learner's subtle bowing speed deviation as a reasonable individualized deviation, while the original analysis logic of the principle and style discrimination module may judge it as an incorrect action because the deviation is not recorded in the style vector library; or the teacher annotates a learner's touch angle deviation that does not conform to the basic playing specification as an incorrect action, while the original logic may misjudge it as a reasonable deviation. The module will accurately extract these distinguishing criteria and then adjust the matching logic of the principle and style discrimination module for basic action expression and individualized style expression, such as optimizing the priority of comparing the two types of expression features, increasing the weight of basic action features that affect the playing specification, and placing the comparison of individualized features after the matching of basic features. At the same time, the module will adjust the difference judgment parameters, such as appropriately relaxing the difference judgment scale of individualized style expression for genres that emphasize style diversity, and tightening the judgment scale of incorrect actions for instruments that emphasize basic specifications, so that the subsequent action classification logic of the principle and style discrimination module fully conforms to the teacher's teaching experience and professional judgment, ensuring that the classification results can accurately guide the learner's practice and improve the teaching effect.

[0039] The above describes one embodiment of the present application in detail, but the content described is only the preferred embodiment of the present application and cannot be considered as limiting the scope of the present application. Any equivalent changes and improvements made within the scope of the present application should still be included in the patent scope of the present application.

Claims

1. A music performance education platform based on motion-sensing interaction, characterized in that, include: The multi-source acquisition module is used to collect multi-dimensional data and construct a continuous temporal motion stream to reconstruct the performance trajectory; The dual-channel training module is used to build and maintain a library of standardized action templates and style vectors, and to train basic action expressions and personalized style expressions according to instrument and genre labels. The principles and style discrimination module is used to simultaneously analyze the matching of learners' actions in basic motor expression and personalized style expression. By comparing the differences between the two types of analysis results, learners' actions are classified as incorrect actions or reasonable personalized deviations. The phase adjustment module is used to form new action evaluation criteria based on learners’ basic performance and learning progress, and to provide these criteria to the principle and style discrimination module, so that the analysis process presents phased differences. The assessment module receives the classification results from the principles and style discrimination module, translates the classification results into technical correction instructions or expression retention prompts, and provides action examples and decomposed practice paths for learners to practice. The teacher interface module is used to receive style examples and review annotations input by teachers. After the teacher annotates the action categories, the style vector is updated and the action analysis logic is adjusted based on the annotations.

2. The music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the multi-source acquisition module, the multidimensional data includes joint angles, motion speed, acceleration, electromyography, and musical instrument tactile signals.

3. The music performance education platform based on motion-sensing interaction according to claim 2, characterized in that, The process of constructing a continuous temporal action flow to reconstruct the performance trajectory is as follows: An inertial measurement unit is deployed at the joints of the performer, electromyography sensors are attached to the surface of the muscles, and pressure sensors are installed at the contact points of the instrument. All sensors are triggered by the same hardware to start data acquisition. The joint angles, motion parameters, electromyographic signals, and tactile signals collected by each sensor are transmitted to the processing unit. Signal synchronization is achieved through hardware timestamp alignment, and then interference signals are removed through filtering algorithms. Using the synchronized signal time axis as a reference, joint angles and motion parameters are mapped to limb movement trajectories. Electromyographic signals and tactile signals are combined to supplement the details of the movements and form a continuous temporal movement flow.

4. A music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the dual-channel training module, the process of constructing and maintaining standardized action templates and style vector libraries, and training basic action expressions and personalized style expressions according to instrument and genre labels, is as follows: Collect standard movement data of professional performers of various instruments and styles, classify them by instrument and style tags, extract movement features to construct standardized movement templates, and extract style difference features to construct a style vector library. It receives new professional movement data and style examples input from the teacher interface module, compares them with the existing standardized movement templates and style vector library, replaces deviation data, and adds content corresponding to new tags to complete the maintenance. Based on standardized movement templates, train basic movement expression according to instrument and genre labels; Using a style vector library as a reference, we train personalized style expressions under the same label.

5. A music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the principle and style discrimination module, the process of simultaneously analyzing the matching of learners' actions in basic action expression and personalized style expression, and classifying learners' actions as incorrect actions or reasonable personalized deviations by comparing the differences between the two types of analysis results, is as follows: The feature data of learners' actions are extracted and compared with the standardized action templates to obtain the matching results of basic action expressions; at the same time, they are compared with the style vector library to obtain the matching results of personalized style expressions. By comparing the matching results of basic action expressions and personalized style expressions, we can identify the differences between the two in terms of action characteristics and determine the specific action steps where the differences lie. If the basic action expression does not match, regardless of the personalized style expression, the action is classified as an incorrect action; if the basic action expression matches, and the difference in personalized style expression is within the range of the style vector library, it is classified as a reasonable personalized deviation.

6. A music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the stage adjustment module, the process of forming new action evaluation criteria based on learners' basic performance and learning progress, and providing these criteria to the principle and style discrimination module to make the analysis process present stage-specific differences, is as follows: Collect learners’ initial action matching results as a basic performance, continuously record action classification results and practice feedback during the learning process, and summarize them to form learning progress data; The initial learning stage is determined based on basic performance, and new learning stages are divided based on the trend of decreasing errors and the stability of reasonable individual deviations during learning progress. For each learning stage, the matching threshold of basic movement expression and the allowable difference range of personalized style expression are adjusted to form the corresponding movement evaluation standard; The action evaluation criteria for each stage are transmitted to the principles and style discrimination module, so that the module can call the corresponding stage's criteria during analysis, thus forming stage-based analysis differences.

7. A music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the evaluation module, the process of receiving the classification results from the principle and style discrimination module and translating these results into technical correction instructions or expression retention prompts is as follows: The evaluation module receives the classification results output by the principle and style discrimination module, including the specific action links of erroneous actions and reasonable personalized deviations; If it is an incorrect action, extract the corresponding joint angle, movement speed, acceleration, electromyography, and instrument tactile signal deviation parameters, and translate them into targeted technical correction instructions; if it is a reasonable personalized deviation, associate it with the corresponding elements in the style vector library and generate expression preservation prompts. For incorrect movements, retrieve the corresponding dynamic video from the standard movement template as a movement example; for reasonable deviations, show videos of similar style movements by performers of the same school as examples. Incorrect movements are broken down into progressive practice steps according to the joint movement sequence, forming a path; reasonable personalized deviations are designed based on style characteristics to create coherent practice combinations, forming a path for learners to practice.

8. A music performance education platform based on motion-sensing interaction according to claim 1, characterized in that, In the teacher interface module, the process of receiving style examples and review annotations input by the teacher, and updating the style vector and adjusting the action analysis logic based on the annotations after the teacher annotates the action categories, is as follows: The teacher interface module receives style example action data input by the teacher and verification annotations of the learner's action categories, and stores the style examples and annotation content according to instrument and genre tags; Once the teacher completes the action category labeling, the action features corresponding to the labeling are extracted, compared with the same label vectors in the style vector library, conflicting vectors are replaced, and new features are added to generate new vectors. Based on the distinction between incorrect actions and reasonable deviations reflected in the teacher's annotations, the matching logic and difference judgment parameters of the two types of expressions in the adjustment principle and style judgment module are adjusted.

Images