Digital piano playing auxiliary system optimized by machine learning algorithm

Abstract

The invention relates to the technical field of piano playing, and provides a digital piano playing auxiliary system optimized by using a machine learning algorithm, the system can configure a piano music score of a to-be-played music of a player in a digital piano in advance, and determine demonstration playing information and accompaniment information for each pronunciation of the piano music score; carrying out accompaniment coupling on each pronunciation time of the accompaniment information and the pronunciation time of the demonstration playing information, and constructing an accompaniment coupling configuration model; performing playing coupling on the demonstration playing information and the pronunciation controllable parameters of the digital piano, and constructing a playing pronunciation coupling configuration model; setting playing auxiliary constraint conditions of the digital piano, determining playing auxiliary optimization data based on the accompaniment coupling configuration model and the playing pronunciation coupling configuration model, taking the playing auxiliary optimization data as sample parameters of a preset machine learning model for training, and generating a target machine learning model; the purpose of the present invention is to optimize playing music when a player performs playing.

Description

Technical Field

[0001] The present invention relates to the field of piano performance, and in particular to a digital piano performance auxiliary system optimized by a machine learning algorithm. Background Art

[0002] Currently, a digital piano, also known as an electric piano, is an electronic musical instrument that simulates the sound and feel of a traditional piano. Digital pianos have a variety of functions and features that make them very popular in music education and home entertainment.

[0003] In terms of the composition of a digital piano, the keyboard usually contains 88 keys, the same number of keys as a standard acoustic piano. The keyboard is equipped with a force sensor to detect the force of the player's key press to simulate the dynamic response of an acoustic piano, and is equipped with touch technology: such as weighted keyboards and hammer-type actions, to simulate the touch and feedback of an acoustic piano. The sound generator is set up inside to process the timbre, effects and audio signals, and store digital audio samples of different timbres and pitches. It is also equipped with a sound source, a built-in speaker and a headphone jack, and a pedal with a sustain function.

[0004] Because it has digital control, it can also realize a variety of timbres, such as piano, harpsichord, organ, guitar, etc. According to the strength of the player's keys, different volumes of sound are produced, simulating the dynamic changes of the original acoustic piano. A variety of rhythms and styles are built in, which can be played with the player. Allow users to record performances and play them back for evaluation and improvement.

[0005] A digital piano can assist performers in performing music. Patent CN201680051919.0 proposes an improved performance assistance device that allows users to easily and effectively read or master the pitch of each note shown on the music score without the need to accurately read the music score image, allowing users to easily practice partial performance of any desired part of a musical work.

[0006] However, it is impossible to adjust the control parameters of the digital piano during the performance, so that the music played by the performer can be more in line with the standard demonstration performance in terms of timbre, pitch and accompaniment, and it is impossible to enhance and optimize the performance of the performer. In addition, the main function of the digital piano is to help the performer learn, but the existing digital piano cannot accurately judge the performer's performance errors and optimization when the performer is learning. Summary of the invention

[0007] The present invention proposes a digital piano performance assistance system optimized by a machine learning algorithm, which is used to solve the problem that the control parameters of the digital piano cannot be adjusted during the performance, so that the music played by the performer can be more in line with the standard demonstrative performance in terms of timbre, pitch and accompaniment playback, and the performance effect of the performer cannot be enhanced and optimized.

[0008] The present invention proposes a digital piano performance assistance system optimized by a machine learning algorithm, comprising:

[0009] Demonstration performance module: used to configure the piano score of the music to be played by the performer in the digital piano in advance, and determine the demonstration performance information and accompaniment information for each pronunciation of the piano score;

[0010] Accompaniment coupling configuration module: used for performing accompaniment coupling on each pronunciation time of accompaniment information and the pronunciation time of demonstration performance information, so as to construct an accompaniment coupling configuration model;

[0011] Pronunciation coupling configuration module: used to perform performance coupling between demonstration performance information and controllable parameters of the digital piano's pronunciation, and to construct a performance pronunciation coupling configuration model;

[0012] Auxiliary variable determination module: used to set the performance auxiliary constraint conditions of the digital piano, and determine the performance auxiliary optimization data based on the accompaniment coupling configuration model and the performance pronunciation coupling configuration model, wherein the performance auxiliary optimization data includes the pronunciation coupling variable, the accompaniment coupling variable and the constraint variable;

[0013] Model training module: used to train the performance auxiliary optimization data as sample parameters of the preset machine learning model to generate a trained target machine learning model;

[0014] Performance assistance optimization module: used to optimize the performance of music through the target machine learning model when the performer is performing.

[0015] Further, the demonstration performance information is piano performance information of each note of the music to be performed;

[0016] The piano performance information includes key strength, timbre characteristics, volume value, soundboard resonance amplitude, effector parameters, metronome parameters, pedal parameters and pitch parameters.

[0017] The piano performance information corresponds to a standby trigger time, which is the trigger interval between two adjacent notes. Each trigger interval corresponds to a unique trigger key, which is determined by the trigger sequence of the digital piano keys on the piano score of the music to be played.

[0018] Further, the accompaniment information includes accompaniment trigger control information and accompaniment style setting information;

[0019] The accompaniment trigger control information includes synchronous trigger control information and asynchronous trigger control information, wherein the synchronous trigger control information is used to indicate that the accompaniment audio and the piano performance information are synchronously triggered by a unique trigger key, and the asynchronous trigger control information is used to indicate that the accompaniment audio information is automatically played according to the piano score after receiving the player's play instruction;

[0020] The accompaniment style setting information is set by the performer through the control terminal of the digital piano before playing.

[0021] Further, the performance coupling includes:

[0022] Set the corresponding coupling point according to each pronunciation in advance;

[0023] Traversing the pronunciation time of each pronunciation in the demonstration performance information according to the coupling point, and obtaining the accompaniment pronunciation and pronunciation order in the accompaniment information corresponding to each pronunciation time;

[0024] The coupling points corresponding to the same pronunciation time and pronunciation order are determined as target coupling points, and the accompaniment control device of the digital piano is controlled to play the corresponding accompaniment at the corresponding target coupling points.

[0025] Furthermore, the construction of the accompaniment coupling configuration model includes the following steps:

[0026] Obtaining a first accompaniment score with accompaniment marking corresponding to the piano score, wherein the first accompaniment score includes an accompaniment start time point, an accompaniment start mechanism, and accompaniment pronunciation parameters of the music to be played;

[0027] Injecting the first accompaniment music score into the training sample library to form a training sample;

[0028] Input the training samples in the training sample library into the preset neural network for training in batches of a preset number;

[0029] Obtaining the corresponding accompaniment coupling rule of the music to be played after training, wherein the accompaniment coupling rule includes determining the playing timing corresponding to each target coupling point;

[0030] The accompaniment coupling rules are classified and integrated according to the different accompaniment timbres of the music to be accompanied, so as to form a configuration model corresponding to different target coupling points of the music to be accompanied.

[0031] Further, the performance coupling includes:

[0032] Determine, based on the demonstration performance information, the controllable pronunciations and corresponding control mechanisms on the digital piano corresponding to each pronunciation;

[0033] According to the controllable pronunciations, a standard parameter of the controllable pronunciation corresponding to each pronunciation is calculated;

[0034] According to the control mechanism, calculating the adjustment interval of the control mechanism corresponding to each pronunciation;

[0035] Matching the adjustment interval of each pronunciation with the standard parameter according to the standard parameter, and generating a pronunciation compensation function;

[0036] According to the pronunciation compensation function, each pronunciation of the demonstration performance information is coupled with the control mechanism of the digital piano to generate a performance coupling compensation model;

[0037] Performance coupling is performed through a performance coupling compensation model.

[0038] Furthermore, the pronunciation coupling configuration model includes the following steps:

[0039] Obtaining configuration pronunciation requirement information and pronunciation controllable range of each pronunciation of the music to be played;

[0040] Determine one or more control mechanisms in the digital piano and a pronunciation compensation function corresponding to the control mechanism based on the pronunciation demand information and the pronunciation controllable interval;

[0041] Based on the pronunciation demand information, determining the connection relationship between the pronunciation compensation functions of the different pronunciation notes and the order relationship between the pronunciation compensation functions;

[0042] Based on the pronunciation notes of each pronunciation of the music to be played, the pronunciation controllable interval, the connection relationship between the pronunciation compensation functions of different pronunciation notes, and the sequential relationship between the pronunciation compensation functions, a pronunciation coupling configuration model is generated according to the note sequence of the piano score.

[0043] Furthermore, the performance assistance constraints include:

[0044] Key velocity constraints, pitch constraints, timbre selection constraints, rhythm constraints, dynamic layering constraints, automatic accompaniment style constraints, performance speed constraints and volume control constraints.

[0045] Furthermore, the target machine learning model is also used to:

[0046] According to the piano score of the music to be played, a random pronunciation verification factor is randomly generated at any time within a preset pronunciation range;

[0047] Generate a performance directional pronunciation feature graph according to the demonstration performance information and the piano score;

[0048] The random pronunciation verification factor is used as the performance pronunciation, and the optimized target pronunciation is obtained by optimizing the target pronunciation through the target and its learning model;

[0049] Determine whether the target pronunciation belongs to any feature in the performance directional pronunciation feature graph and satisfies the same pronunciation time;

[0050] If so, it is determined that the current target machine learning model meets the preset optimization requirements.

[0051] Furthermore, the optimization of the music performance also includes:

[0052] Obtain the piano pressing force of the performer when playing the piano score, and perform force stratification;

[0053] Based on the velocity layer, each pronounced note is set according to the preset timbre composition and playing style, and the corresponding piano key touch sensitivity is set.

[0054] The beneficial effects of the present invention are:

[0055] The present invention is applicable to the situation of the performance of the performer learning and the performance optimization of the performer in real time. The demonstration performance module can provide personalized demonstration performance according to the needs and level of the performer, so as to enhance the auxiliary effect and determine the performance goal during learning; the accompaniment coupling configuration module ensures that the accompaniment is synchronized with the demonstration performance, provides a more harmonious and accurate accompaniment, and can adapt the rhythm and music structure when the performer plays, enhance the richness of the performance, and can also learn more accompaniment styles when learning, making it easier for the performer to learn the coordinated performance of different accompaniments. The pronunciation coupling configuration module combines the demonstration performance with the pronunciation parameters of the digital piano, so that the performer can master more accurate timbre and pronunciation skills when playing, that is, when learning, and can also perform the pronunciation optimization control of the auxiliary performance. The auxiliary variable determination module adjusts the optimization data according to the actual situation of the performer through the set performance auxiliary constraint conditions, so that the auxiliary function of the digital piano is more in line with the needs of the performer. The performance auxiliary optimization module can optimize the performance in real time during the performance, help the performer correct errors and improve the performance quality.

[0056] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0057] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0059] Figure 1A system composition diagram of a digital piano performance assistance system optimized by a machine learning algorithm in an embodiment of the present invention;

[0060] Figure 2 A control flow chart of performance coupling in an embodiment of the present invention;

[0061] Figure 3 This is a flow chart for determining the optimization capability of a machine learning model in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0063] The present invention provides a performance assistance system for a digital piano. The present invention mainly assists a performer in two situations: when the performer is performing and when the performer is learning. When the performer is performing, the system helps the performer to provide accompaniment and required piano timbre, etc.; when learning, the system can accurately judge the differences between demonstration performance information and the music performed in the embodiment during the practice process, thereby judging the deficiencies and defects of the performer, and displaying them to the performer through a terminal device or a display screen.

[0064] like Figure 1 As shown, this embodiment provides a digital piano performance assistance system optimized by machine learning algorithm, including:

[0065] Demonstration performance module: used to configure the piano score of the music to be played by the performer in the digital piano in advance, and determine the demonstration performance information and accompaniment information for each pronunciation of the piano score;

[0066] Accompaniment coupling configuration module: used for performing accompaniment coupling on each pronunciation time of accompaniment information and the pronunciation time of demonstration performance information, so as to construct an accompaniment coupling configuration model;

[0067] Pronunciation coupling configuration module: used to perform performance coupling between demonstration performance information and controllable parameters of the digital piano's pronunciation, and to construct a performance pronunciation coupling configuration model;

[0068] Auxiliary variable determination module: used to set the performance auxiliary constraint conditions of the digital piano, and determine the performance auxiliary optimization data based on the accompaniment coupling configuration model and the performance pronunciation coupling configuration model, wherein the performance auxiliary optimization data includes the pronunciation coupling variable, the accompaniment coupling variable and the constraint variable;

[0069] Model training module: used to train the performance auxiliary optimization data as sample parameters of the preset machine learning model to generate a trained target machine learning model;

[0070] Performance assistance optimization module: used to optimize the performance of music through the target machine learning model when the performer is performing.

[0071] The principle of the above technical solution is:

[0072] The first part of the present invention is a demonstration performance module, which mainly provides a demonstration performance music and sets the best performance standard for the performer to play the music. The piano score of the music to be played by the performer is pre-set inside the digital piano. Then the score information of each music to be played is processed, and this processing includes note analysis, pronunciation strength analysis, timbre analysis, note connection time analysis and accompaniment analysis, etc., for the pronunciation corresponding to each instrument, so as to determine the corresponding demonstration performance information and accompaniment information. The demonstration performance information and accompaniment information are reference and comparison information for the performer during the performance process, and can timely discover the deviation of the performer during the performance when playing this performance.

[0073] In addition to providing demonstration performance information, the present invention also has an accompaniment coupling configuration module, which couples each pronunciation time of the accompaniment information with the pronunciation time of the demonstration performance information during the specific implementation process, thereby constructing an accompaniment coupling configuration model. The accompaniment coupling configuration model enables the accompaniment to closely follow the demonstration performance, improves the overall coordination of the performance, and assists the performer in mastering the structure of the music when the performer is performing, so that the performer can more completely understand the entire performance data.

[0074] In specific implementation, the pronunciation coupling configuration module of the present invention is used to perform performance coupling between the demonstration performance information and the pronunciation controllable parameters of the digital piano to construct a pronunciation coupling configuration model. The pronunciation controllable parameters are control parameters on the digital piano that control the timbre, pitch, volume, etc., so as to combine the information of the demonstration performance with various functional mechanisms of the pronunciation control of the digital piano. During the performance process, when the pronunciation is made, the correlation between the different pronunciation controllable parameters of the digital piano and the demonstration performance information can be determined, so as to ensure that the performer can accurately judge the difference between the strength of the performer's performance, the interval time between notes, and the performance accuracy and the standard performance situation when performing.

[0075] In specific implementation, the auxiliary variable determination module determines the performance auxiliary optimization data by setting the performance auxiliary constraint conditions of the digital piano and on the basis of the accompaniment coupling configuration model and the performance pronunciation coupling configuration model. These optimization data include pronunciation coupling variables, accompaniment coupling variables and constraint variables. The setting of these variables will provide training data for subsequent model training. In order to ensure that the data that needs to be optimized in the real-time performance data of the performer can be accurately determined when the performer is performing, by setting a lot of different conditions, in the process of the performer's performance, very excessive performance data is generated, and continuous data judgment that needs to be optimized is realized, and those data of these performance data that need to be optimized are judged, thereby determining the specific data that needs to be optimized.

[0076] In the specific training process, the model training module uses the existing performance-assisted optimization data to preset a machine learning model and conduct training. This machine learning model is a performance optimization model that can optimize the music during the performance. Its training process is the same as the conventional machine learning model training method, except that the data used is performance-assisted optimization data.

[0077] Finally, the performance auxiliary optimization module of the present invention optimizes the music actually played by using the trained target machine learning model when the performer is performing, and outputs the optimization data. Optimizing the music is to achieve real-time optimization during the performance of the performer, and the output optimization data can assist the performer in discovering his or her performance errors while learning.

[0078] The beneficial effects of the above technical solution are:

[0079] The present invention is applicable to the situation of the performance of the performer learning and the performance optimization of the performer in real time. The demonstration performance module can provide personalized demonstration performance according to the needs and level of the performer, so as to enhance the auxiliary effect and determine the performance goal during learning; the accompaniment coupling configuration module ensures that the accompaniment is synchronized with the demonstration performance, provides a more harmonious and accurate accompaniment, and can adapt the rhythm and music structure when the performer plays, enhance the richness of the performance, and can also learn more accompaniment styles when learning, making it easier for the performer to learn the coordinated performance of different accompaniments. The pronunciation coupling configuration module combines the demonstration performance with the pronunciation parameters of the digital piano, so that the performer can master more accurate timbre and pronunciation skills when playing, that is, when learning, and can also perform the pronunciation optimization control of the auxiliary performance. The auxiliary variable determination module adjusts the optimization data according to the actual situation of the performer through the set performance auxiliary constraint conditions, so that the auxiliary function of the digital piano is more in line with the needs of the performer. The performance auxiliary optimization module can optimize the performance in real time during the performance, help the performer correct errors and improve the performance quality.

[0080] As an embodiment of the present invention: the demonstration performance information is piano performance information of each note of the music to be performed;

[0081] The piano performance information includes key strength, timbre characteristics, volume value, soundboard resonance amplitude, effector parameters, metronome parameters, pedal parameters and pitch parameters.

[0082] The piano performance information corresponds to a standby trigger time, which is the trigger interval between two adjacent notes. Each trigger interval corresponds to a unique trigger key, which is determined by the trigger sequence of the digital piano keys on the piano score of the music to be played.

[0083] The principle of the above technical solution is:

[0084] In the specific implementation process, the demonstration performance information of each note of the present invention is considered to be a part of the music to be played. The demonstration performance information includes information such as key strength, timbre characteristics, volume value, soundboard resonance amplitude, effector parameters, metronome parameters, pedal parameters and pitch parameters. The demonstration performance information not only contains the basic performance characteristics of the notes, but also contains the time relationship between the notes. This time relationship is configured as a "standby trigger time", which is the trigger interval time between two adjacent notes, and each trigger interval time has a unique trigger key, and the trigger key is determined by the trigger sequence of the numeric keyboard on the piano score of the music to be played. When the performer plays the music, according to the performance score, when the trigger sequence is the corresponding note within the corresponding trigger interval time of the corresponding sequence, the corresponding performance information responds in real time, thereby accurately assisting the performer to perform.

[0085] The beneficial effects of the above technical solution are:

[0086] The demonstration performance information is configured with corresponding triggering sequence, triggering note keys and standby triggering time according to the performance score, which can ensure accurate judgment of the performer's performance errors during the performance. At the same time, when the performer needs auxiliary tuning and accompaniment, the tuning accompaniment effect will be better.

[0087] As an embodiment of the present invention: the accompaniment information accompaniment trigger control information and accompaniment style setting information;

[0088] The accompaniment trigger control information includes synchronous trigger control information and asynchronous trigger control information, wherein the synchronous trigger control information is used to indicate that the accompaniment audio and the piano performance information are synchronously triggered by a unique trigger button, and the asynchronous trigger control information is used to indicate that the accompaniment audio information is automatically played according to the piano score after receiving the play instruction of the performer;

[0089] The accompaniment style setting information is set by the performer through the control terminal of the digital piano before playing.

[0090] The principle of the above technical solution is:

[0091] In the specific implementation process, the present invention includes not only the demonstration performance information of the notes, but also an important component - accompaniment information. The accompaniment information includes two parts: one is the accompaniment trigger control information, and the other is the accompaniment style setting information.

[0092] There are two types of accompaniment trigger control information: synchronous trigger control information and asynchronous trigger control information. The function of synchronous trigger control information is to ensure that the accompaniment audio and piano performance information are triggered synchronously through the same trigger button, ensuring that the two change synchronously to form a harmonious performance effect. The asynchronous trigger control information is used to allow the accompaniment audio information to automatically play according to the piano score after receiving the performer's playback instruction, achieving a perfect combination of accompaniment and performance. The performer sets the accompaniment style in advance through the control terminal of the digital piano. The control terminal includes the network terminal device built into the digital piano and the mobile terminal controlled through the APP.

[0093] The beneficial effects of the above technical solution are:

[0094] The accompaniment information of the present invention can be quickly triggered, and when the performer plays, the accompaniment type can be freely set, such as electronic music accompaniment, pop accompaniment or folk accompaniment, etc. When the accompaniment is triggered, the corresponding accompaniment notes are triggered according to the performance rate of the performer, so that the accompaniment can fit the performance of the performer.

[0095] As an embodiment of the present invention: the performance coupling includes:

[0096] Set the corresponding coupling point according to each pronunciation in advance;

[0097] Traversing the pronunciation time of each pronunciation in the demonstration performance information according to the coupling point, and obtaining the accompaniment pronunciation and pronunciation order in the accompaniment information corresponding to each pronunciation time;

[0098] The coupling points corresponding to the same pronunciation time and pronunciation order are determined as target coupling points, and the accompaniment control device of the digital piano is controlled to play the corresponding accompaniment at the corresponding target coupling points.

[0099] The principle of the above technical solution is:

[0100] In the specific implementation process, the present invention sets the corresponding coupling point according to each preset pronunciation. Then, the system traverses the time of each pronunciation in the demonstration performance information, and obtains the accompaniment pronunciation and pronunciation order in the accompaniment information corresponding to each pronunciation time. Then, those coupling points with the same pronunciation time and pronunciation order are determined as target coupling points, and the accompaniment control device of the digital piano is controlled to play the corresponding accompaniment at these target coupling points. Through the precise coupling playback of the accompaniment, the accompaniment of the digital piano can follow the demonstration performance more accurately, achieving a more natural music coordination.

[0101] The beneficial effects of the above technical solution are:

[0102] The present invention can make the pronunciation time and pronunciation order of the accompaniment be played in a coupled manner according to the target coupling point when the accompaniment is pronounced, accurately matching the performance order of the performer, and at the same time, because the entire demonstration performance information is traversed, inaccurate accompaniment pronunciation can be prevented.

[0103] As an embodiment of the present invention, the construction of the accompaniment coupling configuration model includes the following steps:

[0104] Obtaining a first accompaniment score with accompaniment marking corresponding to the piano score, wherein the first accompaniment score includes an accompaniment start time point, an accompaniment start mechanism, and accompaniment pronunciation parameters of the music to be played;

[0105] Injecting the first accompaniment music score into the training sample library to form a training sample;

[0106] Input the training samples in the training sample library into the preset neural network for training in batches of a preset number;

[0107] Obtaining the corresponding accompaniment coupling rule of the music to be played after training, wherein the accompaniment coupling rule includes determining the playing timing corresponding to each target coupling point;

[0108] The accompaniment coupling rules are classified and integrated according to the different accompaniment timbres of the music to be accompanied, so as to form an accompaniment coupling configuration model corresponding to different target coupling points of the music to be accompanied.

[0109] The principle of the above technical solution is:

[0110] In the specific implementation process, the present invention first obtains the first accompaniment score with accompaniment annotation corresponding to the piano score. This first accompaniment score contains information such as the accompaniment start time point, accompaniment start mechanism and accompaniment pronunciation parameters of the music to be played. Then, this information is injected into the training sample library to form training samples. Then, these training samples are input into the preset neural network in batches according to a preset number for training.

[0111] After the training is completed, the corresponding accompaniment coupling rules of the music to be played will be obtained. The accompaniment coupling rules include the specific content of judging the corresponding playback timing of each target coupling point. Finally, these coupling rules will be classified and integrated according to the different accompaniment timbres of the music to be accompanied, thus forming a configuration model corresponding to different target coupling points of the music to be accompanied. In this way, the control of the accompaniment can be adjusted according to the actual situation to achieve more accurate music coordination.

[0112] The beneficial effects of the above technical solution are:

[0113] The present invention can determine the playing timing of accompaniment notes by constructing an accompaniment coupling configuration model, thereby achieving accurate coupling of accompaniment information, accurate playback, and performance of coupling timing.

[0114] As an embodiment of the present invention: the performance coupling includes:

[0115] Determine, based on the demonstration performance information, the controllable pronunciations and corresponding control mechanisms on the digital piano corresponding to each pronunciation;

[0116] According to the controllable pronunciations, a standard parameter of the controllable pronunciation corresponding to each pronunciation is calculated;

[0117] According to the control mechanism, calculating the adjustment interval of the control mechanism corresponding to each pronunciation;

[0118] Matching the adjustment interval of each pronunciation with the standard parameter according to the standard parameter, and generating a pronunciation compensation function;

[0119] According to the pronunciation compensation function, each pronunciation of the demonstration performance information is coupled with the control mechanism of the digital piano to generate a performance coupling compensation model;

[0120] Performance coupling is performed through a performance coupling compensation model.

[0121] The principle of the above technical solution is:

[0122] like Figure 2 As shown, the present invention determines the controllable pronunciations and corresponding control mechanisms on the digital piano corresponding to each pronunciation according to the demonstration performance information. Then, the standard parameters of the controllable pronunciations corresponding to each pronunciation are calculated according to these controllable pronunciations.

[0123] Next, based on the control mechanism, the adjustment interval of the control mechanism corresponding to each pronunciation is calculated. Then, the adjustment interval of each pronunciation is matched with the standard parameters according to the standard parameters, and a pronunciation compensation function is generated. The pronunciation compensation function is used to describe the correspondence between each pronunciation and the control mechanism of the digital piano. In the specific implementation process, the pronunciation compensation function is a technology used for speech synthesis and speech recognition. It can adjust the characteristics of the pronunciation timbre and the like according to the specific standard parameters preset in the digital piano to generate a more natural or more accurate performance output. In the specific implementation process of the pronunciation compensation function, there are different compensation functions for pitch, length, timbre, etc.; for example: the compensation function for pitch is:

[0124] C 音高 (x) = a*x+p, where a is the slope parameter of the pitch change; p is the pitch offset parameter; x is the original pitch value of the note played by the performer in real time; C 音高 (x) is the pitch value after compensation. Compensation is performed one by one for the length of the sound, timbre, etc.

[0125] According to the pronunciation compensation function, each pronunciation of the demonstration performance information is coupled with the control mechanism of the digital piano (the mechanism that controls the sound of the digital piano, such as the timbre control mechanism, the tone length control mechanism and the pitch control mechanism, etc.). This process optimizes and adjusts each pronunciation and the control mechanism of the digital piano through the generated pronunciation compensation function. In this way, the performer can more accurately match the effect of the demonstration performance when performing, achieve the purpose of optimizing the performance, and accurately output the performance optimization data to determine the deviation of the performer during the performance.

[0126] As an embodiment of the present invention, it comprises the following steps:

[0127] Obtaining configuration pronunciation requirement information and pronunciation controllable range of each pronunciation of the music to be played;

[0128] Determine one or more control mechanisms in the digital piano and a pronunciation compensation function corresponding to the control mechanism based on the pronunciation demand information and the pronunciation controllable interval;

[0129] Based on the pronunciation demand information, determining the connection relationship between the pronunciation compensation functions of the different pronunciation notes and the order relationship between the pronunciation compensation functions;

[0130] Based on the pronunciation notes of each pronunciation of the music to be played, the pronunciation controllable interval, the connection relationship between the pronunciation compensation functions of different pronunciation notes, and the sequential relationship between the pronunciation compensation functions, a pronunciation coupling configuration model is generated according to the note sequence of the piano score.

[0131] The principle of the above technical solution is:

[0132] In specific implementation, the present invention will first obtain the configuration pronunciation requirement information and pronunciation controllable interval (i.e., the controllable interval of different control mechanisms in sounding, such as pitch interval) of each pronunciation of the music to be played, which is used to construct a pronunciation coupling configuration model. Then, according to the acquired pronunciation requirement information and pronunciation controllable interval, determine one or more control mechanisms in the digital piano and the pronunciation compensation function corresponding to the control mechanism. It establishes an association between the pronunciation and the corresponding control mechanism and the adjustment interval of the control mechanism, and finds the best compensation function for each pronunciation. Then, according to the pronunciation requirement information, determine the connection relationship between the pronunciation compensation functions of different pronunciation notes and the order relationship between the pronunciation compensation function and the pronunciation compensation function. Thereby determining the consistency and correctness between the pronunciation compensation functions of adjacent pronunciation notes. Finally, according to the pronunciation notes of each pronunciation of the music to be played, the pronunciation controllable interval, the connection relationship between the pronunciation compensation functions of different pronunciation notes, and the order relationship between the pronunciation compensation function and the pronunciation compensation function, according to the note sequence of the piano score, generate a pronunciation coupling configuration model. Through the pronunciation coupling configuration model, the key factors of the performance optimization effect are determined during the performance of the performer.

[0133] As an embodiment of the present invention: the performance assistance constraint conditions include:

[0134] Key velocity constraints, pitch constraints, timbre selection constraints, rhythm constraints, dynamic layering constraints, automatic accompaniment style constraints, performance speed constraints and volume control constraints.

[0135] The principle of the above technical solution is:

[0136] Key velocity constraint: Specifies the velocity range for each note. Performances outside this range will be considered irregular.

[0137] High pitch constraint: limits the highest point of the note. Playing beyond this point will be considered as irregular performance.

[0138] Timbre selection constraint: specifies the timbre that a note should select. If the performer's performance deviates from this timbre, it will be considered an irregular performance.

[0139] Rhythmic constraints: stipulate the interval time between notes and the rhythmic rules of performance. If the performer deviates from these rules, it will be regarded as irregular performance.

[0140] Dynamic layering constraints: stipulate the intensity and duration of notes. If the intensity or duration of the performance does not meet the requirements, it will be regarded as an irregular performance.

[0141] Automatic accompaniment style constraints: specify the style and rules of the accompaniment. If the accompaniment style or rules of the performance do not meet the requirements, it will be regarded as an irregular performance.

[0142] Performance speed constraint: It specifies the performance speed of the notes. If the performance speed does not meet the requirements, it will be regarded as an irregular performance.

[0143] Volume control constraint: specifies the volume range of notes. If the volume of the performance exceeds this range, it will be regarded as an irregular performance.

[0144] As an embodiment of the present invention: the target machine learning model is also used for:

[0145] According to the piano score of the music to be played, a random pronunciation verification factor is randomly generated at any time within a preset pronunciation range;

[0146] Generate a performance directional pronunciation feature graph according to the demonstration performance information and the piano score;

[0147] The random pronunciation verification factor is used as the performance pronunciation, and the optimized target pronunciation is obtained by optimizing the target pronunciation through the target and its learning model;

[0148] Determine whether the target pronunciation belongs to any feature in the performance directional pronunciation feature graph and satisfies the same pronunciation time;

[0149] If so, it is determined that the current target machine learning model meets the preset optimization requirements.

[0150] The principle of the above technical solution is:

[0151] like Figure 3 As shown, the present invention randomly generates a random pronunciation verification factor at any time within a preset pronunciation range according to the piano score of the music to be played. According to the specific situation of the music, a new pronunciation is randomly generated within the preset pronunciation range to verify the optimization performance of the target machine learning model.

[0152] According to the demonstration performance information and the piano score, a performance directed pronunciation feature map is generated. This is a feature map about pronunciation obtained by analyzing the demonstration performance information and the piano score, which shows the relationship between different pronunciations.

[0153] The random pronunciation verification factor is used as the performance pronunciation, and the optimized target pronunciation is obtained through the target and its learning model. The pronunciation is optimized based on the randomly generated pronunciation verification factor and the analysis results of the demonstration performance information and piano score.

[0154] Determine whether the target pronunciation belongs to any feature in the performance directional pronunciation feature graph and satisfies the same pronunciation time. It can ensure that the pronunciation we optimize conforms to the characteristics of the music itself, rather than being randomly generated.

[0155] If yes, then it means that the target machine learning model meets the preset optimization requirements. This is an evaluation phase, which can tell whether the model has really achieved the expected effect. If not, the target machine learning model does not meet the expected effect and needs to be retrained.

[0156] As an embodiment of the present invention: the optimization of the music to be performed further comprises:

[0157] Obtain the piano pressing force of the performer when playing the piano score, and perform force stratification;

[0158] Based on the velocity layer, each pronounced note is set according to the preset timbre composition and playing style, and the corresponding piano key touch sensitivity is set.

[0159] The principle of the above technical solution is:

[0160] The piano pressing force of the performer when playing the piano score is obtained and the force is layered to obtain the change of the performer's force during the performance, so as to more accurately adapt the performer's force during the optimization process. To this end, the piano pressing force data of the performer is obtained through a pressure sensor.

[0161] According to the force layering, each note is set to the corresponding piano key touch sensitivity according to the preset timbre composition and playing style. This makes the played music more in line with the original music style and artistic expression, and also adapts to the player's force control habits. Therefore, it is necessary to set the piano key touch sensitivity corresponding to each note according to the result of force layering and the preset timbre composition to achieve this goal.

[0162] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. A digital piano performance assistance system optimized by machine learning algorithm is characterized by: include: Demonstration performance module: used to configure the piano score of the music to be played by the performer in the digital piano in advance, and determine the demonstration performance information and accompaniment information for each pronunciation of the piano score; Accompaniment coupling configuration module: used for performing accompaniment coupling on each pronunciation time of accompaniment information and the pronunciation time of demonstration performance information, so as to construct an accompaniment coupling configuration model; Pronunciation coupling configuration module: used to perform performance coupling between demonstration performance information and controllable parameters of the digital piano's pronunciation, and to construct a performance pronunciation coupling configuration model; Auxiliary variable determination module: used to set the performance auxiliary constraint conditions of the digital piano, and determine the performance auxiliary optimization data based on the accompaniment coupling configuration model and the performance pronunciation coupling configuration model, wherein the performance auxiliary optimization data includes the pronunciation coupling variable, the accompaniment coupling variable and the constraint variable; Model training module: used to train the performance auxiliary optimization data as sample parameters of the preset machine learning model to generate a trained target machine learning model; Performance auxiliary optimization module: It is used to optimize the performance of music through the target machine learning model when the performer is performing, and output the optimization data.

2. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The demonstration performance information is piano performance information of each note of the music to be performed; The piano performance information includes key strength, timbre characteristics, volume value, soundboard resonance amplitude, effector parameters, metronome parameters, pedal parameters and pitch parameters; The piano performance information corresponds to a standby trigger time, which is the trigger interval between two adjacent notes. Each trigger interval corresponds to a unique trigger key, which is determined by the trigger sequence of the digital piano keys on the piano score of the music to be played.

3. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 2, characterized in that: The accompaniment information, accompaniment trigger control information and accompaniment style setting information; The accompaniment trigger control information includes synchronous trigger control information and asynchronous trigger control information, wherein the synchronous trigger control information is used to indicate that the accompaniment audio and the piano performance information are synchronously triggered by a unique trigger button, and the asynchronous trigger control information is used to indicate that the accompaniment audio information is automatically played according to the piano score after receiving the play instruction of the performer; The accompaniment style setting information is set by the performer through the control terminal of the digital piano before playing.

4. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The performance coupling comprises: Set the corresponding coupling point according to each pronunciation in advance; Traversing the pronunciation time of each pronunciation in the demonstration performance information according to the coupling point, and obtaining the accompaniment pronunciation and pronunciation order in the accompaniment information corresponding to each pronunciation time; The coupling points corresponding to the same pronunciation time and pronunciation order are determined as target coupling points, and the accompaniment control device of the digital piano is controlled to play the corresponding accompaniment at the corresponding target coupling points.

5. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 4, characterized in that: The construction of the accompaniment coupling configuration model includes the following steps: Obtaining a first accompaniment score with accompaniment marking corresponding to the piano score, wherein the first accompaniment score includes an accompaniment start time point, an accompaniment start mechanism, and accompaniment pronunciation parameters of the music to be played; Injecting the first accompaniment music score into the training sample library to form a training sample; Input the training samples in the training sample library into the preset neural network for training in batches of a preset number; Obtaining the corresponding accompaniment coupling rule of the music to be played after training, wherein the accompaniment coupling rule includes determining the playing timing corresponding to each target coupling point; The accompaniment coupling rules are classified and integrated according to the different accompaniment timbres of the music to be accompanied, so as to form an accompaniment coupling configuration model corresponding to different target coupling points of the music to be accompanied.

6. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The performance coupling comprises: Determine, based on the demonstration performance information, the controllable pronunciations and corresponding control mechanisms on the digital piano corresponding to each pronunciation; According to the controllable pronunciations, a standard parameter of the controllable pronunciation corresponding to each pronunciation is calculated; According to the control mechanism, calculating the adjustment interval of the control mechanism corresponding to each pronunciation; Matching the adjustment interval of each pronunciation with the standard parameter according to the standard parameter, and generating a pronunciation compensation function; According to the pronunciation compensation function, each pronunciation of the demonstration performance information is coupled with the control mechanism of the digital piano to generate a performance coupling compensation model; Performance coupling is performed through a performance coupling compensation model.

7. A digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The pronunciation coupling configuration model comprises the following steps: Obtaining configuration pronunciation requirement information and pronunciation controllable range of each pronunciation of the music to be played; Determine one or more control mechanisms in the digital piano and a pronunciation compensation function corresponding to the control mechanism based on the pronunciation demand information and the pronunciation controllable interval; Based on the pronunciation demand information, determining the connection relationship between the pronunciation compensation functions of the different pronunciation notes and the order relationship between the pronunciation compensation functions; Based on the pronunciation notes of each pronunciation of the music to be played, the pronunciation controllable interval, the connection relationship between the pronunciation compensation functions of different pronunciation notes, and the sequential relationship between the pronunciation compensation functions, a pronunciation coupling configuration model is generated according to the note sequence of the piano score.

8. The digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The performance assistance constraints include: Key velocity constraints, pitch constraints, timbre selection constraints, rhythm constraints, dynamic layering constraints, automatic accompaniment style constraints, performance speed constraints and volume control constraints.

9. The digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The target machine learning model is also used to: According to the piano score of the music to be played, a random pronunciation verification factor is randomly generated at any time within a preset pronunciation range; Generate a performance directional pronunciation feature graph according to the demonstration performance information and the piano score; The random pronunciation verification factor is used as the performance pronunciation, and the optimized target pronunciation is obtained by optimizing the target pronunciation through the target and its learning model; Determine whether the target pronunciation belongs to any feature in the performance directional pronunciation feature graph and satisfies the same pronunciation time; If so, it is determined that the current target machine learning model meets the preset optimization requirements.

10. The digital piano performance assistance system optimized by machine learning algorithm as claimed in claim 1, characterized in that: The optimization of the music performance also includes: Obtain the piano pressing force of the performer when playing the piano score, and perform force stratification; Based on the velocity layer, each pronounced note is set according to the preset timbre composition and playing style, and the corresponding piano key touch sensitivity is set.

Images