Method and related device for processing auxiliary training information in musical instrument practice scene

Abstract

The present application provides a method and related device for processing auxiliary training information in a musical instrument practice scenario, the method comprising: responding to a confirmation operation for a first music score and a first musical instrument in a training event, obtaining a first control parameter set including a plurality of first parameter groups and a plurality of first time parameters, each first parameter group including a first motion parameter of a first joint of a manipulator, the first motion parameter and the first time parameter being determined according to the pitch information, duration information and the first musical instrument of a note in the first music score; configuring the first joint of the manipulator according to the first control parameter set; obtaining the first audio data detected in the training event; and outputting the practice record information including the error position according to the first audio data. The present application is conducive to improving the accuracy of auxiliary training information processing in a musical instrument practice scenario and the intelligence of the manipulator.

Description

Technical Field

[0001] The present application relates to the field of manipulator control, and in particular to a method and a related device for processing auxiliary training information in a musical instrument practice scenario. Background Art

[0002] With the development of computer technology and automation technology, the application of various intelligent electromechanical devices has become increasingly common, and has facilitated the daily lives of users to a certain extent. However, in the current instrument practice scenario, instrument learners still mainly rely on instructors to accompany and guide them. Without an instructor, it is difficult for learners to accurately judge whether their performance is accurate, resulting in poor practice results. Therefore, how to improve the intelligence of intelligent electromechanical devices and improve the accuracy of auxiliary training information processing in instrument practice scenarios through intelligent electromechanical devices has become a problem that needs to be solved at present. Summary of the invention

[0003] The embodiments of the present application provide a method and a related device for processing auxiliary training information in a musical instrument practice scenario, in order to improve the accuracy of auxiliary training information processing and the intelligence of a manipulator.

[0004] In a first aspect, an embodiment of the present application provides a method for processing auxiliary training information in a musical instrument practice scenario, which is applied to a computing device in a hand function training system, wherein the hand function training system includes the computing device and a manipulator worn by a user; the method includes:

[0005] In response to a confirmation operation for a first music score and a first musical instrument in a currently processed training event, a first control parameter set of the manipulator is acquired, wherein the first music score includes a plurality of first notes, the first control parameter set includes a plurality of first parameter groups and a plurality of first time parameters corresponding to the plurality of first notes, each first parameter group includes a first motion parameter of a first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitch information and duration information of the plurality of first notes;

[0006] configuring the first joint of the manipulator according to the first control parameter set, wherein the movement time of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group;

[0007] Acquire the first audio data detected in the currently processed training event;

[0008] Output practice record information based on the first audio data, the practice record information including an error position, the error position being used to indicate an audio position where the pitch does not match between the first audio data and the target audio data, the target audio data being standard audio data corresponding to the first music score and the first musical instrument.

[0009] In a second aspect, an embodiment of the present application provides a device for processing auxiliary training information in a musical instrument practice scenario, which is applied to a computing device in a hand function training system, wherein the hand function training system includes the computing device and a manipulator worn by a user; the device includes:

[0010] A first acquisition unit is used to acquire a first control parameter set of the manipulator in response to a confirmation operation for a first music score and a first musical instrument in a currently processed training event, wherein the first music score includes a plurality of first notes, the first control parameter set includes a plurality of first parameter groups and a plurality of first time parameters corresponding to the plurality of first notes, each first parameter group includes a first motion parameter of a first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitch information and duration information of the plurality of first notes;

[0011] a configuration unit, configured to configure the first joint of the manipulator according to the first control parameter set, wherein the movement time of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group;

[0012] A second acquisition unit, configured to acquire first audio data detected in the currently processed training event;

[0013] An output unit is used to output practice record information based on the first audio data, the practice record information including error positions, and the error positions are used to indicate the positions of notes whose pitches do not match in the first audio data and target audio data, and the target audio data is standard audio data corresponding to the first music score and the first musical instrument.

[0014] In a third aspect, an embodiment of the present application provides a computing device, comprising: a processor, a memory, and a communication interface, wherein the processor, memory, and communication interface are interconnected, wherein the communication interface is used to receive or send data, the memory is used to store application code for the computing device to execute the method of the first aspect above, and the processor is configured to execute the method of the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute part or all of the steps described in the method of the first aspect of the embodiment of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0017] It can be seen that in the embodiment of the present application, the computing device in the hand function training system can automatically obtain the first control parameter set of the corresponding manipulator according to the first music score and the first musical instrument selected by the user, and configure the first joint of the manipulator according to the first motion parameter and the first time parameter of the first control parameter set, so that the manipulator leads the user to practice the musical instrument, and can output practice record information including the error position to indicate the audio position where the pitch of the first audio data collected during the practice does not match the pitch in the standard audio data. Leading the user to perform musical instrument training and outputting practice record information through the computing device and the manipulator is beneficial to improving the intelligence of the manipulator, and the first motion parameter and the first time parameter are determined according to the pitch information, duration information and the first musical instrument of the first music score, which is beneficial to improving the accuracy of auxiliary training information processing in the musical instrument practice scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a structural schematic diagram of a hand function training system provided in an embodiment of the present application;

[0020] Figure 2 It is a structural schematic diagram of a manipulator provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0022] Figure 4 It is a flowchart of a method for processing auxiliary training information in a musical instrument practice scenario provided by an embodiment of the present application;

[0023] Figure 5is a schematic diagram of an exercise record display page provided in an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of a retraining segment setting display page provided in an embodiment of the present application;

[0025] Figure 7 It is a structural schematic diagram of a device for processing auxiliary training information in a musical instrument practice scenario provided by an embodiment of the present application;

[0026] Figure 8 It is a structural schematic diagram of another device for processing auxiliary training information in a musical instrument practice scenario provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] The embodiments of the present application are described below with reference to the accompanying drawings.

[0031] The present application embodiment provides a hand function training system 10, referring to Figure 1, the hand function training system 10 may include a computing device 101 and a manipulator 102. The computing device 101 may specifically be a personal computer or a server. The personal computer may be, for example, a smart phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, a head-mounted device, etc., which is installed with an application (i.e., an application client) for controlling the manipulator 102. No specific limitation is made here. In actual applications, the computing device 101 may also be directly set on the manipulator 102 as a part of the manipulator 102. The manipulator 102 and the computing device 101 may be connected to each other by wired or wireless communication to achieve data interaction between them.

[0032] In the embodiment of the present application, when processing the training event, the manipulator 102 is mainly responsible for controlling the active movement of the first joint of the manipulator to drive the user's hand movement to practice the instrument according to the configuration of the computing device 101. The computing device is mainly responsible for detecting the confirmation operation for the first music score and the first musical instrument, and obtaining the first control parameter set of the manipulator 102 corresponding to the first music score and the first musical instrument, and configuring the first joint of the manipulator according to the first motion parameter and the first time parameter in the first control parameter set, and detecting the first audio data detected in the training event, and outputting the practice record information including the error position according to the first audio data, and the error position is specifically used to indicate: the position where the first audio data does not match the pitch in the standard audio data corresponding to the first music score and the first musical instrument.

[0033] Based on this, the computing device in the hand function training system can automatically configure the first joint of the robot according to the first music score and the first musical instrument selected by the user, and then drive the user to practice the musical instrument through the active movement of the robot. It can also detect the first audio data during the practice and output an error position indicating the mismatch between the first audio data and the pitch in the standard audio data, which is conducive to improving the accuracy of auxiliary training information processing in the musical instrument practice scenario and the intelligence of the robot.

[0034] The first preset motion parameter can be used to configure the motion direction and motion angle of the first joint, and the first time parameter corresponding to the first preset time parameter can be used to configure the motion moment of the first joint, that is, the computing device 101 can configure the motion direction, motion angle and motion moment of the first joint of the manipulator according to the first control parameter set. Then, the manipulator 102 can accurately drive the user's hand movement to practice the instrument.

[0035] In addition, in actual applications, in addition to processing practice events in practice mode, the computing device 101 and the manipulator 102 can also process evaluation events in evaluation mode. When processing evaluation events, the computing device 101 can also respond to the confirmation operation for the second music score and the second musical instrument, and obtain the second control parameter set of the manipulator 102 corresponding to the second music score and the second musical instrument, but the computing device 101 will not configure the joints of the manipulator 102, that is, the manipulator 102 will not actively move, but the user will practice the instrument by himself. The computing device 101 then obtains the actual motion parameters of the second joint of the manipulator 102 in the evaluation event, and outputs practice evaluation information based on the actual motion parameters to indicate the difference between the actual motion parameters and the second motion parameters in the second control parameter set, and / or, indicate the difference between the actual time parameters corresponding to the actual motion parameters and the second time parameters in the second control parameter set.

[0036] It should be noted that Figure 1 The number of manipulators 102 is only for illustrative purposes. In actual applications, the number of manipulators 102 to which the computing device 101 is communicatively connected may be one or more, and no specific limitation is made here.

[0037] Reference Figure 2 The manipulator 102 in the present application may specifically include a finger joint driver 01, a battery and a battery cover 02, a power adapter interface 03, a power switch 04, a palm fixing strap 05 and a finger fixing strap 06.

[0038] The manipulator 102 can specifically adjust the movement direction and angle of each joint through the finger joint drivers 01 corresponding to each joint.

[0039] It should be noted that Figure 2 The structure of the manipulator 102 is only an example. In actual applications, the specific structures included in the manipulator 102 may be more or less, and the appearance of the manipulator 102, the type of each specific structure included in the manipulator 102, and the location of each specific structure may also be different. Figure 2For example, a switch button between an evaluation mode and a practice mode may be provided on the manipulator 102 (when it is detected that the switch button corresponds to the practice mode state, the computing device 101 may configure the joints of the manipulator 102 according to the confirmed music score and the control parameter set corresponding to the musical instrument, so that the manipulator 102 actively moves to drive the user to practice the musical instrument; when the switch button corresponds to the evaluation mode, the computing device 101 does not configure the joints of the manipulator 102, but instead detects the actual motion parameters and actual time parameters of the joints of the manipulator 102, outputs practice evaluation information, indicates the difference between the actual motion parameters and the motion parameters corresponding to the confirmed music score and the musical instrument, and / or indicates the difference between the actual time parameters and the time parameters corresponding to the confirmed music score and the musical instrument), or the manipulator 102 may also be provided with a display screen or an audio acquisition device such as a microphone, and no specific restrictions are made here.

[0040] The composition structure of the computing device in this application can be as follows Figure 3 As shown, the computing device may include a processor 110, a memory 120, a communication interface 130, and one or more programs 121, wherein the one or more programs 121 are stored in the above-mentioned memory 120 and are configured to be executed by the above-mentioned processor 110, and the one or more programs 121 include instructions for executing any step in the following method embodiment. Among them, the communication interface 130 is used to support the communication between the computing device and other devices. In a specific implementation, the processor 110 is used to execute any step performed by the computing device in the following method embodiment, and when performing data transmission such as sending, the communication interface 130 may be selectively called to complete the corresponding operation. It should be noted that the structural diagram of the above-mentioned computing device is only an example, and the specific devices included may be more or less, and are not uniquely limited here.

[0041] See also Figure 4 , Figure 4 is a flow chart of a method for processing auxiliary training information in a musical instrument practice scenario provided by an embodiment of the present application, which is applied to a computing device in a hand function training system, wherein the hand function training system includes the computing device and a manipulator worn by a user; Figure 4 As shown, the method for processing auxiliary training information in the instrument practice scenario includes the following steps:

[0042] S201, the computing device obtains a first control parameter set of the robot in response to a confirmation operation on a first music score and a first musical instrument in a currently processed training event.

[0043] Among them, the first music score includes multiple first notes, the first control parameter set includes multiple first parameter groups and multiple first time parameters corresponding to the multiple first notes, each first parameter group includes the first motion parameter of the first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitch information and duration information of the multiple first notes.

[0044] Among them, the first control parameter set can be generated by the computing device according to the first music score information and the first musical instrument, or the computing device can also save the generated control parameter set in association with the corresponding music score and musical instrument after each generation of the control parameter set. After detecting the confirmation operation for the first music score and the first musical instrument, the computing device can first determine whether there is a first control parameter set corresponding to the first music score and the first musical instrument saved in the control parameter set. If so, the first parameter set can be directly obtained. If not, the first control parameter set is generated according to the first music score and the first musical instrument. In the scenario where the user repeatedly uses the same musical instrument to practice the same music score, the processing efficiency of the auxiliary training information can be improved.

[0045] The first motion parameter may specifically include a motion direction and a motion angle. The first time parameter may specifically be time information or duration information.

[0046] In a specific implementation, a computing device may save a first mapping relationship between multiple instrument types and multiple hand motion rules. When the computing device generates a first control parameter set based on the first music score information and the first musical instrument, the computing device may first determine a target hand motion rule from the multiple hand motion rules based on the instrument type of the first musical instrument. The target hand motion rule may include a second mapping relationship between multiple pitch combinations and multiple hand joints. Then, the computing device may determine the motion joints of the manipulator corresponding to each note in the music score based on the pitch information of each note in the first music score and the second mapping relationship, and determine the first joint and the first motion parameter corresponding to the first joint based on the motion joints corresponding to each adjacent note, and determine the first time parameter for the first joint corresponding to each note based on the duration information of the note.

[0047] Among them, a parameter group can correspond to a note, a note can correspond to a pitch combination, and a pitch combination can include one or more pitches played simultaneously, that is, when configuring the first joint of the manipulator, multiple first joints can be configured at one time for a group of pitches played simultaneously. The first time parameter corresponding to each first parameter group can be determined according to the time value information of the note corresponding to the first parameter group.

[0048] Among them, the first time parameter corresponding to each note can be used to indicate: the time difference between the start time of the performance corresponding to the note and the reference time. The reference time can be, for example, the performance time corresponding to the first note, or the reference time corresponding to each note can be the performance time of the previous note.

[0049] Among them, the second mapping relationship between pitch combinations and hand joints can be preset according to the type of musical instrument. The hand joints corresponding to each pitch combination are the joints that need to move when using the musical instrument to play the pitch combination. For example, when the musical instrument type is piano, the hand joints corresponding to each pitch combination are the finger joints that need to press the keys when playing the pitch combination. When the musical instrument type is violin, the hand joints corresponding to different pitch combinations may also include wrist joints. For example, the left hand can adjust the pitch by changing the position and moving it to different positions on the fingerboard.

[0050] After determining the motion joints of the manipulator corresponding to each note, the first joint corresponding to each note and the first motion parameter corresponding to each first joint can be further determined according to the motion joints corresponding to the current note, the motion joints corresponding to the previous note of the current note and / or the motion joints corresponding to the next joint corresponding to the current note. For example, taking the first musical instrument as a piano, if the motion joints corresponding to the current note do not include the left middle finger joint, that is, the current note does not require the left middle finger joint to press the key, and the motion joints corresponding to the previous note of the current note include the left middle finger joint, then the first joint corresponding to the current note also includes the left middle finger joint, and the motion direction corresponding to the left middle finger joint is the preset direction corresponding to the action of leaving the key.

[0051] That is to say, the hand joints that need to move when the instrument sounds each pitch combination can be determined based on the second mapping relationship, and the first joints may also include mechanical hand joints that are not related to the instrument sound but need to be reset.

[0052] Among them, the numerical value of the motion angle can be a preset fixed value. Alternatively, the computing device can determine the numerical value of the motion angle according to one or more information such as age, gender, finger length, palm width, etc. in the user information corresponding to the current training event. Alternatively, the computing device can also set different preset angle values ​​for different types of finger joints, for example, each finger joint corresponds to the same preset angle value, each wrist joint corresponds to the same preset angle value, etc. In other words, the manipulator joints are configured according to the control parameter set in the training event, mainly by prompting the user to play the fingering through the manipulator.

[0053] On this basis, when the computing device configures the first joint of the manipulator according to the first motion parameter and the first time parameter, the joint movement of the manipulator can specifically correspond to each note of the music score, so that the manipulator can actively control the movement of the first joint corresponding to the note at the performance time corresponding to each note, so as to drive the user to play.

[0054] S202: The computing device configures a first joint of the manipulator according to a first control parameter set.

[0055] The movement moment of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group.

[0056] For example, taking the first parameter group including parameter group a as an example, assuming that the first joint corresponding to parameter group a is joint 1, the first motion parameter corresponding to joint 1 includes: moving downward 15 degrees, and the time difference between the first time parameter and the playing time of the first note is 10 seconds. Then, according to the first control parameter set, the first joint of the manipulator is configured, and joint 1 of the manipulator will move downward 15 degrees 10 seconds after the joint movement corresponding to the first note.

[0057] S203: The computing device obtains first audio data detected in the training event currently being processed.

[0058] In a specific implementation, the first audio data may be collected when the robot starts to move, and the first audio data may be collected after a preset time period when the robot ends the movement. The preset time period may be determined based on the duration information of the last note of the first music score to ensure the integrity of the collected first audio data.

[0059] In a specific implementation, the computing device may also process the original audio collected at the start time and the end time of collection, and extract the audio data corresponding to the first musical instrument in the original audio as the first audio data.

[0060] S204: The computing device outputs practice record information according to the first audio data.

[0061] Among them, the practice record information includes an error position, and the error position is used to indicate the audio position where the pitch does not match between the first audio data and the target audio data, and the target audio data is standard audio data corresponding to the first music score and the first musical instrument.

[0062] In a specific implementation, since the manipulator is mainly used to prompt the user to play fingering, the user may press the wrong key or string when playing, or the duration of a single joint action may be wrong when playing, which may lead to a mismatch between the pitch of the detected first audio data and the target audio data. At this time, the computing device may also specifically determine the error type, which may specifically include a pitch error and / or a duration error, and the computing device may also output the error type when outputting the practice record information.

[0063] For example, refer to Figure 5 When the computing device outputs the practice record information, the notes corresponding to the error positions may be highlighted in the first score, for example Figure 5 The note corresponding to the error position is marked by area shading. When a selection operation such as a click operation is detected for the note at the error position, the error type can be further displayed, for example Figure 5 When a selection operation is detected for a shadow position note, the error type is displayed as a pitch error. In addition, Figure 5 As shown, the exercise record may also display the name of the first score (e.g. Figure 5 a), and the instrument type of the first instrument (e.g. Figure 5 The basic information of the instrument a) and the playing suggestion for the error position are displayed. The playing suggestion can be determined according to the error type, for example Figure 5 When the error type is a pitch error, the text information "The correct pitch is x, and the performance pitch is lower than the correct pitch" is output; in addition, when the error type includes a duration error, the output performance information can also be used to indicate the correct duration information, as well as to prompt that the performance duration is greater than the correct duration or the performance duration is less than the correct duration, thereby improving the comprehensiveness of the auxiliary training information output in the instrument practice scenario.

[0064] In particular, in actual applications, in the case of incorrect pitch, the computing device can also obtain a correct hand posture diagram when using the first instrument to play the note based on the instrument type of the first instrument and the note corresponding to the error position, and display the hand posture diagram in the performance suggestion display area, so that the user can understand the correct relative position relationship between the hand and the first instrument when playing the note.

[0065] It can be seen that in the embodiment of the present application, the computing device in the hand function training system can automatically obtain the first control parameter set of the corresponding manipulator according to the first music score and the first musical instrument selected by the user, and configure the first joint of the manipulator according to the first motion parameter and the first time parameter of the first control parameter set, so that the manipulator leads the user to practice the musical instrument, and can output practice record information including the error position to indicate the audio position where the pitch of the first audio data collected during the practice does not match the pitch in the standard audio data. Leading the user to perform musical instrument training and outputting practice record information through the computing device and the manipulator is beneficial to improving the intelligence of the manipulator, and the first motion parameter and the first time parameter are determined according to the pitch information, duration information and the first musical instrument of the first music score, which is beneficial to improving the accuracy of auxiliary training information processing in the musical instrument practice scenario.

[0066] In one possible example, after outputting the practice record information according to the first audio data, the method further includes: in response to a selection operation for a target note in the first music score, determining at least one target parameter group corresponding to the target note from the multiple first parameter groups, the target note including the error position; creating a re-practice event according to the at least one target parameter group, the at least one target parameter group being used to configure the first joint of the robot in the re-practice event; acquiring second audio data detected in the re-practice event; and updating the practice record information according to the second audio data.

[0067] For example, taking the first control parameter set including parameter group 1 corresponding to note 1, parameter group 2 corresponding to note 2 and parameter group 3 corresponding to note 3, assuming that the error position corresponds to note 2, and the computing device determines that the target notes are note 2 and note 3, then the target parameter group includes parameter group 2 and parameter group 3. If parameter group 2 corresponds to joint 1 of the manipulator, and parameter group 3 corresponds to joint 2 of the manipulator, then after the computing device creates a re-practice event, it can configure joint 1 and joint 2 of the manipulator according to parameter group 2 and parameter group 3, and then the manipulator drives the user to re-practice the fragments of note 2 and note 3.

[0068] In a specific implementation, the computing device may first display a re-practice segment setting page, and in the setting page, by displaying a music segment corresponding to a target note, when a confirmation operation for the music segment is detected, it may be determined that a selection operation for the target note is detected.

[0069] For example, see Figure 6, the music score fragment corresponding to the target note can be displayed in the fragment adjustment area of ​​the re-practice fragment setting page in the form of regional shading. Specifically, when the re-practice fragment setting page is initially displayed, the target note recommended by the computing device can be displayed by default. Subsequently, when the computing device detects a selection operation for the drop-down control "∧" in the fragment type selection area, it can expand the fragment type list. The fragment type list can include, for example, default, measure, paragraph and other fragment types. When a selection operation for a specific fragment type is detected, the computing device can determine the measure or paragraph where the error position is located as the target note, and adjust the target note range displayed in the fragment adjustment area.

[0070] In addition, after the computing device automatically determines and displays the target note by selecting a segment type such as default, measure or paragraph, the computing device can also detect adjustment operations such as dragging operations on the target note display area in the segment adjustment area, determine the note corresponding to the target note from the first score according to the adjusted target note display area, and determine the target parameter group according to the note corresponding to the target note. In other words, the computing device can also support the user to customize the retraining range, and flexibly configure the joints of the manipulator according to the user-defined retraining range.

[0071] In addition, if Figure 6 As shown, the computing device may also display a prompt message such as "You can drag to adjust the segment range" in the segment adjustment area. Similarly, the re-practice segment setting page may also display basic information such as the score name and instrument type, for example Figure 6 The music score a and instrument a are displayed in . At the same time, the number of exercises can also be displayed in the re-practice segment setting page, and the number of exercises can be adjusted in an increasing or decreasing manner according to the detected selection operation on the "+" or "-" control. Furthermore, the selection operation for the target note can be a click operation on the start re-practice control. When the computing device detects the click operation on the start re-practice control, it can create a re-practice event according to the target parameter group, and configure the first joint of the manipulator according to the target parameter group to drive the user to re-practice the segment. The number of re-practice can be determined according to the set number of exercises.

[0072] In a specific implementation, the retraining segment setting page may be displayed after a selection operation of a segment retraining control in the practice record display page is detected, such as Figure 5 As shown, the practice record information display page can also display full song re-practice controls and segment re-practice controls. When a selection operation for the full song re-practice control is detected, the computing device can also determine the multiple first parameter groups included in the first control parameter set as target parameter groups, and directly configure the first joint of the manipulator according to the first control parameter set to start full song re-practice.

[0073] Among them, the method of updating the practice record information according to the second audio data can refer to the method of generating the practice record information according to the first audio data, which will not be repeated here.

[0074] It can be seen that in this example, the computing device can also respond to the selection operation for the target note including the error position, create a reconnection event according to the target parameter group corresponding to the target note, configure the first joint of the robot in the retraining event according to the target parameter group, and then re-acquire the second audio data detected in the retraining event, and update the practice record information according to the second audio data, which is conducive to further improving the comprehensiveness of the output auxiliary training information and the intelligence of the robot.

[0075] In one possible example, the target note is determined by the following steps: determining the error type corresponding to the error position based on the pitch and acquisition time of the audio data corresponding to the error position, the error type including: timing error or pitch error; obtaining a fragment extraction rule corresponding to the error type: determining the target note from the multiple first notes based on the fragment extraction rule, the target note including the note corresponding to the error position.

[0076] In a specific implementation, the target note determined in this example can be used as Figure 6 The default exercise snippet that is initially displayed in .

[0077] In a specific implementation, when determining the error position of the pitch mismatch between the first audio data and the target audio data, the computing device can respectively identify the pitches appearing in the first audio data and the target audio data, and mark the acquisition time of each pitch, establish a first time axis for the appearance of the pitches according to the acquisition time of each pitch in the first audio data, establish a second time axis for the appearance of the pitches according to the acquisition time of each pitch in the target audio, and determine the error position and error type by comparing the time axes.

[0078] In a specific implementation, the segment extraction rules corresponding to each error type can be preset by the user, or the computing device can obtain the segment extraction rules corresponding to each error type from the cloud server. In particular, different instrument types can correspond to different rule sets, and the computing device can first determine a target rule set from multiple rule sets according to the instrument type of the first instrument, and then determine the segment extraction rule corresponding to the error type from the multiple segment extraction rules included in the target rule set, so as to further improve the flexibility and accuracy of the target note determination.

[0079] For example, the user presets the segment extraction rule corresponding to the time value error as determining the first note and the second note as the target notes, and the segment extraction rule corresponding to the pitch error as determining the first note and the third note as the target notes, wherein the first note is the first note corresponding to the error position, the second note is the note that is played after the first note in the bar where the first note is located, and the third note is the note that is played before the first note in the bar where the first note is located. Assuming that the bar where the note a corresponding to the error position is located is bar a, and the multiple notes included in bar a are arranged in order from front to back in the order of playing: note b, note a, note c, then when the computing device determines that the error type is a time value error, it will determine that the note a and note c are the target notes.

[0080] In addition, in addition to the fragment extraction rules set by the user, the computing device can also determine the notes in other measures as part of the target note based on the position of the note corresponding to the error position in the measure. For example, when the note corresponding to the error position is the first note in the measure, the first preset number of notes in the previous measure with the last note in the playing order are also obtained as part of the target note; when the note corresponding to the error position is the last note in the measure, the second preset number of notes in the next measure with the first note in the playing order are also obtained as part of the target note.

[0081] It can be seen that in this example, the computing device can determine the error type of the error location based on the pitch and acquisition time of the audio data corresponding to the error location, and obtain the fragment extraction rule corresponding to the error type, and then determine the target note from the multiple first notes included in the first music score according to the fragment extraction rule, which is conducive to improving the flexibility of determining the target note when re-practicing the instrument.

[0082] In one possible example, when there are multiple error positions, the target note is determined by the following steps: determining reference bars corresponding to the multiple error positions from the first music score, the reference bars including the notes corresponding to the error positions; when there are multiple reference bars, determining multiple continuous music score bars from the first music score according to the positions of the multiple reference bars, the multiple music score bars including the multiple reference bars; determining a retraining bar from the multiple music score bars according to the number of bars between two adjacent reference bars, the retraining bar including the reference bar, or including the reference bar and a bar adjacent to the reference bar; determining the note included in the retraining bar as the target note.

[0083] Among them, the reference measure is the measure where the note corresponding to the error position is located.

[0084] In a specific implementation, the method of determining the re-practice measure from multiple score measures can be specifically to determine the number of measures between two reference measures with the closest performance time. When the number of measures between the two reference measures is less than a preset number, the two reference measures and the measure between the two reference measures are both determined as re-practice measures. If the number of measures between the two reference measures is not less than the preset number, only the two reference measures are determined as re-practice measures, and the measure between the two reference measures is not determined as re-practice measures. Furthermore, for segments with relatively concentrated error locations, the computing device can configure a manipulator to drive the user to perform unified practice in one re-practice event without splitting it into multiple re-practice events, which is conducive to further improving the processing efficiency of auxiliary training information.

[0085] The preset number may be determined by the computing device according to user input.

[0086] For example, the multiple bars included in the first music score are arranged in order from front to back in the order of performance: bar 1, bar 2, bar 3, bar 4 and bar 5. Assuming that the reference bars where the error is located are bar 2 and bar 5, the multiple music score bars include bar 2, bar 3, bar 4 and bar 5. In the case where the preset number is 1, the target notes include bar 2 and bar 5. The computing device determines that there are discontinuous notes in the target notes and can create two retraining events, that is, create a retraining event for bar 2 and a retraining event for bar 5. In the case where the preset number is 2, the target notes include bar 2, bar 3, bar 4 and bar 5. Since the notes in the target notes are all continuous and there are no discontinuous notes, the computing device can create a retraining event for bar 2, bar 3, bar 4 and bar 5.

[0087] In a specific implementation, if the number of error positions is 1, or if the notes corresponding to multiple error positions belong to the same reference measure, the computing device can directly determine that the reference measure where the note corresponding to the error position is located is the target note. Alternatively, the computing device can determine that the target note includes: the note corresponding to the error position, and the third preset number of notes before the note corresponding to the error position, and the fourth preset number of notes after the note corresponding to the error position. In particular, the third preset number and the fourth preset number can be determined by the computing device according to user input, or obtained by the computing device from a cloud server.

[0088] It can be seen that in this example, when there are multiple error positions and there are multiple reference measures where the notes corresponding to the error positions are located, the computing device determines the re-training measure from the multiple consecutive music score measures based on the positions of the reference measures in the multiple consecutive music score measures, and determines the notes included in the re-training measure as the target notes, which is conducive to improving the flexibility of determining the target notes.

[0089] In one possible example, the first music score is obtained through the following steps: in response to an upload operation for target audio, identifying the target audio to obtain at least one audio data corresponding to at least one instrument type; generating at least one reference music score corresponding to the at least one instrument type based on the at least one audio data; outputting the at least one instrument type, the at least one instrument type including the first instrument; in response to a confirmation operation for a first instrument in the at least one instrument type, determining that the reference music score corresponding to the first instrument is the first music score.

[0090] In a specific implementation, the computing device can determine the complete music score corresponding to the target audio by identifying the pitch and duration of the target audio, and then identify the specific audio clips (i.e., at least one audio data) of each musical instrument that appears in the target audio, and determine which part of the complete music score each musical instrument plays, to obtain at least one reference music score. At this time, the computing device can output the name of at least one type of musical instrument identified, and when a confirmation operation for the first musical instrument is detected, obtain the reference music score corresponding to the first musical instrument as the target music score.

[0091] For example, taking at least one audio data including data 1 and data 2 as an example, assuming that data 1 corresponds to instrument 1, and music score 1 is generated based on data 1, data 2 corresponds to instrument 2, and music score 2 is generated based on data 2, then when the computing device detects that instrument 1 is selected, it determines that music score 1 is the first music score, and when it detects that instrument 2 is selected, it determines that music score 2 is the first music score.

[0092] It can be seen that in this example, the computing device can identify the target audio uploaded by the user, obtain at least one audio data corresponding to at least one instrument type, and generate at least one reference music score based on the at least one audio data, and then in response to a confirmation operation for the first instrument, determine that the reference music score corresponding to the first instrument is the first music score, thereby improving the flexibility and convenience of obtaining the first music score.

[0093] In one possible example, the method further includes: in response to a confirmation operation for a second music score and a second musical instrument in a currently processed evaluation event, obtaining a second control parameter set of the manipulator corresponding to the second music score and the second musical instrument, the second music score including multiple second notes, the second control parameter set including multiple second parameter groups and multiple second time parameters corresponding to the multiple second notes, each second parameter group including second motion parameters of a second joint of the manipulator, the second motion parameters and the second time parameters being determined according to second music score information and the second musical instrument, the second music score information including pitch information and duration information of the multiple second notes; obtaining actual motion parameters of the second joint in the currently processed evaluation event; outputting practice evaluation information according to the actual motion parameters and the actual time parameters corresponding to the actual motion parameters, the practice evaluation information including first information and / or second information, the first information being used to indicate the difference between the actual motion parameters and the at least one second motion parameter, the second information being used to indicate the difference between the actual time parameters corresponding to the actual motion parameters and the second time parameters.

[0094] Among them, the method of determining the second motion parameter and the second time parameter in the second control parameter set according to the pitch information and duration information of the second note can refer to the aforementioned content of determining the first motion parameter and the first motion parameter, which will not be repeated here.

[0095] Among them, the difference between the actual motion parameters and the second motion parameters may specifically include: whether the corresponding manipulator joints are the same, whether the motion directions of the manipulator joints are the same, etc. The difference between the actual time parameters and the second time parameters may specifically refer to whether the time interval between the start time and the reference time of the movement of the manipulator joints with the same movement sequence is the same, etc.

[0096] In particular, the computing device can also query the mapping relationship between multiple preset quantitative measurement level parameters and multiple difference information (including the aforementioned number of mismatches and the size of the difference) based on the number of mismatches between the actual motion parameter and the second motion parameter, and the difference between the time intervals corresponding to the actual time parameter and the second time parameter, determine the proficiency level corresponding to the number of mismatches and the size of the difference, realize quantitative evaluation of the quantitative measurement level, and improve the refinement level of the auxiliary training information.

[0097] In addition, in other embodiments, the practice evaluation information may include not only the fingering evaluation information determined according to the actual motion parameter, the actual time parameter, and the second control parameter set, but also the intonation evaluation information. The specific computing device may obtain the third audio data collected in the evaluation event, and determine the intonation evaluation information according to the third audio data and the standard audio data corresponding to the second music score and the second instrument. The intonation evaluation information may also include the error position. When determining the proficiency level, the proficiency level may be determined based on the number of mismatches, the difference size, and the number of error positions.

[0098] It can be seen that in this example, the computing device can output exercise evaluation information in the evaluation event based on the actual motion parameters and actual time parameters of each joint of the manipulator detected during the user's autonomous movement, and the second control parameter set corresponding to the second music score and the second musical instrument confirmed in the evaluation event, to indicate the difference between the actual motion parameters and / or actual time parameters and the second control parameter set, which is conducive to improving the comprehensiveness of the auxiliary training information provided by the computing device.

[0099] In one possible example, after outputting the practice evaluation information, the method further includes: obtaining a recommended music score paragraph from a preset music score paragraph based on the first information and / or the second information; outputting the recommended music score paragraph; and when a selection operation for the recommended music score paragraph is detected, configuring the joints of the manipulator according to a set of motion parameters corresponding to the recommended music score paragraph.

[0100] In a specific implementation, each preset music score paragraph may correspond to at least one fingering training type. The computing device may specifically determine the type of difference information between the first information and the second information, and use the type of difference information as a keyword to determine a music score whose fingering training type includes the keyword from multiple preset music score paragraphs as a recommended music score paragraph. For example, when the difference type includes motion joint mismatch, a music score paragraph whose fingering training type includes motion joint mismatch is determined from multiple preset music score paragraphs as a recommended music score paragraph.

[0101] In particular, each preset music score paragraph may also correspond to a proficiency level information. The computing device may first preliminarily screen the music score paragraphs according to the fingering training type. If the number of music score paragraphs after screening is still multiple, the computing device may further screen the music score paragraphs according to the proficiency level of the user in the evaluation event, and use the music score paragraphs whose proficiency level information includes the number of proficiency levels of the user in the evaluation event as recommended music score paragraphs.

[0102] It can be seen that in this example, the computing device can recommend a music score segment to the user based on the difference between the actual motion parameter and the second motion parameter in the exercise evaluation information, and / or the difference between the actual time parameter and the second time parameter, and configure the joints of the robot according to the motion parameter set corresponding to the music score segment to drive the user to practice the recommended music score segment, which is conducive to further improving the flexibility of auxiliary training information processing.

[0103] See also Figure 7 , Figure 7 1 is a schematic diagram of the structure of a device for processing auxiliary training information in a musical instrument practice scenario provided by an embodiment of the present application, which is applied to a computing device in a hand function training system, wherein the hand function training system includes the computing device and a manipulator worn by a user; the device 30 for processing auxiliary training information in the musical instrument practice scenario includes:

[0104] A first acquisition unit 301 is used to acquire a first control parameter set of the manipulator in response to a confirmation operation of a first music score and a first musical instrument in a currently processed training event, wherein the first music score includes a plurality of first notes, the first control parameter set includes a plurality of first parameter groups and a plurality of first time parameters corresponding to the plurality of first notes, each first parameter group includes a first motion parameter of a first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitch information and duration information of the plurality of first notes;

[0105] A configuration unit 302 is configured to configure the first joint of the manipulator according to the first control parameter set, wherein the movement time of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group;

[0106] A second acquisition unit 303 is used to acquire first audio data detected in the currently processed training event;

[0107] The output unit 304 is used to output practice record information based on the first audio data, and the practice record information includes error positions, and the error positions are used to indicate the positions of notes whose pitches do not match in the first audio data and the target audio data. The target audio data is standard audio data corresponding to the first music score and the first musical instrument.

[0108] In one possible example, the processing device 30 for auxiliary training information in the musical instrument practice scenario is also used to, after outputting the practice record information according to the first audio data, determine, in response to a selection operation for a target note in the first music score, at least one target parameter group corresponding to the target note from the multiple first parameter groups, the target note including the error position; create a re-practice event according to the at least one target parameter group, the at least one target parameter group being used to configure the first joint of the robot in the re-practice event; obtain second audio data detected in the re-practice event; and update the practice record information according to the second audio data.

[0109] In one possible example, the target note is determined by the following steps: determining the error type corresponding to the error position based on the pitch and acquisition time of the audio data corresponding to the error position, the error type including: timing error or pitch error; obtaining a fragment extraction rule corresponding to the error type: determining the target note from the multiple first notes based on the fragment extraction rule, the target note including the note corresponding to the error position.

[0110] In one possible example, when there are multiple error positions, the target note is determined by the following steps: determining reference bars corresponding to the multiple error positions from the first music score, the reference bars including the notes corresponding to the error positions; when there are multiple reference bars, determining multiple continuous music score bars from the first music score according to the positions of the multiple reference bars, the multiple music score bars including the multiple reference bars; determining a retraining bar from the multiple music score bars according to the number of bars between two adjacent reference bars, the retraining bar including the reference bar, or including the reference bar and a bar adjacent to the reference bar; determining the note included in the retraining bar as the target note.

[0111] In one possible example, the first music score is obtained through the following steps: in response to an upload operation for target audio, identifying the target audio to obtain at least one audio data corresponding to at least one instrument type; generating at least one reference music score corresponding to the at least one instrument type based on the at least one audio data; outputting the at least one instrument type, the at least one instrument type including the first instrument; in response to a confirmation operation for a first instrument in the at least one instrument type, determining that the reference music score corresponding to the first instrument is the first music score.

[0112] In one possible example, the processing device 30 for auxiliary training information in the instrument practice scenario is also used to, in response to a confirmation operation for a second music score and a second instrument in a currently processed evaluation event, obtain a second control parameter set of the manipulator corresponding to the second music score and the second instrument, the second music score including multiple second notes, the second control parameter set including multiple second parameter groups and multiple second time parameters corresponding to the multiple second notes, each second parameter group including second motion parameters of the second joint of the manipulator, the second motion parameters and the second time parameters are determined according to the second music score information and the second instrument, the second music score information including the pitch information and duration information of the multiple second notes: obtain the actual motion parameters of the second joint in the currently processed evaluation event; output practice evaluation information according to the actual motion parameters and the actual time parameters corresponding to the actual motion parameters, the practice evaluation information including first information and / or second information, the first information being used to indicate the difference between the actual motion parameters and the at least one second motion parameter, and the second information being used to indicate the difference between the actual time parameters corresponding to the actual motion parameters and the second time parameters.

[0113] In one possible example, the processing device 30 for auxiliary training information in the musical instrument practice scenario is also used to, after outputting the practice evaluation information, obtain a recommended music score paragraph from a preset music score paragraph based on the first information and / or the second information; output the recommended music score paragraph; and when a selection operation for the recommended music score paragraph is detected, configure the joints of the manipulator according to the motion parameter set corresponding to the recommended music score paragraph.

[0114] In the case of adopting an integrated unit, a structural diagram of another device for processing auxiliary training information in a musical instrument practice scenario provided by an embodiment of the present application is as follows: Figure 8 As shown. Figure 8 In the embodiment, the device for processing auxiliary training information in the musical instrument practice scene includes: a processing module 310 and a communication module 311. The processing module 310 is used to control and manage the actions of the device for processing auxiliary training information in the musical instrument practice scene, for example, the steps performed by the first acquisition unit 301, the configuration unit 302, the second acquisition unit 303 and the output unit 304, and / or other processes for executing the technology described in this application. The communication module 311 is used to support the interaction between the device for processing auxiliary training information in the musical instrument practice scene 30 and other devices. Figure 8 As shown, the device for processing auxiliary training information in a musical instrument practice scenario may further include a storage module 312, and the storage module 312 is used to store program codes and data of the device for processing auxiliary training information in a musical instrument practice scenario.

[0115] Among them, the processing module 310 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 311 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 312 can be a memory.

[0116] Among them, all relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here. The processing device for auxiliary training information in the above instrument practice scenario can execute the above Figure 4 The steps executed by a computing device in the method for processing auxiliary training information in a musical instrument practice scenario are shown.

[0117] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0118] The present application also provides a computer program product, including a computer program, which implements part or all of the steps of any method described in the above method embodiment when executed by a processor. The computer program product may be a software installation package.

[0119] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and units involved are not necessarily required by the present application.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0122] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0125] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0126] The embodiments of the present application are introduced in detail above. Specific examples are used in the present application to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in the field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing auxiliary training information in a musical instrument practice scenario, characterized in that: A computing device applied to a hand function training system, the hand function training system comprising the computing device and a manipulator worn by a user; the method comprising: In response to a confirmation operation for a first music score and a first musical instrument in a currently processed training event, a first control parameter set of the manipulator is acquired, wherein the first music score includes a plurality of first notes, the first control parameter set includes a plurality of first parameter groups and a plurality of first time parameters corresponding to the plurality of first notes, each first parameter group includes a first motion parameter of a first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitch information and duration information of the plurality of first notes; configuring the first joint of the manipulator according to the first control parameter set, wherein the movement time of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group; Acquire the first audio data detected in the currently processed training event; Outputting practice record information according to the first audio data, the practice record information including an error position, the error position being used to indicate an audio position where the pitch of the first audio data and the target audio data do not match, the target audio data being standard audio data corresponding to the first music score and the first musical instrument; Wherein, the method also includes: in response to a confirmation operation for a second music score and a second musical instrument in a currently processed evaluation event, obtaining a second control parameter set of the manipulator corresponding to the second music score and the second musical instrument, the second control parameter set including a plurality of second parameter groups and a plurality of second time parameters corresponding to the plurality of second parameter groups, each second parameter group including a second motion parameter of a second joint of the manipulator, the second motion parameter and the second time parameter being determined according to second music score information and the second musical instrument, the second music score information including pitch information and duration information of notes in the second music score; obtaining actual motion parameters of the second joint in the currently processed evaluation event; outputting practice evaluation information according to the actual motion parameters and the actual time parameters corresponding to the actual motion parameters, the practice evaluation information including first information and / or second information, the first information being used to indicate a difference between the actual motion parameters and the at least one second motion parameter, the second information being used to indicate a difference between the actual motion parameters and the at least one second motion parameter, and the second time parameter being used to indicate a difference between the actual motion parameters and the at least one second motion parameter. The second information is used to indicate the difference between the actual time parameter corresponding to the actual motion parameter and the second time parameter; based on the first information and / or the second information, a recommended music score paragraph is obtained from a preset music score paragraph, specifically, each preset music score paragraph corresponds to at least one fingering training type, the computing device specifically determines the type of difference information in the first information and the second information, and uses the type of the difference information as a keyword to determine the music score whose fingering training type includes the keyword from multiple preset music score paragraphs as a recommended music score paragraph, each preset music score paragraph also corresponds to a proficiency level information, the computing device first preliminarily screens the music score paragraphs according to the fingering training type, if the number of music score paragraphs after screening is still multiple, the computing device further screens the music score paragraphs according to the proficiency level of the user in the evaluation event; outputs the recommended music score paragraph; when a selection operation for the recommended music score paragraph is detected, configures the joints of the manipulator according to the motion parameter set corresponding to the recommended music score paragraph.

2. The method according to claim 1, characterized in that After outputting the exercise record information according to the first audio data, the method further includes: In response to a selection operation for a target note in the first music score, determining at least one target parameter group corresponding to the target note from the plurality of first parameter groups, the target note including the error position; creating a retraining event according to the at least one target parameter group, the at least one target parameter group being used to configure the first joint of the manipulator in the retraining event; Acquire second audio data detected in the retraining event; The exercise record information is updated according to the second audio data.

3. The method according to claim 2, characterized in that The target note is determined by the following steps: Determine the error type corresponding to the error position according to the pitch and acquisition time of the audio data corresponding to the error position, wherein the error type includes: a time value error or a pitch error; Get the fragment extraction rule corresponding to the error type: The target note is determined from the plurality of first notes according to the segment extraction rule, and the target note includes the note corresponding to the error position.

4. The method according to claim 2, characterized in that: In the case where there are multiple error positions, the target note is determined by the following steps: Determine, from the first music score, reference measures corresponding to the plurality of error positions, wherein the reference measures include notes corresponding to the error positions; In the case where there are multiple reference bars, a plurality of continuous music score bars are determined from the first music score according to the positions of the plurality of reference bars, wherein the plurality of music score bars include the plurality of reference bars; Determine a rehearsal bar from the plurality of score bars according to the number of bars between two adjacent reference bars, wherein the rehearsal bar includes the reference bar, or includes the reference bar and bars adjacent to the reference bar; The notes included in the rehearsal measure are determined to be the target notes.

5. The method according to any one of claims 1 to 4, characterized in that: The first music score is obtained by the following steps: In response to an upload operation for a target audio, identifying the target audio to obtain at least one audio data corresponding to at least one instrument type; generating at least one reference music score corresponding to the at least one instrument type according to the at least one audio data; outputting the at least one instrument type, the at least one instrument type including the first instrument; In response to a confirmation operation for a first musical instrument among the at least one musical instrument type, a reference music score corresponding to the first musical instrument is determined to be the first music score.

6. A device for processing auxiliary training information in a musical instrument practice scene, characterized in that: A computing device used in a hand function training system, the hand function training system comprising the computing device and a manipulator worn by a user; the device comprising: A first acquisition unit is used to acquire a first control parameter set of the manipulator in response to a confirmation operation for a first music score and a first musical instrument in a currently processed training event, wherein the first music score includes a plurality of first notes, the first control parameter set includes a plurality of first parameter groups and a plurality of first time parameters corresponding to the plurality of first notes, each first parameter group includes a first motion parameter of a first joint of the manipulator, the first motion parameter and the first time parameter are determined according to first music score information and the first musical instrument, and the first music score information includes pitches and durations of notes in the first music score; a configuration unit, configured to configure the first joint of the manipulator according to the first control parameter set, wherein the movement time of the first joint corresponding to each first parameter group is determined according to the first time parameter corresponding to each first parameter group; A second acquisition unit, configured to acquire first audio data detected in the currently processed training event; an output unit, configured to output practice record information according to the first audio data, the practice record information including an error position, the error position being used to indicate a note position whose pitch does not match between the first audio data and target audio data, the target audio data being standard audio data corresponding to the first music score and the first musical instrument; Wherein, the device is also used for: in response to a confirmation operation for a second music score and a second musical instrument in a currently processed evaluation event, obtaining a second control parameter set of the manipulator corresponding to the second music score and the second musical instrument, the second control parameter set including a plurality of second parameter groups and a plurality of second time parameters corresponding to the plurality of second parameter groups, each second parameter group including a second motion parameter of a second joint of the manipulator, the second motion parameter and the second time parameter being determined according to second music score information and the second musical instrument, the second music score information including pitch information and duration information of notes in the second music score: obtaining actual motion parameters of the second joint in the currently processed evaluation event; outputting practice evaluation information according to the actual motion parameters and the actual time parameters corresponding to the actual motion parameters, the practice evaluation information including first information and / or second information, the first information being used to indicate a difference between the actual motion parameters and the at least one second motion parameter, the second time parameter being used to indicate a difference between the actual motion parameters and the at least one second motion parameter. The second information is used to indicate the difference between the actual time parameter corresponding to the actual motion parameter and the second time parameter; based on the first information and / or the second information, a recommended music score paragraph is obtained from a preset music score paragraph, specifically, each preset music score paragraph corresponds to at least one fingering training type, the computing device specifically determines the type of difference information in the first information and the second information, and uses the type of the difference information as a keyword to determine the music score whose fingering training type includes the keyword from multiple preset music score paragraphs as a recommended music score paragraph, each preset music score paragraph also corresponds to a proficiency level information, the computing device first preliminarily screens the music score paragraphs according to the fingering training type, if the number of music score paragraphs after screening is still multiple, the computing device further screens the music score paragraphs according to the proficiency level of the user in the evaluation event; outputs the recommended music score paragraph; when a selection operation for the recommended music score paragraph is detected, configures the joints of the manipulator according to the motion parameter set corresponding to the recommended music score paragraph.

7. A computing device, characterized in that The computing device comprises: a processor, a memory and a communication interface, wherein the processor, the memory and the communication interface are connected to each other, wherein the communication interface is used to receive or send data, the memory is used to store application code for the computing device to execute the method as described in any one of claims 1-5, and the processor is configured to execute the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps in the method according to any one of claims 1 to 5.

Images