Musical Instrument Assisted Practice Method, Device, Medium and Electronic Device

Abstract

The present disclosure relates to a musical instrument auxiliary practice method, device, medium and electronic device. The method includes: collecting performance music in real time; sending performance segments within a preset time interval to a server communicatively connected to a terminal at each preset time interval, and generating score-following information for each note played in the performance segments within the preset time interval, and sending the score-following information to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the score-following information for each note played in each performance segment; receiving and displaying the performance feedback result sent by the server. Thus, the performance feedback result can be automatically generated, saving manpower and ensuring the objectivity and accuracy of the performance feedback result. In addition, since the performance segments and the corresponding score-following information are transmitted while performing, thus, the performance feedback result of the current performance music can be quickly generated within a short time after the performance ends, improving the performance experience of the performer.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technologies, and in particular, to a musical instrument assisted practice method, device, medium, and electronic device. Background Art

[0002] Music has been passed down from generation to generation since ancient times with its unique artistic charm and value. Instrument performance, as one of the main forms of music expression, can cultivate sentiment and promote the improvement of aesthetic awareness and music literacy. With the improvement of living standards, people's enthusiasm for learning instrument performance is constantly rising, and parents also pay more and more attention to the cultivation of their children's music literacy. However, learning instrument performance requires professional teacher guidance and a large amount of practice accumulation. In the past, this kind of practice relied more on manual guidance and feedback, which consumed a lot of time and energy. Summary of the Invention

[0003] This part of the content is provided to introduce the concepts in a brief form, which will be described in detail in the following specific implementation part. This part of the content is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a musical instrument assisted practice method, which is applied to a terminal and includes:

[0005] Real-time collecting the played music;

[0006] Sending the playing segment within the preset time interval to a server communicatively connected to the terminal at every preset time interval, and generating the score-following information of each note played in the playing segment within the preset time interval, and sending the score-following information to the server, so that the server generates a playing feedback result of the current played music according to each playing segment of the current played music and the score-following information of each note played in each playing segment;

[0007] Receiving and displaying the playing feedback result sent by the server.

[0008] In a second aspect, the present disclosure provides a musical instrument assisted practice device, which is applied to a terminal and includes:

[0009] A collecting module, configured to collect the played music in real time;

[0010] A sending module, configured to send the performance segments collected by the collecting module within a preset time interval to a server communicatively connected to the terminal at every preset time interval, and generate follow-up score information for each note played in the performance segments within the preset time interval, and send the follow-up score information to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the follow-up score information for each note played in each performance segment;

[0011] A display module, configured to receive and display the performance feedback result sent by the server.

[0012] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method provided in the first aspect of the present disclosure are implemented.

[0013] In a fourth aspect, the present disclosure provides an electronic device, including:

[0014] A storage device, on which at least one computer program is stored;

[0015] At least one processing device, configured to execute the at least one computer program in the storage device to implement the steps of the method provided in the first aspect of the present disclosure.

[0016] In the above technical solution, the performance segments within a preset time interval are sent to a server communicatively connected to the terminal at every preset time interval, and the follow-up score information for each note played in the performance segments within the preset time interval is generated, and the follow-up score information is sent to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the follow-up score information for each note played in each performance segment; the performance feedback result sent by the server is received and displayed. Thereby, the performance feedback result can be automatically generated, saving manpower and ensuring the objectivity and accuracy of the performance feedback result. In addition, since the performance segments and the corresponding follow-up score information are transmitted while performing, the server can quickly generate the performance feedback result of the current performance music within a short time after the performance ends, avoiding the transmission delay problem caused by transmitting large-scale unstructured performance music only at the end of the performance, so that the performer can quickly obtain the performance feedback result and improve the performance experience of the performer.

[0017] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 is a flowchart of a method for assisting in musical instrument practice according to an exemplary embodiment.

[0020] Figure 2 is a flowchart of a method for assisting in musical instrument practice according to another exemplary embodiment.

[0021] Figure 3 is a flowchart of a method for assisting in musical instrument practice according to another exemplary embodiment.

[0022] Figure 4 is a block diagram of a device for assisting in musical instrument practice according to an exemplary embodiment.

[0023] Figure 5 is a block diagram of an electronic device according to an exemplary embodiment. Specific Embodiments

[0024] As discussed in the background art, learning to play a musical instrument requires professional teacher guidance and a large amount of practice accumulation. In the past, this kind of practice relied more on manual guidance and feedback, which required a lot of time and effort. Therefore, it is necessary to generate the performance feedback results automatically with the help of a computer. At present, usually, the terminal collects the performance music, and after the performance ends, the performance music is transmitted to the server, so that the server generates the performance feedback results according to the received performance music and feeds them back to the terminal. Since the performance music is usually large-scale unstructured data, the transmission will take a lot of time. In this way, it will affect the generation efficiency of the performance feedback results and the performance experience of the performer.

[0025] In view of this, the present disclosure provides a method, device, medium, and electronic device for assisting in musical instrument practice.

[0026] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0028] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0029] It should be noted that the concepts such as "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the modifications of "one" and "a plurality" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0031] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0032] Figure 1 is a flowchart of a musical instrument assisted practice method shown according to an exemplary embodiment, wherein the method can be applied to terminals such as smart phones, personal computers, and tablet computers. As Figure 1 shown, the above method may include the following S101 to S103.

[0033] In S101, the performance music is collected in real time.

[0034] In the present disclosure, the performance music is the musical sound data collected by collecting the musical sounds emitted by the performer using the musical instrument, and it is unstructured data.

[0035] In S102, every preset time interval, the performance segment within the preset time interval is sent to a server communicatively connected to the terminal, and the score-following information of each note played in the performance segment within the preset time interval is generated and sent to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the score-following information of each note played in each performance segment.

[0036] In S103, receive and display the performance feedback result sent by the server.

[0037] In the present disclosure, the terminal sends the performance segment within the preset time interval (for example, 10 seconds) to the server communicatively connected to the terminal at every preset time interval, that is, streams the performance music.

[0038] In addition, the score-following information may include the performance pitch, the reference pitch, the performance start time, and the reference start time. The performance feedback result may include the deviated notes and / or the score of the current performance music, where the deviated notes may include misplayed notes (i.e., played as other wrong notes), missed notes, lagged-played notes (i.e., the playing time of the notes lags), and notes with inconsistent note lengths (notes with too long or too short note lengths).

[0039] In one implementation, the performance feedback result includes the deviated notes in the current performance music. In this way, the performer can correct in time according to the deviated notes in subsequent performances.

[0040] In another implementation, the performance feedback result includes the score of the current performance music. In this way, the performer can obtain the overall evaluation of the current performance.

[0041] In yet another implementation, the performance feedback result includes both the deviated notes in the current performance music and the score of the current performance music.

[0042] After the terminal generates the score-following information of each note played in the performance segment within the preset time interval, it can send it to the server; the server receives the performance segments sent by the terminal at every preset time interval and the score-following information of each note played in the performance segment, and at the end of the performance, generates the performance feedback result of the current performance music according to each performance segment of the current performance music and the score-following information of each note played in each performance segment; then, the server feeds back the performance feedback result to the terminal; the terminal receives and displays the performance feedback result. Exemplarily, the terminal may display the communication messages with the server in the form of a session.

[0043] In the above technical solution, the performance segments within a preset time interval are sent to the server communicatively connected to the terminal at every preset time interval, and the score-following information for each note played in the performance segments within the preset time interval is generated and sent to the server. Then, based on each performance segment of the currently played music and the score-following information for each note played in each performance segment, the server generates a performance feedback result for the currently played music; and the performance feedback result sent by the server is received and displayed. In this way, the performance feedback result can be automatically generated, saving manpower and ensuring the objectivity and accuracy of the performance feedback result. Additionally, since the performance segments and the corresponding score-following information are transmitted while playing, the server can quickly generate the performance feedback result for the currently played music within a short time after the performance ends, avoiding the transmission delay problem caused by transmitting large unstructured performance music only at the end of the performance, so that the performer can quickly obtain the performance feedback result and the performance experience of the performer is improved.

[0044] In addition, the terminal can establish a connection with the server in the following way: First, the terminal sends a start message for requesting to establish a connection to the server; after receiving the start message, the server sends a connection response message to the terminal; after receiving the connection response message, the two establish a connection, thus ensuring a reliable connection between the terminal and the server.

[0045] Similarly, the terminal can disconnect from the server in a way similar to establishing a connection with the server. Specifically, first, the terminal sends an end message for requesting to disconnect to the server; after receiving the end message, the server sends a disconnection response message to the terminal; after receiving the disconnection response message, the two disconnect.

[0046] To avoid packet loss and ensure the integrity of data transmission, each time the server receives a performance segment or the score-following information for each note played in the performance segment sent by the terminal, it can send an acknowledgment receipt message to the terminal; if the terminal does not receive the acknowledgment receipt message within a preset time threshold after sending the performance segment or the score-following information for each note played in the performance segment, it sends the performance segment or the score-following information for each note played in the performance segment to the server again.

[0047] Next, the specific implementation manner of generating the score-following information for each note played in the performance segments within the preset time interval in S102 above will be described in detail. Specifically, it can be achieved through the following steps (1) to (3):

[0048] (1) Transcribe the performance segments within the preset time interval to obtain the performance score information corresponding to the performance segments.

[0049] In the present disclosure, the performance score information is structured data, which may specifically include the pitch, performance start time, performance end time, etc. of each note played in the performance segment. It reflects the pitch accuracy at a certain moment, reflects that the performer plays a note at a certain moment, and all information that can reflect relevant information such as pitch, note start time, and note end time in the score is called performance score information. Among them, the score can be a staff notation, a numbered musical notation, etc.

[0050] (2) Obtain the standard score information of the standard music segment corresponding to the performance segment.

[0051] In the present disclosure, the standard score information may include information such as the identifier of each note corresponding to the standard music segment, reference start time, reference end time, reference pitch, the bar it belongs to, the phrase it belongs to, and the hand used for playing (left hand or right hand).

[0052] Specifically, the standard score information of the standard music segment corresponding to the performance segment can be obtained in the following way: The name of the current performance music can be obtained by identifying the first performance segment or according to the name of the performance music input / selected by the performer through the keyboard or voice; afterwards, the standard score information corresponding to the current performance music can be obtained from the local or network through the music name; finally, the score information corresponding to the start and end times of the current performance segment in the standard score information corresponding to the current performance music is determined as the standard score information of the standard music segment corresponding to the performance segment.

[0053] (3) Generate the score-following information of each note played in the performance segment according to the performance score information and the standard score information.

[0054] In the present disclosure, the Dynamic Time Warping (DTW) algorithm can be used to align the performance score information and the standard score information, so as to obtain the score-following information of each note played in the performance segment.

[0055] The following will specifically describe the specific implementation manner in which the above server generates the performance feedback result of the current performance music according to each performance segment of the current performance music and the score-following information of each note played in each performance segment.

[0056] In one embodiment, the score-following information of each note played in each performance segment of the currently played music can be aggregated (i.e., the score-following information of each note played in each performance segment of the currently played music is summarized) to obtain the first deviated note in the currently played music. At the same time, each performance segment of the currently played music can be spliced, and the score-following information of the played music obtained after splicing can be determined, and based on this score-following information, the second deviated note in the currently played music can be generated. After that, the first deviated note and the second deviated note are merged, and the merged deviated note is used as the performance feedback result. By determining and merging the deviated notes in two different ways, the integrity of the deviated notes can be improved.

[0057] In another embodiment, the score-following information of each note played in each performance segment of the currently played music can be aggregated to obtain the first deviated note in the currently played music. At the same time, each performance segment of the currently played music can be spliced, and the score-following information of the played music obtained after splicing can be determined, and based on this score-following information, the second deviated note in the currently played music can be generated. After that, the first deviated note and the second deviated note are merged. Finally, based on the quantity and position of the merged deviated notes, the score of the currently played music is determined, and the score is used as the performance feedback result.

[0058] In yet another embodiment, the score-following information of each note played in each performance segment of the currently played music can be aggregated to obtain the first deviated note in the currently played music. At the same time, each performance segment of the currently played music can be spliced, and the score-following information of the played music obtained after splicing can be determined, and based on this score-following information, the second deviated note in the currently played music can be generated. After that, the first deviated note and the second deviated note are merged. Finally, based on the quantity and position of the merged deviated notes, the score of the currently played music is determined, and the score and the merged deviated notes are jointly used as the performance feedback result.

[0059] Figure 2 is a flowchart of a musical instrument assisted practice method shown according to another exemplary embodiment, where the method is applied to a terminal. As Figure 2 shown, the above method may further include the following S104.

[0060] In S104, the score-following information is displayed.

[0061] In this way, the performer can understand in detail the score-following situation of the current performance segment, reducing the possibility of subsequent performance errors.

[0062] Figure 3 is a flowchart of a musical instrument assisted practice method shown according to another exemplary embodiment, where the method is applied to a terminal. AsFigure 3 As shown in the figure, the above method may further include the following S105 and S106.

[0063] In S105, if no performance feedback result is received within a preset duration after the performance ends, a performance feedback result of the currently performed music is generated according to the score-following information of each note played in each performance segment of the currently performed music.

[0064] In S106, the performance feedback result is displayed.

[0065] In the present disclosure, when there is a communication failure between the terminal and the server or the server fails, the terminal will not be able to obtain the performance feedback result from the server. At this time, the terminal can locally generate the performance feedback result of the currently performed music and display it. In this way, even if there is a communication failure or server failure with the server, the performer can still obtain the performance feedback result.

[0066] In addition, in order to maintain data consistency between the terminal and the server, after the failure is resolved, the terminal can synchronize the locally generated performance feedback result to the server.

[0067] The following will detail the specific implementation manner of generating the performance feedback result of the currently performed music according to the score-following information of each note played in each performance segment of the currently performed music in the above S105.

[0068] In one implementation manner, the score-following information of each note played in each performance segment of the currently performed music can be aggregated to obtain the deviated notes in the currently performed music; then, the deviated notes are used as the performance feedback result.

[0069] In another implementation manner, the score-following information of each note played in each performance segment of the currently performed music can be aggregated to obtain the deviated notes in the currently performed music; then, according to the quantity and position of the deviated notes, the score of the currently performed music is determined, and the score is used as the performance feedback result.

[0070] In yet another implementation manner, the score-following information of each note played in each performance segment of the currently performed music can be aggregated to obtain the deviated notes in the currently performed music; then, according to the quantity and position of the deviated notes, the score of the currently performed music is determined, and the score and the deviated notes are jointly used as the performance feedback result.

[0071] The present disclosure also provides an instrument-assisted practice device, where the terminal can be applied to the terminal. As Figure 4 shown, the device 400 includes:

[0072] An acquisition module 401, configured to acquire the performed music in real time;

[0073] A sending module 402 is configured to send the performance segments within the preset time interval collected by the collection module 401 to a server communicatively connected to the terminal at each preset time interval, and generate follow-up score information for each note played in the performance segments within the preset time interval, and send the follow-up score information to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the follow-up score information for each note played in each performance segment;

[0074] A display module 403 is configured to receive and display the performance feedback result sent by the server.

[0075] In the above technical solution, the performance segments within the preset time interval are sent to a server communicatively connected to the terminal at each preset time interval, and the follow-up score information for each note played in the performance segments within the preset time interval is generated, and the follow-up score information is sent to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the follow-up score information for each note played in each performance segment; receive and display the performance feedback result sent by the server. Thereby, the performance feedback result can be automatically generated, saving manpower and ensuring the objectivity and accuracy of the performance feedback result. In addition, since the performance segments and the corresponding follow-up score information are transmitted while performing, thereby, the server can quickly generate a performance feedback result of the current performance music within a short time after the performance ends, avoiding the transmission delay problem caused by transmitting large-scale unstructured performance music only at the end of the performance, so that the performer can quickly obtain the performance feedback result and improve the performance experience of the performer.

[0076] Optionally, the sending module 402 includes:

[0077] A transcription sub-module is configured to perform music transcription on the performance segments within the preset time interval to obtain performance score information corresponding to the performance segments;

[0078] An acquisition sub-module is configured to acquire standard score information of a standard music segment corresponding to the performance segment;

[0079] A generation sub-module is configured to generate follow-up score information for each note played in the performance segment according to the performance score information and the standard score information.

[0080] Optionally, the follow-up score information includes performance pitch, reference pitch, performance start time, and reference start time.

[0081] Optionally, the performance feedback result includes deviation notes in the current performance music and / or a score of the current performance music.

[0082] Optionally, the display module 403 is further configured to display the score-following information.

[0083] Optionally, the apparatus 400 further includes:

[0084] a generating module, configured to generate a performance feedback result of the current performance music according to the score-following information of each note played in each performance segment of the current performance music if the performance feedback result is not received within a preset duration after the performance ends;

[0085] The display module 403 is further configured to display the performance feedback result.

[0086] Optionally, the performance feedback result includes the deviated notes in the current performance music and the score of the current performance music;

[0087] The generating module includes:

[0088] an aggregating sub-module, configured to aggregate the score-following information of each note played in each performance segment of the current performance music to obtain the deviated notes in the current performance music;

[0089] a determining sub-module, configured to determine the score of the current performance music according to the number and positions of the deviated notes.

[0090] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0091] The present disclosure further provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of the above-described musical instrument-assisted practice method provided by the present disclosure are implemented.

[0092] Next, referring to Figure 5 , which shows a schematic structural diagram of an electronic device (such as a terminal device) 500 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0093] As Figure 5As shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to a program stored in the read-only memory (ROM) 502 or a program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0094] Generally, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or wirelessly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0095] Specifically, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0096] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0097] In some embodiments, the client can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0098] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; it can also exist separately and not be assembled into the electronic device.

[0099] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: collect playing music in real time; send the playing segments within the preset time interval to a server communicatively connected to the terminal at every preset time interval, generate follow-up score information for each note played in the playing segments within the preset time interval, and send the follow-up score information to the server, so that the server generates a playing feedback result of the current playing music according to each playing segment of the current playing music and the follow-up score information for each note played in each playing segment; receive and display the playing feedback result sent by the server.

[0100] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0102] The modules involved in the embodiments of the present disclosure can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the module itself in some cases. For example, the acquisition module can also be described as "the module for real-time acquisition of played music".

[0103] The functions described above herein can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0104] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] According to one or more embodiments of the present disclosure, Example 1 provides a method for musical instrument assisted practice, including: real-time acquisition of played music; sending the played segments within the preset time interval to a server communicatively connected to the terminal at every preset time interval, and generating score-following information for each note played in the played segments within the preset time interval, and sending the score-following information to the server, so that the server generates a performance feedback result of the current played music according to each played segment of the current played music and the score-following information for each note played in each played segment; receiving and displaying the performance feedback result sent by the server.

[0106] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1. Generating the score-following information of each note played in the performance segment within the preset time interval includes: performing music transcription on the performance segment within the preset time interval to obtain the performance score information corresponding to the performance segment; obtaining the standard score information of the standard music segment corresponding to the performance segment; and generating the score-following information of each note played in the performance segment according to the performance score information and the standard score information.

[0107] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1. The score-following information includes performance pitch, reference pitch, performance start time, and reference start time.

[0108] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1. The performance feedback result includes the deviated notes in the currently performed music and / or the score of the currently performed music.

[0109] According to one or more embodiments of the present disclosure, Example 5 provides the method of any one of Examples 1-4. The method further includes: displaying the score-following information.

[0110] According to one or more embodiments of the present disclosure, Example 6 provides the method of any one of Examples 1-4. The method further includes: if the performance feedback result is not received within a preset duration after the performance ends, generating the performance feedback result of the currently performed music according to the score-following information of each note played in each performance segment of the currently performed music; and displaying the performance feedback result.

[0111] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 6. The performance feedback result includes the deviated notes in the currently performed music and the score of the currently performed music. Generating the performance feedback result of the currently performed music according to the score-following information of each note played in each performance segment of the currently performed music includes: aggregating the score-following information of each note played in each performance segment of the currently performed music to obtain the deviated notes in the currently performed music; and determining the score of the currently performed music according to the number and positions of the deviated notes.

[0112] According to one or more embodiments of the present disclosure, Example 8 provides a musical instrument assisted practice device, which is applied to a terminal and includes: a collection module, configured to collect playing music in real time; a sending module, configured to send the playing segment within the preset time interval collected by the collection module to a server communicatively connected to the terminal at every preset time interval, and generate follow - score information for each note played in the playing segment within the preset time interval, and send the follow - score information to the server, so that the server generates a playing feedback result of the current playing music according to each playing segment of the current playing music and the follow - score information for each note played in each playing segment; a display module, configured to receive and display the playing feedback result sent by the server.

[0113] According to one or more embodiments of the present disclosure, Example 9 provides the device of Example 8, and the sending module includes: a transcription sub - module, configured to transcribe the playing segment within the preset time interval to obtain playing score information corresponding to the playing segment; an obtaining sub - module, configured to obtain standard score information of a standard music segment corresponding to the playing segment; a generating sub - module, configured to generate follow - score information for each note played in the playing segment according to the playing score information and the standard score information.

[0114] According to one or more embodiments of the present disclosure, Example 10 provides the device of Example 8, and the follow - score information includes playing pitch, reference pitch, playing start time, and reference start time.

[0115] According to one or more embodiments of the present disclosure, Example 11 provides the device of Example 8, and the playing feedback result includes deviated notes in the current playing music and / or a score of the current playing music.

[0116] According to one or more embodiments of the present disclosure, Example 12 provides the device of any one of Examples 8 - 11, and the display module is further configured to display the follow - score information.

[0117] According to one or more embodiments of the present disclosure, Example 13 provides the device of any one of Examples 8 - 11, and the device further includes: a generating module, configured to generate a playing feedback result of the current playing music according to the follow - score information for each note played in each playing segment of the current playing music if the playing feedback result is not received within a preset duration after the playing ends; the display module is further configured to display the playing feedback result.

[0118] According to one or more embodiments of the present disclosure, Example 14 provides the apparatus of Example 13. The performance feedback result includes the deviated notes in the current performance music and the score of the current performance music. The generation module includes: an aggregation sub-module for aggregating the score-following information of each note played in each performance segment of the current performance music to obtain the deviated notes in the current performance music; and a determination sub-module for determining the score of the current performance music according to the number and positions of the deviated notes.

[0119] According to one or more embodiments of the present disclosure, Example 15 provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processing device, the steps of the method according to any one of Examples 1-7 are implemented.

[0120] According to one or more embodiments of the present disclosure, Example 16 provides an electronic device, including: a storage device having at least one computer program stored thereon; and at least one processing device for executing the at least one computer program in the storage device to implement the steps of the method according to any one of Examples 1-7.

[0121] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the present disclosure.

[0122] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0123] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

Claims

1. An instrument-assisted practice method, applied to a terminal, characterized in that Including: Real-time collection of the played music; At each preset time interval, sending the performance segment within the preset time interval to a server communicatively connected to the terminal, generating follow-up information for each note played in the performance segment within the preset time interval, and sending the follow-up information to the server, so that the server generates a performance feedback result of the current played music based on each performance segment of the current played music and the follow-up information for each note played in each performance segment; Receiving and displaying the performance feedback result sent by the server; Wherein, the performance feedback result of the current played music is obtained by the server merging the first deviation notes in the current played music and the second deviation notes in the current played music, and then obtaining the deviation notes from the merged deviation notes. The first deviation notes are aggregated by the server from the follow-up information for each note played in each performance segment of the current played music; the second deviation notes are obtained by the server splicing each performance segment of the current played music, determining the follow-up information of the played music obtained after splicing, and generating based on the follow-up information of the played music obtained after splicing.

2. The method according to claim 1, wherein The generating the follow-up information for each note played in the performance segment within the preset time interval includes: Performing music transcription on the performance segment within the preset time interval to obtain performance score information corresponding to the performance segment; Obtaining standard score information of a standard music segment corresponding to the performance segment; Generating the follow-up information for each note played in the performance segment based on the performance score information and the standard score information.

3. The method according to claim 1, characterized in that, The follow-up information includes playing pitch, reference pitch, playing start time, and reference start time.

4. The method according to claim 1, wherein The performance feedback result includes the merged deviation notes and / or the score of the current played music, wherein the score is determined according to the quantity and position of the merged deviation notes.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Displaying the follow-up information.

6. The method according to any one of claims 1-4, characterized in that The method further includes: If the performance feedback result is not received within a preset duration after the performance ends, generating the performance feedback result of the current played music based on the follow-up information for each note played in each performance segment of the current played music; Displaying the performance feedback result.

7. The method according to claim 6, wherein The performance feedback result includes the deviation notes in the current played music and the score of the current played music; The generating the performance feedback result of the current played music based on the follow-up information for each note played in each performance segment of the current played music includes: Aggregating the follow-up information for each note played in each performance segment of the current played music to obtain the deviation notes in the current played music; Determining the score of the current played music according to the quantity and position of the deviation notes.

8. An instrument auxiliary practice device, applied to a terminal, characterized in that Including: A collection module for real-time collection of the played music; A sending module, configured to send the performance segments collected by the collection module within the preset time interval to a server communicatively connected to the terminal at each preset time interval, generate follow-up score information for each note played in the performance segments within the preset time interval, and send the follow-up score information to the server, so that the server generates a performance feedback result of the current performance music according to each performance segment of the current performance music and the follow-up score information for each note played in each performance segment; A display module, configured to receive and display the performance feedback result sent by the server; Wherein, the performance feedback result of the current performance music is obtained by the server merging the first deviation notes in the current performance music and the second deviation notes in the current performance music, and then obtaining the deviation notes from the merged deviation notes. The first deviation notes are aggregated by the server based on the follow-up score information for each note played in each performance segment of the current performance music; the second deviation notes are generated by the server splicing each performance segment of the current performance music, determining the follow-up score information of the spliced performance music, and generating based on the follow-up score information of the spliced performance music.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a processing device, it implements the steps of the method according to any one of claims 1-7.

10. An electronic device, characterized in that, Comprising: A storage device storing at least one computer program thereon; At least one processing device, configured to execute the at least one computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

Images