Artificial intelligence-based scoring system for harp playing and use method thereof

Abstract

The invention discloses a scoring system for harp playing based on artificial intelligence, and relates to the technical field of harp playing scoring. The scoring hardware comprises optical detection sources fixedly installed on the two sides of the harp frame and a sound wave detection source fixedly installed in the harp base, the scoring hardware obtains a playing sound source and string vibration data when a person plays the harp, and examination tracks for harp playing scoring are arranged in the intelligent music library. The scoring software provides scores based on the playing level of the player, the scoring software is interconnected with the intelligent music library, a whole or fragment test question track is selected after the playing level of the player is referred, the system further comprises a CNN convolution algorithm, and the CNN convolution algorithm is rooted in the scoring hardware, the intelligent music library and the scoring software. The scoring hardware is used for learning the playing posture of the player, the intelligent music library is used for learning the music playing difficulty, and the scoring software is used for automatically selecting the graded playing music, so that the music suitable for the player to examine or play can be selected for the player, and the scoring result matched with the playing level can be obtained.

Description

Technical Field

[0001] The invention relates to the technical field of harp performance scoring, in particular to an artificial intelligence-based harp performance scoring system and a use method thereof. Background Art

[0002] A harp is a stringed instrument that usually has forty-eight strings mounted on an upright triangular frame. The player produces music by plucking these strings.

[0003] In the performance part, candidates need to choose at least one piece from each of the three repertoire lists (Groups A, B and C) in the syllabus, and can also choose a piece of their own choice. These pieces need to be played in a continuous and uninterrupted manner, and the overall performance time should not exceed the maximum time set for this level. In the scale and arpeggio parts, candidates need to play the keys within the specified range from memory.

[0004] However, examiners are required to participate in the harp grading review. Examiners are prone to fatigue after a long period of review, which may cause deviations in the review results. The scoring results for some candidates are biased, and there is room for improvement. Summary of the invention

[0005] The object of the present invention is to provide a harp performance scoring system based on artificial intelligence and a method of using the same to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a harp performance scoring system based on artificial intelligence, comprising scoring hardware, wherein the scoring hardware comprises optical detection sources fixedly mounted on both sides of a harp frame and a sound wave detection source fixedly mounted inside a harp seat, wherein the scoring hardware obtains performance sound sources and string vibration data when a person performs; an intelligent music library, wherein the intelligent music library contains test repertoires for harp performance scoring; and scoring software, wherein the scoring software proposes a score based on the performance level of the performer; the scoring software is interconnected with the intelligent music library, and selects a whole or fragmentary test repertoire after referring to the performance level of the performer.

[0007] A harp performance scoring system based on artificial intelligence and a method of using the same, comprising:

[0008] Step S1: collecting the hand performance movements and performance sound source data of the performer during the performance through the scoring hardware;

[0009] Step S2: The simulation unit establishes a coordinate axis and corresponds the hand performance action and the performance sound source data to the same time point;

[0010] Step S3: The CNN convolution algorithm simulates the performer's hand performance movements and generates frame-by-frame animations;

[0011] Step S4: CNN convolution algorithm analyzes the sound source data and determines the inharmonious sound source;

[0012] Step S5: CNN convolution algorithm combines the animation and the discordant sound source to judge the performer's level;

[0013] Step S6: the intelligent music library integrates the CNN convolution algorithm to select the corresponding whole or fragment music according to the performer's level;

[0014] Step S7: The scoring software scores the performance.

[0015] As a further solution of the present invention: a harp performance scoring system based on artificial intelligence also includes a CNN convolution algorithm, which is rooted in the scoring hardware, the intelligent music library and the scoring software, and the scoring hardware learns the performer's playing posture, the intelligent music library learns the difficulty of the performance repertoire, and the scoring software automatically selects graded performance repertoires.

[0016] As a further solution of the present invention: a scoring system for harp performance based on artificial intelligence, the scoring software includes a user ER model, the user ER model is connected to the Internet and can crawl harp performance repertoire information on the Internet through python, the repertoire information includes repertoire transcription information, comment information, and repertoire collection and download number information, the CNN convolution algorithm comprehensively judges the difficulty of the performance repertoire based on the repertoire transcription information, comment information, and repertoire collection and download number information crawled by python, and divides the difficulty level.

[0017] As a further solution of the present invention: a scoring system for harp performance based on artificial intelligence, the intelligent music library includes a virtual truncation device, the virtual truncation device cooperates with the CNN convolution algorithm and can selectively truncate the performance repertoire through the CNN convolution algorithm, and the truncation standard comes from the difficulty level determined by the CNN convolution algorithm.

[0018] As a further solution of the present invention: a scoring system for harp performance based on artificial intelligence, the optical detection source includes an IPC element, the IPC element includes an image recording unit, a point cloud conversion unit, a point cloud judgment unit and a model output unit, the image recording unit records images of the performer's hands and harp together, the point cloud conversion unit converts the pixels contained in the recorded image into a point cloud with readable color, the point cloud judgment unit judges and distinguishes the model of the person's hand, harp or external environment according to the color, and the model output unit outputs the person's hand model.

[0019] As a further solution of the present invention: a scoring system for harp performance based on artificial intelligence, the sound wave detection source includes a wave sensor, the wave sensor includes a collection unit and a wave phase output unit, the collection unit collects sound wave data generated when the strings vibrate, and the wave phase output unit outputs visualized waveform data according to the sound wave data.

[0020] As a further solution of the present invention: an artificial intelligence-based scoring system for harp performance also includes a simulation unit, which includes a coordinate simulation unit, and the coordinate simulation unit simulates the data collected or output by the optical detection source and the sound wave detection source at the same time point.

[0021] As a further solution of the present invention: a harp performance scoring system based on artificial intelligence also includes a noise reduction unit fixedly installed in the harp seat, the noise reduction unit includes a noise collection unit, a noise inversion unit and an isolation unit, the noise collection unit collects noise other than the performance, the noise inversion unit performs phase analysis on the collected noise and simulates an inverted waveform, and the isolation unit outputs an inverted waveform to reduce the noise sound and alleviate the interference of noise on the performance sound.

[0022] As a further solution of the present invention: a grading system for harp performance based on artificial intelligence, the grades include S, A, B, C and failing, and the difficulty levels of S, A, B, C are arranged in descending order.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. Since the scoring hardware includes an optical detection source fixedly installed on both sides of the piano frame and a sound wave detection source fixedly installed inside the piano seat, the scoring hardware obtains the performance sound source and string vibration data when the person performs, and the intelligent music library is provided with test repertoires for harp performance scoring. The scoring software proposes a score based on the performer's performance level. The scoring software is interconnected with the intelligent music library, and selects the entire or fragmentary test repertoire after referring to the performer's performance level. It also includes the CNN convolution algorithm. The CNN convolution algorithm is rooted in the scoring hardware, the intelligent music library and the scoring software. The scoring hardware learns the performer's performance posture, the intelligent music library learns the difficulty of the performance repertoire, and the scoring software automatically selects graded performance repertoires. Therefore, it is possible to select repertoires suitable for the performer's grading or performance, and obtain a scoring result that matches his or her performance level.

[0025] 2. Since the user ER model is connected to the Internet and can crawl the harp performance information on the Internet through Python, including the transcription information, comment information, and the number of collections and downloads of the songs, the CNN convolution algorithm determines the performance level of the person based on the scoring hardware, and then crawls the songs on the Internet based on Python, automatically classifies the difficulty, and matches the performance level of the performer before conducting an assessment, thus achieving a quantifiable scoring standard at the same level.

[0026] 3. Since the noise reduction unit includes a noise collection unit, a noise inversion unit and an isolation unit, the noise collection unit collects noise other than the performance, the noise inversion unit performs phase analysis on the collected noise and simulates an inverted waveform, and the isolation unit outputs an inverted waveform to reduce the noise sound and reduce the interference of noise on the performance sound. Therefore, the setting of the noise reduction unit can effectively reduce the impact of environmental noise on the performance score and improve the accuracy of the score. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic flow chart of the steps and methods in a harp performance scoring system based on artificial intelligence and a method for using the same according to the present invention;

[0028] Figure 2 It is a schematic structural diagram of scoring hardware in an artificial intelligence-based harp performance scoring system and a method for using the same according to the present invention;

[0029] Figure 3 A schematic diagram of a simulated coordinate axis in a harp performance scoring system based on artificial intelligence and a method for using the same in the present invention;

[0030] Figure 4 The present invention provides a principle diagram of noise inversion in an artificial intelligence-based harp performance scoring system and a method for using the same. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In the description of the present invention, it should be noted that, for those skilled in the art, the specific meanings of the above terms in the present invention can be understood in specific situations. The following describes an embodiment of the present invention based on its overall structure.

[0033] In an embodiment of the present invention, a scoring system for harp performance based on artificial intelligence includes scoring hardware, an intelligent music library and scoring software, wherein the scoring hardware includes an optical detection source fixedly installed on both sides of a harp frame and a sound wave detection source fixedly installed inside a harp seat, the scoring hardware acquires performance sound source and string vibration data when a person performs, the intelligent music library includes test repertoires for harp performance scoring, the scoring software proposes a score based on the performance level of the performer, the scoring software is interconnected with the intelligent music library, selects a whole or fragment test repertoire after referring to the performance level of the performer, and further includes a CNN convolution algorithm, the CNN convolution algorithm is rooted in the scoring hardware, the intelligent music library and the scoring software, and the scoring hardware learns the performance posture of the performer, the intelligent music library learns the difficulty of the performance repertoire, and the scoring software automatically selects graded performance repertoires;

[0034] See also Figure 1 , a harp performance scoring system based on artificial intelligence and a method of using the same, comprising

[0035] Step S1: collecting the hand performance movements and performance sound source data of the performer during the performance through the scoring hardware;

[0036] Step S2: The simulation unit establishes a coordinate axis and corresponds the hand performance action and the performance sound source data to the same time point;

[0037] Step S3: The CNN convolution algorithm simulates the performer's hand performance movements and generates frame-by-frame animations;

[0038] Step S4: CNN convolution algorithm analyzes the sound source data and determines the inharmonious sound source;

[0039] Step S5: CNN convolution algorithm combines the animation and the discordant sound source to judge the performer's level;

[0040] Step S6: the intelligent music library integrates the CNN convolution algorithm to select the corresponding whole or fragment music according to the performer's level;

[0041] Step S7: The scoring software scores the performance.

[0042] Specifically, the optical detection source includes an IPC element, and the IPC element includes an image recording unit, a point cloud conversion unit, a point cloud judgment unit and a model output unit. The image recording unit records images of the performer's hands and harp together, and the point cloud conversion unit converts pixels contained in the recorded image into a point cloud with readable color. The point cloud judgment unit judges and distinguishes the model of the person's hand, harp or external environment according to the color, and the model output unit outputs the person's hand model. The sound wave detection source includes a wave sensor, and the wave sensor includes a collection unit and a wave phase output unit. The collection unit collects sound wave data generated when the strings vibrate, and the wave phase output unit outputs visualized waveform data according to the sound wave data.

[0043] At the same time, it also includes a simulation unit, which includes a coordinate simulation unit. The coordinate simulation unit simulates the data collected or output by the optical detection source and the sound wave detection source at the same time point.

[0044] Other embodiments of the present invention: please refer to Figure 2 and Figure 3 In summary, after the optical detection source in the scoring hardware is installed on the side of the harp frame and the sound wave detection source is installed inside the harp seat, on the one hand, the IPC element in the optical detection source detects the hand movements of the person and converts them into continuous frame images, and on the other hand, the sound wave detection source is combined to convert the recorded sound source into a continuous waveform and put it into the same coordinate axis with time as the scale. Then, the CNN convolution algorithm intervenes in the coordinate axis to capture the sections with abnormal waveform changes (including sudden large-span changes in peaks and troughs, short and rapid band changes, and other sound bands that do not conform to the beautiful music waveform) and correspond them to the corresponding time points, and preliminarily define them as suspicious bad performances;

[0045] Then the CNN convolution algorithm intelligently simulates the continuous frame images collected by the IPC component. The specific method of the intelligent simulation is to further refine the number of frames and analyze and learn the frame number, and then simulate the hand motion trajectory animation of the person and assign the animation to the timeline. After that, the animation is matched with the sound wave in the timeline at the same time to determine whether the person’s hand posture is correct, and finally evaluate the person’s performance effect.

[0046] Other embodiments of the present invention: the scoring software includes a user ER model, the user ER model is connected to the Internet and can crawl the information of harp performances on the Internet through python, the music information includes music transcription information, comment information, and music collection and download number information, the CNN convolution algorithm comprehensively judges the difficulty of the performance music according to the music transcription information, comment information, and music collection and download number information crawled by python, and divides the difficulty level, and the intelligent music library includes a virtual truncation device, the virtual truncation device cooperates with the CNN convolution algorithm and can selectively truncate the performance music through the CNN convolution algorithm, and the truncation standard comes from the difficulty level determined by the CNN convolution algorithm, the levels include S, A, B, C and failure, the difficulty levels of S, A, B, C are arranged in descending order, and S, A, B, C are the scoring standards for different grading difficulties;

[0047] In this embodiment, since the user ER model is connected to the Internet and can crawl the harp performance information on the Internet through Python, including the transcription information, comment information, and the number of collections and downloads of the songs, the CNN convolution algorithm determines the performance level of the person based on the scoring hardware, and then crawls the songs on the Internet based on Python, automatically classifies the difficulty, and matches the performance level of the performer before conducting an assessment, thereby realizing a quantifiable scoring standard at the same level.

[0048] Other embodiments of the present invention: please refer to Figure 4 , also includes a noise reduction unit fixedly installed in the piano seat, the noise reduction unit includes a noise collection unit, a noise inversion unit and an isolation unit, the noise collection unit collects noise other than performance, the noise inversion unit performs phase analysis on the collected noise and simulates an inverted waveform, the isolation unit outputs an inverted waveform to reduce the noise sound and alleviate the interference of noise on the performance sound. Therefore, the setting of the noise reduction unit can effectively reduce the influence of environmental noise on the performance score and improve the accuracy of the score.

[0049] The working principle of the present invention is: after the optical detection source in the scoring hardware is installed on the side of the harp frame and the sound wave detection source is installed inside the harp seat, on the one hand, the IPC element in the optical detection source is used to detect the hand movements of the personnel and convert them into continuous frame images, and on the other hand, the sound wave detection source is combined to convert the recorded sound source into a continuous waveform and store it in the same coordinate axis with time as the scale, and then the CNN convolution algorithm intervenes in the coordinate axis, and the sections with abnormal waveform changes (including sudden large-span changes in peaks and troughs, short and rapid band changes, and other sound wave bands that do not conform to the beautiful music waveform) are captured and corresponded to the corresponding time points, and preliminarily defined as suspicious and bad performances; then the CNN convolution algorithm performs intelligent simulation on the continuous frame images collected by the IPC element, and the specific method of the intelligent simulation is to further refine the frame number and analyze and learn the frame number, and then simulate the animation of the movement trajectory of the personnel's hands and assign the animation to the time axis, and then correspond the animation with the sound waves in the time axis at the same time, judge whether the personnel's hand posture is correct, and finally evaluate the performance effect of the personnel.

[0050] At the same time, the user ER model is connected to the Internet and can crawl the harp music information on the Internet through Python. The music information includes music transcription information, comment information, and the number of music collections and downloads. The CNN convolution algorithm comprehensively judges the difficulty of the music according to the music transcription information, comment information, and the number of music collections and downloads crawled by Python, and divides the difficulty level. In addition, the intelligent music library includes a virtual truncation device, which cooperates with the CNN convolution algorithm and can selectively truncate the music through the CNN convolution algorithm. The truncation standard comes from the difficulty level determined by the CNN convolution algorithm, etc. The levels include S, A, B, C and fail. The difficulty levels of S, A, B, C are arranged in descending order. S, A, B, C are the scoring standards for different levels of difficulty. Specifically, since the user ER model is connected to the Internet and can crawl the harp performance information on the Internet through python, including the score transcription information, comment information, and the number of collections and downloads of the songs, the CNN convolution algorithm determines the performance level of the person based on the scoring hardware, and then crawls the songs on the Internet based on python, automatically grades the difficulty and matches the performance level of the performer for assessment, thus realizing a quantifiable scoring standard at the same level.

[0051] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A harp performance scoring system based on artificial intelligence, characterized in that: The scoring hardware includes an optical detection source fixedly mounted on both sides of the piano frame and a sound wave detection source fixedly mounted inside the piano seat, and the scoring hardware obtains the performance sound source and string vibration data when the person performs; An intelligent music library, wherein the intelligent music library includes examination repertoire for harp performance scoring; as well as Scoring software that provides scores based on the performer's performance level; The scoring software is interconnected with the intelligent music library and selects a whole or a fragment of the test music after referring to the performance level of the performer.

2. The harp performance scoring system based on artificial intelligence according to claim 1, characterized in that: It also includes a CNN convolution algorithm, which is rooted in the scoring hardware, intelligent music library and scoring software. The scoring hardware learns the performer's playing posture, the intelligent music library learns the difficulty of the performance repertoire, and the scoring software automatically selects graded performance repertoires.

3. The harp performance scoring system based on artificial intelligence according to claim 2, characterized in that: The scoring software includes a user ER model, which is connected to the Internet and can crawl harp music information on the Internet through python. The music information includes music transcription information, comment information, and the number of music collections and downloads. The CNN convolution algorithm comprehensively judges the difficulty of the music to be performed based on the music transcription information, comment information, and the number of music collections and downloads crawled by python, and divides the difficulty level.

4. The harp performance scoring system based on artificial intelligence according to claim 3, characterized in that: The intelligent music library includes a virtual truncation device, which cooperates with the CNN convolution algorithm and can selectively truncate the performance repertoire through the CNN convolution algorithm, and the truncation standard comes from the difficulty level determined by the CNN convolution algorithm.

5. The harp performance scoring system based on artificial intelligence according to claim 4, characterized in that: The optical detection source includes an IPC element, which includes an image recording unit, a point cloud conversion unit, a point cloud judgment unit and a model output unit. The image recording unit records images of the performer's hands and harp together. The point cloud conversion unit converts the pixels contained in the recorded image into a point cloud with readable color. The point cloud judgment unit judges and distinguishes the models of the person's hands, harp or external environment based on the color. The model output unit outputs the person's hand model.

6. The harp performance scoring system based on artificial intelligence according to claim 5, characterized in that: The sound wave detection source includes a wave sensor, and the wave sensor includes a collection unit and a wave phase output unit. The collection unit collects the sound wave data generated when the strings vibrate, and the wave phase output unit outputs visualized waveform data according to the sound wave data.

7. The harp performance scoring system based on artificial intelligence according to claim 6, characterized in that: It also includes a simulation unit, which includes a coordinate simulation unit. The coordinate simulation unit simulates the data collected or output by the optical detection source and the sound wave detection source at the same time point.

8. The harp performance scoring system based on artificial intelligence according to claim 7, characterized in that: It also includes a noise reduction unit fixedly installed in the piano seat, which includes a noise collection unit, a noise inversion unit and an isolation unit. The noise collection unit collects noise other than performance, the noise inversion unit performs phase analysis on the collected noise and simulates an inversion waveform, and the isolation unit outputs an inversion waveform to reduce the noise sound and alleviate the interference of noise on the performance sound.

9. The harp performance scoring system based on artificial intelligence according to claim 8, characterized in that: The grades include S, A, B, C and fail, and the difficulty levels of S, A, B, C are arranged in descending order.

10. The artificial intelligence-based harp performance scoring system and its use method according to claim 9, characterized in that: include Step S1: collecting the hand performance movements and performance sound source data of the performer during the performance through the scoring hardware; Step S2: The simulation unit establishes a coordinate axis and corresponds the hand performance action and the performance sound source data to the same time point; Step S3: The CNN convolution algorithm simulates the performer's hand performance movements and generates frame-by-frame animations; Step S4: CNN convolution algorithm analyzes the sound source data and determines the inharmonious sound source; Step S5: CNN convolution algorithm combines the animation and the discordant sound source to judge the performer's level; Step S6: the intelligent music library integrates the CNN convolution algorithm to select the corresponding whole or fragment music according to the performer's level; Step S7: The scoring software scores the performance.

Images