Intelligent identification and visual playing system for ancient music score

Abstract

The invention relates to the technical field of ancient music score intelligent processing, and discloses an ancient music score intelligent identification and visual playing system, which comprises an image preprocessing module, a stroke enhancement module, a symbol segmentation module, a symbol identification module, a semantic analysis module, a visualization module, an acoustic synthesis module and the like. Through a specially designed image processing and symbol analysis method, the aging problem of the ancient music score can be accurately repaired, adhered symbols are separated, a complex structure is identified, music semantics are inferred in combination with a knowledge graph, and a multi-modal visualization effect and a high-sampling ancient charm tone are generated. The system supports feedback optimization and automatic workflow, significantly improves the efficiency and accuracy of ancient music score recognition, translation and presentation, and provides technical support for ancient music research and propagation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent processing of ancient music scores, in particular to an ancient music score intelligent recognition and visualized performance system. BACKGROUND

[0002] As an important carrier for recording ancient music, ancient music scores have high historical and cultural value. However, due to the long time, limited preservation conditions and the particularity of the notation method, many ancient music scores have problems such as paper aging, ink diffusion, symbol blurring and stroke sticking, which seriously affect the accuracy of their digital recognition and translation. At present, traditional image processing and optical symbol recognition technology is mainly designed for modern printed music scores, and it is often difficult to effectively deal with the complex structural characteristics and diversified symbol variants of ancient music scores when processing them, with low recognition accuracy.

[0003] In the prior art, the processing of ancient music scores relies on manual interpretation and manual translation, which is low in efficiency and easily affected by subjective factors. Although some research has tried to apply general OCR technology to ancient music score recognition, due to the lack of targeted optimization of the characteristics of ancient music score images, it is difficult to effectively handle typical problems such as uneven illumination, symbol sticking, incomplete and blurred symbols. In addition, existing systems are limited to symbol-level recognition in semantic analysis, lacking the ability to infer deep musical semantics such as ancient music rhythm patterns and performance techniques, making it difficult to truly restore the musical connotation of ancient music.

[0004] Therefore, there is an urgent need in the art for an intelligent system that can comprehensively handle ancient music score image restoration, symbol recognition, semantic analysis and multi-modal presentation, to overcome the many shortcomings of existing technology in ancient music score recognition, translation and performance restoration, and to promote the digitalization process of ancient music research and dissemination. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes an ancient music score intelligent recognition and visualized performance system, which can realize accurate processing of complex symbol structures and blurred images by designing special image processing and symbol analysis methods for the particularity of ancient music scores, and provide multi-modal visualized and acoustic synthesis schemes, thereby overcoming the shortcomings of traditional technology in ancient music score recognition, translation and presentation.

[0006] The embodiment of the application provides an ancient music score intelligent recognition and visual performance system, which comprises an image preprocessing module, a stroke enhancement module, an illumination correction module, a symbol segmentation module, a symbol recognition module, a semantic analysis module, a visualization module and an acoustic synthesis module.

[0007] The image preprocessing module comprises a denoising unit, an illumination correction unit and an edge enhancement unit. The denoising unit uses an adversarial generative network to repair the ancient music score image to remove noise caused by paper aging and ink diffusion. The illumination correction unit analyzes the brightness distribution of the image, calculates the non-uniform illumination area and compensates. The edge enhancement unit uses morphological operation to extract the symbol contour and enhance the boundary definition. The stroke enhancement module comprises a sticking separation unit, a stroke thinning unit and a fuzzy enhancement unit. The sticking separation unit separates the sticking symbols by morphological erosion operation. The stroke thinning unit extracts the core strokes of the symbols by using a skeletonization algorithm. The fuzzy enhancement unit uses a local contrast enhancement algorithm to improve the distinguishability of the fuzzy strokes.

[0008] The symbol segmentation module comprises a phrase positioning unit, a symbol segmentation unit and a hierarchical segmentation unit. The phrase positioning unit detects the phrase separation line in the ancient music score by using a sliding window. The symbol segmentation unit further divides the phrase into independent symbol units based on the music theory rules. The hierarchical segmentation unit divides the symbol part layer by layer according to the up-down-left-right combination relationship for complex symbol structures such as reduction score. The symbol recognition module comprises a deep learning model unit, a data labeling unit and a post-processing error correction unit. The deep learning model unit uses an improved YOLO or CRNN architecture to train the symbol recognition model. The data labeling unit stores a large number of artificially labeled ancient music score samples, covering symbol variants of different dynasties and schools. The post-processing error correction unit embeds the ancient music theory rules as constraint conditions to correct errors in the recognition result.

[0009] The semantic analysis module includes a knowledge graph unit that stores mapping relationships between ancient music score symbols and music semantics and related historical background knowledge; an inference unit that infers unmarked rhythm patterns and performance techniques according to the knowledge graph; and a structured output unit that converts the recognition result into a JSON or MusicXML extended format, and labels the pitch, time value, and performance technique of each note. The visualization module includes an original score highlighting unit that highlights the symbols currently being played in real time on the scanned ancient music score image; a modern score comparison unit that synchronously generates a staff or staff view and compares it with the original score; an abstract information visualization unit that dynamically renders melody lines, rhythm beats, and emotional fluctuations through graphics; and a technique demonstration unit that displays the performance techniques of a specific musical instrument in the form of 2D / 3D animation.

[0010] The acoustic synthesis module includes a sound source driving unit that calls a high-sampling ethnic musical instrument sound source library to generate timbre according to the analysis result; a parameterized synthesis unit that designs a physical modeling synthesizer to generate timbre in real time according to the ancient music score analysis parameters; and a technique simulation unit that adjusts timbre details in combination with performance technique information to restore ancient rhythm characteristics. The system further includes a feedback optimization module that records the user's interactive correction of the recognition or translation result, and continuously optimizes the model performance through incremental learning.

[0011] According to some embodiments of the present application, the image preprocessing module further includes an image quality assessment unit for analyzing the resolution and clarity of the input image and prompting the user to rescan or take a low-quality image; and an adaptive parameter adjustment unit for dynamically adjusting the parameters of the denoising and illumination correction algorithm according to the image characteristics. The symbol recognition module further includes a missing symbol completion unit that completes missing symbol parts through morphological dilation operations and template matching algorithms; and a variant symbol classification unit that classifies similar symbols and labels their variant types using clustering algorithms.

[0012] According to some embodiments of the present application, the system further includes a multi-language support module for processing ancient music scores annotated in different languages and translating them into a unified symbol representation; and a historical version comparison module for comparing the differences between versions of the same music score in different dynasties or genres and labeling the changed parts. The visualization module further includes a user interaction unit that allows the user to manually adjust the highlighted area or switch between different visualization modes; and a multi-device synchronization unit that synchronizes the visualization content to multiple display devices through a communication component.

[0013] According to some embodiments of the present application, the acoustic synthesis module further comprises: a timbre library extension unit supporting user uploading of custom sound source files and integrating them into the sound source library; a real-time modulation unit dynamically adjusting the frequency, amplitude and harmonic components of the timbre according to user input parameters. The feedback optimization module further comprises: a model update unit periodically downloading the latest training data and model weights to improve recognition accuracy; a user behavior analysis unit recording user operation habits and optimizing system default settings.

[0014] According to some embodiments of the present application, the system further comprises: an automated workflow module integrating image input, preprocessing, recognition, semantic parsing, visualization generation and synchronous playback into a seamless pipeline; a performance monitoring module monitoring the running state of each module in real time and issuing an alarm when an exception occurs; a resource management module optimizing computing resource allocation to improve system running efficiency.

[0015] The embodiments of the present application at least have the following beneficial effects: The system of the embodiments of the present application solves the recognition problem of ancient music scores caused by paper aging, ink diffusion and stroke blurring by designing a special image preprocessing process. Through the symbol segmentation and recognition model fused with domain knowledge, the parsing ability for complex symbol structure is improved, and high-accuracy symbol recognition is realized by embedding music theory rules. The semantic parsing module combines with the knowledge graph to infer the rhythm type and performance technique not explicitly marked, and converts the fuzzy ancient notation into accurate music instructions. The visualization module provides a multi-modal synchronous presentation method to enhance user experience, and the acoustic synthesis module generates realistic ancient timbre through a high-sampling sound source library and parameterized synthesis technology. The whole system realizes continuous improvement through the feedback optimization mechanism, forms a highly automated and collaborative overall architecture, and significantly improves the efficiency and accuracy of ancient music score recognition, translation and presentation. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 Fig. 1 is a schematic diagram of the overall architecture of the system of the present application; Figure 2 Fig. 2 is a flowchart of the image preprocessing module in the embodiments of the present application; Figure 3 Fig. 3 is a functional schematic diagram of the visualization module in the embodiments of the present application; Figure 4 Fig. 4 is a structural diagram of the acoustic synthesis module in the embodiments of the present application. DETAILED DESCRIPTION

[0017] The technical solutions of the present application will be described clearly and completely below in combination with the drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0018] The present application provides an ancient music score intelligent recognition and visualization performance system, the overall architecture of which is shown in Figure 1 The modules are connected in sequence through data flow to form a complete processing chain. The image preprocessing module receives the original image and performs repair and enhancement processing, and then transmits the processed image to the symbol recognition module for symbol segmentation and recognition. The output result of the symbol recognition module is sent to the semantic analysis module for inferring music semantic information. The result of the semantic analysis module is further transmitted to the visualization module and the acoustic synthesis module to generate multi-modal visualization effects and high-sampling timbre output, respectively. The cooperation relationship between the modules ensures seamless connection from image input to audio output.

[0019] The specific process of the image preprocessing module is shown in Figure 2 It includes a denoising unit, an illumination correction unit and an edge enhancement unit. The denoising unit uses a generative adversarial network to repair the ancient music score image. First, the noise distribution characteristics in the image are analyzed, and then a generative adversarial network model is used to generate the repaired image area to remove the interference caused by paper aging and ink diffusion. The illumination correction unit detects non-uniform illumination areas by calculating the brightness distribution of the image, and adjusts the overall brightness of the image using a local brightness compensation algorithm, so that the image can maintain consistent clarity under different illumination conditions. The edge enhancement unit uses morphological operations to extract symbol contours and enhances symbol boundaries through dilation and erosion operations to improve the recognizability of symbols. The above three units are executed in sequence, and finally output the repaired and enhanced image to provide a high-quality data basis for subsequent symbol recognition.

[0020] The symbol recognition module includes a deep learning model unit, a data annotation unit, and a post-processing correction unit. The deep learning model unit trains the symbol recognition model using an improved YOLO or CRNN architecture. The YOLO architecture is used for fast positioning of symbol locations, while the CRNN architecture is used for accurate recognition of symbol categories. The data annotation unit stores a large number of artificially annotated ancient music score samples, covering symbol variants of different dynasties and genres, providing rich data support for model training. The post-processing correction unit embeds ancient music theory rules as constraint conditions to correct the recognition results. For example, when there are symbol combinations that do not conform to the rules of music theory in the recognition results, the unit will automatically correct the errors according to the context information. The output of the symbol recognition module is a structured symbol sequence, including the position, category, and variant type of each symbol.

[0021] The semantic analysis module includes a knowledge graph unit, an inference unit, and a structured output unit. The knowledge graph unit stores the mapping relationship between ancient music score symbols and music semantics and related historical background knowledge, providing data support for the inference unit. The inference unit infers the rhythm patterns and performance techniques that are not explicitly marked in the ancient music score based on the knowledge graph. For example, when the specific rhythm pattern is not marked in the ancient music score, the inference unit will infer the reasonable rhythm arrangement based on the context information and historical performance habits. The structured output unit converts the recognition results into JSON or MusicXML extended format, labeling the pitch, time value, and performance technique of each note. This structured output form facilitates the processing and display of subsequent modules.

[0022] The visualization module functions as shown in Figure 3 The original score highlighting unit highlights the symbols of the current performance in real time on the scanned ancient music score image, and synchronizes with the audio playback by dynamically updating the position of the highlighted area. The modern score comparison unit synchronously generates a staff or tablature view and displays it with the original score, helping users understand the musical content of the ancient music score. The abstract information visualization unit provides a more intuitive musical performance form by dynamically rendering melody lines, rhythm beats, and emotional fluctuations. The above three units work together through data flow to ensure the diversity and synchronization of the visualization effect.

[0023] The structure of the acoustic synthesis module is shown in Figure 4As shown, the system includes a sound source driving unit, a parametric synthesis unit, and a technique simulation unit. The sound source driving unit generates timbres by calling a high-sampled library of ethnic musical instrument sound sources based on the analysis results, supporting the selection of timbres from various traditional instruments. The parametric synthesis unit designs a physically modeled synthesizer to generate timbres in real time based on the analysis parameters of ancient musical scores. For example, when the analysis results indicate that a certain note requires a special playing technique, the parametric synthesis unit adjusts the frequency, amplitude, and harmonic components of the timbre to reproduce the characteristics of ancient music. The technique simulation unit further adjusts the timbre details by incorporating playing technique information, such as simulating techniques like glissando and vibrato to enhance the realism of the timbre. These three units work together to generate a highly accurate audio output.

[0024] The feedback optimization module records user interactive corrections to the recognition or translation results and continuously optimizes model performance through incremental learning. For example, when a user discovers an error in recognizing a symbol, they can manually correct the error through the interface, and the system will incorporate the corrected data into the training set and retrain the model. In addition, the model update unit regularly downloads the latest training data and model weights to improve recognition accuracy, while the user behavior analysis unit records user operating habits and optimizes the system's default settings.

[0025] The multilingual support module handles ancient musical scores with annotations in different languages ​​and translates them into a unified symbolic representation. For example, when an ancient musical score contains Chinese annotations, this module translates it into an internationally recognized symbolic representation to facilitate cross-cultural communication. The historical version comparison module compares the differences between versions of the same musical score in different dynasties or schools and marks the changed parts, providing a reference for researchers.

[0026] The automated workflow module integrates image input, preprocessing, recognition, semantic parsing, visualization generation, and synchronized playback into a seamless pipeline. The performance monitoring module monitors the operational status of each module in real time and issues alerts when anomalies occur, ensuring stable system operation. The resource management module optimizes computing resource allocation to improve system efficiency, for example, by dynamically adjusting the load distribution between the CPU and GPU for high-efficiency processing.

[0027] In this embodiment, the system workflow is as follows: First, the user acquires the original image of the ancient musical score by scanning or photographing and inputs it into the image preprocessing module. The image preprocessing module processes the image sequentially through a denoising unit, an illumination correction unit, and an edge enhancement unit, outputting a high-quality preprocessed result. Subsequently, the symbol recognition module performs symbol segmentation and recognition on the preprocessed image, generating a structured symbol sequence. The semantic parsing module infers the musical semantic information based on the symbol sequence and converts it into JSON or MusicXML extended format. The visualization module generates multimodal visualization effects based on the semantic parsing results, including highlighting the original score, comparing it with modern scores, and visualizing abstract information. The acoustic synthesis module generates high-sampled audio output based on the semantic parsing results and restores the ancient musical characteristics through a technique simulation unit. Finally, the feedback optimization module records the user's interactive corrections and continuously optimizes the model performance to ensure the long-term availability of the system.

[0028] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A system for intelligent recognition and visual performance of ancient musical scores, characterized in that, include: The image preprocessing module is used to receive the original image of the ancient musical score and perform noise reduction and illumination correction on it; The symbol recognition module is used for symbol segmentation and recognition in the preprocessed image; The semantic parsing module is used to infer semantic information in music based on symbol recognition results; The visualization module is used to generate multimodal visualizations synchronized with audio playback. The acoustic synthesis module is used to generate high-sampled timbre output based on the semantic parsing results.

2. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, The image preprocessing module includes: The noise reduction unit uses a generative adversarial network to repair the ancient musical score image to remove noise; The illumination correction unit calculates and compensates for non-uniform illumination areas by analyzing the image brightness distribution. Edge enhancement units utilize morphological operations to extract symbol outlines and enhance boundary clarity.

3. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, The symbol recognition module includes: The deep learning model unit uses an improved YOLO or CRNN architecture to train a symbol recognition model; The data annotation unit stores manually annotated samples of ancient musical scores; The post-processing error correction unit embeds ancient music score rules as constraints to correct the recognition results.

4. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, The semantic parsing module includes: Knowledge graph units store the mapping relationship between ancient musical notation symbols and musical semantics; The reasoning unit infers unmarked rhythmic patterns and performance techniques based on a knowledge graph. The structured output unit converts the recognition results into JSON or MusicXML extended formats.

5. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, The visualization module includes: The original score highlighting unit highlights the currently played symbols in real time on the scanned ancient music score image; The modern music score comparison unit simultaneously generates staff or numbered musical notation views and displays them in comparison with the original score; Abstract information visualization units use dynamic graphics to render melodic lines, rhythmic patterns, and emotional fluctuations.

6. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, The acoustic synthesis module includes: The sound source driver unit calls a high-sampled ethnic musical instrument sound source library to generate timbres; Parametric synthesis unit: Design a physical model synthesizer to generate timbres based on analytical parameters; The technique simulation unit adjusts the tonal details by incorporating performance technique information.

7. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, It also includes a feedback optimization module that records the user's interactive corrections to the recognition or translation results and continuously optimizes the model performance through incremental learning.

8. The intelligent recognition and visual performance system for ancient musical scores according to claim 1, characterized in that, It also includes an automated workflow module that integrates image input, preprocessing, recognition, semantic parsing, visualization generation, and synchronized playback into a seamless pipeline.

Images