Full-tone nonlinear mapping for digital musical notation modulation and its application implementation

Abstract

This invention discloses a modulation rule for digital musical notation and its app implementation. The modulation rule is based on a bidirectional nonlinear mapping library of the twelve-tone equal temperament. The multifunctional digital musical notation modulation app, utilizing OpenCV and CNN, is suitable for both PC and mobile devices, belonging to the field of computer music software technology. The software has a three-layer architecture: interface layer, business logic layer, and data and system service layer, integrating seven core modules: image or text import preprocessing, recognition, parsing, manual correction, modulation, export rendering, and playback verification. Modulation uses C major as an intermediary, achieving bidirectional nonlinear conversion from any key to any key through 22 pre-set mapping tables. It recognizes symbols classified by OpenCV preprocessed images and ONNX Runtime-loaded CNN models. It supports multi-format export and Markov analysis, and provides two verification methods: audio playback and key signature rule verification. This invention solves the problems of existing tools having single input / output formats, insufficient processing efficiency, and lack of verification, making it suitable for music teaching, academic research, and other scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer music software and data processing technology, specifically to a full-tone nonlinear mapping library based on the twelve-tone equal temperament and a multifunctional simplified notation modulation software using OpenCV and CNN. It is applicable to the entire process of simplified notation import, recognition, modulation conversion, manual correction, export, and audio playback in scenarios such as music teaching, academic research, music score typesetting, and music data alignment. Background Technology

[0002] Existing simplified musical notation modulation tools and methods have the following core flaws, making it difficult to meet users' actual needs:

[0003] (1) Single input and output format support: It only supports a few image formats (such as JPEG), is not compatible with new formats such as WebP, HEIF / HEIC, and has no multi-format export function (such as TXT text, Markov sequence);

[0004] (2) Inefficient: The processing time for a single 100-bar simplified musical score exceeds 10 seconds, which cannot meet the requirements for efficient processing;

[0005] (3) Lack of verification functions: There is no audio playback verification and no key change rule verification, which cannot meet the needs of users in multiple scenarios.

[0006] While traditional manual modulation methods can guarantee a certain level of accuracy, they are cumbersome and time-consuming, making them unsuitable for transposing complex musical scores. Therefore, there is an urgent need for a highly efficient and multifunctional modulation technology to address the shortcomings of existing techniques. Summary of the Invention

[0007] This invention aims to solve the three core problems of existing simplified musical notation modulation tools: limited support for import and export formats, insufficient efficiency, and lack of verification functions. It provides a simplified musical notation modulation mapping rule and software that features "full format adaptation, high-efficiency processing, and multi-functional verification".

[0008] To address the aforementioned problems, the technical solution of this invention comprises four parts: modulation mapping construction, software architecture design, functional module implementation, and core algorithm flow, as detailed below:

[0009] (1) Modulation mapping construction: The arbitrary modulation technology of the whole mode in this invention is based on the twelve equal temperament as the underlying theory. It realizes bidirectional communication between all modes through "intermediate benchmark normalization and table lookup for precise matching". The specific principle and implementation path are as follows:

[0010] The twelve-tone equal temperament system is constructed by following the universal standards of modern music tuning. An octave is precisely divided into 12 equal semitones in a geometric progression, forming a complete semitone sequence containing C, C#, D, D#, E, F, F#, G, G#, A, A#, and B. The ratio of the vibration frequencies of each adjacent semitone is a fixed constant. This ensures that the pitch relationships of all modes are consistent, providing a rigorous mathematical basis for cross-mode conversion.

[0011] Modulation path optimization design: To address the complexity and error-prone nature of direct modulation between arbitrary keys, an innovative "two-step modulation" strategy is adopted—first, any original key is uniformly normalized to an intermediate reference key—C major, and then mapped from C major to the target key. This path significantly simplifies the logical complexity of multi-key conversions by establishing a unified modulation reference, while improving modulation accuracy and efficiency.

[0012] Based on the "whole-whole-half-whole-whole-whole-half" interval structure of the natural major scale, the semitone difference (or offset) between the other 11 keys (excluding C major) and C major is calculated. This allows for the creation of a solfège-pitch conversion table for C major in each of the seven scales, as shown in Table 1 (D major to C major conversion table). Similarly, the semitone difference (or offset) between C major and the other 11 keys (excluding C major) is calculated, creating a solfège-pitch conversion table for C major to various keys in the seven scales, as shown in Table 2 (C major to D# major conversion table). For normalized conversion from the original key to C major, the reference pitch of the original key is first analyzed, and then the pitch after conversion is found by consulting the corresponding nonlinear mapping library. During the conversion, only the absolute position of the pitch is adjusted. For mapping conversion from C major to the target key, the reference pitch of C major is first analyzed, and then the pitch after conversion is found by consulting the corresponding table. During the conversion, only the absolute position of the pitch is adjusted. This allows for efficient and accurate conversion between any key and any major using C major as a medium.

[0013] Construction of a Nonlinear Mapping Library: Based on the aforementioned modulation principle, this invention constructs a nonlinear mapping library containing bidirectional mapping relationships between "various keys to C major" and "C major to various keys," totaling 22 mapping tables. Eleven non-C major keys correspond to one forward table and one reverse table, respectively, totaling 1848 key-value entries. This covers the commonly used range of numbered musical notation, with an octave offset ranging from −2 to +4. The mapping tables store the correspondence between solfège syllables and pitches of the original and target keys at the same absolute pitch in key-value pairs. This correspondence is embedded as compile-time constants in the engineering implementation, ensuring query speed and cross-platform portability.

[0014] Table 1: C to D key mapping table

[0015] 1= C 0 [-2]1 [-2]#1 [-2]2 [-2]#2 [-2]3 [-2]4 [-2]#4 [-2]5 [-2]#5 [-2]6 [-2]#6 [-2]7 1=#D 0 [-3]6 [-3]#6 [-3]7 [-2]1 [-2]#1 [-2]2 [-2]#2 [-2]3 [-2]4 [-2]#4 [-2]5 [-2]#5 1= C 1 [-1]1 [-1]#1 [-1]2 [-1]#2 [-1]3 [-1]4 [-1]#4 [-1]5 [-1]#5 [-1]6 [-1]#6 [-1]7 1=#D 1 [-2]6 [-2]#6 [-2]7 [-1]1 [-1]#1 [-1]2 [-1]#2 [-1]3 [-1]4 [-1]#4 [-1]5 [-1]#5 1= C 2 [0]1 [0]#1 [0]2 [0]#2 [0]3 [0]4 [0]#4 [0]5 [0]#5 [0]6 [0]#6 [0]7 1=#D 2 [-1]6 [-1]#6 [-1]7 [0]1 [0]#1 [0]2 [0]#2 [0]3 [0]4 [0]#4 [0]5 [0]#5 1= C 3 [1]1 [1]#1 [1]2 [1]#2 [1]3 [1]4 [1]#4 [1]5 [1]#5 [1]6 [1]#6 [1]7 1=#D 3 [0]6 [0]#6 [0]7 [1]1 [1]#1 [1]2 [1]#2 [1]3 [1]4 [1]#4 [1]5 [1]#5 1= C 4 [2]1 [2]#1 [2]2 [2]#2 [2]3 [2]4 [2]#4 [2]5 [2]#5 [2]6 [2]#6 [2]7 1=#D 4 [1]6 [1]#6 [1]7 [2]1 [2]#1 [2]2 [2]#2 [2]3 [2]4 [2]#4 [2]5 [2]#5 1= C 5 [3]1 [3]#1 [3]2 [3]#2 [3]3 [3]4 [3]#4 [3]5 [3]#5 [3]6 [3]#6 [3]7 1=#D 5 [2]6 [2]#6 [2]7 [3]1 [3]#1 [3]2 [3]#2 [3]3 [3]4 [3]#4 [3]5 [3]#5 1= C 6 [4]1 [4]#1 [4]2 [4]#2 [4]3 [4]4 [4]#4 [4]5 [4]#5 [4]6 [4]#6 [4]7 1=#D 6 [3]6 [3]#6 [3]7 [4]1 [4]#1 [4]2 [4]#2 [4]3 [4]4 [4]#4 [4]5 [4]#5

[0016] Table 2: D-to-C major conversion mapping table

[0017] 1= D 0 [-3]#6 [-3]7 [-2]1 [-2]#1 [-2]2 [-2]#2 [-2]3 [-2]4 [-2]#4 [-2]5 [-2]#5 [-2]6 1= C 0 [-2]1 [-2]#1 [-2]2 [-2]#2 [-2]3 [-2]4 [-2]#4 [-2]5 [-2]#5 [-2]6 [-2]#6 [-2]7 1= D 1 [-2]#6 [-2]7 [-1]1 [-1]#1 [-1]2 [-1]#2 [-1]3 [-1]4 [-1]#4 [-1]5 [-1]#5 [-1]6 1= C 1 [-1]1 [-1]#1 [-1]2 [-1]#2 [-1]3 [-1]4 [-1]#4 [-1]5 [-1]#5 [-1]6 [-1]#6 [-1]7 1= D 2 [-1]#6 [-1]7 [0]1 [0]#1 [0]2 [0]#2 [0]3 [0]4 [0]#4 [0]5 [0]#5 [0]6 1= C 2 [0]1 [0]#1 [0]2 [0]#2 [0]3 [0]4 [0]#4 [0]5 [0]#5 [0]6 [0]#6 [0]7 1= D 3 [0]#6 [0]7 [1]1 [1]#1 [1]2 [1]#2 [1]3 [1]4 [1]#4 [1]5 [1]#5 [1]6 1= C 3 [1]1 [1]#1 [1]2 [1]#2 [1]3 [1]4 [1]#4 [1]5 [1]#5 [1]6 [1]#6 [1]7 1= D 4 [1]#6 [1]7 [2]1 [2]#1 [2]2 [2]#2 [2]3 [2]4 [2]#4 [2]5 [2]#5 [2]6 1= C 4 [2]1 [2]#1 [2]2 [2]#2 [2]3 [2]4 [2]#4 [2]5 [2]#5 [2]6 [2]#6 [2]7 1= D 5 [2]#6 [2]7 [3]1 [3]#1 [3]2 [3]#2 [3]3 [3]4 [3]#4 [3]5 [3]#5 [3]6 1= C 5 [3]1 [3]#1 [3]2 [3]#2 [3]3 [3]4 [3]#4 [3]5 [3]#5 [3]6 [3]#6 [3]7 1= D 6 [3]#6 [3]7 [4]1 [4]#1 [4]2 [4]#2 [4]3 [4]4 [4]#4 [4]5 [4]#5 [4]6 1= C 6 [4]1 [4]#1 [4]2 [4]#2 [4]3 [4]4 [4]#4 [4]5 [4]#5 [4]6 [4]#6 [4]7

[0018] (2) Software architecture design: A three-layer architecture of "interface layer + business logic layer + data and system service layer" is adopted. The business logic layer integrates the core algorithm sublayer and the engine sublayer, as follows:

[0019] Interface layer: Implemented based on Jetpack Compose + Material 3, including image selection interface, confirmation interface, operation interface (including manual correction, mode selection, playback control, and export functions), supporting zoom, pan, and point-and-click interaction;

[0020] Business logic layer: Unified scheduling by MainViewModel, integrating the core algorithm sublayer and engine sublayer, responsible for image loading, recognition triggering, pitch conversion calculation, playback status management, export and distribution, and error handling; the business logic layer embeds:

[0021] Core algorithm sublayer: includes the definition of the pitch model of simplified musical notation, the full-tone mapping library based on the twelve equal temperament, pitch encoding and decoding algorithms, pitch interval analysis, and text simplified musical notation parsing and formatting;

[0022] Engine sub-layers include the OpenCV image preprocessing engine, CNN+ONNX recognition engine, export rendering engine, and audio synthesis and playback engine.

[0023] Data and System Services Layer: Responsible for loading Assets resources (models, SoundFont files), reading and writing files (MediaStore), and playing local audio (AudioTrack).

[0024] (3) Functional module implementation: The software includes 7 core functional modules and 4 auxiliary functional modules, such as Figure 2 The detailed implementation of each module is as follows:

[0025] Core functional modules:

[0026] Image and text import and preprocessing: Using the "image / *" MIME type, it supports users importing all Android natively decoded image formats, including PNG, JPEG, WebP, BMP, GIF, HEIF / HEIC (Android 8.0+), AVIF (Android 12+), etc. It automatically performs preprocessing operations such as image normalization, tilt correction, and font size adaptation to ensure subsequent recognition accuracy. Structured data text import must follow the structure "[a]b", where a represents an octave and b represents a pitch level.

[0027] Simplified musical notation recognition: Extracting elements from the preprocessed simplified musical notation image to accurately identify key elements such as pitch, octave, and sharps / flats.

[0028] Simplified musical notation analysis: A structured storage system is established for the identified pitches, octaves, sharps, and flats, recording information such as the bounding box coordinates and attribute characteristics of each element, providing data support for modulation processing.

[0029] Key modulation: Based on the twelve-tone equal temperament theory, it enables bidirectional modulation between 12 common modes, including C / D / E / F / G / A / B (including sharps and flats), to meet the needs of different tonality adaptation.

[0030] Manual correction: The recognition result is marked at the original pitch position of the simplified musical notation (displayed in blue text), allowing users to manually verify the recognition content and easily modify the pitch with minor recognition deviations, ensuring the accuracy of key transposition.

[0031] Export and Rendering: Visualizes and renders the manually corrected musical notation data, exporting the transposed data as a JPG image file, fully preserving the original layout and visual structure. It also allows exporting the pitch sequence text (txt) before and after transposition.

[0032] Playback: For manually corrected musical notation data, the system supports audio playback of the extracted pitch information. Users can further verify the accuracy of the pitch recognition results and the reliability of the core data before modulation through auditory verification.

[0033] Auxiliary function modules:

[0034] Configuration management module: Preset default D-major to C-major conversion configuration, supports user-defined original and target keys;

[0035] Error handling module: Captures issues such as call reversal errors, file export failures, and crashes, and automatically records error logs;

[0036] Markov analysis function: Supports first-order Markov analysis of simplified musical notation images and outputs a sequence of pitch results;

[0037] Extracting pitch text from sheet music images: Convert manually corrected sheet music images to their original key, extract the pitch text sequence, and use this sequence as text input to convert to any key multiple times, eliminating the need for manual correction of image input.

[0038] (4) Core algorithm flow: The overall flow is image import → preprocessing → recognition → parsing → manual correction → pitch conversion calculation → export / playback; detailed flow of the pitch conversion algorithm ( Figure 3 )as follows:

[0039] The system parses and extracts structured note data. Using the NoteSymbol output from the preceding module as its core, the system reads the raw information recorded within, such as pitch levels, accidentals, octave shifts, and layout coordinates, through parsing logic. During processing, the system automatically identifies rests or unresolved unknown symbols; this non-pitch data is not entered into the mapping library but is directly passed through to maintain its original state.

[0040] The system searches for the mapping representation of the original key note in the key of C. Based on the note's scale degree, sharps, flats, and octave information, the system directly looks up the corresponding pitch position of the note in C in a preset original key to C key mapping table. If the original note contains flats, the system first normalizes it to enharmonic sharps or enharmonic natural tones before performing the search. The significance of this step is to use C key as an intermediary to eliminate the notation differences between different keys, establishing a unified reference standard for subsequent target key conversions.

[0041] Perform a forward mapping from C major to the target key. In the preset "C major → target key" mapping table, the system directly looks up the corresponding note representation of the target key using the C major notation obtained in the previous step as the key. Since the mapping table has been embedded as a key-value structure during compilation, this step is an O(1) lookup, requiring no numerical calculations; the mapping table entries have been manually calibrated during the construction phase according to the twelve-tone equal temperament and numbered musical notation rules, ensuring that the generated result conforms to music theory logic while also taking into account writing habits.

[0042] The system performs reverse decoding of the target key. It directly retrieves the note representation of the target key from a preset mapping table, prioritizing natural tones. If the original note is in flat notation and the lookup result is in sharp notation, the system rewrites the result in flat notation according to the preset enharmonic correspondence table (#1→b2, #2→b3, #4→b5, #5→b6, #6→b7) to preserve the user's original writing habits. For #3 and #7, since there are no flat enharmonics in standard notation, the sharp is output as is. This mechanism ensures accurate pitch while greatly improving the readability of the score after modulation.

[0043] The system encapsulates and outputs the transposition results. Based on the decoded data, the system generates a new `JianpuPitch` object and precisely writes it back to the corresponding `NoteSymbol` field. This process strictly limits updates to only pitch-related information, without affecting existing non-pitch attributes such as the layout bounding box coordinates and confidence level. The processed structured data forms a complete closed loop, allowing direct integration with subsequent rendering, export, and audio playback modules to achieve stable transposition across 12 modes.

[0044] In terms of engineering implementation, this solution uses Kotlin to write the core business logic, and uses TransposeUseCase to connect the entire process of data organization and bidirectional mapping. Through the collaboration of structured objects such as KeySignature and NoteSymbol, the algorithm not only ensures accurate alignment at the musical level, but also achieves high usability in the form of simplified musical notation.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The system boasts strong import and export format compatibility. On Android, it supports mainstream formats such as PNG / JPEG / WebP / BMP (API 28+) and HEIF / AVIF (API 31+) via the system image decoder; on desktop (Windows / macOS / Linux), it supports PNG / JPG / JPEG / BMP formats. All formats support structured text data import. Processing efficiency meets real-time response requirements, with a processing time of ≤1 second for single 100-bar, 12MP musical scores, satisfying high-efficiency processing needs. It provides two verification methods: audio playback verification and key signature rule verification, covering both core dimensions to ensure the auditory consistency and theoretical correctness of the transposition results. Even non-professional users can independently complete the transposition and export process after a brief explanation. The interface is simple, the operation process is standardized, and non-professional users can master the basic operations within 5 minutes. Attached Figure Description

[0047] The accompanying drawings are provided to further illustrate the invention. There are five drawings in total, and the reference numerals and descriptions are as follows:

[0048] Figure 1 This is a system architecture diagram of the present invention;

[0049] Figure 2 This is a functional module division diagram of the present invention;

[0050] Figure 3 This is a flowchart of the modulation algorithm of the present invention;

[0051] Figure 4 This is a flowchart of the simplified spectrum analysis process of the present invention;

[0052] Figure 5 This is a schematic diagram of the core operation interface of the present invention. Detailed Implementation

[0053] This software adopts the Kotlin Multiplatform architecture, supporting both mobile and desktop platforms. Both platforms share the core algorithm and data model, and adapt to the system API through the platform abstraction layer.

[0054] The Android mobile application runtime environment is Android 7.0+ (API 24+), with 4GB of RAM recommended. The development tools are Android Studio 2025.3.1 (Narwhal Feature Drop), Gradle 9.1.0, Android GradlePlugin 9.0.0, and JDK 11. The development languages ​​are Kotlin 2.0.21 and C++ (for audio synthesis, TinySoundFont / Bungee). The dependencies are OpenCV Android 4.5.3.0, ONNX Runtime Android 1.17.0, and JetpackCompose BOM 2024.09.00. The compilation configuration is compileSdk 36, minSdk 24, and targetSdk 36.

[0055] The desktop runtime environment is Windows 10+, macOS 11+, and Linux (glibc ≥ 2.28). The development tools are IntelliJ IDEA 2025.3, Gradle 8.10.2, and JDK 17. The development language is Kotlin 2.1.10. The dependent libraries are OpenCV Java 4.9.0, ONNX Runtime 1.17.0, and Compose Multiplatform 1.7.3. The distribution formats are Windows MSI, macOS DMG, and Linux DEB.

[0056] This system is implemented using a pure offline Android three-layer architecture, which is divided into a UI layer, a business logic layer, and a data and system service layer, such as... Figure 1 As shown.

[0057] The user interface layer, built on Jetpack Compose, is responsible for image import, result display, original / target key setting, manual correction, export feedback, and playback control. The business logic layer handles task arrangement, simplified musical notation structure parsing, key conversion calculation, export scheduling, and playback queue organization. The data and system service layer is responsible for underlying capabilities such as image decoding, OpenCV image processing, ONNXRuntime inference, MediaStore file storage, AudioTrack audio output, and JNI native timbre synthesis. This three-layer structure decouples user interface interaction, business rules, and underlying implementation, facilitating future expansion and maintenance.

[0058] In terms of development tools and implementation methods, this system uses Android Studio as the integrated development environment, Kotlin as the main development language, and C++ to implement some low-level audio synthesis logic. CMake and JNI are used to connect the Kotlin layer and the native layer. For image recognition, OpenCV for Android is used to perform operations such as grayscale conversion, denoising, binarization, line segmentation, and contour analysis. For symbol classification, ONNX Runtime Android loads a lightweight CNN model to complete local inference. Interface state management uses the ViewModel and StateFlow mechanism. Image and text file exports are uniformly written to the system album and download directory through MediaStore, meeting Android's partitioned storage specifications.

[0059] The overall system processing flow is as follows: When a user imports a musical score image into the interface layer, the interface layer first decodes the external URI into a Bitmap for preview. After user confirmation, the business logic layer calls the recognition engine to convert the Bitmap into a Mat object, and then performs preprocessing, candidate box extraction, symbol classification, and rule correction in sequence to obtain the initial recognition result. Subsequently, the business logic layer further parses the recognition result into a structured musical score object, forming a unified NoteSymbol dataset. If the user finds a recognition error, they can directly select a note in the interface layer for correction, and the correction result is written back to the unified data object in real time. Finally, the business logic layer completes the transposition calculation based on the unified data object and sends the result to the export rendering module or playback module. The interface layer and business logic layer use an "event triggering—state update—interface redrawing" working mechanism. When the user performs operations such as selecting an image, confirming recognition, modifying the key signature, editing notes, exporting, or playing in the interface layer, the corresponding use case function in the business logic layer is triggered. After the business logic layer completes processing, it sends the new recognition result, transposition result, or playback status back to the interface layer, which automatically refreshes the displayed content based on the latest status. This unidirectional data flow approach avoids direct coupling between multiple modules, improving system stability. For text imports, structured text is input, the business logic layer is directly invoked, the built mapping library performs the data object conversion calculation, and the result is sent to the export rendering module.

[0060] The business logic layer is based on a unified structured data model. Each note output during the recognition phase is encapsulated as a NoteSymbol object, recording five basic fields: id, symbol type, pitch, bounding box coordinates, and confidence level. This object is reused repeatedly during manual correction, transposition, export, and playback stages, eliminating the need for multiple intermediate format conversions. This reduces data loss between modules and ensures consistency between the recognition result, correction result, transposition result, and export result.

[0061] In the data and system service layer, OpenCV primarily handles image-level processing tasks, ONNX Runtime mainly handles local symbol classification and inference tasks, MediaStore is responsible for saving exported images and text results, AudioTrack is responsible for the final audio output, and the SoundFont native synthesizer is responsible for generating more natural instrument timbres. Since recognition, modulation, rendering, and playback are all completed locally on the terminal device, the entire system does not rely on any network connection, offering advantages such as fast response speed, good privacy, and simple deployment, making it suitable for offline musical notation modulation scenarios on mobile devices.

[0062] The core functional modules of this system consist of 7 main parts.

[0063] For the image and text import and preprocessing module, taking the Android client as an example, users can manually import simplified musical notation images and structured text data in Android's native decoding formats such as PNG, JPEG, WebP, BMP, GIF, HEIF / HEIC (Android 8.0+), and AVIF (Android 12+). The imported simplified musical notation images are preprocessed using OpenCV to perform grayscale conversion, noise reduction, binarization, geometric correction, cropping, line segmentation, and candidate box extraction, normalizing the original image to a standardized input. The Otsu binarization algorithm is used to automatically determine the threshold, the horizontal projection method is used to perform line segmentation, and contour detection is used to extract note candidate boxes.

[0064] For the simplified musical notation recognition module (such as...) Figure 4 As shown, the standardized musical notation images are used to identify notes, sharps, flats, and octaves. The identified content is then segmented into candidate Regions of Interest (ROIs). A CNN model is used to automatically learn the local texture, shape, and orientation features of notes, sharps, flats, etc., and then corrects these features based on positional relationship verification, baseline constraints, and ornament filtering rules. Finally, a lightweight CNN model (CNN_middle.onnx) is loaded onto the ONNX Runtime to classify symbols based on the highest possible probability. The recognition accuracy is ≥91% on three musical notation images of different complexities.

[0065] For the simplified musical notation content parsing module, the recognition results are encapsulated into NoteSymbol objects, recording the ID, symbol type, pitch, bbox coordinates, and confidence level, and then converted into simplified musical notation structured data storage; the musical notation order is established through row sorting and coordinate clustering to ensure consistency in transposition, playback, and export.

[0066] For the modulation algorithm module, firstly, based on the twelve-tone equal temperament, an octave is divided into 12 equidistant semitones: C, C#, D, D#, E, F, F#, G, G#, A, A#, and B. An indirect method is used to transpose any key to any key, i.e., transpose any key to C major, and then transpose C major to any key. Based on the twelve-tone equal temperament and the major scale's interval structure of "whole-whole-half-whole-whole-whole-half," the solfège-pitch correspondence between non-C major and C major at each pitch level is calculated, establishing a bidirectional nonlinear mapping library of 22 mapping tables for "any key → C major" and "C major → any key." For any key change, first look up the C major notation of the original note in the "Original Key → C Major" table, and then look up the target key notation in the "C Major → Target Key" table. If the original note is a flat notation, first normalize it to an enharmonic sharp or enharmonic natural tone before looking it up. If the lookup result is a sharp, rewrite it as a flat according to the preset enharmonic correspondence (#1 → b2, etc.).

[0067] For manual correction modules (such as) Figure 5 As shown in the image, after recognition, a recognition box and blue recognition result text will be superimposed on the original image. It supports zooming and panning to locate notes. When errors are found manually, an editing dialog box will pop up for modification. The dialog box supports modifying pitch, sharps, octaves, deleting incorrect notes, or adding missed notes. The modification results are written back to the NoteSymbol list in real time.

[0068] For the export rendering module, the corrected musical notation image is transposed, supporting the export of transposed musical notation images (JPG), pitch sequences before and after transposition (txt), and Markov sequences (txt). The rendering of the transposed musical notation image is based on partial erasing and redrawing of the original image, preserving the original layout structure, with "Original Key → Target Key" marked at the top, and consistent font size and layout. Text export will display the storage path: images are saved to Pictures / JianpuTrans, and text is saved to Downloads / JianpuTrans. The default export path on the desktop is the JianpuTrans subdirectory under the user's Documents directory (both images and text are saved here). Users can customize the export directory in application preferences; the path is persistently saved via the system Preferences API.

[0069] For the playback module, the stored structured data is converted into MIDI values ​​and then combined with time signature / tempo / duration to generate a playback queue. The playback speed is adjustable through 8 preset levels, covering 40–200 BPM (extremely slow, slow, medium-slow, medium speed, medium-fast, fast, very fast, extremely fast), with actual playback BPMs of 50 / 68 / 86 / 102 / 114 / 130 / 154 / 180 respectively. The synthesized audio is output via AudioTrack.

[0070] The following is a detailed implementation of the aforementioned core functions.

[0071] For image preprocessing, the approach is as follows: first, the input musical notation Bitmap is validated for legality; then, it is converted into an OpenCV Mat object. Next, grayscale processing is performed to remove color information; 3×3 Gaussian blur is used for image denoising; Otsu's automatic thresholding algorithm is used for inverse binarization to separate symbols from the background; then, Hough line detection is used to calculate the image tilt angle and perform rotation correction; contour detection is used to locate the maximum effective region and expand the safe boundary outwards to complete background cropping; finally, the start and end positions of the musical notation lines are identified based on the horizontal projection value; after filtering out invalid small lines, the entire musical notation is divided into single-line Mat sequences, providing standardized image input for subsequent recognition.

[0072] Key code snippets:

[0073] fun preprocessImage(bitmap: Bitmap): List <mat>{

[0074] val mat = bitmap.toMat() / / Convert to an OpenCV Mat object

[0075] / / 1. Grayscale conversion: val grayMat = Mat()

[0076] Imgproc.cvtColor(mat, grayMat, Imgproc.COLOR_BGR2GRAY)

[0077] / / 2. Denoising (Gaussian blur)

[0078] val blurMat = Mat()

[0079] Imgproc.GaussianBlur(grayMat, blurMat, Size(3.0, 3.0), 0.0)

[0080] / / 3. Binarization (Otsu automatic thresholding)

[0081] val binaryMat = Mat()

[0082] Imgproc.threshold(blurMat, binaryMat, 0.0, 255.0, Imgproc.THRESH_BINARY_INV + Imgproc.THRESH_OTSU)

[0083] / / 4. Geometric correction (tilt correction)

[0084] val correctedMat = correctSkew(binaryMat)

[0085] / / 5. Crop (remove excess background)

[0086] val croppedMat = cropBackground(correctedMat)

[0087] / / 6. Line splitting

[0088] val lineMats = splitLines(croppedMat)

[0089] return lineMats

[0090] }

[0091] For the implementation of the simplified musical notation recognition module, the approach is as follows: First, perform contour detection and size filtering on the single-line simplified musical notation image to extract candidate regions of symbols that meet the specifications. Normalize each candidate region image to a size of 28×28 and complete numerical standardization. Then, input the processed image into a lightweight CNN model to complete symbol classification prediction. Subsequently, combine the simplified musical notation layout rules to correct the prediction results, constraining that sharps and flats must be attached to the left of the note, octave points must be located directly above or below the note, and notes must retain only the digits 1-7. Isolated invalid symbols are filtered out. Finally, the corrected symbol type, position coordinates, confidence level, and other information are encapsulated into a DetectedSymbol object for output.

[0092] Key code snippets:

[0093] fun recognizeLine(lineMat: Mat): List <detectedsymbol>{

[0094] / / 1. Detect all candidate regions for symbols in a single row of images (filter noise and locate possible symbol positions).

[0095] val candidates = findCandidateRegions(lineMat)

[0096] / / Initialize the recognition result list to store the final corrected symbol data.

[0097] val result = mutableListOf <detectedsymbol>()

[0098] / / 2. Traverse each candidate region and complete the identification and correction one by one.

[0099] for (roi in candidates) {

[0100] / / Normalize the candidate region image to the 28×28 size required by the CNN model.

[0101] val normalized = normalizeTo28x28(roi.image)

[0102] / / Call the CNN classifier to predict the symbol type of the current region

[0103] val label = cnnClassifier.predict(normalized)

[0104] / / Apply simplified musical notation layout rules to correct the original CNN prediction results (to resolve context-dependent symbol misidentification).

[0105] val fixedLabel = applyLayoutRules(

[0106] rawLabel = label, / / CNN's original predicted label

[0107] bbox = roi.bbox, / / The rectangle containing the current symbol's position.

[0108] neighbors = candidates / / All candidate regions in the same row (used for context validation)

[0109] / / Encapsulate the corrected recognition results and add them to the list

[0110] result += DetectedSymbol(

[0111] type = fixedLabel, / / Modified symbol type

[0112] bbox = roi.bbox, / / The position of the symbol in the image

[0113] confidence = roi.confidence / / Recognition confidence level (0-1) )

[0115] }

[0116] / / 3. Return the final single-line symbol recognition result

[0117] return result

[0118] }

[0119] For the simplified musical notation parsing module, the approach is to first sort the identified symbols from top to bottom and from left to right according to the row number and horizontal coordinate, retain only the valid numerical notes and skip the auxiliary symbols, then match the sharps and flats within the left range of the same row for each note, detect the octave points above and below the note to determine the octave offset, and finally generate a unique identifier for each note. The pitch, position, confidence level and other information are integrated to construct a structured NoteSymbol object, forming standard note data that can be directly used for key changes and playback.

[0120] Key code snippets:

[0121] fun parseSymbols(symbols: List <detectedsymbol>): List <notesymbol>{

[0122] / / 1. Symbol sorting: Sort by line number (from top to bottom) and horizontal axis (from left to right), conforming to the reading order of simplified musical notation.

[0123] val sorted = symbols.sortedWith(compareBy({ it.rowIndex}, {it.bbox.left}))

[0124] / / Initialize the structured note list and store the final parsing results

[0125] val notes = mutableListOf <notesymbol>()

[0126] / / 2. Traverse the sorted symbols, processing only numbered notes (filtering out auxiliary symbols such as sharps, flats, and octaves).

[0127] for (symbol in sorted) {

[0128] / / Skip non-digit symbols (only parse the note symbols 1-7)

[0129] if (!symbol.isDigit()) continue

[0130] / / Analyze the sharps and flats (# / b / none) associated with the left side of the note.

[0131] val accidental = findLeftAccidental(symbol, sorted)

[0132] / / Detect the octave points above and below the note, and calculate the octave offset (+1 = higher octave, -1 = lower octave, 0 = original octave).

[0133] val octaveShift = detectOctaveDots(symbol, sorted)

[0134] / / 3. Encapsulate the structured note object and add it to the results list

[0135] notes += NoteSymbol( id = generateId(), / / Generate a unique identifier ID for the note)

[0136] kind = SymbolKind.NOTE, / / Symbol type is marked as musical note

[0137] pitch = JianpuPitch( / / Construct a simplified musical pitch object (note value + octave + sharp / sharp)

[0138] symbol.degree, / / Core pitch levels (1-7)

[0139] octaveShift, / / Octave offset

[0140] accidental / / Accidental / Presentation / Descending attribute

[0141] ),

[0142] bbox = symbol.bbox, / / The rectangular box containing the position of the musical note in the image.

[0143] confidence = symbol.confidence, / / Recognize confidence level (0-1)

[0144] }

[0145] / / 4. Return the parsed list of structured musical notes

[0146] return notes

[0147] }

[0148] For the implementation of the modulation algorithm module, the approach is to first preprocess and normalize the input notes based on the twelve-tone equal temperament, uniformly normalizing the flat notation of the original notes to the corresponding sharp or natural tone notation, thus establishing a standard retrieval key. Subsequently, the program enters a two-level mapping process of "original key—C major—target key" to achieve precise interval conversion. To maintain the consistency of the notation, the algorithm retrieves the sharps and flats of the original notes and restores the conversion result to the initial notation preference through an enharmonic write-back mechanism. Finally, while maintaining a high degree of consistency between the note spatial coordinates and non-pitch attributes such as confidence, the system completes the reconstruction of pitch information and returns the object. Finally, keeping the note position, confidence, and other attributes unchanged, only the pitch information is replaced to obtain the modulated note object, which is then returned.

[0149] Part of the main code (TranspositionLookup.kt):

[0150] fun transpose(pitch: JianpuPitch, fromKey: KeySignature, toKey:KeySignature): JianpuPitch {

[0151] if (fromKey == toKey) return pitch

[0152] / / Step 1: Normalize the falling signs (b2→#1, b3→#2, b5→#4, b6→#5, b7→#6; b1→7 down one octave; b4→3)

[0153] val queryPitch = normalizeFlatToSharp(pitch)

[0154] / / Step 2: Source key → C key (reverse lookup table)

[0155] val cPitch: JianpuPitch = when (fromKey) {

[0156] KeySignature.C -> queryPitch

[0157] else -> TARGET_TO_C[fromKey]!![queryPitch]!!

[0158] }

[0159] / / Step 3: C major → Target major (forward lookup table)

[0160] val targetPitch: JianpuPitch = when (toKey) {

[0161] KeySignature.C -> cPitch

[0162] else -> C_TO_TARGET[toKey]!![cPitch]!!

[0163] }

[0164] / / Step 4: Write back the flat / press preference (if the original input was a flat, rewrite the result #X as an enharmonic bY)

[0165] return if (pitch.accidental == Accidental.FLAT) tryFlatSpelling(targetPitch) else targetPitch

[0166] }

[0167] For the manual correction module, the approach is to build an interactive interface based on Jetpack Compose. A blue border recognition box that precisely matches the position of the note is drawn on top of the original musical score, and the corresponding recognized pitch text is displayed inside the box. Users can zoom and pan to locate the target note, and clicking the recognition box will bring up an editing dialog box. Users can modify the pitch, sharps, flats, octaves of the note, or directly delete incorrect notes or add missed notes. The modification operation will be written back to the note data list in real time, and the interface will refresh synchronously to display the corrected results, ensuring the accuracy of recognition and modulation results.

[0168] Part of the main code (EditScreen.kt):

[0169] @Composable

[0170] fun EditScreen(notes: List <notesymbol>, onNoteUpdate: (NoteSymbol) -> Unit) {

[0171] / / Original image overlaid with recognition box

[0172] val imageBitmap = remember(bitmap) { bitmap.asImageBitmap()}

[0173] Image(

[0174] bitmap = imageBitmap,

[0175] contentDescription = "Original image of simplified musical notation" )

[0177] / / Draw the recognition box and blue text

[0178] notes.forEach { note ->

[0179] Box

[0180] modifier = Modifier

[0181] .offset(note.bbox.left.dp, note.bbox.top.dp)

[0182] .size(note.bbox.width.dp, note.bbox.height.dp)

[0183] .border(2.dp, Color.Blue)

[0184] .clickable {

[0185] / / A correction dialog box pops up

[0186] showEditDialog(note, onNoteUpdate)

[0187] }

[0188] ) {

[0189] Text(

[0190] text = note.pitch?.let { "${it.accidental.symbol}${it.degree}"}?: "Hugh",

[0191] color = Color.Blue )

[0193] }

[0194] }

[0195] }

[0196] For the implementation of the export rendering module, the approach is to first perform a deep copy of the original sheet music image to obtain an editable canvas, traverse the original notes and perform old symbol erasure operations at the corresponding positions, then draw the new note symbols after modulation according to the original position and layout style, add annotation text from the original key to the target key at the top of the canvas, and after completing all the drawing, save the processed image to the specified directory of the system gallery according to the Android media storage specification. At the same time, it can output the pitch sequence and Markov sequence text files before and after modulation, completely preserving the original layout and core modulation data of the sheet music.

[0197] Key code snippets:

[0198] fun exportImage(

[0199] sourceBitmap: Bitmap,

[0200] originalNotes: List <notesymbol>,

[0201] transposedNotes: List <notesymbol>): Uri {

[0202] / / 1. Create an editable copy of the image: ARGB_8888 format ensures color integrity, true indicates that it can be modified.

[0203] val canvasBitmap = sourceBitmap.copy(Bitmap.Config.ARGB_8888, true)

[0204] / / Initialize the canvas object and bind an editable Bitmap for drawing operations.

[0205] val canvas = Canvas(canvasBitmap)

[0206] / / 2. Erase original notes: Iterate through the original note list and clear the old symbols at the corresponding positions (to avoid overlap between old and new notes).

[0207] originalNotes.forEach { eraseOldSymbol(canvas, it)}

[0208] / / Draw the transposition note: Draw the new transposition mark in the original position, keeping the layout consistent with the original. Figure 1 To

[0209] transposedNotes.forEach { drawNewSymbol(canvas, it)}

[0210] / / Draw a key title: Add transposition description text (such as "Original Key -> Target Key") to the canvas to improve readability.

[0211] drawKeyTitle(canvas, "Original Key -> Target Key")

[0212] / / 3. Save to system gallery and return image URI

[0213] return saveToGallery(canvasBitmap)

[0214] }

[0215] For the playback module implementation, the approach is to first sort the structured notes according to the performance order, convert the pitch of each note into a standard MIDI value, and then convert the note duration into a corresponding playback duration in milliseconds by combining the set playback speed and time signature. A playback event sequence containing MIDI values ​​and durations is constructed. The SoundFont native audio synthesizer is used as the priority playback engine. When no available sound source is available, it falls back to the Kotlin waveform player, generating and playing audio one by one in sequence. The playback speed and time signature can be adjusted in real time to achieve accurate auditory verification of the transposed score.

[0216] Key code snippets:

[0217] fun playNotes(notes: List <notesymbol>, tempo: Int, meter:TimeSignature) {

[0218] val orderedNotes = notes.sortedWith(

[0219] / / TODO: add your comparator logic here )

[0221] val events = orderedNotes.map { note ->

[0222] PlaybackEvent(

[0223] midi = pitchToMidi(note.pitch),

[0224] durationMs = 60_000L / tempo / / One beat per note by default )

[0226] }

[0227] for (event in events) {

[0228] synth.play(event.midi, event.durationMs)

[0229] }

[0230] }

[0231] The following section will test and verify the transposition app developed for this system.

[0232] Functional testing involved taking 300 musical scores of varying complexity as examples and performing key conversions: #C→C, B→D, and #F→A. The converted files were then exported to verify whether all core functions were functioning correctly.

[0233] Test Case 1: Low-complexity sheet music ("The Moon Represents My Heart", C# major to C major)

[0234] Input: A PNG format sheet music image;

[0235] Process: Import → Preprocessing → Recognition → Manual correction (2 note recognition deviations) → Key transposition → Export image + TXT;

[0236] Results: 100% accuracy in transposition, processing time 2.1 seconds, correct exported file format, and correct layout. Figure 1 To.

[0237] Test Case 2: Medium Complexity Simplified Musical Notation ("Good Luck in 2025", B to D major)

[0238] Input: WebP format sheet music image;

[0239] Process: Import → Preprocessing → Recognition → Manual correction (3 note recognition deviations) → Key transposition → Export image + TXT;

[0240] Results: 100% accuracy in transposition; processing time approximately 2.8 seconds; exported file format correct; layout consistent with the original. Figure 1 To.

[0241] Test Case 3: High-complexity simplified musical score ("A Clear Sky Can't Compare to Having You in My Heart", F# to A major)

[0242] Input: HEIC format simplified musical notation image;

[0243] Process: Import → Preprocessing → Recognition → Manual correction (5 recognition deviations) → Key transposition → Playback verification;

[0244] Result: 100% pitch accuracy, correct playback pitch and rhythm, no stuttering.

[0245] The performance test conditions involved importing a complex musical score of 100 bars, performing 10 key conversions, testing the parsing time and transposition response time, and running it continuously for 1 hour to check for software lag and crashes. The test results were: preprocessing 0.092 seconds, recognition 0.445 seconds, transposition 0.156 milliseconds, export 0.039 seconds, and total processing time ≤ 1 second; no crashes or memory leaks were observed during 1 hour of continuous operation, and memory usage remained stable below 200MB.

[0246] The usability test involved inviting 10 non-technical users to independently complete the entire process: "User opens the software → selects the musical score image to be transposed → confirms selection → manually verifies correct recognition → selects the original and target keys → selects the export type → export successful." Operation time and feedback were recorded. First, the basic uses of the software were explained to the users; then, they were allowed to operate independently, and their operation time was recorded; finally, user feedback was collected, including the difficulty of operation and the intuitiveness of the interface. The test results showed that all 10 users could complete the entire operation within 5 minutes, with an average operation time of 3.5 minutes. Feedback indicated a simple interface, easy operation, and no obvious usability issues, meeting the usability objectives.

[0247] The test results show that all core functions, performance, and usability of the software meet the development goals. No major defects were found during the above tests. The software can run normally, meet the user's switching needs, and can be used in practical applications.< / notesymbol> < / notesymbol> < / notesymbol> < / notesymbol> < / notesymbol> < / notesymbol> < / detectedsymbol> < / detectedsymbol> < / detectedsymbol> < / mat>

Claims

1. Nonlinear mapping of full-tone modulation in digital musical notation and its application implementation. The 12-key nonlinear mapping library, connected to the key modulation module, is used to store the full-range pitch nonlinear mapping table and key signature correspondence rules for 12 keys: C, C#, D, D#, E, F, F#, G, G#, A, A#, and B. (1) According to the twelve equal temperament, an octave is divided into semitones with the same interval, namely C, C#, D, D#, E, F, F#, G, G#, A, A#, B; (2) In order to quickly achieve the goal of any key change, a "double-step key change" strategy is innovatively adopted: first change the original key to C major, and then change from C major to any key. (3) When changing from the original key to C key, according to the structure of "whole whole half whole whole whole half", calculate the number of semitones that the original key differs from C key, i.e. the offset. Then, shift the pitch of the original key by this semitone to obtain the pitch of the original key when changing to C key. (4) To change from C major to the target key, according to the structure of "whole whole half whole whole whole half", calculate the number of semitones that the C major is to change to the target key, i.e. the offset. Then, shift the pitch of the C major as a whole by this semitone to obtain the pitch of the C major to the target key. (5) Based on this principle, a nonlinear mapping library containing pitch, scale and solfège syllables for converting various modes to C major and C major to various modes was invented and constructed. The 12-tone nonlinear mapping library includes: A non-linear mapping table for 7 octaves of pitch stores the pitch value and interval relationship for each note name in 11 modes converted to C mode and C mode converted to 11 modes.

2. According to claim 1, a smart simplified musical notation modulation system based on the twelve-tone equal temperament is characterized in that, include: (1) Image input module, used to import source files containing simplified musical notation content and parse the source files into Bitmap format images to be recognized; (2) Preprocessing module, connected to image input module, used to perform orientation correction, grayscale conversion, noise reduction, binarization and spectral line segmentation on the Bitmap format image to be identified, so as to obtain a normalized spectral surface; (3) Symbol recognition module, connected to the preprocessing module, is used to perform pitch detection and recognition on the normalized simplified spectrum bitmap and output a set of simplified spectrum symbols with position and category; (4) The modulation operation module is connected to the symbol recognition module. It is used to extract pitch information based on the set of simplified musical notation symbols, call the modulation rules of the twelve equal temperament to perform pitch mapping, and output the set of standard simplified musical notation symbols after modulation. (5) Interactive correction module, connected to the transposition operation module, is used to provide a visual editing interface to add, delete and modify the pitch after transposition and perform position calibration, and output the corrected standard numbered musical notation symbol set; (6) Rendering output module, connected to the interactive correction module, is used to render the corrected standard numbered musical notation symbol set into a high-definition numbered musical notation image according to the standard numbered musical notation format, and output it to local storage or cloud. It also supports outputting the numbered musical notation transposed sequence and Markov sequence as txt files and saving the output to the local file directory. (7) Playback module, connected to the recognition module, is used to play the recognized standard musical notation symbols set pitch and according to the selectable fixed beat.

3. The system according to claim 2, characterized in that, The preprocessing module further includes: (1) Grayscale unit, used to convert the Bitmap format image to be identified into a grayscale image; (2) Denoising unit, used to call openAI's functions to denoise and remove invalid connections; (3) Binarization unit, used to completely separate the simplified spectrum symbols from the background by using adaptive threshold binarization; (4) Tilt correction unit, used to calculate the tilt angle of the page by Hough line detection and perform rotation correction to keep the musical notation lines horizontal; (5) Page segmentation unit, used to initially divide the score into lines by horizontal projection, and combined with the splitting of blank areas, the removal of weak lines and the merging of adjacent fragments to obtain independent musical score lines and symbol areas.

4. The system according to claim 2, characterized in that: (1) Supports users to import all Android native decoded digital musical notation image formats including PNG, JPEG, WebP, BMP, GIF, HEIF / HEIC (Android 8.0+), and AVIF (Android 12+); (2) Supports users to input text in the form of [a]b, where a represents an octave and b represents the pitch value.

5. The system according to claim 2, characterized in that: (1) Output the pitch information after transposition as a high-definition image of simplified musical notation, ensuring clear symbols and standardized format, and save the image to the local album in JPG format; (2) Supports outputting the original key and the pitch sequence after modulation of the simplified musical notation, and saves the file in txt format to the local simplified musical notation directory; (3) Supports outputting the original key of the simplified musical notation and its corresponding Markov pitch sequence, and saves the file in txt format to the local simplified musical notation directory.

6. The system according to claim 2, characterized in that, The modulation calculation module further includes: (1) Information parsing unit, used to parse the pitch information of the simplified musical notation; (2) A transposition mapping unit, used to retrieve the corresponding bidirectional conversion rules and notation priority rules in the full-tone nonlinear mapping library according to the original key and the target key; (3) Nonlinear recoding unit, used to recode the original pitch into the target pitch according to the rules, while keeping the absolute pitch relationship of the corresponding notes unchanged; (4) Notation output unit, used to output the target key notation representation according to the notation priority of the original key notation, and generate the target key structured notation symbol set for subsequent manual correction, rendering output and playback control.

7. The system according to claim 2, characterized in that, The interactive correction module further includes: (1) Manually identify pitches that are misrecognized by the system, as well as characters that are over-recognized or under-recognized; (2) Provides a visual editing box, which allows users to correct pitch numbers, sharps and flats, and octave information. Information that is over-recognized can be deleted. (3) Select speed and play the corrected pitch information for multimodal verification; (4) Based on the simplified musical notation information, this software provides a prompt box for transposition verification rules, which makes it easy for users to check whether the transposition is correct.

8. The system according to claim 2, characterized in that, The intelligent simplified musical notation transposition system is configured to perform the following steps: (1) Obtain the source file containing the simplified musical notation content, and parse the source file into a Bitmap format image to be recognized; (2) The Bitmap format image to be identified is subjected to orientation correction, grayscale conversion, noise reduction, binarization and spectral line segmentation to obtain a normalized spectral surface; (3) Mark the note numbers, sharps, flats, and octaves on the normalized simplified musical notation bitmap to generate a structured simplified musical notation symbol set with the original coordinates; (4) The structured spectrum symbol set is manually corrected; (5) Extract the pitch, sharps and flats and octave information from the structured simplified musical notation set, and receive the target key information. Call the full-mode nonlinear mapping library based on the twelve equal temperament to perform bidirectional conversion on the 12 modes with key signatures C, C#, D, D#, E, F, F#, G, G#, A, A# and B to obtain the target key structured simplified musical notation set; (6) The output format can be selected in multiple modes. The obtained target simplified spectrum symbol set can be overlaid and rendered according to the original spectrum coordinates, and the corrected standard simplified spectrum symbol set can be output. It can output the target key structured musical notation set as a text sequence; it can obtain the Markov transition probability and transition probability matrix of the pitch; it can generate an audio-format structured musical notation set based on the pitch in the structured musical notation set and the target key musical notation set, according to selectable tempo information.

Images