Natural language driven song generation and intelligent instrument guiding method and system

Abstract

The application relates to the technical field of intelligent musical instruments, and discloses a natural language driving song generation and intelligent musical instrument guiding method and system, which comprises the following steps: seven types of information are extracted from a natural language description through a multidimensional music intention analysis engine, reasoning and conflict resolution are carried out in combination with a knowledge graph; structured instructions are input into a collaborative generation engine, a song is collaboratively generated through a structure, chord and melody generator coupled through cross attention, and a cyclic correction mechanism is used to ensure harmonic consistency; the generation result is encoded into a unified music intermediate representation and is compiled and output; based on the unified representation, an optimal fingering is solved through dynamic programming, and an LED sequence containing a pre-highlight guide is generated; and the LED sequence is sent to an intelligent guitar driving fretboard to guide performance. The application realizes full-link automation from natural language to entity musical instrument follow-the-light performance, cyclic correction improves music quality, LED pre-highlight reduces follow-the-light performance difficulty, and supports performance feedback and adaptive adjustment.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent musical instruments, and more particularly to a method and system for song generation and intelligent musical instrument guidance based on natural language. Background Technology

[0002] With the rapid development of artificial intelligence technology, computer-aided music composition has become an active research area. Traditional computer-aided composition systems require users to manually input chord progressions and select style templates, demanding a certain level of music theory knowledge and thus having a relatively high barrier to entry. In recent years, deep learning-based music generation models have been able to generate audio based on simple text prompts; however, these solutions generally suffer from the following shortcomings:

[0003] First, the granularity of control is coarse. Existing models can usually only receive simple text prompts and cannot understand natural language instructions containing specific musical composition intentions, such as "write a fast-paced pop love song in C major with some electronic elements, and the chorus should be memorable." For core elements of music composition such as tonality, tempo, paragraph structure, chord progression preferences, and instrumentation, existing technologies lack systematic analysis and structured control capabilities.

[0004] Secondly, the output format is limited. Existing solutions only generate audio waveform files and do not output editable sheet music formats. If users want to actually play the generated songs, they still need to manually transcribe the music—that is, to restore the audio to staff notation, numbered musical notation, or guitar fingering chart through auditory analysis. This process is extremely time-consuming and requires professional musical training.

[0005] Third, there is a lack of an interaction layer with physical instruments. Current smart guitar products on the market guide users to press the strings by embedding LED lights on the fretboard, but their tracks are all pre-loaded after being manually transcribed, and cannot be loaded with AI-generated tracks in real time. Users cannot instantly create a new song and play it directly on the smart guitar following the lights. Furthermore, existing LED fretboard systems only illuminate the position of the note to be played, lacking visual previews of upcoming notes, making it difficult for beginners to follow along.

[0006] Fourth, existing music generation models process song structure generation, chord arrangement, and melody creation as independent modules in a sequential manner, lacking global consistency constraints. Since there is only a one-way conditional dependency between the chord generator and the melody generator (chords are determined first, then melody), the melody generator cannot influence chord decisions in reverse, which may lead to dissonance between chords and melody, affecting the musical quality of the generated result. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and to provide a method and system for song generation and intelligent musical instrument guidance based on natural language.

[0008] The objective of this invention is achieved through the following technical solution: a method for song generation and intelligent instrument guidance based on natural language, comprising the following steps:

[0009] S1. Receive the natural language description text input by the user, extract tonality, tempo, style, paragraph structure, orchestration requirements, emotional tendencies and special instructions through a multi-dimensional music intent analysis engine, and generate structured music instructions by implicitly constraining reasoning and conflict resolution of incomplete input through a music knowledge graph.

[0010] S2. Input the structured music instructions from step S1 into the collaborative generation engine. Generate the song generation result through the paragraph structure generator, chord progression generator and melody line generator that are mutually constrained by the cross-attention mechanism, and ensure harmony consistency through the loop correction mechanism.

[0011] S3. Convert the song generation result of step S2 into a unified music intermediate representation, and use a multi-format music score compiler to compile and output the unified music intermediate representation as a reference score and MIDI file. The reference score includes staff notation, numbered musical notation, chord function notation, and guitar six-line fingering notation.

[0012] S4. Based on the unified music intermediate representation in step S3, the optimal fingering is solved by a dynamic programming algorithm with position continuity constraints, and an LED light control sequence with a pre-brightening guidance mechanism is generated based on the optimal fingering.

[0013] S5. Send the LED lighting control sequence from step S4 to the smart guitar to drive the fretboard LEDs to light up in sequence to guide the playing.

[0014] Preferably, the multi-dimensional music intent analysis engine in step S1 includes

[0015] Music attribute extractor, used to extract tonality, tempo and beat, style tags, orchestration requirements, paragraph structure requirements, emotional inclination and special instructions from natural language;

[0016] An implicit constraint inferencer, connected to a music knowledge graph, is used to automatically infer and complete missing constraints when user input is incomplete.

[0017] The conflict detector and resolver is used to detect contradictory commands in user input, identify conflict types, and automatically resolve conflicts according to a preset priority strategy.

[0018] Preferably, the collaborative generation engine in step S2 adopts a hierarchical generation architecture, wherein:

[0019] The paragraph structure generator outputs a paragraph structure containing paragraph type, number of sections, and repetition pattern at a coarse-grained level.

[0020] The chord progression generator uses a dual-attention Transformer network. The first attention head focuses on the paragraph structure context, and the second attention head focuses on the harmonic function constraints of the current key, outputting a measure-by-measure chord sequence.

[0021] The melody line generator takes the chord progression of the chord sequence as a condition, selects the chord tones of the current chord on the strong beat using a chord tone priority strategy, and fills the weak beats with passing tones and auxiliary tones to produce the song generation result.

[0022] Preferably, the cyclic correction mechanism in step S2 specifically includes the following steps:

[0023] S201. Calculate the interval conflict between the melody notes and the current chord, and generate a harmonic consistency score;

[0024] S202. When the harmonic consistency score in step S201 is lower than the preset threshold, mark the conflict position, trigger the melody line generator to resample the measure, and re-detect the harmonic consistency score until the score exceeds the preset threshold.

[0025] S203. If the harmonic consistency score still fails to meet the standard after more than a preset number of resampling attempts, the chord progression corresponding to the conflict position is adjusted in reverse. Chord fine-tuning is performed while preserving the harmonic function, and then melody generation is triggered again.

[0026] Preferably, the unified music intermediate representation in step S3 includes the metadata of the encoded song, paragraph structure, chord sequence, melody sequence, and fingering instructions for each instrument track.

[0027] Preferably, the state transition cost function of the dynamic programming algorithm with position continuity constraints in step S4 simultaneously considers finger movement distance, open string utilization priority, and pre-lighting time window adaptation, and outputs the optimal fingering that minimizes the total cost at a specified difficulty level.

[0028] Preferably, the pre-brightening guidance mechanism in step S4 includes: while driving the smart guitar fretboard LED to light up the currently played note, lighting up the target fret LED to be played with a weak light color that is lower than the normal brightness by a preset time offset, and the brightness of the weak light color increases linearly to the normal brightness as the playing time approaches.

[0029] Preferably, the LED light control sequence in step S4 adopts a color coding strategy to distinguish at least one of the following performance elements with different colors: melody tone, chord root tone, power chord tone, performance technique transition and pre-lighting indication;

[0030] Different lighting rendering modes are used for different playing techniques. The lighting rendering modes include: string bending uses sequential flashing along the fret direction, glissando uses dynamic gradual change from the starting LED to the ending LED, string striking and string pulling use pulse-like high brightness, and vibrato uses periodic sinusoidal fluctuation of LED brightness.

[0031] Preferably, the method further includes step S6, which specifically includes the following steps:

[0032] S601: The smart guitar collects the user's actual playing data in real time through its built-in sensors, compares the pitch and rhythm with the generated reference sheet music, and calculates the playing accuracy.

[0033] S602. When the performance accuracy in step S601 is lower than the first threshold, the chord density of subsequent measures is automatically simplified, the technical embellishments are reduced, or the playback speed is reduced.

[0034] S603. When the performance accuracy in step S601 is higher than the second threshold, the performance challenge of subsequent measures is automatically increased.

[0035] A natural language-driven song generation and intelligent instrument guidance system, including

[0036] The multi-dimensional music intent analysis module is used to extract tonality, tempo, style, paragraph structure, orchestration requirements, emotional tendencies and special instructions from the natural language description text input by the user. It includes a music knowledge graph reasoning unit to perform implicit constraint completion when the input information is incomplete, and a conflict resolution unit to detect and resolve contradictory instructions.

[0037] The collaborative generation engine module includes a paragraph structure generator, a chord progression generator, and a melody line generator. These three components are coupled together through a cross-attention network and include a loop correction unit for detecting and eliminating harmonic conflicts between melody and chords.

[0038] The Unified Music Score Compiler Module is used to encode the output of the Cooperative Generation Engine Module into a unified music intermediate representation and compile the output to include at least multiple target formats such as reference scores, MIDI files, and smart instrument LED light control sequences;

[0039] The intelligent musical instrument adaptation module includes a fingering optimization solver for solving the optimal fingering using a dynamic programming algorithm with position continuity constraints; and an LED timeline renderer for generating an LED lighting control sequence with a pre-brightening guidance mechanism based on the optimal fingering.

[0040] The present invention has the following advantages and beneficial effects compared with the prior art:

[0041] This invention achieves end-to-end automation from natural language description to complete song playing on a physical instrument with LED guidance. Users can complete song creation and performance learning without any music theory knowledge. Through a global constraint network, structure generation, chord arrangement, and melody creation are collaboratively generated with a cyclical correction mechanism, avoiding the music quality problems caused by the fragmentation of modules in existing technologies. It not only generates audio, but more importantly, it generates editable and playable sheet music formats such as staff notation, simplified notation, and guitar tablature. For the first time, it realizes real-time interaction between AI-generated tracks and smart instrument hardware through a proprietary MIDI-to-LED link. The LED pre-lighting guidance mechanism provides visual previews in the physical fretboard space, combined with multi-color encoding and technique visualization rendering, significantly reducing the difficulty for beginners and improving the performance learning experience. It supports performance feedback and adaptive difficulty adjustment, forming a complete intelligent music education closed loop of "creation → performance → evaluation → adjustment". Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall architecture of the natural language-driven song generation and intelligent musical instrument guidance system of this invention;

[0043] Figure 2 yes Figure 1 A schematic diagram of the cross-attention network structure of the collaborative generation engine;

[0044] Figure 3 This is a schematic diagram of the user interface. Detailed Implementation

[0045] The invention's objective will be further described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments cannot be described in detail here, but the implementation of the invention is not limited to the following embodiments.

[0046] Example 1

[0047] The method for natural language-driven song generation and intelligent instrument guidance includes the following steps:

[0048] S1. Receives natural language description text input by the user, extracts tonality, tempo, style, paragraph structure, orchestration requirements, emotional inclination, and special instructions through a multi-dimensional music intent parsing engine, and uses a music knowledge graph to perform implicit constraint reasoning and conflict resolution on incomplete input to generate structured music instructions. The multi-dimensional music intent parsing engine supports a multi-turn dialogue interaction mode, where the user makes targeted modifications to the generated song through incremental natural language instructions. The intent parsing engine identifies the modification target and direction, performing incremental modifications within the existing context without regenerating the entire song.

[0049] The multi-dimensional music intent analysis engine includes a music attribute extractor, an implicit constraint inferencer, and a conflict detection and resolution system. The music attribute extractor extracts seven categories of information from natural language: tonality, tempo and beat, style tags, orchestration requirements, paragraph structure requirements, emotional tone, and special instructions. The implicit constraint inferencer connects to a music knowledge graph and automatically infers and completes missing constraints when user input is incomplete; it maps tempo descriptions to specific BPM ranges, automatically completes orchestration and paragraph structure templates based on style tags, and determines a default tonality recommendation based on emotional tone. The conflict detection and resolution system detects contradictory instructions in user input, identifies conflict types, and automatically resolves them according to a preset priority strategy.

[0050] The user enters a natural language command in the input box of the mobile app: "Write a fast-paced pop love song in C major with some electronic elements. The chorus should be memorable. I want to play it on my smart guitar. Arrange it into a beginner-friendly version."

[0051] The music attribute extractor in the intent parsing engine performs semantic analysis on the input text and extracts the following structured information:

[0052] Key: C major

[0053] Speed: Fast-paced, mapped to 128 BPM via knowledge graph reasoning.

[0054] Time signature: Inferred from the pop music style, it is 4 / 4 time.

[0055] Style tags: pop + electronic

[0056] Instrumentation requirements: Guitar (explicitly specified by the user), synthesizer (electronic elements automatically associated), and drums (standard for pop music).

[0057] Paragraph structure requirements: Standard structure for popular songs (intro, verse, pre, chorus, chorus, ...)

[0058] Emotional inclination: Romantic (love songs)

[0059] Special instructions: The chorus should be memorable (using repetitive motifs in the main melody) and beginner-friendly (marked as easy).

[0060] The reasoning for "fast tempo is 128 BPM" was completed using a music knowledge graph, which is associated with the mapping rule that "fast tempo is the median of 128 within the range of 120-140 BPM". No contradictory instructions were found after the conflict detection and resolution scanner scan.

[0061] S2. Input the structured music instructions from step S1 into the collaborative generation engine. Generate the song generation result through the paragraph structure generator, chord progression generator, and melody line generator that are mutually constrained by the cross-attention mechanism, and ensure harmonic consistency through the loop correction mechanism.

[0062] The collaborative generation engine employs a hierarchical generation architecture. The paragraph structure generator outputs paragraph structures at a coarse-grained level, including paragraph type, number of measures, and repetition pattern. The chord progression generator uses a dual-attention Transformer network: the first attention head focuses on the paragraph structure context, and the second attention head focuses on the harmonic functional constraints of the current key, outputting a measure-by-measure chord sequence. The melody line generator, conditioned on the chord progression of the chord sequence, selects the chord tones of the current chord on strong beats using a chord tone priority strategy, and fills the weak beats with passing tones and auxiliary tones of the key scale, outputting the generated song.

[0063] A structured music instruction input collaborative generation engine. First, the paragraph structure generator outputs the song paragraph skeleton:

[0064] [Intro: 4 measures] → [Verse 1: 8 measures] → [Pre-Chorus: 4 measures] → [Chorus: 8 measures] → [Verse 2: 8 measures] → [Pre-Chorus: 4 measures] → [Chorus: 8 measures] → [Bridge: 8 measures, transition to Am] → [Chorus: 8 measures] → [Outro: 4 measures, diminuendo]

[0065] Subsequently, the chord progression generator outputs the chord sequence segment by segment under the constraints of the C major harmonic function. Taking Verse 1 and Chorus as an example:

[0066] Verse 1:|C|G / B|Am7|F|C|G|Am|F|

[0067] Chorus: |C|G|Am|F|C|G|F|G| (Terminal V→I regression C)

[0068] The melody line generator generates melodies measure by measure, conditioned on chord sequences and prioritizing chord tones. In the chorus section, to meet the "memorable point" requirement, the system incorporates repeated ascending leap motifs (such as E4→G4→C5) in measures 1-2 and 5-6 to enhance the melody's recognizability and singability.

[0069] During the generation process, a note (B) appeared in the melody of the third measure that clashed minor second with the third (C) of the Am chord on a strong beat, causing the harmonic consistency score to drop to 0.71 (below the threshold of 0.85). The loop correction mechanism marked this position, triggering the melody generator to resample the third measure, replacing B with C (the chord third). After re-detection, the harmonic score rose to 0.92, and the correction was successful.

[0070] Arrangement of electronic elements: The synthesizer track generates a bass pad tone in the Verse section as a base, and adds an eighth-note Arp pattern in the Chorus section to increase rhythmic drive.

[0071] The cross-attention mechanism in the collaborative generation engine is as follows: Let the paragraph structure be encoded as P, the chord sequence as C, and the melody sequence as M. During the generation process, the chord generator uses P as the query and itself as the key and value for self-attention, while simultaneously using P as the key and value for cross-attention, thereby perceiving the paragraph structure constraints; the melody generator uses C as the query and itself and P as the key and value for multi-head cross-attention, thereby simultaneously perceiving the chord progression and paragraph structure. The three components iteratively optimize through a shared latent space.

[0072] The cyclical correction mechanism includes the following steps:

[0073] S201. Calculate the interval conflict between the melody notes and the current chord, and generate a harmonic consistency score;

[0074] S202. When the harmonic consistency score in step S201 is lower than the preset threshold, mark the conflict position, trigger the melody generator to resample the measure, and re-detect the harmonic consistency score until the score exceeds the preset threshold.

[0075] S203. If the harmonic consistency score still fails to meet the standard after more than a preset number of resampling attempts, the chord progression corresponding to the conflict position is adjusted in reverse. Chord fine-tuning is performed while preserving the harmonic function, and then melody generation is triggered again.

[0076] To verify the technical effectiveness of the loop correction mechanism, an ablation experiment was conducted on 100 example songs of different styles and tones to compare the harmonic consistency performance when the loop correction mechanism was turned off and on.

[0077] Experimental setup: Harmonic Consistency Score (HCS, range 0.0-1.0) was calculated for 100 songs generated from natural language commands with and without the loop correction mechanism enabled. The number of conflicts that triggered the correction in each song was recorded, and the success rate of the correction was calculated.

[0078] Table 1 Comparative Experimental Results

[0079] index Close the loop correction Enable cyclical correction Average HCS 0.72 0.91 The proportion of songs with HCS < 0.85 78% 8% Average number of conflicts per song 12.4 2.1 (Residual after correction) Correction success rate — 94.3% Melody resampling trigger count (per song) — 3.2 times Reverse adjustment of chord trigger counts (per song) — 0.6 times Average additional calculation time per song — +1.8s

[0080] Experimental results show that after enabling the loop correction mechanism, the average HCS increased from 0.72 to 0.91 (an improvement of 26.4%), with only 8% of songs having sections with an HCS below 0.85, far lower than the 78% when it was disabled. Of the 12.4 conflicts, 94.3% could be corrected through melody resampling (an average of 3.2 times per song), and only 5.7% of persistent conflicts (approximately 0.7 conflicts per song) required triggering inverse chord fine-tuning. The additional computational overhead was only 1.8 seconds, well within an acceptable range for user experience.

[0081] Taking an E major rock song as an example, the HCS change trajectory during the cyclic correction process is further demonstrated:

[0082] Initial generation (first pass): HCS = 0.68 (7 conflict sections in total)

[0083] Second resampling: HCS=0.79 (4 conflicts corrected, 3 remaining)

[0084] Third resampling: HCS=0.85 (2 conflicts corrected, 1 remaining)

[0085] Fourth resampling pass: HCS=0.85 (stubborn conflicts not improved)

[0086] Trigger chord fine-tuning: Fine-tunes the chord in conflict position from F#m to A (both belong to the same harmonic function group and are subdominant functions).

[0087] The final HCS value was 0.93.

[0088] This trajectory shows that the cyclic correction mechanism can solve the problem in the vast majority of cases (within 4 resamplings), with only a very few stubborn conflicts requiring chord inversion fine-tuning.

[0089] S3. Convert the song generation result from step S2 into a Unified Music Representation (UMR), and use a multi-format score compiler to compile and output the UMR as reference scores (including staff notation, numbered musical notation, chord function notation, and guitar fingering notation) and MIDI files. The UMR is used to encode the song's metadata, paragraph structure, chord sequences, melody sequences, and fingering instructions for each instrument track.

[0090] The output of the collaborative generation engine is encoded into a unified music intermediate representation, and then compiled by a multi-format score compiler into the following format:

[0091] MusicXML format sheet music (for professional musicians).

[0092] Simplified musical notation and chord progressions (for band rehearsals)

[0093] Guitar tablature (beginner-friendly version, using open chord fingering on the first three frets)

[0094] S4. Based on the unified music intermediate representation in step S3, the optimal fingering is solved by a dynamic programming algorithm with position continuity constraints, and an LED light control sequence with a pre-brightening guidance mechanism is generated based on the optimal fingering.

[0095] Among them, the state transition cost function of the dynamic programming algorithm with position continuity constraint considers the finger movement distance, the priority of open string utilization, and the adaptation of the pre-lighting time window, and outputs the optimal fingering that minimizes the total cost under the specified difficulty level.

[0096] The dynamic programming algorithm with position continuity constraints described in step S4 has the following state transition equation:

[0097]

[0098] in, The note's sequence number in the performance sequence ( ≥1), For the first The position selected by each note For the first The position of each note. This refers to the set of all legal positions on the guitar fretboard (including combinations of frets for each string, such as frets 0-12). Indicates before the performance ends The note and the The note falls in position. Minimum cumulative cost at time For the fingers from the position Move to the position The cost is positively correlated with the difference in fret and string number between finger movements, and is used to penalize large jumps to ensure positional continuity. To open the string reward item (take negative value), when the position is... A negative penalty is imposed when dealing with open string notes, encouraging the use of open strings to reduce difficulty. For the first The duration of each note's performance (in beats or milliseconds, converted according to BPM). To compensate for the cost of adapting to the pre-brightness time window, when at time ( When setting a pre-brightness advance (e.g., one beat), the fingerboard LED resources or finger position cannot simultaneously meet the pre-brightness requirements. When a requirement is made, a high price is imposed; otherwise, the price is zero or very small.

[0099] Using the recursive equation described above, the algorithm calculates each note sequentially starting from the first note, ultimately selecting the position sequence {p1,p2,...,pN} that minimizes the total cost Cost(N,pN) as the optimal fingering, where N is the total number of notes. This algorithm, at a specified difficulty level, adjusts the weights of various costs (e.g., increasing...) Weighting to restrict cross-product use and increase The absolute value is used to encourage open strings, and the output is the optimal fingering that satisfies the difficulty constraints.

[0100] The pre-brightening guidance mechanism includes: while driving the fretboard LED of the smart guitar to light up the currently played note, lighting up the target fret LED of the upcoming note with a weak light color that is lower than the normal brightness by a preset time offset, and the brightness of the weak light color increases linearly to the normal brightness as the playing time approaches.

[0101] The LED lighting control sequence adopts a color coding strategy, using different colors to distinguish the main melody tone, chord root tone, power chord tone, transition of playing techniques (glissando / hammer / pull-off, etc.) and pre-brightness indication; different lighting rendering modes are used for different playing techniques, including: string bending uses sequential flashing along the fret direction, glissando uses dynamic gradation from the start to the end, hammer-off and pull-off use beat pulse highlighting, and vibrato uses periodic sine wave brightness fluctuation.

[0102] The fingering optimization solver aims for a "beginner-friendly" difficulty level. Under conditions of position continuity constraints (finger movement distance ≤ 3 frets, priority use of open strings, smooth position switching), it uses a dynamic programming algorithm to solve for the optimal fingering for guitar tracks. The LED timeline renderer generates an LED control sequence based on the optimal fingering and overlays a pre-lighting guidance mechanism. The LED pre-lighting signal for each note illuminates the fretboard with a 30% brightness white light one beat in advance.

[0103] Color coding scheme: Green for the melody note, blue for the root note of the chord, red for the power chord note, and a soft white light for the pre-brightness indicator. At the points where string bending techniques appear in the Bridge section, LEDs flash sequentially along the frets to indicate the direction and amplitude of the bend.

[0104] To verify the technical effectiveness of the pre-brightness guidance mechanism, the applicant organized a control experiment.

[0105] Experimental Design: Forty guitar beginners were recruited (defined as volunteers who had been playing guitar for less than 3 months and could not independently sight-read tablature). They were randomly divided into an experimental group (n=20, guided by LEDs with a pre-lighting mechanism) and a control group (n=20, guided by traditional LEDs that only illuminate the current note). Both groups used the same smart guitar hardware and the same "beginner" songs (BPM 80, 4 / 4 time, simple chord progression in C major), practicing each song 3 times (9 times in total). The experimental metrics were: first-time pass rate (the percentage of students who could play along completely on the first try), average playing accuracy after three tries, and the number of interruption requests (users actively pausing due to falling behind) during practice.

[0106] Table 2 Comparison of experimental results

[0107] index Control group (no pre-brightness) Experimental group (with pre-brightness) First pass rate 15% (3 out of 20 people) 55% (11 out of 20 people) Average playing accuracy after three attempts 64.3% 83.7% Average number of interruption requests per pass 4.7 times 1.3 times Root note localization error rate 32.1% 8.5% Shifting error rate 28.6% 9.2% Glide / Bend Technique Completion Rate 41.0% 74.3% Total practice time (three times) 18.2min 11.5min

[0108] Experimental results show that the pre-illumination guidance mechanism increased the first-time pass rate for beginners from 15% to 55% (an increase of 266.7%), and the average playing accuracy after three practice sessions increased from 64.3% to 83.7% (an absolute increase of 19.4 percentage points), while reducing the number of interruption requests per session by 72.3%. The pre-illumination mechanism significantly improved error-prone scenarios: the root note positioning error rate and position shifting error rate decreased from 32.1% and 28.6% to 8.5% and 9.2%, respectively. For techniques such as glissando and string bending, the completion rate increased from 41.0% to 74.3% (an increase of 81.2%), attributed to the visual rendering of techniques (string bending flashing prompts, glissando gradation, etc.) providing intuitive previews of the direction and amplitude of the operation.

[0109] In addition, the subjective experience of users was assessed using a 5-point Likert scale: the experimental group scored significantly lower than the control group (4.3±0.6, p<0.01) on "whether you feel heel pressure" (2.1±0.8) and significantly higher than the control group (3.2±1.0, p<0.01) on "whether you are willing to continue using it" (4.6±0.5).

[0110] S5. Send the LED lighting control sequence from step S4 to the smart guitar to drive the fretboard LEDs to light up in sequence to guide the playing. The LED lighting control sequence uses a color coding strategy to distinguish playing elements with different colors. Playing elements include melody notes, chord root notes, power chord notes, playing technique transitions, and pre-lighting indicators.

[0111] Different lighting rendering modes are used for different playing techniques. The lighting rendering modes include: string bending uses sequential flashing along the fret direction, glissando uses dynamic gradual change from the starting LED to the ending LED, string striking and string pulling use pulse-like high brightness, and vibrato uses periodic sinusoidal fluctuation of LED brightness.

[0112] The smart guitar receives an LED lighting control sequence via Bluetooth, and the fretboard LEDs light up sequentially, beat by beat. A pre-lit white light appears on the fret the user is about to play, increasing linearly over time, and then switches to the target color once the normal brightness is reached. The user completes the performance based on the light guidance.

[0113] like Figure 1 As shown, the natural language-driven song generation and intelligent instrument guidance system includes a multi-dimensional music intent parsing module, a collaborative generation engine module, a unified score compiler module, and an intelligent instrument adaptation module.

[0114] The multi-dimensional music intent analysis module is used to extract tonality, tempo, style, paragraph structure, orchestration requirements, emotional tendencies and special instructions from the natural language description text input by the user. It includes a music knowledge graph reasoning unit to perform implicit constraint completion when the input information is incomplete, and a conflict resolution unit to detect and resolve contradictory instructions.

[0115] The collaborative generation engine module includes a paragraph structure generator, a chord progression generator, and a melody line generator. These three components are coupled together through a cross-attention network and include a loop correction unit for detecting and eliminating harmonic conflicts between melody and chords.

[0116] The unified score compiler module is used to encode the output of the collaborative generation engine module into a unified music intermediate representation, and compile the output to include at least a variety of target formats, including staff notation, numbered musical notation, guitar tablature, MIDI files, and smart instrument LED light control sequences;

[0117] The intelligent musical instrument adaptation module includes a fingering optimization solver for solving the optimal fingering using a dynamic programming algorithm with position continuity constraints; and an LED timeline renderer for generating an LED lighting control sequence with a pre-brightening guidance mechanism based on the optimal fingering.

[0118] Example 2

[0119] Except for the following technical features, the other technical features in this embodiment are the same as those in Embodiment 1:

[0120] The system in this embodiment also includes a performance feedback and adaptive adjustment module, which receives sensor data from the smart guitar, evaluates the user's performance accuracy in real time, and dynamically adjusts the difficulty level of subsequent scores based on the evaluation results.

[0121] This embodiment also includes an adaptive difficulty adjustment mechanism in step S6, which specifically includes the following steps:

[0122] S601 uses the built-in sensors of the smart guitar to collect the user's actual playing data in real time, compares it with the generated reference score in terms of pitch and rhythm, and calculates the playing accuracy.

[0123] The guitar's built-in piezoelectric pickups capture the user's playing audio in real time, and after pitch detection, compares it with the reference sheet music. The performance feedback module calculates that the user's pitch accuracy is 78% (above the first threshold of 60%, below the second threshold of 90%), maintains the current difficulty level, and continues playing the complete piece.

[0124] S602. When the performance accuracy is lower than the first threshold, automatically simplify the chord density of subsequent measures, reduce technical embellishments, or reduce the playback speed.

[0125] S603. When the performance accuracy is higher than the second threshold, automatically increase the performance challenge of subsequent measures, including adding hit-and-pull techniques, string bending, or increasing BPM.

[0126] To verify the effectiveness of the adaptive difficulty adjustment mechanism, the differences in the system's output score were tested under three preset difficulty levels. The same natural language command, "A gentle folk song in G major," was input, and the difficulty levels were set to easy, medium, and hard, respectively. The output results were then compared.

[0127] Table 3 Output Comparison Results

[0128] Output metrics easy medium hard Total number of chords 24 36 52 Chord type Major / minor triads only Add seventh chords and suspended chords Includes ninth chords, inversions, and modulated chords. Average number of chords per measure 1.5 2.2 3.3 melodic range C4-G4 (5 degrees) A3-C5 (10 degrees) G3-E5 (13 degrees) Melodic ornament ratio 8% 22% 41% Guitar Track Technique Only pluck Adding a pull-off Add string bending, glissando, and vibrato. Position range Top 3 Top 5 The first 12 items Maximum span 2nd grade 4th grade 5th grade Total song duration 3:12 3:36 4:08 Generation time 2.1s 3.4s 5.7s LED color options 2 colors (green + blue) 3 colors 5-color (full encoding scheme)

[0129] Experiments show that the three difficulty levels exhibit clear stratification in terms of musical density, technical complexity, and fingering range, validating the effectiveness of the difficulty control mechanism. The easy level restricts chords to triads, positions to the first three frets, and fingering ranges to within two frets, making it suitable for beginners with no prior experience. The hard level fully utilizes modulated chords, inversions, full-position fingering, and all playing techniques, providing a challenge close to real performance.

[0130] Further testing was conducted in adaptive difficulty mode: the system was run with a simulated trajectory of performance accuracy (initial accuracy 52%, evaluated once per measure). When the accuracy remained below the first threshold (60%) for three consecutive measures, the system automatically downgraded subsequent measures from the current medium level to an easy simplified version. Chord density was reduced from an average of 2.2 per measure to 1.5, the span limit was narrowed to 2 frets, and slides and hammer-ons / pull-off techniques were removed. After the downgrade, the simulated accuracy rebounded to 73% (above the threshold) in the 5th measure, and the system maintained the simplified version to complete the performance. This validation demonstrates that the adaptive difficulty mechanism can respond in real time to the learner's performance during the performance, dynamically matching their current ability level.

[0131] Example 3

[0132] Except for the following technical features, the other technical features in this embodiment are the same as those in Embodiment 1:

[0133] Based on the song generated in Example 1, the user enters a second natural language command in the APP: "The chorus is too ordinary, change it to be more explosive."

[0134] In step S1, the multi-dimensional music intent analysis engine identifies the modification target as the Chorus section and the modification direction as enhancing dynamics and intensity. While preserving the rest of the original song, the multi-dimensional music intent analysis engine sends targeted modification instructions to the collaborative generation engine:

[0135] (1) The chord density of the chord is increased from 2 chords per measure to 3 chords per measure, which increases the sense of progression;

[0136] (2) The overall melody range is raised by a minor third, and a high sustain (G5, lasting 2 beats) is inserted at Climax.

[0137] (3) The drum track switches to a double bass rhythm pattern in the Chorus section;

[0138] (4) Add a distortion guitar tone to the synthesizer to increase the "explosiveness".

[0139] After the modifications are completed, the system only recompiles the score and LED sequence for the Chorus and related transition measures; the rest of the music remains unchanged. Once the user confirms the changes, the new LED sequence overwrites the original sequence and is sent to the smart guitar.

[0140] The above-described specific embodiments are preferred embodiments of the present invention and are not intended to limit the present invention. Any other changes or equivalent substitutions made without departing from the technical solution of the present invention are included within the protection scope of the present invention.

Claims

1. A method for song generation and intelligent instrument guidance based on natural language, characterized by: Includes the following steps: S1. Receive the natural language description text input by the user, extract tonality, tempo, style, paragraph structure, orchestration requirements, emotional tendencies and special instructions through a multi-dimensional music intent analysis engine, and generate structured music instructions by implicitly constraining reasoning and conflict resolution of incomplete input through a music knowledge graph. S2. Input the structured music instructions from step S1 into the collaborative generation engine. Generate the song generation result through the paragraph structure generator, chord progression generator and melody line generator that are mutually constrained by the cross-attention mechanism, and ensure harmony consistency through the loop correction mechanism. S3. Convert the song generation result of step S2 into a unified music intermediate representation, and use a multi-format music score compiler to compile and output the unified music intermediate representation as a reference score and MIDI file. The reference score includes staff notation, numbered musical notation, chord function notation, and guitar six-line fingering notation. S4. Based on the unified music intermediate representation in step S3, the optimal fingering is solved by a dynamic programming algorithm with position continuity constraints, and an LED light control sequence with a pre-brightening guidance mechanism is generated based on the optimal fingering. S5. Send the LED lighting control sequence from step S4 to the smart guitar to drive the fretboard LEDs to light up in sequence to guide the playing.

2. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: Step S1's multi-dimensional music intent analysis engine includes Music attribute extractor, used to extract tonality, tempo and beat, style tags, orchestration requirements, paragraph structure requirements, emotional inclination and special instructions from natural language; An implicit constraint inferencer, connected to a music knowledge graph, is used to automatically infer and complete missing constraints when user input is incomplete. The conflict detector and resolver is used to detect contradictory commands in user input, identify conflict types, and automatically resolve conflicts according to a preset priority strategy.

3. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The collaborative generation engine in step S2 adopts a hierarchical generation architecture, wherein: The paragraph structure generator outputs a paragraph structure containing paragraph type, number of sections, and repetition pattern at a coarse-grained level. The chord progression generator uses a dual-attention Transformer network. The first attention head focuses on the paragraph structure context, and the second attention head focuses on the harmonic function constraints of the current key, outputting a measure-by-measure chord sequence. The melody line generator takes the chord progression of the chord sequence as a condition, selects the chord tones of the current chord on the strong beat using a chord tone priority strategy, and fills the weak beats with passing tones and auxiliary tones to produce the song generation result.

4. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The cyclic correction mechanism in step S2 specifically includes the following steps: S201. Calculate the interval conflict between the melody notes and the current chord, and generate a harmonic consistency score; S202. When the harmonic consistency score in step S201 is lower than the preset threshold, mark the conflict position, trigger the melody line generator to resample the measure, and re-detect the harmonic consistency score until the score exceeds the preset threshold. S203. If the harmonic consistency score still fails to meet the standard after more than a preset number of resampling attempts, the chord progression corresponding to the conflict position is adjusted in reverse. Chord fine-tuning is performed while preserving the harmonic function, and then melody generation is triggered again.

5. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The unified music intermediate representation mentioned in step S3 includes the metadata of the encoded song, paragraph structure, chord sequence, melody sequence, and fingering instructions for each instrument track.

6. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The state transition cost function of the dynamic programming algorithm with position continuity constraints in step S4 simultaneously considers finger movement distance, open string utilization priority, and pre-lighting time window adaptation, and outputs the optimal fingering that minimizes the total cost at a specified difficulty level.

7. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The pre-brightening guidance mechanism in step S4 includes: while driving the smart guitar fretboard LED to light up the currently played note, lighting up the target fret LED to be played with a weak light color that is lower than the normal brightness by a preset time offset, and the brightness of the weak light color increases linearly to the normal brightness as the playing time approaches.

8. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: The LED light control sequence in step S4 adopts a color coding strategy to distinguish at least one of the following performance elements with different colors: melody tone, chord root tone, power chord tone, performance technique transition and pre-light indication; Different lighting rendering modes are used for different playing techniques. The lighting rendering modes include: string bending uses sequential flashing along the fret direction, glissando uses dynamic gradual change from the starting LED to the ending LED, string striking and string pulling use pulse-like high brightness, and vibrato uses periodic sinusoidal fluctuation of LED brightness.

9. The method for song generation and intelligent instrument guidance based on natural language as described in claim 1, characterized in that: It also includes step S6, which specifically includes the following steps: S601: The smart guitar collects the user's actual playing data in real time through its built-in sensors, compares the pitch and rhythm with the generated reference sheet music, and calculates the playing accuracy. S602. When the performance accuracy in step S601 is lower than the first threshold, the chord density of subsequent measures is automatically simplified, the technical embellishments are reduced, or the playback speed is reduced. S603. When the performance accuracy in step S601 is higher than the second threshold, the performance challenge of subsequent measures is automatically increased.

10. A natural language-driven song generation and intelligent instrument guidance system, characterized in that: include The multi-dimensional music intent analysis module is used to extract tonality, tempo, style, paragraph structure, orchestration requirements, emotional tendencies and special instructions from the natural language description text input by the user. It includes a music knowledge graph reasoning unit to perform implicit constraint completion when the input information is incomplete, and a conflict resolution unit to detect and resolve contradictory instructions. The collaborative generation engine module includes a paragraph structure generator, a chord progression generator, and a melody line generator. These three components are coupled together through a cross-attention network and include a loop correction unit for detecting and eliminating harmonic conflicts between melody and chords. The Unified Music Score Compiler Module is used to encode the output of the Cooperative Generation Engine Module into a unified music intermediate representation and compile the output to include at least multiple target formats such as reference scores, MIDI files, and smart instrument LED light control sequences; The intelligent musical instrument adaptation module includes a fingering optimization solver for solving the optimal fingering using a dynamic programming algorithm with position continuity constraints; and an LED timeline renderer for generating an LED lighting control sequence with a pre-brightening guidance mechanism based on the optimal fingering.

Images