Violin playing auxiliary system based on artificial intelligence

Abstract

The invention relates to the technical field of music data processing, in particular to a violin playing auxiliary system based on artificial intelligence, and the system comprises an audio collection module which employs a microphone array to capture a violin playing sound field, and generates an original audio stream. A coding adaptation module generates a transmission code stream while extracting side information metadata including a noise masking threshold and a transient signal flag. The feature extraction module outputs a high-dimensional feature vector. The reconstruction module generates reconstructed audio. And the evaluation feedback module performs quantitative evaluation on the reconstructed audio, generates scores of intonation deviation degree, string rubbing depth and overtone purity, feeds the scores back to the coding adaptation module, and dynamically adjusts frequency weighting parameters of the psychological acoustic model. The music interface module outputs reconstructed audio and structured evaluation data. The system drives the coding adaptation module to adaptively optimize a compression strategy through score feedback of the evaluation feedback module, and the reconstruction module optimizes a generation process by using historical evaluation data to form a closed-loop data stream.

Description

Technical Field

[0001] This invention relates to the field of music data processing technology, and in particular to an artificial intelligence-based violin playing assistance system. Background Technology

[0002] The psychoacoustic model based on universal audio coding simulates the perceptual characteristics of the human auditory system, such as the masking effect and critical band, and prioritizes the preservation of auditory-sensitive information in audio signal processing, thereby optimizing data representation. In the violin performance assistance scenario, the model analyzes the spectral characteristics of the performance audio, identifies pitch accuracy, timbre purity, and dynamic range changes based on psychoacoustic principles, and then infers the performance quality and provides real-time guidance. This reasoning mechanism based on auditory perception enables the assistance tool to better meet the needs of musical expression.

[0003] Existing compression technologies based on general audio coding psychoacoustic models suffer from the following technical challenges: When achieving high compression ratios, the model, relying on the masking effect of human hearing, classifies some high-frequency components as redundant information and reduces them, resulting in excessive smoothing of the extremely high-order harmonic structures excited by the friction of the violin bow strings. In real-time performance assistance scenarios, when a learner attempts to play artificial harmonics, the high-frequency harmonic series generated by their precise finger touch and bow pressure becomes incomplete due to information loss during the encoding process, making it impossible for the system to accurately identify and reflect the timbre quality of the performance. In digital music production, the lack of detail in recorded violin tremolos makes it difficult to reproduce the full emotion expressed by the performer through subtle vibrato in subsequent mixing stages, severely affecting the realism and expressiveness of the music production. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an artificial intelligence-based violin playing assistance system. This system solves the technical problem that the loss of accuracy in real-time auxiliary analysis and digital music production of subtle violin playing techniques (such as vibrato and overtones) is caused by the excessive smoothing of high-frequency overtone details in redundancy reduction by the general audio coding psychoacoustic model.

[0005] To solve the above-mentioned technical problems, the specific contents of the present invention are as follows: This invention relates to an artificial intelligence-based violin playing assistance system, comprising an audio acquisition module, an encoding adaptation module, a feature extraction module, a reconstruction module, an evaluation feedback module, and a music interface module; The audio acquisition module uses a microphone array to capture the sound field of a violin performance and generate a raw audio stream; The encoding adaptation module receives the raw audio stream and the score from the evaluation feedback module, performs psychoacoustic model compression to generate the transmission bitstream, and extracts side information metadata, which includes noise masking threshold and transient signal markers. The feature extraction module receives the transmitted code stream and side information metadata, performs time series modeling through a neural network, and outputs feature vectors, which include harmonic signal-to-noise ratio fluctuations and formant drift modes. The reconstruction module uses the feature vector as a condition and combines the masking threshold in the side information metadata to perform frequency domain detail reconstruction and generate reconstructed audio. The evaluation feedback module performs a quantitative evaluation of the reconstructed audio, generates score data for pitch deviation, vibrato depth, and overtone purity, stores the score data to form historical score data, and sends the score data to the encoding adaptation module. The encoding adaptation module adjusts the frequency weighting parameters of the psychoacoustic model based on the scoring data; The music interface module receives the reconstructed audio and the scoring data from the evaluation feedback module, and outputs the reconstructed audio and the scoring data. The reconstruction module uses historical scoring data from the evaluation feedback module to optimize audio generation.

[0006] Furthermore, in the artificial intelligence-based violin playing assistance system described in this invention, the audio acquisition module adopts a MEMS microphone array, which enhances the violin sound source through beamforming technology and generates a pulse code modulation stream; The encoding adaptation module receives the pulse code modulation stream and executes the main compression link and the auxiliary analysis link in parallel. The main compression link adopts a psychoacoustic model with an equivalent rectangular bandwidth scale, while the auxiliary analysis link extracts frequency domain features through constant Q transform and marks transient components.

[0007] Furthermore, in the artificial intelligence-based violin playing assistance system of the present invention, the feature extraction module adopts a gated recurrent unit network, receives the transmitted bitstream and side information metadata, and outputs a feature vector through a self-attention mechanism. The feature vector includes a timestamp and 256-dimensional data. The generator of the reconstruction module adopts a U-Net structure and embeds a feature filtering mechanism at the skip connections. It takes feature vectors and side information metadata as input to generate reconstructed audio.

[0008] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: The evaluation feedback module integrates a multi-task classifier, which analyzes pitch trajectory smoothness, vibrato modulation depth, and spectral centroid offset. The multi-task classifier compares the analysis results with the standard performance template, generates frequency correction coefficients, and encapsulates the frequency correction coefficients as PID control instructions. The encoding adaptation module receives PID control commands through the metadata bus and adjusts the critical frequency band division scheme and masking threshold parameters of the psychoacoustic model. The evaluation feedback module outputs PID control commands when the overtone reconstruction quality deteriorates, thereby reducing the masking threshold in the high-frequency region.

[0009] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: The discriminator in the reconstruction module calculates the Wasserstein distance between the reconstructed audio and features of historical high-quality performance clips; The discriminator generates a gradient signal based on the Wasserstein distance, and the gradient signal optimizes the generator through backpropagation. The feature extraction module receives the intermediate feature vectors output by the reconstruction module and updates the attention weights of the gated recurrent unit network. When the historical score queue of the evaluation feedback module shows that the reconstruction quality of a specific musical phrase is poor, the feature extraction module upsamples the low-frequency components in the transmitted bitstream.

[0010] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: The music interface module adopts a dual-channel output architecture. The audio stream channel transmits the reconstructed pulse code modulation data, while the metadata channel encapsulates the structured analysis results generated by the evaluation module. The music interface module detects the connection to the digital audio workstation, enables the timbre mapping function, and converts the evaluation indicators into MIDI control signals; The evaluation feedback module establishes a historical quality score queue, and when the score is lower than a preset threshold, it triggers the fine-tuning mode of the reconstruction module. In fine-tuning mode, the reconstruction module increases the depth of the generative network, and the feature extraction module improves the resolution of time-frequency analysis.

[0011] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: During continuous performance, the encoding adaptation module switches the critical frequency band division scheme based on the frequency correction coefficient transmitted in real time by the evaluation feedback module. The feature extraction module parses the time-frequency masking map in the side information metadata and adjusts the focus area of ​​the neural network attention mask based on the reconstruction quality score of the evaluation feedback module. When processing vibrato phrases, the feature extraction module monitors the signal-to-noise ratio changes of harmonics near the fundamental frequency and shares the monitoring results with the reconstruction module through the metadata bus.

[0012] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: The reconstruction module adopts a progressive course learning strategy during the training phase, and trains the generator according to the difficulty level of high-frequency reconstruction. The evaluation feedback module transmits the quantitative score to the encoding adaptation module in real time via the metadata bus to correct the frequency linear weighting parameters of the psychoacoustic model. The encoding adaptation module recalculates the masking threshold of each critical frequency band based on the corrected weighting parameters, forming a dynamic encoding optimization loop based on performance quality.

[0013] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: During the training phase, the reconstruction module trains the generator according to the difficulty level of high-frequency reconstruction. The evaluation feedback module transmits the quantitative score to the encoding adaptation module via the metadata bus to correct the frequency weighting parameters of the psychoacoustic model; The encoding adaptation module recalculates the masking threshold of the critical frequency band based on the corrected weighting parameters, and the quantization score, weighting parameters, and masking threshold form a data loop.

[0014] Furthermore, the artificial intelligence-based violin playing assistance system of the present invention also includes: The evaluation feedback module sends control signals to the encoding adaptation module through the metadata bus, and the encoding adaptation module adjusts the frequency band division strategy according to the control signals; The reconstruction module sends intermediate feature vectors to the feature extraction module, and the feature extraction module updates the weight parameters of the gated recurrent unit network based on the intermediate feature vectors. When processing artificial harmonic performances, the evaluation feedback module, encoding adaptation module, feature extraction module, and reconstruction module exchange data multiple times through the metadata bus to optimize the audio generation effect.

[0015] Beneficial effects of this invention; This invention captures the sound field of a violin performance using an ultra-high frequency sampling rate through an audio acquisition module, preserving complete acoustic information, including extremely high harmonics, from the source. The encoding and adaptation module extracts side information metadata while generating the transmission bitstream through a parallel processing link, recording noise masking thresholds and transient signal markers to ensure that high-frequency details are not completely lost during compression. The feature extraction module analyzes harmonic signal-to-noise ratio fluctuations and formant drift patterns using a gated recurrent unit network and a self-attention mechanism, outputting high-dimensional feature vectors to capture the dynamic characteristics of performance techniques. The reconstruction module reconstructs frequency domain details using the high-dimensional feature vectors in conjunction with the masking thresholds in the side information metadata, where the generator employs a U-N... The et structure prioritizes the recovery of suppressed high-frequency components; the evaluation feedback module generates a score for the quantitative evaluation of the reconstructed audio and feeds it back to the encoding adaptation module, dynamically adjusting the frequency weighting parameters and masking threshold of the psychoacoustic model to form a forward optimization mechanism; the music interface module outputs high-fidelity audio and structured evaluation data, supporting real-time parameter adjustment in digital music production; through closed-loop data flow and cross-module collaboration, the system iteratively approximates the original acoustic characteristics when continuously processing techniques such as artificial overtones, thereby effectively compensating for the high-frequency attenuation defects of the psychoacoustic model, improving the accuracy of high-frequency overtone reconstruction, and ensuring that the subtle techniques of violin playing maintain accuracy and expressiveness in real-time auxiliary analysis and digital music production. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0017] Figure 1 This is a system architecture diagram of the artificial intelligence-based violin playing assistance system of the present invention. Detailed Implementation

[0018] To make the technical solution of the present invention clearer, the present invention will be clearly and completely described below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The present invention provided by various embodiments will be described in detail below with reference to the accompanying drawings. To better understand the purpose of the present invention, the present invention will be described in further detail below.

[0019] Please see Figure 1The violin playing assistance system based on artificial intelligence provided by the present invention includes an audio acquisition module, an encoding adaptation module, a feature extraction module, a reconstruction module, an evaluation feedback module, and a music interface module. The audio acquisition module uses a microphone array to capture the sound field of a violin performance and generate a raw audio stream. The encoding adaptation module receives the original audio stream, performs psychoacoustic model compression to generate a transmission bitstream, and simultaneously extracts side information metadata, which includes noise masking thresholds and transient signal markers. The feature extraction module receives the transmitted code stream and the side information metadata, performs time series modeling through a neural network, and outputs a high-dimensional feature vector, which includes harmonic signal-to-noise ratio fluctuation and formant drift mode. The reconstruction module uses the high-dimensional feature vector as a condition and combines the masking threshold in the side information metadata to perform frequency domain detail reconstruction and generate reconstructed audio. The evaluation feedback module performs a quantitative evaluation of the reconstructed audio, generates scores for pitch deviation, vibrato depth, and overtone purity, and feeds the scores back to the encoding adaptation module to dynamically adjust the frequency weighting parameters of the psychoacoustic model. The music interface module outputs the reconstructed audio and structured evaluation data; The evaluation feedback module's scoring feedback enables the encoding adaptation module to adaptively optimize the compression strategy, the reconstruction module utilizes historical evaluation data to optimize the generation process, and the system improves the high-frequency overtone reconstruction accuracy through closed-loop data flow.

[0020] The audio acquisition module captures the sound field vibrations produced by the violin playing through a microphone array arranged in the performance space. The array uses a beamforming algorithm to spatially locate and enhance the violin sound source, suppressing environmental noise interference. The acquired analog sound wave signal is converted into a digital pulse code modulation stream by a high-precision analog-to-digital converter circuit. This process uses oversampling technology to shift the quantization noise spectrum to the ultra-high frequency range, preserving complete harmonic components for subsequent processing.

[0021] The encoding adaptation module performs parallel dual-link processing on the input pulse-code modulation stream. The main link applies a psychoacoustic model based on the equivalent rectangular bandwidth scale to perform frame-by-frame windowing processing on the audio signal, calculates the masking threshold for each critical frequency band, achieves compression coding conforming to international standards, and generates the transmission bitstream. The auxiliary link extracts the time-frequency features of the signal through multi-resolution constant Q-transform, combines it with a transient detection algorithm to identify the time intervals of overtone bursts, and generates side information metadata data packets including noise masking thresholds and transient markers. The outputs of both links together constitute the data representation of the coding layer.

[0022] The feature extraction module receives the transport bitstream and side information metadata from the encoding adaptation module. The gated recurrent unit network within the module first performs linear prediction coefficient inversion on the transport bitstream to reconstruct the basic spectral profile; simultaneously, it parses the time-frequency masking map in the side information metadata and dynamically adjusts the network's self-attention weight allocation. When processing musical phrases including vibrato techniques, the network focuses on the signal-to-noise ratio variation patterns of harmonics near the fundamental frequency, combining transient labeling information to output a high-dimensional feature vector with temporal context information. This vector characterizes the harmonic dynamics of the performed audio.

[0023] The reconstruction module uses the high-dimensional feature vector generated by the feature extraction module as conditional input, and combines it with the masking threshold information in the side information metadata to perform detailed reconstruction in the frequency domain. The generator within the module adopts an encoder-decoder architecture, introducing a feature filtering mechanism at skip connections to prioritize the reconstruction of high-frequency overtone components suppressed by the psychoacoustic model. The discriminator employs a multi-scale analysis strategy, comparing the reconstructed output with the spectral features of high-quality performance samples, and continuously optimizing the generator's reconstruction accuracy through an adversarial training mechanism.

[0024] The evaluation feedback module performs multi-dimensional quantitative analysis on the reconstructed audio, employing an improved fundamental frequency tracking algorithm to assess pitch deviation, extracting vibrato modulation depth through Hilbert transform, and calculating overtone purity indices using spectral centroid analysis. The evaluation results are quantified as frequency correction coefficients and fed back to the encoding adaptation module via the metadata bus, dynamically adjusting the critical band division scheme and masking threshold parameters of the psychoacoustic model. When the system detects a decrease in high-frequency overtone reconstruction quality, the feedback mechanism automatically lowers the masking threshold for the corresponding frequency band, forming a forward optimization loop.

[0025] The music interface module employs a low-latency audio driver protocol to synchronously transmit reconstructed high-fidelity audio data and structured evaluation results. Internally, the module integrates an adaptive audio rendering engine that dynamically adjusts audio processing parameters based on the sound quality score provided by the evaluation feedback module. When connected to a digital audio workstation, the module automatically maps evaluation metrics to standardized control signals, enabling real-time adjustment of virtual instrument parameters and meeting the precise control requirements for timbre details in digital music production.

[0026] The system achieves deep data interaction between modules through a metadata bus. The quality score generated by the evaluation feedback module directly controls the frequency band allocation strategy of the encoding adaptation module; the intermediate features generated by the reconstruction module are fed back to the feature extraction module to optimize network weights; and the oversampling parameters of the audio acquisition module are dynamically adjusted according to the reconstruction quality. This cross-module collaborative mechanism enables the system to continuously approach the original acoustic characteristics through multiple iterations when processing complex performance techniques such as artificial harmonics, forming an adaptive ability to compensate for the defects of the psychoacoustic model.

[0027] This invention's audio acquisition module employs a microelectromechanical system (MEMS) microphone array distributed in a five-element array configuration. A generalized sidelobe cancellation algorithm is used to achieve directional enhancement of the violin sound source, effectively suppressing ambient noise. The sampling circuit uses a second-order Σ-Δ modulation architecture, combined with oversampling technology to push quantization noise to the high-frequency band, thereby generating a pulse-code modulation stream at an ultra-high frequency sampling rate. After receiving the pulse-code modulation stream, the encoding adaptation module executes the main compression link and the auxiliary analysis link in parallel. The main compression link uses the core algorithm of the G.722.1C standard, but its Barker band mapping is changed to an equivalent rectangular bandwidth scale, which better matches the high-frequency harmonic distribution characteristics of the violin. The auxiliary analysis link calculates the time-frequency spectrum through an improved constant Q-transform, and combines it with the Teager energy operator to detect transient components, generating metadata including a time-frequency masking threshold. The main compression link generates the transmission bitstream, and the auxiliary analysis link generates side information metadata. The two links output synchronously through a shared memory area, achieving data time alignment and providing complete input for subsequent feature extraction.

[0028] The feature extraction module of this invention employs a bidirectional gated recurrent unit network combined with a self-attention mechanism. The network input is the feature sequence of the compressed bitstream after linear predictive coding and cepstral decoding, while simultaneously receiving masking threshold information transmitted via the metadata bus. Temporal envelope features are extracted through convolutional gated recurrent unit layers, while the self-attention layer focuses on the correlation between bow hair friction noise and fundamental frequency harmonics, ultimately outputting a 256-dimensional feature vector with timestamps. The reconstruction module's generator uses a densely connected U-Net structure, introducing a feature filtering mechanism at skip connections. The generator takes high-dimensional feature vectors and side information metadata as input, concatenates the feature vectors and metadata at the bottleneck layer, and calculates frequency band importance weights through the attention mechanism, prioritizing the reconstruction of high-frequency components suppressed by the psychoacoustic model. The generator performs feature dimensionality reduction through the encoder layer and frequency domain detail reconstruction at the decoder layer, achieving a nonlinear mapping from the compressed domain to the high-fidelity domain.

[0029] This invention's evaluation feedback module integrates a multi-task learning model based on the LightGBM framework, simultaneously calculating three evaluation metrics: pitch trajectory smoothness, vibrato modulation depth, and spectral centroid offset. The multi-task classifier compares the analysis results with a standard performance template to generate frequency correction coefficients, which are then encapsulated as control commands. The encoding adaptation module receives these control commands via a metadata bus, dynamically adjusting the critical bandwidth and masking threshold parameters of the psychoacoustic model. The feedback mechanism employs a proportional-integral-derivative controller principle, quantifying the evaluation results into control signals. When the system detects a continuous decline in overtone reconstruction quality, the control command automatically lowers the masking threshold in the high-frequency region, forming a forward optimization mechanism. The output of the evaluation feedback module is transmitted in real-time to the encoding adaptation module via the metadata bus, correcting the frequency linear weighting parameters of the psychoacoustic model and achieving a dynamic encoding optimization loop based on performance quality.

[0030] The discriminator in the reconstruction module of this invention employs a multi-scale discrimination strategy and adds a Wasserstein distance constraint based on gradient penalty. The discriminator input includes both the spectral features of the reconstructed output audio and features of historical high-quality performance segments from the evaluation feedback module. A gradient signal is generated by calculating the Wasserstein distance through comparison. This gradient signal optimizes the generator's reconstruction strategy through backpropagation, driving the generator to adjust network parameters to improve reconstruction accuracy. The feature extraction module receives the intermediate feature vector output from the reconstruction module and dynamically updates the attention weights of the gated recurrent unit network. When the historical scoring queue of the evaluation feedback module shows poor reconstruction quality for a specific musical phrase, the feature extraction module upsamples the low-frequency components in the transmitted bitstream to improve the resolution of time-frequency analysis. The weight update of the feature extraction module synergizes with the optimization process of the reconstruction module, continuously improving the high-frequency overtone reconstruction effect through iterative learning.

[0031] The music interface module of this invention adopts a dual-channel output architecture. The audio stream channel transmits the reconstructed pulse-code modulation data, while the metadata channel encapsulates the structured analysis results generated by the evaluation feedback module. The music interface module detects the connection status of the digital audio workstation through the protocol stack and automatically enables the timbre mapping function, converting the evaluation indicators of pitch deviation, vibrato depth, and overtone purity into MIDI control signals. The evaluation feedback module establishes a rolling updated historical quality score queue and uses a sliding window mechanism to maintain the most recent score data. When multiple consecutive scores in the queue are lower than a preset threshold, the fine-tuning mode of the reconstruction module is triggered. In the fine-tuning mode, the reconstruction module increases the model capacity by increasing the convolutional layer depth of the generator network, the feature extraction module enhances the detail capture capability by improving the resolution of time-frequency analysis, and the music interface module synchronously adjusts the audio rendering parameters, forming an optimized closed loop for specific performance scenarios.

[0032] In this invention, the encoding adaptation module receives frequency correction coefficients transmitted in real time from the evaluation feedback module during continuous performance and dynamically switches the critical frequency band division scheme. The frequency correction coefficients are encoded through control commands, and the encoding adaptation module adjusts the frequency band boundary parameters of the psychoacoustic model after parsing the commands. The feature extraction module parses the time-frequency masking map in the side information metadata and adjusts the focusing area of ​​the neural network attention mask according to the reconstruction quality score. When processing vibrato phrases, the feature extraction module prioritizes monitoring the signal-to-noise ratio changes of harmonics near the fundamental frequency and shares the monitoring results with the reconstruction module through the metadata bus. After receiving the harmonic monitoring results, the reconstruction module adds high-frequency reconstruction weights to the generator attention layer, and the encoding adaptation module simultaneously optimizes the frequency band division strategy to achieve collaborative processing of vibrato techniques.

[0033] This invention's reconstruction module employs a progressive learning strategy during the training phase, training the generator according to the difficulty level of high-frequency reconstruction. Initially, the generator is trained to reconstruct low- and mid-frequency components, gradually increasing the difficulty of high-frequency overtone reconstruction. The evaluation feedback module transmits quantitative scores to the encoding adaptation module in real-time via a metadata bus, correcting the frequency linear weighting parameters of the psychoacoustic model. The encoding adaptation module recalculates the masking thresholds for each critical frequency band based on the corrected weighting parameters, forming a dynamic encoding optimization loop based on performance quality. The generator in the reconstruction module gradually adapts to the high-frequency reconstruction task through adversarial training, the discriminator adjusts its discrimination criteria based on the characteristics of historical high-quality performance segments, and the evaluation feedback module's scoring mechanism iterates synchronously with the training process, improving the overall reconstruction accuracy of the system.

[0034] The sampling circuit of the audio acquisition module of this invention adopts a Sigma-Delta modulation architecture, using oversampling technology to push the quantization noise spectrum to the ultra-high frequency band, preserving the integrity of the violin's very high harmonics. The music interface module integrates an adaptive segmented convolution algorithm, dynamically adjusting reverberation parameters based on the sound quality score output by the evaluation feedback module; the algorithm adaptively selects the convolution kernel length and attenuation coefficient by analyzing the overtone purity score. When the system processes artificial overtones, the music interface module simultaneously outputs the overtone purity score and reconstructed audio. After receiving the score data, the digital audio workstation automatically corrects the virtual instrument's timbre parameters, achieving accurate reproduction of performance details.

[0035] This invention achieves cross-module collaborative evolution through a metadata bus: the output of the evaluation feedback module directly controls the frequency band division strategy of the encoding adaptation module, and the intermediate feature vectors of the reconstruction module are fed back to the feature extraction module to optimize the weight parameters of the gated recurrent unit network. During continuous processing of artificial overtone performances, the system iteratively approaches the original acoustic characteristics; historical scoring data from the evaluation feedback module drives the encoding adaptation module to adjust its frequency band sensitivity; the feature extraction module updates its attention mechanism based on the intermediate features from the reconstruction module; and the reconstruction module utilizes the optimized feature vectors to improve generation quality, forming an adaptive mechanism that compensates for the defects of the psychoacoustic model.

[0036] This invention uses a microphone array in the audio acquisition module to capture the sound field of a violin performance and generates a raw audio stream at an ultra-high frequency sampling rate, thus preserving complete acoustic information, including extremely high harmonics, from the source. After receiving the raw audio stream, the encoding and adaptation module executes the main compression link and the auxiliary analysis link in parallel. The main compression link uses a psychoacoustic model for standard compression to generate the transmission bitstream, while the auxiliary analysis link extracts frequency domain features through constant Q-transform and generates side information metadata, including noise masking thresholds and transient signal markers. This design ensures that crucial high-frequency overtone details are recorded in the metadata during compression rather than being completely discarded. The feature extraction module receives the transmission bitstream and the side information metadata, analyzes harmonic signal-to-noise ratio fluctuations and formant drift modes through a gated recurrent unit network and a self-attention mechanism, and outputs a high-dimensional feature vector, thereby capturing the dynamic characteristics of subtle performance techniques. The reconstruction module uses high-dimensional feature vectors as conditions, combined with masking thresholds from side information metadata, to perform detailed reconstruction in the frequency domain. The generator employs a U-Net structure and embeds a feature filtering mechanism, prioritizing the reconstruction of high-frequency components suppressed by the psychoacoustic model, effectively restoring the acoustic features of techniques such as vibrato and overtones. The evaluation feedback module quantitatively evaluates the reconstructed audio, generating scores for pitch deviation, vibrato depth, and overtone purity. These scores are fed back to the encoding adaptation module via the metadata bus, dynamically adjusting the frequency weighting parameters and masking thresholds of the psychoacoustic model, forming a forward optimization mechanism. When the system detects a decrease in overtone reconstruction quality, the feedback instruction automatically lowers the masking threshold in the high-frequency region to avoid over-smoothing. The music interface module outputs the reconstructed audio and structured evaluation data for use in digital music production. The timbre mapping function converts evaluation metrics into MIDI control signals, adjusting virtual instrument parameters in real time. The system of this invention uses the scoring feedback of the evaluation feedback module to drive the adaptive optimization compression strategy of the encoding adaptation module, and the reconstruction module uses historical evaluation data to optimize the generation process, forming a closed-loop data flow. This allows the system to continuously approach the original acoustic characteristics through multiple iterations when processing complex techniques such as artificial overtones, thereby compensating for the high-frequency attenuation defects of the psychoacoustic model and improving the accuracy of high-frequency overtone reconstruction.

[0037] This invention provides an artificial intelligence-based violin playing assistance system. The invention improves the accuracy of high-frequency overtone reconstruction through modular design, and the specific implementation method is as follows.

[0038] The audio acquisition module uses a microphone array to capture the sound field of a violin performance. A MEMS microphone array is preferred, employing beamforming technology to directionally enhance the violin sound source and suppress environmental noise interference. During acquisition, an ultra-high frequency sampling rate is used to generate a pulse-code modulation stream, preserving complete acoustic information, including extremely high harmonics, from the source. The encoding adaptation module receives the raw audio stream generated by the audio acquisition module and executes the main compression link and auxiliary analysis link in parallel. The main compression link uses a psychoacoustic model with an equivalent rectangular bandwidth scale to perform frame-by-frame windowing processing on the audio signal, calculates the masking threshold for each critical frequency band, and generates a transmission bitstream conforming to international standards. The auxiliary analysis link extracts frequency domain features through constant Q-transform and combines it with a transient detection algorithm to mark transient components, generating side information metadata. This side information metadata includes noise masking thresholds and transient signal markers. This parallel processing design ensures that crucial high-frequency overtone details are recorded in the metadata during compression, rather than being completely discarded.

[0039] The feature extraction module receives the transmission bitstream and side information metadata output by the encoding and adaptation module, and performs temporal modeling through a gated recurrent unit network. The network incorporates a self-attention mechanism, focusing on the correlation between bowstring friction noise and the fundamental frequency, outputting a timestamped feature vector. This feature vector includes harmonic signal-to-noise ratio fluctuations and formant drift patterns to capture the dynamic characteristics of playing techniques. The reconstruction module uses the feature vector generated by the feature extraction module as a condition, combined with a masking threshold from the side information metadata, to perform frequency domain detail reconstruction. The generator in the reconstruction module adopts a U-Net structure, embedding a feature selection mechanism at skip connections to prioritize the reconstruction of high-frequency components suppressed by the psychoacoustic model, generating the reconstructed audio.

[0040] The evaluation feedback module quantitatively assesses the reconstructed audio, generating scores for pitch deviation, vibrato depth, and overtone purity. This module integrates a multi-task classifier to analyze pitch trajectory smoothness, vibrato modulation depth, and spectral centroid shift, comparing the results with a standard performance template to generate frequency correction coefficients. These coefficients are encapsulated as PID control instructions and sent to the encoding adaptation module via the metadata bus. The encoding adaptation module dynamically adjusts the frequency weighting parameters and masking threshold parameters of the psychoacoustic model based on the PID control instructions. For example, when the overtone reconstruction quality deteriorates, the masking threshold in the high-frequency region is reduced, forming a forward optimization mechanism. The evaluation feedback module also stores the scoring data to form historical scoring data for subsequent optimization.

[0041] The music interface module employs a dual-channel output architecture. The audio stream channel transmits the reconstructed pulse-code modulation data, while the metadata channel encapsulates the structured analysis results generated by the evaluation feedback module. When the music interface module detects a digital audio workstation connection, it enables timbre mapping, converting evaluation metrics into MIDI control signals to support real-time parameter adjustment in digital music production. The system achieves cross-module collaboration through a metadata bus. For example, during continuous processing of artificial harmonic performances, the evaluation feedback module, encoding adaptation module, feature extraction module, and reconstruction module exchange data multiple times to optimize the audio generation effect. The reconstruction module adjusts the generation process using historical scoring data from the evaluation feedback module, and the feature extraction module updates the attention weights of the gated recurrent unit network based on the intermediate feature vectors from the reconstruction module, forming a closed-loop data stream.

[0042] The system of this invention, through the above-described embodiments, effectively compensates for the high-frequency attenuation defects of psychoacoustic models, enhancing the expressiveness of subtle violin playing techniques in real-time assisted analysis and digital music production. Data flow between modules is seamless; evaluation feedback drives the adaptive optimization of compression strategies in the encoding and adaptation module; and the reconstruction module utilizes historical data to improve the generation process, achieving high-frequency overtone reconstruction accuracy that meets performance requirements.

[0043] In a real-time performance assistance scenario, when a learner attempts to play artificial harmonics, the system captures the performance sound field using a MEMS microphone array via an audio acquisition module. The microphone array uses beamforming technology to directionally enhance the violin sound source and generates a pulse-code modulation stream at an ultra-high frequency sampling rate, preserving extremely high-order harmonic components at the source. After receiving the original audio stream, the encoding adaptation module executes the main compression link and the auxiliary analysis link in parallel. The main compression link generates the transmission bitstream using a psychoacoustic model with an equivalent rectangular bandwidth scale. The auxiliary analysis link extracts frequency domain features and marks transient components using a constant Q-transform, generating side information metadata including a noise masking threshold and transient signal markers. The feature extraction module receives the transmission bitstream and side information metadata, outputting feature vectors through a gated recurrent unit network and a self-attention mechanism. These feature vectors include harmonic signal-to-noise ratio fluctuations and formant drift patterns to capture the dynamic characteristics of artificial harmonic performance. The reconstruction module uses the feature vectors as conditions, combined with the masking threshold in the side information metadata, to perform frequency domain detail reconstruction. The generator uses a U-Net structure to prioritize the recovery of suppressed high-frequency harmonic components, generating the reconstructed audio. The evaluation feedback module quantitatively assesses the reconstructed audio, generating scores for pitch deviation, vibrato depth, and overtone purity. This score data is then sent to the encoding adaptation module via a metadata bus. When a decrease in overtone reconstruction quality is detected, the evaluation feedback module outputs a PID control command to lower the masking threshold in the high-frequency region, forming a forward optimization mechanism. The music interface module outputs the reconstructed audio and score data, allowing learners to adjust their playing techniques in real time. Through a closed-loop data flow, the system enables the encoding adaptation module to dynamically adjust the frequency weighting parameters of the psychoacoustic model based on the score data, while the reconstruction module utilizes historical score data to optimize audio generation, thereby compensating for the high-frequency attenuation defects in the psychoacoustic model.

[0044] In a digital music production scenario, the second embodiment of this invention uses a system to record violin tremolo and reconstruct the emotional details of the performance. The audio acquisition module generates a pulse-code modulation stream at an ultra-high frequency sampling rate, ensuring complete capture of subtle harmonic information in the tremolo. The auxiliary analysis link of the encoding adaptation module uses a constant Q-transform to mark transient components, and side information metadata records the noise masking threshold of the tremolo segment, providing a foundation for subsequent reconstruction. The feature extraction module analyzes the transmitted bitstream and side information metadata through a self-attention mechanism, outputting a timestamped feature vector. This feature vector includes formant drift patterns to characterize the spectral changes of vibrato techniques. The generator of the reconstruction module adopts a U-Net structure, embedding a feature filtering mechanism at skip connections, and reconstructs high-frequency overtones using the masking threshold to generate high-fidelity audio. The evaluation feedback module integrates a multi-task classifier to analyze pitch trajectory smoothness and tremolo modulation depth, generating frequency correction coefficients and encapsulating them as PID control instructions. The encoding adaptation module receives instructions through the metadata bus, adjusting the critical frequency band division scheme and masking threshold parameters to avoid losing tremolo details. The music interface module employs a dual-channel output architecture: the audio stream channel transmits the reconstructed audio, while the metadata channel encapsulates evaluation metrics. When a digital audio workstation connection is detected, the module activates timbre mapping, converting the evaluation metrics into MIDI control signals to adjust virtual instrument parameters in real time. When processing continuous vibrato phrases, the feature extraction module monitors the signal-to-noise ratio changes of harmonics near the fundamental frequency and shares the monitoring results with the reconstruction module. The historical scoring queue of the evaluation feedback module triggers a fine-tuning mode, the reconstruction module increases the depth of the generative network, and the feature extraction module improves the time-frequency analysis resolution, further enhancing reconstruction accuracy. Through cross-module collaboration via the metadata bus, the system approximates the original acoustic characteristics through multiple iterations, achieving accurate reproduction of the performance's emotional expression.

[0045] The technical features of this invention are explained below: The audio acquisition module uses a microphone array to capture the sound field of a violin performance and generate a raw audio stream. In this invention, the module uses a MEMS microphone array and beamforming technology to directionally enhance the violin sound source, generating a pulse code modulation stream at an ultra-high frequency sampling rate. This preserves complete acoustic information, including extremely high harmonics, from the source, providing high-quality input for subsequent processing.

[0046] The encoding adaptation module receives the original audio stream and the scoring data from the evaluation feedback module, performs psychoacoustic model compression to generate the transmission bitstream, and extracts side information metadata. In this invention, the module executes the main compression link and the auxiliary analysis link in parallel. The main compression link uses a psychoacoustic model with an equivalent rectangular bandwidth scale to achieve standard compression. The auxiliary analysis link extracts frequency domain features and marks transient components through constant Q transform, generating metadata including noise masking threshold and transient signal marking, so that high-frequency overtone details are recorded rather than lost during the compression process.

[0047] The feature extraction module receives the transmitted bitstream and side information metadata, performs time-series modeling through a neural network, and outputs feature vectors. In this invention, the module uses a gated recurrent unit network combined with a self-attention mechanism to analyze harmonic signal-to-noise ratio fluctuations and formant drift modes, and outputs timestamped feature vectors to capture the dynamic spectral characteristics of violin playing techniques such as vibrato and overtones.

[0048] The reconstruction module uses feature vectors as conditions and combines the masking threshold in the side information metadata to perform frequency domain detail reconstruction and generate reconstructed audio. In the invention, the module's generator adopts a U-Net structure and embeds a feature filtering mechanism at the skip connections to prioritize the reconstruction of high-frequency components suppressed by the psychoacoustic model, effectively recovering the acoustic features of vibrato and artificial overtones and improving reconstruction accuracy.

[0049] The evaluation feedback module quantifies the reconstructed audio, generating score data for pitch deviation, vibrato depth, and overtone purity, and feeds the score data back to the encoding adaptation module. In this invention, the module integrates a multi-task classifier to analyze pitch trajectory smoothness, vibrato modulation depth, and spectral centroid offset, generates frequency correction coefficients and encapsulates them as PID control instructions, and dynamically adjusts the frequency weighting parameters and masking threshold of the psychoacoustic model through the metadata bus, forming a closed-loop optimization mechanism based on performance quality.

[0050] The music interface module adopts a dual-channel output architecture. The audio stream channel transmits the reconstructed pulse code modulation data, while the metadata channel encapsulates the structured analysis results generated by the evaluation feedback module. In this invention, the module enables the timbre mapping function when it detects a digital audio workstation connection, converting the pitch deviation, vibrato depth, and overtone purity evaluation indicators into MIDI control signals to achieve real-time parameter adjustment in digital music production and support the synchronous output of high-fidelity audio and evaluation data.

[0051] The microphone array is deployed in the performance space using MEMS technology. It uses beamforming algorithms to spatially locate and enhance the violin sound source and suppress environmental noise interference. In this invention, the array generates a pulse code modulation stream with an ultra-high frequency sampling rate, which enables the capture of complete acoustic information, including extremely high harmonics, from the source, providing a high-quality input foundation for subsequent processing.

[0052] The psychoacoustic model is based on the masking effect and critical band characteristics of the human auditory system. It performs frame-by-frame windowing processing on the audio signal and calculates the masking threshold of each frequency band. In this invention, the coding adaptation module uses a psychoacoustic model with an equivalent rectangular bandwidth scale to achieve standard compression. While generating the transmission bitstream, it optimizes the data representation and avoids excessive smoothing of high-frequency overtone details.

[0053] The side information metadata is generated by the auxiliary analysis link of the encoding adaptation module, including noise masking threshold and transient signal markers. In this invention, the metadata extracts frequency domain features through constant Q transform and combines it with transient detection algorithm to identify overtone burst intervals, providing key time-frequency information for feature extraction and reconstruction modules and compensating for the loss of high-frequency information during compression.

[0054] The noise masking threshold is a core parameter of the psychoacoustic model, representing the level of noise energy that can be masked within each critical frequency band. In this invention, the threshold is transmitted to the reconstruction module via a metadata bus and dynamically adjusted in conjunction with the scoring of the evaluation feedback module. For example, when the overtone reconstruction quality deteriorates, the threshold in the high-frequency region is reduced to achieve priority recovery of high-frequency overtone details.

[0055] Transient signal markers are a component of the side information metadata. They are generated by the auxiliary analysis link of the encoding and adaptation module through constant Q-transform and transient detection algorithm. They are used to identify the time interval of transient components such as overtone bursts in the audio signal. In this invention, the marker information is parsed by the feature extraction module, and the weights of the neural network are adjusted by combining a self-attention mechanism to prioritize the processing of transient features related to performance skills, so that high-frequency details are accurately captured during the reconstruction process.

[0056] The feature vector is the data representation output by the feature extraction module after performing time-series modeling of the transmitted bitstream and side information metadata through a gated recurrent unit network and a self-attention mechanism. It includes harmonic signal-to-noise ratio fluctuations and formant drift modes. In this invention, the vector is used as a conditional input to the reconstruction module and, combined with a masking threshold, guides the reconstruction of frequency domain details to quantitatively characterize the dynamic spectral characteristics of violin playing.

[0057] Harmonic signal-to-noise ratio fluctuation is a core element of the feature vector, reflecting the dynamic ratio change of harmonic components and noise energy in the audio signal. In this invention, the fluctuation pattern is analyzed through the self-attention mechanism of the feature extraction module to identify the influence of bowstring friction noise on the fundamental frequency harmonics, providing a basis for the reconstruction module to recover high-frequency overtones.

[0058] Formant drift pattern is a key feature of feature vectors, describing the time-varying trajectory of formant frequencies in the audio spectrum. In this invention, the pattern captures timbre changes caused by vibrato or tremolo techniques through neural network modeling, enabling the reconstruction module to optimize high-frequency overtone reconstruction based on the drift pattern and improve the emotional fidelity of the performance.

[0059] Frequency domain detail reconstruction is the core operation of the reconstruction module. It uses feature vectors as conditions and combines the masking threshold in the side information metadata to restore and enhance high-frequency components in the frequency domain. In this invention, the reconstruction process uses a generator with a U-Net structure, which prioritizes the reconstruction of overtone details suppressed by the psychoacoustic model through a feature selection mechanism to compensate for compression loss.

[0060] Pitch deviation is one of the scoring data indicators generated by the evaluation feedback module. It is calculated by an improved fundamental frequency tracking algorithm to quantify the deviation between the played pitch and the standard value. In this invention, the deviation data is used to generate frequency correction coefficients, and the psychoacoustic model parameters of the encoding adaptation module are dynamically adjusted by PID control instructions to optimize pitch feedback.

[0061] Vibrato depth is a component of the scoring data in the evaluation feedback module. The amplitude change of the vibrato modulation signal is extracted through Hilbert transform. In this invention, the depth index is compared with the standard performance template to generate a correction instruction, which drives the coding adaptation module to adjust the critical frequency band division and enhance the acoustic expressiveness of the vibrato technique.

[0062] Overtone purity is one of the evaluation metrics of the evaluation feedback module, which uses spectral centroid analysis to calculate the clarity of overtone components. In this invention, the purity score is fed back to the encoding adaptation module through the metadata bus. When a decrease in purity is detected, the high-frequency masking threshold is automatically reduced to avoid loss of overtone details.

[0063] The scoring data is the output of the evaluation feedback module's multi-dimensional quantitative evaluation of the reconstructed audio, including structured indicators such as pitch deviation, vibrato depth, and overtone purity. In this invention, the data is sent to the encoding adaptation module and the music interface module via the metadata bus for real-time adjustment of system parameters and support for interaction with external devices.

[0064] Historical scoring data is a queue formed by the evaluation feedback module through storing rolling updated scoring data, used to record the system optimization process; in the invention, the data is called by the reconstruction module to optimize the audio generation strategy, such as increasing network depth or increasing resolution in fine-tuning mode, forming an adaptive mechanism based on long-term performance.

[0065] Frequency weighting parameters are core configuration parameters of psychoacoustic models, used to adjust the frequency band priority during compression based on the differences in sensitivity of the human auditory system to different frequencies. In this invention, the evaluation feedback module sends the scoring data to the encoding adaptation module through the metadata bus to dynamically adjust the frequency weighting parameters. For example, when the quality of overtone reconstruction decreases, the weight of the high-frequency region is reduced to optimize the preservation of high-frequency overtone details and reconstruction accuracy.

[0066] Pulse code modulation stream is an audio acquisition module that converts analog sound wave signals into digital format encoding representations through analog-to-digital conversion circuits, and uses ultra-high frequency sampling rate to achieve signal integrity. In this invention, the module generates pulse code modulation stream through MEMS microphone array and beamforming technology, preserving the extremely high-order harmonic components in violin playing from the source, providing a high-fidelity data foundation for subsequent encoding and reconstruction.

[0067] The main compression link is the processing path in the encoding adaptation module that performs standard psychoacoustic model compression. It uses an equivalent rectangular bandwidth scale to perform frame-by-frame windowing and masking threshold calculation on the audio signal. In this invention, the main compression link generates a transmission bitstream that conforms to international standards, achieving data compression efficiency while working in conjunction with the auxiliary analysis link to maintain the basic framework of audio quality.

[0068] The auxiliary analysis link is a processing path in the encoding adaptation module that runs parallel to the main compression link. It extracts frequency domain features through constant Q-transform and marks transient components in combination with transient detection algorithms. In this invention, the metadata of the link generation side includes noise masking threshold and transient signal marking, which provides key time-frequency information for the feature extraction module and compensates for high-frequency details that may be lost in the main compression link.

[0069] The equivalent rectangular bandwidth scale is a frequency band division method that simulates the critical frequency band characteristics of human hearing. It divides the audio spectrum into multiple sub-bands with bandwidth proportional to the center frequency. In this invention, the main compression link of the coding adaptation module uses the equivalent rectangular bandwidth scale to calculate the masking threshold, which is more in line with the distribution pattern of high-frequency harmonics of the violin and improves the adaptability of the psychoacoustic model to the violin timbre.

[0070] The constant Q transform is a time-frequency analysis method whose bandwidth resolution increases with frequency, making it particularly suitable for analyzing the harmonic structure in music signals. In this invention, the auxiliary analysis link extracts the spectral characteristics of violin performance audio through the constant Q transform and combines it with the Teager energy operator to detect transient components, thereby enhancing the time-domain localization capability of overtones and vibrato techniques.

[0071] Gated recurrent unit networks are a type of recurrent neural network structure that controls information flow through update gates and reset gates, effectively handling long-term dependencies in time-series data. In this invention, the feature extraction module uses a gated recurrent unit network to perform time-series modeling of the transmitted bitstream and side information metadata, outputting time-stamped feature vectors to capture the dynamic evolution patterns of performance techniques such as vibrato and overtones.

[0072] Self-attention mechanism is a neural network computing technique that dynamically allocates attention weights at different positions in the input sequence through a query-key-value model, focusing on key features. In this invention, the feature extraction module combines the self-attention mechanism with a gated recurrent unit network to prioritize the analysis of the correlation between bowstring friction noise and fundamental harmonics, thereby improving the guidance accuracy for high-frequency overtone reconstruction.

[0073] The U-Net structure is a convolutional neural network with an encoder and decoder architecture. It achieves multi-scale feature fusion through feature concatenation at skip connections. In this invention, the generator of the reconstruction module adopts the U-Net structure. It performs feature dimensionality reduction through the encoder layer and frequency domain detail reconstruction through the decoder layer. Combined with the feature selection mechanism at the skip connections, it prioritizes the recovery of high-frequency overtone components, thereby achieving a nonlinear mapping from the compressed domain to the high-fidelity domain.

[0074] The feature selection mechanism is an attention weight calculation module embedded in the U-Net skip connection, which dynamically adjusts the information transmission path by evaluating the importance of features. In this invention, the mechanism combines the masking threshold in the side information metadata to calculate the frequency band importance weight, strengthens high-frequency overtone related features during feature fusion, suppresses redundant information interference, and improves the reconstruction module's ability to recover the suppressed components of the psychoacoustic model.

[0075] The multi-task classifier is a machine learning model that integrates evaluation feedback modules. It adopts an architecture that uses a shared underlying network and an independent output layer to process multiple related tasks simultaneously. In this invention, the classifier simultaneously analyzes three evaluation indicators: pitch trajectory smoothness, vibrato modulation depth, and spectral centroid offset. It improves feature extraction efficiency through parameter sharing and compares the output results with a standard performance template to generate frequency correction coefficients.

[0076] The critical band division scheme is the core configuration of the psychoacoustic model, which divides the audio spectrum into multiple sub-bands based on the equivalent rectangular bandwidth characteristics of human hearing. In this invention, the coding adaptation module dynamically adjusts the critical band boundary parameters according to the frequency correction coefficients transmitted by the evaluation feedback module. For example, in the overtone reconstruction scenario, it increases the accuracy of the high-frequency region bandwidth division and optimizes the protection strategy for extremely high harmonics.

[0077] The masking threshold parameter is a threshold of noise energy that can be masked within each critical frequency band in a psychoacoustic model, based on the masking effect principle of the auditory system. In this invention, the parameter is dynamically adjusted by receiving PID control commands through a metadata bus. When the system detects a decrease in overtone purity, it automatically lowers the masking threshold in the high-frequency region to prevent high-frequency harmonics generated by the friction of the violin bow strings from being incorrectly identified as redundant information.

[0078] Wasserstein distance is a metric for measuring the difference between two probability distributions and provides a more stable gradient signal in generative adversarial networks. In this invention, the discriminator of the reconstruction module calculates the Wasserstein distance between the reconstructed audio and historical high-quality performance clips, and generates a smooth optimization direction by combining gradient penalty constraints, thereby driving the generator to improve the reconstruction quality of high-frequency overtones.

[0079] The gradient signal is a parameter adjustment direction vector calculated during the backpropagation of the neural network, representing the partial derivative of the loss function with respect to the network weights. In this invention, the gradient signal generated by the discriminator based on the Wasserstein distance is transmitted to the generator through backpropagation to guide the network parameter update strategy, such as strengthening the weight adjustment magnitude of the high-frequency reconstruction path.

[0080] Backpropagation is a core algorithm in neural network training. It uses the chain rule to propagate the output layer error forward layer by layer and calculate the gradient. In this invention, the algorithm is applied to the adversarial training process of the reconstruction module. It backpropagates the gradient signal generated by the discriminator to the entire generator network to achieve end-to-end parameter optimization, especially for adjusting network weights for high-frequency detail reconstruction tasks.

[0081] Attention weights are feature importance scores calculated through query-key value matching in the self-attention mechanism, used to dynamically focus on key information. In this invention, the feature extraction module receives the reconstruction quality score through the metadata bus and dynamically adjusts the self-attention weight allocation of the gated recurrent unit network, for example, increasing the attention weight of the fundamental frequency adjacent harmonic region when processing vibrato phrases.

[0082] Upsampling is a digital signal processing technique that increases the sampling rate of a signal through interpolation algorithms to enhance the resolution of time-frequency analysis. In this invention, when the historical scoring queue of the evaluation feedback module shows that the reconstruction quality of a specific musical phrase is poor, the feature extraction module upsamples the low-frequency components in the transmitted bitstream and combines them with the interpolated high-frequency harmonic information to improve the granularity of time-frequency analysis, providing a more refined feature representation for subsequent reconstruction.

[0083] The dual-channel output architecture is a data output design, in which the audio stream channel transmits the reconstructed pulse code modulation data, and the metadata channel encapsulates the structured analysis results generated by the evaluation module. In this invention, the music interface module adopts this architecture to achieve synchronous output of high-fidelity audio and evaluation data, supporting real-time processing and analysis of digital audio workstations.

[0084] The fine-tuning mode is a system optimization state that is triggered when the historical scoring queue of the evaluation feedback module shows a score lower than a preset threshold. In this invention, the reconstruction module increases the depth of the generative network and the feature extraction module improves the resolution of time-frequency analysis in the fine-tuning mode, so as to improve the reconstruction accuracy for specific performance scenarios.

[0085] The frequency correction coefficient is a parameter generated by the evaluation feedback module through multi-task classifier analysis, used to correct the frequency weighting of the psychoacoustic model. In this invention, the coefficient is encapsulated as a PID control instruction and sent to the encoding adaptation module through the metadata bus to dynamically adjust the critical frequency band division and masking threshold, thereby optimizing the compression strategy.

[0086] The time-frequency masking map is a representation in the side information metadata, describing the masking characteristics of the audio signal in the time and frequency domains. In this invention, the feature extraction module parses the time-frequency masking map and adjusts the focusing area of ​​the neural network attention mask according to the reconstruction quality score of the evaluation feedback module, thereby enhancing the processing of key frequency bands.

[0087] Attention masking is a mechanism in neural networks used to control the allocation of attention weights and focus on specific regions. In this invention, the feature extraction module adjusts the focusing region of the attention mask based on the time-frequency masking map and the reconstruction quality score, prioritizing the processing of high-frequency overtone-related features to improve the ability to reconstruct details.

[0088] The progressive learning strategy is a training method that trains the model according to the difficulty of the task. In this invention, the reconstruction module adopts this strategy during the training phase. In the initial stage, the generator is trained to reconstruct low- and mid-frequency components, and the difficulty of reconstructing high-frequency overtones is gradually increased, so that the generator gradually adapts to complex tasks.

[0089] The Sigma-Delta modulation architecture is an analog-to-digital conversion technique that pushes quantization noise to higher frequencies through oversampling and noise shaping. In this invention, the audio acquisition module uses this architecture to generate a pulse-code modulation stream at an ultra-high frequency sampling rate, preserving the integrity of the violin's very high harmonics.

[0090] The adaptive segmented convolution algorithm is a signal processing technique that dynamically adjusts convolution parameters based on the input. In this invention, the music interface module integrates this algorithm to dynamically adjust reverberation parameters based on the sound quality score from the evaluation feedback module, thereby optimizing the audio rendering effect.

Claims

1. An artificial intelligence-based violin playing assistance system, characterized in that, It includes an audio acquisition module, an encoding adaptation module, a feature extraction module, a reconstruction module, an evaluation feedback module, and a music interface module; The audio acquisition module uses a microphone array to capture the sound field of a violin performance and generate a raw audio stream; The encoding adaptation module receives the raw audio stream and the score from the evaluation feedback module, performs psychoacoustic model compression to generate the transmission bitstream, and extracts side information metadata, which includes noise masking threshold and transient signal markers. The feature extraction module receives the transmitted code stream and side information metadata, performs time series modeling through a neural network, and outputs feature vectors, which include harmonic signal-to-noise ratio fluctuations and formant drift modes. The reconstruction module uses the feature vector as a condition and combines the masking threshold in the side information metadata to perform frequency domain detail reconstruction and generate reconstructed audio. The evaluation feedback module performs a quantitative evaluation of the reconstructed audio, generates score data for pitch deviation, vibrato depth, and overtone purity, stores the score data to form historical score data, and sends the score data to the encoding adaptation module. The encoding adaptation module adjusts the frequency weighting parameters of the psychoacoustic model based on the scoring data; The music interface module receives the reconstructed audio and the scoring data from the evaluation feedback module, and outputs the reconstructed audio and the scoring data. The reconstruction module uses historical scoring data from the evaluation feedback module to optimize audio generation.

2. The violin playing assistance system based on artificial intelligence according to claim 1, characterized in that, The audio acquisition module uses a MEMS microphone array to enhance the violin sound source through beamforming technology and generate a pulse code modulation stream. The encoding adaptation module receives the pulse code modulation stream and executes the main compression link and the auxiliary analysis link in parallel. The main compression link adopts a psychoacoustic model with an equivalent rectangular bandwidth scale, while the auxiliary analysis link extracts frequency domain features through constant Q transform and marks transient components.

3. The artificial intelligence-based violin playing assistance system according to claim 2, characterized in that, The feature extraction module uses a gated recurrent unit network to receive the transmitted bitstream and side information metadata, and outputs a feature vector through a self-attention mechanism. The feature vector includes a timestamp and 256-dimensional data. The generator of the reconstruction module adopts a U-Net structure and embeds a feature filtering mechanism at the skip connections. It takes feature vectors and side information metadata as input to generate reconstructed audio.

4. The artificial intelligence-based violin playing assistance system according to claim 3, characterized in that, Also includes: The evaluation feedback module integrates a multi-task classifier, which analyzes pitch trajectory smoothness, vibrato modulation depth, and spectral centroid offset. The multi-task classifier compares the analysis results with the standard performance template, generates frequency correction coefficients, and encapsulates the frequency correction coefficients as PID control instructions. The encoding adaptation module receives PID control commands through the metadata bus and adjusts the critical frequency band division scheme and masking threshold parameters of the psychoacoustic model. The evaluation feedback module outputs PID control commands when the overtone reconstruction quality deteriorates, thereby reducing the masking threshold in the high-frequency region.

5. The artificial intelligence-based violin playing assistance system according to claim 4, characterized in that, Also includes: The discriminator in the reconstruction module calculates the Wasserstein distance between the reconstructed audio and features of historical high-quality performance clips; The discriminator generates a gradient signal based on the Wasserstein distance, and the gradient signal optimizes the generator through backpropagation. The feature extraction module receives the intermediate feature vectors output by the reconstruction module and updates the attention weights of the gated recurrent unit network. When the historical score queue of the evaluation feedback module shows that the reconstruction quality of a specific musical phrase is poor, the feature extraction module upsamples the low-frequency components in the transmitted bitstream.

6. The violin playing assistance system based on artificial intelligence according to claim 5, characterized in that, Also includes: The music interface module adopts a dual-channel output architecture. The audio stream channel transmits the reconstructed pulse code modulation data, while the metadata channel encapsulates the structured analysis results generated by the evaluation module. The music interface module detects the connection to the digital audio workstation, enables the timbre mapping function, and converts the evaluation indicators into MIDI control signals; The evaluation feedback module establishes a historical quality score queue, and when the score is lower than a preset threshold, it triggers the fine-tuning mode of the reconstruction module. In fine-tuning mode, the reconstruction module increases the depth of the generative network, and the feature extraction module improves the resolution of time-frequency analysis.

7. The artificial intelligence-based violin playing assistance system according to claim 6, characterized in that, Also includes: During continuous performance, the encoding adaptation module switches the critical frequency band division scheme based on the frequency correction coefficient transmitted in real time by the evaluation feedback module. The feature extraction module parses the time-frequency masking map in the side information metadata and adjusts the focus area of ​​the neural network attention mask based on the reconstruction quality score of the evaluation feedback module. When processing vibrato phrases, the feature extraction module monitors the signal-to-noise ratio changes of harmonics near the fundamental frequency and shares the monitoring results with the reconstruction module through the metadata bus.

8. The artificial intelligence-based violin playing assistance system according to claim 7, characterized in that, Also includes: The reconstruction module adopts a progressive course learning strategy during the training phase, and trains the generator according to the difficulty level of high-frequency reconstruction. The evaluation feedback module transmits the quantitative score to the encoding adaptation module in real time via the metadata bus to correct the frequency linear weighting parameters of the psychoacoustic model. The encoding adaptation module recalculates the masking threshold of each critical frequency band based on the corrected weighting parameters, forming a dynamic encoding optimization loop based on performance quality.

9. The violin playing assistance system based on artificial intelligence according to claim 8, characterized in that, Also includes: During the training phase, the reconstruction module trains the generator according to the difficulty level of high-frequency reconstruction. The evaluation feedback module transmits the quantitative score to the encoding adaptation module via the metadata bus to correct the frequency weighting parameters of the psychoacoustic model; The encoding adaptation module recalculates the masking threshold of the critical frequency band based on the corrected weighting parameters, and the quantization score, weighting parameters, and masking threshold form a data loop.

10. The violin playing assistance system based on artificial intelligence according to claim 9, characterized in that, Also includes: The evaluation feedback module sends control signals to the encoding adaptation module through the metadata bus, and the encoding adaptation module adjusts the frequency band division strategy according to the control signals; The reconstruction module sends intermediate feature vectors to the feature extraction module, and the feature extraction module updates the weight parameters of the gated recurrent unit network based on the intermediate feature vectors. When processing artificial harmonic performances, the evaluation feedback module, encoding adaptation module, feature extraction module, and reconstruction module exchange data multiple times through the metadata bus to optimize the audio generation effect.

Images