Music work collaborative AI editing method and system for non-professional users

Abstract

The application relates to a music work collaborative AI editing method and system for non-professional users, and belongs to the technical field of human-computer interaction and audio processing. Technical threshold reduction effect: non-professional users can complete professional-level editing by intuitive circle selection and natural language instructions without complex music theory or software operation; editing precision improvement effect: through multi-level music structure analysis and note level time sequence marking, the paragraph recognition accuracy and note positioning precision are improved; human-computer cooperation efficiency effect: through multi-version generation and instant feedback mechanism, the original hours of adjustment work are reduced to minutes; personalized adaptation effect: after 5 editing conversations, the acceptance rate of the system recommended version is improved, and the preference learning mechanism effectively improves the personalized adaptation ability; music quality guarantee effect: through the music coherence guarantee technology, the editing and original area boundary smoothness reaches 96.3%, and the overall unity of the music is ensured, and there is no obvious artificial processing trace.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and audio processing technology, and in particular to a collaborative artificial intelligence editing method, corresponding device, computer equipment and storage medium for music works for non-professional users. Background Technology

[0002] With the widespread application of artificial intelligence technology in music creation, more and more non-professional users are beginning to try using AI-assisted tools for music creation. However, existing AI music editing tools still have many limitations in practical applications, seriously affecting user experience and editing efficiency.

[0003] The existing technology has the following main problems: 1. High professional threshold: Non-professional users lack the ability to communicate effectively with AI systems using professional terminology, making it difficult to accurately express their intention to modify music clips; 2. Unintuitive interaction: Existing tools mostly use professional parameter adjustment interfaces, requiring users to operate through complex sliders, numerical input boxes and other controls, which cannot achieve an intuitive "point-and-click" editing experience; 3. Inaccurate positioning: Users find it difficult to accurately select and edit specific segments on the audio timeline, especially on mobile devices, where interface limitations and operational precision issues make selecting specific time points or segments even more difficult; 4. Long feedback cycle: There is a significant feedback delay between the user's feedback and the AI-generated result, which usually requires multiple trials to achieve the desired effect, resulting in low editing efficiency; 5. Lack of personalization: The system lacks the ability to learn and adapt to user preferences, and cannot optimize subsequent suggestions based on historical choices, resulting in each edit having to start from scratch. Summary of the Invention

[0004] The purpose of this invention is to provide a collaborative AI editing method and system for music works for non-professional users, in order to solve the technical problem in the prior art that non-professional users have difficulty in accurately optimizing specific sections of AI-generated music using professional terminology.

[0005] Technical solution A collaborative AI editing method for music works aimed at non-professional users includes: Users are categorized into professional users and non-professional users. If the user is a non-professional user, the collaborative function will process the music selected by the non-professional user. The music structure analysis model is used to perform multi-level structural analysis on the input music work, automatically identify the structural sections in the music, extract music features, and establish a mapping relationship between feature vectors and music elements. It provides a visual editing interface on the audio timeline, receiving the area to be edited selected by the user through a circle operation, as well as editing instructions entered through natural language or preset options; Semantic analysis is performed on the natural language commands input by the user to extract key editing operations and music parameters, and subjective descriptions are transformed into precise music parameter adjustment amounts; Based on the set of editing instructions, the selected interval is locally optimized to maintain musical consistency through a music generation model, generating multiple optimized versions that meet the user's intent. At the same time, multiple optimized versions are presented for users to listen to and compare, and users' listening behavior and selection results are recorded. Based on users' choices, their music preferences are learned and the user preference model is updated.

[0006] Furthermore, in the steps of performing multi-level structural analysis on the input musical work using a music structure analysis model, automatically identifying structural sections in the music, extracting musical features, and establishing a mapping relationship between feature vectors and musical elements, By using deep learning algorithms to perform multi-level structural analysis on input musical works, the intro, verse, chorus, interlude, and outro structural sections in the music are automatically identified. Extract low-level audio features and high-level semantic features from musical works, and establish a mapping relationship between feature vectors and musical elements; Perform time sequence analysis at the note level on musical works to generate timestamp data accurate to the start and end times of notes.

[0007] Furthermore, in the step of providing a visual editing interface on the audio timeline, receiving the area to be edited selected by the user through a selection operation, and the editing instructions input through natural language or preset options, The selection operation in the user interaction steps includes selection on the touch screen, sliding the slider to adjust the selection boundary, or clicking the paragraph label to quickly select the entire paragraph. The natural language input allows users to directly input their editing intent, and the preset editing options provide relevant editing presets based on the selected music segment type.

[0008] Furthermore, in the step of performing semantic analysis on the natural language commands input by the user, extracting key editing operations and music parameters, and converting subjective descriptions into precise music parameter adjustment amounts, Semantic analysis and keyword extraction: Perform word segmentation and syntactic analysis on the natural language input by the user to extract keywords related to music editing; Music parameter mapping: Natural language descriptions are transformed into specific changes in music parameters through a mapping table, which is obtained by machine learning from music editing cases and the decision-making process of professional music producers; Context-aware adjustment: Adaptively adjust the parameter mapping results based on the currently selected music segment type and the overall music style.

[0009] Furthermore, in the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval based on the set of editing instructions and using a music generation model to maintain musical consistency, The music generation model employs a conditional variational autoencoder architecture, and the music generation model includes: The encoder part uses a combination of temporal convolutional networks and self-attention mechanisms to extract multi-level features from the input music segments; The conditional injection mechanism injects the user's editing intent as conditional information into the latent space; The decoder section is based on an autoregressive architecture, which generates music waveforms that meet editing requirements based on the latent variables and conditional information of the samples.

[0010] Furthermore, in the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval based on the set of editing instructions and using a music generation model to maintain musical consistency, This also includes changes in musical style and adjustments to the emotional tone of the music: The music style conversion uses an algorithm based on a cycle-consistent adversarial network, which achieves style conversion through spectrogram conversion, content preservation constraints, and cycle consistency constraints. The music emotion adjustment adopts an algorithm based on the Valence-Arousal two-dimensional emotion model, which achieves emotion adjustment through emotion feature extraction, emotion trajectory adjustment and multi-dimensional parameter coordination.

[0011] Furthermore, in the steps of simultaneously presenting multiple optimized versions for users to listen to and compare, recording users' listening behavior and selection results, and learning their music preferences based on user choices to update the user preference model, The multi-version listening interface arranges multiple edited versions in parallel in the form of cards, providing a parallel listening mechanism and A / B comparison function; User behavior tracking and analysis builds multi-dimensional user preference profiles through explicit feedback collection and implicit behavior analysis; The preference model is continuously updated through incremental learning algorithms, including feature weight adjustment, clustering-enhanced recommendation, and context-aware modeling.

[0012] A collaborative AI editing system for music works for non-professional users, implementing the method of any one of claims 1-7, includes: The music structure analysis module is used to perform multi-level structural analysis on input musical works, extract musical features, and establish a music structure database. The user interaction module provides a visual editing interface, which receives the user's range selection operation and editing command input; The intent parsing module performs semantic analysis on the user's natural language commands and translates them into specific music editing operations; The music editing module performs local optimization based on the user's intent and selected range using a music generation model. The trial and learning module provides a comparison of multiple versions, records user preferences, and optimizes subsequent suggestions.

[0013] Furthermore, the music structure analysis module includes: The audio decoding unit is responsible for reading audio files and converting them into a unified internal representation format; The feature extraction unit extracts time-domain features, frequency-domain features, and perceptual features from the audio signal; The structural analysis unit uses a deep learning model to automatically analyze the structure of music. The metadata management unit manages the metadata information of musical works.

[0014] A computer device includes a processor, a memory, an input device, an output device, and an audio interface, wherein the memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-7.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-7.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. Reduced technical barrier: Experiments have shown that the average time for non-professional users to complete music editing tasks using the method described in this invention is reduced from 28 minutes using traditional methods to 3.5 minutes, reducing the learning cost by 87.5%. Users do not need to learn complex music theory or software operation; they can complete professional-level music editing through intuitive selection and natural language commands. 2. Improved editing accuracy: Through multi-level music structure analysis and note-level timing annotation, a 98.2% paragraph recognition accuracy and a 95.7% note positioning accuracy were achieved, solving the technical problem of "low accuracy of user finger swipe operation" in traditional music editing. 3. Human-Computer Collaboration Efficiency: Through multi-version generation and real-time feedback mechanisms, the adjustment work that traditionally takes hours in music editing is reduced to within minutes. User satisfaction tests show that 85% of users are satisfied or very satisfied, with key satisfaction points including "simple and intuitive operation," "accurate understanding of my ideas," and "the generated results sound very professional." 4. Personalized Adaptation Effect: After 5 editing sessions, the acceptance rate of the system's recommended version increased from the initial 42% to 78%, indicating that the preference learning mechanism effectively improved the system's personalized adaptation capability; 5. Music quality assurance effect: Through music continuity assurance technology, the smoothness of the boundary between the edited area and the original area reaches 96.3%, ensuring the overall unity of the music work and eliminating obvious traces of manual processing. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of a collaborative AI editing method for music works for non-professional users provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the construction process of a music structure analysis model in one embodiment of the present invention; Figure 3 This is a schematic diagram of the user interface layout provided in an embodiment of the present invention; Figure 4 This is a detailed flowchart of the user editing intent parsing process in one embodiment of the present invention; Figure 5 This is a flowchart illustrating the workflow of the music editing module generating multiple versions of optimized results in one embodiment of the present invention; Figure 6 This is an interactive schematic diagram of the audio-visual comparison and preference learning process provided in an embodiment of the present invention; Figure 7 This is a module structure diagram of a collaborative AI editing system for music works for non-professional users provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a computer device suitable for implementing the method of the present invention, provided by an embodiment of the present invention. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that the terms "comprising," "including," and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses. Terms such as "first" and "second" in the claims, specification, and accompanying drawings of this application, as well as relational terms, are used merely to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] Reference Appendix Figures 1-8 This is an overall flowchart of a collaborative AI editing method for music works for non-professional users in one embodiment of this application; A collaborative AI editing method for music works aimed at non-professional users includes: Users are categorized into professional users and non-professional users. If the user is a non-professional user, the collaborative function will process the music selected by the non-professional user. The music structure analysis model is used to perform multi-level structural analysis on the input music work, automatically identify the structural sections in the music, extract music features, and establish a mapping relationship between feature vectors and music elements. It provides a visual editing interface on the audio timeline, receiving the area to be edited selected by the user through a circle operation, as well as editing instructions entered through natural language or preset options; Semantic analysis is performed on the natural language commands input by the user to extract key editing operations and music parameters, and subjective descriptions are transformed into precise music parameter adjustment amounts; Based on the set of editing instructions, the selected interval is locally optimized to maintain musical consistency through a music generation model, generating multiple optimized versions that meet the user's intent. At the same time, multiple optimized versions are presented for users to listen to and compare, and users' listening behavior and selection results are recorded. Based on users' choices, their music preferences are learned and the user preference model is updated.

[0023] In this embodiment, in the steps of performing multi-level structural analysis on the input musical work using a music structure analysis model, automatically identifying structural sections in the music, extracting musical features, and establishing a mapping relationship between feature vectors and musical elements, By using deep learning algorithms to perform multi-level structural analysis on input musical works, the intro, verse, chorus, interlude, and outro structural sections in the music are automatically identified. Extract low-level audio features and high-level semantic features from musical works, and establish a mapping relationship between feature vectors and musical elements; Perform time sequence analysis at the note level on musical works to generate timestamp data accurate to the start and end times of notes. Specifically, The music structure analysis model is constructed as follows: First, multi-resolution feature extraction is performed on the input music. Short-time Fourier transform and Mel frequency cepstral coefficients are used to extract the time-frequency features of the music. The sampling rate is 44.1kHz, the frame length is 2048 sampling points, and the frame shift is 512 sampling points. At the same time, a deep learning-based sentiment classification model is used to extract the high-level semantic features of the music. This model achieves a classification accuracy of 82.3% when trained on the EmoMusic dataset. These features together constitute the multi-feature representation of the music. Secondly, music structure segmentation recognition is performed. A transformer model based on a self-attention mechanism is used to analyze the music feature sequence. The model contains 12 attention heads and has a hidden layer dimension of 768. The model is trained on a dataset containing 50,000 labeled music tracks and can recognize common music structure segments such as intro, verse, chorus, interlude, and outro with an accuracy of 92.5%. Next, note-level timing annotation was performed using a hybrid model combining convolutional neural networks and recurrent neural networks. The convolutional layers used one-dimensional convolutions with a kernel size of 3, and the recurrent layers used LSTMs with 128 units. This model was trained on the MAESTRO dataset and achieved an F1 score of 0.893 for note start and end time detection. Finally, a music feature database is constructed, which associates and stores the extracted music features, structural segmentation information, and note timing data to form a complete description of the music structure. This database not only includes traditional lyrics text-timestamp data pairs, but also adds multi-dimensional representations of music elements such as chord progressions, rhythmic patterns, and timbre features.

[0024] In this embodiment, in the step of providing a visual editing interface on the audio timeline, receiving the editing interval selected by the user through a selection operation, and the editing instructions input through natural language or preset options, The selection operation in the user interaction steps includes selection on the touch screen, sliding the slider to adjust the selection boundary, or clicking the paragraph label to quickly select the entire paragraph. The natural language input allows users to directly input their editing intent, and the preset editing options provide relevant editing presets based on the selected music segment type.

[0025] Specifically, The user interface has been specifically optimized for the usage habits of non-professional users, as detailed below: The audio timeline display area uses waveform visualization technology, using different colors to distinguish different structural sections of the music. The chorus is marked in orange, the intro and interlude are marked in blue, and the verse is marked in green. Key musical event points, such as the starting position of lyrics and the entry position of drum beats, are marked on the timeline. The range selection mechanism supports multiple operation methods: users can select continuous areas by circling on the touch screen with a selection accuracy of ±0.5 seconds; they can precisely adjust the selection boundary by sliding the slider with an accuracy of ±0.1 seconds; or they can quickly select the entire paragraph by clicking the paragraph label, with the selected area highlighted and the start and end times displayed in real time. The editing command input area offers two main input methods: one is a natural language input box, which supports Chinese and English input, such as "make this chorus more exciting"; the other is preset editing options, where the system provides relevant editing presets based on the selected music segment type, such as options for the chorus such as "enhance the rhythm", "enrich the harmonic layers", and "increase the emotional intensity". The real-time preview function allows users to listen to the effect of the selected area before making a final edit, ensuring the accuracy of the selection. Near the editing boundary, the system provides a local zoom function with a magnification of 200%, which makes it easy for users to make fine adjustments. In this embodiment, in the step of performing semantic analysis on the natural language commands input by the user, extracting key editing operations and music parameters, and converting subjective descriptions into precise music parameter adjustment amounts, Semantic analysis and keyword extraction: Perform word segmentation and syntactic analysis on the natural language input by the user to extract keywords related to music editing; Music parameter mapping: Natural language descriptions are transformed into specific changes in music parameters through a mapping table, which is obtained by machine learning from music editing cases and the decision-making process of professional music producers; Context-aware adjustment: Adaptively adjust the parameter mapping results based on the currently selected music segment type and the overall music style.

[0026] Specifically, User editing intent parsing is a crucial step in converting ambiguous natural language commands from users into precise music parameters. The intent parsing process in this invention is specifically implemented as follows: Semantic analysis and keyword extraction: First, the natural language input by the user is segmented and analyzed by syntax. A pre-trained model based on BERT is used. This model is further fine-tuned on music text and achieves an accuracy of 89.7% in the task of understanding editing instructions. For example, for the instruction "make this chorus more exciting", the system extracts two key components: "chorus" and "more exciting". Music Parameter Mapping: A mapping table was established between natural language descriptions and music parameters. This mapping table was obtained through machine learning on the decision-making processes of 10,000 music editing cases and 50 professional music producers. The specific mapping relationships are shown in the table below: Natural language description of the corresponding music parameter changes and adjustment range; The tempo is increased by 5-10%, the pitch is raised by a semitone to a whole tone, the rhythm density is increased by 15-25%, brass instruments are added, and the harmonic complexity is increased by 0.2-0.4 based on adaptive adjustment according to music style. The speed is reduced by 5-10%, the pitch is lowered by a semitone, more strings and piano are used, and high-frequency components are reduced. The music style is adaptively adjusted. It adds richer harmonic layers, increases pad parts, incorporates subtle percussion elements, increases spatial reverb by 15-25% while maintaining musical clarity, and has a signal-to-noise ratio > 20dB; The design is more concise, removing some harmonic layers, reducing the number of notes that sound simultaneously, simplifying the rhythmic pattern, and reducing reverb by 10-20% while retaining the core melody and rhythm. Context-aware adjustment: The system adapts the parameter mapping results by combining the currently selected music segment type and the overall music style. For example, the same "more exciting" instruction will increase the speed more when applied to rock music, but less when applied to classical music. Command specification: The mapped music parameter combinations are transformed into a set of specific music editing commands. For example, "make this chorus more exciting" is transformed into the following specific commands: increase the tempo from 120 BPM to 126 BPM; raise the overall melody pitch by a semitone; add snare drum sounds on the second and fourth beats; add brass instrument accents on the strong beats; increase the harmonic complexity from 0.35 to 0.52. In this embodiment, in the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval based on the set of editing instructions and using a music generation model to maintain musical consistency, The music generation model employs a conditional variational autoencoder architecture, and the music generation model includes: The encoder part uses a combination of temporal convolutional networks and self-attention mechanisms to extract multi-level features from the input music segments; The conditional injection mechanism injects the user's editing intent as conditional information into the latent space; The decoder section is based on an autoregressive architecture, which generates music waveforms that meet editing requirements based on the latent variables and conditional information of the samples.

[0027] Specifically, The music editing module adopts a conditional variational autoencoder architecture, which is implemented as follows: The encoder part uses a combination of temporal convolutional networks and self-attention mechanisms. The temporal convolutional network contains 8 residual blocks, each with 64 filters and a kernel size of 3. The self-attention mechanism has 8 attention heads and a hidden dimension of 512. The encoder encodes the input music into a 128-dimensional Gaussian distribution in the latent space. The conditional injection mechanism injects the user's editing intent as conditional information into the latent space. Specifically, the music parameter adjustment amount output by the intent parsing module is mapped into a conditional vector through a fully connected layer, and the sampling process of the latent variables is affected by the conditional affine transformation. The decoder is based on an autoregressive architecture similar to WaveNet, containing 30 dilated convolutional layers with an exponentially increasing dilation factor from 1 to 512. Local consistency constraints are introduced during the decoding process, and adversarial training is used to ensure that the beginning and end of the generated segments are smoothly connected to the original music.

[0028] In this embodiment, in the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval based on the set of editing instructions and using a music generation model to maintain musical consistency, This also includes changes in musical style and adjustments to the emotional tone of the music: The music style conversion uses an algorithm based on a cycle-consistent adversarial network, which achieves style conversion through spectrogram conversion, content preservation constraints, and cycle consistency constraints. The music emotion adjustment adopts an algorithm based on the Valence-Arousal two-dimensional emotion model, and achieves emotion adjustment through emotion feature extraction, emotion trajectory adjustment and multi-dimensional parameter coordination; Specifically, The music style transfer uses an algorithm based on a recurrent adversarial network. The generator adopts a U-Net architecture, which contains 8 layers each for the encoder and decoder. The discriminator uses a 70×70 PatchGAN. During training, L1 loss with a weight of 10, adversarial loss with a weight of 1, and recurrent consistency loss with a weight of 10 are used. Music emotion adjustment adopts an algorithm based on the Valence-Arousal two-dimensional emotion model. Principal component analysis is used to extract Valence and Arousal components from music features, and then partial least squares regression is used to establish the mapping relationship between emotion features and music parameters. The multi-version generation adopts a parameter space sampling strategy. Within the parameter range that meets the user's editing intent, Latin hypercube sampling is performed on 5 key dimensions to generate 4 versions with significant differences. The generation time is controlled within 5 seconds to ensure real-time interaction.

[0029] In this embodiment, the steps of simultaneously presenting multiple optimized versions for users to listen to and compare, recording users' listening behavior and selection results, and learning their music preferences based on user selections and updating the user preference model are included. The multi-version listening interface arranges multiple edited versions in parallel in the form of cards, providing a parallel listening mechanism and A / B comparison function; User behavior tracking and analysis builds multi-dimensional user preference profiles through explicit feedback collection and implicit behavior analysis; The preference model is continuously updated through incremental learning algorithms, including feature weight adjustment, clustering-enhanced recommendation, and context-aware modeling.

[0030] Specifically, The multi-version listening interface arranges multiple edited versions in parallel in the form of cards. Each card displays a version thumbnail, a description of key features, and playback controls. The description of key features includes dimensions such as speed changes, pitch changes, and instrument changes. The AB comparison function allows users to select two versions for focused comparison. During the comparison, the main differences between the two versions in terms of musical features are highlighted. User behavior tracking is conducted through multiple dimensions: explicit feedback includes 1-5 star ratings and final version selection; implicit behavior analysis includes listening time for each version, number of repeated listenings, and skipping behavior during the listening process. For example, repeatedly listening to a specific paragraph indicates that the user is particularly concerned about the editing effect of that paragraph. The preference model is updated using an incremental learning algorithm, updating the model every time user feedback is received. The feature weights are adjusted using gradient descent with a learning rate of 0.01. Clustering-enhanced recommendation uses the k-nearest neighbor algorithm with k=5. Context-aware modeling is achieved through multi-task learning, sharing the underlying feature representation but separating the output layers. Long-term preference evolution detects preference drift through a sliding window with a window size of 20 interactions. The exploration-utilization balance is achieved through an upper confidence bound algorithm with a confidence parameter c=0.1. The optimal version is recommended in 90% of cases, and the exploratory version is recommended in 10% of cases.

[0031] In this embodiment, the music structure analysis module is used to perform multi-level structural analysis on the input music work, extract music features, and establish a music structure database. The user interaction module provides a visual editing interface, which receives the user's range selection operation and editing command input; The intent parsing module performs semantic analysis on the user's natural language commands and translates them into specific music editing operations; The music editing module performs local optimization based on the user's intent and selected range using a music generation model. The trial and learning module provides a comparison of multiple versions, records user preferences, and optimizes subsequent suggestions.

[0032] Specifically, Intro style change Scenario description: The user wants to convert the intro of an electronic song into a jazz style to create a unique auditory experience.

[0033] Implementation process: 1. The user selects the intro section (0:00-0:30) and chooses "Style Change → Jazz" from the preset options.

[0034] 2. The intent parsing module identifies key elements of the jazz style transition: swing rhythm (swing ratio 0.6-0.7), jazz harmony (increasing the proportion of seventh and ninth chords to 40-60%), and acoustic instrument combination (piano, bass, drum kit).

[0035] 3. The music editing module applies a style transfer algorithm to generate three versions: Version A: Traditional jazz style, using a piano trio configuration, swing ratio=0.65 Version B: Blends jazz style with some electronic elements; swing ratio = 0.6 Version C: Light jazz style, emphasizing rhythm rather than harmonic complexity, swing ratio = 0.55 4. After listening to the version, the user selected version B, which lasted 38 seconds, but requested that the saxophone volume be lowered. The user gave the version a 4-star rating.

[0036] 5. Based on this feedback, the system generates adjusted versions B1 (saxophone volume reduced by 30%) and B2 (saxophone volume reduced by 50%, while enhancing the piano part).

[0037] 6. The user ultimately selected version B1, listened for 25 seconds, and gave it a 5-star rating. The system confirmed the editing and updating of the preference profile: the user prefers a blended style (style blending degree 0.8) and is sensitive to saxophone volume (volume sensitivity 0.9).

[0038] Technical Results: The system achieves accurate cross-style conversion, with the style classifier showing a confidence level of 0.91 for the converted jazz style while preserving 85% of the original musical characteristics. The system completes editing within 1 minute and 45 seconds through iterative refinement, compared to 35 minutes using traditional methods.

[0039] In this embodiment, the music structure analysis module includes: The audio decoding unit is responsible for reading audio files and converting them into a unified internal representation format; The feature extraction unit extracts time-domain features, frequency-domain features, and perceptual features from the audio signal; The structural analysis unit uses a deep learning model to automatically analyze the structure of music. The metadata management unit manages the metadata information of musical works.

[0040] Specifically, Device implementation like Figure 7 As shown, the collaborative AI editing device for music works for non-professional users provided by this invention includes the following modules: Music structure analysis module 710 includes: Audio decoding unit 711: Supports MP3, WAV, AAC and other formats, with sampling rates of 44.1kHz, 48kHz and 96kHz, and bit depth of 16 / 24-bit. Feature extraction unit 712: Extracts 128-dimensional MFCC features, 256-dimensional chromatographic features, and 512-dimensional deep learning features. Structural analysis unit 713: Based on the converter model, with 125M parameters and real-time inference speed (<50ms delay). Metadata Management Unit 714: Based on SQLite database, it supports version management and fast retrieval.

[0041] User interaction module 720 includes: Visualization Unit 721: Based on WebGL technology, it supports smooth waveform rendering at 60fps. Gesture recognition unit 722: Supports multi-touch, recognition accuracy ±2 pixels, response time <100ms. Command input unit 723: Supports Chinese and English speech recognition (accuracy rate 95.3%) and text input. Real-time feedback unit 724: audio latency <20ms, visual feedback latency <50ms.

[0042] Intent parsing module 730 includes: Natural Language Processing Unit 731: Based on the BERT model, with 340M parameters, fine-tuned for music instruction understanding tasks. Music semantic understanding unit 732: contains 5,000 music term mapping rules. Instruction conversion unit 733: Outputs a 25-dimensional music parameter vector. Context Integration Unit 734: Based on graph attention network, it integrates music structure and user context.

[0043] The music editing and processing module 740 includes: Algorithm scheduling unit 741: A scheduler based on reinforcement learning with a decision accuracy of 92.8%. Multi-version generation unit 742: Generates 4 versions in parallel, with a generation time of less than 5 seconds. Quality assessment unit 743: Based on multiple music quality indicators, the assessment consistency is >0.85. Continuity guarantee unit 744: based on temporal stretching of phase vocoder, smoothness >95%.

[0044] Module 750 of the trial and learning materials includes: Auditory Management Unit 751: Supports A / B comparison, loop playback, and random playback. Feedback collection unit 752: Collects 15-dimensional user behavior characteristics. Preference modeling unit 753: Based on the factorization machine model, with real-time update frequency. Recommendation optimization unit 754: Based on the multi-armed gambling machine algorithm, the recommendation accuracy is 78.3%.

[0045] Data storage and management module 760 includes: User Profile Database 761: Stores user preference models, with a capacity supporting 10,000 users. Music Model Library 762: Stores 10 music generation models with a total of 2.3B parameters. Case Database 763: Stores 50,000 edited cases, with a retrieval time of <200ms. Version control system 764: Git-based version control that supports branching and merging.

[0046] A computer device includes a processor, a memory, an input device, an output device, and an audio interface, wherein the memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-7. Specifically, Computer equipment implementation Figure 8This is a schematic diagram of a computer device according to an embodiment of the present invention. The computer device includes a processor 810, a memory 820, an input device 830, an output device 840, and an audio interface 850. These components are interconnected and communicate with each other via a system bus 860.

[0047] The processor 810 is an Intel i7-11800H 8-core processor, equipped with an NVIDIA RTX 3060 GPU, providing AI acceleration capabilities. For music generation tasks, TensorRT is used to optimize inference speed, with latency <100ms.

[0048] The memory 820 includes 16GB of DDR4 RAM and a 1TB NVMe SSD. The SSD stores computer programs that, when executed by a processor, implement the methods described in this invention. The RAM is used for temporary storage of audio data and intermediate features.

[0049] The input device 830 features a 13-inch capacitive touchscreen supporting 10-point multi-touch and 1024 levels of pressure sensitivity. It also includes a professional audio interface, supports ASIO drivers, and has an input latency of <10ms.

[0050] The output device 840 is a 14-inch 4K display covering 100% of the sRGB color gamut. Audio output supports 24-bit / 192kHz, with a signal-to-noise ratio >110dB and THD+N <0.0005%.

[0051] The audio interface 850 is a USB-C audio interface that supports 32-bit / 384kHz sampling, has a dynamic range of 120dB, and is equipped with Phantom Power.

[0052] The system architecture adopts a cloud-based collaborative model: simple interactions and rendering are processed locally, while complex music generation and style transfer are executed on the cloud server. The cloud server is configured with dual Xeon Gold 6248 processors, 4 NVIDIA A100 GPUs, and 80GB of memory. Network latency requirements are <50ms and bandwidth >100Mbps.

[0053] In this embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0054] in, Computer-readable storage medium implementation The present invention also provides a computer-readable storage medium, a 1TB NVMe SSD, on which a computer program is stored, which, when executed by a processor, implements the method described in the present invention. The storage medium uses the EXT4 file system and supports fast random read and write operations.

[0055] Computer programs consist of multiple code modules: Music structure analysis module: 15,000 lines of code, based on the PyTorch framework. User interaction module: 8,000 lines of code, based on React and Web Audio API. Intent parsing module: 6,000 lines of code, based on Hugging Face Transformers. Music editing module: 25,000 lines of code, based on TensorFlow and LibROSA. The trial and learning module consists of 7,000 lines of code, based on Scikit-learn and Redis.

[0056] The program adopts a microservice architecture, with modules communicating via gRPC, supporting horizontal scaling. Containerized deployment uses Docker and Kubernetes, supporting automatic scaling.

[0057] Experimental Data and Performance Evaluation To verify the technical effectiveness of this invention, we conducted a series of experimental evaluations: System performance testing: Under standard testing conditions (Intel i7 processor, 16GB RAM, NVIDIA RTX 3060 graphics card), the response times for various system operations are as follows: Operation type average response time test conditions Musical structure analysis of a 3-minute song (2.1 ± 0.3 seconds, 44.1 kHz) Range selection recognition time: 0.05 ± 0.01 seconds (1080p touchscreen) Natural language command parsing takes 0.3 ± 0.1 seconds to process Chinese commands, which are 5-15 characters long. Multiple versions generate a 4.5±0.8 second 30-second clip, 4 versions. User preference model updates every 0.1 ± 0.02 seconds based on single selection feedback. Editing quality expert evaluation: Twenty professional music producers were invited to conduct a blind review of the system's editing results, using a 5-point scale. Evaluation Dimensions Mean Score Standard Deviation Evaluation Summary Command compliance 4.20.3: The system can accurately understand and implement the user's editing intent. The music quality, edited to 4.00.4, is good in terms of musicality. Technical Quality 4.30.2 The audio processing technology is in place and has no obvious flaws. 4.10.3 The editing area seamlessly connects with the original music. Creativity 3.80.5 and multiple versions offer meaningful creative options. User experience research: Sixty non-professional users were recruited for testing, and the results are as follows: Usability assessment: The System Usability Scale (SUS) score is 82.5±5.2, which is considered "excellent". Editing efficiency comparison: Traditional software takes an average of 28±12 minutes, while this system takes an average of 3.5±1.2 minutes, resulting in an efficiency improvement of 87.5%. User satisfaction: 85% of users expressed satisfaction or high satisfaction, with the main points of satisfaction including "simple and intuitive operation", "able to accurately understand my ideas" and "the generated results sound very professional".

[0058] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A collaborative AI editing method for music works for non-professional users, characterized in that, include: Users are categorized into professional users and non-professional users. If the user is a non-professional user, the collaborative function will process the music selected by the non-professional user. The music structure analysis model is used to perform multi-level structural analysis on the input music work, automatically identify the structural sections in the music, extract music features, and establish a mapping relationship between feature vectors and music elements. It provides a visual editing interface on the audio timeline, receiving the area to be edited selected by the user through a circle operation, as well as editing instructions entered through natural language or preset options; Semantic analysis is performed on the natural language commands input by the user to extract key editing operations and music parameters, and subjective descriptions are transformed into precise music parameter adjustment amounts; Based on the set of editing instructions, the selected interval is locally optimized to maintain musical consistency through a music generation model, generating multiple optimized versions that meet the user's intent. At the same time, multiple optimized versions are presented for users to listen to and compare, and users' listening behavior and selection results are recorded. Based on users' choices, their music preferences are learned and the user preference model is updated.

2. The method according to claim 1, characterized in that, In the steps described above, a multi-level structural analysis of the input musical work is performed using a music structure analysis model, automatically identifying structural sections in the music, extracting musical features, and establishing a mapping relationship between feature vectors and musical elements. By using deep learning algorithms to perform multi-level structural analysis on input musical works, the intro, verse, chorus, interlude, and outro structural sections in the music are automatically identified. Extract low-level audio features and high-level semantic features from musical works, and establish a mapping relationship between feature vectors and musical elements; Perform time sequence analysis at the note level on musical works to generate timestamp data accurate to the start and end times of notes.

3. The method according to claim 1, characterized in that, In the steps of providing a visual editing interface on the audio timeline, receiving the area to be edited selected by the user through a selection operation, and the editing instructions input through natural language or preset options, The selection operation in the user interaction steps includes selection on the touch screen, sliding the slider to adjust the selection boundary, or clicking the paragraph label to quickly select the entire paragraph. The natural language input allows users to directly input their editing intent, and the preset editing options provide relevant editing presets based on the selected music segment type.

4. The method according to claim 1, characterized in that, In the steps of performing semantic analysis on the natural language commands input by the user, extracting key editing operations and music parameters, and converting subjective descriptions into precise music parameter adjustment amounts, Semantic analysis and keyword extraction: Perform word segmentation and syntactic analysis on the natural language input by the user to extract keywords related to music editing; Music parameter mapping: Natural language descriptions are transformed into specific changes in music parameters through a mapping table, which is obtained by machine learning from music editing cases and the decision-making process of professional music producers; Context-aware adjustment: Adaptively adjust the parameter mapping results based on the currently selected music segment type and the overall music style.

5. The method according to claim 1, characterized in that, In the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval while maintaining musical consistency based on the set of editing instructions and a music generation model, The music generation model employs a conditional variational autoencoder architecture, and the music generation model includes: The encoder part uses a combination of temporal convolutional networks and self-attention mechanisms to extract multi-level features from the input music segments; The conditional injection mechanism injects the user's editing intent as conditional information into the latent space; The decoder section is based on an autoregressive architecture, which generates music waveforms that meet editing requirements based on the latent variables and conditional information of the samples.

6. The method according to claim 1, characterized in that, In the step of generating multiple optimized versions that conform to the user's intent by performing local optimization on the selected interval while maintaining musical consistency based on the set of editing instructions and a music generation model, This also includes changes in musical style and adjustments to the emotional tone of the music: The music style conversion uses an algorithm based on a cycle-consistent adversarial network, which achieves style conversion through spectrogram conversion, content preservation constraints, and cycle consistency constraints. The music emotion adjustment adopts an algorithm based on the Valence-Arousal two-dimensional emotion model, which achieves emotion adjustment through emotion feature extraction, emotion trajectory adjustment and multi-dimensional parameter coordination.

7. The method according to claim 1, characterized in that, The process involves simultaneously presenting multiple optimized versions for users to listen to and compare, recording users' listening behavior and selection results, and learning their music preferences based on their choices to update the user preference model. The multi-version listening interface arranges multiple edited versions in parallel in the form of cards, providing a parallel listening mechanism and A / B comparison function; User behavior tracking and analysis builds multi-dimensional user preference profiles through explicit feedback collection and implicit behavior analysis; The preference model is continuously updated through incremental learning algorithms, including feature weight adjustment, clustering-enhanced recommendation, and context-aware modeling.

8. A collaborative AI editing system for music works for non-professional users, implementing the method of any one of claims 1-7, characterized in that, include: The music structure analysis module is used to perform multi-level structural analysis on input musical works, extract musical features, and establish a music structure database. The user interaction module provides a visual editing interface, which receives the user's range selection operation and editing command input; The intent parsing module performs semantic analysis on the user's natural language commands and translates them into specific music editing operations; The music editing module performs local optimization based on the user's intent and selected range using a music generation model. The trial and learning module provides a comparison of multiple versions, records user preferences, and optimizes subsequent suggestions.

9. The system according to claim 8, characterized in that, The music structure analysis module includes: The audio decoding unit is responsible for reading audio files and converting them into a unified internal representation format; The feature extraction unit extracts time-domain features, frequency-domain features, and perceptual features from the audio signal; The structural analysis unit uses a deep learning model to automatically analyze the structure of music. The metadata management unit manages the metadata information of musical works.

10. A computer device, comprising a processor, a memory, an input device, an output device, and an audio interface, characterized in that, The memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images