High-fidelity music reconstruction method and device, equipment and medium

Abstract

The invention relates to the technical field of voice semantics, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a high-fidelity music reconstruction method, device, equipment and medium, and the method comprises the steps: obtaining a to-be-processed music signal, carrying out the signal coding of the to-be-processed music signal, and obtaining a low-dimensional music signal; performing multi-scale convolution on the low-dimensional music signal to obtain a multi-scale music feature; performing multi-head self-attention calculation on the multi-scale music features to obtain attention values of the multi-scale music features; performing feature fusion on the multi-scale music features according to the attention values to obtain fused music features; performing non-autoregression decoding on the fused music features to obtain target acoustic features; and performing waveform reconstruction on the to-be-processed music signal according to the target acoustic feature to obtain a target music signal. According to the invention, the music signal reconstruction quality and reconstruction efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of speech semantics technology, and in particular to a high-fidelity music reconstruction method, device, equipment and medium. Background Art

[0002] With the acceleration of globalization and the popularization of digital technology, the demand for music signal reconstruction and processing in scenarios such as healthcare and financial technology has increased significantly. Traditional music vocoders based on parametric methods (such as LPC, CELP), spectral envelope-based technologies (such as WORLD) or rule-based methods for speech or music signal reconstruction are difficult to effectively restore music signal details and consume high reconstruction resources.

[0003] For example, in healthcare settings, music therapy requires dynamically generating or adjusting therapeutic music (such as soothing alpha wave music or personalized rhythmic music) based on the patient's emotional state. This therapy requires real-time synthesis of high-fidelity music signals to stimulate emotional resonance in patients and assist in psychological treatment. However, traditional neural vocoders, such as HiFi-GAN, lack high-frequency harmonics in the music they generate. This results in a lack of delicate emotional expression and insufficient detail restoration, which weakens the therapeutic effect.

[0004] For example, in the fintech sector, virtual customer service agents need to synthesize emotionally charged speech with background music, such as soothing music and voice guidance, to enhance the user experience. This solution requires maintaining a high-fidelity mixed output of speech and music over low-bandwidth transmission. However, when traditional vocoders optimize the mixed generation for a single signal type (speech or music), spectral conflicts lead to reduced speech clarity or significant music masking, making it impossible to integrate speech or music features.

[0005] Existing technologies mainly use neural network-based vocoders (such as WaveNet, WaveGlow, HiFi-GAN, etc.) to reconstruct music signals. However, most of these neural vocoders are designed for speech signals. When directly used on music signals, they may not be able to fully capture subtle changes in the music, resulting in the reconstructed music lacking artistic appeal and realism. At the same time, many high-fidelity neural vocoders in existing technologies use autoregressive models or deep network structures, which require frame-by-frame prediction during the generation process, resulting in large computational complexity and high inference latency.

[0006] Therefore, how to improve the quality and efficiency of music signal reconstruction has become an urgent problem to be solved. Summary of the Invention

[0007] The present invention provides a high-fidelity music reconstruction method, device, equipment and medium, the main purpose of which is to solve the problems of low music signal reconstruction quality and poor reconstruction efficiency.

[0008] In a first aspect, to achieve the above-mentioned objectives, the present invention provides a high-fidelity music reconstruction method, comprising:

[0009] Acquiring a music signal to be processed, and performing signal encoding on the music signal to be processed to obtain a low-dimensional music signal;

[0010] Performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features;

[0011] Performing multi-head self-attention calculation on the multi-scale music feature to obtain an attention value of the multi-scale music feature;

[0012] Performing feature fusion on the multi-scale music features according to the attention value to obtain a fused music feature;

[0013] performing non-autoregressive decoding on the fused music features to obtain target acoustic features;

[0014] The waveform of the music signal to be processed is reconstructed according to the target acoustic characteristics to obtain a target music signal.

[0015] In a second aspect, the present invention further provides a high-fidelity music reconstruction device, comprising:

[0016] a signal encoding module, configured to obtain a music signal to be processed, and perform signal encoding on the music signal to be processed to obtain a low-dimensional music signal;

[0017] A multi-scale convolution module, configured to perform multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features;

[0018] A multi-head attention calculation module, configured to perform multi-head self-attention calculation on the multi-scale music features to obtain an attention value of the multi-scale music features;

[0019] a feature fusion module, configured to fuse the multi-scale music features according to the attention value to obtain a fused music feature;

[0020] a non-autoregressive decoding module, configured to perform non-autoregressive decoding on the fused music features to obtain target acoustic features;

[0021] The music reconstruction module is used to reconstruct the waveform of the music signal to be processed according to the target acoustic characteristics to obtain the target music signal.

[0022] In a third aspect, the present invention further provides an electronic device, comprising:

[0023] at least one processor; and,

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the high-fidelity music reconstruction method described above.

[0026] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned high-fidelity music reconstruction method.

[0027] In the embodiment of the present invention, the interference of environmental noise, equipment noise, etc. can be effectively eliminated through signal enhancement processing, the signal-to-noise ratio and clarity of the music signal can be improved, and the high-dimensional spectrum features can be mapped to a preset low-dimensional space, which significantly reduces the data complexity; multi-scale convolution is performed on the low-dimensional music signal to separate and extract features of different frequency bands, avoiding the omission of single-scale convolution. At the same time, multi-scale features complement each other, adapt to the diversity of music signals, and reduce inter-layer dependence; the multi-head self-attention mechanism allows parallel attention to different subspaces of music features, avoiding the information omission of a single attention mechanism, significantly enhancing the ability to analyze complex music structures, and improving feature capture capabilities; by guiding the attention value It can adaptively focus on key information in multi-scale features, avoid feature redundancy caused by fixed weight fusion, improve feature utilization, and enhance the global consistency of feature expression; use the acoustic decoder to expand the time-frequency dimensions of features in parallel, quickly generate the initial acoustic structure, and dynamically inject dynamic information such as rhythm and energy through the rhythm fusion mechanism to enhance the natural expressiveness of acoustic features and improve the detail richness and musicality of the target acoustic features; through phase-amplitude decoupling and layered generation strategies, avoid the high computational overhead of traditional point-by-point synthesis, significantly improve the waveform reconstruction speed, map abstract acoustic features into interpretable phase and amplitude parameters, reduce the complexity of music editing, and reduce post-processing requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 A schematic diagram of an application environment of a high-fidelity music reconstruction method according to an embodiment of the present invention;

[0030] Figure 2 A schematic flow chart of a high-fidelity music reconstruction method provided by one embodiment of the present invention;

[0031] Figure 3 A schematic diagram of a process for performing multi-scale convolution on the low-dimensional music signal according to an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of modules of a high-fidelity music reconstruction device provided by one embodiment of the present invention;

[0033] Figure 5 A schematic structural diagram of an electronic device for implementing a high-fidelity music reconstruction method provided by an embodiment of the present invention;

[0034] Figure 6 Another structural diagram of an electronic device for implementing a high-fidelity music reconstruction method provided by an embodiment of the present invention.

[0035] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] An embodiment of the present application provides a high-fidelity music reconstruction method, and the execution subject of the high-fidelity music reconstruction method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the device provided by the embodiment of the present application. In other words, the high-fidelity music reconstruction method can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0039] The present invention provides a high-fidelity music reconstruction method, which can be applied in Figure 1 application environment. Among them, the client communicates with the server through the network. The server can obtain the music signal to be processed through the client, and can effectively eliminate interference such as environmental noise and equipment noise by performing signal enhancement processing on the music signal to be processed, thereby improving the signal-to-noise ratio and clarity of the music signal, and mapping the high-dimensional spectrum features to the preset low-dimensional space, which significantly reduces the data complexity; performing multi-scale convolution on the low-dimensional music signal can separate and extract features of different frequency bands, avoid the omission of single-scale convolution, and at the same time, multi-scale features complement each other, adapt to the diversity of music signals, and reduce inter-layer dependence; the multi-head self-attention mechanism allows for parallel attention to different subspaces of music features, avoids information omission of a single attention mechanism, significantly enhances the ability to parse complex music structures, and improves feature capture capabilities; through the attention value The system guides adaptive focus on key information in multi-scale features, avoids feature redundancy caused by fixed weight fusion, improves feature utilization, and enhances the global consistency of feature expression; uses acoustic decoders to expand the time-frequency dimensions of features in parallel, quickly generates initial acoustic structures, and dynamically injects dynamic information such as rhythm and energy through a rhythmic fusion mechanism to enhance the natural expressiveness of acoustic features and improve the detail richness and musicality of target acoustic features; through phase-amplitude decoupling and layered generation strategies, avoids the high computational overhead of traditional point-by-point synthesis, significantly improves waveform reconstruction speed, maps abstract acoustic features into interpretable phase and amplitude parameters, reduces the complexity of music editing, reduces post-processing requirements, and finally outputs the target music signal back to the client. Among them, the client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0040] Reference Figure 2 FIG. 1 is a flow chart of a high-fidelity music reconstruction method provided by an embodiment of the present invention. In this embodiment, the high-fidelity music reconstruction method includes:

[0041] S1. Obtain a music signal to be processed, and perform signal encoding on the music signal to be processed to obtain a low-dimensional music signal.

[0042] In an embodiment of the present invention, the music signal to be processed refers to the original music data that has not been processed by any specific algorithm or technology, and usually exists in the form of a digital audio file such as WAV, MP3, FLAC, etc. The music signal to be processed contains the physical characteristics of the music such as frequency, amplitude, phase, etc., and is the basic input for audio processing.

[0043] The present invention can directly record music signals through audio acquisition devices such as microphones, voice recorders, etc. to generate digital audio files, or obtain existing music files from public or private audio databases (such as Freesound, professional music libraries) through audio file libraries.

[0044] For example, the music signals to be processed in medical and health scenarios are usually used for music therapy, sleep assistance or mood regulation. The core is to influence the physiological or psychological state of the human body through music signals of specific frequency or rhythm. The present invention includes natural environment sound effects such as the sound of waves, rain, birdsong, etc., which are used to relieve anxiety or improve sleep quality; specific frequency music such as 432Hz "healing frequency" music, or binaural beats audio, which are used to regulate brain wave frequency.

[0045] Specifically, it can be used for music therapy by analyzing the patient's emotional state and generating corresponding music signals to relieve depression or anxiety. It can also be used for sleep assistance by using white noise or music signals of a specific frequency to improve sleep quality.

[0046] For example, the music signals to be processed in the FinTech scenario are usually used for sentiment analysis, user behavior prediction or brand marketing. The core is to combine the feature extraction of music signals with machine learning models to explore user emotions or behavior patterns.

[0047] The present invention can obtain the target user's playback history from the music streaming platform, analyze their music preferences and emotional tendencies, or use it as background music signals for financial product advertisements, embed specific sound effects in the trading platform, and influence user decisions through the rhythm or tone of the music signal.

[0048] Specifically, it can be used for brand marketing by selecting music signals that match the tonality of the financial brand to enhance user memory points, or by adjusting the frequency or rhythm of the trading environment sound effects to reduce user anxiety and improve the trading experience.

[0049] In an embodiment of the present invention, encoding the music signal to be processed to obtain a low-dimensional music signal includes:

[0050] performing signal enhancement processing on the music signal to be processed to obtain an enhanced music signal;

[0051] performing signal framing on the enhanced music signal to obtain a framed music signal;

[0052] Extracting signal features of the framed music signal to obtain signal spectrum features;

[0053] The signal spectrum features are mapped to a preset unified dimensional space to obtain a low-dimensional music signal.

[0054] In an embodiment of the present invention, the signal enhancement processing includes denoising processing, etc., which eliminates noise in the music signal, such as environmental interference and equipment noise, and improves signal quality. The music signal can be converted into the frequency domain through Fourier transform, and the noise spectrum can be subtracted from the signal to improve the signal-to-noise ratio and make the spectrum structure clearer.

[0055] The present invention divides the continuous music signal into short time frames to facilitate subsequent time-frequency analysis. A Hamming window or a Hanning window can be used to reduce spectral leakage. The frame length is set to 20-40ms (corresponding to the short-term stationarity assumption), and the frame shift is 50%-75% of the frame length (to ensure overlap between frames). Frames are divided according to the frame length and frame shift to obtain a framed music signal, where each frame is a discrete sequence in the time domain, preserving local time-frequency characteristics.

[0056] In detail, frequency domain features are extracted from the framed signal and the frequency components of the music signal are quantized. Each frame of the signal can be converted to the frequency domain through short-time Fourier transform (STFT) to obtain a complex spectrum (amplitude + phase), which is used as the signal spectrum feature (such as Mel spectrum matrix).

[0057] The present invention maps high-dimensional spectral features to low-dimensional space, reduces computational complexity and retains key information. It projects the features to the direction with the largest variance through linear transformation, retains the first one or several principal components with the farthest distance, and can compress the features to a low-dimensional latent space through a preset autoencoder while minimizing reconstruction errors.

[0058] In the embodiments of the present invention, signal enhancement processing (such as spectral subtraction and deep learning noise reduction) can be used to effectively eliminate interference such as environmental noise and equipment noise, improve the signal-to-noise ratio and clarity of the music signal, and ensure the accuracy of subsequent analysis. In particular, in low-quality recordings or complex acoustic environments, the signal availability can be significantly improved; high-dimensional spectral features are mapped to a preset low-dimensional space (such as through PCA or autoencoders), which significantly reduces data complexity while retaining key information, thereby improving the real-time performance and efficiency of the system.

[0059] S2. Perform multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features.

[0060] In an embodiment of the present invention, features of different scales are extracted from the low-dimensional music signal according to three different-scale convolution kernels in a preset multi-scale convolutional network, and the extracted features of different scales are aggregated into multi-scale music features.

[0061] like Figure 3 As shown, in the embodiment of the present invention, performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features includes:

[0062] Determine a first convolution kernel, a second convolution kernel, and a third convolution kernel of a preset multi-scale convolutional neural network;

[0063] Extracting high-frequency features from the low-dimensional music signal using the first convolution kernel to obtain high-frequency signal features;

[0064] performing intermediate frequency feature extraction on the low-dimensional music signal according to the second convolution kernel to obtain intermediate frequency signal features;

[0065] Extracting low-frequency features of the low-dimensional music signal according to the third convolution kernel to obtain low-frequency signal features;

[0066] The high-frequency signal features, the intermediate-frequency signal features, and the low-frequency signal features are collected to obtain multi-scale music features.

[0067] In an embodiment of the present invention, high-frequency, medium-frequency and low-frequency features of the music signal are extracted in parallel by convolution kernels of different scales to capture multi-level time-frequency patterns. The first convolution kernel is relatively small in size (such as 3×3 or 1×5), corresponding to high-frequency details (such as percussion instruments and transient signals), and can capture rapidly changing time-frequency components such as drum beats and note starting points; the second convolution kernel is of medium size (such as 5×5 or 3×7), corresponding to medium-frequency energy distribution (such as human voice and main melody), and can be used to extract the main frequency components and harmonic structure of music; the third convolution kernel is relatively large in size (such as 7×7 or 5×15), corresponding to low-frequency trends (such as bass rhythm and background), and can capture slowly changing fundamental frequencies and rhythmic patterns.

[0068] In detail, the first convolution kernel is used to slide on the low-dimensional music signal, and a convolution operation of the local receptive field is performed to obtain high-frequency signal features such as edge detection results; the second convolution kernel is used to extract medium-scale features, balancing local details and global structures. The feature diversity can be improved by increasing the number of convolution kernels (such as 64 channels) to obtain medium-frequency signal features (such as chord progressions and rhythm types); the third convolution kernel is used to capture a wide range of low-frequency patterns and suppress high-frequency noise. The maximum pooling is combined to reduce the resolution, highlight the trend, and obtain low-frequency signal features (such as fundamental frequency trajectory and overall rhythm).

[0069] The present invention splices features of different scales into a comprehensive representation to enhance the model's ability to recognize complex music patterns. Specifically, it can be channel splicing, which splices the high-frequency signal features, the mid-frequency signal features, and the low-frequency signal features along the channel dimension to form multi-scale music features.

[0070] In an embodiment of the present invention, multi-scale convolution is performed on the low-dimensional music signal to separate and extract features of different frequency bands, avoiding omissions caused by single-scale convolution. At the same time, multi-scale features complement each other, adapt to the diversity of music signals, and reduce inter-layer dependence.

[0071] S3. Perform multi-head self-attention calculation on the multi-scale music feature to obtain an attention value of the multi-scale music feature.

[0072] In an embodiment of the present invention, the multi-scale music features are multiplied with the preset query, key, and value weight matrices respectively to generate a query matrix, a key matrix, and a value matrix. The three are combined into an attention matrix, and the attention matrix is divided into multiple subspaces, namely, attention heads. The attention weight is determined by the matching degree between the query and the key, and then the value matrix is weighted and summed to obtain the attention value of each head.

[0073] In an embodiment of the present invention, performing multi-head self-attention calculation on the multi-scale music feature to obtain the attention value of the multi-scale music feature includes:

[0074] Multiplying the multi-scale music features with the preset first query weight, second key weight, and third value weight respectively to obtain a first query matrix, a second key matrix, and a third value matrix;

[0075] Assembling the first query matrix, the second key matrix, and the third value matrix into an attention matrix;

[0076] The pre-built multi-head attention mechanism is used to calculate the attention values corresponding to different attention heads according to the attention matrix.

[0077] In an embodiment of the present invention, the first query weight is used to map the multi-scale music features to the "query" space, representing the information that currently needs attention; the second key weight is used to map the input features to the "key" space, representing the information available for reference; the third value weight is used to map the input features to the "value" space, representing the actual content information carried.

[0078] In detail, the multi-scale music features are multiplied by the first query weight to obtain a query matrix, which represents the music pattern that currently needs to be paid attention to, such as a specific melody or rhythm. The multi-scale music features are multiplied by the second key weight to obtain a key matrix. The function of the key matrix is to provide a set of "key vectors" for matching with the query vector to determine which parts to focus on. The multi-scale music features are multiplied by the third value weight to obtain a value matrix. The value matrix contains the actual feature content, which will be weighted and aggregated later according to the matching results of the query and the key.

[0079] Specifically, the query matrix, key matrix and value matrix are spliced or combined into a three-dimensional attention matrix according to a specific dimension, usually the feature dimension, to provide a unified input for subsequent multi-head attention calculations. The attention matrix is divided into multiple subspaces along a certain dimension (such as the channel dimension), and each subspace is called an "attention head". For example, if 8-head attention is used, the matrix is divided into 8 parts.

[0080] For each attention head, the present invention calculates the similarity (such as dot product) between the query vector and the key vector to obtain an attention score, which represents the matching degree between the query and the key. The attention score is normalized (such as Softmax) and converted into a probability distribution to represent the attention weights of different positions. The value matrix is weighted and summed according to the normalized weights to obtain the attention value of the current head. The outputs of all attention heads are spliced or averaged to obtain the final multi-head attention value, thereby enhancing the expressive ability.

[0081] For example, in the field of medical health, music therapy is widely used in scenarios such as regulating patients' emotions, relieving pain, and improving sleep quality. By analyzing the multi-scale characteristics of music and combining it with the patient's physiological data (such as heart rate, respiratory rate, etc.), the music therapy plan can be personalized to improve the treatment effect.

[0082] The following is a specific example that demonstrates how to perform multi-head self-attention calculations on multi-scale music features to optimize a music therapy recommendation system. This system uses wearable devices or audio acquisition devices to collect audio signals for music therapy in real time, extracting time-domain features such as short-term energy and zero-crossing rate from the audio signals to reflect instantaneous changes in the audio signals.

[0083] The present invention extracts the spectral features of the audio signal (such as Mel spectrum, MFCC, etc.) through methods such as Fourier transform to reflect the frequency distribution of the audio signal. It combines time domain and frequency domain information to extract the time-frequency features of the audio signal (such as wavelet transform coefficients) to capture the frequency changes of the audio signal on different time scales.

[0084] Using the pre-built multi-head attention mechanism, the attention matrix is divided into multiple attention heads. By analyzing the attention values calculated by the multi-head self-attention, the music features that have significant effects on the patient's mood regulation or pain relief are identified. Combined with the patient's real-time physiological data (such as heart rate, respiratory rate, etc.), the attention value is used to dynamically adjust the music therapy plan and recommend the music clips or playlists that best suit the patient's current condition.

[0085] In an embodiment of the present invention, the multi-head self-attention mechanism allows for parallel attention to different subspaces of musical features, such as simultaneously capturing rhythmic patterns and melody trends, avoiding information omissions of a single attention mechanism, significantly enhancing the ability to parse complex musical structures, and improving feature capture capabilities; by calculating the attention value, the weights of different musical features can be dynamically adjusted, for example, highlighting bass features during chord transitions and strengthening treble features during melody climaxes, thereby improving the targeted expression of features and meeting the needs of real-time music generation or interactive systems.

[0086] S4. Perform feature fusion on the multi-scale music features according to the attention value to obtain a fused music feature.

[0087] In an embodiment of the present invention, the attention value reflects the real-time importance of different musical patterns (such as rhythm, melody, and chords), and differentially weights the features of each scale according to its size. By combining the attention value with the multi-scale features, it is possible to highlight the key information in the current context while suppressing irrelevant features, thereby generating a fusion of music features that has both local details and global consistency, thereby improving the ability to express complex musical structures.

[0088] In the embodiment of the present invention, the step of fusing the multi-scale music features according to the attention value to obtain the fused music features includes:

[0089] Obtaining a distribution weight matrix corresponding to the multi-scale music features;

[0090] Calculating the feature residual of the multi-scale music feature according to the assigned weight matrix;

[0091] The feature residuals are weightedly fused according to the attention value to obtain a fused music feature.

[0092] In this embodiment of the present invention, the weighting matrix assigns importance weights to different scales or channels in the multi-scale music feature set, reflecting the contribution of each feature to the fusion process. For example, low-frequency features may be given a higher weight to highlight the melodic foundation, while high-frequency features may be given a lower weight to avoid noise interference.

[0093] Specifically, a weighted processing is performed on the multi-scale music features using an assigned weight matrix to obtain the residual of each feature scale or channel. The feature residual represents the difference between the original feature and the weighted feature, and is used to capture information not directly expressed by the weight matrix. For example, if the actual contribution of a certain scale feature does not match the expected weight, its residual will retain this deviation information to avoid information loss.

[0094] Specifically, the multi-head attention value calculated previously is used as a dynamic adjustment factor, the feature residual is weighted, and the weighted feature residual is fused with the original multi-scale music feature by element-by-element addition or splicing to ensure that the original feature and the dynamically adjusted residual information complement each other. For example, the weighted residual may correct local deviations in the original feature while retaining global structural information, and finally generate a fused music feature with both rich details and overall coordination.

[0095] For example, in the quantitative trading system of FinTech business, it is necessary to integrate financial data features such as price, trading volume, and volatility at different time scales, such as minute, hour, and day, to generate more robust trading signals.

[0096] In detail, for the same financial asset such as stocks, time series features at the minute level (high-frequency fluctuations), hourly level (short-term trends) and daily level (long-term trends) are extracted respectively to form a multi-scale feature matrix. The self-attention mechanism is used to calculate the attention weight of each scale feature to reflect the contribution of different time scales to the current market status. For example, the minute-level weight is higher in a volatile market, and the daily-level weight is higher in a trending market.

[0097] The present invention uses a learnable weight matrix to perform linear transformation on multi-scale features, generates initial distribution weights, and then calculates feature residuals to capture the deviations of each scale feature from the global mean (such as abnormal fluctuations in minute-level data during breaking news). The residuals are weighted and fused according to the attention value, and the fused features are input into the trading strategy model to achieve adaptive adjustment of time scale dependence under different market conditions and improve the strategy winning rate.

[0098] For example, in healthcare scenarios, the results of multi-head self-attention calculations are used to obtain a weight matrix corresponding to multi-scale music features. This weight matrix reflects the importance of features at different scales in the fusion process. Based on this weight matrix, the feature residuals of the multi-scale music features are calculated. Feature residuals represent the differences between features at different scales, helping to preserve more detailed features.

[0099] This method performs a weighted fusion of feature residuals based on attention values. By adjusting the contribution of different feature residuals through attention values, the fused music features focus more on those that significantly alleviate the patient's mood or pain. The resulting fused music feature integrates multi-scale music information and highlights features that are sensitive to the patient's condition.

[0100] In an embodiment of the present invention, by analyzing the fused music features, music clips or styles that have significant effects on patient mood regulation or pain relief are identified, and key information related to the patient's physiological state in the music signal is deeply mined, thereby realizing personalized recommendation and dynamic adjustment of music therapy. This method not only improves the therapeutic effect of music therapy, but also provides new ideas and methods for big data analysis and artificial intelligence applications in the medical and health fields.

[0101] In an embodiment of the present invention, attention values are used to guide adaptive focus on key information in multi-scale features (such as the core of the melody or the strong beat of the rhythm), avoiding feature redundancy caused by fixed weight fusion and improving feature utilization. The fusion process integrates features of different scales (such as time-frequency resolution) and combines the attention mechanism to dynamically balance local details (such as note transitions) and global structures (such as phrase division), thereby enhancing the global consistency of feature expression.

[0102] S5. Perform non-autoregressive decoding on the fused music features to obtain target acoustic features.

[0103] In an embodiment of the present invention, non-autoregressive decoding avoids the point-by-point generation delay of the acoustic decoder by parallel expansion of the time-frequency dimensions, significantly improving the decoding speed. The time-frequency dimension expansion is combined with jump connections to enable the target acoustic features to simultaneously retain high-level semantics (such as melodic structure) and low-level details (such as timbre texture), thereby improving the clarity and realism of the generated audio.

[0104] In the embodiment of the present invention, performing non-autoregressive decoding on the fused music features to obtain target acoustic features includes:

[0105] Expanding the fused music features in time-frequency dimensions according to a preset acoustic decoder to obtain initial acoustic features;

[0106] Performing prosodic fusion on the initial acoustic features to obtain intermediate acoustic features;

[0107] A skip connection mechanism is used to perform feature aggregation on the intermediate acoustic features and the corresponding decoding layer in the acoustic decoder to obtain the target acoustic features.

[0108] In an embodiment of the present invention, the fused music features are processed by a preset acoustic decoder, and their time-frequency dimensions are gradually expanded, so that abstract music features (such as notes and rhythms) are converted into a representation that is closer to the acoustic signal. The time or frequency dimension can be increased through deconvolution or upsampling operations to generate initial acoustic features.

[0109] For example, if the fused feature is a low-dimensional vector, the decoder may expand it into a feature map containing more time steps and frequency points through multi-layer upsampling to simulate the time-frequency structure of the acoustic signal.

[0110] In detail, the rhythmic fusion of the initial acoustic features is usually performed by introducing additional rhythmic information such as fundamental frequency and energy contour dynamic adjustment features to enhance the rhythmic expressiveness of the acoustic features and make them closer to the ups and downs of real music or speech. For example, the rhythmic information is integrated into the initial features through a gating mechanism or weighted summation.

[0111] For example, if the initial features lack emotional expression, prosody fusion may give the features more vivid dynamic changes by adjusting the energy weights at different time steps.

[0112] Specifically, at each layer or specific layer of the acoustic decoder, the intermediate acoustic features are aggregated with the features of the corresponding layer of the decoder through skip connections, which can alleviate the gradient vanishing problem and promote the fusion of multi-scale information. A certain layer of the decoder may receive abstract information from the fused music features, while fusing the shallow time-frequency details through skip connections, and ultimately generate more refined target acoustic features.

[0113] In an embodiment of the present invention, an acoustic decoder is used to expand the time-frequency dimensions of features in parallel to quickly generate an initial acoustic structure; then, a rhythmic fusion mechanism is used to dynamically inject dynamic information such as rhythm and energy to enhance the natural expressiveness of the acoustic features; finally, combined with the jump connection technology, shallow detail features and deep semantic features are aggregated across layers to avoid information loss, significantly improving the detail richness and musicality of the target acoustic features.

[0114] S6. Reconstruct the waveform of the music signal to be processed according to the target acoustic characteristics to obtain a target music signal.

[0115] In the embodiment of the present invention, the phase and amplitude of the target acoustic features are extracted through the feature mapping module, the abstract features are converted into waveform control parameters, the initial waveform is generated using the relevant waveform control parameters, and multiple simple waveforms are superimposed to form an initial composite waveform to quickly construct a signal structure. Finally, through fine-grained adjustments such as spectrum correction and time domain smoothing, the sound quality and expressiveness are optimized and the mechanical feel is eliminated.

[0116] In an embodiment of the present invention, the waveform reconstruction of the music signal to be processed according to the target acoustic features to obtain the target music signal includes:

[0117] Extracting phase and amplitude of the target acoustic features to obtain waveform characteristic parameters;

[0118] generating an initial reconstructed waveform of the music signal to be processed according to the waveform characteristic parameters;

[0119] Fine-grained adjustment is performed on the initial reconstructed waveform to obtain a target music signal.

[0120] In an embodiment of the present invention, the target acoustic features are decomposed to extract key waveform parameters thereof, including phase information and amplitude information. The phase information describes the position offset of the signal on the time axis, and the amplitude information reflects the intensity change of the signal.

[0121] In detail, the abstract acoustic features are converted into operational waveform control parameters. For example, the phase parameter is used to locate the starting moment of the note, and the amplitude parameter is used to adjust the volume dynamics of the note. The target acoustic features are mapped into a continuous value sequence of phase and amplitude through a feature mapping module such as a fully connected layer or a convolutional layer.

[0122] For example, if the target acoustic feature contains a 128-dimensional vector, it may be decoded into a three-dimensional representation of time-phase-amplitude through a specific network structure.

[0123] Specifically, the timing distribution of the signal is determined based on the phase information. In music synthesis, the phase parameter may correspond to the trigger time of the note. The signal strength is adjusted according to the amplitude information. The attack, decay, sustain and release stages of the note are controlled by the envelope generator (such as ADSR envelope). The simple waveforms corresponding to multiple phase-amplitude pairs (such as sine waves and sawtooth waves) are superimposed to form a preliminary composite waveform.

[0124] For example, in chord generation, phase-amplitude pairs of different notes may be superimposed into a polyphonic waveform.

[0125] The present invention makes fine adjustments to the initial reconstructed waveform to eliminate the mechanical feel or distortion of the initial waveform and make it closer to the natural music signal. The spectral distribution of the initial waveform can be analyzed through short-time Fourier transform (STFT) to adjust the harmonic components or suppress noise. For example, if the initial waveform has high-frequency burrs, the spectrum can be smoothed through low-pass filtering.

[0126] The present invention can also avoid sudden changes by interpolating or weighted averaging transient changes in the waveform, for example, using a fade-in and fade-out effect at the connection of notes to reduce "click" sounds.

[0127] In an embodiment of the present invention, a phase-amplitude decoupling and layered generation strategy is used to avoid the high computational overhead of traditional point-by-point synthesis, significantly improve the waveform reconstruction speed, map abstract acoustic features into interpretable phase and amplitude parameters, reduce the complexity of music editing, and reduce post-processing requirements.

[0128] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0129] like Figure 4 FIG. 1 is a functional module diagram of a high-fidelity music reconstruction device provided by an embodiment of the present invention.

[0130] In an embodiment of the present disclosure, a high-fidelity music reconstruction device is provided, and the high-fidelity music reconstruction device corresponds one-to-one to the high-fidelity music reconstruction method in the above embodiment. Figure 4 As shown, the high-fidelity music reconstruction device 100 can be installed in an electronic device. According to the functions to be implemented, the high-fidelity music reconstruction device 100 includes a signal encoding module 101, a multi-scale convolution module 102, a multi-head attention calculation module 103, a feature fusion module 104, a non-autoregressive decoding module 105, and a music reconstruction module 106. The functional modules are described in detail as follows:

[0131] a signal encoding module, configured to obtain a music signal to be processed, and perform signal encoding on the music signal to be processed to obtain a low-dimensional music signal;

[0132] A multi-scale convolution module, configured to perform multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features;

[0133] A multi-head attention calculation module, configured to perform multi-head self-attention calculation on the multi-scale music features to obtain an attention value of the multi-scale music features;

[0134] a feature fusion module, configured to fuse the multi-scale music features according to the attention value to obtain a fused music feature;

[0135] a non-autoregressive decoding module, configured to perform non-autoregressive decoding on the fused music features to obtain target acoustic features;

[0136] The music reconstruction module is used to reconstruct the waveform of the music signal to be processed according to the target acoustic characteristics to obtain a target music signal.

[0137] In one embodiment, when performing audio enhancement processing on the original audio to obtain the audio to be processed, the signal encoding module 101 is configured to:

[0138] Performing sampling rate unification processing on the original audio to obtain unified audio;

[0139] De-noising the unified audio to obtain interference-free audio;

[0140] Normalization is performed on the interference-free audio to obtain audio to be processed.

[0141] In one embodiment, when the signal encoding module 101 performs signal encoding on the music signal to be processed to obtain a low-dimensional music signal, it is configured to:

[0142] performing signal enhancement processing on the music signal to be processed to obtain an enhanced music signal;

[0143] performing signal framing on the enhanced music signal to obtain a framed music signal;

[0144] Extracting signal features of the framed music signal to obtain signal spectrum features;

[0145] The signal spectrum features are mapped to a preset unified dimensional space to obtain a low-dimensional music signal.

[0146] In one embodiment, when performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features, the multi-scale convolution module 102 is configured to:

[0147] Determine a first convolution kernel, a second convolution kernel, and a third convolution kernel of a preset multi-scale convolutional neural network;

[0148] Extracting high-frequency features from the low-dimensional music signal using the first convolution kernel to obtain high-frequency signal features;

[0149] performing intermediate frequency feature extraction on the low-dimensional music signal according to the second convolution kernel to obtain intermediate frequency signal features;

[0150] Extracting low-frequency features of the low-dimensional music signal according to the third convolution kernel to obtain low-frequency signal features;

[0151] The high-frequency signal features, the intermediate-frequency signal features, and the low-frequency signal features are collected to obtain multi-scale music features.

[0152] In one embodiment, when performing multi-head self-attention calculation on the multi-scale music feature to obtain the attention value of the multi-scale music feature, the multi-head attention calculation module 103 is used to:

[0153] Multiplying the multi-scale music features with the preset first query weight, second key weight, and third value weight respectively to obtain a first query matrix, a second key matrix, and a third value matrix;

[0154] Assembling the first query matrix, the second key matrix, and the third value matrix into an attention matrix;

[0155] The pre-built multi-head attention mechanism is used to calculate the attention values corresponding to different attention heads according to the attention matrix.

[0156] In one embodiment, when the feature fusion module 104 performs feature fusion on the multi-scale music features according to the attention value to obtain the fused music features, it is configured to:

[0157] Obtaining a distribution weight matrix corresponding to the multi-scale music features;

[0158] Calculating the feature residual of the multi-scale music feature according to the assigned weight matrix;

[0159] The feature residuals are weightedly fused according to the attention value to obtain a fused music feature.

[0160] In one embodiment, when performing non-autoregressive decoding on the fused music features to obtain the target acoustic features, the non-autoregressive decoding module 105 is configured to:

[0161] Expanding the fused music features in time-frequency dimensions according to a preset acoustic decoder to obtain initial acoustic features;

[0162] Performing prosodic fusion on the initial acoustic features to obtain intermediate acoustic features;

[0163] A skip connection mechanism is used to perform feature aggregation on the intermediate acoustic features and the corresponding decoding layer in the acoustic decoder to obtain the target acoustic features.

[0164] In one embodiment, when the music reconstruction module 106 performs waveform reconstruction on the music signal to be processed according to the target acoustic features to obtain the target music signal, it is configured to:

[0165] Extracting phase and amplitude of the target acoustic features to obtain waveform characteristic parameters;

[0166] generating an initial reconstructed waveform of the music signal to be processed according to the waveform characteristic parameters;

[0167] Fine-grained adjustment is performed on the initial reconstructed waveform to obtain a target music signal.

[0168] In the present invention, the specific definition of a high-fidelity music reconstruction device can be found in the definition of a high-fidelity music reconstruction method described above and will not be repeated here. Each module in the high-fidelity music reconstruction device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0169] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a high-fidelity music reconstruction method.

[0170] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a high-fidelity music reconstruction method.

[0171] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0172] Acquiring a music signal to be processed, and performing signal encoding on the music signal to be processed to obtain a low-dimensional music signal;

[0173] Performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features;

[0174] Performing multi-head self-attention calculation on the multi-scale music feature to obtain an attention value of the multi-scale music feature;

[0175] Performing feature fusion on the multi-scale music features according to the attention value to obtain a fused music feature;

[0176] performing non-autoregressive decoding on the fused music features to obtain target acoustic features;

[0177] The waveform of the music signal to be processed is reconstructed according to the target acoustic characteristics to obtain a target music signal.

[0178] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and actual implementation may employ other division methods.

[0179] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0180] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0181] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0182] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can implement:

[0183] Acquiring a music signal to be processed, and performing signal encoding on the music signal to be processed to obtain a low-dimensional music signal;

[0184] Performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features;

[0185] Performing multi-head self-attention calculation on the multi-scale music feature to obtain an attention value of the multi-scale music feature;

[0186] Performing feature fusion on the multi-scale music features according to the attention value to obtain a fused music feature;

[0187] performing non-autoregressive decoding on the fused music features to obtain target acoustic features;

[0188] The waveform of the music signal to be processed is reconstructed according to the target acoustic characteristics to obtain a target music signal.

[0189] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0190] The computer-readable storage medium may also store at least one computer-executable program / instruction, such as a computer-readable instruction. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0191] In addition, the computer device may also include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.).

[0192] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0193] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-described methods. In particular, any reference to memory, storage, database, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory.

[0194] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0195] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented with a dedicated hardware-based system that performs the specified function or action, or may be implemented with a combination of dedicated hardware and computer instructions.

[0196] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

[0197] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

Claims

1. A high-fidelity music reconstruction method, characterized in that: The method comprises: Acquiring a music signal to be processed, and performing signal encoding on the music signal to be processed to obtain a low-dimensional music signal; Performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features; Performing multi-head self-attention calculation on the multi-scale music feature to obtain an attention value of the multi-scale music feature; Performing feature fusion on the multi-scale music features according to the attention value to obtain a fused music feature; performing non-autoregressive decoding on the fused music features to obtain target acoustic features; The waveform of the music signal to be processed is reconstructed according to the target acoustic characteristics to obtain a target music signal.

2. The high-fidelity music reconstruction method according to claim 1, wherein: The step of encoding the music signal to be processed to obtain a low-dimensional music signal comprises: performing signal enhancement processing on the music signal to be processed to obtain an enhanced music signal; performing signal framing on the enhanced music signal to obtain a framed music signal; Extracting signal features of the framed music signal to obtain signal spectrum features; The signal spectrum features are mapped to a preset unified dimensional space to obtain a low-dimensional music signal.

3. The high-fidelity music reconstruction method according to claim 1, wherein: The performing multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features includes: Determine a first convolution kernel, a second convolution kernel, and a third convolution kernel of a preset multi-scale convolutional neural network; Extracting high-frequency features from the low-dimensional music signal using the first convolution kernel to obtain high-frequency signal features; performing intermediate frequency feature extraction on the low-dimensional music signal according to the second convolution kernel to obtain intermediate frequency signal features; Extracting low-frequency features of the low-dimensional music signal according to the third convolution kernel to obtain low-frequency signal features; The high-frequency signal features, the intermediate-frequency signal features, and the low-frequency signal features are collected to obtain multi-scale music features.

4. The high-fidelity music reconstruction method according to claim 1, wherein: The performing multi-head self-attention calculation on the multi-scale music feature to obtain the attention value of the multi-scale music feature includes: Multiplying the multi-scale music features with the preset first query weight, second key weight, and third value weight respectively to obtain a first query matrix, a second key matrix, and a third value matrix; Assembling the first query matrix, the second key matrix, and the third value matrix into an attention matrix; The pre-built multi-head attention mechanism is used to calculate the attention values corresponding to different attention heads according to the attention matrix.

5. The high-fidelity music reconstruction method according to claim 1, wherein: The step of fusing the multi-scale music features according to the attention value to obtain a fused music feature includes: Obtaining a distribution weight matrix corresponding to the multi-scale music features; Calculating the feature residual of the multi-scale music feature according to the assigned weight matrix; The feature residuals are weightedly fused according to the attention value to obtain a fused music feature.

6. The high-fidelity music reconstruction method according to claim 1, wherein: The performing non-autoregressive decoding on the fused music features to obtain target acoustic features includes: Expanding the fused music features in time-frequency dimensions according to a preset acoustic decoder to obtain initial acoustic features; Performing prosodic fusion on the initial acoustic features to obtain intermediate acoustic features; A skip connection mechanism is used to perform feature aggregation on the intermediate acoustic features and the corresponding decoding layer in the acoustic decoder to obtain the target acoustic features.

7. The high-fidelity music reconstruction method according to claim 1, wherein: The waveform reconstruction of the music signal to be processed according to the target acoustic feature to obtain the target music signal includes: Extracting phase and amplitude of the target acoustic features to obtain waveform characteristic parameters; generating an initial reconstructed waveform of the music signal to be processed according to the waveform characteristic parameters; Fine-grained adjustment is performed on the initial reconstructed waveform to obtain a target music signal.

8. A high-fidelity music reconstruction device, characterized in that: The device comprises: a signal encoding module, configured to obtain a music signal to be processed, and perform signal encoding on the music signal to be processed to obtain a low-dimensional music signal; A multi-scale convolution module, configured to perform multi-scale convolution on the low-dimensional music signal to obtain multi-scale music features; A multi-head attention calculation module, configured to perform multi-head self-attention calculation on the multi-scale music features to obtain an attention value of the multi-scale music features; a feature fusion module, configured to fuse the multi-scale music features according to the attention value to obtain a fused music feature; a non-autoregressive decoding module, configured to perform non-autoregressive decoding on the fused music features to obtain target acoustic features; The music reconstruction module is used to reconstruct the waveform of the music signal to be processed according to the target acoustic characteristics to obtain the target music signal.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform a high-fidelity music reconstruction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the high-fidelity music reconstruction method according to any one of claims 1 to 7 is implemented.

Images