Bluetooth sound box data processing method and system, and storage medium

Abstract

The invention relates to the technical field of Bluetooth sound boxes, in particular to a Bluetooth sound box data processing method and system and a storage medium. The method comprises the following steps: performing multi-modal data capture on a client to obtain original multi-modal client data; carrying out self-adaptive multi-resolution compression on the original multi-modal client data to obtain compressed multi-modal client data; transmitting the compressed multi-modal client data to the Bluetooth sound box through the client to obtain received compressed multi-modal data; performing multi-modal data decoupling on the received compressed multi-modal data to obtain separated audio data; performing dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data; and performing music preference type determination on the user to obtain a user music preference type identifier. According to the invention, the limitation of a traditional Bluetooth sound box in the aspects of personalized service and environmental adaptability is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of Bluetooth speakers, and in particular to a method, system and storage medium for processing Bluetooth speaker data. Background Art

[0002] In traditional Bluetooth speakers, a significant technical difficulty is the lack of consideration for users' personalized music preferences, resulting in no targeted optimization of audio data processing. This means that different users will get different listening experiences when listening to the same music because the speakers cannot recognize and adapt to personal preferences. This lack of personalized optimization limits the performance of speakers in meeting personalized needs. Due to the lack of personalized optimization, speakers use a unified standard when processing audio data and cannot dynamically adjust to the preferences of different users. This results in the inability of speakers to perform targeted optimization of audio signals, such as enhancing certain frequency responses to meet the listening needs of specific music styles. In addition, the acoustic environment in which the speaker is located has a significant impact on the sound quality, but traditional speakers often ignore this. Different environments, such as indoors, outdoors, open or confined spaces, will have different effects on audio signals. If the speaker cannot adjust the audio output according to the acoustic environment in which it is located, it cannot guarantee the best listening experience in all environments. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a method, system and storage medium for processing Bluetooth speaker data to solve at least one of the above technical problems.

[0004] To achieve the above object, a method for processing Bluetooth speaker data includes the following steps:

[0005] Step S1: Capturing multimodal data of a client to obtain original multimodal client data; performing adaptive multi-resolution compression on the original multimodal client data to obtain compressed multimodal client data;

[0006] Step S2: transmitting the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; performing multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; and performing dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data;

[0007] Step S3: determining the music preference type of the user to obtain a user music preference type identifier; performing phase response on the reconstructed audio data according to the user music preference type identifier to obtain phase optimized audio data;

[0008] Step S4: construct an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; perform acoustic environment compensation on the phase-optimized audio data according to the environmental acoustic feature database to obtain final optimized audio data.

[0009] The present invention performs adaptive multi-resolution compression on multimodal data, which can dynamically adjust the compression rate of data transmission according to actual network conditions, optimize the size of data packets, and reduce data loss and distortion caused by network fluctuations. This not only improves the robustness of data transmission, but also ensures the clarity and stability of sound quality, and maintains high audio quality even in poor network conditions. Through multimodal data decoupling and dynamic spectrum reconstruction optimization technology, audio data can be separated and processed more accurately, thereby restoring audio signals more realistically. This enables the played audio signal to better retain the details and dynamic range of the original sound, provide a richer and more three-dimensional auditory experience, and especially significantly improve the layering and detail performance of music. By identifying the user's personalized music preferences and performing corresponding phase response optimization, customized audio output can be provided for different users. This means that each user can obtain the best listening experience according to their preferences, thereby improving the adaptability of the Bluetooth speaker and the user's satisfaction. By constructing an environmental acoustic feature database and performing acoustic environment compensation on the audio signal based on these data, the speaker can automatically adjust the audio output in different environments to adapt to the current acoustic conditions. This intelligent compensation mechanism ensures a consistent high-quality audio experience in all environments, and users can enjoy optimized sound quality whether indoors, outdoors, in open or confined spaces. In summary, the present invention effectively solves the limitations of traditional Bluetooth speakers in data transmission, audio processing, personalized services and environmental adaptability, bringing users a more stable, clear, rich and personalized audio experience, while also improving the performance of Bluetooth speakers in different environments.

[0010] Preferably, the present invention further provides a system for processing Bluetooth speaker data, which is used to execute the method for processing Bluetooth speaker data as described above, and the system for processing Bluetooth speaker data comprises:

[0011] A data compression module is used to capture multimodal data of a client to obtain original multimodal client data; and to perform adaptive multi-resolution compression on the original multimodal client data to obtain compressed multimodal client data;

[0012] The audio reconstruction module is used to transmit the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; perform multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; perform dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data;

[0013] The audio phase optimization module is used to determine the music preference type of the user and obtain the user's music preference type identifier; perform phase response on the reconstructed audio data according to the user's music preference type identifier to obtain phase optimized audio data;

[0014] The environmental acoustic compensation module is used to construct an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; perform acoustic environment compensation on the phase optimized audio data according to the environmental acoustic feature database to obtain the final optimized audio data.

[0015] In the present invention, the data compression module can dynamically adjust the compression strategy according to the network status and data characteristics by adopting adaptive multi-resolution compression technology. This intelligent compression method not only reduces the bandwidth demand for data transmission, but also improves the efficiency of data transmission, ensuring that high-quality audio data can be stably transmitted in various network environments. The audio reconstruction module can effectively restore and enhance the audio signal by decoupling and dynamically reconstructing the received compressed data. This process not only reduces the distortion that occurs during data transmission, but also improves the clarity and dynamic range of the audio signal, making the played audio closer to the original sound quality, providing a richer and more delicate auditory experience. The audio phase optimization module can provide a customized audio processing solution for each user by analyzing the user's listening history and preferences. This personalized processing method enables the speaker to adjust the phase response of the audio according to the user's preferences, so that each user can obtain the audio experience that best suits their personal taste, enhancing the user's satisfaction and the intelligence level of the speaker. The environmental acoustic compensation module can intelligently identify the acoustic environment in which the speaker is located by constructing an environmental acoustic feature database, and compensate the audio signal according to the environmental characteristics. This compensation mechanism enables the speaker to automatically adjust the audio output in different environments to adapt to the current acoustic conditions, ensuring a consistent high-quality audio experience in various environments, such as indoors, outdoors, in open or confined spaces. Through the collaborative work of various modules, the entire process of intelligent processing from data capture, compression, transmission, decoupling, reconstruction, optimization to compensation is realized. This end-to-end intelligent processing not only improves the efficiency and quality of audio processing, but also enables the system to better adapt to different usage scenarios and user needs, improving the flexibility of the system and user experience.

[0016] Preferably, a computer-readable storage medium stores a method for processing Bluetooth speaker data as described above which can be loaded and executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, objects and advantages of the present invention will become more apparent from reading the detailed description with reference to the following drawings:

[0018] Figure 1 A schematic flow chart of the steps of a method for processing Bluetooth speaker data according to an embodiment is shown.

[0019] Figure 2 A detailed flow chart of step S23 of an embodiment is shown.

[0020] Figure 3 A detailed flow chart of step S25 of an embodiment is shown. DETAILED DESCRIPTION

[0021] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0023] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0024] To achieve this, please refer to Figures 1 to 3 The present invention provides a method for processing Bluetooth speaker data, comprising the following steps:

[0025] Step S1: Capturing multimodal data of a client to obtain original multimodal client data; performing adaptive multi-resolution compression on the original multimodal client data to obtain compressed multimodal client data;

[0026] Step S2: transmitting the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; performing multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; and performing dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data;

[0027] Step S3: determining the music preference type of the user to obtain a user music preference type identifier; performing phase response on the reconstructed audio data according to the user music preference type identifier to obtain phase optimized audio data;

[0028] Step S4: construct an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; perform acoustic environment compensation on the phase-optimized audio data according to the environmental acoustic feature database to obtain final optimized audio data.

[0029] In this embodiment, first, a multifunctional data collector, such as a smart phone or a dedicated audio processing device, can be used to capture the user's multimodal data. This includes audio signals, user interaction data, and device status information. Using the built-in microphone and sensor of the device, audio signals can be collected in real time, and user interaction and device status data can be obtained through the network interface. Next, adaptive multi-resolution compression technology, such as a compression algorithm based on wavelet transform, is used to compress the original multimodal data. This algorithm can dynamically adjust the compression ratio according to the data content and network conditions to optimize the data transmission efficiency. For example, the PyWavelets library of MATLAB or Python can be used to implement this compression process. Then, the compressed multimodal data is wirelessly transmitted to the Bluetooth speaker through the Bluetooth module of the client device. At the speaker end, an audio decoder is used to receive and decouple the compressed data and separate the audio data. This involves applying a specific decompression algorithm, such as Huffman decoding or LZW decoding. Then, the separated audio data is dynamically reconstructed and optimized. This can be done by analyzing the spectral content of the audio signal using a short-time Fourier transform (STFT) and applying spectral repair techniques, such as spectral subtraction or spectral filling, to enhance the spectral characteristics of the audio signal. The reconstructed audio data is optimized for phase response based on the user's music preference type identification. For example, if the user prefers music with heavy beats, the sense of rhythm can be enhanced by enhancing the phase consistency of the low-frequency components. Then, a database of environmental acoustic characteristics is constructed. This can be achieved by deploying microphone arrays at different locations and performing acoustic simulations using ray tracing software such as Raynoise to obtain spatial acoustic parameters. Finally, the phase-optimized audio data is compensated for the acoustic environment based on the environmental acoustic characteristics database. This involves applying filters, such as equalizers or dynamic range compressors, to adjust the audio signal to adapt it to a specific listening environment. Through this series of steps, the final optimized audio data can be obtained.

[0030] Preferably, step S1 comprises the following steps:

[0031] Step S11: Capturing multimodal data of the client to obtain original multimodal client data;

[0032] Specifically, audio data can be captured in real time through the client's microphone and sensors, and control instructions and device status updates can be received through the user interface. In this process, the audio signal is stored in the form of a waveform, the control instruction is stored in the form of a command code, and the device status is stored in the form of a status report. All of this data will be packaged together to form a raw multimodal dataset.

[0033] Step S12: performing modal feature separation on the original multimodal client data to obtain client multimodal feature data; wherein the client multimodal feature data includes audio data, control instruction data and device status data;

[0034] Specifically, spectrum analysis technology can be used to identify audio data, command parsing algorithms can be used to extract control instruction data, and status monitoring logic can be used to collect device status data. After separation, client multimodal feature data is obtained, wherein the client multimodal feature data includes audio data, control instruction data, and device status data.

[0035] Step S13: performing run-length encoding and differential pulse encoding on the control instruction data to obtain compressed control instruction data;

[0036] Specifically, the control instruction data can first be run-length encoded (RLE), which is a simple form of encoding that replaces a continuous sequence of identical data values ​​with a single value and a count value. For example, if there is a control instruction in the control instruction data that is ten consecutive "on" commands, RLE will encode it as "on x10". Next, differential pulse code modulation (DPCM) is applied to the data, which is a technique that uses the difference between signal values ​​rather than absolute values ​​for encoding. DPCM achieves data compression by calculating the differences between consecutive samples and then encoding these differences, ultimately obtaining compressed control instruction data.

[0037] Step S14: performing incremental encoding based on change detection on the device status data to obtain compressed device status data;

[0038] Specifically, device status parameters such as volume, battery level, and connection status can be continuously monitored. When a state change is detected, the values ​​before and after the change are recorded, and the difference between the two is calculated. For example, if the volume is adjusted from 75% to 80%, this change is recorded and a delta code is generated, indicating that the volume has increased by 5%. This change-based encoding method only stores information about state changes, rather than the absolute state value at each point in time, which significantly reduces the size of the data and ultimately results in compressed device state data.

[0039] Step S15: performing multi-scale signal decomposition on the audio data to obtain multi-scale audio data of the client, and performing wavelet transform on the multi-scale audio data of the client to obtain feature-enhanced audio data;

[0040] Specifically, you can use the AudioProcessor tool for multiscale signal decomposition, which integrates the functions of multiscale decomposition and wavelet transform. First, the tool performs multiscale decomposition on the audio data, which involves decomposing the signal into subbands of different frequency bands, each of which represents a specific scale of the signal. This can be achieved through wavelet transform, where an appropriate wavelet basis function is selected to match the characteristics of the audio signal. For example, you can use the Daubechies wavelet, which is suitable for the decomposition of audio signals because it can capture the time-frequency characteristics of the signal well. During the multiscale decomposition process, the audio signal is decomposed into a series of approximation coefficients and detail coefficients, which represent the low-frequency and high-frequency components of the signal, respectively. Next, the tool performs wavelet transform on these coefficients to further extract the features of the audio signal. Wavelet transform can enhance the important features of the signal, such as sharp transients and smooth background noise. In this way, feature-enhanced audio data can be obtained in the end.

[0041] Step S16: performing enhanced Huffman coding on the feature-enhanced audio data to obtain compressed audio data;

[0042] Specifically, enhanced Huffman coding is a lossless compression algorithm that assigns a unique code to each symbol in the data by constructing a variable-length coding tree. The frequency of occurrence of each symbol (e.g., sample value) in the feature-enhanced audio data can be first counted. Then, a Huffman tree is constructed based on these frequencies, in which the least frequent symbol is assigned the longest code and the most frequent symbol is assigned the shortest code. For example, if a particular frequency component in the audio data is very common, it will be assigned a short code, such as "01", and the uncommon component will be assigned a long code, such as "1110". Through encoding, compressed audio data is finally obtained.

[0043] Step S17: performing embedded multimodal fusion on the compressed control instruction data, the compressed device status data and the compressed audio data to obtain compressed multimodal client data.

[0044] Specifically, an embedded coding scheme, such as embedded Huffman coding (ECC), can be used to fuse compressed control instruction data, compressed device status data, and compressed audio data. ECC allows multiple data streams to be merged into a single data stream without destroying the integrity of the individual data streams. For example, assuming there is an audio data stream, ECC can be used to embed control instructions and device status data in the coding tree of the audio data. In this way, there will be a mark next to each audio data block indicating whether there are control instructions or device status data associated with it. If present, these data will be attached to the corresponding audio data block. Ultimately, the fused data stream will be packaged into a compressed multimodal client data packet, which can be efficiently transmitted to a Bluetooth speaker or other target device.

[0045] The present invention significantly reduces the size of data while retaining key information by performing run-length encoding, differential pulse encoding and incremental encoding based on change detection on control instruction data and device status data respectively. Such an encoding method reduces redundancy in the data transmission process and improves the efficiency and reliability of data transmission. The application of multi-scale signal decomposition and wavelet transform enables the audio data to be feature enhanced before compression, which not only improves the quality of the audio data, but also enables the compressed audio data to better retain the important features of the original audio, thereby providing a better audio experience after decompression. The compressed control instruction data, device status data and audio data are effectively combined together through embedded multimodal fusion to form compressed multimodal client data. This fusion method not only reduces the storage space of data, but also simplifies the transmission and processing flow of data, and improves the processing efficiency of the overall system. Through targeted compression strategies, the present invention reduces the demand for storage and processing resources. This is particularly beneficial for resource-constrained devices because it allows these devices to store and process more data without sacrificing the quality or function of the data due to resource constraints.

[0046] Preferably, step S2 comprises the following steps:

[0047] Step S21: The client transmits the compressed multimodal client data to the Bluetooth speaker and receives the data to obtain received compressed multimodal data;

[0048] Specifically, a client device can be used to package compressed multimodal data into a format suitable for Bluetooth transmission. This involves splitting the data into small blocks and adding the necessary Bluetooth transmission protocol header information to each block to ensure that the data can be correctly routed and transmitted in the Bluetooth network. Then, a Bluetooth protocol stack (such as Bluetooth Classic or Bluetooth Low Energy) is used to establish a connection with the Bluetooth speaker. Once the connection is established, the data will be sent to the speaker via wireless signals. After arriving at the Bluetooth speaker, the Bluetooth module on the speaker will receive these signals and use the corresponding Bluetooth protocol stack to parse and reassemble the data, and finally obtain the complete received compressed multimodal data.

[0049] Step S22: performing multimodal data decoupling on the received compressed multimodal data to obtain separated audio data, and performing time-frequency feature extraction on the separated audio data to obtain audio time-frequency feature data;

[0050] Specifically, the MultiModalDecoder tool can be used to process the received compressed multimodal data. First, the multimodal decoder identifies and separates the different data streams in the compressed multimodal data, such as audio data, control instruction data, and device status data. This involves parsing the markers in the embedded coding scheme to determine which parts belong to the audio data. Once the audio data is separated, the tool decodes it to restore its original form. After decoding, the AudioFeatureExtractor tool is used to extract the time-frequency features of the audio data. This tool uses the short-time Fourier transform (STFT) to convert the audio signal into a time-frequency representation, thereby extracting features such as frequency, amplitude, and phase. STFT does this by segmenting the audio signal into short time frames and performing a fast Fourier transform (FFT) on each frame. From the FFT results of each frame, key time-frequency features such as Mel-frequency cepstral coefficients (MFCCs) and chroma features are extracted, and finally audio time-frequency feature data is obtained.

[0051] Step S23: adjusting the preset initial analysis window length according to the audio time-frequency feature data to obtain an adjustment window length parameter;

[0052] Specifically, please refer to the sub-steps of step S23 for the detailed implementation process of this embodiment.

[0053] Step S24: performing short-time Fourier transform on the separated audio data according to the adjusted window length parameter to obtain an initial spectrum diagram;

[0054] Specifically, the separated audio data can be loaded. Then, a window function, such as a Hamming window or a Hanning window, is applied to perform windowing processing on the audio signal. Next, the windowed audio signal is divided into short time frames according to the adjustment window length parameter calculated in step S23. For example, if the adjustment window length parameter is 25 milliseconds, a segment of the audio signal is taken every 25 milliseconds for processing. For each short time frame, a fast Fourier transform (FFT) is performed, which is an efficient algorithm for calculating the spectrum of the signal. FFT converts each short time frame from the time domain to the frequency domain to generate corresponding spectrum data. These spectrum data represent the amplitude of different frequency components in each short time frame. Finally, the spectrum data of all short time frames are aggregated to form a two-dimensional initial spectrum diagram. The horizontal axis represents time, the vertical axis represents frequency, and the color or brightness represents the amplitude at different times and frequencies. For example, assume that there is an audio signal with a length of 1 second and a sampling rate of 44.1kHz. In step S23, the calculated adjustment window length parameter is 25 milliseconds, and the corresponding FFT point number is 1024. Then, 1024 samples are taken every 25 milliseconds for FFT to generate a series of spectrum data of short time frames, and finally form an initial spectrum diagram, which shows the time-frequency characteristics of this 1 second audio signal.

[0055] Step S25: performing key frequency band identification on the initial frequency spectrum to obtain audio key frequency band data, and performing high frequency spectrum density analysis on the audio key frequency band in the separated audio data according to the audio key frequency band data to obtain key frequency band enhanced time-frequency spectrum data;

[0056] Specifically, please refer to the sub-steps of step S25 for the detailed implementation process of this embodiment.

[0057] Step S26: updating the initial spectrum graph according to the spectrum data of the key frequency band enhancement to obtain an optimized spectrum graph;

[0058] Specifically, an initial spectrogram can be loaded. At the same time, the key frequency band enhanced time spectrum data is also loaded, which is obtained by performing key frequency band identification and enhancement processing on the initial spectrogram in step S25, highlighting the most important frequency bands in the audio signal. Next, an enhancement algorithm is applied to each key frequency band in the initial spectrogram. For example, a gain function based on an auditory psychology model is used, which can adjust the amplitude of the spectrum according to the sensitivity of the human ear to different frequencies. For each key frequency band, its amplitude is increased so that these frequency bands are more prominent in the final spectrogram. After processing all key frequency bands, the initial spectrogram is updated to generate an optimized spectrogram. For example, it is assumed that the low-frequency and high-frequency parts of the audio signal are identified as key frequency bands in step S25, and they are enhanced. In this step, the initial spectrogram is loaded, and the gain function and dynamic range compression are applied to these key frequency bands. After processing, a new spectrogram is generated, in which the amplitudes of the low-frequency and high-frequency parts are improved, and the dynamic range of the entire spectrum is effectively controlled.

[0059] Step S27: Performing harmonic fundamental frequency analysis on the optimized frequency spectrum to obtain fundamental frequency harmonic structure data, and reconstructing the fundamental frequency harmonic structure data to obtain reconstructed audio data.

[0060] Specifically, please refer to the sub-steps of step S27 for the detailed implementation process of this embodiment.

[0061] The present invention transmits compressed multimodal data to a Bluetooth speaker through a client, thereby reducing the bandwidth required for data in a wireless transmission process, thereby improving transmission efficiency and ensuring rapid and stable data transmission. Audio data can be accurately separated and key time-frequency features can be extracted through multimodal data decoupling. This improves the accuracy of audio processing. By dynamically adjusting the length of the analysis window according to the audio time-frequency feature data, the short-time Fourier transform is more in line with the actual characteristics of the audio signal. This optimized window setting can more accurately capture the details of the audio signal and improve the accuracy of spectrum analysis. By performing key frequency band identification and high spectral density analysis on the initial spectrogram, the most important part of the audio can be highlighted. This enables the key frequency band to be enhanced, thereby significantly improving the quality and perception of the audio. The initial spectrogram is updated in real time based on the key frequency band enhanced time-frequency spectrum data to generate an optimized spectrogram. This dynamic update mechanism enables audio processing to adapt to different audio content and characteristics, and provides more flexible and accurate processing results. By performing harmonic fundamental frequency analysis and reconstruction of fundamental frequency harmonic structure data on the optimized spectrogram, a high-quality audio signal can be restored. This reconstruction process retains the key information of the audio, so that the reconstructed audio signal has better sound quality and higher fidelity.

[0062] Preferably, step S23 includes the following steps:

[0063] Step S231: performing spectral centroid identification on the audio time-frequency feature data to obtain a spectral centroid distribution diagram, and performing time-varying spectral feature identification on the audio time-frequency feature data according to the spectral centroid distribution diagram to obtain an audio time-varying spectral feature matrix;

[0064] Specifically, the audio time-frequency feature data can be loaded, and the spectral centroid is calculated for the spectrum data of each short time frame. The spectral centroid is the weighted average of the spectrum, where the weight is the amplitude of each frequency component. The calculation formula of the spectral centroid is: Among them, f i is the frequency of the ith frequency component, P i is the corresponding amplitude, and N is the total number of frequency components; after calculating the spectral centroids of all frames, a spectral centroid distribution plot is generated, which shows the spectral centroid values ​​changing over time. Next, this distribution plot is analyzed to identify time-varying spectral features. This involves finding local maxima and minima of the spectral centroid, or identifying rapid changes in the spectral centroid that indicate important events or structural changes in the audio signal. Through these analyses, an audio time-varying spectral feature matrix can be constructed, which contains the spectral centroids and other related features that change over time.

[0065] Step S232: identifying the main frequency period of the audio time-frequency feature data according to the audio time-varying spectrum feature matrix to obtain a main frequency period sequence;

[0066] Specifically, the audio time-varying spectrum feature matrix can be loaded, and the spectrum data of each short time frame can be analyzed to identify the most significant frequency component, that is, the main frequency. This involves finding the frequency component with the largest amplitude in each frame, or using statistical methods to determine the most stable and significant frequency in multiple consecutive frames. Once the main frequency is identified, the changes of these frequencies over time are tracked to determine the main frequency period. This involves using an autoregressive (AR) model or a periodogram method to estimate the periodicity of the frequency. By analyzing the appearance and disappearance of the main frequency in consecutive frames, a main frequency period sequence can be constructed, which describes the main rhythm and melodic structure in the audio signal. For example, if the audio signal is a musical clip containing human voice and musical instruments, the fundamental frequency of the human voice and the main harmonics of the musical instrument are identified as the main frequencies, and the changes of these frequencies over time are tracked to generate a main frequency period sequence reflecting the rhythm and melodic structure of the music.

[0067] Step S233: Calculate the time scale ratio of the main frequency period sequence and the preset initial analysis window length to obtain the window-signal time scale ratio coefficient;

[0068] Specifically, a sequence of dominant frequency cycles can be loaded. At the same time, a preset initial analysis window length is obtained, which is an optimal value determined based on standard practices of audio analysis or previous experiments. Next, each dominant frequency cycle is compared with the initial analysis window length. The formula for calculating the time scale ratio is as follows: This scaling factor reflects the relative size of the main frequency period and the analysis window length, and can be used to adjust the analysis window to better fit the characteristics of the audio signal. For example, if the main frequency period is much smaller than the initial analysis window length, this indicates that a smaller window is needed to more accurately capture the details of the audio signal.

[0069] Step S234: performing transient capture performance evaluation on the preset initial analysis window length according to the window-signal time scale ratio coefficient to obtain a transient capture performance index;

[0070] Specifically, a window-signal time scale ratio coefficient can be loaded and analyzed to evaluate the effect of different window lengths on transient capture. Transient capture performance evaluation involves simulating the effect of different window lengths on transient features in audio signals, such as the percussion sound of percussion instruments, sudden volume changes, etc. A set of predefined evaluation criteria, such as transient response index or signal distortion metric, are used to evaluate the transient capture performance of different window lengths. For example, the ratio of the peak value to the average value of the transient signal under different window lengths is calculated, or the ratio of the peak value of the transient signal to the window length is calculated to evaluate the accuracy and efficiency of transient capture. Through these evaluations, a transient capture performance index is generated, which reflects the effect of different window lengths on the ability to capture transient features. For example, if the transient capture performance index indicates that a smaller window length can better capture transient features, then a smaller window length is selected in subsequent audio analysis.

[0071] Step S235: adjusting the preset initial analysis window length according to the transient capture performance index to obtain a window length adjustment factor, and performing dynamic range compression on the preset initial analysis window length according to the window length adjustment factor to obtain a preliminary adjusted window length;

[0072] Specifically, a transient capture performance indicator may be loaded and analyzed to determine whether the initial analysis window length needs to be adjusted. For example, if the transient capture performance indicator shows that a smaller window length can more effectively capture transient features, a window length adjustment factor is calculated. This factor may be calculated by the following formula: Among them, the optimal window length is the length recommended based on the transient capture performance index. Next, the window length adjustment factor is applied to the initial analysis window length for dynamic range compression. The process of dynamic range compression involves adjusting the amplitude of the audio signal to ensure that the dynamic range of the signal can be effectively controlled under the new window length. Use compression algorithms, such as RMS (root mean square) compression or peak compression, to adjust the amplitude of the signal. For example, if the adjustment factor is 0.8, multiply the initial analysis window length by 0.8 to obtain a new preliminary adjustment window length. Finally, output this preliminary adjustment window length.

[0073] Step S236: Binary quantize the preliminary adjustment window length to obtain an adjustment window length parameter.

[0074] Specifically, the initial adjusted window length can be loaded, which is a floating point number expressed in milliseconds or seconds. Next, a binary quantization process is performed. Quantization involves mapping continuous values ​​to the nearest discrete value. In this case, the floating point value of the window length is converted to binary format. This involves multiplying the floating point number by a specific scaling factor (for example, if the window length is in milliseconds, multiply it by 1000 to convert it to microseconds), then rounding the result to the nearest integer, and finally converting the integer to its binary representation. For example, if the initial adjusted window length is 25 milliseconds, it is converted to microseconds (25 milliseconds * 1000 = 25000 microseconds) and then rounded to the nearest integer, which is 25000. Then, 25000 is converted to its binary form. This binary value will be used as the adjustment window length parameter.

[0075] The present invention can more accurately capture the essential characteristics of an audio signal by performing spectral centroid recognition and time-varying spectral feature recognition on audio time-frequency feature data. The main frequency period sequence is identified by analyzing the audio time-varying spectral feature matrix, so that the identification of the main frequency period of the audio signal is more accurate. According to the time scale ratio calculation of the main frequency period sequence and the initial analysis window length, and the evaluation of transient capture efficiency, the length of the analysis window can be dynamically adjusted. This makes the analysis window more in line with the actual characteristics of the audio signal, thereby improving the accuracy and effect of audio analysis. Through the evaluation of transient capture efficiency and the dynamic range compression of the window length, the transient characteristics in the audio signal can be more effectively captured. This is particularly important for improving the dynamic range and detail performance of the audio signal, especially when processing music and sound effects containing rich transient information. By performing wavelet multi-resolution decomposition on the preliminary adjustment window length and screening based on the principle of information entropy maximization, the optimal window length can be selected from the multi-scale window length candidate set. This optimization selection mechanism makes the analysis window more suitable for the multi-scale characteristics of the audio signal, and improves the flexibility and adaptability of audio analysis. By binary quantizing the optimized window length, the window length used in audio processing can be accurately controlled.

[0076] Preferably, step S25 comprises the following steps:

[0077] Step S251: performing spectrum energy density calculation on the initial spectrum graph to obtain a spectrum energy density distribution graph, and performing spectrum peak detection on the initial spectrum graph to obtain a spectrum peak feature set;

[0078] Specifically, the data of the initial spectrogram can be loaded, which contains the amplitude information of the audio signal at different frequencies and times. The amplitude of each frequency component at each time frame is squared to obtain an energy value, and then the energy values ​​of the same frequency component at all time frames are summed to calculate the total energy of the frequency component. In this way, a spectrum energy density distribution graph is obtained, which shows the total energy distribution of the audio signal at different frequencies. Next, the spectrum energy density distribution graph is subjected to spectrum peak detection. The purpose of spectrum peak detection is to identify significant peaks in the spectrum, which represent the most important frequency components in the audio signal. To this end, a threshold-based detection method is used, such as setting a global threshold or a local threshold, to distinguish between noise and true spectrum peaks. For example, a threshold is set, which is the average value of all energy values ​​in the spectrum energy density distribution graph plus a certain proportion of the standard deviation. All frequency components above this threshold will be considered as spectrum peaks and will be collected into the spectrum peak feature set. Finally, the spectrum energy density distribution graph and the spectrum peak feature set are output.

[0079] Step S252: determining the key frequency band energy threshold of the initial frequency spectrum according to the frequency spectrum energy density distribution diagram and the spectrum peak feature set to obtain the key frequency band energy threshold;

[0080] Specifically, a spectral energy density distribution map and a spectral peak feature set can be loaded. Then, this data is analyzed to determine which frequency bands contain key energy information. This involves analyzing the spectral peak feature set to identify the most significant frequency range, or clustering the spectral energy density distribution map to identify areas where energy is concentrated. Next, the energy threshold of the key frequency band is determined based on these analysis results. This threshold can be fixed or dynamic, depending on the characteristics of the audio signal and the desired processing goals. For example, a threshold is selected that is the median or average of the energy values ​​within the key frequency band, or a threshold that can distinguish between key frequency bands and non-key frequency bands. Finally, the key frequency band energy threshold is output.

[0081] Step S253: marking and merging high-energy frequency bands of the initial spectrum according to the key frequency band energy threshold to obtain preliminary key frequency band mapping data;

[0082] Specifically, an initial spectrogram and a key band energy threshold can be loaded. The energy value of each frequency component of the initial spectrogram is compared with the key band energy threshold. For frequency bands with energy values ​​higher than the threshold, they are marked as high-energy bands. These high-energy bands represent the most important parts of the audio signal, such as melody lines or rhythmic hits. Next, these marked high-energy bands are merged. The purpose of merging is to merge adjacent or overlapping high-energy bands into a larger band. The merging strategy is based on the frequency range, energy value or time continuity of the bands. For example, if two high-energy bands are adjacent in frequency and their energy values ​​are significantly higher than the threshold, they are merged into a single preliminary key band. Finally, preliminary key band mapping data is output, which includes information of all marked and merged high-energy bands.

[0083] Step S254: performing auditory weighted filtering on the initial spectrogram to obtain an auditory weighted spectrogram;

[0084] Specifically, an initial spectrogram can be loaded. An auditory weighting filter based on the human auditory system is used, which can simulate the sensitivity of the human ear to sounds of different frequencies. For example, the human ear is more sensitive to mid-frequency sounds than to low-frequency or high-frequency sounds, so the filter will give more weight to the mid-frequency components. This auditory weighting filter is applied to each frequency component of the initial spectrogram. The filtering process involves multiplying the energy value of each frequency component by the corresponding auditory weight to adjust the energy distribution of the spectrogram. In this way, some parts of the spectrogram will be strengthened and other parts will be weakened to better reflect the perceptual characteristics of human hearing. Finally, an auditory weighted spectrogram is output, which takes into account the nonlinear characteristics of human hearing.

[0085] Step S255: performing perceptual importance scoring on the preliminary key frequency band mapping data according to the auditory weighted spectrogram to obtain a key frequency band importance matrix;

[0086] Specifically, auditory weighted spectrograms and preliminary key band mapping data can be loaded. The auditory weighted spectrograms are analyzed to determine the perceptual importance of each preliminary key band. This involves considering the energy, frequency position, and relationship of the band to other bands. For example, higher scores are given to bands that have high energy and are within the sensitive frequency range of the human ear. Next, a perceptual importance score is assigned to each preliminary key band, which reflects the contribution of the band to the overall auditory perception. The score can be linear or nonlinear, depending on the specific auditory model and algorithm. For example, a psychoacoustic-based model, such as the Zwicker psychoacoustic model, is used to calculate the perceptual importance of each band. Finally, a key band importance matrix is ​​output, which contains the perceptual importance score of each preliminary key band.

[0087] Step S256: clustering the key frequency band importance matrix to obtain an optimized key frequency band set, and performing adaptive band-pass filtering on the optimized key frequency band set to obtain a key frequency band enhanced audio signal;

[0088] Specifically, a key frequency band importance matrix can be loaded. A clustering algorithm, such as K-means or hierarchical clustering, is used to group the key frequency bands. The purpose of clustering is to group frequency bands with similar perceptual importance into one category. For example, frequency bands with scores above a certain threshold are classified as "high importance" and frequency bands with scores below the threshold are classified as "low importance". Next, adaptive bandpass filtering is performed on each clustered optimized key frequency band set. The purpose of adaptive bandpass filtering is to enhance the signals of these frequency bands while reducing noise and other unimportant signals. An adaptive filter-based algorithm, such as the LMS (least mean square) algorithm, is used to dynamically adjust the parameters of the filter to adapt to the characteristics of the audio signal. Finally, the key frequency band enhanced audio signal is output.

[0089] Step S257: performing time-frequency transformation on the key-band enhanced audio signal to obtain key-band enhanced time-frequency spectrum data.

[0090] Specifically, the enhanced key frequency band audio signal can be loaded. Next, the audio signal is converted from the time domain to the time-frequency domain by applying time-frequency transformation techniques, such as short-time Fourier transform (STFT), wavelet transform or Wigner-Ville distribution. This transformation can reveal the frequency components of the audio signal that change over time and provide richer time-frequency feature information. Taking STFT as an example, a suitable window function, such as a Hamming window, is selected and the audio signal is windowed. Then, the windowed signal is divided into short time frames, and a fast Fourier transform (FFT) is performed on each short time frame to calculate the spectral content of each frame. By changing the position of the window and repeating this process, a time-frequency spectrum can be constructed, which shows the amplitude and phase information of the frequency components that change over time. Finally, the key frequency band enhanced time-frequency spectrum data is output.

[0091] The present invention can accurately identify the key frequency bands and significant features in the audio signal by performing spectrum energy density calculation and spectrum peak detection on the initial spectrum graph. By combining the spectrum energy density distribution and the spectrum peak feature set to determine the key frequency band energy threshold, the key frequency band in the audio signal can be more accurately identified. This energy threshold-based method improves the reliability of key frequency band identification. By marking and merging the high-energy frequency bands of the initial spectrum graph, the high-energy part in the audio signal can be identified and integrated. This helps to highlight the most important part of the audio signal, so that the audio processing can focus more on the core content of the signal. Through auditory weighted filtering and perceptual importance scoring, the audio signal can be optimized according to the auditory characteristics of the human ear. This method makes the processing of audio signals more in line with human auditory habits, thereby improving the auditory perception quality of the audio signal. By clustering the key frequency band importance matrix and adaptive bandpass filtering, the key frequency band of the audio signal can be intelligently optimized. This not only improves the signal quality of the key frequency band, but also makes the audio signal clearer and more natural. By performing time-frequency transformation on the key frequency band enhanced audio signal, the time-frequency characteristics of the audio signal can be further improved. This helps to enhance the dynamic range and detail of the audio signal while maintaining its quality.

[0092] Preferably, step S27 includes the following steps:

[0093] Step S271: performing logarithmic spectrum enhancement on the optimized spectrum graph to obtain an enhanced logarithmic spectrum graph;

[0094] Specifically, data of an optimized spectrogram can be loaded, which contains the amplitude information of the audio signal at different frequencies. The logarithm of the amplitude value of each frequency component is taken to obtain a spectrum representation that is closer to the loudness perceived by the human ear. The logarithmic transformation helps to compress the dynamic range of the audio signal, so that the low-energy frequency bands (usually difficult for the human ear to detect) are enhanced while avoiding distortion of the high-energy frequency bands. In the specific operation, the natural logarithm or the logarithm with a base of 10 is used, and the logarithmic spectrum is offset to avoid the negative infinity problem in the logarithmic operation. For example, if the amplitude values ​​in the original spectrogram range from 0.01 to 1000, after taking the logarithm, these values ​​will be converted to a range that is easier to handle, such as from -4.6 to 3. Next, the spectrum after taking the logarithm is further enhanced, including dynamic range compression or gain adjustment, to highlight the key features in the spectrum. Finally, an enhanced logarithmic spectrogram is output, which shows the spectrum of the audio signal after logarithmic transformation and enhancement.

[0095] Step S272: performing cepstrum fundamental frequency estimation on the enhanced logarithmic spectrum to obtain an audio fundamental frequency estimation sequence, and performing theoretical harmonic frequency calculation on the optimized spectrum based on the audio fundamental frequency estimation sequence to obtain a theoretical harmonic frequency map;

[0096] Specifically, an enhanced log spectrum graph can be loaded. Cepstrum analysis is performed, which is an analysis method that converts a log spectrum into a cepstrum. Cepstrum is a representation method that displays the periodic components of a spectrum, which can reveal fundamental frequency information in an audio signal. In the specific operation, the log spectrum is transformed using a discrete cosine transform (DCT) to obtain cepstrum coefficients. The peak positions of the cepstrum coefficients correspond to the fundamental frequency period of the audio signal. These peaks are identified and the corresponding fundamental frequency estimation sequence is extracted. Next, the theoretical harmonic frequencies of the optimized spectrum graph are calculated based on the audio fundamental frequency estimation sequence. This involves calculating the harmonic series, i.e., integer multiples of the fundamental frequency, for each fundamental frequency estimate. A predefined harmonic range, such as up to the 20th harmonic, is used to calculate the harmonic frequency of each fundamental frequency. Finally, a theoretical harmonic frequency graph is output, which shows the theoretical harmonic frequency distribution calculated based on the fundamental frequency estimation sequence.

[0097] Step S273: performing harmonic peak detection on the optimized spectrum diagram according to the theoretical harmonic frequency diagram to obtain an actual harmonic feature set;

[0098] Specifically, a theoretical harmonic frequency map and an optimized spectrum map can be loaded. The theoretical harmonic frequency map provides the expected harmonic frequency positions in the audio signal, while the optimized spectrum map contains the actual spectrum data of the audio signal. Peak detection is performed on each theoretical harmonic frequency position in the optimized spectrum map. This involves searching for local maxima in the spectrum map near each theoretical harmonic frequency. In order to improve the accuracy of the detection, a peak detection algorithm is used, such as a threshold-based method or a local maximum-based method. For example, a threshold is set, and only when the amplitude value in the spectrum is higher than the threshold, a harmonic peak is considered to exist at the position. Once the peaks are detected, the positions and amplitudes of these peaks are recorded and combined into an actual harmonic feature set.

[0099] Step S274: performing harmonic energy ratio calculation on the actual harmonic feature set to obtain a harmonic energy ratio matrix, and performing time-varying harmonic structure extraction on the actual harmonic feature set to obtain a time-varying harmonic structure sequence;

[0100] Specifically, an actual harmonic feature set can be loaded. An energy ratio calculation is performed on each detected harmonic peak. This involves dividing the amplitude value of each harmonic by the sum of the amplitudes of all harmonics to obtain the energy ratio of each harmonic. This ratio reflects the energy contribution of each harmonic in the overall audio signal. Next, a harmonic energy ratio matrix is ​​constructed, which contains the energy ratio information of each harmonic. In addition, a time-varying harmonic structure extraction is performed on the actual harmonic feature set. This involves analyzing the changes in harmonic peaks over time to identify time-varying harmonic structures in audio signals. A time series analysis method, such as a hidden Markov model (HMM) or a recurrent neural network (RNN), is used to model and extract time-varying harmonic structures. Finally, a harmonic energy ratio matrix and a time-varying harmonic structure sequence are output.

[0101] Step S275: generating harmonic structure data for the harmonic energy ratio matrix and the time-varying harmonic structure sequence according to the enhanced logarithmic spectrum to obtain fundamental frequency harmonic structure data; wherein the fundamental frequency harmonic structure data includes fundamental frequency trajectory data, harmonic frequency sequence and harmonic intensity matrix;

[0102] Specifically, the enhanced logarithmic spectrogram, harmonic energy ratio matrix, and time-varying harmonic structure sequence can be loaded. These data are analyzed to generate fundamental frequency harmonic structure data, including fundamental frequency trajectory data, harmonic frequency sequence, and harmonic intensity matrix. First, the fundamental frequency trajectory, that is, the change of the most significant frequency component over time, is identified in the enhanced logarithmic spectrogram. Then, the harmonic frequency sequence is extracted, which is obtained based on the theoretical harmonic frequency graph and the actual harmonic peak detection results. The harmonic intensity matrix is ​​calculated based on the harmonic energy ratio matrix and represents the relative intensity of each harmonic. For example, suppose there is an audio signal with a fundamental frequency of 100 Hz and 3 significant harmonics with frequencies of 200 Hz, 300 Hz, and 400 Hz. The enhanced logarithmic spectrogram is analyzed to determine the change of these frequency components over time and record their amplitudes. In this way, the fundamental frequency trajectory data (for example, the change of the amplitude of 100 Hz over time), the harmonic frequency sequence ([200 Hz, 300 Hz, 400 Hz]), and the harmonic intensity matrix (the amplitude value of each harmonic) are obtained. Finally, the fundamental frequency harmonic structure data is output.

[0103] Step S276: identifying the phase relationship between the fundamental frequency trajectory data and the harmonic frequency sequence to obtain a harmonic phase relationship diagram, and performing harmonic space redistribution on the fundamental frequency harmonic structure data according to the harmonic phase relationship diagram and the harmonic intensity matrix to obtain redirected harmonic structure data;

[0104] Specifically, fundamental frequency trajectory data and harmonic frequency sequences can be loaded. These data are analyzed to identify the phase relationship between harmonics. This involves calculating the phase difference between each harmonic and its fundamental frequency and tracking the changes of these phase differences over time. For example, a short-time Fourier transform (STFT) is used to calculate the phase of the fundamental frequency and each harmonic in each time frame, and then the phase difference between them is calculated. By analyzing the changes of these phase differences over time, a harmonic phase relationship diagram is constructed, which shows the phase relationship between the fundamental frequency and each harmonic. Next, the harmonic phase relationship diagram and the harmonic intensity matrix are used to perform harmonic space redistribution on the fundamental frequency harmonic structure data. This involves adjusting the phase and amplitude of the harmonics to improve the sound quality of the audio signal or adapt to specific audio processing goals. For example, the amplitude of a specific harmonic is increased, or its phase is changed to enhance a specific feature of the audio signal. Finally, the redirected harmonic structure data is output, which includes the adjusted fundamental frequency trajectory, the harmonic frequency sequence, and the harmonic intensity matrix.

[0105] Step S277: performing harmonic synthesis on the redirected harmonic structure data to obtain a preliminary reconstructed audio signal, and performing envelope modulation on the preliminary reconstructed audio signal to obtain an envelope modulated audio signal;

[0106] Specifically, you can load the redirected harmonic structure data, which includes the fundamental frequency trajectory, the harmonic frequency sequence, and the adjusted harmonic intensity matrix. Use this data to synthesize the audio signal. The synthesis process involves setting the frequency, amplitude, and phase information of each harmonic component sine wave according to the redirected harmonic structure data. In specific operation, the following formula is applied to each harmonic component to generate the signal: Where s(t) is the audio signal at time t, M is the total number of harmonics, and A m is the amplitude of the mth harmonic, y m is the frequency of the mth harmonic, ψ m is the phase of the mth harmonic, t is the time variable, and after synthesizing the preliminary reconstructed audio signal, envelope modulation will be performed. Envelope modulation involves modulating the amplitude of each harmonic component according to an envelope function E(t), which can be expressed as:

[0107]

[0108] Where A1 is the peak amplitude of the envelope; t is the time variable; e is the base of the natural logarithm; T attack is the attack time, i.e. the time it takes for the signal to appear from nothing; t decay is the decay time point, that is, the time from attack to the end of decay; T decay is the decay time, i.e. the time it takes for the signal to decay from the peak value to the sustained level; t sustain is the duration point, that is, the time from the end of decay to the beginning of release; T release is the release time, i.e. the time it takes for the signal to decay from a sustained level to nothing. This envelope function can be the amplitude envelope of an audio signal, which describes the time-varying amplitude of a signal. For example, an ADSR (Attack, Decay, Sustain, Release) envelope is used, which is a common envelope shape in music and audio processing used to simulate the natural start and end of a sound. Ultimately, the output envelope modulates the audio signal, which reflects the natural characteristics of the audio in the time domain, such as the dynamic changes in the start, sustain, and end phases of the sound.

[0109] Step S278: performing phase correction on the envelope modulated audio signal to obtain reconstructed audio data.

[0110] Specifically, an envelope modulated audio signal can be loaded. The phase content of the signal is analyzed and compared with an ideal phase reference to identify and correct any phase deviation or distortion. This involves using a phase correction algorithm, such as the Hilbert transform, to estimate the instantaneous phase and amplitude of the signal. In specific operation, the Hilbert transform is applied to the signal to obtain an analytical signal, and then the instantaneous phase of the analytical signal is calculated. Next, this instantaneous phase is compared with the theoretical phase (for example, the phase of a sine wave), and the difference between the two is calculated. Based on this difference, the phase of the audio signal is adjusted to align it with the theoretical phase. Finally, phase-corrected audio data is output, which reflects the ideal characteristics of the audio signal in both the frequency domain and the time domain.

[0111] The present invention can analyze the spectral characteristics of the audio signal more deeply by performing logarithmic spectrum enhancement on the optimized spectrum graph. This makes the details in the spectrum graph more prominent and helps to capture subtle changes in the audio signal. Accurate fundamental frequency and harmonic frequency information are provided for the audio signal by cepstrum fundamental frequency estimation and theoretical harmonic frequency calculation. By performing harmonic peak detection and actual harmonic collection on the optimized spectrum graph, the harmonic components in the audio signal can be identified and strengthened. This helps to improve the richness and expressiveness of the audio signal, so that the reconstructed audio signal is closer to the original sound quality. The energy distribution of the audio signal can be better understood and adjusted by calculating the harmonic energy ratio and extracting the time-varying harmonic structure. This optimization helps to improve the dynamic range and balance of the audio signal, making the audio output more natural and pleasant. By extracting and analyzing the time-varying harmonic structure sequence, the structural characteristics of the audio signal that change with time can be captured. This time-varying analysis helps to maintain the coherence and fluency of the audio signal and improve the listening experience. By identifying the phase relationship and spatial redistribution of the fundamental frequency harmonic structure data, as well as subsequent harmonic synthesis and envelope modulation, the audio signal can be accurately reconstructed. This not only preserves the key information of the audio signal, but also improves the overall quality of the signal. The final phase correction step ensures the phase accuracy of the reconstructed audio signal, which is crucial to ensuring the clarity and stability of the sound quality. The phase-corrected audio signal can provide a purer and more accurate listening experience.

[0112] Preferably, step S3 comprises the following steps:

[0113] Step S31: collecting historically played tracks from the client to obtain a historically played track data set, and identifying the audio playing mode of the user based on the historically played track data set to obtain user audio playing feature data; wherein the user audio playing feature data includes user style preference distribution data, user playing time mode and user music element preference data;

[0114] Specifically, a historical play track dataset can be collected from the client's music playback application. This involves accessing a database or reading the user's music playback log to obtain a list of music played by the user in the past period of time. Detailed information about each song, such as artist, album, number of plays, play duration, and play timestamp, is recorded. Next, a machine learning algorithm is used to analyze this data to identify the user's audio playback pattern. For example, a clustering algorithm (such as K-means) is used to group the music played by the user to identify the user's preferred music style. In addition, the user's playback time pattern is analyzed, such as the time period of the day when the user usually listens to music and the duration of each listening session. Music Information Retrieval (MIR) technology is used to analyze the characteristics of each song, such as rhythm, tonality, pitch, timbre, and intensity. A feature vector is constructed to represent these elements of each song, and these feature vectors are used to train a classifier, such as a support vector machine (SVM), to identify the user's preference for different music elements. Finally, the user's audio playback feature data, including user style preference distribution data, user playback time pattern, and user music element preference data, will be output.

[0115] Step S32: constructing a user profile of the user according to the user style preference distribution data, the user playback time pattern and the user music element preference data to obtain a user audio profile, and using a preset preference recognition model to recognize the user preference type of the user audio profile to obtain a user music preference type identifier; wherein the user music preference type identifier is any one of a rhythm-oriented listener, a melody-oriented listener and an atmosphere-oriented listener;

[0116] Specifically, user style preference distribution data, user play time pattern and user music element preference data can be loaded. Next, data mining technology is used to build a user audio profile. This involves using machine learning algorithms such as decision trees, random forests or gradient boosting machines to analyze user data and find patterns of user preferences. For example, it is found that users often listen to jazz at night and prefer to listen to music with a strong sense of rhythm on certain days of the week. Then, a preset preference recognition model is used to classify the user audio profile. This model is based on previous user research and divides users into different music preference types, such as rhythm-oriented listeners, melody-oriented listeners and atmosphere-oriented listeners. A classifier, such as a support vector machine (SVM) or a neural network, is trained to identify the user's music preference type based on the feature data of the user's audio profile. For example, if a user's music element preference data shows that music with complex melodies is often listened to, and the play time pattern shows that music is usually listened to when relaxing, then the model will classify the user as a melody-oriented listener. Finally, the user music preference type identifier is output.

[0117] Step S33: if the user's music preference type is identified as a rhythm-oriented listener, low-frequency phase consistency enhancement is performed on the reconstructed audio data to obtain rhythm-enhanced audio data;

[0118] Specifically, low-frequency phase consistency enhancement is performed on reconstructed audio data of a user identified as a rhythm-oriented listener to obtain rhythm-enhanced audio data. The reconstructed audio data can be loaded. The low-frequency parts of the audio data will be focused on because these parts usually contain rhythm information. A bandpass filter is used to isolate the low-frequency band, for example, the cutoff frequency is set below 200 Hz. Next, phase consistency enhancement is performed on these low-frequency components. This involves analyzing and adjusting the phase of adjacent samples to ensure phase continuity and consistency of the rhythm components. One method is to use phase shaping techniques, such as Hilbert transform, to generate an analytical signal and adjust the phase to enhance rhythm perception. For example, if the phase of the rhythm components in the audio data is discontinuous in some parts, the phase of these parts is adjusted to smooth the transition, thereby enhancing the perception of the rhythm. Finally, rhythm-enhanced audio data is output, which has enhanced phase consistency and amplitude in the low-frequency band.

[0119] Step S34: if the user's music preference type is identified as a melody-oriented listener, the mid-high frequency phase relationship of the reconstructed audio data is enhanced to obtain melody-prominent audio data;

[0120] Specifically, the mid-high frequency phase relationship is enhanced for the reconstructed audio data of the user identified as a melody-oriented listener to obtain melody-prominent audio data. The reconstructed audio data can be loaded. The mid-high frequency parts of the audio data will be focused on because these parts usually contain melody information. A bandpass filter is used to isolate the mid-high frequency band, for example, the cutoff frequency is set between 200Hz and 4000Hz. Next, the phase relationship of these mid-high frequency components is enhanced. This involves analyzing and adjusting the phase of adjacent frequency components to ensure the clarity and coherence of the melody line. One method is to use spectrum editing techniques, such as phase-based spectrum shaping, to enhance the melody components in a specific frequency range. For example, if the phase of the melody components in the audio data is inconsistent at certain frequencies, the phase of these frequencies is adjusted to ensure the coherence of the melody line. Finally, melody-prominent audio data is output, which has enhanced phase relationships and amplitudes in the mid-high frequency bands.

[0121] Step S35: If the user's music preference type is identified as an atmosphere-oriented listener, full-band phase spatial enhancement is performed on the reconstructed audio data to obtain spatially enhanced audio data;

[0122] Specifically, the reconstructed audio data of the user identified as an atmosphere-oriented listener is subjected to full-band phase spatial enhancement to obtain spatially enhanced audio data. The reconstructed audio data can be loaded. The entire frequency band will be focused on to enhance the spatial sense of the audio. Stereo expansion techniques such as HRTF (head-related transfer function) or virtual speaker technology are used to simulate a wider sound field. Next, the audio data is subjected to phase spatial enhancement. This involves using cross-feedback delay networks to increase the perception of depth and width of the sound, or using environment-based filters to simulate the reflection and reverberation characteristics of different acoustic environments. For example, each frequency component in the audio is analyzed, and its phase and amplitude are adjusted according to its frequency characteristics to simulate the propagation of sound in a specific environment. This includes increasing early reflections and late reverberation, thereby creating a richer and more immersive auditory experience. Finally, spatially enhanced audio data is output, which has an enhanced phase spatial sense in the full frequency band.

[0123] Step S36: performing phase modulation processing on the reconstructed audio data according to the rhythm-enhanced audio data, the melody-emphasized audio data, or the spatial-enhanced audio data to obtain phase-optimized audio data.

[0124] Specifically, the reconstructed audio data and the corresponding enhanced audio data (rhythm enhancement, melody prominence or spatial enhancement) can be loaded. These data are analyzed and the phase modulation strategy that needs to be applied is determined. Next, the reconstructed audio data is phase modulated. This involves using phase modulation techniques, such as phase shifting or phase rotation, to adjust the phase characteristics of the audio signal. For example, a phase modulator is used that dynamically adjusts the phase of the original audio signal based on the characteristics of the enhanced audio data. If the enhanced audio data is rhythm enhancement, focus on enhancing the phase consistency of the rhythm component. If the enhanced audio data is melody prominence, enhance the phase clarity of the melody line. If the enhanced audio data is spatial enhancement, adjust the phase to expand the sound field. Finally, output phase optimized audio data.

[0125] The present invention can collect user audio playback feature data reflecting user preferences by collecting the user's historical playback tracks and identifying their audio playback modes. This helps to more accurately understand the user's listening habits and preferences, thereby providing audio content that is more in line with the user's taste. By constructing a user audio portrait using data such as user style preferences, playback time modes, and music element preferences, user classification is made more detailed and accurate. According to the user's music preference type identification (rhythm-oriented, melody-oriented, or atmosphere-oriented listeners), the audio data can be targeted for phase enhancement and modulation processing. This makes the audio output more able to meet the auditory needs of different users. For rhythm-oriented listeners, enhancing low-frequency phase consistency can make the rhythm more prominent; for melody-oriented listeners, enhancing mid- and high-frequency phase relationships can make the melody clearer; for atmosphere-oriented listeners, enhancing the full-band phase spatial sense can provide a richer auditory background. By performing phase modulation processing on the reconstructed audio data, the perceived quality of the audio can be further improved. Whether it is the sense of rhythm, melody, or space, it can be optimized accordingly, so that the audio output is more in line with the user's personal preferences.

[0126] Preferably, step S4 comprises the following steps:

[0127] Step S41: deploy a multi-point microphone array in the environment where the Bluetooth speaker is located, and perform spatial sound field sampling and positioning to obtain a spatial sampling point distribution map, perform wide-band frequency sweep excitation on the environment where the Bluetooth speaker is located through the microphone array, and collect response signals to obtain a multi-channel environmental response signal, perform time-frequency recognition on the multi-channel environmental response signal, and obtain a time-varying spectrum atlas;

[0128] Specifically, a multi-point microphone array can be deployed in the environment where the Bluetooth speaker is located. This array consists of four or more microphones, which are placed at different locations in the room to ensure comprehensive spatial sound field sampling. The microphones are connected to a central data acquisition system by wire or wireless means. Next, the spatial sound field sampling and positioning process is started. This involves playing a known sound source, such as white noise or a maximum length sequence, and recording it through the microphone array. By analyzing these recordings, the position of each microphone and the relative distance between them can be determined, thereby generating a spatial sampling point distribution map. Then, the microphone array is used to perform wide-band swept frequency excitation on the environment. This involves playing a series of single tones of different frequencies, covering the audible range of the human ear (usually 20Hz to 20kHz). Each single tone will generate reflections and reverberations in the environment, and the microphone array will capture these response signals. The collected response signals are multi-channel, and each microphone channel will record a unique version of the signal. Time-frequency identification is performed on these multi-channel environmental response signals, usually using short-time Fourier transform (STFT) or other time-frequency analysis methods to obtain a set of time-varying spectrum maps. These atlases reveal the response characteristics of the environment to sounds of different frequencies, including reflection, absorption, and reverberation. Finally, time-varying spectrum atlases are output, which contain the response information of each sampling point in the environment to sounds of different frequencies.

[0129] Step S42: extracting spatial acoustic parameters of the ambient sound field based on the time-varying spectrum atlas to obtain a spatial acoustic feature matrix;

[0130] Specifically, digital signal processing technology can be used to analyze each set of time-varying spectrograms (a set of time-varying spectrograms contains multiple sets of time-varying spectrograms). One method is to estimate the reverberation time by measuring the attenuation slope of the signal at different time points. This can be achieved by linearly fitting the energy attenuation of each frequency component and then calculating the slope. The steeper the slope, the shorter the reverberation time; the flatter the slope, the longer the reverberation time. In addition, the resonance mode of the room can be determined by spectrum envelope analysis. This involves analyzing the spectrum energy of each time frame to identify areas where energy is concentrated at specific frequencies, which correspond to the resonant frequencies of the room. After extracting parameters such as reverberation time and resonance mode, these parameters are organized into a spatial acoustic feature matrix. In this matrix, each row represents a sampling point, and each column represents a specific acoustic parameter, such as reverberation time, sound pressure level distribution, and frequency response. In this way, a data set that comprehensively describes the acoustic characteristics of the environment is obtained, which contains key acoustic parameters at various locations in the environment.

[0131] Step S43: performing reverberation feature recognition on the spatial acoustic feature matrix to obtain a reverberation feature vector;

[0132] Specifically, relevant acoustic parameters, such as the reverberation time and frequency response of each sampling point, can be extracted from the spatial acoustic feature matrix. These parameters provide basic data on the behavior of sound waves in the environment. Next, autocorrelation analysis is used to determine the time distribution of reverberation. The autocorrelation function can reveal the correlation between the signal and itself at different time delays, thereby identifying the duration of the reverberation. By calculating the autocorrelation values ​​at different time delays, the decay curve of the reverberation can be estimated, and then the reverberation time can be obtained. In addition, in order to identify early reflections, the signal delays of different microphone channels can be compared. Early reflections refer to the sound waves reflected back after the sound waves first hit the room surface. They have an important impact on the sense of positioning and spatiality of the sound. By measuring the time difference of the early reflections received by different microphones, the distance between the sound source and each reflecting surface can be estimated, and then the geometric structure of the room can be inferred. After completing the above analysis, the obtained data is integrated into a reverberation feature vector. This vector contains key information describing the reverberation characteristics of the environment, such as reverberation time, reflection intensity, and the performance of reverberation at different frequencies. The reverberation time reflects the duration of a sound wave in an environment, the reflection intensity describes the strength of early reflections, and the frequency content of the reverberation reveals how the reverberation is distributed across different frequencies.

[0133] Step S44: performing acoustic ray tracing simulation on the spatial sampling point distribution map to obtain an acoustic energy distribution model, and extracting acoustic features and constructing a database for the reverberation feature vector based on the acoustic energy distribution model to obtain an environmental acoustic feature database;

[0134] Specifically, acoustic ray tracing simulations can be performed using professional acoustic simulation software, such as Odeon or Raynoise, which can simulate the propagation behavior of sound waves in three-dimensional space. First, use the acoustic simulation software to import a spatial sampling point distribution map, which contains the geometric structure of the room and the location information of the sampling points. Then, the software simulates the propagation path of the sound wave from the sound source to each sampling point according to the size and material properties of the room, including phenomena such as reflection, refraction and diffraction of the sound wave. During the simulation, the software tracks each interaction between the sound wave and the room interface and calculates the attenuation of the sound wave on different paths. This involves considering the reflection coefficient of the sound wave on the surface of different materials and the energy loss of the sound wave during propagation. In this way, the distribution of sound energy at various points in the room can be predicted to generate a detailed sound energy distribution model. Next, the sound energy distribution model is used to extract the acoustic parameters in the reverberation feature vector. For example, the energy attenuation of sound waves of different frequencies in the model can be analyzed to estimate the reverberation time of the room. In addition, the diffusion of sound waves in the room can be evaluated to calculate parameters such as the clarity and intensity of the room. Finally, these extracted acoustic parameters are organized into an environmental acoustic feature database, which records the room's reverberation time, clarity, intensity, and other relevant acoustic characteristics in detail.

[0135] Step S45: performing spectrum compensation on the phase-optimized audio data based on the environmental acoustic feature database to obtain an environmental compensation spectrum;

[0136] Specifically, an environmental acoustic feature database can be accessed, which contains detailed room acoustic parameters such as reverberation time, sound absorption coefficient, diffusion characteristics, etc. Next, a spectral analysis is performed on the phase-optimized audio data, which is achieved by fast Fourier transform (FFT) to convert the audio signal from the time domain to the frequency domain to obtain its spectral representation. The spectrum represents the energy distribution of the audio signal at different frequencies. According to the parameters in the environmental acoustic feature database, how the acoustic characteristics of the room affect the audio signal is mapped. For example, a room with a long reverberation time will attenuate the high-frequency components of the audio signal, while materials with a high sound absorption coefficient will reduce the energy of a specific frequency. Based on the above mapping, the spectrum of the audio signal is adjusted. If the reverberation time of the room is too long, the clarity of the speech is improved by enhancing the high-frequency components of the audio signal. This can be achieved through a high-pass filter or gain boost. On the contrary, if the reverberation time is short, the warmth of the sound is increased by enhancing the low-frequency components, which can be achieved through a low-pass filter or gain boost. In this way, an environmental compensation spectrum can be generated, which takes into account the impact of the environment on the audio signal, thereby providing a consistent listening experience in different environments.

[0137] Step S46: performing blind deconvolution on the environment compensation spectrum to obtain a dereverberation environment compensation spectrum, and performing spatial sound image reconstruction on the dereverberation environment compensation spectrum to obtain an audio space optimization spectrum;

[0138] Specifically, the ambient compensation spectrum can first be analyzed in detail to identify the characteristics of reverberation. This includes analyzing the delayed response and decay characteristics in the spectrum, which are typical signs of reverberation. The spectrum is converted to the time domain using Fourier transforms in order to identify the duration and intensity of the reverberation. A mathematical model is constructed to describe the reverberation process based on parameters in the ambient acoustic characteristics database, such as room dimensions, material properties, and known reverberation time. This model will predict the propagation and reflection behavior of sound waves in the room. Using the mathematical model of reverberation, a blind deconvolution algorithm is applied to try to reverse the reverberation process. Blind deconvolution is a signal processing technique that attempts to recover the original signal from the observed signal, even if the specific details of the reverberation are not known. Algorithms such as Wiener filtering or least squares are used to estimate and eliminate the effects of reverberation, thereby recovering the original, unaffected spectrum. Next, the dereverberated ambient compensation spectrum is spatially reimaged. This involves using sound field analysis techniques such as wave field synthesis or stereo processing techniques to adjust the spatial characteristics of the audio signal. Analyze the room's geometry and material properties, and then adjust the audio signal's left-right channel balance, stereo width, and depth accordingly. This can be achieved by adjusting the audio signal's delay, phase, and amplitude to simulate the effects of different sound source locations and environmental reflections. Finally, the adjusted audio signal is processed to output an audio space optimized spectrum.

[0139] Step S47: performing phase correction inverse transformation on the audio space optimized spectrum to obtain an audio time domain compensation signal, and performing loudness equalization on the audio time domain compensation signal to obtain final optimized audio data.

[0140] Specifically, the audio spatially optimized spectrum can be loaded and an inverse transform, such as an inverse Fourier transform, can be applied to convert the frequency domain signal back to the time domain signal. During this process, special attention needs to be paid to correcting the phase of the signal to ensure that the converted time domain signal has the correct timing and phase relationship. This involves analyzing the phase content of the spectrum and adjusting the phase as necessary to compensate for any phase distortion introduced by the previous processing steps. Next, the audio time domain compensated signal is subjected to loudness equalization. Loudness equalization techniques, such as dynamic range compression or multi-band compression, are used to adjust the loudness of the audio signal so that it maintains a consistent perceived loudness at different volume levels. The loudness of the audio signal is analyzed and the amplitude of the signal is adjusted based on a loudness model, such as the Zwicker loudness model, to improve the audibility of low-loudness parts while preventing distortion of high-loudness parts. Finally, the final optimized audio data will be output, which has been spatially optimized and loudness equalized while maintaining the original audio characteristics.

[0141] The present invention can accurately capture the acoustic characteristics of the environment and generate a detailed spatial sampling point distribution map through multi-point microphone array deployment and spatial sound field sampling positioning. By performing time-frequency recognition on multi-channel environmental response signals, comprehensive parameters of the environmental sound field can be extracted to form a spatial acoustic feature matrix. This helps to deeply understand the impact of the environment on the audio signal. The reverberation characteristics in the environment can be identified by obtaining the reverberation feature vector, which is crucial for adjusting the audio signal to adapt to different environments. This recognition helps to reduce the negative impact of reverberation on sound quality. Based on the sound energy distribution model and the environmental acoustic feature database, the audio data can be spectrally compensated to adapt to different acoustic environments. This adaptability enables the audio output to maintain high sound quality in different environments. Through blind deconvolution and spatial sound image reconstruction, the influence of reverberation can be removed and the spatial performance of audio can be optimized. This makes the positioning of audio signals in space more accurate, enhancing the listener's sense of immersion and sense of space. Phase correction inverse transformation and loudness equalization processing help to improve the clarity and stability of audio signals. After these processes, the audio signal can adapt to different volume levels while maintaining its original quality, providing a consistent auditory experience.

[0142] Preferably, the present invention further provides a system for processing Bluetooth speaker data, which is used to execute the method for processing Bluetooth speaker data as described above, and the system for processing Bluetooth speaker data comprises:

[0143] A data compression module is used to capture multimodal data of a client to obtain original multimodal client data; and to perform adaptive multi-resolution compression on the original multimodal client data to obtain compressed multimodal client data;

[0144] The audio reconstruction module is used to transmit the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; perform multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; perform dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data;

[0145] The audio phase optimization module is used to determine the music preference type of the user and obtain the user's music preference type identifier; perform phase response on the reconstructed audio data according to the user's music preference type identifier to obtain phase optimized audio data;

[0146] The environmental acoustic compensation module is used to construct an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; perform acoustic environment compensation on the phase optimized audio data according to the environmental acoustic feature database to obtain the final optimized audio data.

[0147] Preferably, a computer-readable storage medium stores a method for processing Bluetooth speaker data as described above which can be loaded and executed by a processor.

[0148] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0149] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for processing Bluetooth speaker data, characterized in that: The following steps are involved: Step S1: Capturing multimodal data of the client to obtain original multimodal client data; Performing adaptive multi-resolution compression on original multimodal client data to obtain compressed multimodal client data; Step S2: transmitting the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; performing multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; performing dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data; Step S3: determining the music preference type of the user to obtain a user music preference type identifier; performing phase response on the reconstructed audio data according to the user music preference type identifier to obtain phase optimized audio data; Step S4: constructing an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; performing acoustic environment compensation on the phase-optimized audio data according to the environmental acoustic feature database to obtain final optimized audio data.

2. The method for processing Bluetooth speaker data according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Capturing multimodal data of the client to obtain original multimodal client data; Step S12: performing modal feature separation on the original multimodal client data to obtain client multimodal feature data; wherein the client multimodal feature data includes audio data, control instruction data and device status data; Step S13: performing run-length encoding and differential pulse encoding on the control instruction data to obtain compressed control instruction data; Step S14: performing incremental encoding based on change detection on the device status data to obtain compressed device status data; Step S15: performing multi-scale signal decomposition on the audio data to obtain multi-scale audio data of the client, and performing wavelet transform on the multi-scale audio data of the client to obtain feature-enhanced audio data; Step S16: performing enhanced Huffman coding on the feature-enhanced audio data to obtain compressed audio data; Step S17: performing embedded multimodal fusion on the compressed control instruction data, the compressed device status data and the compressed audio data to obtain compressed multimodal client data.

3. The method for processing Bluetooth speaker data according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: The client transmits the compressed multimodal client data to the Bluetooth speaker and receives the data to obtain received compressed multimodal data; Step S22: performing multimodal data decoupling on the received compressed multimodal data to obtain separated audio data, and performing time-frequency feature extraction on the separated audio data to obtain audio time-frequency feature data; Step S23: adjusting the preset initial analysis window length according to the audio time-frequency feature data to obtain an adjustment window length parameter; Step S24: performing short-time Fourier transform on the separated audio data according to the adjusted window length parameter to obtain an initial spectrum diagram; Step S25: performing key frequency band identification on the initial frequency spectrum to obtain audio key frequency band data, and performing high frequency spectrum density analysis on the audio key frequency band in the separated audio data according to the audio key frequency band data to obtain key frequency band enhanced time-frequency spectrum data; Step S26: updating the initial spectrum graph according to the spectrum data of the key frequency band enhancement to obtain an optimized spectrum graph; Step S27: Performing harmonic fundamental frequency analysis on the optimized frequency spectrum to obtain fundamental frequency harmonic structure data, and reconstructing the fundamental frequency harmonic structure data to obtain reconstructed audio data.

4. The method for processing Bluetooth speaker data according to claim 3, characterized in that: Step S23 includes the following steps: Step S231: performing spectral centroid identification on the audio time-frequency feature data to obtain a spectral centroid distribution diagram, and performing time-varying spectral feature identification on the audio time-frequency feature data according to the spectral centroid distribution diagram to obtain an audio time-varying spectral feature matrix; Step S232: identifying the main frequency period of the audio time-frequency feature data according to the audio time-varying spectrum feature matrix to obtain a main frequency period sequence; Step S233: Calculate the time scale ratio of the main frequency period sequence and the preset initial analysis window length to obtain the window-signal time scale ratio coefficient; Step S234: performing transient capture performance evaluation on the preset initial analysis window length according to the window-signal time scale ratio coefficient to obtain a transient capture performance index; Step S235: adjusting the preset initial analysis window length according to the transient capture performance index to obtain a window length adjustment factor, and performing dynamic range compression on the preset initial analysis window length according to the window length adjustment factor to obtain a preliminary adjusted window length; Step S236: Binary quantize the preliminary adjustment window length to obtain an adjustment window length parameter.

5. The method for processing Bluetooth speaker data according to claim 3, characterized in that: Step S25 includes the following steps: Step S251: performing spectrum energy density calculation on the initial spectrum graph to obtain a spectrum energy density distribution graph, and performing spectrum peak detection on the initial spectrum graph to obtain a spectrum peak feature set; Step S252: determining the key frequency band energy threshold of the initial frequency spectrum according to the frequency spectrum energy density distribution diagram and the spectrum peak feature set to obtain the key frequency band energy threshold; Step S253: marking and merging high-energy frequency bands of the initial spectrum according to the key frequency band energy threshold to obtain preliminary key frequency band mapping data; Step S254: performing auditory weighted filtering on the initial spectrogram to obtain an auditory weighted spectrogram; Step S255: performing perceptual importance scoring on the preliminary key frequency band mapping data according to the auditory weighted spectrogram to obtain a key frequency band importance matrix; Step S256: clustering the key frequency band importance matrix to obtain an optimized key frequency band set, and performing adaptive band-pass filtering on the optimized key frequency band set to obtain a key frequency band enhanced audio signal; Step S257: performing time-frequency transformation on the key-band enhanced audio signal to obtain key-band enhanced time-frequency spectrum data.

6. The method for processing Bluetooth speaker data according to claim 3, characterized in that: Step S27 includes the following steps: Step S271: performing logarithmic spectrum enhancement on the optimized spectrum graph to obtain an enhanced logarithmic spectrum graph; Step S272: performing cepstrum fundamental frequency estimation on the enhanced logarithmic spectrum to obtain an audio fundamental frequency estimation sequence, and performing theoretical harmonic frequency calculation on the optimized spectrum based on the audio fundamental frequency estimation sequence to obtain a theoretical harmonic frequency map; Step S273: performing harmonic peak detection on the optimized spectrum diagram according to the theoretical harmonic frequency diagram to obtain an actual harmonic feature set; Step S274: performing harmonic energy ratio calculation on the actual harmonic feature set to obtain a harmonic energy ratio matrix, and performing time-varying harmonic structure extraction on the actual harmonic feature set to obtain a time-varying harmonic structure sequence; Step S275: generating harmonic structure data for the harmonic energy ratio matrix and the time-varying harmonic structure sequence according to the enhanced logarithmic spectrum to obtain fundamental frequency harmonic structure data; wherein the fundamental frequency harmonic structure data includes fundamental frequency trajectory data, harmonic frequency sequence and harmonic intensity matrix; Step S276: identifying the phase relationship between the fundamental frequency trajectory data and the harmonic frequency sequence to obtain a harmonic phase relationship diagram, and performing harmonic space redistribution on the fundamental frequency harmonic structure data according to the harmonic phase relationship diagram and the harmonic intensity matrix to obtain redirected harmonic structure data; Step S277: performing harmonic synthesis on the redirected harmonic structure data to obtain a preliminary reconstructed audio signal, and performing envelope modulation on the preliminary reconstructed audio signal to obtain an envelope modulated audio signal; Step S278: performing phase correction on the envelope modulated audio signal to obtain reconstructed audio data.

7. The method for processing Bluetooth speaker data according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: collecting historically played tracks from the client to obtain a historically played track data set, and identifying the audio playing mode of the user based on the historically played track data set to obtain user audio playing feature data; wherein the user audio playing feature data includes user style preference distribution data, user playing time mode and user music element preference data; Step S32: constructing a user profile of the user according to the user style preference distribution data, the user playback time pattern and the user music element preference data to obtain a user audio profile, and using a preset preference recognition model to recognize the user preference type of the user audio profile to obtain a user music preference type identifier; wherein the user music preference type identifier is any one of a rhythm-oriented listener, a melody-oriented listener and an atmosphere-oriented listener; Step S33: if the user's music preference type is identified as a rhythm-oriented listener, low-frequency phase consistency enhancement is performed on the reconstructed audio data to obtain rhythm-enhanced audio data; Step S34: if the user's music preference type is identified as a melody-oriented listener, the mid-high frequency phase relationship of the reconstructed audio data is enhanced to obtain melody-prominent audio data; Step S35: If the user's music preference type is identified as an atmosphere-oriented listener, full-band phase spatial enhancement is performed on the reconstructed audio data to obtain spatially enhanced audio data; Step S36: performing phase modulation processing on the reconstructed audio data according to the rhythm-enhanced audio data, the melody-emphasized audio data, or the spatial-enhanced audio data to obtain phase-optimized audio data.

8. The method for processing Bluetooth speaker data according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: deploy a multi-point microphone array in the environment where the Bluetooth speaker is located, and perform spatial sound field sampling and positioning to obtain a spatial sampling point distribution map, perform wide-band frequency sweep excitation on the environment where the Bluetooth speaker is located through the microphone array, and collect response signals to obtain a multi-channel environmental response signal, perform time-frequency recognition on the multi-channel environmental response signal, and obtain a time-varying spectrum atlas; Step S42: extracting spatial acoustic parameters of the ambient sound field based on the time-varying spectrum atlas to obtain a spatial acoustic feature matrix; Step S43: performing reverberation feature recognition on the spatial acoustic feature matrix to obtain a reverberation feature vector; Step S44: performing acoustic ray tracing simulation on the spatial sampling point distribution map to obtain an acoustic energy distribution model, and extracting acoustic features and constructing a database for the reverberation feature vector based on the acoustic energy distribution model to obtain an environmental acoustic feature database; Step S45: performing spectrum compensation on the phase-optimized audio data based on the environmental acoustic feature database to obtain an environmental compensation spectrum; Step S46: performing blind deconvolution on the environment compensation spectrum to obtain a dereverberation environment compensation spectrum, and performing spatial sound image reconstruction on the dereverberation environment compensation spectrum to obtain an audio space optimization spectrum; Step S47: performing phase correction inverse transformation on the audio space optimized spectrum to obtain an audio time domain compensation signal, and performing loudness equalization on the audio time domain compensation signal to obtain final optimized audio data.

9. A system for processing Bluetooth speaker data, characterized in that: A method for performing Bluetooth speaker data processing as claimed in claim 1, wherein the Bluetooth speaker data processing system comprises: A data compression module is used to capture multimodal data of a client to obtain original multimodal client data; and to perform adaptive multi-resolution compression on the original multimodal client data to obtain compressed multimodal client data; The audio reconstruction module is used to transmit the compressed multimodal client data to the Bluetooth speaker through the client to obtain received compressed multimodal data; perform multimodal data decoupling on the received compressed multimodal data to obtain separated audio data; perform dynamic spectrum reconstruction optimization on the separated audio data to obtain reconstructed audio data; The audio phase optimization module is used to determine the music preference type of the user and obtain the user's music preference type identifier; perform phase response on the reconstructed audio data according to the user's music preference type identifier to obtain phase optimized audio data; The environmental acoustic compensation module is used to construct an environmental acoustic feature database for the environment where the Bluetooth speaker is located to obtain an environmental acoustic feature database; perform acoustic environment compensation on the phase optimized audio data according to the environmental acoustic feature database to obtain the final optimized audio data.

10. A computer-readable storage medium storing a method for processing Bluetooth speaker data as claimed in any one of claims 1 to 8 that can be loaded and executed by a processor.

Images