A music search method and system based on big data

Abstract

This invention discloses a music search method and system based on big data. It obtains sentiment characteristics by processing user historical playback data using a Natural Language Processing (NLP) model; acquires real-time semantic features based on real-time user input search context data using a BRET model; dynamically adjusts the matching weights of audio features, sentiment characteristics, and real-time semantic features using a reinforcement learning model; adjusts the feature weights based on the matching results to obtain fused features; inputs the fused features into an MTNN multi-task neural network, fuses the outputs of each branch through an attention mechanism to generate a music matching score, and generates a music recommendation list based on the music matching score. This improves the user search experience and platform conversion efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information retrieval, in particular to a music search method and system based on big data. BACKGROUND

[0002] With the outbreak of the digital music industry, the size of the online music library has reached millions, and users' demand for accurate search is increasingly urgent. Traditional music search technology mainly relies on keyword matching and single audio feature extraction. Keyword search relies on accurate user input, is difficult to understand ambiguous semantics, and cannot capture implicit requirements such as context and emotion. Audio feature extraction focuses on time-frequency domain information, ignoring key dimensions such as phonetic sequence and energy peak, resulting in insufficient similarity style recognition accuracy. It does not integrate user dynamic requirements and only makes static recommendations based on historical data, which cannot adapt to real-time scenario changes. At the same time, users' demand for personalized experience is increasing, and the click-through rate and play completion rate of existing technologies are low. SUMMARY

[0003] The purpose of the present application is to solve the above problems, and a music search method and system based on big data are designed.

[0004] To achieve the above purpose, the technical scheme of the present application is as follows: further, in the music search method based on big data, the music search method based on big data comprises the following steps:

[0005] Frame processing is performed on the audio data of the database to extract time-frequency domain features, phonetic sequences and energy peak point pairs to obtain audio features. User historical playback data is processed by an NLP natural language model to obtain emotional tendency features;

[0006] Based on the search context data input by the user in real time, real-time semantic features are obtained through a BRET model;

[0007] The matching weights of the audio features, emotional tendency features and real-time semantic features are dynamically adjusted using a reinforcement learning model. The fusion features are obtained by adjusting the feature weights according to the matching results;

[0008] The fusion features are input into an MTNN multi-task neural network, and the outputs of each branch are fused through an Attention attention mechanism to generate a music matching score. A music recommendation list is generated according to the music matching score.

[0009] Further, in the music search method based on big data, the frame processing of the audio data of the database to extract time-frequency domain features, phonetic sequences and energy peak point pairs to obtain audio features, and processing user historical playback data by an NLP natural language model to obtain emotional tendency features, comprise:

[0010] The audio data in the database is preprocessed to remove broken, noisy, and incompatible files, while retaining standard audio data. An overlapping framing strategy is used, with a frame length of 20ms and a frame shift of 10ms. A Hanning window is then used for windowing to obtain the preprocessed audio data.

[0011] The preprocessed audio data is subjected to FFT (Fast Fourier Transform) to obtain a spectrogram. Based on the spectrogram, Mel frequency cepstral coefficients, Mel spectral features, and time-domain statistical features are extracted. The three types of features are concatenated to obtain a multidimensional time-frequency domain feature vector.

[0012] The preprocessed audio data is denoised and endpoints are detected to locate valid audio segments. A pre-trained acoustic model is loaded using the Kaldi speech recognition toolkit to map audio frames into phoneme candidates. Repeated phonemes are removed and the sequence is optimized using the CTC connection temporal classification decoding algorithm to obtain the phonetic symbol sequence for each audio.

[0013] Peak detection is performed on the short-time energy sequence of the preprocessed audio data. The peak threshold is set to 1.5 times the average energy of the audio. Adjacent peak points are paired in time order. The time interval, energy difference and energy ratio between the pairs are calculated to obtain a multidimensional peak feature vector.

[0014] Furthermore, in the aforementioned music search method based on big data, the audio data in the database is processed by frame segmentation to extract time-frequency domain features, phonetic symbols, and energy peak point pairs to obtain audio features. Then, user historical playback data is processed using an NLP (Natural Language Processing) model to obtain sentiment characteristics, including:

[0015] Collect users' historical playback data over the past 90 days, including played tracks, playback duration, favorites behavior, download behavior, comment behavior, and skip frequency, to obtain users' historical playback data;

[0016] A natural language processing (NLP) model is built based on a fine-tuned BERT-base-chinese model. Preprocessed user history playback data is input into the NLP model, which outputs three types of sentiment probabilities: positive, neutral, and negative. By weighted fusion of behavioral sentiment and comment sentiment, sentiment tendency features are obtained.

[0017] Furthermore, in the aforementioned music search method based on big data, the step of obtaining real-time semantic features based on the user's real-time input search context data through the BRET model includes:

[0018] The system acquires real-time search context data from the user, cleans and standardizes the search context data to obtain a real-time text sequence, inputs the real-time text sequence into a fine-tuned BERT model, takes the output vector of the last layer of the model, performs Layer Norm standardization, and then performs dimensionality reduction through a fully connected layer to obtain real-time semantic features.

[0019] Furthermore, in the aforementioned music search method based on big data, the method utilizes a reinforcement learning model to dynamically adjust the matching weights of audio features, sentiment features, and real-time semantic features, and adjusts the feature weights based on the matching results to obtain fused features, including:

[0020] The matching degree between the three types of features and the music features in the database is calculated by using cosine similarity. The reinforcement learning model agent adjusts the action by outputting weights according to the current state vector. After updating the weights, the fusion matching degree is calculated. The three types of features are weighted and summed according to the adjusted weights to obtain the fused features.

[0021] Furthermore, in the aforementioned music search method based on big data, the step of inputting the fused features into an MTNN multi-task neural network, fusing the outputs of each branch through an attention mechanism to generate a music matching score, and generating a music recommendation list based on the music matching score includes:

[0022] The outputs of the three branches of the MTNN multi-task neural network are used as inputs to the attention mechanism, and the attention weights of each branch are calculated: the outputs of each branch are converted into attention scores through a fully connected layer;

[0023] The attention score is normalized using the Softmax function to obtain the attention weight. The branch outputs are then weighted and fused according to the attention weight to obtain the fusion vector.

[0024] Furthermore, in the aforementioned music search method based on big data, the step of inputting the fused features into an MTNN multi-task neural network, fusing the outputs of each branch through an attention mechanism to generate a music matching score, and generating a music recommendation list based on the music matching score includes:

[0025] The fused vector is input into the output layer of the MTNN and mapped to the [0,1] interval by the Sigmoid activation function to obtain the music matching score;

[0026] The top 100 songs are filtered in descending order of matching score. A greedy algorithm is used to ensure that the genre coverage of the recommended list is greater than 50%. The top 20 songs are then used to generate a music recommendation list, which includes the matching score.

[0027] Furthermore, in a big data-based music search system, the big data-based music search system includes the following modules:

[0028] The audio feature extraction module is used to perform frame-by-frame processing on the audio data in the database, extract time-frequency domain features, phonetic symbol sequences, and energy peak point pairs to obtain audio features, and process the user's historical playback data through an NLP natural language model to obtain sentiment characteristics.

[0029] The semantic feature extraction module is used to obtain real-time semantic features based on the search context data input by the user in real time, through the BRET model;

[0030] The feature vector fusion module is used to dynamically adjust the matching weights of audio features, sentiment features, and real-time semantic features using a reinforcement learning model. Based on the matching results, the feature weights are adjusted to obtain fused features.

[0031] The music matching and recommendation module is used to input the fused features into the MTNN multi-task neural network, fuse the outputs of each branch through the Attention mechanism, generate a music matching score, and generate a music recommendation list based on the music matching score.

[0032] Furthermore, in a music search system based on big data, the music matching and recommendation module includes the following sub-modules:

[0033] The computation submodule is used to take the outputs of the three branches of the MTNN multi-task neural network as input to the attention mechanism and calculate the attention weights of each branch: the outputs of each branch are converted into attention scores through a fully connected layer;

[0034] The fusion submodule is used to normalize the attention score using the Softmax function to obtain the attention weights, and then weight and fuse the branch outputs according to the attention weights to obtain the fusion vector.

[0035] Furthermore, in a music search system based on big data, the music matching and recommendation module includes the following sub-modules:

[0036] The input submodule is used to input the fusion vector into the MTNN output layer, which is then mapped to the [0,1] interval by the Sigmoid activation function to obtain the music matching score.

[0037] The matching submodule is used to filter the top 100 music tracks in descending order of matching score. It uses a greedy algorithm to ensure that the genre coverage of the recommended list is greater than 50%, and takes the top 20 music tracks to generate a music recommendation list with matching scores.

[0038] Its beneficial effects are as follows: First, by integrating multiple technologies, it breaks through traditional bottlenecks, resulting in more comprehensive feature extraction. Frame-by-frame processing combines time-frequency domain, phonetic sequence, and energy peak features with NLP-analyzed sentiment tendencies to construct a multi-dimensional feature system, improving audio similarity recognition accuracy by more than 35%. Second, it achieves accurate real-time semantic understanding. Based on a fine-tuned BERT model, it captures search context and updates semantic features within 500ms, improving fuzzy search matching accuracy and effectively responding to scenario-based needs. Third, it exhibits strong dynamic weight adaptability. The reinforcement learning model optimizes weights based on user feedback, making feature fusion align with real-time preferences, increasing click-through rates for new user recommendations and playback completion rates for existing users. It balances matching accuracy and efficiency. MTNN, combined with the Attention mechanism, integrates multi-task outputs. Supported by the Spark distributed framework, the response time for retrieving 100,000 audio tracks is ≤300ms, balancing large-scale data processing and accurate recommendations, improving user search experience and platform conversion efficiency. Attached Figure Description

[0039] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0040] Figure 1 This is a schematic diagram of the first embodiment of a music search method based on big data in this invention.

[0041] Figure 2 This is a schematic diagram of a second embodiment of a music search method based on big data in this invention.

[0042] Figure 3 This is a schematic diagram of the first embodiment of a music search system based on big data in this invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0045] The present invention will now be described in detail with reference to the accompanying drawings.Figure 1 As shown, a music search method based on big data includes the following steps:

[0046] Step 101: Perform frame segmentation on the audio data in the database, extract time-frequency domain features, phonetic symbols, and energy peak point pairs to obtain audio features, and process the user's historical playback data through an NLP natural language model to obtain emotional tendency features.

[0047] Specifically, in this embodiment, the audio data in the database is preprocessed, removing corrupted, noisy, and incompatible files, retaining standard audio data, and using an overlapping framing strategy with a frame length of 20ms and a frame shift of 10ms. A Hanning window is then used for windowing to obtain preprocessed audio data. An FFT (Fast Fourier Transform) is performed on the preprocessed audio data to obtain a spectrogram. Based on the spectrogram, Mel frequency cepstral coefficients, Mel spectral features, and time-domain statistical features are extracted. These three types of features are then concatenated to obtain a multidimensional time-frequency domain feature vector. The preprocessed audio data... Noise reduction and endpoint detection are performed to locate valid audio segments. A pre-trained acoustic model is loaded using the Kaldi speech recognition toolkit to map audio frames as phoneme candidates. A CTC-connected temporal classification decoding algorithm is used to remove duplicate phonemes and optimize the sequence, resulting in a phonetic sequence for each audio track. Peak detection is performed on the short-time energy sequence of the preprocessed audio data, setting a peak threshold of 1.5 times the average energy of the audio. Adjacent peak points are paired in chronological order, and the time interval, energy difference, and energy ratio between the pairs are calculated to obtain a multi-dimensional peak feature vector. Historical playback data from the past 90 days is collected, including played tracks, playback duration, favorites, downloads, comments, and skip frequency, to obtain the user's historical playback data. An NLP (Natural Language Processing) model is built based on a fine-tuned BERT-base-chinese model. The preprocessed historical playback data is input into the NLP model, which outputs three types of sentiment probabilities: positive, neutral, and negative. Sentiment tendency features are obtained by weighted fusion of behavioral sentiment and comment sentiment.

[0048] Audio data preprocessing and frame segmentation are the primary steps in feature extraction. Their core value lies in eliminating data noise and standardizing data formats, providing standardized input for subsequent feature extraction. During the data screening and cleaning stage, audio files with a signal-to-noise ratio (SNR) below 20dB are removed because when the SNR is below this threshold, environmental noise severely interferes with the audio's spectral characteristics, leading to deviations in subsequent time-frequency domain feature extraction and affecting similarity matching accuracy. The standard of MP3 and WAV formats, a 44.1kHz sampling rate, and a 16-bit depth is chosen because these are currently the mainstream lossless audio formats. A 44.1kHz sampling rate can fully cover the audible frequency range of 20Hz-20kHz, and a 16-bit depth ensures sufficient dynamic range, preventing audio information loss. The configuration of the framing parameters has been verified through a large number of experiments. The 20ms frame length can ensure that each frame contains enough audio information to reflect local features, while avoiding excessive redundancy of information within the frame. The 10ms frame shift achieves 50% frame overlap, which can effectively solve the problem of discontinuity of information between frames and prevent feature breakage caused by framing. Hanning window is used for windowing processing.

[0049] In time-frequency domain feature extraction, the Fast Fourier Transform (FFT) converts the time-domain audio signal into a frequency-domain spectrogram, completing the time-to-frequency dimension conversion and providing a foundation for subsequent feature extraction. Mel-frequency cepstral coefficients (MFCCs) are selected using a combination of 13-dimensional coefficients, first-order differences, and second-order differences. The 13-dimensional MFCCs effectively reflect the spectral envelope characteristics of the audio, while the first and second-order differences supplement the dynamic changes in the audio, improving the temporal continuity of the features. The setting of 40 Mel filter banks closely matches the characteristics of human hearing, as the Mel scale has a linear relationship with human subjective frequency perception, better simulating the sensitivity of the human ear to different frequencies. The introduction of time-domain statistical features such as short-time energy and zero-crossing rate supplements the time-domain information that frequency-domain features cannot cover. Short-time energy reflects the intensity changes of audio, the zero-crossing rate can be used to distinguish between speech and instrumental audio, and features such as spectral centroid and spectral bandwidth can characterize the distribution characteristics of the spectrum. The phonetic symbol sequence extraction adopts a complete process of preprocessing, acoustic modeling, and sequence decoding. In the preprocessing stage, Wiener filtering further improves audio quality by estimating and suppressing noise. The dual-threshold method can accurately locate effective audio segments, eliminate silent parts, and reduce invalid calculations. The Kaldi speech recognition toolkit is used to load a pre-trained acoustic model with a CNN+LSTM architecture. Kaldi has a mature model framework in the field of speech recognition. CNN can effectively extract local spectral features of audio, while LSTM can capture the temporal dependencies of audio. The combination of the two improves the accuracy of phoneme mapping. The application of the CTC decoding algorithm solves the alignment problem between phonemes and audio frames, effectively removes repeated phonemes, and optimizes the accuracy of the phonetic symbol sequence.

[0050] Historical data collection selects the playback data of the past 90 days. This time span can cover the user's recent emotional preferences and avoid the problem of outdated preferences caused by too long time. The multi-dimensional behavioral data such as playback duration, collection, and comments collected can comprehensively reflect the user's emotional attitude towards music. Playback duration > 90% of the track length is marked as positive feedback. Complete playback and a high duration ratio usually indicate the user's love for the music. Playback < 30% (i.e., skipped) is marked as negative feedback, which means the user has a low acceptance of the music. For the processing of review texts, Jieba word segmentation combined with stop word removal is adopted. Jieba word segmentation has a high accuracy in the field of Chinese word segmentation. Stop word removal can remove meaningless words such as "le" and improve the efficiency of text feature extraction. The NLP model selects the fine-tuned BERT-base-chinese model. The BERT model has strong context semantic understanding ability. By supplementing review data in the music field for fine-tuning, the recognition accuracy of the model for music-related emotional words can be improved. In the calculation of emotional tendency, the Softmax activation function outputs three types of emotional probabilities to ensure the probability normalization of the output results. The weighted fusion method of behavioral emotion (weight 0.7) and review emotion (weight 0.3) is adopted. Behavioral data can more directly reflect the user's real emotional preferences. There may be subjective expression biases in review texts. This weight allocation has been verified by a large amount of user data and can achieve accurate quantification of emotional tendency, and finally be converted into a 32-dimensional feature vector for easy fusion with subsequent audio features and semantic features.

[0051] Step 102: Based on the search context data input by the user in real time, obtain real-time semantic features through the BRET model;

[0052] Specifically, in this embodiment, the search context data input by the user in real time is obtained, and the search context data is cleaned and standardized to obtain a real-time text sequence. The real-time text sequence is input into the fine-tuned BERT model, and the output vector of the last layer of the model is taken. After being standardized by Layer Norm, it is dimension-reduced through a fully connected layer to obtain real-time semantic features.

[0053] Comprehensively collect relevant context data during the user's real-time search process. The core data includes the search keywords input by the user, such as song names, singer names, music genre descriptions, lyric fragments, etc., input time sequence, the order and pause intervals of keyword input. The auxiliary data includes the current search scene information, such as search device type, network environment, current time, geographical location, search trigger method, active input, speech-to-text conversion, historical search recommendation clicks, and real-time interaction behaviors, such as keyword deletion, replacement, and relevant recommendation browsing records during the search process;

[0054] Clean and standardize the collected search context data. First, filter out invalid data, removing meaningless characters, duplicate inputs, and interference information, such as mis-entered garbled characters and spaces. Second, standardize text data, including unifying case, Chinese word segmentation. Use the jieba word segmentation tool and optimize the word segmentation effect by combining with a music domain dictionary. For example, correctly split "ancient style folk songs" into "ancient style" and "folk songs" instead of individual words, and remove stop words, such as words without actual semantic meaning like "de", "le", "he", etc. Finally, unify the formats of time series data and scenario data, convert time to the standard timestamp format, geographical locations to administrative division codes, and classification data such as device types and network environments to unified enumeration values, forming a structured context data set.

[0055] Model customization and fine-tuning are the key guarantees for the accuracy of semantic extraction. Select the BERT-base model as the basic framework. Benefiting from the powerful context semantic modeling ability of its Transformer architecture, supplement 500,000 music search logs, 200,000 music reviews and label data to build a domain corpus, which solves the problem of insufficient semantic understanding of general pre-trained models in music scenarios, enabling the model to accurately identify music-specific emotional words such as healing and exciting. The fine-tuning parameters are optimized through multiple rounds of experiments: a learning rate of 2e-5 can prevent the model from oscillating due to too high a learning rate, a batch size of 32 takes into account both gradient stability and computational efficiency, 10 training rounds enable the model to fully learn domain features and avoid overfitting, and the cross-entropy loss function adapts to the optimization requirements of semantic classification tasks. In the semantic vector generation step, take the 768-dimensional output vector of the last layer [CLS] mark of the model. This vector aggregates the semantic information of the entire input sequence and is an effective quantification of the overall intention of the text. Layer Norm standardization eliminates the dimensional difference between vector dimensions and ensures the consistency of feature distribution. The fully connected layer reduces the dimension to 64, which not only reduces the subsequent computational overhead but also keeps consistent with the audio feature dimension, laying a foundation for feature fusion. The design of the feature real-time update mechanism fully adapts to the dynamic needs of users. When the user modifies keywords or the scenario switches from bedtime to exercise, the quick update response within 500ms ensures that semantic features can match the demand changes in a timely manner, avoiding matching deviations caused by feature lag. The real-time guarantee enables the search system to dynamically adapt to the flow of user intentions.

[0056] Step 103: Use the reinforcement learning model to dynamically adjust the matching weights of audio features, emotional tendency features, and real-time semantic features, and adjust the feature weights according to the matching results to obtain fused features.

[0057] Specifically, in this embodiment, the matching degree between the three types of features and the music features in the database is calculated by cosine similarity. The reinforcement learning model agent adjusts the action by outputting weights according to the current state vector. After updating the weights, the fusion matching degree is calculated. The three types of features are weighted and summed according to the adjusted weights to obtain the fused features.

[0058] In the reinforcement learning model construction phase, the definition of core elements closely aligns with the characteristics of the music search scenario: the agent, as the decision-making body for weight adjustment, directly determines the fusion effect through its decision quality; the 48-dimensional state vector covers the matching degree of three types of features—audio, emotion, and semantic—as well as feedback from the last five recommendation clicks, comprehensively reflecting the current matching status and user preferences, providing sufficient basis for decision-making; the action space is designed as a set of weight increments of ±0.05 and ±0.1, ensuring both the flexibility of weight adjustment and avoiding abrupt matching changes due to excessive magnitude, while adhering to the constraint w1+w2+w3=1; the reward mechanism is based on CTR and playback completion rate, with differentiated scores: +10 for clicks, +20 for complete playback, and -5 for no clicks, ensuring that the rewards accurately reflect the recommendation effect and guide the agent to optimize towards higher click-through rates and higher completion rates. The model architecture uses DQN (Deep Q-Network), whose experience replay pool solves the sample correlation problem, and the target network updated every 100 steps improves training stability. The structure of 48-dimensional input - 128-dimensional ReLU hidden layer - 64-dimensional ReLU hidden layer - action space dimension output can fully fit the complex mapping relationship between state and action.

[0059] The multi-feature matching and weight initialization process provides a foundation for dynamic adjustment: cosine similarity is used to calculate the matching degree because it can effectively measure the directional consistency of high-dimensional vectors and is suitable for feature similarity evaluation needs; music emotion tags are pre-annotated by professional annotators, and text tags cover dimensions such as music style, ensuring the accuracy of the matching benchmark; the initial weights w1=0.4, w2=0.3, and w3=0.3 are set based on domain experience, highlighting the fundamental position of audio features, while the adjustment to w1=0.5 for new users takes into account that the reliability of audio features is higher when there is no historical data. Iterative weight optimization achieves adaptive adjustment through a closed loop of decision-execution-feedback-learning: After the agent outputs weight adjustment actions based on the state vector, it calculates the fusion matching degree S=w1*S1+w2*S2+w3*S3, and then stores the weights, states, and user feedback rewards in the experience replay pool to provide data for model training; the parameters are updated after processing every 100 search requests, enabling the model to learn new user behavior patterns in a timely manner. When the younger user group recently prefers popular music styles, the model will gradually increase the weight of music style-related dimensions in the semantic features. In the fusion feature generation stage, the weighted summation formula F=w1*audio features+w2*sentiment tendency features+w3*real-time semantic features achieves the organic fusion of the three types of features. L2 normalization processing unifies the norm of the fusion features, improving the consistency of subsequent model inputs. Finally, the 64-dimensional fusion features not only retain the core information of each dimension, but also adapt to user personalization and real-time needs through dynamic weights, providing high-quality input for subsequent matching score calculation.

[0060] Step 104: Input the fused features into the MTNN multi-task neural network, fuse the outputs of each branch through the Attention mechanism to generate a music matching score, and generate a music recommendation list based on the music matching score.

[0061] Specifically, in this embodiment, the outputs of the three branches of the MTNN multi-task neural network are used as inputs to the attention mechanism, and the attention weights of each branch are calculated: the outputs of each branch are converted into attention scores through a fully connected layer; the attention scores are normalized using the Softmax function to obtain attention weights, and the branch outputs are weighted and fused according to the attention weights to obtain a fusion vector. The fusion vector is input into the MTNN output layer, mapped to the [0,1] interval by the Sigmoid activation function to obtain the music matching score; the top 100 music tracks are selected in descending order of matching score, and a greedy algorithm is used to ensure that the genre coverage in the recommendation list is greater than 50%, and the top 20 music tracks are selected to generate a music recommendation list with matching scores.

[0062] The core objective of music matching score calculation and recommendation list generation (based on MTNN and Attention) is to transform fused features into accurate matching scores and generate a recommendation list tailored to user needs through intelligent sorting. The key technological highlight lies in the collaborative optimization of MTNN multi-task modeling and the Attention mechanism. The MTNN architecture employs a shared layer + branch layer + fusion layer structure to achieve synergistic improvement in multi-dimensional matching objectives: the shared layer extracts general features through a fully connected layer that transforms 64-dimensional input into 32-dimensional output, and the ReLU activation function injects non-linear expressive power, reducing parameter redundancy while enhancing feature reuse; the branch layer sets up three parallel task branches, corresponding to audio, sentiment, and semantic matching respectively. Each branch progressively extracts task-specific features using a two-layer fully connected network (32-dimensional → 16-dimensional → 8-dimensional), ensuring both task independence and feature sharing. Training parameters have been validated and optimized through engineering: The Adam optimizer improves convergence efficiency by leveraging its adaptive learning rate, with a learning rate of 5e-4, a batch size of 64, and 50 training rounds balancing training effectiveness and efficiency; the loss function uses a weighted sum of the MSE losses of the three tasks, logically corresponding to the feature weights, ensuring that the score has the smallest error relative to the actual needs. The Attention mechanism strengthens the contribution of key features, taking the 8-dimensional output of the three branches as input, transforming it into a 1-dimensional score through a fully connected layer, and then normalizing it with Softmax to obtain the weights. For example, when a user searches for sad music, the weight α2 of the emotion branch is automatically increased, and the weighted fusion generates a 16-dimensional vector. The matching score is mapped to the [0,1] interval by Sigmoid. The recommendation list is optimized by the process of initial screening of the first 100 songs - greedy algorithm for diversity adjustment (music style coverage ≥50%) - output of the first 20 songs, with accompanying explanations of the score and matching dimensions, and a response time within 100ms to ensure a real-time experience.

[0063] Key technology safeguards and iteration mechanisms are the core support for the long-term stable operation of the system. A full-chain safeguard system is constructed through three dimensions: data updates, model iterations, and performance optimization. Data updates adopt a timed incremental mode: the audio feature database is incrementally updated with new data every day at 3 AM, avoiding the resource consumption of full updates. Relying on the Spark distributed framework, processing 100,000 audio tracks in a single batch takes only 2 hours. User sentiment features are incrementally updated every 24 hours based on the day's playback data, calculating only for users with active behavior, reducing unnecessary overhead. Model iteration is driven by user feedback: reinforcement learning and MTNN models are incrementally trained weekly based on the previous week's search logs. The process includes data cleaning, feature transformation, model fine-tuning, offline evaluation, and canary release, ensuring the model adapts to dynamically changing user needs. Performance optimization focuses on response speed and scalability: Redis is used to cache popular music feature vectors, significantly reducing the response time for popular searches, keeping it within 300ms overall. The Spark distributed framework supports parallel processing of large-scale audio feature extraction, breaking through the performance bottleneck of a single machine.

[0064] Its beneficial effects lie in breaking through traditional bottlenecks through the integration of multiple technologies, resulting in more comprehensive feature extraction. Frame-by-frame processing combines time-frequency domain, phonetic sequence, and energy peak features with NLP-analyzed sentiment tendencies to construct a multi-dimensional feature system, improving audio similarity recognition accuracy by more than 35%. Real-time semantic understanding is accurate, capturing search context based on a fine-tuned BERT model, updating semantic features within 500ms, improving fuzzy search matching accuracy, and effectively responding to scenario-based needs. Thirdly, dynamic weights have strong adaptability; the reinforcement learning model optimizes weights based on user feedback, making feature fusion fit real-time preferences, increasing click-through rates for new users and playback completion rates for existing users. It balances matching accuracy and efficiency; MTNN combined with the Attention mechanism integrates multi-task outputs, and supported by the Spark distributed framework, the response time for retrieving 100,000 audio tracks is ≤300ms, balancing large-scale data processing and accurate recommendations, improving user search experience and platform conversion efficiency.

[0065] Please see Figure 2 In a music search method based on big data, a multi-task deep learning network model is established to process the audio data in the database by frame segmentation, extracting time-frequency domain features, phonetic symbols, and energy peak point pairs to obtain audio features. Then, user historical playback data is processed through an NLP (Natural Language Processing) model to obtain sentiment characteristics, including the following steps:

[0066] Step 201: Preprocess the audio data in the database, remove damaged, noisy and incompatible files, retain standard audio data, use the overlapping frame splitting strategy, set the frame length to 20ms and the frame shift to 10ms, and use Hanning window for windowing to obtain preprocessed audio data.

[0067] Step 202: Perform FFT (Fast Fourier Transform) on the preprocessed audio data to obtain a spectrogram. Extract Mel frequency cepstral coefficients, Mel spectral features, and time-domain statistical features based on the spectrogram. Concatenate the three types of features to obtain a multidimensional time-frequency domain feature vector.

[0068] Step 203: Denoise and endpoint detection are performed on the preprocessed audio data to locate the effective audio segments. The pre-trained acoustic model is loaded using the Kaldi speech recognition toolkit to map the audio frames as phoneme candidates. The CTC connection temporal classification decoding algorithm is used to remove duplicate phonemes and optimize the sequence to obtain the phonetic symbol sequence of each audio.

[0069] Step 204: Perform peak detection on the short-time energy sequence of the preprocessed audio data, set the peak threshold to 1.5 times the average energy of the audio, form peak point pairs by chronological order of adjacent peak points, calculate the time interval, energy difference and energy ratio between the point pairs, and obtain the multidimensional peak feature vector.

[0070] The above describes an embodiment of the music search method based on big data according to the present invention. Please refer to [link / reference]. Figure 3 In a big data-based music search system, the following modules are included:

[0071] The audio feature extraction module is used to perform frame-by-frame processing on the audio data in the database, extract time-frequency domain features, phonetic symbol sequences, and energy peak point pairs to obtain audio features, and process the user's historical playback data through an NLP natural language model to obtain sentiment characteristics.

[0072] The semantic feature extraction module is used to obtain real-time semantic features based on the search context data input by the user in real time, through the BRET model;

[0073] The feature vector fusion module is used to dynamically adjust the matching weights of audio features, sentiment features, and real-time semantic features using a reinforcement learning model. Based on the matching results, the feature weights are adjusted to obtain fused features.

[0074] The music matching and recommendation module is used to input fused features into the MTNN multi-task neural network, fuse the outputs of each branch through the attention mechanism, generate a music matching score, and generate a music recommendation list based on the music matching score.

[0075] This application can be used in a wide range of general-purpose or special-purpose computer system environments or configurations: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. This application can be described in the general context of computer-executable instructions executed by a computer, specifically program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0076] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A music search method based on big data, characterized in that, The big data-based music search method includes the following steps: The audio data in the database is processed by frame segmentation, and time-frequency domain features, phonetic symbol sequences and energy peak point pairs are extracted to obtain audio features. User historical playback data is processed by NLP natural language model to obtain sentiment characteristics. Based on real-time user input search context data, real-time semantic features are obtained through the BRET model; The matching weights of audio features, sentiment features, and real-time semantic features are dynamically adjusted using a reinforcement learning model. The feature weights are then adjusted based on the matching results to obtain fused features. The fused features are input into the MTNN multi-task neural network. The outputs of the three branches of the MTNN multi-task neural network are used as inputs to the attention mechanism. The attention weights of each branch are calculated: the outputs of each branch are converted into attention scores through a fully connected layer; the attention scores are normalized by the Softmax function to obtain attention weights; the branch outputs are weighted and fused according to the attention weights to obtain a fusion vector; the fusion vector is input into the MTNN output layer and mapped to the [0,1] interval by the Sigmoid activation function to obtain the music matching score; and a music recommendation list is generated based on the music matching score.

2. The music search method based on big data as described in claim 1, characterized in that, The audio data in the database is processed by frame segmentation, and time-frequency domain features, phonetic symbol sequences, and energy peak point pairs are extracted to obtain audio features. Then, user historical playback data is processed using an NLP (Natural Language Processing) model to obtain sentiment characteristics, including: The audio data in the database is preprocessed to remove broken, noisy, and incompatible files, while retaining standard audio data. An overlapping framing strategy is used, with a frame length of 20ms and a frame shift of 10ms. A Hanning window is then used for windowing to obtain the preprocessed audio data. The preprocessed audio data is subjected to FFT (Fast Fourier Transform) to obtain a spectrogram. Based on the spectrogram, Mel frequency cepstral coefficients, Mel spectral features, and time-domain statistical features are extracted. The three types of features are concatenated to obtain a multidimensional time-frequency domain feature vector. The preprocessed audio data is denoised and endpoints are detected to locate valid audio segments. A pre-trained acoustic model is loaded using the Kaldi speech recognition toolkit to map audio frames into phoneme candidates. Repeated phonemes are removed and the sequence is optimized using the CTC connection temporal classification decoding algorithm to obtain the phonetic symbol sequence for each audio. Peak detection is performed on the short-time energy sequence of the preprocessed audio data. The peak threshold is set to 1.5 times the average energy of the audio. Adjacent peak points are paired in time order. The time interval, energy difference and energy ratio between the pairs are calculated to obtain a multidimensional peak feature vector.

3. The music search method based on big data as described in claim 1, characterized in that, The audio data in the database is processed by frame segmentation, and time-frequency domain features, phonetic symbol sequences, and energy peak point pairs are extracted to obtain audio features. Then, user historical playback data is processed using an NLP (Natural Language Processing) model to obtain sentiment characteristics, including: Collect users' historical playback data over the past 90 days, including played tracks, playback duration, favorites behavior, download behavior, comment behavior, and skip frequency, to obtain users' historical playback data; A natural language processing (NLP) model is built based on a fine-tuned BERT-base-chinese model. Preprocessed user history playback data is input into the NLP model, which outputs three types of sentiment probabilities: positive, neutral, and negative. By weighted fusion of behavioral sentiment and comment sentiment, sentiment tendency features are obtained.

4. The music search method based on big data as described in claim 1, characterized in that, The search context data based on real-time user input, which obtains real-time semantic features through the BRET model, includes: The system acquires real-time search context data from the user, cleans and standardizes the search context data to obtain a real-time text sequence, inputs the real-time text sequence into a fine-tuned BERT model, takes the output vector of the last layer of the model, performs Layer Norm standardization, and then performs dimensionality reduction through a fully connected layer to obtain real-time semantic features.

5. The music search method based on big data as described in claim 1, characterized in that, The method utilizes a reinforcement learning model to dynamically adjust the matching weights of audio features, sentiment features, and real-time semantic features. Based on the matching results, the feature weights are adjusted to obtain fused features, including: The matching degree between the three types of features and the music features in the database is calculated by using cosine similarity. The reinforcement learning model agent adjusts the action by outputting weights according to the current state vector. After updating the weights, the fusion matching degree is calculated. The three types of features are weighted and summed according to the adjusted weights to obtain the fused features.

6. The music search method based on big data as described in claim 1, characterized in that, The process of inputting the fused features into the MTNN multi-task neural network, fusing the outputs of each branch through the attention mechanism to generate a music matching score, and generating a music recommendation list based on the music matching score includes: The top 100 songs are filtered in descending order of matching score. A greedy algorithm is used to ensure that the genre coverage of the recommended list is greater than 50%. The top 20 songs are then used to generate a music recommendation list, which includes the matching score.

7. A music search system based on big data, characterized in that, The big data-based music search system includes the following modules: The audio feature extraction module is used to perform frame-by-frame processing on the audio data in the database, extract time-frequency domain features, phonetic symbol sequences, and energy peak point pairs to obtain audio features, and process the user's historical playback data through an NLP natural language model to obtain sentiment characteristics. The semantic feature extraction module is used to obtain real-time semantic features based on the search context data input by the user in real time, through the BRET model; The feature vector fusion module is used to dynamically adjust the matching weights of audio features, sentiment features, and real-time semantic features using a reinforcement learning model. Based on the matching results, the feature weights are adjusted to obtain fused features. The music matching recommendation module is used to input the fused features into the MTNN multi-task neural network, and use the outputs of the three branches of the MTNN multi-task neural network as the input of the attention mechanism to calculate the attention weight of each branch: the output of each branch is converted into an attention score through a fully connected layer; The attention score is normalized using the Softmax function to obtain the attention weight. The branch outputs are then weighted and fused according to the attention weight to obtain the fusion vector. The fusion vector is input into the MTNN output layer and mapped to the [0,1] interval by the Sigmoid activation function to obtain the music matching score. A music recommendation list is then generated based on the music matching score.

8. A music search system based on big data as described in claim 7, characterized in that, The music matching and recommendation module includes the following sub-modules: The matching submodule is used to filter the top 100 music tracks in descending order of matching score. It uses a greedy algorithm to ensure that the genre coverage of the recommended list is greater than 50%, and takes the top 20 music tracks to generate a music recommendation list with matching scores.

Images