Music recommendation system based on big data
Through a music recommendation system based on big data, using the GANs model to generate music data that users may like but have not yet interacted with, solving the problem that users and music relationships are difficult to capture, and achieving more accurate personalized music recommendations.
Patent Information
- Application Number
- CN202510514696.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2025-08-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, since users usually interact with only a small portion of music in the music library, it is difficult for the model to fully capture the complex relationship between the user and the music during training, and existing methods such as limited collection of user behavior data, collaborative filtering and matrix decomposition have limitations.
A music recommendation system based on big data is adopted, including data acquisition, preprocessing, feature extraction, model training and optimization modules, and a GANs model is used to generate music data that users may like but have not yet interacted with, and a collaborative filtering algorithm for matrix decomposition is combined with a personalized recommendation model.
Through the generator, the data distribution characteristics are learned, the interactive data between users and music is completed, the data sparseness problem is alleviated, and the personalization and accuracy of the recommendation system are improved.
Smart Images
Figure CN120448581A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of music recommendation, and in particular to a music recommendation system based on big data. Background Art
[0002] The application of big data technology has revolutionized music recommendation systems. These systems can deeply mine and analyze multi-dimensional data such as users' listening history, preferences, and social behaviors, providing users with more accurate and personalized music recommendations.
[0003] In existing technologies, users typically only interact with a small fraction of the music in their library, making it difficult for models to fully capture the complex relationships between users and music during training. While introducing more user behavior data (such as browsing, searching, and sharing) or leveraging techniques like collaborative filtering and matrix factorization can alleviate data sparsity to some extent, these methods still have limitations. For example, the collection of user behavior data may be constrained by privacy protection and data quality, while methods like collaborative filtering and matrix factorization may face challenges such as cold start problems and scalability. Therefore, this paper proposes a music recommendation system based on big data. Summary of the Invention
[0004] The purpose of this invention is to solve the shortcomings of the existing technology, that is, since users usually only interact with a small part of the music in the music library, it is difficult for the model to fully capture the complex relationship between users and music during training, and to propose a music recommendation system based on big data.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A music recommendation system based on big data, comprising:
[0007] Data collection module: responsible for collecting user behavior data, music metadata and social information;
[0008] Data preprocessing module: performs preprocessing operations such as cleaning, formatting, and normalization on the collected data;
[0009] Feature extraction module: extracts feature vectors related to music recommendation from preprocessed data;
[0010] Model training and optimization module: Build a GANs model, which includes a generator and a discriminator. Adaptive training strategies are used to enable the generator to learn the distribution characteristics of the data. The generator is responsible for generating music data that users may like but have not yet interacted with, completing the interaction data between users and music, and using this completed interaction data and extracted feature vector features to train the music recommendation model.
[0011] Recommendation generation module: Generates a personalized music recommendation list based on the user's current context and the trained recommendation model;
[0012] User feedback collection and analysis module: collects user feedback on recommendation results, analyzes user satisfaction and behavior patterns, and uses this to optimize the music recommendation model.
[0013] The above technical solution further includes:
[0014] Furthermore, the data collection module records the user's interactive behaviors such as clicks, browsing, playing, pausing, skipping, searching, etc. through an application or website installed on the user's device. The data collection module uses web crawler technology to capture music metadata of songs from music websites or social media platforms. The data collection module obtains the user's social information from the API of a third-party social platform.
[0015] Furthermore, the data preprocessing module establishes association relationships between data based on the attributes or relationships of the data. The data preprocessing module identifies key attributes in different data, such as user ID, song ID, timestamp, etc., and matches and associates different data based on the key attributes. For example, user behavior data is associated with social information based on user ID, and the similarity between different data is calculated:
[0016]
[0017] Among them, r ui is the metric value of the first data u to the second data i, r vi It is the measurement value of the first data v to the second data i, and the value range of cosine similarity is [-1,1].
[0018] Furthermore, the feature extraction module extracts feature vectors related to music recommendation from the preprocessed data, and uses correlation analysis to evaluate features that have a significant impact on music recommendation. The correlation analysis calculation formula is: Among them, r is the correlation coefficient, x i and y i are the observed values of the feature and music recommendation variables, and are the means of the two sets of data respectively. The correlation coefficient r ranges from -1 to 1. The larger the absolute value, the stronger the correlation between the two sets of data.
[0019] Furthermore, the model training and optimization module includes a generator unit, a discriminator unit and a recommendation model training unit. The generator unit learns the distribution characteristics of the data and generates music data that matches the user preferences but has not yet been actually interacted with by the user. The discriminator unit distinguishes between real user interaction data and data generated by the generator, and helps the generator improve the quality of the data it generates through feedback signals. The recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model.
[0020] Furthermore, the structure of the generator unit includes:
[0021] Input layer: Receives a random noise vector that follows a Gaussian or uniform distribution. This vector serves as the starting point of the generation process and contains all the potential information required to generate interaction data between users and music.
[0022] Convolutional transpose layer: used to upsample the input vector to the spatial resolution. Through the inverse process of the convolution operation, the low-dimensional features are mapped to the high-dimensional space to generate the interaction data between the user and the music. The calculation formula is expressed as Y = ConvTranspose(X,W,b,s,p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the spatial resolution of the output feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map;
[0023] Batch normalization layer: Normalize the output of the convolutional transpose layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number;
[0024] Activation function layer: introduces nonlinear factors to enable the generator to learn complex feature representations;
[0025] Output layer: generated music data, used to complete the interaction data between users and music.
[0026] Furthermore, the structure of the discriminator unit includes:
[0027] Input layer: receives data from the real dataset or generator as input;
[0028] Convolution layer: extracts the features of the input data through convolution operation. The convolution kernel slides on the input data, calculates the local features of each position, and generates a feature map. The calculation formula is expressed as Y = Conv(X, W, b, s, p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the sliding distance of the convolution kernel on the input feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map;
[0029] Batch Normalization Layer: The batch normalization layer normalizes the output of the convolutional layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number;
[0030] Activation function layer: The activation function layer introduces nonlinear factors, allowing the discriminator to learn complex feature representations;
[0031] Fully connected layer: The fully connected layer converts the output of the convolutional layer and the activation function layer into a fixed-length feature vector, which is used for the final classification decision;
[0032] Output layer: The output layer is a single neuron that uses the Sigmoid activation function to output a probability value, which indicates the confidence that the input data is real data. The closer the probability value is to 1, the more real the input data is; the closer the probability value is to 0, the more false the input data is.
[0033] Furthermore, the recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model, including the following steps:
[0034] Data preparation:
[0035] Completing interaction data: The generator in the GANs model generates music data that the user may like but has not yet interacted with, to complete the interaction data between the user and the music;
[0036] Feature vector extraction: Extract feature vectors from music datasets. These feature vectors represent various attributes of music, such as genre, rhythm, and melody. These feature vectors will be used to train recommendation models to capture the potential relationship between users and music.
[0037] Model training: The completed interaction data and extracted feature vector features are input into a collaborative filtering algorithm based on matrix decomposition. The model parameters are iteratively updated using the gradient descent method to minimize the loss function, thereby obtaining the optimal recommendation model.
[0038] Model evaluation: Use the test dataset to evaluate the performance of the trained recommendation model. Common evaluation metrics include accuracy, recall, F1 score, AUC, etc. Adjust and optimize the model based on the evaluation results to improve the recommendation effect.
[0039] The present invention has the following beneficial effects:
[0040] In this paper, a GAN model is trained to generate music data that users may like but have not yet interacted with. This approach maintains data authenticity while increasing the amount of interaction data between users and music, alleviating data sparsity. After obtaining this complete interaction data, this data is combined with existing recommendation algorithms to build a more accurate recommendation system. By integrating the advantages of multiple data sources and algorithms, the performance of the recommendation system is improved, providing users with more personalized music recommendation services. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 This is a system block diagram of a music recommendation system based on big data proposed by the present invention. DETAILED DESCRIPTION
[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0043] See also Figure 1 As shown, the present invention is a music recommendation system based on big data, comprising:
[0044] Data collection module: responsible for collecting user behavior data, music metadata and social information;
[0045] Data preprocessing module: performs preprocessing operations such as cleaning, formatting, and normalization on the collected data;
[0046] Feature extraction module: extracts feature vectors related to music recommendation from preprocessed data;
[0047] Model training and optimization module: Build a GANs model, which includes a generator and a discriminator. Adaptive training strategies are used to enable the generator to learn the distribution characteristics of the data. The generator is responsible for generating music data that users may like but have not yet interacted with, completing the interaction data between users and music, and using this completed interaction data and extracted feature vector features to train the music recommendation model.
[0048] Recommendation generation module: Generates a personalized music recommendation list based on the user's current context and the trained recommendation model;
[0049] User feedback collection and analysis module: collects user feedback on recommendation results, analyzes user satisfaction and behavior patterns, and uses this to optimize the music recommendation model.
[0050] In one embodiment, the data collection module records the user's interactive behaviors such as clicks, browsing, playing, pausing, skipping, and searching through an application or website installed on the user's device. The data collection module uses web crawler technology to capture music metadata of songs from music websites or social media platforms. The data collection module obtains the user's social information from the API of a third-party social platform.
[0051] In one embodiment, the data preprocessing module establishes associations between data based on attributes or relationships of the data. The data preprocessing module identifies key attributes in different data, such as user ID, song ID, timestamp, etc., and matches and associates different data based on the key attributes. For example, user behavior data is associated with social information based on user ID, and similarity between different data is calculated:
[0052]
[0053] Among them, r ui is the metric value of the first data u to the second data i, r vi It is the measurement value of the first data v to the second data i, and the value range of cosine similarity is [-1,1].
[0054] In one embodiment, the feature extraction module extracts feature vectors related to music recommendation from the preprocessed data and uses correlation analysis to evaluate features that have a significant impact on music recommendation. The correlation analysis calculation formula is: Among them, r is the correlation coefficient, x i and y i are the observed values of the feature and music recommendation variables, and are the means of the two sets of data respectively. The correlation coefficient r ranges from -1 to 1. The larger the absolute value, the stronger the correlation between the two sets of data.
[0055] In one embodiment, the model training and optimization module includes a generator unit, a discriminator unit and a recommendation model training unit. The generator unit learns the distribution characteristics of the data and generates music data that matches the user preferences but has not yet been actually interacted with by the user. The discriminator unit distinguishes between real user interaction data and data generated by the generator, and helps the generator improve the quality of the data it generates through feedback signals. The recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model.
[0056] In one embodiment, the structure of the generator unit includes:
[0057] Input layer: Receives a random noise vector that follows a Gaussian or uniform distribution. This vector serves as the starting point of the generation process and contains all the potential information required to generate interaction data between users and music.
[0058] Convolutional transpose layer: used to upsample the input vector to the spatial resolution. Through the inverse process of the convolution operation, the low-dimensional features are mapped to the high-dimensional space to generate the interaction data between the user and the music. The calculation formula is expressed as Y = ConvTranspose(X,W,b,s,p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the spatial resolution of the output feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map;
[0059] Batch normalization layer: Normalize the output of the convolutional transpose layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number;
[0060] Activation function layer: introduces nonlinear factors to enable the generator to learn complex feature representations;
[0061] Output layer: generated music data, used to complete the interaction data between users and music.
[0062] In one embodiment, the structure of the discriminator unit includes:
[0063] Input layer: receives data from the real dataset or generator as input;
[0064] Convolution layer: extracts the features of the input data through convolution operation. The convolution kernel slides on the input data, calculates the local features of each position, and generates a feature map. The calculation formula is expressed as Y = Conv(X, W, b, s, p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the sliding distance of the convolution kernel on the input feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map;
[0065] Batch Normalization Layer: The batch normalization layer normalizes the output of the convolutional layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number;
[0066] Activation function layer: The activation function layer introduces nonlinear factors, allowing the discriminator to learn complex feature representations;
[0067] Fully connected layer: The fully connected layer converts the output of the convolutional layer and the activation function layer into a fixed-length feature vector, which is used for the final classification decision;
[0068] Output layer: The output layer is a single neuron that uses the Sigmoid activation function to output a probability value, which indicates the confidence that the input data is real data. The closer the probability value is to 1, the more real the input data is; the closer the probability value is to 0, the more false the input data is.
[0069] In one embodiment, the recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model, including the following steps:
[0070] Data preparation:
[0071] Completing interaction data: The generator in the GANs model generates music data that the user may like but has not yet interacted with, to complete the interaction data between the user and the music;
[0072] Feature vector extraction: Extract feature vectors from music datasets. These feature vectors represent various attributes of music, such as genre, rhythm, and melody. These feature vectors will be used to train recommendation models to capture the potential relationship between users and music.
[0073] Model training: The completed interaction data and extracted feature vector features are input into a collaborative filtering algorithm based on matrix decomposition. The model parameters are iteratively updated using the gradient descent method to minimize the loss function, thereby obtaining the optimal recommendation model.
[0074] Model evaluation: Use the test dataset to evaluate the performance of the trained recommendation model. Common evaluation metrics include accuracy, recall, F1 score, AUC, etc. Adjust and optimize the model based on the evaluation results to improve the recommendation effect.
[0075] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.
Claims
1. A music recommendation system based on big data, characterized in that: include: Data collection module: responsible for collecting user behavior data, music metadata and social information; Data preprocessing module: performs preprocessing operations on the collected data; Feature extraction module: extracts feature vectors related to music recommendation from preprocessed data; Model training and optimization module: Build a GANs model, which includes a generator and a discriminator. Adaptive training strategies are used to enable the generator to learn the distribution characteristics of the data. The generator is responsible for generating music data that users may like but have not yet interacted with, completing the interaction data between users and music, and using this completed interaction data and extracted feature vector features to train the music recommendation model. Recommendation generation module: Generates a personalized music recommendation list based on the user's current context and the trained recommendation model; User feedback collection and analysis module: collects user feedback on recommendation results, analyzes user satisfaction and behavior patterns, and uses this to optimize the music recommendation model.
2. A music recommendation system based on big data according to claim 1, characterized in that: The data collection module records the user's interactive behavior through an application or website installed on the user's device. The data collection module uses web crawler technology to capture the music metadata of songs from music websites or social media platforms. The data collection module obtains the user's social information from the API of a third-party social platform.
3. The music recommendation system based on big data according to claim 1, characterized in that: After preprocessing the user behavior data, music metadata, and social information, the data preprocessing module integrates the preprocessed data to form a unified data set. At the same time, based on the attributes or relationships of the data, an association relationship between the data is established.
4. A music recommendation system based on big data according to claim 3, characterized in that: The data preprocessing module establishes association relationships between data based on the attributes or relationships of the data. The data preprocessing module identifies key attributes in different data, matches and associates different data based on the key attributes, and calculates the similarity between different data: Among them, r ui is the metric value of the first data u to the second data i, r vi It is the measurement value of the first data v to the second data i, and the value range of cosine similarity is [-1,1].
5. The music recommendation system based on big data according to claim 1, characterized in that: The feature extraction module extracts feature vectors related to music recommendation from the preprocessed data and uses correlation analysis to evaluate features that have a significant impact on music recommendation. The correlation analysis calculation formula is: Among them, r is the correlation coefficient, x i and y i are the observed values of the feature and music recommendation variables, and are the means of the two sets of data respectively. The correlation coefficient r ranges from -1 to 1. The larger the absolute value, the stronger the correlation between the two sets of data.
6. The music recommendation system based on big data according to claim 1, characterized in that: The model training and optimization module includes a generator unit, a discriminator unit and a recommendation model training unit. The generator unit learns the distribution characteristics of the data and generates music data that matches the user preferences but has not yet been actually interacted with by the user. The discriminator unit distinguishes between real user interaction data and data generated by the generator, and helps the generator improve the quality of the data it generates through feedback signals. The recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model.
7. The music recommendation system based on big data according to claim 6, characterized in that: The structure of the generator unit includes: Input layer: Receives a random noise vector that follows a Gaussian or uniform distribution. This vector serves as the starting point of the generation process and contains all the potential information required to generate interaction data between users and music. Convolutional transpose layer: used to upsample the input vector to the spatial resolution. Through the inverse process of the convolution operation, the low-dimensional features are mapped to the high-dimensional space to generate the interaction data between the user and the music. The calculation formula is expressed as Y = ConvTranspose(X,W,b,s,p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the spatial resolution of the output feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map; Batch normalization layer: Normalize the output of the convolutional transpose layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number; Activation function layer: introduces nonlinear factors to enable the generator to learn complex feature representations; Output layer: generated music data, used to complete the interaction data between users and music.
8. The music recommendation system based on big data according to claim 6, characterized in that: The structure of the discriminator unit includes: Input layer: receives data from the real dataset or generator as input; Convolution layer: extracts the features of the input data through convolution operation. The convolution kernel slides on the input data, calculates the local features of each position, and generates a feature map. The calculation formula is expressed as Y = Conv(X, W, b, s, p), where X is the input feature map, W is the convolution kernel weight; b is the bias term; s is the step size, which controls the sliding distance of the convolution kernel on the input feature map; p is padding, which is used to add extra zero values around the input feature map to control the size of the output feature map; Batch Normalization Layer: The batch normalization layer normalizes the output of the convolutional layer so that the input data of each batch has the same distribution. The normalization process is expressed as Where X is the input feature map; μ and σ 2 are the mean and variance of the input feature map respectively; γ and β are learnable parameters used to adjust the normalized feature map; ε is a positive number; Activation function layer: The activation function layer introduces nonlinear factors, allowing the discriminator to learn complex feature representations; Fully connected layer: The fully connected layer converts the output of the convolutional layer and the activation function layer into a fixed-length feature vector, which is used for the final classification decision; Output layer: The output layer is a single neuron that uses the Sigmoid activation function to output a probability value, which indicates the confidence that the input data is real data. The closer the probability value is to 1, the more real the input data is; the closer the probability value is to 0, the more false the input data is.
9. The music recommendation system based on big data according to claim 6, characterized in that: The recommendation model training unit uses the completed interaction data and the extracted feature vector features to train the music recommendation model, including the following steps: Data preparation: Completing interaction data: The generator in the GANs model generates music data that the user may like but has not yet interacted with, to complete the interaction data between the user and the music; Feature vector extraction: extracting feature vectors from music datasets, which represent various attributes of music; Model training: The completed interaction data and extracted feature vector features are input into a collaborative filtering algorithm based on matrix decomposition. The model parameters are iteratively updated using the gradient descent method to minimize the loss function. Model evaluation: Use the test dataset to evaluate the performance of the trained recommendation model, and adjust and optimize the model based on the evaluation results.