A movie rating system and method based on big data

Abstract

The application discloses a kind of movie rating system and method based on big data, including data acquisition layer, data storage and processing layer, model construction layer, service layer and user interface layer: data acquisition layer: carry out data acquisition;Data storage and processing layer: the data cleaning and pretreatment to the data collected, extract features from the data after pretreatment;Model construction layer: responsible for processing and analyzing big data, generating movie rating, using naive bayes analysis user behavior data, quantifying positive / negative sentiment, when the sentiment of a particular group is imbalanced, use focal loss to reduce the weight of easily classified samples, based on movie features, multi-dimensional features and time features, use multi-task learning to train multiple XGBoost models, predict ratings in different dimensions respectively, naive bayes analysis results provide sentiment input for different dimensions, at the same time, combined with group characteristics, provide personalized ratings for different groups.

Description

Technical Field

[0001] This invention relates to the field of film rating technology, and in particular to a film rating system and method based on big data. Background Technology

[0002] The rapid development of internet technology has made the collection, storage, and analysis of massive amounts of data possible. Big data technology can extract valuable information from this data, providing strong technical support for film rating systems. By collecting and analyzing audience reviews, film rating systems can provide viewers with a relatively objective and comprehensive evaluation of films. This helps viewers gain a preliminary understanding of a film before watching it, avoiding wasting time and money on movies they don't like.

[0003] Existing film rating systems focus only on the overall score, neglecting the correlation between different dimensions of a film (such as plot, acting, and directing). In other words, under single-task learning methods, rating information from different dimensions may not be considered or fully utilized individually. The model might simply combine these factors to generate an overall score. However, this approach may fail to accurately reflect the specific evaluations and preferences of viewers or critics across different dimensions. Therefore, this paper proposes a film rating system and method based on big data. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies where, under single-task learning methods, rating information from different dimensions may not be considered or fully utilized individually, thus failing to accurately reflect the specific evaluations and preferences of viewers or film critics in different dimensions. Therefore, this invention proposes a film rating system and method based on big data.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A movie rating system based on big data includes a data acquisition layer, a data storage and processing layer, a model building layer, a service layer, and a user interface layer.

[0007] Data Acquisition Layer: This layer collects data, including user behavior data, movie metadata, social media data, and professional film review data. User behavior data includes viewing history, search history, comments, likes, and shares. Movie metadata includes basic movie information (such as director, actors, genre, and release year), synopsis, trailer views, and poster click rates. Social media data includes discussions and sentiments about the movie scraped from social platforms such as Twitter and Reddit. Professional film review data includes ratings and reviews from well-known film review websites and magazines.

[0008] Data storage and processing layer: Hadoop HDFS is used to store the collected data, and the collected data is cleaned and preprocessed. Features are extracted from the preprocessed data, including user preference features (comedy, action preference), group features (such as circle influence, team activity), film features (director influence, cast), multi-dimensional film features (such as director style rating, actor acting rating), and time features (initial release popularity). The user preference features are extracted from user behavior data, the group features are extracted from user behavior data and social media data, the film features are extracted from film metadata, the multi-dimensional film features are extracted from social media data and professional film review data, and the time features are extracted from user behavior data and film metadata.

[0009] Model building layer: Responsible for processing and analyzing big data, generating movie ratings, using Naive Bayes to analyze user preference features and group characteristics, quantifying positive / negative sentiment, and using focus loss to reduce the weight of easily classified samples when the sentiment distribution of a specific group is unbalanced. Based on movie features, multi-dimensional features, and time features, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. The Naive Bayes analysis results provide sentiment input for different dimensions. At the same time, combined with group characteristics, personalized ratings are provided for different groups.

[0010] Service layer: Responsible for providing API interfaces and real-time processing functions, supporting data interaction and response between the model building layer and the user interface layer;

[0011] User interface layer: responsible for displaying the data generated by the model building layer and the functions provided by the service layer.

[0012] The above technical solution further includes:

[0013] Preferably, the user behavior data reflects users' movie-watching habits, preferences, and social behaviors, which is key to understanding user needs and preferences. The movie metadata provides basic information and market performance of the movie. The social media data is crawled from social platforms to reveal public discussions and emotional tendencies about the movie. The professional film review data includes ratings and comments from professional film critics.

[0014] Preferably, the user preference features are extracted by calculating the user's viewing frequency or rating for different types of movies, using nodes and edges in social networks to represent the relationship between users and groups, and then extracting group features by calculating the degree of nodes. The movie features are extracted from movie metadata, and the multi-dimensional movie features are extracted based on professional film reviews and user comments through sentiment analysis and topic modeling, and time series analysis is used to extract time features.

[0015] Preferably, the specific steps for using Naive Bayes to analyze user comments and quantify positive / negative sentiment are as follows:

[0016] Data preparation: Collect user behavior data, including viewing records, search history, comments, likes, shares, etc. Preprocess the text data in the user behavior data, such as removing stop words and punctuation marks, and performing stemming or lemmatization. Convert the preprocessed text data into numerical features, usually using methods such as Bag of Words, TF-IDF (Term Frequency-Inverse Document Frequency), or word embedding (such as Word2Vec, BERT).

[0017] Bernoulli Naive Bayes Model Construction: It assumes that features are independent (the assumption of Naive Bayes) and that each feature is binary (Bernoulli distribution), meaning it only takes values ​​of 0 or 1. Based on the preprocessed data, the prior probability of each feature is calculated, which is the probability of the feature appearing in all samples. The posterior probability of each class is calculated using Bayes' theorem, which is the probability of belonging to a certain class given the observed data. The posterior probability P(C|X) is expressed as... Where P(C|X) is the posterior probability of class C given feature X, P(X|C) is the likelihood probability of feature X given class C, P(C) is the prior probability of class C, and P(X) is the prior probability of feature X. In Bernoulli Naive Bayes, it is assumed that the feature vector X has n binary features, i.e., X = (x1, x2, ..., xn). n ), where x i ∈{0,1}, given a class C, each feature x i It appears independently, that is, x i The probability that = 1 is θ i,C The likelihood probability P(X|C) is expressed as in, It is feature x i The probability of being 1 under category C is (1-θ) i,C ) is a feature x i The probability of being 0 in category C is expressed by the decision rule of the Bernoulli Naive Bayes classifier as follows: in, It is the predicted category;

[0018] Sentiment quantification: Input text data into a Bernoulli Naive Bayes model to obtain the probability that each comment belongs to positive or negative sentiment. Set a threshold based on the probability value to classify comments into positive or negative sentiment. Summarize the sentiment of all comments to obtain the user's overall sentiment towards the movie.

[0019] Preferably, the specific process of using focus loss to reduce the weight of easily classified samples when the distribution of sentiment tendencies in a specific group is unbalanced is as follows:

[0020] Sentiment distribution analysis: Analyze the sentiment distribution of a specific group to identify whether there is an imbalance (such as a large number of positive sentiment samples compared to negative sentiment samples). When the sentiment distribution of a specific group is unbalanced, i.e. the number of positive or negative comments differs significantly, focus loss is used to reduce the weight of easily classified samples.

[0021] Focus loss function definition: For imbalanced distributions, a focus loss function is defined. This function introduces a modulation factor to reduce the weight of easily classified samples, causing the model to focus on difficult-to-classify samples. The formula for the focus loss function is FL(p...). i )=-α i (1-p i ) γ log(p i ), where p i α is the probability that the model predicts a sample belongs to the true category (for positive or negative sentiment). i These are class weights (used to handle class imbalance) and γ modulation factors (used to reduce the weight of easily classified samples);

[0022] Model optimization: During the training of the Bernoulli Naive Bayes model, the focus loss function is used instead of the cross-entropy loss function.

[0023] Preferably, based on historical rating data, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. The Naive Bayes analysis results provide sentiment input for different dimensions, including the following steps:

[0024] Data preparation: Collect features provided by the data storage and processing layer;

[0025] Multi-task learning model construction: A multi-task learning framework is adopted, and an XGBoost model is constructed for each scoring dimension. The objective function of the XGBoost model is expressed as follows: Where N is the number of users, M is the number of rating dimensions, and y ij It is user i's actual rating on dimension j. It is the model's predicted score, w k These are the model parameters, and λ is the regularization coefficient.

[0026] Naive Bayes analysis: Sentiment analysis results are input as additional features into the multi-task learning model to help the model predict ratings more accurately;

[0027] Model training and evaluation: Train the model using cross-validation, evaluate the model performance, and adjust the model parameters and feature set based on the evaluation results to optimize the model performance;

[0028] Prediction and Output: Use the trained model to predict ratings for new movies and present the prediction results to users in an intuitive and easy-to-understand way.

[0029] Preferably, the specific steps for adjusting the model prediction results based on the characteristics of the user's group, providing personalized scores for different groups, using the K-means clustering algorithm to group users, and then adjusting the scores based on group characteristics are as follows:

[0030] Data preprocessing: Preprocessing the collected data;

[0031] Data integration and feature engineering: Integrating preprocessed data to generate a unified feature matrix;

[0032] Select a K value: Preset a K value based on the actual situation;

[0033] Clustering using the K-means algorithm: randomly select K initial cluster centers, assign each data point to its nearest cluster center, update the centroid of the cluster based on the mean of the data points in each cluster, repeat the assignment and update steps until the cluster centers no longer change, or the preset maximum number of iterations is reached;

[0034] Analyze the clustering results: Based on the data characteristics in each cluster, analyze the representativeness of the clusters, assign labels to each cluster, classify the clusters according to user preferences, and visualize the clustering results using a 3D scatter plot;

[0035] Personalized ratings: Adjust ratings based on category results. For example, for users who like action movies, the weight of the plot rating can be slightly increased; for users who like comedy movies, the weight of the actor performance rating can be slightly increased.

[0036] Preferably, a movie rating system based on big data and a movie rating method based on big data include:

[0037] S1: Data Collection and Integration: Collect data, including user behavior data, movie metadata, social media data, and professional film review data;

[0038] S2: Data storage and processing: Use Hadoop HDFS to store the collected data, perform data cleaning and preprocessing on the collected data, and extract features from the preprocessed data;

[0039] S3: Sentiment Analysis: Using the Naive Bayes algorithm to perform sentiment analysis on user behavior data and social media data, quantifying positive / negative sentiment tendencies. To address the problem of imbalanced sentiment distribution in specific groups, a focus loss function is introduced to reduce the weight of easily classified samples.

[0040] S4: Multi-task learning: Based on historical rating data, multiple XGBoost models are trained using a multi-task learning framework. Each model is responsible for predicting the rating of one dimension (such as overall rating, plot rating, actor performance rating, etc.). The sentiment tendency obtained from the Naive Bayes analysis in S3 is used as a feature input into the XGBoost model to provide reference information on sentiment tendency for different dimensions and improve the accuracy of rating prediction.

[0041] S5: Personalized Scoring: The model prediction results are adjusted in a personalized manner based on the characteristics of the user's group, providing different groups with scores that match their preferences.

[0042] The present invention has the following beneficial effects:

[0043] 1. In this invention, a multi-task learning framework is used to train multiple XGBoost models, each responsible for predicting the rating for one dimension. This method can capture the intrinsic relationships between different dimensions and fully utilize the correlations between them, improving the accuracy of rating prediction. Simultaneously, the sentiment tendency obtained from Naive Bayes analysis is used as a feature input into the XGBoost model, providing reference information on sentiment tendency for different dimensions, further enhancing the accuracy and personalization of the rating.

[0044] 2. In this invention, the Naive Bayes algorithm is used to perform sentiment analysis on user behavior data and social media data. This quantifies positive / negative sentiment tendencies, providing a reference for the sentiment dimension of movie ratings. To address the issue of imbalanced sentiment distribution among specific groups, a focus loss function is introduced, reducing the weight of easily classified samples and increasing the model's attention to difficult-to-classify samples. This helps improve the model's prediction accuracy and generalization ability across different groups. Attached Figure Description

[0045] Figure 1 This is a system architecture diagram of a movie rating system based on big data proposed in this invention;

[0046] Figure 2 This is a flowchart of a movie rating method based on big data proposed in this invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, a movie rating system based on big data includes a data acquisition layer, a data storage and processing layer, a model building layer, a service layer, and a user interface layer.

[0049] Data Acquisition Layer: This layer collects data, including user behavior data, movie metadata, social media data, and professional film review data. User behavior data includes viewing history, search history, comments, likes, and shares. Movie metadata includes basic movie information (such as director, actors, genre, and release year), synopsis, trailer views, and poster click rates. Social media data includes discussions and sentiments about the movie scraped from social platforms such as Twitter and Reddit. Professional film review data includes ratings and reviews from well-known film review websites and magazines.

[0050] Data storage and processing layer: Hadoop HDFS is used to store the collected data, and the collected data is cleaned and preprocessed. Features are extracted from the preprocessed data, including user preference features (comedy, action preference), group features (such as circle influence, team activity), film features (director influence, cast), multi-dimensional film features (such as director style rating, actor acting rating), and time features (initial release popularity). The user preference features are extracted from user behavior data, the group features are extracted from user behavior data and social media data, the film features are extracted from film metadata, the multi-dimensional film features are extracted from social media data and professional film review data, and the time features are extracted from user behavior data and film metadata.

[0051] Model building layer: Responsible for processing and analyzing big data, generating movie ratings, using Naive Bayes to analyze user preference features and group characteristics, quantifying positive / negative sentiment, and using focus loss to reduce the weight of easily classified samples when the sentiment distribution of a specific group is unbalanced. Based on movie features, multi-dimensional features, and time features, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. The Naive Bayes analysis results provide sentiment input for different dimensions, thereby helping the system to more accurately predict user ratings in various dimensions. At the same time, combined with the characteristics of the user's group, personalized ratings are provided for different groups.

[0052] Service layer: Responsible for providing API interfaces and real-time processing functions, supporting data interaction and response between the model building layer and the user interface layer;

[0053] User interface layer: responsible for displaying the data generated by the model building layer and the functions provided by the service layer.

[0054] In one embodiment, the user behavior data reflects users' movie-watching habits, preferences, and social behaviors, and is key to understanding user needs and preferences. The movie metadata provides basic information and market performance of the movie. The social media data is crawled from social platforms to reveal public discussions and emotional tendencies about the movie. The professional film review data includes ratings and comments from professional film critics.

[0055] In one embodiment, the user preference features are extracted by calculating the user's viewing frequency or rating for different types of movies, using nodes and edges in a social network to represent the relationship between users and groups, and then extracting group features by calculating the degree of nodes. The movie features are extracted from movie metadata, and the multi-dimensional movie features are extracted based on professional film reviews and user comments through sentiment analysis and topic modeling, and time series analysis is used to extract time features.

[0056] In one embodiment, the specific steps for using Naive Bayes to analyze user reviews and quantify positive / negative sentiment are as follows:

[0057] Data preparation: Collect user behavior data, including viewing records, search history, comments, likes, shares, etc. Preprocess the text data in the user behavior data, such as removing stop words and punctuation marks, and performing stemming or lemmatization. Convert the preprocessed text data into numerical features, usually using methods such as Bag of Words, TF-IDF (Term Frequency-Inverse Document Frequency), or word embedding (such as Word2Vec, BERT).

[0058] Bernoulli Naive Bayes Model Construction: It assumes that features are independent (the assumption of Naive Bayes) and that each feature is binary (Bernoulli distribution), meaning it only takes values ​​of 0 or 1. Based on the preprocessed data, the prior probability of each feature is calculated, which is the probability of the feature appearing in all samples. The posterior probability of each class is calculated using Bayes' theorem, which is the probability of belonging to a certain class given the observed data. The posterior probability P(C|X) is expressed as... Where P(C|X) is the posterior probability of class C given feature X, P(X|C) is the likelihood probability of feature X given class C, P(C) is the prior probability of class C, and P(X) is the prior probability of feature X. In Bernoulli Naive Bayes, it is assumed that the feature vector X has n binary features, i.e., X = (x1, x2, ..., xn). n ), where x i ∈{0,1}, given a class C, each feature x i It appears independently, that is, x i The probability that = 1 is θ i,C The likelihood probability P(X|C) is expressed as in, It is feature x i The probability of being 1 under category C is (1-θ) i,C ) is a feature x i The probability of being 0 in category C is expressed by the decision rule of the Bernoulli Naive Bayes classifier as follows: in, It is the predicted category;

[0059] Sentiment quantification: Input text data into a Bernoulli Naive Bayes model to obtain the probability that each comment belongs to positive or negative sentiment. Set a threshold based on the probability value to classify comments into positive or negative sentiment. Summarize the sentiment of all comments to obtain the user's overall sentiment towards the movie.

[0060] In one embodiment, the specific process of using focus loss to reduce the weight of easily classified samples when the sentiment distribution of a specific group is unbalanced is as follows:

[0061] Sentiment distribution analysis: Analyze the sentiment distribution of a specific group to identify whether there is an imbalance (such as a large number of positive sentiment samples compared to negative sentiment samples). When the sentiment distribution of a specific group is unbalanced, i.e. the number of positive or negative comments differs significantly, focus loss is used to reduce the weight of easily classified samples.

[0062] Focus loss function definition: For imbalanced distributions, a focus loss function is defined. This function introduces a modulation factor to reduce the weight of easily classified samples, causing the model to focus on difficult-to-classify samples. The formula for the focus loss function is FL(p...). i )=-α i (1-p i ) γ log(p i ), where p i α is the probability that the model predicts a sample belongs to the true category (for positive or negative sentiment). i These are class weights (used to handle class imbalance) and γ modulation factors (used to reduce the weight of easily classified samples);

[0063] Model optimization: During the training of the Bernoulli Naive Bayes model, the focus loss function is used instead of the cross-entropy loss function.

[0064] Suppose that in a certain group, positive sentiment comments account for 80% and negative sentiment comments account for 20%. When training a model using the traditional cross-entropy loss function, the model may overemphasize positive sentiment comments, leading to a decrease in its ability to predict negative sentiment comments. This can be addressed by introducing a focus loss function and appropriately adjusting α. i By adjusting the values ​​of γ, the model can handle positive and negative sentiment comments in a more balanced way, thereby improving overall performance.

[0065] In one embodiment, based on historical rating data, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. The Naive Bayes analysis results provide sentiment input for different dimensions, including the following steps:

[0066] Data preparation: Collect features provided by the data storage and processing layer;

[0067] Multi-task learning model construction: A multi-task learning framework is adopted, and an XGBoost model is constructed for each scoring dimension. The objective function of the XGBoost model is expressed as follows: Where N is the number of users, M is the number of rating dimensions, and y ij It is user i's actual rating on dimension j. It is the model's predicted score, w k These are the model parameters, and λ is the regularization coefficient.

[0068] Naive Bayes analysis: Sentiment analysis results are input as additional features into the multi-task learning model to help the model predict ratings more accurately;

[0069] Model training and evaluation: Train the model using cross-validation, evaluate the model performance, and adjust the model parameters and feature set based on the evaluation results to optimize the model performance;

[0070] Prediction and Output: Use the trained model to predict ratings for new movies and present the prediction results to users in an intuitive and easy-to-understand way.

[0071] In one embodiment, the specific steps of adjusting the model prediction results based on the characteristics of the user's group to provide personalized scores for different groups, using the K-means clustering algorithm to group users, and then adjusting the scores according to group characteristics are as follows:

[0072] Data preprocessing: Preprocessing the collected data;

[0073] Data integration and feature engineering: Integrating preprocessed data to generate a unified feature matrix;

[0074] Select a K value: Preset a K value based on the actual situation;

[0075] Clustering using the K-means algorithm: randomly select K initial cluster centers, assign each data point to its nearest cluster center, update the centroid of the cluster based on the mean of the data points in each cluster, repeat the assignment and update steps until the cluster centers no longer change, or the preset maximum number of iterations is reached;

[0076] Analyze the clustering results: Based on the data characteristics in each cluster, analyze the representativeness of the clusters, assign labels to each cluster, classify the clusters according to user preferences, and visualize the clustering results using a 3D scatter plot;

[0077] Personalized ratings: Adjust ratings based on category results. For example, for users who like action movies, the weight of the plot rating can be slightly increased; for users who like comedy movies, the weight of the actor performance rating can be slightly increased.

[0078] like Figure 2 As shown, a movie rating method based on big data includes:

[0079] S1: Data Collection and Integration: Collect data, including user behavior data, movie metadata, social media data, and professional film review data;

[0080] S2: Data storage and processing: Use Hadoop HDFS to store the collected data, perform data cleaning and preprocessing on the collected data, and extract features from the preprocessed data;

[0081] S3: Sentiment Analysis: Using the Naive Bayes algorithm to perform sentiment analysis on user behavior data and social media data, quantifying positive / negative sentiment tendencies. To address the problem of imbalanced sentiment distribution in specific groups, a focus loss function is introduced to reduce the weight of easily classified samples.

[0082] S4: Multi-task learning: Based on historical rating data, multiple XGBoost models are trained using a multi-task learning framework. Each model is responsible for predicting the rating of one dimension (such as overall rating, plot rating, actor performance rating, etc.). The sentiment tendency obtained from the Naive Bayes analysis in S3 is used as a feature input into the XGBoost model to provide reference information on sentiment tendency for different dimensions and improve the accuracy of rating prediction.

[0083] S5: Personalized Scoring: The model prediction results are adjusted in a personalized manner based on the characteristics of the user's group, providing different groups with scores that match their preferences.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A movie rating system based on big data, characterized in that, It includes a data acquisition layer, a data storage and processing layer, a model building layer, a service layer, and a user interface layer: Data Acquisition Layer: This layer collects data, including user behavior data, movie metadata, social media data, and professional film review data. The movie metadata includes basic movie information, plot synopsis, trailer view counts, and poster click-through rates. Data storage and processing layer: Hadoop HDFS is used to store the collected data, and the collected data is cleaned and preprocessed. Features are extracted from the preprocessed data, including user preference features, group features, movie features, multi-dimensional movie features, and time features. User preference features are extracted from user behavior data, group features are extracted from user behavior data and social media data, movie features are extracted from movie metadata, multi-dimensional movie features are extracted from social media data and professional film review data, and time features are extracted from user behavior data and movie metadata. Model building layer: Responsible for processing and analyzing big data, generating movie ratings, using Naive Bayes to analyze user preference features and group characteristics, quantifying positive / negative sentiment, and using focus loss to reduce the weight of easily classified samples when the sentiment distribution of a specific group is unbalanced. Based on movie features, multi-dimensional movie features, and time features, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. The Naive Bayes analysis results provide sentiment input for different dimensions. At the same time, combined with group characteristics, personalized ratings are provided for different groups. Service layer: Responsible for providing API interfaces and real-time processing functions, supporting data interaction and response between the model building layer and the user interface layer; User interface layer: responsible for displaying the data generated by the model building layer and the functions provided by the service layer.

2. The movie rating system based on big data according to claim 1, characterized in that, The user behavior data reflects users' viewing habits, preferences, and social behaviors, and is key to understanding user needs and preferences. The film metadata provides basic information about the film and its market performance. The social media data is crawled from social platforms to reveal public discussions and emotional tendencies about the film. The professional film review data includes ratings and comments from professional film critics.

3. A movie rating system based on big data according to claim 1, characterized in that, The user preference features are extracted by calculating the viewing frequency or rating of users for different types of movies. Nodes and edges in social networks are used to represent the relationship between users and groups. Group features are then extracted by calculating the degree of nodes. Movie features are extracted from movie metadata. The multi-dimensional movie features are extracted based on professional film reviews and user comments through sentiment analysis and topic modeling. Time series analysis is used to extract time features.

4. A movie rating system based on big data according to claim 1, characterized in that, The specific steps for using Naive Bayes to analyze user reviews and quantify positive / negative sentiment are as follows: Data preparation: Collect user behavior data, preprocess the text data in the user behavior data, and convert the preprocessed text data into numerical features; Bernoulli Naive Bayes Model Construction: It assumes that features are independent (the assumption of Naive Bayes) and that each feature is binary (Bernoulli distribution), meaning it only takes values ​​of 0 or 1. Based on the preprocessed data, the prior probability of each feature is calculated, which is the probability of the feature appearing in all samples. The posterior probability of each class is calculated using Bayes' theorem, which is the probability of belonging to a certain class given the observed data. The posterior probability P(C|X) is expressed as... Where P(C|X) is the posterior probability of class C given feature X, P(X|C) is the likelihood probability of feature X given class C, P(C) is the prior probability of class C, and P(X) is the prior probability of feature X. In Bernoulli Naive Bayes, it is assumed that the feature vector X has n binary features, i.e. ,in Given a category C, each feature To appear independently, that is The probability is The likelihood probability P(X|C) is expressed as ,in, It is a feature The probability of being 1 under category C. It is a feature The decision rule of the Bernoulli Naive Bayes classifier, which has a probability of 0 in category C, is expressed as follows: ,in, It is the predicted category; Sentiment quantification: Input text data into a Bernoulli Naive Bayes model to obtain the probability that each comment belongs to positive or negative sentiment. Set a threshold based on the probability value to classify comments into positive or negative sentiment. Summarize the sentiment of all comments to obtain the user's overall sentiment towards the movie.

5. A movie rating system based on big data according to claim 1, characterized in that, The specific process of using focus loss to reduce the weight of easily classified samples when the distribution of sentiment tendencies in a specific group is unbalanced is as follows: Sentiment distribution analysis: Analyze the sentiment distribution of a specific group to identify whether there is an imbalance. When the sentiment distribution of a specific group is unbalanced, that is, the number of positive or negative comments differs greatly, focus loss is used to reduce the weight of easily classified samples. Focus loss function definition: For imbalanced distributions, a focus loss function is defined. This function introduces a modulation factor to reduce the weight of easily classified samples, causing the model to focus on difficult-to-classify samples. The formula for the focus loss function is: ,in, It is the probability that the model predicts the sample belongs to the true category. It is the category weight. Modulation factor; Model optimization: During the training of the Bernoulli Naive Bayes model, the focus loss function is used instead of the cross-entropy loss function.

6. A movie rating system based on big data according to claim 1, characterized in that, Based on historical rating data, multiple XGBoost models are trained using multi-task learning to predict ratings in different dimensions. Naive Bayes analysis results provide sentiment input for different dimensions, including the following steps: Data preparation: Collect features provided by the data storage and processing layer; Multi-task learning model construction: A multi-task learning framework is adopted, and an XGBoost model is constructed for each scoring dimension. The objective function of the XGBoost model is expressed as follows: Where N is the number of users and M is the number of rating dimensions. It is user i's actual rating on dimension j. It is a model-predicted score. These are model parameters. It is the regularization coefficient; Naive Bayes analysis: Sentiment analysis results are input as additional features into the multi-task learning model; Model training and evaluation: Train the model using cross-validation, evaluate the model performance, and adjust the model parameters and feature set based on the evaluation results to optimize the model performance; Prediction and Output: Use the trained model to predict the rating of new movies.

7. A movie rating system based on big data according to claim 1, characterized in that, The specific steps for providing personalized scores for different groups by combining group characteristics are as follows: Data preprocessing: Preprocessing the collected data; Data integration and feature engineering: Integrating preprocessed data to generate a unified feature matrix; Select a K value: Preset a K value based on the actual situation; Clustering using the K-means algorithm: randomly select K initial cluster centers, assign each data point to its nearest cluster center, update the centroid of the cluster based on the mean of the data points in each cluster, repeat the assignment and update steps until the cluster centers no longer change, or the preset maximum number of iterations is reached; Analyze the clustering results: Based on the data characteristics of each cluster, analyze the representativeness of the clusters, assign labels to each cluster, and classify the clusters according to user preferences; Personalized scoring: Adjust the scoring results based on the classification results.

8. A movie rating method based on big data corresponding to the movie rating system based on big data as described in claim 1, characterized in that, include: S1: Data Collection and Integration: Collect data, including user behavior data, movie metadata, social media data, and professional film review data; S2: Data storage and processing: Use Hadoop HDFS to store the collected data, perform data cleaning and preprocessing on the collected data, and extract features from the preprocessed data; S3: Sentiment Analysis: Using the Naive Bayes algorithm to perform sentiment analysis on user behavior data and social media data, quantifying positive / negative sentiment tendencies. To address the problem of imbalanced sentiment distribution in specific groups, a focus loss function is introduced to reduce the weight of easily classified samples. S4: Multi-task learning: Based on historical rating data, multiple XGBoost models are trained using a multi-task learning framework. Each model is responsible for predicting the rating of one dimension. The sentiment tendency obtained from the Naive Bayes analysis in S3 is used as a feature input into the XGBoost model to provide reference information on sentiment tendency for different dimensions. S5: Personalized Scoring: The model prediction results are adjusted in a personalized manner based on the characteristics of the user's group, providing different groups with scores that match their preferences.

Images