Home page function module management method and system based on big data and medium

Through big data and reinforced learning technology, home page functional modules are dynamically adjusted, which solves the problem that traditional home page design is difficult to respond to user needs, and achieves a highly personalized user experience and business efficiency improvement.

CN119090573AInactive Publication Date: 2024-12-06EVERGRANDE PEIGUAN EDUCATION TECH CO LTD

Patent Information

Application Number
CN202410630750.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-21
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional homepage design and management are difficult to respond to changes in large-scale user needs in real time and accurately, resulting in limited user experience and business results.

Method used

Adopt the home page functional module management method based on big data, combined with reinforcement learning technology, collect and preprocess user behavior data, extract features, perform cluster analysis and Markov decision-making process optimization, and dynamically adjust the home page functional module to achieve more humane Personalized recommendations.

Benefits of technology

Through continuous trial and error and learning, the system can gradually optimize the display strategy, improve user satisfaction and retention rate, quickly respond to market changes, efficiently allocate resources, maximize user participation and conversion rate, and optimize resource use.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090573A_ABST
    Figure CN119090573A_ABST
Patent Text Reader

Abstract

The invention provides a home page function module management method and system based on big data and a medium, and the method comprises the steps: collecting user behavior data, and carrying out the preprocessing of the user behavior data; performing feature extraction on the user behavior data to obtain user behavior features; performing clustering analysis on the homepage function module, and constructing a user behavior analysis model; setting a comprehensive operation index score, and regarding a home page function module as a Markov decision process definition reinforcement learning environment based on the comprehensive operation index score; and monitoring the use condition of the home page function module in real time, collecting new user behavior data, optimizing a reinforcement learning environment based on strategy gradient optimization and a loss function, maximizing the environment to obtain long-term accumulated rewards, and dynamically adjusting the home page function module. Through continuous learning and optimization, the system can predict and promote the long-term behaviors of the user, such as improving the loyalty of the user and increasing the re-purchase rate, so that the maximization of the long-term value of an enterprise is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of e-commerce, and in particular relates to a method, system and medium for managing homepage function modules based on big data. Background Art

[0002] With the rapid development of the internet and mobile internet, the amount of user-generated data has exploded, making big data technology crucial for understanding and leveraging this data. For websites and applications, the homepage is the first interface for users to interact with the service, and the optimization of its functional modules directly impacts user experience and business outcomes. Traditional homepage design and management often relies on designers' intuition and limited user feedback, making it difficult to respond to the changing needs of large-scale users in real time and with precision. Summary of the Invention

[0003] The purpose of this invention is to propose a homepage function module management method, system and medium based on big data, combine reinforcement learning to manage the homepage module, make more humane personal recommendations, and meet user needs.

[0004] In order to achieve the above-mentioned object, a first aspect of the present invention provides a method for managing homepage function modules based on big data, the method comprising:

[0005] Step 1: Collect user behavior data and pre-process the user behavior data;

[0006] Step 2: Based on the pre-processed user behavior data, extract features from the user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features, and statistical features;

[0007] Step 3: Based on user behavior characteristics, perform cluster analysis on the homepage function modules and build a user behavior analysis model to classify function modules with similar user behavior characteristics into the same category;

[0008] Step 4: Based on the cluster analysis results, set the comprehensive operation index score. Based on the comprehensive operation index score, the homepage function module is regarded as a Markov decision process to define the reinforcement learning environment;

[0009] Step 5: Monitor the usage of the homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules;

[0010] The fourth step specifically includes:

[0011] S4.1. Based on the results of user cluster analysis, user group categories are obtained. Each category represents a typical behavior pattern, and a comprehensive operation index score is designed. total , which is expressed as follows:

[0012] Score total =α·Score1+β·Score2+.....γ·Scorei

[0013] 1=α+β+......+γ

[0014] Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n];

[0015] S4.2. Define the state as the current user group category, the layout of the homepage function modules for users in this category, and the corresponding comprehensive operation index score. Define the action as the layout adjustment of the homepage function modules. Design a reward function based on the change in the comprehensive operation index score. Define the strategy function π to determine the action to take based on the current state. Design the value function Critic and the strategy Actor to iteratively optimize together.

[0016] The method further comprises at least one of the following steps:

[0017] Step A: Collect multi-source data to optimize and expand the accuracy of the model, and fine-tune the model to obtain the best prediction performance;

[0018] Step B: Design a user satisfaction score. By analyzing individual user historical behavior and user satisfaction scores, generate personalized homepage function module recommendations.

[0019] Step C: Based on the comprehensive operation indicator score, continuously optimize steps 4 and 5, and dynamically adjust the homepage function modules.

[0020] Furthermore, the user behavior data includes at least one of the time, frequency, dwell time, click module and browsing path of the user visiting the homepage.

[0021] Furthermore, card detection is used to test the relevance of user behavior features in turn. The relevance L is expressed as follows:

[0022]

[0023] Where, f represents the actual frequency, f e Indicates the expected frequency;

[0024] The relevance of user behavior data is ranked according to the correlation degree to obtain the ranked user behavior data features, the features that are most highly correlated with the target variable are retained, the features are standardized, and then formed into feature vectors.

[0025] Furthermore, the cluster analysis is PSO-K means clustering, which is used to classify the homepage functional modules into different categories, including:

[0026] Randomly select s from the homepage module as the initial cluster center points, cluster the user behavior data from the initial cluster center points, and obtain X clustering results, where X ≥ s and X is a positive integer;

[0027] Obtaining M user behavior data corresponding to the X clustering results, and determining user behavior data of s initial cluster centers from the M user behavior data;

[0028] The loss function L of clustering is defined as:

[0029]

[0030] Among them, S i is the number of centers, d represents the distance, G ij Represents the j-th cluster center coordinate vector of the current user behavior data i, G j Represents the cluster center of the current user behavior data j;

[0031] Cluster user behavior data into K clusters, where K ≥ 2;

[0032] Calculate the silhouette coefficient of each user behavior data, then calculate the average silhouette coefficient, compare the average silhouette coefficients under different numbers of clusters, find the maximum value, and the corresponding k value is the optimal number of clusters.

[0033] Furthermore, in step S4.2, the iterative optimization of the design value function Critic and the strategy Actor specifically includes:

[0034] Let the state be represented as (s, L, Score total ), where s represents the current user group, and L represents the layout of the homepage function modules of the current category of users;

[0035] Set the action set to A = {a1, a2, ...a i ,a n}, where action a i Indicates different layout adjustments of home page function modules;

[0036] The reward function R(s,a,s') is designed based on the change in the comprehensive operation index score and is expressed as the reward difference:

[0037] R(s,a,s')=Score(s')-Score(s)R(s,a,s')=Score(s')-Score(s)

[0038] Use the Softmax strategy to select the optimal action based on the current state, as shown below:

[0039]

[0040] Among them, a' represents the optimal action of the current state;

[0041] Critic uses the value function V(s) to estimate the value of the state, which represents the expected long-term reward in state s. Actor updates the strategy based on the feedback of the value function.

[0042] Furthermore, the update function of the Critic is expressed as follows:

[0043] V(s)←V(s)+μ[R(s,a,s′)+γV(s′)-V(s)]

[0044] Where V(s) is the value function, γ is the discount factor, and μ represents the learning rate;

[0045] The Actor updates the policy function π according to the gradient ascent to maximize the long-term reward, which is expressed as follows:

[0046]

[0047] in, represents the gradient of J(θ), where J(θ) is a function of the parameters θ.

[0048] Furthermore, the learning rate is updated using the Adam algorithm, which is expressed as follows:

[0049] m t =β1m t-1 +(1-β1)g t

[0050]

[0051]

[0052]

[0053]

[0054] Among them, m t represents the time gradient g t The exponentially weighted moving average of t represents time t, m t-1represents the weighted moving average of the gradient at time t-1, β1 represents the decay rate of the exponentially weighted moving average, and v t Represents the gradient The exponentially weighted moving average of , β2 represents the decay rate of the exponentially weighted moving average, and They represent the deviation correction coefficients, and ∈ represents a constant that keeps the denominator stable.

[0055] Furthermore, the silhouette coefficient is calculated as follows:

[0056] For each data point i, calculate the following value:

[0057] a(i): average distance to other points in the same cluster;

[0058] b(i): the average distance to all points in the nearest cluster;

[0059] s(i)=(b(i)-a(i)) / max(a(i),b(i))

[0060] Calculate the silhouette coefficient of all data points and calculate the average as the overall silhouette coefficient. The silhouette coefficient range is [-1, 1]. The value closest to 1 indicates the best clustering effect.

[0061] According to the second aspect of the present invention, a homepage function module management system based on big data includes:

[0062] User behavior data collection module, used to collect user behavior data and pre-process the user behavior data;

[0063] A feature extraction module is used to extract features from the pre-processed user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features and statistical features;

[0064] Cluster analysis module, which is used to perform cluster analysis on homepage function modules based on user behavior characteristics, build a user behavior analysis model and classify function modules with similar user behavior characteristics into the same category;

[0065] The reinforcement learning module is used to set the comprehensive operation index score based on the cluster analysis results. Based on the comprehensive operation index score, the homepage function module is regarded as a Markov decision process to define the reinforcement learning environment;

[0066] The real-time monitoring module is used to monitor the usage of the homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules;

[0067] The steps performed by the reinforcement learning module specifically include:

[0068] S4.1. Based on the results of user cluster analysis, user group categories are obtained. Each category represents a typical behavior pattern, and a comprehensive operation index score is designed. total , which is expressed as follows:

[0069] Score total =α·Score1+β·Score2+.....γ·Scorei

[0070] 1=α+β+......+γ

[0071] Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n];

[0072] S4.2. Define the state as the current user group category, the layout of the homepage function modules for users in this category, and the corresponding comprehensive operation index score. Define the action as the layout adjustment of the homepage function modules. Design a reward function based on the change in the comprehensive operation index score. Define the strategy function π to determine the action to take based on the current state. Design the value function Critic and the strategy Actor to iteratively optimize together.

[0073] The method further comprises at least 0 of the following modules:

[0074] Module A is used to collect multi-source data to optimize and expand the accuracy of the model, and fine-tune the model to obtain the best prediction performance;

[0075] Module B is used to design user satisfaction scores. By analyzing individual user historical behavior and user satisfaction scores, it generates personalized homepage function module recommendations.

[0076] Module C is used to continuously optimize steps four and five based on the comprehensive operational indicator scores, and to dynamically adjust the homepage functional modules.

[0077] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium stores program instructions thereon, and when the program instructions are executed by a processor, the method as described in any one of the embodiments of the first aspect of the present invention is implemented.

[0078] The beneficial technical effects of the present invention are at least as follows:

[0079] This invention utilizes reinforcement learning to enable the system to continuously trial and error and learn the most effective display strategies based on historical user interaction data. By assigning positive or negative rewards to different functional module layouts and content presentations, the system can gradually optimize, presenting highly personalized homepage content to each user, thereby improving user satisfaction and retention. An adaptive learning rate allows the system to automatically adjust its learning speed based on current performance during the learning process. This means that in the early stages of exploration, the system can quickly try multiple strategies. Once the optimal solution is neared, the learning rate is slowed down and fine-tuned to ensure the optimal configuration is achieved efficiently and accurately. This enables the homepage functional modules to quickly respond to market changes and dynamic adjustments in user preferences. By using a reward function to guide the decision-making process, the system can more accurately identify which functional modules or content are most valuable to users, thereby efficiently allocating the limited homepage display space. This mechanism helps maximize user engagement and conversion rates while optimizing resource utilization and reducing unnecessary display costs. The reinforcement learning exploration mechanism encourages the system to continuously experiment with new content combinations. Combined with clustering techniques to segment user groups, this allows the homepage to not only optimize for individual users but also discover and promote new content or products that appeal to specific user groups, enhancing its ability to identify new opportunities. Through continuous learning and optimization, the system can predict and promote users' long-term behavior, such as improving user loyalty and increasing repurchase rates, thereby maximizing the long-term value of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0081] Figure 1 This is a flow chart of the homepage function module management method based on big data of the present invention.

[0082] Figure 2 This is a framework diagram of the homepage function module management system based on big data of the present invention. DETAILED DESCRIPTION

[0083] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0084] like Figure 1 As shown, the embodiment of the present invention provides a method for managing homepage function modules based on big data, the method comprising:

[0085] Step 1: Collect user behavior data and pre-process the user behavior data;

[0086] Step 2: Based on the pre-processed user behavior data, extract features from the user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features, and statistical features;

[0087] Step 3: Based on user behavior characteristics, perform cluster analysis on the homepage function modules and build a user behavior analysis model to classify function modules with similar user behavior characteristics into the same category;

[0088] Step 4: Based on the cluster analysis results, set the comprehensive operation index score. Based on the comprehensive operation index score, the homepage function module is regarded as a Markov decision process to define the reinforcement learning environment;

[0089] Step 5: Monitor the usage of the homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules;

[0090] The fourth step specifically includes:

[0091] S4.1. Based on the results of user cluster analysis, user group categories are obtained. Each category represents a typical behavior pattern, and a comprehensive operation index score is designed. total , which is expressed as follows:

[0092] Score total =α·Score1+β·Score2+.....γ·Scorei

[0093] 1=α+β+......+γ

[0094] Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n];

[0095] S4.2. Define the state as the current user group category, the layout of the homepage function modules for users in this category, and the corresponding comprehensive operation index score. Define the action as the layout adjustment of the homepage function modules. Design a reward function based on the change in the comprehensive operation index score. Define the strategy function π to determine the action to take based on the current state. Design the value function Critic and the strategy Actor to iteratively optimize together.

[0096] The method further comprises at least one of the following steps:

[0097] Step A: Collect multi-source data to optimize and expand the accuracy of the model, and fine-tune the model to obtain the best prediction performance;

[0098] Step B: Design a user satisfaction score. By analyzing individual user historical behavior and user satisfaction scores, generate personalized homepage function module recommendations.

[0099] Step C: Based on the comprehensive operation indicator score, continuously optimize steps 4 and 5, and dynamically adjust the homepage function modules.

[0100] Furthermore, the user behavior data includes at least one of the time, frequency, dwell time, click module and browsing path of the user visiting the homepage.

[0101] Furthermore, card detection is used to test the relevance of user behavior features in turn. The relevance L is expressed as follows:

[0102]

[0103] Where, f represents the actual frequency, f e Indicates the expected frequency;

[0104] The relevance of user behavior data is ranked according to the correlation degree to obtain the ranked user behavior data features, the features that are most highly correlated with the target variable are retained, the features are standardized, and then formed into feature vectors.

[0105] Furthermore, the cluster analysis is PSO-K means clustering, which is used to classify the homepage functional modules into different categories, including:

[0106] Randomly select s from the homepage module as the initial cluster center points, cluster the user behavior data from the initial cluster center points, and obtain X clustering results, where X ≥ s and X is a positive integer;

[0107] Obtaining M user behavior data corresponding to the X clustering results, and determining user behavior data of s initial cluster centers from the M user behavior data;

[0108] The loss function L of clustering is defined as:

[0109]

[0110] Among them, S i is the number of centers, d represents the distance, G ij Represents the j-th cluster center coordinate vector of the current user behavior data i, G j Represents the cluster center of the current user behavior data j;

[0111] Cluster user behavior data into K clusters, where K ≥ 2;

[0112] Calculate the silhouette coefficient of each user behavior data, then calculate the average silhouette coefficient, compare the average silhouette coefficients under different numbers of clusters, find the maximum value, and the corresponding k value is the optimal number of clusters.

[0113] Furthermore, in step S4.2, the iterative optimization of the design value function Critic and the strategy Actor specifically includes:

[0114] Let the state be represented as (s, L, Score total ), where s represents the current user group, and L represents the layout of the homepage function modules of the current category of users;

[0115] Set the action set to A = {a1, a2, ...a i ,a n}, where action a i Indicates different layout adjustments of home page function modules;

[0116] The reward function R(s,a,s') is designed based on the change in the comprehensive operation index score and is expressed as the reward difference:

[0117] R(s,a,s')=Score(s')-Score(s)R(s,a,s')=Score(s')-Score(s)

[0118] Use the Softmax strategy to select the optimal action based on the current state, as shown below:

[0119]

[0120] Among them, a' represents the optimal action of the current state;

[0121] Critic uses the value function V(s) to estimate the value of the state, which represents the expected long-term reward in state s. Actor updates the strategy based on the feedback of the value function.

[0122] Furthermore, the update function of the Critic is expressed as follows:

[0123] V(s)←V(s)+μ[R(s,a,s′)+γV(s′)-V(s)]

[0124] Where V(s) is the value function, γ is the discount factor, and μ represents the learning rate;

[0125] The Actor updates the policy function π according to the gradient ascent to maximize the long-term reward, which is expressed as follows:

[0126]

[0127] in, represents the gradient of J(θ), where J(θ) is a function of the parameters θ.

[0128] Furthermore, the learning rate is updated using the Adam algorithm, which is expressed as follows:

[0129] m t =β1m t-1 +(1-β1)g t

[0130]

[0131]

[0132]

[0133]

[0134] Among them, m t represents the time gradient g t The exponentially weighted moving average of t represents time t, m t-1 represents the weighted moving average of the gradient at time t-1, β1 represents the decay rate of the exponentially weighted moving average, and v t Represents the gradient The exponentially weighted moving average of , β2 represents the decay rate of the exponentially weighted moving average, and They represent the deviation correction coefficients, and ∈ represents a constant that keeps the denominator stable.

[0135] Furthermore, the silhouette coefficient is calculated as follows:

[0136] For each data point i, calculate the following value:

[0137] a(i): average distance to other points in the same cluster;

[0138] b(i): the average distance to all points in the nearest cluster;

[0139] s(i)=(b(i)-a(i)) / max(a(i),b(i))

[0140] Calculate the silhouette coefficient of all data points and calculate the average as the overall silhouette coefficient. The silhouette coefficient range is [-1, 1]. The value closest to 1 indicates the best clustering effect.

[0141] Furthermore, in step A, data is collected from various channels and sources, including but not limited to log files, API interfaces, public databases, social media, and sensor data. This ensures both breadth and depth of data coverage to fully reflect the diversity of the prediction targets. The data is then preprocessed to address missing values, outliers, perform data deduplication, and perform other preprocessing tasks such as data type conversion. ETL (Extract, Transform, Load) tools or custom scripts are used to integrate data from various sources, ensuring a consistent data format for subsequent analysis.

[0142] Based on domain knowledge and statistical analysis, select the features that have the greatest impact on the prediction target. Feature selection can be performed using methods such as feature importance assessment and correlation analysis. Based on business logic and data characteristics, create new features, such as derived features and aggregated features, to improve the model's expressiveness. Encode categorical variables (e.g., one-hot encoding, label encoding), and normalize or standardize continuous variables.

[0143] Select an appropriate machine learning or deep learning model based on the nature of the problem (regression, classification, clustering, etc.) and the size of the data. Try a variety of models, such as linear regression, random forest, support vector machine, and neural network. Use cross-validation (such as K-fold cross-validation) to assess model generalization and avoid overfitting. Train the selected model on the cleaned and processed data, paying attention to adjusting hyperparameters to optimize model performance.

[0144] Use appropriate evaluation metrics (such as precision, recall, F1 score, AUC-ROC curve, etc.) to quantify model performance. Based on the evaluation results, make detailed adjustments to the model, including but not limited to adjusting the learning rate, changing the network structure, employing regularization techniques, and adjusting the feature set. Use adaptive learning rate algorithms (such as Adam and RMSprop) to automatically adjust the learning rate to accelerate convergence and improve model accuracy. Finally, use ensemble learning methods such as voting, bagging, boosting, or stacking to combine the predictions of multiple models to further improve prediction accuracy and robustness.

[0145] Furthermore, in step B, first, collect the user's historical behavior data, including but not limited to browsing history, click-through rate, dwell time, purchase history, search keywords, and interactive participation (such as likes and comments). At the same time, collect user satisfaction rating data, which may come from questionnaires, product reviews, or direct feedback. Remove incomplete, erroneous, or abnormal data records, handle missing values, and unify data formats. Extract useful features from the raw data, such as user active time periods, preferred categories, consumption frequency, etc., to prepare for subsequent model training.

[0146] Based on user behavior and satisfaction data, clustering algorithms (such as K-means and DBSCAN) are used to group users and identify groups of users with similar behavior patterns and preferences. A detailed profile is created for each user or user group, including basic information, interests, preferences, behavioral habits, and satisfaction trends.

[0147] Finally, based on the user profile and historical satisfaction scores, features highly correlated with satisfaction are selected. A satisfaction prediction model is trained using machine learning models (such as linear regression, decision trees, random forests, gradient boosting trees, or neural networks). The input features are user behavior and historical data, and the output is the predicted satisfaction score. Model performance is evaluated through methods such as cross-validation and A / B testing, and model parameters are adjusted based on the evaluation results to optimize prediction accuracy. Based on user-feature module interaction data, collaborative filtering algorithms (user-based, item-based) are used to discover users' potential interests and preferences. The content attributes of the feature modules (such as categories and tags) are analyzed to match them with the preferences in the user profile, and feature modules with similar content are recommended. This is combined with model-based recommendations (such as matrix factorization and deep learning recommendation systems) that comprehensively consider user behavior, satisfaction prediction results, and content attributes to generate a personalized recommendation list. This real-time feedback data is fed back into the model to continuously update the user profile and adjust the recommendation algorithm weights, achieving continuous optimization of personalized recommendations. Regular A / B testing is conducted to compare the effectiveness of different recommendation strategies and select the optimal one.

[0148] Furthermore, in step C, periodically or when sufficient new data is collected, the policy network is trained with the latest data, and the policy π(a|s) is updated using the policy gradient method. During the policy gradient training process, an appropriate loss function (such as the negative log-likelihood of the advantage function estimate) is defined to guide the direction of gradient updates and optimize the policy to achieve higher cumulative rewards. Based on the optimized policy, the layout of the homepage functional modules is dynamically adjusted, the optimal policy is implemented, and the real-time monitoring phase is re-entered, forming a closed-loop feedback system.

[0149] In addition, the system of the embodiment of the present invention is a homepage function module management system based on big data, including:

[0150] User behavior data collection module 101, used to collect user behavior data and pre-process the user behavior data;

[0151] A feature extraction module 102 is configured to extract features from the pre-processed user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features, and statistical features;

[0152] Cluster analysis module 103, used to perform cluster analysis on homepage function modules based on user behavior characteristics, build a user behavior analysis model and classify function modules with similar user behavior characteristics into the same category;

[0153] A reinforcement learning module 104 is configured to set a comprehensive operation index score based on the cluster analysis results, and define a reinforcement learning environment by treating the homepage function module as a Markov decision process based on the comprehensive operation index score;

[0154] Real-time monitoring module 105 is used to monitor the usage of homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules;

[0155] The steps performed by the reinforcement learning module specifically include:

[0156] S4.1. Based on the results of user cluster analysis, user group categories are obtained. Each category represents a typical behavior pattern, and a comprehensive operation index score is designed. total , which is expressed as follows:

[0157] Score total =α·Score1+β·Score2+.....γ·Scorei

[0158] 1=α+β+......+γ

[0159] Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n];

[0160] S4.2. Define the state as the current user group category, the layout of the homepage function modules for users in this category, and the corresponding comprehensive operation index score. Define the action as the layout adjustment of the homepage function modules. Design a reward function based on the change in the comprehensive operation index score. Define the strategy function π to determine the action to take based on the current state. Design the value function Critic and the strategy Actor to iteratively optimize together.

[0161] The method further comprises at least 0 of the following modules:

[0162] Module A is used to collect multi-source data to optimize and expand the accuracy of the model, and fine-tune the model to obtain the best prediction performance;

[0163] Module B is used to design user satisfaction scores. By analyzing individual user historical behavior and user satisfaction scores, it generates personalized homepage function module recommendations.

[0164] Module C is used to continuously optimize steps four and five based on the comprehensive operational indicator scores, and to dynamically adjust the homepage functional modules.

[0165] It should be appreciated that the method steps in the embodiments of the present invention can be implemented or executed by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can be run on a programmed application-specific integrated circuit.

[0166] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.

[0167] Further, the method can be implemented in any type of Android computing platform that is operably connected to a suitable one. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described herein, the present invention can also include the computer itself.

[0168] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data that is stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0169] Corresponding to the aforementioned embodiments, the present invention also provides system embodiments. For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0170] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a division of logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0172] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A homepage function module management method based on big data, characterized in that: The method comprises: Step 1: Collect user behavior data and pre-process the user behavior data; Step 2: Based on the preprocessed user behavior data, extract features from the user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features and statistical features; Step 3: Based on user behavior characteristics, cluster analysis is performed on the homepage function modules, and a user behavior analysis model is constructed to classify function modules with similar user behavior characteristics into the same category; Step 4: Based on the cluster analysis results, set the comprehensive operation index score, and define the reinforcement learning environment by considering the homepage function module as a Markov decision process based on the comprehensive operation index score; Step 5: Monitor the usage of the homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules; Wherein, the step 4 specifically includes: S4.

1. Based on the results of user cluster analysis, we obtain user group categories. Each category represents a typical behavior pattern and designs a comprehensive operation index score. total , which is expressed as follows: Score total =α·Score1+β·Score2+.....γ·Scorei 1=α+β+......+γ Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n]; S4.2, define the state as the current user group category and the layout of the homepage function modules of the current category users and the corresponding comprehensive operation index score. The action is defined as the layout adjustment of the homepage function modules. Design the reward function based on the change of the comprehensive operation index score, define the strategy function π, decide what action to take according to the current state, and design the value function Critic and the strategy Actor to jointly iterate and optimize; The method further comprises at least one of the following steps: Step A: Collect multi-source data to optimize and expand the accuracy of the model, and fine-tune and optimize the model to obtain the best prediction performance; Step B: Design user satisfaction ratings, and generate personalized homepage function module recommendations by analyzing individual user historical behaviors and user satisfaction ratings; Step C: Based on the comprehensive operation indicator score, continuously optimize steps 4 and 5, and dynamically adjust the home page function modules.

2. A method for managing homepage function modules based on big data according to claim 1, characterized in that: The user behavior data includes at least one of the time, frequency, dwell time, click module and browsing path of the user visiting the homepage.

3. A method for managing homepage function modules based on big data according to claim 2, characterized in that: Use card detection to test the relevance of user behavior features in turn. The relevance L is expressed as follows: Where, f represents the actual frequency, f e represents the expected frequency; The relevance of user behavior data is ranked according to the degree of association to obtain the ranked user behavior data features, the features most highly correlated with the target variable are retained, the features are standardized, and then formed into feature vectors.

4. A method for managing homepage function modules based on big data according to claim 3, characterized in that: The cluster analysis is PSO-K means clustering, which is used to classify the homepage function modules into different categories, including: Randomly select s from the homepage modules as the initial clustering centers, cluster the user behavior data from the initial clustering centers, and obtain X clustering results, where X ≥ s, and X is a positive integer; Obtaining M user behavior data corresponding to the X-type clustering results, and determining user behavior data of s initial cluster center points from the M user behavior data; The loss function L of clustering is defined as: Among them, S i is the number of centers, d represents the distance, G ij represents the j-th cluster center coordinate vector of the current user behavior data i, G j Represents the cluster center of the current user behavior data j; Cluster the user behavior data into K clusters, where K ≥ 2; Calculate the silhouette coefficient of each user behavior data, then calculate the average silhouette coefficient, compare the average silhouette coefficients under different numbers of clusters, find the maximum value, and the corresponding k value is the optimal number of clusters.

5. The method for managing homepage function modules based on big data according to claim 1, characterized in that: In step S4.2, the design of the value function Critic and the strategy Actor for joint iterative optimization specifically includes: Let the state be represented as (s, L, Score total ), where s represents the current user group, and L represents the layout of the homepage function modules of the current category of users; Set the action set to A = {a1, a2, ...a i ,a n }, where action a i Indicates the layout adjustments of different home page function modules; The reward function R(s,a,s') is designed based on the change in the comprehensive operation index score and is expressed using the reward difference: R(s,a,s')=Score(s')-Score(s)R(s,a,s')=Score(s')-Score(s) Use the Softmax strategy to select the optimal action based on the current state, as shown below: Among them, a' represents the optimal action of the current state; Critic uses the value function V(s) to estimate the value of the state, which represents the expected long-term reward in state s. Actor updates the strategy based on the feedback of the value function.

6. A method for managing homepage function modules based on big data according to claim 5, characterized in that: The update function of the Critic is expressed as follows: V(s)←V(s)+μ[R(s,a,s′)+γV(s′)-V(s)] Among them, V(s) is the value function, γ is the discount factor, and μ represents the learning rate; The Actor updates the strategy function π according to the gradient ascent to maximize the long-term reward, which is expressed as follows: in, represents the gradient of J(θ), where J(θ) is a function of the parameters θ.

7. A method for managing homepage function modules based on big data according to claim 6, characterized in that: in, The learning rate is updated using the Adam algorithm, expressed as follows: m t =β1m t-1 +(1-β1)g t Among them, m t represents the time gradient g t The exponentially weighted moving average of , t represents time t, m t-1 represents the weighted moving average of the gradient at time t-1, β1 represents the decay rate of the exponentially weighted moving average, and v t Represents the gradient The exponentially weighted moving average of , β2 represents the decay rate of the exponentially weighted moving average, and They represent the deviation correction coefficients respectively, and ∈ represents a constant that keeps the denominator stable.

8. A method for managing homepage function modules based on big data according to claim 4, characterized in that: The silhouette coefficient is calculated as follows: For each data point i, calculate the following value: a(i): average distance to other points in the same cluster; b(i): the average distance to all points in the nearest cluster; s(i)=(b(i)-a(i)) / max(a(i),b(i)) Calculate the silhouette coefficient of all data points and calculate the average value as the overall silhouette coefficient. The silhouette coefficient range is [-1, 1]. The value closest to 1 indicates the best clustering effect.

9. A homepage function module management system based on big data, characterized in that: include: A user behavior data collection module is used to collect user behavior data and pre-process the user behavior data; A feature extraction module is used to extract features from the user behavior data based on the preprocessed user behavior data to obtain user behavior features; wherein the features include basic features, time features, sequence features, social network features, advanced features and statistical features; The cluster analysis module is used to perform cluster analysis on the homepage function modules according to the user behavior characteristics, and to build a user behavior analysis model to classify the function modules with similar user behavior characteristics into the same category; A reinforcement learning module is used to set a comprehensive operation index score based on the cluster analysis results, and define a reinforcement learning environment by considering the homepage function module as a Markov decision process based on the comprehensive operation index score; Real-time monitoring module, used to monitor the usage of homepage function modules in real time, collect new user behavior data, optimize the reinforcement learning environment based on policy gradient optimization and loss function, maximize the environment to obtain long-term accumulated rewards, and dynamically adjust the homepage function modules; The steps performed by the reinforcement learning module specifically include: S4.

1. Based on the results of user cluster analysis, we obtain user group categories. Each category represents a typical behavior pattern and designs a comprehensive operation index score. total , which is expressed as follows: Score total =α·Score1+β·Score2+.....γ·Scorei 1=α+β+......+γ Among them, α, β and γ are dynamic weights used to adjust the importance of different indicators; Score1, Score2, Scorei represent the scores of different indicators, i∈[1,2......i,n]; S4.2, define the state as the current user group category and the layout of the homepage function modules of the current category users and the corresponding comprehensive operation index score. The action is defined as the layout adjustment of the homepage function modules. Design the reward function based on the change of the comprehensive operation index score, define the strategy function π, decide what action to take according to the current state, and design the value function Critic and the strategy Actor to jointly iterate and optimize; The method further comprises at least 0 of the following modules: Module A is used to collect multi-source data to optimize and expand the accuracy of the model, and fine-tune and optimize the model to obtain the best prediction performance; Module B is used to design user satisfaction scores. It generates personalized homepage function module recommendations by analyzing individual user historical behaviors and user satisfaction scores. Module C is used to continuously optimize steps 4 and 5 based on the comprehensive operation indicator score, and dynamically adjust the home page function modules. 10 . A computer-readable storage medium having program instructions stored thereon, wherein the program instructions are executed by a processor to implement the method according to claim 1 .

Citation Information

Patent Citations

  • Method and equipment for displaying personalized page

    CN108280128A

  • Business data association analysis method, device and equipment

    CN112487021A

  • Personalized recommendation method and device, storage medium and computer equipment

    CN117455621A

Cited By

  • Intelligent graphical interface operation optimization method based on image processing

    CN120104233A