Social platform distribution strategy detection method, system and equipment based on improved Apriori algorithm

Abstract

The invention discloses a social platform distribution strategy detection method, system and device based on an improved Apriori algorithm, and belongs to the technical field of information recommendation. The method comprises the following steps: S1, integrating a crawler platform, and obtaining data of each platform; s2, preprocessing the platform data, and storing the platform data in a database; s3, importing the database data into a model, and performing embedding to obtain a user vector; s4, inputting the user vector into a feature extraction model for feature extraction to obtain recommended features; and S5, discretizing the recommendation features, mapping the discretized recommendation features into recommendation tags, mining the importance degree of the recommendation tags by using an improved Apriori algorithm, and obtaining a platform distribution strategy according to the importance degree. A solution is provided for the accuracy and efficiency of distribution strategy detection in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of information recommendation, and more specifically to a social platform distribution strategy detection method, system and device based on an improved Apriori algorithm. Background Art

[0002] In today's digital age, people are faced with a large amount of information pouring in from the Internet every day and selectively accepting and interpreting this information. This information comes from various online platforms, including e-commerce, social media, news websites, music streaming, and online videos. This information overload problem has become a major challenge in the digital society, because users are usually unable to effectively filter and process a large amount of information while maintaining satisfaction and efficiency. At this time, recommendation systems have emerged to solve this core problem. The core goal of recommendation systems is to help users discover content, products, or services that they may be interested in without getting lost in the vast amount of information. These systems are information filtering tools that analyze users' personal preferences, historical behaviors, social connections, and the characteristics of items to provide personalized suggestions and recommendations. By based on users' needs and interests, recommendation systems help reduce information overload, improve user satisfaction, and help companies increase user engagement and sales conversion rates.

[0003] As mentioned above, recommendation systems have become an indispensable part of online platforms and applications. These systems are designed to help users discover content, products or services that they may be interested in, in order to improve user experience, user engagement and business performance. However, while providing personalized suggestions, recommendation systems also raise a series of problems and challenges, such as the accuracy and efficiency of distribution strategy detection in existing technologies.

[0004] Therefore, how to provide a social platform distribution strategy detection method, system and device based on an improved Apriori algorithm that can solve the above problems is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a social platform distribution strategy detection method, system and device based on the improved Apriori algorithm, which extracts recommendation features from the recommendation algorithm platform, discretizes the recommendation features, maps them to recommendation tags, and uses the improved Apriori algorithm to mine the importance of recommendation tags, and derives the platform distribution strategy according to the importance. The accuracy and efficiency of social platform distribution strategy detection are improved.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A social platform distribution strategy detection method based on an improved Apriori algorithm, comprising:

[0008] S1: Integrate crawler platforms to obtain data from various platforms;

[0009] S2: pre-process the platform data and save it to a database;

[0010] S3: importing the data of the database into the model and embedding it to obtain a user vector;

[0011] S4: Inputting the user vector into a feature extraction model to extract features and obtain recommended features;

[0012] S5: Discretize the recommendation features, map them into recommendation tags, and use the improved Apriori algorithm to mine the importance of the recommendation tags, and derive a platform distribution strategy based on the importance.

[0013] Preferably, the S1 comprises:

[0014] S1.1: Use web crawlers to write code for pages on multiple platforms, encapsulate the crawler codes of multiple platforms into corresponding modules, and build a crawler platform using the front-end and back-end;

[0015] S1.2: Use a crawler platform to crawl data from each platform, where the platform data includes user data, user behavior data, and platform push data.

[0016] Preferably, S2 includes:

[0017] The platform data is cleaned to remove illegal characters and empty characters, and the cleaned platform data is stored in the database.

[0018] Preferably, S3 includes:

[0019] The database is connected through assembly language, and the data in the database is input into the RoBERTa model and embedded to obtain a user vector. The RoBERTa model introduces layer normalization after self-attention output, and each sample is independently standardized at each position. The RoBERTa model uses a multi-layer Transformer Encoder to obtain the final text representation to obtain a user vector.

[0020] Preferably, the feature extraction model includes forward propagation, backward propagation, a bidirectional long short-term memory network Bi-LSTM module and an Attention mechanism.

[0021] Preferably, S5:

[0022] S5.1: discretize the recommendation features and map them into recommendation labels;

[0023] S5.2: The recommended tag data of each user obtained in S5.1 is stored in the database, and the corresponding bit vector conversion is performed. The conversion rule is: if the corresponding recommended tag exists, the corresponding value is set to 1, otherwise it is set to 0;

[0024] S5.3: By scanning the database once, construct a bit vector representation of all user recommendation label data to capture the distribution of user recommendation label data in the entire database;

[0025] S5.4: Use the inner product operation of the bit vector to obtain the frequency of each recommended tag in the database, and use the frequency as the local support;

[0026] S5.5: Merge item sets with the same prefix by performing an AND operation on the bit vectors;

[0027] S5.6: Eliminate infrequent item sets and generate candidate item sets;

[0028] S5.7: Divide the data into multiple blocks according to the user recommendation tag data in the database, and distribute them to multiple processors through hash partitioning. Each processor independently maps the assigned user recommendation tag data to a bit vector and performs bit operations. The local support calculated by all processors is merged to obtain the global support.

[0029] S5.8: The global support is used as the importance of the recommended tag, and the platform distribution strategy is derived according to the importance.

[0030] A social platform distribution strategy detection system based on an improved Apriori algorithm, comprising:

[0031] Data collection module, integrating crawler platform to obtain data from various platforms;

[0032] A database construction module pre-processes the platform data and saves it to the database;

[0033] A vectorization module imports the database data into the model and embeds it to obtain a user vector;

[0034] A recommendation feature extraction module inputs the user vector into a feature extraction model to extract features and obtain recommendation features;

[0035] The intelligent recommendation strategy generation module discretizes the recommendation features, maps them into recommendation tags, and uses the improved Apriori algorithm to mine the importance of the recommendation tags, and derives the platform distribution strategy based on the importance.

[0036] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a social platform distribution strategy detection method based on an improved Apriori algorithm is implemented.

[0037] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method, system and device for detecting the distribution strategy of a social platform based on an improved Apriori algorithm, which extracts recommendation features from the recommendation algorithm platform, discretizes the recommendation features, maps them to recommendation tags, and uses the improved Apriori algorithm to mine the importance of the recommendation tags, and derives the platform distribution strategy based on the importance. The accuracy and efficiency of the detection of the distribution strategy of the social platform are improved, and the specific beneficial effects are:

[0038] (1) The present invention improves the Apriori algorithm by adopting bit vector mapping and bit operation methods, and combining parallel computing to obtain an improved Apriori algorithm. This improvement not only optimizes the performance of the algorithm when processing large-scale data sets, but also improves the operating efficiency of the algorithm, mainly in terms of reducing the number of database scans and parallel processing;

[0039] (2) The present invention uses crawler technology and combines front-end and back-end technologies to build a crawler platform to efficiently collect platform user data, user information, and platform push data, and on this basis realizes the visualization of data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0042] Figure 2 The method flow chart of the present invention;

[0043] Figure 3 This is a diagram of the crawler framework used in the data collection module;

[0044] Figure 4 It is a feature extraction framework diagram of the feature extraction model;

[0045] Figure 5 A framework for data mining-based distribution strategy detection and evaluation;

[0046] Figure 6 This is the architecture diagram of the platform data collection system based on front-end and back-end technologies;

[0047] Figure 7 The main idea diagram for the improved Apriori algorithm implementation;

[0048] Figure 8 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a social platform distribution strategy detection method, system and device based on the improved Apriori algorithm. Figure 1 , the method includes: S1: integrating crawler platforms to obtain data from various platforms; S2: preprocessing the platform data and saving them to a database; S3: importing the database data into a model and embedding them to obtain a user vector; S4: inputting the user vector into a feature extraction model to extract features and obtain recommendation features; S5: discretizing the recommendation features and mapping them to recommendation tags, and using an improved Apriori algorithm to mine the importance of the recommendation tags, and deriving a platform distribution strategy based on the importance. A solution is generally provided for the accuracy and efficiency of distribution strategy detection in the prior art.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] See also Figure 2 The embodiment of the present invention discloses a social platform distribution strategy detection method based on an improved Apriori algorithm, which specifically includes:

[0053] S1: Through front-end and back-end technologies, the platform crawlers are integrated to form a crawler platform. Taking the Douyin platform as an example, after configuring the corresponding parameters through the crawler platform, such as the number of crawls and the range of crawled data, user information, user history, user online time, user interaction number (likes, favorites, and followers), platform push, user preferences and other data are obtained, and then the obtained data is saved in json format;

[0054] S2: After obtaining the data of relevant users on the Douyin platform, clean the data, remove illegal characters and empty characters, and then store the data in the MySQL database;

[0055] S3: Connect to the database, retrieve the saved user data, take user behavior, user information, and platform push data as input, embed them through the Bert model, and obtain the user vector;

[0056] S4: The user vector obtained in S3 is used as input to the feature extraction model (Bi-LSTM+Attention model) to extract features and obtain recommendation features.

[0057] S5: Discretize the recommendation features obtained in S4 and map them to recommendation tags, such as global popularity and classification popularity tags. Use the improved Apriori algorithm to mine the importance of the recommendation tags, and derive the distribution strategy of the platform based on the importance of the recommendation tags.

[0058] In a specific embodiment, S1 specifically includes:

[0059] S1.1: First, use crawler technology to write code for pages on multiple platforms. The crawler process framework is as follows: Figure 3 As shown in the figure, ensure that the code can run normally and the data can be obtained correctly, then encapsulate the crawler codes of these platforms into modules, and then use the front-end and back-end technologies vue+springboot+mysql to build the crawler platform, such as Figure 6 As shown, the user can input the URL and then configure parameters, such as the number of crawls and the range of crawled data. After clicking Start, data collection can be carried out, and the progress of data collection and the amount of data obtained can be displayed.

[0060] S1.2: Use the crawler platform to crawl the user data, user behavior data and platform push data of the Douyin platform, and save the data in json format.

[0061] In a specific embodiment, S2 specifically includes:

[0062] Use the pandas library in the python library to clean the json data and remove illegal data such as duplication and missing data. For some difficult data, regular expressions can be used to clean the data and save the cleaned data to the mysql database for subsequent data analysis.

[0063] Feature extraction framework based on deep learning feature extraction model such as Figure 4 As shown, S3 is specifically:

[0064] Connect to the database, retrieve the saved user data, take user behavior, user information, and platform push data as input, embed them through the RoBERTa model, and get the user vector. Some of the core formulas included in RoBERTa are as follows:

[0065] Self-attention formula:

[0066]

[0067] in, is the attention weight, which is the output obtained by weighted summation of the input sequence, is the first elements, is the first Elements, is the length of the input sequence.

[0068] Layer normalization,RoBERTa introduces layer normalization after the self-attention output, and performs independent normalization on each sample at each position.

[0069]

[0070] Among them, μ m and σ m are the mean and standard deviation of the mth position respectively.

[0071] Positional embedding:

[0072]

[0073]

[0074] Here, i is the position of the word in the sequence, j is the index of the embedding dimension, and d is the embedding dimension.

[0075] Output of Transformer Encoder:

[0076] The RoBERTa model uses a multi-layer Transformer Encoder model to obtain the final text representation, and its output is represented as:

[0077] H=RoBERTa(X)

[0078] =TransformerEncoder(LayerNorm(SelfAttention(X)

[0079] +PositionalEmbeddings(X)))(5)

[0080] In the formula, X represents user text data.

[0081] In a specific embodiment, S4 specifically includes:

[0082] The user vector obtained in S4 is used as input to the feature extraction model (Bi-LSTM+Attention model) for feature extraction to obtain the recommendation features. In Bi-LSTM, since it is bidirectional, there are two directions: forward and backward:

[0083] Forward propagation:

[0084]

[0085] Backward propagation:

[0086]

[0087] in, and Represent the forward and backward hidden states respectively, x t is the input sequence at time t.

[0088] In the Bi-LSTM model, the forward and backward hidden states are merged together through a concatenation operation to form the final bidirectional hidden state:

[0089]

[0090] The Attention mechanism is used to weight the information at different positions in the sequence, so as to focus more on the parts that are useful for the current task:

[0091] a t =softmax(v·tanh(W·h t ,c])) 9)

[0092] c t =∑ t′ α t′ ·h t′ (10)

[0093] Among them, a t is the attention weight at position t, c t is the context vector computed from these weights, and W and v are learning parameters.

[0094] Reference Figure 5 The data mining distribution strategy detection and evaluation framework diagram, S5:

[0095] The recommendation features obtained by S4 are discretized through a clustering algorithm, mapped to the recommendation tags, and then the improved Apriori algorithm is used to mine the importance of the recommendation tags.

[0096] Reference Figure 7 As shown, in a specific embodiment, the Apriori algorithm is improved by using bit vector mapping and bit operation methods, and combined with parallel computing, to obtain an improved Apriori algorithm, wherein the specific optimization strategy of the Apriori algorithm is:

[0097] S5.1: discretize the recommendation features and map them into recommendation labels;

[0098] S5.2: Bit vector mapping: The recommended tag data corresponding to each user is stored in the database, and the corresponding bit vector conversion is performed. The conversion rule is: if the corresponding recommended tag exists, the corresponding value is set to 1, otherwise it is set to 0;

[0099] S5.3: One-time scan: By scanning the database once, a bit vector representation of all user recommendation tag data is constructed, thereby capturing the distribution of user recommendation tag data in the entire database;

[0100] S5.4: Inner product calculation support: Using the inner product operation of the bit vector, the frequency of each recommended tag in the database, i.e., the support, is quickly obtained. The calculation result of the inner product directly reflects the number of concurrent occurrences of the item;

[0101] S5.5: Generate candidate item sets through “AND” operation: Through the “AND” operation of bit vectors, item sets with the same prefix are merged to efficiently generate candidate item sets, laying the foundation for the next step of frequent item set identification;

[0102] S5.6: Pre-pruning optimization: In the process of generating candidate item sets, by excluding those known infrequent item sets, ineffective scanning of low-dimensional frequent item sets is avoided;

[0103] S5.7: Parallel computing: Divide the data into multiple blocks according to the user recommendation label data in the database, and distribute them to multiple processors through hash partitioning. Each processor independently maps the assigned user recommendation label data to a bit vector, and then performs bit operations. Finally, the local support calculated by all processors is merged to obtain the global support.

[0104] S5.8: The global support is used as the importance of the recommended tag, and the platform distribution strategy is derived according to the importance.

[0105] In a specific embodiment, taking the Douyin platform as an example:

[0106] Assume that the recommended labels mapped out by user 1's data are A, B, and C, the recommended labels mapped out by user 2's data are A, C, and D, and the recommended label mapped out by user 3's data is B. The recommended labels of these three users are bit-mapped to vectors, and the itemset A is 110, the itemset B is 101, the itemset C is 110, and the itemset D is 010. Then the inner product is calculated. The inner product of itemset A is 2, the inner product of itemset B is 2, the inner product of itemset C is 2, and the inner product of itemset D is 1. Then the frequent itemsets are mapped two by two. Row AND operation, item set (A, B) = 110&101 = 100, item set (A, C) = 110&110 = 110, item set (B, C) = 101&110 = 100, and finally calculate the inner product, item set (A, B) = 1, item set (A, C) = 2, item set (B, C) = 1, and finally the item set (A, C) is the maximum support, that is, the recommendation label combination of A and C is more important. Finally, the distribution strategy of the platform is obtained according to the importance of the recommended labels.

[0107] See attached Figure 8 As shown, a social platform distribution strategy detection system based on an improved Apriori algorithm includes:

[0108] Data collection module, integrating crawler platform to obtain data from various platforms;

[0109] A database construction module pre-processes the platform data and saves it to the database;

[0110] A vectorization module imports the database data into the model and embeds it to obtain a user vector;

[0111] A recommendation feature extraction module inputs the user vector into a feature extraction model to extract features and obtain recommendation features;

[0112] The intelligent recommendation strategy generation module discretizes the recommendation features, maps them into recommendation tags, and uses the improved Apriori algorithm to mine the importance of the recommendation tags, and derives the platform distribution strategy based on the importance.

[0113] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a social platform distribution strategy detection method based on an improved Apriori algorithm is implemented.

[0114] The social platform distribution strategy detection method provided by the present invention first combines crawler technology with front-end and back-end technologies to form a visual data collection system, which can make users use it more conveniently and store data more securely. Then, a deep learning model is used to extract features of user information and platform push data obtained from the data collection system for subsequent data analysis and other operations. Finally, data mining technology is used to mine and analyze the recommended features to obtain the platform's distribution strategy.

[0115] In a specific embodiment, the present invention can be applied to social media platforms such as TikTok and Bilibili, and is expected to achieve the following effects:

[0116] (1) Content recommendation optimization: By detecting and analyzing the distribution strategy of the platform, we can discover the strengths and weaknesses of the current recommendation system, and then optimize the algorithm to improve the accuracy of content recommendations and user satisfaction.

[0117] (2) Enhance user stickiness: By understanding users’ preferences and interaction patterns for different types of content, the platform can adjust its distribution strategy and provide content that is more in line with users’ interests, thereby increasing users’ stay time on the platform and improving user stickiness.

[0118] (3) Improving content exposure: By detecting distribution strategies, the platform can identify which content is more likely to gain high exposure, thereby providing guidance to content creators to help them create more popular content.

[0119] (4) Risk management: By detecting and analyzing distribution strategies, the platform can promptly detect and respond to potential risks, such as content abuse and the spread of false information.

[0120] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0121] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A social platform distribution strategy detection method based on an improved Apriori algorithm, characterized in that: include: S1: Integrate crawler platforms to obtain data from various platforms; S2: pre-process the platform data and save it to a database; S3: importing the data of the database into the model and embedding it to obtain a user vector; S4: Inputting the user vector into a feature extraction model to extract features and obtain recommended features; S5: Discretize the recommendation features, map them into recommendation tags, and use the improved Apriori algorithm to mine the importance of the recommendation tags, and derive a platform distribution strategy based on the importance.

2. According to claim 1, a social platform distribution strategy detection method based on an improved Apriori algorithm is characterized in that: The S1 includes: S1.1: Use web crawlers to write code for pages on multiple platforms, encapsulate the crawler codes of multiple platforms into corresponding modules, and build a crawler platform using the front-end and back-end; S1.2: Use a crawler platform to crawl data from each platform, where the platform data includes user data, user behavior data, and platform push data.

3. According to claim 2, a social platform distribution strategy detection method based on an improved Apriori algorithm is characterized in that: The S2 includes: The platform data is cleaned to remove illegal characters and empty characters, and the cleaned platform data is stored in the database.

4. According to claim 1, a social platform distribution strategy detection method based on an improved Apriori algorithm is characterized in that: The S3 includes: The database is connected through assembly language, and the data in the database is input into the RoBERTa model and embedded to obtain a user vector. The RoBERTa model introduces layer normalization after self-attention output, and each sample is independently standardized at each position. The RoBERTa model uses a multi-layer Transformer Encoder to obtain the final text representation to obtain a user vector.

5. According to the social platform distribution strategy detection method based on the improved Apriori algorithm of claim 1, it is characterized in that: The feature extraction model includes forward propagation, backward propagation, a bidirectional long short-term memory network Bi-LSTM module and an Attention mechanism.

6. According to the social platform distribution strategy detection method based on the improved Apriori algorithm of claim 1, it is characterized in that: The S5: S5.1: discretize the recommendation features and map them into recommendation labels; S5.2: The recommended tag data of each user obtained in S5.1 is stored in the database, and the corresponding bit vector conversion is performed. The conversion rule is: if the corresponding recommended tag exists, the corresponding value is set to 1, otherwise it is set to 0; S5.3: By scanning the database once, construct a bit vector representation of all user recommendation label data to capture the distribution of user recommendation label data in the entire database; S5.4: Use the inner product operation of the bit vector to obtain the frequency of each recommended tag in the database, and use the frequency as the local support; S5.5: Merge item sets with the same prefix by performing an AND operation on the bit vectors; S5.6: Eliminate infrequent item sets and generate candidate item sets; S5.7: Divide the data into multiple blocks according to the user recommendation tag data in the database, and distribute them to multiple processors through hash partitioning. Each processor independently maps the assigned user recommendation tag data to a bit vector and performs bit operations. The local support calculated by all processors is merged to obtain the global support. S5.8: The global support is used as the importance of the recommended tag, and the platform distribution strategy is derived according to the importance.

7. A social platform distribution strategy detection system based on an improved Apriori algorithm using the social platform distribution strategy detection method based on an improved Apriori algorithm as described in any one of claims 1 to 6, characterized in that: include: Data collection module, integrating crawler platform to obtain data from various platforms; A database construction module pre-processes the platform data and saves it to the database; A vectorization module imports the database data into the model and embeds it to obtain a user vector; A recommendation feature extraction module inputs the user vector into a feature extraction model to extract features and obtain recommendation features; The intelligent recommendation strategy generation module discretizes the recommendation features, maps them into recommendation tags, and uses the improved Apriori algorithm to mine the importance of the recommendation tags, and derives the platform distribution strategy based on the importance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the social platform distribution strategy detection method based on the improved Apriori algorithm as described in any one of claims 1 to 6 is implemented.

Images