Big data-based e-commerce live broadcast data flow management optimization method and system

Abstract

The application discloses an e-commerce live broadcast data flow guide management optimization method and system based on big data, relates to the technical field of live broadcast data management, and comprises the following steps: obtaining a user click behavior sequence and performing trajectory matching on the sequence by adopting a DTW algorithm to obtain a first abnormal traffic set; constructing a user feature matrix based on the first abnormal traffic set, and performing screening according to a clustering algorithm to obtain a second abnormal traffic set; constructing a graph model based on the second abnormal traffic set, and analyzing user IP region jumping by adopting a Louvain algorithm to obtain a marked list of a volume brushing community; constructing a user conversion path set, and calculating a first probability value of each path based on the marked list of the volume brushing community; screening advertisement delivery traffic according to the first probability value to obtain an optimized flow guide decision instruction; and the application solves the problem of false traffic interference in existing live broadcast flow guide management, and can optimize advertisement delivery effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of live data management optimization, more specifically, the present application relates to an e-commerce live data flow management optimization method and system based on big data. BACKGROUND

[0002] In recent years, with the popularization of mobile Internet, the rise of social e-commerce and the digital transformation of consumer shopping behavior, e-commerce live has become the core channel for brand and consumer interaction. E-commerce live data flow refers to the process of analyzing user behavior data generated during live streaming, such as viewing time, interaction frequency, and click conversion, and combining algorithm marketing strategies to guide potential consumers to the live streaming room or product page, thereby improving the traffic scale and conversion efficiency of the live streaming room and achieving sales growth.

[0003] For example, the data analysis method and system for e-commerce live shopping disclosed in the invention patent with publication number CN117714722A receives target anchor verification requests and initial bonuses sent by willing sellers, compares them with pre-stored blacklists, conducts potential risk assessment on target anchors according to live streaming record data, establishes a relationship graph for target anchors with potential risk labels, judges whether there is an anomaly according to the relationship graph, establishes a live streaming traffic authenticity monitoring model, and establishes first and second judgment criteria for live monitoring of potential risk label and dangerous label anchors. If fake traffic is monitored, the target anchor is added to the blacklist, and a warning report is sent to the willing seller, efficiently and accurately identifying and responding to fake traffic, and improving the credibility and user satisfaction of the live shopping platform.

[0004] For example, the live e-commerce content intelligent management platform based on big data disclosed in the invention patent with publication number CN112019871B includes a live content segmentation and classification module, a video frame position matching module, a play rule database, a play rule selection module, a video screening and play module, and a play rule intelligent recommendation module. The present application divides the complete live playback video of the product according to the content and labels the keywords, forms a product feature keyword video list, and through the video frame position matching module, the video frame position matching to the keyword is obtained from the complete live playback video of the product according to the artificial input product keyword, and a certain play rule is manually selected for playing, realizing intelligent management of the live video content of the product, having the characteristics of strong operability, making up for the poor operability, low efficiency and low matching degree caused by manual adjustment of the video progress bar, improving the adjustment efficiency and enhancing the viewing experience of consumers watching live streaming.

[0005] The above-mentioned technical solutions have at least the following technical problems:

[0006] E-commerce live broadcast has become a core means of commodity promotion, but there are false flows such as robot brushing and IP address forgery in the live broadcast room, which cannot accurately distinguish between real users and brushing users, resulting in reduced advertising efficiency. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art, embodiments of the present application provide an e-commerce live broadcast data flow management optimization method and system based on big data, which solves the problem of false flow interference in existing live broadcast flow management by fusing user behavior feature extraction, abnormal flow detection, social group brushing analysis and path probability correction technology.

[0008] To achieve the above object, the present application provides the following technical scheme:

[0009] The e-commerce live broadcast data flow management optimization method based on big data comprises the following steps: obtaining a user click behavior sequence and performing trajectory matching on the sequence using a DTW algorithm to obtain a first abnormal flow set; constructing a user feature matrix based on the first abnormal flow set and filtering according to a clustering algorithm to obtain a second abnormal flow set; constructing a graph model based on the second abnormal flow set and analyzing user IP regional jumps using a Louvain algorithm to obtain a brushing social group marker list; constructing a user conversion path set and calculating a first probability value of each path based on the brushing social group marker list; and correcting the advertising flow according to the first path probability to obtain an optimized flow decision instruction.

[0010] In a preferred embodiment, the user click behavior sequence is obtained and trajectory matching is performed on the sequence using a DTW algorithm to obtain a first abnormal flow set, specifically: the user click behavior sequence is obtained, and a trajectory time series graph is constructed; a user behavior intensity feature sequence is obtained by performing feature extraction on the trajectory time series graph based on a preset neural network model; the user behavior intensity feature sequence is matched with a preset reference trajectory using a DTW algorithm; the matching result is analyzed to identify abnormal sequences, and a first abnormal flow set is constructed.

[0011] In a preferred embodiment, the user feature matrix is constructed based on the first abnormal flow set, and the second abnormal flow set is obtained by filtering according to a clustering algorithm, specifically: the user feature matrix is constructed based on the first abnormal flow set, and the user feature matrix is subjected to density clustering based on a preset density threshold using a clustering algorithm; the clustering clusters with a spatial distance less than a preset first threshold are merged, and the clustering clusters containing a number of users greater than a preset second threshold are marked as second abnormal flows to obtain a second abnormal flow set.

[0012] In a preferred implementation, the second abnormal traffic set is used to build a graph model, and the Louvain algorithm is used to analyze the user IP regional jump to obtain a list of fake traffic community markers. Specifically, each user in the second abnormal traffic set is taken as a node to build a graph model; the Louvain algorithm is used to divide the graph model into communities, and the communities are abnormally identified to obtain a list of fake traffic community markers.

[0013] In a preferred implementation, the Louvain algorithm is used to divide the graph model into communities, and the communities are abnormally identified to obtain a list of fake traffic community markers. Specifically, the Louvain algorithm is used to initially divide the graph model into communities to obtain a plurality of communities; the geographical distances between all IPs in the communities are obtained, and the maximum geographical distance and the average geographical distance in the community are calculated, and the ratio of the maximum geographical distance to the average geographical distance is taken as a regional jump index; the modularity of each community is calculated, and the modularity of each community is corrected according to the regional jump index to obtain a corrected modularity; the community merging strategy is adjusted according to the corrected modularity to obtain a plurality of adjusted communities; the adjusted communities are abnormally identified to obtain a list of fake traffic community markers.

[0014] In a preferred implementation, the user conversion path set is constructed, and the first probability value of each path is calculated based on the list of fake traffic community markers. Specifically, the behavior timestamps of the user in the live room are obtained, and the behavior chain sequence is constructed according to a preset time sequence; the key nodes in the behavior chain sequence are extracted to obtain a user conversion path set; the user conversion path set is divided based on the list of fake traffic community markers to obtain a real path subset and a fake traffic path subset; the real path subset and the fake traffic path subset are calculated using a preset kernel density calculation formula to obtain real path feature values and fake traffic path feature values; the ratio of the real path feature values to the fake traffic path feature values is calculated, and the first probability value of each path is calculated according to the Bayes formula.

[0015] In a preferred implementation, the first probability value is used to filter the advertising traffic to obtain an optimized traffic decision instruction. Specifically, the advertising channel data is obtained, and the first traffic quality evaluation value of all paths under each advertising channel is calculated according to the first path probability; the advertising traffic is filtered according to the first traffic quality evaluation value to obtain a filtered advertising traffic pool; the traffic decision is optimized according to the filtered advertising traffic pool to obtain an optimized traffic decision instruction.

[0016] Also provided is an e-commerce live broadcast data flow management optimization system based on big data, comprising a data acquisition module, an abnormality screening module, a community identification module, a probability calculation module, and a traffic screening module: the data acquisition module is used to acquire a user click behavior sequence and perform trajectory matching on the sequence using a DTW algorithm to obtain a first abnormal traffic set; the abnormality screening module is used to construct a user feature matrix based on the first abnormal traffic set and screen according to a clustering algorithm to obtain a second abnormal traffic set; the community identification module is used to construct a graph model based on the second abnormal traffic set and analyze user IP regional jumps using a Louvain algorithm to obtain a fake traffic community marker list; the probability calculation module is used to construct a user conversion path set and calculate a first probability value of each path based on the fake traffic community marker list; and the traffic screening module is used to screen advertising traffic according to the first probability value to obtain an optimized flow decision instruction. The technical effects and advantages of the e-commerce live broadcast data flow management optimization method and system based on big data of the present application are as follows:

[0017] 1. The present application precisely identifies and eliminates abnormal traffic by collecting user click behavior sequences and performing trajectory matching using a dynamic time warping (DTW) algorithm; secondly, the quality of traffic is further optimized by constructing a user feature matrix and using a clustering algorithm for screening. By performing density clustering on abnormal traffic, real users and fake traffic users can be effectively distinguished, helping advertisers better understand the characteristics of user groups and thus improving the accuracy of target audiences. In addition, the Louvain algorithm is used for community identification to analyze user IP regional jumps, further accurately marking fake traffic communities, and effectively avoiding the impact of fake traffic behavior on advertising effectiveness.

[0018] 2. The present application constructs a user conversion path set and calculates a first probability value of each path based on the fake traffic community marker list, and uses the Bayes formula to accurately predict the path. This deep analysis not only helps to identify high-quality conversion paths, but also effectively evaluates the traffic quality of each path to optimize advertising effectiveness; in addition, the advertising traffic is screened according to the first path probability, and the optimized flow decision instruction can accurately match efficient traffic, thereby improving the ROI of advertising. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The flowchart of the e-commerce live broadcast data flow management optimization method based on big data of the present application.

[0020] Figure 2 The structural diagram of the e-commerce live broadcast data flow management optimization system based on big data of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0022] It should be noted that the acquisition and processing of all information or data in the present application are performed under the premise of complying with the corresponding data protection regulations and policies of the country and on the premise of being authorized by the authorized manager.

[0023] Embodiment 1, Figure 1 The e-commerce live broadcast data flow management optimization method based on big data of the present application is given, including the following steps:

[0024] S1, obtaining a user click behavior sequence and performing trajectory matching on the sequence by using a DTW algorithm to obtain a first abnormal traffic set;

[0025] In this example, a user click behavior sequence is obtained, and trajectory matching is performed on the sequence by using a DTW algorithm to obtain a first abnormal traffic set, specifically as follows:

[0026] Obtaining a user click behavior sequence and constructing a trajectory time sequence diagram:

[0027] Performing feature extraction on the trajectory time sequence diagram based on a preset neural network model to obtain a user behavior intensity feature sequence;

[0028] Matching the user behavior intensity feature sequence and a preset reference trajectory by using a DTW algorithm;

[0029] Performing data analysis on the matching result, identifying an abnormal sequence, and constructing a first abnormal traffic set.

[0030] It should be noted that the click, stay, and jump behavior sequence of a user in a live broadcast room is collected, and a trajectory time sequence diagram is generated according to a timestamp, for example, the behavior sequence of a certain user in a live broadcast room is [entering a live broadcast room (10:00) → clicking a product link (10:02) → collecting a product (10:05) → leaving the live broadcast room (10:08)]; the input layer of the preset neural network model is the trajectory time sequence diagram, the hidden layer includes a time sequence convolution layer, an attention mechanism layer, and a full connection layer, wherein the time sequence convolution layer extracts a click frequency feature; the attention mechanism layer is used to extract high-frequency operation nodes, such as a flash sale button and repeated clicks; and the full connection layer outputs a user behavior intensity feature sequence.

[0031] The specific calculation formula for matching the user behavior intensity feature sequence and the preset reference trajectory by using the DTW algorithm is as follows:

[0032]

[0033] where D is the matching distance, π is the regular path, S i is the user behavior intensity feature sequence, R j is the preset reference trajectory.

[0034] Further, first by constructing the user click behavior sequence and generating the trajectory timing diagram, detailed and orderly user behavior data is provided for subsequent analysis. This step ensures comprehensive capture of user behavior and lays the foundation for in-depth analysis of data.

[0035] Then, based on the preset neural network model, the trajectory timing diagram is feature extracted, further enhancing the recognition ability of user behavior patterns. By extracting the user behavior intensity feature sequence, the intensity and behavior pattern of user interaction during live streaming can be accurately captured, and the real interest points and activity level can be better reflected. Compared with traditional single behavior analysis method, this feature extraction method can effectively improve the recognition ability of complex user behavior and reduce misjudgment.

[0036] One of the core advantages of this method is to use the DTW algorithm to match the user behavior intensity feature sequence with the preset reference trajectory. The DTW algorithm can handle the nonlinear alignment problem in time series, so as to realize accurate matching and comparison of user behavior, and can identify abnormal sequences with large differences from the standard behavior pattern. In this way, real users and fake users can be effectively distinguished, and abnormal traffic that does not conform to the expected behavior pattern can be excluded, ensuring the authenticity and effectiveness of traffic data.

[0037] Finally, through in-depth data analysis of the matching results, abnormal behavior sequences can be identified, and the first abnormal traffic set can be constructed, providing reliable data support for subsequent traffic screening and conversion path optimization, achieving accurate identification of abnormal traffic, minimizing the interference of invalid traffic, and providing a solid data basis for advertising decision-making, ensuring the effectiveness of advertising and the improvement of ROI.

[0038] S2, based on the first abnormal traffic set, a user feature matrix is constructed, and a clustering algorithm is used for screening to obtain a second abnormal traffic set;

[0039] In this example, based on the first abnormal traffic set, a user feature matrix is constructed, and a clustering algorithm is used for screening to obtain a second abnormal traffic set, specifically:

[0040] Based on the first abnormal traffic set, a user feature matrix is constructed, and a clustering algorithm is used to perform density clustering on the user feature matrix based on a preset density threshold;

[0041] merge the cluster clusters with a spatial distance less than a preset first threshold value, and mark the cluster cluster containing a number of users greater than a preset second threshold value as a second abnormal traffic, to obtain a second abnormal traffic set.

[0042] It should be noted that the user feature matrix includes behavior density, operation type distribution, and device fingerprint, wherein the behavior density is the number of user operations per unit time; the operation type distribution is the click / favorite / purchase ratio; and the device fingerprint is a multi-account login marker of the same device.

[0043] When filtering according to the clustering algorithm, the parameters are set as follows: the neighborhood radius is 0.5, the minimum sample number is 5, and if the cluster center distance is less than 0.3, they are merged into the same cluster; and if the number of users in the cluster is greater than 100, it is determined as a large-scale brushing cluster, and a second abnormal traffic set is generated.

[0044] It should be noted that the user feature matrix is constructed based on the first abnormal traffic set, which integrates the behavior characteristics of different users and provides a comprehensive user feature view for subsequent clustering analysis; the user feature matrix is subjected to density clustering by using a clustering algorithm, and the users are grouped by a preset density threshold. This density clustering method can effectively identify user groups with similar behavior patterns and distinguish them from other groups. Compared with traditional classification methods, density clustering can better handle irregular data distribution and avoid misclassification problems caused by improper threshold setting or uneven data distribution; in addition, by merging the cluster clusters with a spatial distance less than a preset first threshold value, the division of user groups can be further refined to ensure that users with similar behavior patterns and frequent interactions are correctly classified. This strategy helps to avoid confusing normal users with brushing users and ensures the accuracy of traffic analysis. At the same time, the cluster cluster containing a number of users greater than a preset second threshold value is marked as a second abnormal traffic, which can effectively identify brushing behavior. These cluster clusters usually have obvious feature differences, such as user concentration and abnormal behavior, and through this screening method, invalid traffic can be further eliminated to provide more accurate traffic data for advertising.

[0045] S3, based on the second abnormal traffic set, a graph model is constructed, and Louvain algorithm is used to analyze user ip regional jumps to obtain a brushing community label list;

[0046] In this example, based on the second abnormal traffic set, a graph model is constructed, and Louvain algorithm is used to analyze user ip regional jumps to obtain a brushing community label list, specifically as follows:

[0047] A graph model is constructed with each user in the second abnormal traffic set as a node;

[0048] The Louvain algorithm is used to divide the graph model into communities and identify anomalies in the communities to obtain a list of communities marked for fraudulent traffic.

[0049] Specifically, the second abnormal traffic is used to construct a graph model by treating each user as a node, and the edges of the graph model are constructed based on the relationships between users, such as shared IP, device, and payment account.

[0050] In this example, the Louvain algorithm is used to segment the graph model into communities and identify anomalies within these communities, resulting in a list of communities flagged for fraudulent activity. Specifically:

[0051] The Louvain algorithm is used to perform initial community partitioning on the graph model, resulting in several communities;

[0052] Obtain the geographical distance between all IPs in the community, calculate the maximum and average geographical distance within the community, and use the ratio of the maximum and average geographical distance as the geographical jump index.

[0053] Calculate the modularity of each community and adjust the modularity of each community according to the regional jump index to obtain the adjusted modularity.

[0054] Based on the revised modularity, the community merging strategy was adjusted to obtain several adjusted communities;

[0055] Anomaly detection is performed on the adjusted communities to obtain a list of communities flagged for fraudulent traffic.

[0056] The modularity of each community is adjusted based on the geographical jump index to obtain the adjusted modularity. The specific calculation formula is as follows:

[0057] Q new =Q-λ·T

[0058] Among them, Q new Q represents the modified modularity, λ represents the modularity of each community, T represents the regional jump index, and λ represents the preset weight coefficient.

[0059] It should be noted that first, each user in the second abnormal traffic set is taken as a node in the graph model, and the connection between users can be constructed according to their behavior patterns or other related characteristics. In this way, the relationship and interaction between users can be systematically captured, thereby providing strong data support for community division and anomaly identification; in addition, the Louvain algorithm is an efficient community detection algorithm that can automatically divide communities according to the connection strength of nodes in the graph, ensuring detailed division of user groups. Through this community division, potential brushing communities can be identified, that is, user groups with abnormal behavior patterns and frequent regional jumps. This method has stronger adaptability and flexibility than traditional rule screening methods and can cope with complex and variable user behavior.

[0060] Further, by calculating the geographic distance between IPs within the community, the maximum geographic distance and the average geographic distance are obtained, and the ratio of the two is taken as the regional jump index, which effectively identifies unnatural patterns in user behavior. The regional jump index can accurately reflect the IP jump phenomenon in the brushing behavior, helping to discover malicious users who try to evade monitoring by changing geographic locations. By modifying the modularity and adjusting the community merging strategy according to the regional jump index, the community division result can be optimized, further improving the identification accuracy of brushing communities.

[0061] S4, constructing a user conversion path set and calculating a first probability value of each path based on the brushing community marking list;

[0062] In this example, a user conversion path set is constructed, and a first probability value of each path is calculated based on the brushing community marking list, specifically:

[0063] Obtain the behavior timestamp of the user in the live room and construct a behavior chain sequence according to the preset time sequence;

[0064] Extract the key nodes in the behavior chain sequence to obtain a user conversion path set;

[0065] Divide the user conversion path set based on the brushing community marking list to obtain a real path subset and a brushing path subset;

[0066] Calculate the feature values of the real path subset and the brushing path subset using a preset kernel density calculation formula to obtain real path feature values and brushing path feature values;

[0067] Calculate the ratio of the real path feature values and the brushing path feature values, and calculate the first probability value of each path according to the Bayes formula.

[0068] The preset kernel density calculation formula is specifically:

[0069]

[0070] wherein f s (x) is the eigenvalue of the real path, n is the number of real paths in the real path subset, h is a preset bandwidth parameter, K is a kernel function, x i is the eigenvalue of the i-th real path in the real path subset, x is the feature dimension.

[0071] It should be noted that constructing the user conversion path set and calculating the first probability value of each path based on the brushing amount community marking list can achieve precise traffic screening and optimize advertisement placement in e-commerce live streaming. This method can effectively identify the conversion paths of real users and brushing amount users through detailed behavior chain sequence analysis, combined with the brushing amount community marking list and the Bayesian algorithm, thereby improving the authenticity, effectiveness and placement accuracy of traffic.

[0072] Firstly, the behavior time stamps of users in the live streaming room are obtained, and a behavior chain sequence is constructed according to a preset time sequence. This step can accurately record all interactive behaviors of each user during the live streaming process. By constructing the behavior chain in the order of time stamps, the behavior trajectory of the user can be fully reflected, and each important node of the user in the live streaming, such as clicking, watching time, interaction, etc., can be captured, thereby providing rich data support for the construction of conversion paths.

[0073] Then, by extracting the key nodes in the behavior chain sequence, the user's behavior can be effectively converted into a conversion path set. The extraction of key nodes can help accurately identify the critical behaviors in the user's conversion process, thereby constructing representative conversion paths. These conversion paths not only help to identify the whole process from watching to purchasing, but also distinguish different types of user groups, especially in e-commerce live streaming, the accurate identification of conversion paths is crucial for advertisement placement.

[0074] On this basis, the user conversion paths are divided into real path subset and brushing amount path subset in combination with the brushing amount community marking list. By dividing the conversion paths, brushing amount traffic can be effectively filtered out.

[0075] Then, a preset kernel density calculation formula is used to calculate the features of the real path subset and the brushing amount path subset. This method can further improve the accuracy of traffic analysis. The kernel density estimation method can estimate the probability density according to the behavior characteristics of the user, and calculate the eigenvalue of each path. The eigenvalues of real paths and brushing amount paths can reflect the differences between different paths, thereby better identifying the boundaries between normal paths and brushing amount paths.

[0076] Finally, by calculating the ratio of the real path eigenvalue and the brushing path eigenvalue, and calculating the first probability value of each path according to the Bayes formula, an accurate conversion probability value can be assigned to each user path according to the characteristics of the traffic. This step gives the authenticity probability of each path under the current data environment through the Bayes algorithm combined with prior knowledge and path eigenvalue. In this way, the real effective user path and the brushing path can be effectively distinguished, and the advertising placement decision is optimized.

[0077] S5, according to the first probability value, the advertising placement traffic is corrected to obtain an optimized flow decision instruction.

[0078] In this example, the advertising placement traffic is screened according to the first path probability to obtain an optimized flow decision instruction, specifically:

[0079] Obtain advertising placement channel data, and calculate the first traffic quality evaluation value of all paths under each advertising placement channel according to the first path probability;

[0080] Screen the placement traffic according to the first traffic quality evaluation value to obtain a screened advertising traffic pool;

[0081] Optimize the flow decision according to the screened advertising traffic pool to obtain an optimized flow decision instruction.

[0082] It should be noted that the advertising placement channel data is pulled in real time from the advertising platform (such as JUHAO engine, Tencent advertising) API, the channel name (such as Douyin live, fast shop, Taobao direct car); the user conversion path set under each channel (such as "home page→product detail page→payment completion"); the exposure, click, conversion rate, average stay time and other indicators corresponding to the path.

[0083] The first traffic quality evaluation value formula is as follows:

[0084]

[0085] Wherein, r c is the first traffic quality evaluation value of channel c, t is the number of paths of channel c, p i is the first probability value of the i-th path, l i is the preset historical conversion amount of the i-th path under channel c, l c is the total conversion amount of all paths under channel c.

[0086] It should be noted that by acquiring the advertising channel data and calculating the traffic quality evaluation value according to the first path probability of each path, the effectiveness of each advertising path can be quantified. This step can help advertisers filter out traffic paths with higher conversion potential from the vast amount of traffic. Since the probability value of each path reflects the authenticity and path conversion effect of the path, advertisers can accurately determine which paths can bring better investment returns based on the probability value; secondly, by calculating and analyzing the first traffic quality evaluation value, the low-quality traffic can be effectively filtered out. This screening process ensures that each path in the advertising traffic pool has high conversion ability, effectively avoiding unnecessary waste of advertising costs. Through this precise screening, advertising is not only more accurate, but also improves the exposure effect of the advertisement, so that the advertising budget can be more efficiently invested in potential high-quality user groups; based on the screened advertising traffic pool, the traffic decision is optimized to ensure that the advertising decision is more effective.

[0087] Embodiment 2, Figure 2 The e-commerce live broadcast data traffic management optimization system based on big data is given, which comprises a data acquisition module, an abnormality screening module, a community identification module, a probability calculation module and a traffic screening module:

[0088] The data acquisition module is used for acquiring user click behavior sequences and performing trajectory matching on the sequences by using a DTW algorithm to obtain a first abnormal traffic set;

[0089] The abnormality screening module is used for constructing a user feature matrix based on the first abnormal traffic set, and screening according to a clustering algorithm to obtain a second abnormal traffic set;

[0090] The community identification module is used for constructing a graph model based on the second abnormal traffic set, and analyzing user IP region jumps by using a Louvain algorithm to obtain a list of brushing volume community labels;

[0091] The probability calculation module is used for constructing a user conversion path set, and calculating a first probability value of each path based on the list of brushing volume community labels;

[0092] The traffic screening module is used for screening advertising traffic according to the first probability value to obtain an optimized traffic decision instruction.

[0093] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized in the form of a computer program product, wholly or partially.

[0094] Those skilled in the art can understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0095] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0096] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0097] Finally: the above is only the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for managing and optimizing data diversion of e-commerce live broadcast data based on big data, characterized in that, The method comprises the following steps: obtaining a user click behavior sequence and performing trajectory matching on the sequence by using a DTW algorithm to obtain a first abnormal traffic set; constructing a user feature matrix based on the first abnormal traffic set and screening according to a clustering algorithm to obtain a second abnormal traffic set; constructing a graph model based on the second abnormal traffic set and analyzing user IP regional jumps by using a Louvain algorithm to obtain a list of brushing volume community markers, specifically: constructing a graph model with each user in the second abnormal traffic set as a node; performing community division on the graph model by using the Louvain algorithm and performing abnormal identification on the community to obtain the list of brushing volume community markers, specifically: performing initial community division on the graph model by using the Louvain algorithm to obtain a plurality of communities; obtaining the geographical distance between all IPs in the community and calculating the maximum geographical distance and the average geographical distance in the community, and taking the ratio of the maximum geographical distance and the average geographical distance as a regional jump index; calculating the modularity of each community and correcting the modularity of each community according to the regional jump index to obtain a corrected modularity; adjusting the community merging strategy according to the corrected modularity to obtain a plurality of adjusted communities; performing abnormal identification on the adjusted community to obtain the list of brushing volume community markers; constructing a user conversion path set and calculating a first probability value of each path based on the list of brushing volume community markers, specifically: obtaining the behavior timestamp of the user in the live room and constructing a behavior chain sequence according to a preset time sequence; extracting key nodes in the behavior chain sequence to obtain a user conversion path set; dividing the user conversion path set based on the list of brushing volume community markers to obtain a real path subset and a brushing volume path subset; calculating the feature values of the real path subset and the brushing volume path subset by using a preset kernel density calculation formula to obtain real path feature values and brushing volume path feature values; calculating the ratio of the real path feature values and the brushing volume path feature values, and calculating the first probability value of each path according to the Bayes formula; screening the advertising traffic according to the first probability value to obtain an optimized traffic diversion decision instruction.

2. The big data-based e-commerce live broadcast data diversion management optimization method of claim 1, wherein, The method comprises the following steps: obtaining a user click behavior sequence and performing trajectory matching on the sequence by using a DTW algorithm to obtain a first abnormal traffic set, specifically: obtaining a user click behavior sequence and constructing a trajectory time sequence graph; extracting features from the trajectory time sequence graph based on a preset neural network model to obtain a user behavior density feature sequence; matching the user behavior density feature sequence with a preset reference trajectory by using the DTW algorithm; 3. The big data-based e-commerce live broadcast data diversion management optimization method of claim 2, wherein, performing data analysis on the matching result, identifying abnormal sequences, and constructing a first abnormal traffic set. The method comprises the following steps: constructing a user feature matrix based on the first abnormal traffic set and performing density clustering on the user feature matrix based on a preset density threshold by using a clustering algorithm; merging clustering clusters with a spatial distance less than a preset first threshold, and marking clustering clusters containing a number of users greater than a preset second threshold as second abnormal traffic to obtain a second abnormal traffic set.

4. The big data-based e-commerce live broadcast data diversion management optimization method of claim 3, wherein, The advertisement delivery traffic is screened according to the first probability value, and an optimized flow decision instruction is obtained, specifically as follows: Obtain advertisement delivery channel data, and calculate a first traffic quality evaluation value of all paths under each advertisement delivery channel according to a first path probability; Screen the delivery traffic according to the first traffic quality evaluation value, and obtain a screened advertisement traffic pool; Optimize the flow decision according to the screened advertisement traffic pool, and obtain an optimized flow decision instruction.

5. The big data-based e-commerce live broadcast data diversion management optimization method of claim 4, wherein, The preset kernel density calculation formula is specifically as follows: wherein, is a true path eigenvalue, is a number of true paths in a true path subset, is a preset bandwidth parameter, is a kernel function, is an eigenvalue of an i-th true path in a true path subset, is a feature dimension.

6. The e-commerce live broadcast data flow management optimization system based on big data, applied to the e-commerce live broadcast data flow management optimization method based on big data in any one of claims 1-5, characterized in that, The method comprises a data acquisition module, an abnormality screening module, a community identification module, a probability calculation module, and a traffic screening module: The data acquisition module is used for acquiring a user click behavior sequence and performing trajectory matching on the sequence by using a DTW algorithm to obtain a first abnormal traffic set; The abnormality screening module is used for constructing a user feature matrix based on the first abnormal traffic set, and screening according to a clustering algorithm to obtain a second abnormal traffic set; The community identification module is used for constructing a graph model based on the second abnormal traffic set, and analyzing user ip regional jumps by using a Louvain algorithm to obtain a brushing volume community marker list; The probability calculation module is used for constructing a user conversion path set, and calculating a first probability value of each path based on the brushing volume community marker list; The traffic screening module is used for screening the advertisement delivery traffic according to the first probability value, and obtaining an optimized flow decision instruction.

Images