Live broadcast stream pushing service analysis method and device based on large model
By combining the BERT model and GraphSAGE graph neural network with the TFT time series model, the problem of inaccurate identification of live streaming traffic is solved, enabling accurate division of CDN node affiliation and dynamic adaptation of traffic flow, providing future traffic trend prediction, and supporting network resource optimization and operational decision-making.
Patent Information
- Application Number
- CN202511876285.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies cannot adapt to newly launched live streaming platforms or identify domain name changes in a timely manner, and the accuracy of CDN node attribution judgment is insufficient, resulting in inaccurate identification of live streaming traffic and poor performance of traditional association methods.
The BERT model is used for dynamic identification of live streaming domain names. A CDN node distribution table is constructed by combining the GraphSAGE graph neural network. The TFT time series model is used to predict traffic flow. By parsing AAA and DNS logs, an address feature library is constructed to accurately identify CDN node affiliation and traffic flow.
It achieves accurate identification and dynamic adaptation of live streaming traffic, solving the problems of identification omissions and inaccurate attribution judgments in traditional methods, and provides future traffic trend and peak prediction, providing data support for network resource allocation and operation decisions.
Smart Images
Figure CN121864608A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of traffic flow analysis, and in particular to a method and apparatus for analyzing live streaming services based on a large model. Background Technology
[0002] With the rapid development of short video and live streaming services, a large number of individual broadcasters are pushing audio and video streams to CDN nodes through home broadband or enterprise dedicated lines, generating continuous high uplink traffic. This type of traffic is characterized by its sudden bursts, frequent cross-regional scheduling, and dynamic domain name changes. Operators urgently need to accurately grasp the core business data of live streaming to support operational decisions.
[0003] However, existing analysis methods rely on manually maintained domain keyword databases (such as those containing "push" and "send"), which cannot adapt to newly launched live streaming platforms or domain name changes of existing platforms in a timely manner, resulting in omissions in live streaming push traffic identification. CDN node identification relies solely on manually classifying CDN nodes by IP address range, without considering the network topology and application association characteristics of CDN nodes, leading to insufficient accuracy in attribution judgment and inaccurate identification of live streaming push behavior. Furthermore, DNS, AAA, and Flow data have heterogeneous formats and poor time synchronization, making it difficult for traditional rule-based association methods to establish a complete "account-client address-CDN node" link, resulting in poor association performance. Summary of the Invention
[0004] To address the aforementioned problems with existing analytical methods, this invention provides a live streaming service analysis method and apparatus based on a large model. Relying on the BERT model and GraphSAGE graph neural network, it accurately identifies the CDN node affiliation and the traffic flow direction of live streaming. Simultaneously, it uses TFT time series prediction to learn the future traffic flow characteristics of live streaming, providing data support for operators to optimize network resource allocation, formulate pricing policies, and conduct business operation decisions.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] In one embodiment of the present invention, a live streaming service analysis method based on a large model is proposed, the method comprising:
[0007] Dynamically identify the characteristics of live streaming domain names based on the BERT model;
[0008] Build an address signature database and parse AAA and DNS logs;
[0009] Construct a CDN node distribution table based on the GraphSAGE algorithm;
[0010] Use Flow traffic to analyze user push traffic and push traffic received by CDN nodes;
[0011] The flow of live streaming traffic is analyzed and predicted based on the TFT model.
[0012] Furthermore, based on the BERT model, the characteristics of the live streaming domain name are dynamically identified, including:
[0013] Using a pre-trained BERT model for scenario transfer learning of live streaming domain names, the semantic features of live streaming domain names, including root domain names and streaming keywords, are automatically mined.
[0014] Historical DNS logs are input into a pre-trained BERT model to filter out live streaming domains, extract core features including live streaming domain, application name, and root domain, and form an initial domain feature library.
[0015] New DNS logs are collected regularly, and live streaming domains that are not included in the initial domain feature library are automatically identified through a pre-trained BERT model. The initial domain feature library is then updated after verification.
[0016] Furthermore, an address signature database is constructed, and AAA logs and DNS logs are parsed, including:
[0017] Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database;
[0018] Extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic and downlink traffic, and build an online user data table based on the AAA logs and the initial AAA online users;
[0019] Extract the request address, request port, A record address, request domain name, and request time from the DNS log; match the request domain name from the DNS log against the initial domain name feature database and tag the matching request domain name with an application label; match the request address and A record address from the DNS log against the address feature database and tag the matching request address and A record address with the province, city, and network type.
[0020] Associate the request time, request address, and request port in the DNS log with the timestamp, NAT address, and port range in the AAA log, and tag the associated AAA log with the user.
[0021] Furthermore, a CDN node distribution table is constructed based on the GraphSAGE algorithm, including:
[0022] Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS logs according to application tag + home province + home city; use the A record addresses in the DNS logs as nodes in an undirected graph, and construct edges between nodes in the undirected graph based on the same application tag + home province + home city and its corresponding A record address, forming an undirected graph containing A record addresses and network topology;
[0023] Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of A record addresses containing network topology information.
[0024] Based on the feature vector of the A record address containing network topology information, the A record address is clustered into CDN nodes, and CDN node units are divided into three dimensions: application tag, home province, and home city, and the home province and home city of each CDN node unit are clearly defined.
[0025] A CDN node distribution table is constructed by associating CDN nodes with application tags, home province, home city, and network type.
[0026] Furthermore, using Flow traffic analysis, we can analyze user push traffic and push traffic received by CDN nodes, including:
[0027] Based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses;
[0028] Use the source IP address and source port in the Flow traffic to match the online user data table, and tag the matched source IP address and source port with user tags;
[0029] Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic;
[0030] If the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application;
[0031] If the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic analysis is performed according to the application tag on the destination IP address in the Flow traffic, and the Flow traffic is counted as the cross-network traffic of the CDN node.
[0032] If the DNS logs do not match the flow traffic, then based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
[0033] Furthermore, based on the TFT model, the traffic flow of live streaming is analyzed and predicted, including:
[0034] Collect historical push traffic and integrate characteristics that affect traffic, such as application type, holidays, and network load.
[0035] The flow prediction model is trained based on the TFT model, allowing the flow prediction model to learn the pattern of flow changes over time;
[0036] The trained traffic prediction model outputs the trend, peak period, and scale of future traffic flow.
[0037] In one embodiment of the present invention, a live streaming service analysis device based on a large model is also proposed, the device comprising:
[0038] The live streaming domain name recognition module is used to dynamically identify the characteristics of live streaming domain names based on the BERT model.
[0039] The log parsing module is used to build an address signature database and parse AAA logs and DNS logs;
[0040] The CDN node distribution table construction module is used to construct a CDN node distribution table based on the GraphSAGE algorithm.
[0041] The push stream traffic analysis module is used to analyze user push stream traffic and push stream traffic received by CDN nodes using Flow traffic.
[0042] The live streaming traffic prediction module is used to analyze and predict the traffic flow direction of live streaming based on the TFT model.
[0043] Furthermore, based on the BERT model, the characteristics of the live streaming domain name are dynamically identified, including:
[0044] Using a pre-trained BERT model for scenario transfer learning of live streaming domain names, the semantic features of live streaming domain names, including root domain names and streaming keywords, are automatically mined.
[0045] Historical DNS logs are input into a pre-trained BERT model to filter out live streaming domains, extract core features including live streaming domain, application name, and root domain, and form an initial domain feature library.
[0046] New DNS logs are collected regularly, and live streaming domains that are not included in the initial domain feature library are automatically identified through a pre-trained BERT model. The initial domain feature library is then updated after verification.
[0047] Furthermore, an address signature database is constructed, and AAA logs and DNS logs are parsed, including:
[0048] Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database;
[0049] Extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic and downlink traffic, and build an online user data table based on the AAA logs and the initial AAA online users;
[0050] Extract the request address, request port, A record address, request domain name, and request time from the DNS log; match the request domain name from the DNS log against the initial domain name feature database and tag the matching request domain name with an application label; match the request address and A record address from the DNS log against the address feature database and tag the matching request address and A record address with the province, city, and network type.
[0051] Associate the request time, request address, and request port in the DNS log with the timestamp, NAT address, and port range in the AAA log, and tag the associated AAA log with the user.
[0052] Furthermore, a CDN node distribution table is constructed based on the GraphSAGE algorithm, including:
[0053] Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS logs according to application tag + home province + home city; use the A record addresses in the DNS logs as nodes in an undirected graph, and construct edges between nodes in the undirected graph based on the same application tag + home province + home city and its corresponding A record address, forming an undirected graph containing A record addresses and network topology;
[0054] Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of A record addresses containing network topology information.
[0055] Based on the feature vector of the A record address containing network topology information, the A record address is clustered into CDN nodes, and CDN node units are divided into three dimensions: application tag, home province, and home city, and the home province and home city of each CDN node unit are clearly defined.
[0056] A CDN node distribution table is constructed by associating CDN nodes with application tags, home province, home city, and network type.
[0057] Furthermore, using Flow traffic analysis, we can analyze user push traffic and push traffic received by CDN nodes, including:
[0058] Based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses;
[0059] Use the source IP address and source port in the Flow traffic to match the online user data table, and tag the matched source IP address and source port with user tags;
[0060] Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic;
[0061] If the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application;
[0062] If the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic analysis is performed according to the application tag on the destination IP address in the Flow traffic, and the Flow traffic is counted as the cross-network traffic of the CDN node.
[0063] If the DNS logs do not match the flow traffic, then based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
[0064] Furthermore, based on the TFT model, the traffic flow of live streaming is analyzed and predicted, including:
[0065] Collect historical push traffic and integrate characteristics that affect traffic, such as application type, holidays, and network load.
[0066] The flow prediction model is trained based on the TFT model, allowing the flow prediction model to learn the pattern of flow changes over time;
[0067] The trained traffic prediction model outputs the trend, peak period, and scale of future traffic flow.
[0068] In one embodiment of the present invention, a computer device is also proposed, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned live streaming service analysis based on a large model.
[0069] In one embodiment of the present invention, a computer-readable storage medium is also proposed, which stores a computer program that performs live streaming service analysis based on a large model.
[0070] Beneficial effects:
[0071] 1. This invention is based on the BERT model to realize the automatic identification of live streaming domain names and dynamic updating of the domain name feature library, which solves the problem of lagging behind traditional manual maintenance, ensures accurate identification of streaming traffic, and can adapt to the dynamic changes of live streaming platform domain names.
[0072] 2. This invention innovatively introduces GraphSAGE graph neural network, which, combined with network topology features, enables accurate division of CDN node affiliation, overcoming the limitations of traditional IP segment division.
[0073] 3. This invention trains a traffic prediction model based on the TFT model, learns the pattern of traffic changes over time, and outputs future traffic trends and peak values, providing forward-looking support for network resource scheduling. Attached Figure Description
[0074] Figure 1 This is a schematic diagram of the live streaming business analysis method based on a large model according to the present invention;
[0075] Figure 2 This is a schematic diagram of the live streaming business analysis device based on a large model according to the present invention.
[0076] Figure 3 This is a schematic diagram of the computer device structure of the present invention. Detailed Implementation
[0077] The principles and spirit of the present invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.
[0078] Those skilled in the art will recognize that embodiments of the present invention can be implemented as an apparatus, device, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.
[0079] According to an embodiment of the present invention, a live streaming push service analysis method based on a large model is proposed. The method includes: dynamically identifying the characteristics of live streaming push domain names based on the BERT model; constructing an address feature library and parsing AAA logs and DNS logs; constructing a CDN node distribution table based on the GraphSAGE algorithm; using Flow traffic to analyze user push traffic and push traffic received by CDN nodes; and analyzing and predicting the traffic flow direction of live streaming push based on the TFT model.
[0080] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.
[0081] This invention utilizes a collaborative technology system of "BERT model + GraphSAGE graph neural network + TFT time series prediction" to achieve a systematic analysis of traffic flow in live streaming services.
[0082] The definitions are as follows:
[0083] A flow refers to a set of data packets that share the same key characteristics (such as the five-tuple: source / destination IP, source / destination port, and transport layer protocol) and are transmitted within a specific time window. It is the basic unit for traffic classification, statistics, and behavioral analysis in network traffic monitoring and management.
[0084] DNS: Domain Name System, is the core distributed directory service of the Internet. Through hierarchical resolution, it translates human-readable domain names into machine-routable IP addresses, forming the foundation for network access and service addressing.
[0085] AAA (Authorization, Authorization, and Accounting) is a framework for network security and operations management. It sequentially verifies entity identity, determines access permissions, and performs auditing and cost accounting based on resource usage.
[0086] BERT model: A bidirectional encoder representation model based on the Transformer architecture. Pre-trained through self-supervised tasks such as masked language modeling, it can generate deep, context-sensitive word vectors and is one of the cornerstone models in the field of natural language processing.
[0087] GraphSAGE: A graph sampling and aggregation algorithm, it is an inductive graph neural network. It generates embedding vectors for new, unseen nodes by sampling neighboring nodes and iteratively aggregating their features, making it suitable for large-scale dynamic graph learning.
[0088] TFT: Time Series Fusion Transformer, an advanced time series model designed for multi-step forecasting. It explicitly models the trends, seasonality, and feature dependencies of time series through gating mechanisms and variable selection networks.
[0089] Figure 1 This is a schematic diagram of the live streaming business analysis method based on a large model, as described in this invention. Figure 1 As shown, the implementation steps are as follows:
[0090] S1. Dynamically identify the characteristics of live streaming domain names based on the BERT model;
[0091] 1. Use a pre-trained BERT model for scenario transfer learning of live streaming domain names to automatically mine the semantic features of live streaming domain names, including root domain names and streaming keywords, so that it can automatically distinguish between live streaming domain names and non-live streaming domain names.
[0092] 2. Input historical DNS logs into the pre-trained BERT model to filter out live streaming domains, extract core features, including live streaming domain, application name and root domain, to form an initial domain feature library;
[0093] 3. Regularly collect new DNS logs and automatically identify live streaming domains that are not included in the initial domain feature library using a pre-trained BERT model. After verification, update the initial domain feature library to ensure that the initial domain feature library adapts to changes in the live streaming platform's domains.
[0094] S2. Construct an address signature database and parse AAA and DNS logs;
[0095] 1. Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database;
[0096] 2. Parse the AAA logs and build an online user data table;
[0097] Furthermore, extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic, and downlink traffic;
[0098] Furthermore, an online user data table is constructed based on the AAA logs and the initial AAA online users (the AAA logs contain user online status; after the initial collection of AAA logs, the online users in the AAA logs are obtained as the initial AAA online users);
[0099] 3. parse the DNS logs and, based on the initial domain name feature database, address feature database, and online user data table, add application tags, province of origin, city of origin, network type, and user tags;
[0100] Furthermore, extract the request address, request port, A record address, request domain name, and request time from the DNS log;
[0101] Furthermore, the requested domain name in the DNS log is used to match the initial domain name feature database, and the matching requested domain name is labeled with an application tag; the requested address and A record address in the DNS log are used to match the address feature database, and the matching requested address and A record address are labeled with the province, city, and network type.
[0102] Furthermore, the request time, request address, and request port in the DNS log are associated with the timestamp, NAT address, and port range in the AAA log, and user tags are added to the associated AAA logs.
[0103] S3. Construct a CDN node distribution table based on the GraphSAGE algorithm;
[0104] 1. Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS log according to application tag + home province + home city; use the A record addresses in the DNS log as nodes of the undirected graph, and construct edges between nodes of the undirected graph based on the same application tag + home province + home city and its corresponding A record address, to form an undirected graph containing A record addresses and network topology;
[0105] The formula for an undirected graph is as follows:
[0106] G=(V,E)
[0107] Where V is the set of addresses of record A, and E is the set of edges associated with the addresses of record A;
[0108] 2. Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of the A record address containing network topology information;
[0109] 3. Based on the feature vector of the A record address containing network topology information, cluster the A record addresses into CDN nodes, and divide them into CDN node units with three dimensions: application tag, home province, and home city, and clarify the home province and home city of each CDN node unit;
[0110] 4. Use CDN nodes to associate application tags, home province, home city and network type to build a CDN node distribution table.
[0111] S4. Use Flow traffic to analyze user push traffic and push traffic received by CDN nodes;
[0112] 1. Aggregate flow traffic according to preset time granularity, and associate it with CDN node distribution table and online user data table;
[0113] Furthermore, Flow traffic includes timestamp, source province, destination province, source network type, destination network type, source IP address, destination IP address, source port, destination port, inflow rate, outflow rate, IP address type (IPv4 / IPv6), and protocol type;
[0114] Furthermore, based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses;
[0115] Furthermore, the source IP address and source port in the Flow traffic are used to match the online user data table, and user tags are added to the matching source IP address and source port;
[0116] 2. Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic;
[0117] Furthermore, if the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application;
[0118] Furthermore, if the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic is analyzed according to the application tag on the destination IP address in the Flow traffic, and the Flow traffic is counted as the cross-network traffic of the CDN node.
[0119] Furthermore, if the DNS logs do not match the Flow traffic, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application, based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs.
[0120] S5. Analyze and predict the traffic flow direction of live streaming based on the TFT model;
[0121] 1. Collect historical push traffic data and integrate characteristics that affect traffic, such as application type, holidays, and network load;
[0122] 2. Train the flow prediction model based on the TFT model, so that the flow prediction model learns the pattern of flow change over time;
[0123] 3. The trained traffic prediction model outputs the trend, peak period and scale of future traffic flow.
[0124] It should be noted that although the operation of the method of the present invention has been described in a specific order in the above embodiments and figures, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.
[0125] To provide a clearer explanation of the network device configuration file verification method based on the Drools rule engine, several specific embodiments are described below. However, it is worth noting that these embodiments are only for better illustrating the present invention and do not constitute an improper limitation of the present invention.
[0126] Example:
[0127] The implementation steps are as follows:
[0128] S1. Dynamically identify the characteristics of live streaming domain names based on the BERT model;
[0129] 1. Use a pre-trained BERT model to perform transfer learning for live streaming domain scenarios;
[0130] 2. Construct a set of 100,000 labeled samples, including domain names of mainstream live streaming platforms such as Douyin (douyincdn.com) and Huya (huya.com) (10,000 positive samples; live streaming platform domain names generally contain keywords such as push / rtc / livepush / send / rtmp / rtsp, and the specific live streaming platform domain information can be determined by these keywords + the root domain) and non-live streaming domain names (90,000 negative samples). Each sample is labeled with "whether it is a live streaming push" and "corresponding platform".
[0131] 3. Freeze the parameters of the bottom 6 layers of the BERT model, fine-tune the upper layers, optimize the cross-entropy loss function, set the learning rate to 2e-5, train for 10 epochs, and finally achieve a precision ≥90%, accuracy ≥90%, and recall ≥90% on the test set.
[0132] 4. Input the requested domain names from the historical DNS logs into the fine-tuned BERT model in batches, filter the requested domain names with a confidence score ≥ 0.9, and extract features such as root domain name, push keywords (push / rtc / livepush / send / rtmp / rtsp, etc.), subdomain structure, etc. to form an initial domain name feature library;
[0133] 5. Collect new DNS logs weekly, automatically identify unlabeled live streaming domains using a fine-tuned BERT model, manually verify samples with a confidence level of 0.85-0.9, and update them to the initial domain feature database after confirmation to ensure real-time adaptation to new live streaming platforms.
[0134] S2. Construct an address signature database and parse AAA and DNS logs;
[0135] 1. Obtain the IP address range and its associated ASN number through the BGP routing table (each city has a different ASN number, and the province where the city is located can be determined from the city number); associate the home province, home city and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database;
[0136] 2. Parse the AAA logs (obtained from the routing device) and use regular expressions to filter out invalid data such as empty accounts, incorrect IPs, and invalid port ranges (outside 0-65535);
[0137] 3. Based on the initial full set of online user data, the user status (online / offline) is updated using a 5-minute sliding window algorithm and written to HDFS in the format of "NAT address + port range + account + time";
[0138] 4. Parse the DNS logs (obtained from the routing device), input the requested domain name into the fine-tuned BERT model, and obtain the DNS request records for live streaming; based on the initial domain name feature library, address feature library, and online user data table, add application tags, province of origin, city of origin, network type, and user tags.
[0139] S3. Construct a CDN node distribution table based on the GraphSAGE algorithm;
[0140] 1. Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS log according to application tag + home province + home city; use the A record addresses in the DNS log as nodes of the undirected graph, and construct edges between nodes of the undirected graph based on the same application tag + home province + home city and its corresponding A record address, to form an undirected graph containing A record addresses and network topology;
[0141] The formula for an undirected graph is as follows:
[0142] G=(V,E)
[0143] Where V is the set of addresses of record A, and E is the set of edges associated with the addresses of record A;
[0144] 2. Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of the A record address containing network topology information;
[0145] 3. Based on the feature vector of the A record address containing network topology information, cluster the A record addresses into CDN nodes, and divide them into CDN node units with three dimensions: application tag, home province, and home city, and clarify the home province and home city of each CDN node unit;
[0146] 4. Use CDN nodes to associate application tags, home province, home city and network type to build a CDN node distribution table.
[0147] S4. Use Flow traffic to analyze user push traffic and push traffic received by CDN nodes;
[0148] 1. Aggregate flow traffic (obtained from routing devices) at 5-minute granularity, and associate it with the CDN node distribution table and online user data table;
[0149] Furthermore, based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered (in the above steps, the CDN nodes have been identified as the CDN nodes used by the live streaming application, which is equivalent to obtaining all the Flow traffic of the live streaming application), and the filtered destination IP addresses are tagged with CDN node tags and application tags.
[0150] Furthermore, the source IP address and source port in the Flow traffic are used to match the online user data table, and user tags are added to the matching source IP address and source port;
[0151] 2. Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic;
[0152] Furthermore, if the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application, and the traffic scale of this user's request to the application can be obtained based on the Flow traffic.
[0153] Furthermore, if the Flow traffic does not match the DNS log, that is, there is no data in the DNS log that requests the address to the A record address, it means that the source IP address in the Flow traffic is not a user of this network or a DNS user of the operator. Then, according to the application tag on the destination IP address in the Flow traffic, the traffic analysis is performed, and the Flow traffic is also added to the application corresponding to the destination IP address in the Flow traffic, but it belongs to the application's cross-network traffic.
[0154] Furthermore, if the DNS logs cannot match the Flow traffic, it may be because the Flow traffic has been sampled and there is no corresponding Flow traffic in the DNS logs. In this case, based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
[0155] S5. Analyze and predict the traffic flow direction of live streaming based on the TFT model;
[0156] 1. Collect historical push traffic data and integrate characteristics that affect traffic, such as application type, holidays, and network load;
[0157] 2. Train the flow prediction model based on the TFT model, so that the flow prediction model learns the pattern of flow change over time;
[0158] 3. The trained traffic prediction model outputs the trend, peak period and scale of future traffic flow.
[0159] In practice, historical push traffic (hourly granularity) for 7 days is collected, and a prediction dataset is constructed by combining characteristics such as application type, holidays, and network load.
[0160] In practice, the TFT model is trained according to the following characteristics and parameters:
[0161] Model structure: The input layer contains temporal feature embeddings and application feature embeddings, the hidden layer is set to 256 dimensions, and the attention mechanism focuses on key influencing features;
[0162] Training parameters: learning rate 1e-4, training 30 epochs, loss function is MAE (mean absolute error).
[0163] In practice, the model outputs the trend of push traffic, peak periods and peak scale for the next 24 hours, providing support for network resource scheduling.
[0164] Based on the same inventive concept, this invention also proposes a live streaming service analysis device based on a large model. The implementation of this device can refer to the implementation of the methods described above, and repeated details will not be repeated. The term "module" used below can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0165] Figure 2 This is a schematic diagram of the live streaming service analysis device based on a large model according to the present invention. Figure 2 As shown, the device includes:
[0166] The live streaming domain name recognition module 101 is used to dynamically identify the characteristics of live streaming domain names based on the BERT model, including:
[0167] Using a pre-trained BERT model for scenario transfer learning of live streaming domain names, the semantic features of live streaming domain names, including root domain names and streaming keywords, are automatically mined.
[0168] Historical DNS logs are input into a pre-trained BERT model to filter out live streaming domains, extract core features including live streaming domain, application name, and root domain, and form an initial domain feature library.
[0169] New DNS logs are collected regularly, and live streaming domains that are not included in the initial domain feature library are automatically identified through a pre-trained BERT model. The initial domain feature library is then updated after verification.
[0170] Log parsing module 102 is used to build an address signature database and parse AAA logs and DNS logs, including:
[0171] Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database;
[0172] Extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic and downlink traffic, and build an online user data table based on the AAA logs and the initial AAA online users;
[0173] Extract the request address, request port, A record address, request domain name, and request time from the DNS log; match the request domain name from the DNS log against the initial domain name feature database and tag the matching request domain name with an application label; match the request address and A record address from the DNS log against the address feature database and tag the matching request address and A record address with the province, city, and network type.
[0174] Associate the request time, request address, and request port in the DNS log with the timestamp, NAT address, and port range in the AAA log, and tag the associated AAA log with the user.
[0175] CDN node distribution table construction module 103 is used to construct a CDN node distribution table based on the GraphSAGE algorithm, including:
[0176] Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS logs according to application tag + home province + home city; use the A record addresses in the DNS logs as nodes in an undirected graph, and construct edges between nodes in the undirected graph based on the same application tag + home province + home city and its corresponding A record address, forming an undirected graph containing A record addresses and network topology;
[0177] Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of A record addresses containing network topology information.
[0178] Based on the feature vector of the A record address containing network topology information, the A record address is clustered into CDN nodes, and CDN node units are divided into three dimensions: application tag, home province, and home city, and the home province and home city of each CDN node unit are clearly defined.
[0179] A CDN node distribution table is constructed by associating CDN nodes with application tags, home province, home city, and network type.
[0180] The push stream traffic analysis module 104 is used to analyze user push stream traffic and push stream traffic received by CDN nodes using Flow traffic, including:
[0181] Based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses;
[0182] Use the source IP address and source port in the Flow traffic to match the online user data table, and tag the matched source IP address and source port with user tags;
[0183] Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic;
[0184] If the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application;
[0185] If the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic is analyzed according to the application tag on the destination IP address in the Flow traffic and the traffic is counted as the cross-network traffic of the CDN node.
[0186] If the DNS logs do not match the flow traffic, then based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
[0187] The live streaming traffic prediction module 105 is used to analyze and predict the traffic flow direction of live streaming based on the TFT model, including:
[0188] Collect historical push traffic and integrate characteristics that affect traffic, such as application type, holidays, and network load.
[0189] The flow prediction model is trained based on the TFT model, allowing the flow prediction model to learn the pattern of flow changes over time;
[0190] The trained traffic prediction model outputs the trend, peak period, and scale of future traffic flow.
[0191] It should be noted that although several modules of the live streaming business analysis device based on a large model are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.
[0192] Based on the aforementioned inventive concept, such as Figure 3 As shown, the present invention also proposes a computer device 200, including a memory 210, a processor 220, and a computer program 230 stored in the memory 210 and executable on the processor 220. When the processor 220 executes the computer program 230, it implements the aforementioned live streaming service analysis method based on a large model.
[0193] Based on the aforementioned inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program that executes the aforementioned live streaming business analysis method based on a large model.
[0194] This invention proposes a live streaming push business analysis method and device based on a large model. Through multi-dimensional data fusion and correlation analysis, it accurately identifies the client accounts, traffic scale, CDN node distribution, and cross-regional / cross-operator flow of live streaming push, providing data support for operators to optimize network resource allocation, formulate tariff policies, and carry out business operation decisions.
[0195] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
[0196] Regarding the limitation of the scope of protection of this invention, those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solution of this invention are still within the scope of protection of this invention.
Claims
1. A live streaming push business analysis method based on a large model, characterized in that, The method includes: Dynamically identify the characteristics of live streaming domain names based on the BERT model; Build an address signature database and parse AAA and DNS logs; Construct a CDN node distribution table based on the GraphSAGE algorithm; Use Flow traffic to analyze user push traffic and push traffic received by CDN nodes; The flow of live streaming traffic is analyzed and predicted based on the TFT model.
2. The live streaming business analysis method based on a large model according to claim 1, characterized in that, Dynamically identify the characteristics of live streaming domain names based on the BERT model, including: Using a pre-trained BERT model for scenario transfer learning of live streaming domain names, the semantic features of live streaming domain names, including root domain names and streaming keywords, are automatically mined. Historical DNS logs are input into a pre-trained BERT model to filter out live streaming domains, extract core features including live streaming domain, application name, and root domain, and form an initial domain feature library. New DNS logs are collected regularly, and live streaming domains that are not included in the initial domain feature library are automatically identified through a pre-trained BERT model. The initial domain feature library is then updated after verification.
3. The live streaming business analysis method based on a large model according to claim 1, characterized in that, Build an address signature database and parse AAA and DNS logs, including: Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database; Extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic and downlink traffic, and build an online user data table based on the AAA logs and the initial AAA online users; Extract the request address, request port, A record address, request domain name, and request time from the DNS log; match the request domain name from the DNS log against the initial domain name feature database, and tag the matching request domain name with an application label; match the request address and A record address from the DNS log against the address feature database, and tag the matching request address and A record address with the province, city, and network type. Associate the request time, request address, and request port in the DNS log with the timestamp, NAT address, and port range in the AAA log, and tag the associated AAA log with the user.
4. The live streaming business analysis method based on a large model according to claim 1, characterized in that, A CDN node distribution table is constructed based on the GraphSAGE algorithm, including: Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS logs according to application tag + home province + home city; use the A record addresses in the DNS logs as nodes in an undirected graph, and construct edges between nodes in the undirected graph based on the same application tag + home province + home city and its corresponding A record address, forming an undirected graph containing A record addresses and network topology; Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of A record addresses containing network topology information. Based on the feature vector of the A record address containing network topology information, the A record address is clustered into CDN nodes, and CDN node units are divided into three dimensions: application tag, home province, and home city, and the home province and home city of each CDN node unit are clearly defined. A CDN node distribution table is constructed by associating CDN nodes with application tags, home province, home city, and network type.
5. The live streaming business analysis method based on a large model according to claim 1, characterized in that, Using Flow traffic, analyze user push traffic and push traffic received by CDN nodes, including: Based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses; Use the source IP address and source port in the Flow traffic to match the online user data table, and tag the matched source IP address and source port with user tags; Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic; If the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application; If the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic analysis is performed according to the application tag on the destination IP address in the Flow traffic, and the Flow traffic is counted as the cross-network traffic of the CDN node. If the DNS logs do not match the flow traffic, then based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
6. The live streaming business analysis method based on a large model according to claim 1, characterized in that, The flow of live streaming traffic is analyzed and predicted based on the TFT model, including: Collect historical push traffic and integrate characteristics that affect traffic, such as application type, holidays, and network load. The flow prediction model is trained based on the TFT model, allowing the flow prediction model to learn the pattern of flow changes over time; The trained traffic prediction model outputs the trend, peak period, and scale of future traffic flow.
7. A live streaming service analysis device based on a large model, characterized in that, The device includes: The live streaming domain name recognition module is used to dynamically identify the characteristics of live streaming domain names based on the BERT model. The log parsing module is used to build an address signature database and parse AAA logs and DNS logs; The CDN node distribution table construction module is used to construct a CDN node distribution table based on the GraphSAGE algorithm. The push stream traffic analysis module is used to analyze user push stream traffic and push stream traffic received by CDN nodes using Flow traffic. The live streaming traffic prediction module is used to analyze and predict the traffic flow direction of live streaming based on the TFT model.
8. The live streaming service analysis device based on a large model according to claim 7, characterized in that, Dynamically identify the characteristics of live streaming domain names based on the BERT model, including: Using a pre-trained BERT model for scenario transfer learning of live streaming domain names, the semantic features of live streaming domain names, including root domain names and streaming keywords, are automatically mined. Historical DNS logs are input into a pre-trained BERT model to filter out live streaming domains, extract core features including live streaming domain, application name, and root domain, and form an initial domain feature library. New DNS logs are collected regularly, and live streaming domains that are not included in the initial domain feature library are automatically identified through a pre-trained BERT model. The initial domain feature library is then updated after verification.
9. The live streaming service analysis device based on a large model according to claim 7, characterized in that, Build an address signature database and parse AAA and DNS logs, including: Obtain IP address ranges and their associated ASN numbers through the BGP routing table; associate the home province, home city, and network type corresponding to the ASN number with the request address and A record address in the DNS log to build an address feature database; Extract the core fields from the AAA logs, including user account, NAT address, port range, timestamp, access type, uplink traffic and downlink traffic, and build an online user data table based on the AAA logs and the initial AAA online users; Extract the request address, request port, A record address, request domain name, and request time from the DNS log; match the request domain name from the DNS log against the initial domain name feature database, and tag the matching request domain name with an application label; match the request address and A record address from the DNS log against the address feature database, and tag the matching request address and A record address with the province, city, and network type. Associate the request time, request address, and request port in the DNS log with the timestamp, NAT address, and port range in the AAA log, and tag the associated AAA log with the user.
10. The live streaming service analysis device based on a large model according to claim 7, characterized in that, A CDN node distribution table is constructed based on the GraphSAGE algorithm, including: Summarize application tags, home province, home city, network type, and user tags, and summarize the A record addresses in the DNS logs according to application tag + home province + home city; use the A record addresses in the DNS logs as nodes in an undirected graph, and construct edges between nodes in the undirected graph based on the same application tag + home province + home city and its corresponding A record address, forming an undirected graph containing A record addresses and network topology; Based on the GraphSAGE algorithm, each A record address in the DNS log learns the application label, home province, and home city of its neighboring A record addresses, thereby generating a feature vector of A record addresses containing network topology information. Based on the feature vector of the A record address containing network topology information, the A record address is clustered into CDN nodes, and CDN node units are divided into three dimensions: application tag, home province, and home city, and the home province and home city of each CDN node unit are clearly defined. A CDN node distribution table is constructed by associating CDN nodes with application tags, home province, home city, and network type.
11. The live streaming service analysis device based on a large model according to claim 7, characterized in that, Using Flow traffic, analyze user push traffic and push traffic received by CDN nodes, including: Based on the CDN node distribution table, the destination IP addresses in the Flow traffic are filtered, and CDN node tags and application tags are added to the filtered destination IP addresses; Use the source IP address and source port in the Flow traffic to match the online user data table, and tag the matched source IP address and source port with user tags; Use the request address and A record address in the DNS log to match the source IP address and destination IP address in the flow traffic to determine the application and user associated with the flow traffic; If the DNS log matches the Flow traffic, then the Flow traffic is the push traffic from the user to the application; If the Flow traffic does not match the DNS log, it means that the source IP address in the Flow traffic is not a user of this network or a user of the carrier's DNS. In this case, the traffic analysis is performed according to the application tag on the destination IP address in the Flow traffic, and the Flow traffic is counted as the cross-network traffic of the CDN node. If the DNS logs do not match the flow traffic, then based on the application tag corresponding to the A record address in the DNS logs and the uplink traffic in the AAA logs, the uplink traffic in the AAA logs will be used as the push traffic from the user to the application.
12. The live streaming service analysis device based on a large model according to claim 7, characterized in that, The flow of live streaming traffic is analyzed and predicted based on the TFT model, including: Collect historical push traffic and integrate characteristics that affect traffic, such as application type, holidays, and network load. The flow prediction model is trained based on the TFT model, allowing the flow prediction model to learn the pattern of flow changes over time; The trained traffic prediction model outputs the trend, peak period, and scale of future traffic flow.
13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-6.
14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that performs the method according to any one of claims 1-6.