Content delivery network (CDN) system merging back-to-source method and device

Abstract

The invention relates to a CDN (Content Delivery Network) system merging back-to-source method and device. The method comprises the following steps: in response to received streaming media requests, if the streaming media requests have the same protocol, merging the streaming media requests and returning to a source; wherein the Hash keys merged back to the source are obtained based on an anonymous communication network; and if the streaming media request has different protocols, unifying the source return path of the streaming media request based on an anonymous processing mechanism of protocol conversion. According to the method and the device, the requests can be effectively aggregated, the back-to-source path is unified, and the back-to-source efficiency of the CDN system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a CDN system merging back source method and device. BACKGROUND

[0002] In the architecture design of CDN live streaming system, a three-layer distributed deployment mode is generally adopted, including an edge node layer, a parent node layer and a central source station layer. In order to protect the privacy and anonymity of data transmission, all service nodes are deployed in the Tor anonymous network environment. Under this architecture, the communication between any two nodes must comply with the three-hop forwarding mechanism of Tor network, that is, the data packet needs to be encrypted and forwarded by the entry node (Entry Node), the intermediate node (Middle Node) and the exit node (Exit Node) in turn, so as to ensure that the communication link cannot be tracked.

[0003] However, the back source efficiency of the traditional CDN system back source mode is poor. SUMMARY

[0004] Therefore, it is necessary to provide a CDN system merging back source method and device capable of improving the back source efficiency of the CDN system in view of the above technical problems.

[0005] In a first aspect, the present application provides a CDN system merging back source method, which comprises:

[0006] In response to receiving a streaming media request, if the streaming media requests have the same protocol, the streaming media requests are merged and back sourced; wherein the hash key of the merged and back sourced streaming media requests is obtained based on an anonymous communication network;

[0007] If the streaming media requests have different protocols, the back source paths of the streaming media requests are unified based on a protocol conversion anonymous processing mechanism.

[0008] In one embodiment, if the streaming media requests have the same protocol, the streaming media requests are merged and back sourced, which comprises:

[0009] For the streaming media requests with the same protocol, domain name, publication point and stream name, the corresponding hash key is obtained based on the anonymous communication network and the consistent hash algorithm in the edge node layer;

[0010] The streaming media requests with the same hash key value are directed to the same target edge node based on a preset hash mapping rule, and an aggregated request is generated;

[0011] In response to the aggregated request, the optimal service process is selected based on a dynamic weight distribution algorithm to perform a back source streaming pulling operation; wherein the dynamic weight distribution algorithm is used to ensure that a single process carries the traffic of all streaming media requests with the same hash key value.

[0012] In one embodiment, the corresponding hash key is obtained based on the consistent hash algorithm of the anonymous communication network at the edge node layer, comprising:

[0013] A first hash key is obtained based on the domain name, the publication point and the stream name.

[0014] A second hash key is obtained according to the first hash key and the node weighting factor of the anonymous communication network.

[0015] In one embodiment, the method further comprises:

[0016] A plurality of virtual nodes are generated at each physical node, and a virtual hash value is calculated based on the second hash key.

[0017] In one embodiment, if the stream media requests have different protocols, the source return path of the stream media requests is unified based on a protocol conversion anonymous processing mechanism, comprising:

[0018] The stream media requests with different protocols and the same domain name, publication point and stream name are protocol-converted to unify the source return path.

[0019] The protocol conversion is encrypted based on a plurality of encryption tunnels of the anonymous communication network.

[0020] The exit node of the protocol conversion link is dynamically selected based on a weighting algorithm.

[0021] In one embodiment, the method further comprises:

[0022] The sharding transmission of the source return data is performed based on a multi-path concurrent transmission optimization strategy and a dynamic sharding adjustment algorithm; the dynamic sharding adjustment algorithm is used to adjust the sharding size and quantity of the source return data according to real-time network conditions to obtain a plurality of data pieces; the multi-path concurrent transmission optimization strategy comprises: allocating an independent anonymous communication circuit for each data piece to enable the plurality of data pieces to be transmitted simultaneously on different anonymous communication circuits.

[0023] In a second aspect, the application provides a CDN system source return merging device, comprising:

[0024] A source return merging module is configured to, in response to receiving a stream media request, merge the source return of the stream media request if the stream media request has the same protocol; wherein the hash key of the source return is obtained based on the anonymous communication network.

[0025] A path unification module is configured to, if the stream media requests have different protocols, unify the source return path of the stream media requests based on a protocol conversion anonymous processing mechanism.

[0026] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0027] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0028] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0029] The aforementioned CDN system's origin merging method and apparatus, in response to receiving streaming media requests, merges and performs origin merging on streaming media requests if they share the same protocol. The hash key for origin merging is obtained based on an anonymous communication network. If the streaming media requests have different protocols, the origin merging path is unified based on an anonymous processing mechanism using protocol conversion. This achieves effective request aggregation and unified origin merging paths, thereby improving the origin merging efficiency of the CDN system. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a diagram illustrating the application environment of the CDN system's origin merging method in one embodiment.

[0032] Figure 2 This is a flowchart illustrating the CDN system's origin merging method in one embodiment;

[0033] Figure 3 This is a schematic diagram illustrating the merging and origin-based return of streaming media requests using the same protocol in one embodiment.

[0034] Figure 4 This is a schematic diagram of a P2P request link before merging streaming media requests from different protocols in one embodiment.

[0035] Figure 5 This is a schematic diagram of a P2P request link after merging streaming media requests from different protocols in one embodiment;

[0036] Figure 6 This is a schematic diagram of the protocol anonymity conversion mechanism and dynamic egress node scheduling in one embodiment;

[0037] Figure 7 This is a schematic diagram of a concurrent back-to-source process design in one embodiment;

[0038] Figure 8 This is a schematic diagram of a dynamic fragmentation adjustment algorithm in one embodiment;

[0039] Figure 9 This is a structural block diagram of a CDN system merging back-to-source device in one embodiment;

[0040] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0043] In the architecture design of CDN live streaming systems, a three-layer distributed deployment model is commonly adopted, including an edge node layer, a parent node layer, and a central origin server layer. To ensure the privacy and anonymity of data transmission, all service nodes are deployed in a Tor anonymous network environment. Under this architecture, communication between any two nodes must follow the Tor network's three-hop forwarding mechanism, that is, data packets must be encrypted and forwarded sequentially through the entry node, middle node, and exit node, thereby ensuring that the communication link cannot be traced. While this design effectively achieves network anonymity, it also introduces additional transmission latency and performance overhead. Therefore, this architecture and communication method have the following problems:

[0044] (1) High resource consumption: Each machine in the edge layer needs to pull the stream to the upper layer, resulting in high CPU consumption and redundant stream cache.

[0045] (2) High parent layer load: A large number of requests are sent directly to the parent layer, which increases the QPS (queryper second) of the parent layer and increases the machine load.

[0046] (3) Redundant request problem: Traditional edge nodes independently retrieve the same streaming media request (same domain, publisher, stream name triplet), causing multiple servers in the edge layer to repeatedly pull the same content from the parent node in the upper layer, resulting in a waste of CPU resources of the edge nodes (actual tests show that the redundant computing overhead accounts for 35%-42%), and the QPS pressure of the parent node increases linearly (for every 1,000 additional edge nodes, the parent load increases by 18%-23%). Stream data is repeatedly cached in multiple nodes in the edge layer, and the memory utilization rate decreases by 27%-31%.

[0047] (4) Complex Protocol Processing: Requests using different protocols (such as FLV and P2P) require separate processing, increasing system complexity. In traditional solutions, FLV requests are directly processed by the edge gateway and sent back to the parent layer. P2P requests require navigating through the edge gateway → edge CLPS (P2P service) → parent gateway → parent CLPS, resulting in a complex and inefficient path. When both FLV and P2P requests exist simultaneously, FLV and P2P requests with the same content are sent back to the parent layer separately, leading to repeated streaming at the edge layer, increasing CPU and bandwidth consumption. Simultaneously, the parent gateway must handle a large number of duplicate requests. The same stream of data may be stored multiple times at the edge layer using different protocols, reducing caching efficiency.

[0048] (5) Low communication transmission efficiency: In the Tor anonymous network architecture, to ensure the anonymity of communication, data transmission between any two nodes must go through a strict three-hop node forwarding mechanism. Although this multi-hop forwarding feature effectively protects communication privacy, it inevitably introduces significant transmission delay and bandwidth loss. Especially when dealing with large-scale video streaming scenarios with stringent real-time requirements, this multi-hop forwarding mechanism will form a significant performance bottleneck, resulting in a significant decrease in end-to-end transmission efficiency, making it difficult to meet the needs of high-concurrency, low-latency video streaming transmission.

[0049] Based on the aforementioned traditional technologies, this application provides a CDN system origin merging method based on anonymous network link selection. This method addresses the problems of high resource consumption, high parent layer load, numerous redundant requests, complex protocol processing, and low communication transmission efficiency inherent in traditional CDN live streaming system architectures. By designing a weighted hash key based on the Tor network, proposing an anonymization mechanism for protocol conversion, and introducing a multi-path concurrent transmission optimization strategy, it aims to comprehensively improve the performance, resource utilization, and security of CDN live streaming systems in the Tor anonymous network environment. Optionally, this application can be applied to confidential organizations such as governments, enterprises, and the military using CDNs, particularly in scenarios requiring the protection of data transmission privacy and anonymity, ensuring security while improving transmission efficiency.

[0050] Specifically, the embodiments of this application can reduce resource consumption and redundant requests. By introducing Tor node-related attributes (such as bandwidth, latency, and stability coefficients) as weighting factors, a weighted hash key based on the Tor network is designed to merge requests with the same protocol for origin pull. This design aims to reduce CPU consumption and stream caching at the edge layer, while reducing the request volume (QPS) and machine load at the parent layer, thereby effectively solving the problems of high resource consumption and numerous redundant requests.

[0051] Furthermore, the embodiments of this application can also improve resource utilization and security. The embodiments of this application propose an anonymization mechanism for protocol conversion. This mechanism dynamically converts P2P protocol requests into FLV format through the edge layer CLPS service, unifies the origin-back path, and transmits them within a Tor three-layer encrypted tunnel. This mechanism not only eliminates redundant traffic generated by multi-layer P2P hops and shortens the origin-back path, but also improves caching efficiency and resource utilization.

[0052] Furthermore, the embodiments of this application can also improve data transmission efficiency and system robustness. By introducing multiple independent Tor circuits to achieve data fragmentation transmission, data transmission efficiency is improved, and a dynamic fragmentation adjustment algorithm is used to ensure high reliability of transmission and system robustness. Even when the Tor network is unstable or subject to interference, efficient and stable data transmission can be maintained.

[0053] Therefore, compared with traditional technologies, the inventiveness of this application's embodiments lies in: 1. Innovative merging and origin-based mechanism: Combining the characteristics of the Tor network, a weighted hash key based on Tor nodes is designed to achieve intelligent merging of requests with the same protocol, solving the problem of redundant requests in traditional architectures. 2. Flexible protocol conversion and anonymization processing: An anonymization processing mechanism for protocol conversion is proposed, which can dynamically convert protocols according to actual needs and maintain anonymity during transmission, enhancing the system's flexibility and security. 3. Efficient data transmission strategy: Multi-path concurrent transmission and dynamic fragmentation adjustment algorithms are introduced, fully utilizing the characteristics of the Tor network, improving data transmission efficiency and system robustness, and providing a new solution for CDN live streaming systems.

[0054] For example, possible future application scenarios for the embodiments of this application include: 1. High-definition video live streaming: In high-definition video live streaming scenarios, the method of the embodiments of this application can significantly improve data transmission efficiency, reduce latency, and enhance user experience. 2. Online education: Online education platforms require stable and efficient video transmission services, and the method of the embodiments of this application can ensure the real-time and smooth transmission of online education content. 3. Telemedicine: Telemedicine services have high requirements for the real-time performance and security of data transmission, and the method of the embodiments of this application can provide efficient and stable data transmission support while ensuring anonymity.

[0055] Furthermore, the embodiments of this application can also: 1. Improve user experience: By improving data transmission efficiency and reducing latency, the method of the embodiments of this application can significantly improve the user experience when watching videos, participating in online education, or telemedicine services. 2. Reduce operating costs: By reducing resource consumption and redundant requests, the method of the embodiments of this application can help CDN service providers reduce operating costs and improve profitability. 3. Promote technological innovation: The method of the embodiments of this application combines the advantages of anonymous communication and CDN technology, providing new ideas and directions for technological innovation in related fields. With the continuous development of technology and the expansion of application scenarios, the method of the embodiments of this application is expected to generate broader social and economic benefits.

[0056] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to the above content, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.

[0057] Before introducing the specific embodiments of this application, the technical terms involved in this application will be explained:

[0058] Tor (The Onion Router) is an anonymous communication system consisting of a directory server, an Onion Router (OR), and an Onion Proxy (OP). Users can communicate anonymously on the Internet through Tor.

[0059] An Onion Router (or simply "Onion") is a router that periodically communicates with other Tors, thus forming a virtual circuit within the Tor network.

[0060] CDN (Content Delivery Network) is a technology that speeds up user access by caching content on multiple geographically dispersed servers. To achieve efficient and stable content delivery, CDN technology deploys node servers in multiple locations across the internet, analyzing network traffic, node connection status, load levels, user distance, and service response time in real time. It then redirects user requests to the nearest service node, reducing bottlenecks in data transmission efficiency and stability. This enables users on the internet to quickly access the content they need, improving website response times.

[0061] FLV (Flash Video) is a streaming media container format developed by Adobe. It uses a lightweight binary structure to store audio and video data, supports H.264 / AAC encoding, and is suitable for network transmission. Its file consists of a Header and a Body. The Body contains alternating Tags (video / audio / script data) and a PreviousTagSize field, featuring low latency and easy parsing.

[0062] P2P (Peer-to-Peer) is a decentralized network architecture that allows nodes to directly transmit data without a central server, significantly improving transmission efficiency and reducing bandwidth costs. Its core advantages lie in multi-source concurrent downloading, support for downloading while playing, and dynamic expansion capabilities, making it suitable for high-bandwidth scenarios such as video streaming.

[0063] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0064] The CDN system origin merging method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0065] In one exemplary embodiment, such as Figure 2 As shown, a method for merging origin pulls in a CDN system is provided, which is then applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 204. Wherein:

[0066] Step 202: In response to receiving a streaming media request, if the streaming media requests have the same protocol, the streaming media requests are merged and returned to the origin; wherein, the hash key for merging and returning to the origin is obtained based on the anonymous communication network.

[0067] In this context, streaming media request can refer to FLV request and / or P2P request, or simply request; protocol can refer to FLV protocol and / or P2P protocol, but this application is not limited thereto; hash key can also be called hash key; anonymous communication network can refer to Tor network.

[0068] Specifically, in response to receiving a streaming media request, if the streaming media requests have the same protocol, the streaming media requests are merged and returned to the origin; wherein, the hash key for merging and returning to the origin is obtained based on an anonymous communication network.

[0069] For example, by introducing node-related attributes of the Tor network (such as bandwidth, latency, and stability coefficient) as weighting factors, a weighted hash key based on the Tor network is designed to merge requests of the same protocol for origin retrieval, thereby reducing CPU consumption and stream caching at the edge layer, while reducing the request volume (QPS) and machine load of the parent layer, thus effectively solving the problems of high resource consumption and many redundant requests.

[0070] Step 204: If the streaming media requests have different protocols, the origin path of the streaming media requests is unified based on the anonymous processing mechanism of protocol conversion.

[0071] Specifically, if streaming media requests have different protocols, an anonymous processing mechanism based on protocol conversion will convert the different protocols into the same protocol, thereby unifying the origin path of streaming media requests.

[0072] Optionally, the protocol-conversion-based anonymization mechanism dynamically converts P2P protocol requests into FLV format through edge-layer P2P streaming components, such as CLPS services, unifying the origin-back path and transmitting them within a Tor layer-3 encrypted tunnel. This mechanism not only eliminates redundant traffic generated by multi-level P2P redirects and shortens the origin-back path, but also improves caching efficiency and resource utilization.

[0073] In the aforementioned CDN system's origin merging method, in response to a received streaming media request, if the streaming media requests have the same protocol, the streaming media requests are merged and returned to the origin. The hash key for merging and returning to the origin is obtained based on an anonymous communication network. If the streaming media requests have different protocols, the origin return path of the streaming media requests is unified based on an anonymous processing mechanism for protocol conversion. This achieves effective aggregation of requests and a unified origin return path, thereby improving the origin return efficiency of the CDN system.

[0074] In an exemplary embodiment, step 202, if the streaming media requests have the same protocol, involves merging the streaming media requests back to the origin, including: for streaming media requests with the same domain name, publishing point, and stream name and the same protocol, obtaining the corresponding hash key using a consistent hashing algorithm at the edge node layer based on an anonymous communication network; directing streaming media requests with the same hash key value to the same target edge node based on a preset hash mapping rule, and generating an aggregation request; in response to the aggregation request, selecting the optimal service process based on a dynamic weight allocation algorithm to perform the back-to-origin streaming operation; wherein, the dynamic weight allocation algorithm is used to ensure that a single process carries the traffic of all streaming media requests with the same hash key value.

[0075] Specifically, refer to Figure 3This diagram illustrates the merging and origin-pull processing of streaming media requests using the same protocol in one embodiment. In this CDN program, the concept of an edge node refers to a machine group (the entire box labeled "edge" in the diagram). A machine group can include machines 1 and 2. Each machine may receive streaming requests, which are merged into a single request based on a hash key and forwarded upwards. LVS is a component that load balances requests. All requests are first sent to the edge LVS machine, which then distributes the requests to the machines on each edge node to prevent excessive QPS from damaging hotspot machines. The concept of a parent node is the same as that of an edge node. An edge node is a machine group containing many machines, each with the ability to independently process requests. Each machine has multiple processes, and a streaming request is executed by a specific process within that machine.

[0076] For example, this application proposes a request aggregation mechanism. For requests with the same domain name, publication point, and flow name under the same protocol, a consistent hashing algorithm is used at the edge node layer to generate a hash key. Requests with the same key value are directed to the same target edge node through a preset hash mapping rule, thus achieving request aggregation. Furthermore, to ensure process-level load balancing, after receiving the aggregation request, the target edge node selects the optimal service process based on a dynamic weight allocation algorithm to perform a back-to-origin traffic pull operation, ensuring that a single process carries all request traffic with the same hash key value. That is, request merging is achieved through three layers of hashing: [flow identifier hash] → [machine-level routing] → [process-level load balancing].

[0077] In an exemplary embodiment, a consistent hashing algorithm is used at the edge node layer based on the anonymous communication network to obtain the corresponding hash key, including: obtaining a first hash key based on the domain name, publication point, and stream name; and obtaining a second hash key based on the first hash key and the node weighting factor of the anonymous communication network.

[0078] For example, this application proposes a weighted hash key based on the Tor network. This hash key not only considers traditional request-level factors (such as domain name, publishing point, and stream name), but also incorporates Tor node-related attributes as weighting factors. The calculation process of the hash key is as follows:

[0079] First, define the flow identifier triple F=(D,P,S) (domain name, publication point, flow name), and use the improved FNV-1a hash algorithm:

[0080]

[0081] in, It is the basic hash value calculated based on the domain name, publishing point, and stream name, which is the first hash value, ensuring that the same F has the same hash value; It is the largest prime number not greater than the total number of Tor nodes (here, we take 16063).

[0082] Secondly, the weighting factor for Tor nodes is introduced into the node state vector N=(b,l,σ) (bandwidth / Mbps, latency / ms, stability coefficient). The weighting formula is as follows:

[0083]

[0084] in, Is The second hash value is a weighted hash value formed by adding performance metrics of Tor nodes to the existing hash value. The higher the bandwidth, the lower the latency, and the higher the stability of the node, the higher its hash value. The higher the value, the greater the dynamic weighting coefficient. =0.6, =0.3, =0.1; stability coefficient =Total successful requests ∈ [0,1].

[0085] In one exemplary embodiment, the method further includes: generating multiple virtual nodes on each physical node and calculating a virtual hash value based on a second hash key.

[0086] Optionally, due to the potential for frequent changes in streaming machines caused by the weighted hash key design based on the Tor network, anti-jitter optimization is implemented. This is achieved by using virtual node mapping to reduce the frequency of target edge node changes due to hash key variations. Each physical node generates... Virtual nodes are used to reduce the probability of physical node switching by increasing the hash space dimension. Virtual hash value. The calculation formula is as follows:

[0087] .

[0088] In an exemplary embodiment, step 204, if the streaming media requests have different protocols, unifies the back-to-origin path of the streaming media requests based on the anonymous processing mechanism of protocol conversion, including: converting the protocols of streaming media requests with different protocols that have the same domain name, publishing point and stream name to unify the back-to-origin path; encrypting the protocol conversion based on multiple encrypted tunnels of the anonymous communication network; and dynamically selecting the exit node of the protocol conversion link based on a weight algorithm.

[0089] For example, refer to Figure 4 and Figure 5 This application proposes a design for merging back-to-source protocols for different protocols, wherein... Figure 4 This is a schematic diagram of a P2P request chain before merging streaming media requests from different protocols in one embodiment.Figure 5 This is a schematic diagram of a P2P request chain after merging streaming media requests from different protocols in one embodiment. Here, clps refers to the p2p streaming component and the gateway refers to the flv streaming component. They run on different processes on the same machine. It can be understood that after merging different protocols through the method of this application embodiment, the merging and back-to-source of the same hash key can continue.

[0090] Specifically, currently, FLV requests are directly sent to the gateway, which processes them and then sends a request back to the parent layer's origin server. For P2P requests, a P2P service called clps is deployed on the same machine as the gateway, in addition to the gateway service. When a P2P request arrives, it first goes to the gateway, which then requests clps. clps then sends a request back to the parent layer's gateway, which in turn requests the parent layer's clps. The entire process uses the P2P request suffix.

[0091] The FLV request back-to-origin path includes, in sequence: edge node → edge gateway node → parent gateway node → origin server. Communication between nodes is achieved through a three-hop anonymous link in the Tor network. The content transmission path is defined as a four-tuple as follows:

[0092] Path = (EdgeNode, EdgeGateway, ParentGateway, OriginServer)

[0093] Where: EdgeNode ∈ E (set of edge nodes); EdgeGateway ∈ G_e (set of edge gateways); ParentGateway ∈ G_p (set of parent gateways); OriginServer ∈ S (set of origin servers).

[0094] The above transmission process satisfies: e∈E, !g_e∈G_e, !g_p∈G_p, !s∈S, such that e → g_e→ g_p → s constitutes a valid transmission chain.

[0095] The p2p request back-to-origin path includes, in sequence: edge computing node → edge gateway node → edge layer CLPS node → parent layer gateway node → parent layer CLPS node → origin server. Communication between nodes at each level is achieved through a three-hop encrypted link in the Tor network, and the content location service node is responsible for dynamic routing decisions. The transmission path is defined as an ordered six-tuple as follows:

[0096] Path = (EN, EG, clps_e, PG, clps_p, OS)

[0097] Where: EN∈EdgeNodes; EG∈EdgeGateways; clps_e∈ContentLocationService_edge; PG∈ParentGateways; clps_p∈ContentLocationService_parent; OS∈OriginServers.

[0098] The above transmission process satisfies the Markov chain property (where X_n represents the state of the nth layer node):

[0099] P(X_{n+1}=x|X_n=x_n,...,X_0=x_0) = P(X_{n+1}=x|X_n=x_n).

[0100] Furthermore, based on the aforementioned transmission path, this application proposes a protocol merging trigger condition. When the same triplet (domain name + publication point + flow name) is detected to exist simultaneously in both FLV and P2P requests: for different protocol requests with the same domain name, publication point, and flow name, if both FLV and P2P requests exist simultaneously, the suffix is ​​modified to FLV during edge layer CLPS processing, returning to the gateway of this layer, thus merging with the existing FLV request for origin return. The origin return merging scheme for different protocols proposed in this application eliminates redundant traffic generated by multi-layer P2P hops, shortens the path by 2-3 hops after merging, and requires only one copy of the same content after protocol merging.

[0101] Optionally, the protocol anonymity conversion mechanism proposed in this application embodiment can be as follows: Figure 6 As shown, it specifically includes:

[0102] (1) Edge layer protocol normalization

[0103] The CLPS service dynamically converts P2P protocol (.xs suffix) requests to FLV format, unifying the origin path. The conversion process preserves the original streaming media parameters (such as bitrate and segment sequence number), only modifying the protocol identifier.

[0104] (2) Tor three-layer encrypted tunnel

[0105] Entry node (Guard Relay): Receives conversion instructions and encrypts target gateway information using RSA-2048;

[0106] Middle Relay: It uses X25519 key exchange for secondary encryption to remove the correlation between the entry node;

[0107] Exit Relay: Employs AES-256-GCM to encrypt FLV data returning to the source, hiding protocol conversion traces.

[0108] (3) Dynamic exit node scheduling

[0109] Dynamic selection of egress nodes for protocol conversion links serves two purposes. First, by periodically changing egress nodes (default every 60 seconds), the long-term correlation between traffic characteristics and user identity is severed, making it difficult for attackers to pinpoint targets through time-based correlation analysis and improving anonymity during the request process. Second, real-world testing shows that the average lifespan of static Tor egress IPs is only 2.3 hours, while dynamic scheduling can significantly improve availability. Furthermore, to avoid overloading a single node, requests are evenly distributed using a weighted algorithm (such as bandwidth / latency weighting).

[0110] The exit node is dynamically selected based on a comprehensive weight. The weight calculation formula is as follows:

[0111]

[0112] in, For bandwidth, For delay, For geographical distance, a node switch is forced every 60 seconds, the old node key is immediately destroyed, and a new node is selected based on the weight of the exit node.

[0113] In one exemplary embodiment, the method further includes:

[0114] The data back to the source is fragmented and transmitted based on a multi-path concurrent transmission optimization strategy and a dynamic fragmentation adjustment algorithm. The dynamic fragmentation adjustment algorithm is used to adjust the size and number of fragments of the data back to the source according to the real-time network conditions, thereby obtaining multiple data fragments. The multi-path concurrent transmission optimization strategy includes allocating an independent anonymous communication circuit for each data fragment, so that multiple data fragments can be transmitted simultaneously on different anonymous communication circuits.

[0115] For example, to improve data transmission efficiency and system robustness, embodiments of this application introduce a multi-path concurrent transmission optimization strategy. Specifically, refer to... Figure 7 When the edge layer receives data from the parent layer, it uses three independent Tor circuits to fragment and transmit the data. Each data fragment is assigned to a different Tor circuit (a three-hop encrypted link consisting of an ingress node, an intermediate node, and an egress node). Data received from the source station by the parent layer is transmitted in parallel to the edge layer through these circuits. At the edge layer, a data reassembly module reassembles the received data fragments into a complete data stream. Therefore, in scenarios where the edge layer uses three independent Tor circuits to transmit fragmented data, the fragment size and number of fragments are crucial, directly affecting transmission efficiency, data integrity, and resource utilization.

[0116] Furthermore, the embodiments of this application employ a dynamic fragmentation adjustment algorithm to adaptively adjust the fragment size and number based on real-time network conditions (such as latency, packet loss rate, bandwidth fluctuations, etc.) to optimize transmission efficiency and ensure high-reliability transmission even when the Tor network is unstable or subject to interference.

[0117] like Figure 8 As shown, the core logic of the dynamic sharding adjustment algorithm is as follows:

[0118] (1) Processing input parameters:

[0119] The sliding window averages of real-time collected bandwidth (B), latency (L), packet loss rate (P), and file size (F) are used as basic constraints.

[0120] (2) Calculation of slice size:

[0121]

[0122] (3) Calculation of the number of slices:

[0123] ⌈ ⌉

[0124] (4) Dynamic adjustment mechanism: The weighting coefficient is adjusted every 5 seconds based on the transmission effect. The code can be as follows:

[0125] if throughput < 0.8 * expected:

[0126] α *= 0.95 # Bias towards reducing fragmentation

[0127] if packet_loss > threshold:

[0128] β *= 1.05 # Bias towards increasing redundancy

[0129] Based on the above sharding adjustment and calculation strategies, the complete application process during the source retrieval is as follows:

[0130] (1) Data sharding:

[0131] The data file to be transmitted is divided into multiple data fragments according to the calculated fragment size. Each data fragment contains complete metadata and a header to ensure that it can be independently decoded and reassembled during transmission.

[0132] (2) Path allocation:

[0133] Each data segment is assigned an independent Tor circuit for transmission. A Tor circuit consists of an ingress node, intermediate nodes, and an egress node, ensuring the anonymity and security of data transmission. By transmitting data segments in parallel through multiple independent circuits, network bandwidth can be fully utilized, improving transmission efficiency.

[0134] (3) Concurrent transmission:

[0135] The concurrent transmission process is initiated, with multiple data slices being transmitted simultaneously on different Tor circuits. Edge nodes monitor the transmission status of each circuit, including transmission rate, latency, and packet loss rate, in order to adjust transmission strategies in a timely manner.

[0136] (4) Data restructuring:

[0137] At the receiving end, after the edge node receives all data fragments, it uses a data reassembly module to reassemble them into a complete data stream according to order and integrity requirements. Error detection and verification are performed during the reassembly process to ensure the accuracy and integrity of the data.

[0138] (5) Dynamic adjustment:

[0139] Based on real-time monitoring results, the size and number of fragments can be dynamically adjusted. For example, when network bandwidth increases, the fragment size can be appropriately increased to improve transmission efficiency; when packet loss rate rises, the fragment size can be decreased and the number of fragments increased to improve transmission robustness.

[0140] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0141] Based on the same inventive concept, this application also provides a CDN system merging and origin-pull-out device for implementing the CDN system merging and origin-pull-out method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more CDN system merging and origin-pull-out device embodiments provided below can be found in the limitations of the CDN system merging and origin-pull-out method described above, and will not be repeated here.

[0142] In one exemplary embodiment, such as Figure 9 As shown, a CDN system origin merging device 900 is provided, including: an origin merging module 901 and a path unification module 902, wherein:

[0143] The merge-origin module 901 is used to respond to a received streaming media request. If the streaming media requests have the same protocol, the streaming media requests are merged and returned to the origin. The hash key for merging and returning to the origin is obtained based on an anonymous communication network.

[0144] The path unification module 902 is used to unify the origin path of streaming media requests based on an anonymous processing mechanism that converts protocols if the streaming media requests have different protocols.

[0145] In one embodiment, the merging origin pull module 901 is further configured to: obtain the corresponding hash key for streaming media requests with the same domain name, publishing point, and stream name using a consistent hashing algorithm at the edge node layer based on an anonymous communication network; direct streaming media requests with the same hash key value to the same target edge node based on a preset hash mapping rule, and generate an aggregation request; and, in response to the aggregation request, select the optimal service process based on a dynamic weight allocation algorithm to perform the origin pull operation; wherein, the dynamic weight allocation algorithm is used to ensure that a single process carries the traffic of all streaming media requests with the same hash key value.

[0146] In one embodiment, the merged source module 901 is further configured to: obtain a first hash key based on the domain name, publication point, and stream name; and obtain a second hash key based on the first hash key and the node weighting factor of the anonymous communication network.

[0147] In one embodiment, the CDN system merging origin device 900 further includes: a virtual hash value acquisition module, used to generate multiple virtual nodes on each physical node and calculate the virtual hash value based on the second hash key.

[0148] In one embodiment, the path unification module 902 is further configured to: convert streaming media requests with different protocols that have the same domain name, publishing point, and stream name to unify the origin path; encrypt the protocol conversion based on multiple encrypted tunnels of the anonymous communication network; and dynamically select the exit node of the protocol conversion link based on a weight algorithm.

[0149] In one embodiment, the CDN system merging origin access device 900 further includes:

[0150] The fragmented transmission module is used to perform fragmented transmission of data returning to the source based on a multi-path concurrent transmission optimization strategy and a dynamic fragmentation adjustment algorithm. The dynamic fragmentation adjustment algorithm is used to adjust the size and number of fragments of the data returning to the source according to the real-time network conditions, thereby obtaining multiple data fragments. The multi-path concurrent transmission optimization strategy includes allocating an independent anonymous communication circuit for each data fragment, allowing multiple data fragments to be transmitted simultaneously on different anonymous communication circuits.

[0151] The modules in the aforementioned CDN system's origin merging device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0152] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores CDN system merge-to-origin data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a CDN system merge-to-origin method.

[0153] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0154] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0155] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0156] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A CDN system origin merging method, characterized in that, The method includes: In response to receiving a streaming media request, if the streaming media requests have the same protocol, the streaming media requests are merged and returned to the origin; wherein, the hash key of the merged and returned source is obtained based on an anonymous communication network; If the streaming media requests have different protocols, the origin path of the streaming media requests is unified based on the anonymous processing mechanism of protocol conversion.

2. The method according to claim 1, characterized in that, The step of merging and returning the streaming media requests to the origin if they have the same protocol includes: For streaming media requests with the same domain name, publishing point, and stream name using the same protocol, the corresponding hash key is obtained at the edge node layer based on the anonymous communication network using a consistent hashing algorithm. Based on preset hash mapping rules, streaming media requests with the same hash key value are directed to the same target edge node, and an aggregation request is generated; In response to the aggregation request, the optimal service process is selected based on a dynamic weight allocation algorithm to perform the origin pull operation; wherein, the dynamic weight allocation algorithm is used to ensure that a single process carries the traffic of all streaming media requests with the same hash key value.

3. The method according to claim 2, characterized in that, The step of obtaining the corresponding hash key at the edge node layer using a consistent hashing algorithm based on the anonymous communication network includes: Obtain the first hash key based on the domain name, publication point, and stream name; The second hash key is obtained based on the first hash key and the node weighting factor of the anonymous communication network.

4. The method according to claim 3, characterized in that, The method further includes: Multiple virtual nodes are generated on each physical node, and a virtual hash value is calculated based on the second hash key.

5. The method according to claim 1, characterized in that, If the streaming media requests have different protocols, a protocol-based anonymous processing mechanism is used to unify the origin-side path of the streaming media requests, including: Streaming media requests with the same domain name, publishing point, and stream name but different protocols are converted to unify the origin path; Encryption is used for protocol conversion based on multiple encrypted tunnels in an anonymous communication network; Based on a weighted algorithm, the exit node of the protocol conversion link is dynamically selected.

6. The method according to claim 1, characterized in that, The method further includes: The data back to the source is transmitted in fragments based on a multi-path concurrent transmission optimization strategy and a dynamic fragmentation adjustment algorithm. The dynamic fragmentation adjustment algorithm is used to adjust the size and number of fragments of the data back to the source according to the real-time network conditions to obtain multiple data fragments. The multi-path concurrent transmission optimization strategy includes: allocating an independent anonymous communication circuit for each data fragment, so that multiple data fragments can be transmitted simultaneously on different anonymous communication circuits.

7. A CDN system merging and back-to-origin device, characterized in that, The device includes: The merge-origin module is used to respond to a received streaming media request and, if the streaming media requests have the same protocol, merge the streaming media requests back to the origin; wherein, the hash key of the merge-origin is obtained based on an anonymous communication network; The path unification module is used to unify the origin path of streaming media requests based on an anonymous processing mechanism that performs protocol conversion if the streaming media requests have different protocols.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images