Feature extraction method and device and storage medium

Abstract

The embodiment of the invention provides a feature extraction method and device and a storage medium, relates to the technical field of communication, and is used for improving the comprehensiveness of feature extraction of a content distribution network. The method comprises the following steps: acquiring a service log of a content distribution network; performing feature extraction on the service log to obtain feature data of N time periods; n is a positive integer, the feature data of one time period comprises at least one of the following types: traffic feature data, cache content feature data and session feature data, and the feature data of one time period is used for representing the operation change condition of the content distribution network in the time period.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a feature extraction method, device and storage medium. Background Art

[0002] With the development of the digital economy, computing power, as a key production factor of the digital economy, has become the core support and driving force for tapping the value of data elements and promoting the development of the digital economy. From the perspective of computing power, artificial intelligence computing and edge computing have become important forces for market growth. As a platform technology, edge computing provides important carrying capacity for emerging technologies such as the fifth-generation mobile communication technology (5G), the Internet of Things, robots, and artificial intelligence. As the core basic application of edge computing, the content delivery network (CDN) is a platform-based infrastructure that provides network content caching and accelerated distribution services. It can provide caching and acceleration services for web pages and video media files to a large number of content providers (CP) in a multi-tenant shared service model. CDN is indispensable in various application scenarios of the future industrial Internet and computing power networks.

[0003] Currently, the comprehensiveness of feature extraction of content distribution networks is not high, and the extracted feature data cannot fully reflect the operational changes of content distribution networks. Summary of the invention

[0004] The present application provides a feature extraction method, device and storage medium for improving the comprehensiveness of feature extraction of a content distribution network.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a feature extraction method is provided, the method comprising:

[0007] Get the content distribution network's service logs;

[0008] Feature extraction is performed on the service log to obtain feature data of N time periods; N is a positive integer, and the feature data of one time period includes at least one of the following types: traffic feature data, cache content feature data, and session feature data. The feature data of one time period is used to characterize the operation changes of the content distribution network within the time period.

[0009] In a second aspect, a feature extraction device is provided, the device comprising:

[0010] An acquisition unit, used for acquiring a service log of a content distribution network;

[0011] The processing unit is used to extract features from the service log to obtain feature data of N time periods; N is a positive integer, and the feature data of one time period includes at least one of the following types: traffic feature data, cache content feature data, and session feature data. The feature data of one time period is used to characterize the operation changes of the content distribution network within the time period.

[0012] In a third aspect, a feature extraction device is provided, comprising: a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the feature extraction implements the method provided in the first aspect above.

[0013] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0014] According to a fifth aspect, a computer program product comprising computer instructions is provided. When the computer instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0015] In the embodiment of the present application, after feature extraction is performed on the service log of the content distribution network, feature data of N time periods are obtained. Since the feature data of one time period includes at least one of traffic feature data, cache content feature data, and session feature data, the comprehensiveness of feature extraction of the content distribution network is improved, thereby being able to fully reflect the operational changes of the content distribution network. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0017] Figure 1 A schematic diagram of the structure of a content distribution network provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a flow chart of a feature extraction method provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the composition of characteristic data of a time period provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the composition of a back-to-source bandwidth provided in an embodiment of the present application;

[0021] Figure 5A schematic diagram of a flow chart of another feature extraction method provided in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of a modeling data extraction process for a user behavior model provided in an embodiment of the present application;

[0023] Figure 7 A schematic diagram of a modeling data extraction process for a traffic model of a content distribution network cache node provided in an embodiment of the present application;

[0024] Figure 8 A schematic diagram summarizing characteristic data of a traffic model of a content distribution network cache node provided in an embodiment of the present application;

[0025] Fig. 9 A schematic diagram of displaying characteristic data provided by an embodiment of the present application;

[0026] Fig.10 A schematic diagram of the composition of a feature extraction device provided in an embodiment of the present application;

[0027] Fig.11 A schematic diagram of the structure of a feature extraction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0030] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0031] Content distribution network is a new network architecture added to the existing Internet protocol (IP) transmission network, which mainly includes distributed storage, load balancing, redirection of network requests and content management. Content management and global network traffic management are the core of CDN. CDN publishes the content of the website to the "edge" of the network closest to the user. By judging the proximity of the user and the server load, the user can obtain the required content nearby, solving the problems of network congestion and slow response speed of user access network caused by small network bandwidth, large user visits and uneven distribution of network points.

[0032] Due to the different business characteristics of each CP, the traffic sent by clients of different CPs to CDN cache nodes is also different, and the back-to-source traffic caused by cache misses of the requested content is also different. Therefore, it is necessary to extract features of the content distribution network to reflect the operational changes of the content distribution network.

[0033] However, the current feature extraction methods for content distribution networks mainly extract feature data related to traffic, which mainly includes the following items:

[0034] 1. The service traffic requested by users per unit time and the resulting cache misses, resulting in back-to-source traffic from obtaining file content from the upper-level CDN node (or CP source station).

[0035] 2. The ratio of service traffic (also called service bandwidth) to total traffic (also called total throughput bandwidth): bandwidth hit rate.

[0036] 3. The number of user requests and the number of back-to-origin requests per unit time.

[0037] 4. The ratio of service requests to total requests: request hit rate.

[0038] From the above content, it can be seen that the feature data extracted by the current feature extraction method is relatively small and not comprehensive. Many detailed feature data of user traffic are not taken into account, and it cannot be directly mapped to the hardware resource bottleneck of the content distribution network, and cannot fully reflect the operation changes of the content distribution network.

[0039] Based on this, the embodiments of the present application provide a feature extraction method, device and storage medium. After obtaining the service log of the content distribution network, feature extraction is performed on the service log to obtain feature data of N time periods. Since the feature data of a time period includes at least one of traffic feature data, cache content feature data, and session feature data, compared with the current feature extraction method, the types of extracted feature data are more comprehensive, which improves the comprehensiveness of the feature extraction of the content distribution network, thereby being able to fully reflect the operation changes of the content distribution network.

[0040] The scheme of the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0041] Figure 1 A schematic diagram of the structure of a content distribution network provided in an embodiment of the present application. Figure 1 As shown, the content distribution network 10 includes multiple CP nodes (eg, CP node 11, CP node 12), a CDN grouping node 21, a CDN cache node 31, and multiple CP clients (eg, CP client 41, CP client 42, CP client 43, and CP client 44).

[0042] The CP node is used to provide metadata for the content distribution network 10, that is, to provide content for the content distribution network 10 for use by users of the CP client. The CP node may also have other names, for example, it may also be called a CP source station.

[0043] In some embodiments, the CDN grouping node 21 is connected to multiple CP nodes and the CDN cache node 31 respectively, and is used to obtain the back-to-source traffic of the CDN cache node 31, and group the back-to-source traffic of the CDN cache node 31 and send it to different CP nodes. The CDN grouping node 21 can be an independent physical device, such as a server or a computer.

[0044] In some embodiments, the CDN cache node 31 is connected to multiple CP clients, and is used to cache content provided by multiple CP nodes and respond to requests from multiple CP clients.

[0045] The CDN cache node 31 can be an independent physical device, such as a server. It can also be a server cluster or distributed system composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data service networks. The CDN cache node 31 can also be called by other names, such as CDN nodes, CDN edge nodes, CDN cache service nodes, etc.

[0046] In some embodiments, the CDN grouping node 21 and the CDN cache node 31 may be integrated on the same physical device. In this case, the CDN cache node 31 may also have the function of grouping the back-to-source traffic.

[0047] In some embodiments, for one CP client among multiple CP clients (e.g., CP client 41), the CP client may be a device that provides interactive functions for users, such as a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc.

[0048] It should be understood that Figure 1 is an exemplary schematic diagram, Figure 1 The number of devices included in the content distribution network shown is not limited, for example, the number of CP nodes is not limited. Figure 1 In addition to the equipment shown, Figure 1 The content distribution network shown may also include other devices, which is not limited to this.

[0049] The execution subject of the feature extraction method provided in the embodiment of the present application is a feature extraction device. As an example, the feature extraction device can be the above-mentioned CDN cache node 31; or, the feature extraction device can also be a processor in the above-mentioned CDN cache node 31; or, the feature extraction device can also be an application (application, APP) installed in the above-mentioned CDN cache node 31 for executing the feature extraction method; or, the feature extraction device can also be a functional module with feature extraction function in the above-mentioned CDN cache node 31; or, the feature extraction device can also be a server with processing function connected to the above-mentioned CDN cache node 31. This application example does not limit this.

[0050] Next, if Figure 2 FIG. 1 is a flow chart of a feature extraction method provided in an embodiment of the present application, and the method comprises the following steps:

[0051] S101: Obtain service logs of a content distribution network.

[0052] The service log of the content distribution network is generated during the operation of the content distribution network.

[0053] In some embodiments, after the feature extraction device turns on the feature extraction function, it executes the feature extraction method provided in the embodiment of the present application. Correspondingly, after the feature extraction device turns off the feature extraction function, the feature extraction device does not execute or stops executing the feature extraction method provided in the embodiment of the present application.

[0054] In some embodiments, the feature extraction device obtains the service log of the content distribution network after starting the feature extraction function.

[0055] As a possible example, the feature extraction device turns on the feature extraction function by default.

[0056] As another possible example, the feature extraction device periodically turns on the feature extraction function. For example, the feature extraction device turns on the feature extraction function from 8:00 am to 8:00 pm, and turns off the feature extraction function from 8:00 pm to 8:00 am.

[0057] As another possible example, the feature extraction device determines to turn on / off the feature extraction function according to the user's instruction. For example, after receiving the user's instruction to turn on the feature extraction function, the feature extraction device turns on the feature extraction function in response to the instruction to obtain the service log of the content distribution network.

[0058] S102: Extract features from the service log to obtain feature data for N time periods.

[0059] Wherein, N is a positive integer.

[0060] In some embodiments, step S102 can be implemented as the following steps:

[0061] X1. Sort service logs based on time order.

[0062] The time sequence may be the time sequence of access by the CP client.

[0063] As an example, the service logs are sorted based on the time sequence, and the service logs can be sorted in positive order based on the time sequence of the CP client access to obtain the sorted service logs.

[0064] As another example, the service logs may be sorted based on the time sequence, or may be sorted in reverse order based on the time sequence of the CP client access, to obtain sorted service logs.

[0065] X2. Perform feature extraction on the sorted service logs to obtain feature data for N time periods.

[0066] The length of each time period may be preset by the network operation and maintenance personnel and may be any fixed time length. For example, a time period T may be 1 second, 1 minute, or 1 day, etc., without limitation.

[0067] It should be understood that the service logs of the content distribution network may be disordered. The service logs are sorted based on time sequence, and then feature extraction is performed on the sorted service logs to obtain feature data of N time periods, so that the feature data of each time period is ordered, thereby effectively reflecting the operation changes of the content distribution network within the time period.

[0068] In some embodiments, the characteristic data of a time period includes at least one of the following types: traffic characteristic data, cache content characteristic data, and session characteristic data. The characteristic data of a time period is used to characterize the operation changes of the content distribution network during the time period.

[0069] As an example, the traffic characteristic data includes at least one of the following types: service bandwidth, back-to-source bandwidth, total throughput bandwidth, back-to-source service ratio, bandwidth hit rate, request hit rate, number of service requests, number of back-to-source requests, average service fragment size, average back-to-source fragment size, new fragment rate, and cold fragment rate. Among them, the total throughput bandwidth is the sum of the service bandwidth and the back-to-source bandwidth; the back-to-source service ratio is the ratio of the back-to-source bandwidth to the service bandwidth; the bandwidth hit rate is the ratio of the service bandwidth to the total throughput bandwidth; the number of service requests is used to indicate the number of requests initiated by the client per unit time; the request hit rate is obtained according to the ratio of the number of back-to-source requests to the number of service requests; the average service fragment size is used to indicate the average size of each service request made by the client; the average back-to-source fragment size is used to indicate the average size of each back-to-source request made by the client; the request content size is used to indicate the size of the cache storage space occupied by the content requested by the user in the content distribution network.

[0070] As an example, the cache content feature data includes at least one of the following types: requested content size, content concentration. The requested content size is used to indicate the size of the cache storage space of the content distribution network occupied by the content requested by the user; the content concentration is obtained according to the ratio of the requested content size to the service bandwidth.

[0071] As an example, the session feature data includes at least one of the following types: the number of session transactions, the average number of shards per session, the average service traffic per session, and the average back-to-source traffic per session; wherein the number of session transactions is the number of client session transactions initiated per unit time, the average number of shards per session is the average number of shards accessed by each session, and the average service traffic per session is the average service traffic generated by each session.

[0072] Exemplarily, the characteristic data of one time period in N time periods may be as follows: Figure 3 As shown. Figure 3 It can be seen that for each of the N time periods, for example, the characteristic data of the time period TN may include traffic characteristic data, cache content characteristic data, and session characteristic data. Among them, the traffic characteristic data includes service bandwidth, back-to-source bandwidth, total throughput bandwidth, back-to-source service ratio, bandwidth hit rate, request hit rate, number of service requests, number of back-to-source requests, average service shard size, average back-to-source shard size, new shard rate, and cold shard rate. The cache content characteristic data includes the request content size and content concentration, and the session characteristic data includes the number of session transactions, the average number of shards per session, the average service traffic per session, and the average back-to-source traffic per session.

[0073] See also Figure 3 The back-to-source service ratio can be represented by the symbol γ, with the unit of %. The service bandwidth can be represented by the symbol x, with the unit of Gbps. The service bandwidth can also be called the service traffic requested per unit time, which is the network bandwidth resources of the CDN occupied by the service traffic requested by the user per unit time.

[0074] The back-to-source bandwidth can be represented by the symbol y, with the unit of Gbps. The back-to-source bandwidth can also be called the flow of back-to-source requests per unit time, or the back-to-source flow, which is used to represent the back-to-source flow generated by cache miss requests per unit time.

[0075] The new slice rate can be represented by ε new It is expressed in %, which is the ratio of the number of cache misses to the size of the requested content in the first access, and the new piece rate ε new Determined by the user behavior of the CP client, ε new ∈[0,1].

[0076] The cold film rate can be represented by the symbol ε cold It is expressed in %, which is the ratio of the size of cache misses to the size of requested content for non-first access, and the cold slice rate ε cold Determined by the user behavior of the CP client, the upper limit of the cold slice rate range is constrained by the joint constraints of the average number of session transactions and the cache capacity of the CDN cache node. Assuming that the cache capacity of the CDN cache node is D, the cold slice rate range can be expressed as It can be seen from this that when P>D, the value range of the cold film rate also tends to [0,1], and is independent of the cache capacity D of the CDN cache node. P is the total amount of content in the CP node.

[0077] It should be understood that the range of the cold slice rate is related to the size of the group content storage space: a larger group will have a higher probability of obtaining a lower cold slice rate under the same traffic and the same content aggregation, thereby improving the bandwidth hit rate. As time goes by, the average number of CP session transactions P tends to be much larger than the cache capacity D of the CDN cache node. At this time, the cold slice rate ε cold The value of will tend to be in the range of [0,1], and has nothing to do with the cache capacity D of the CDN cache node. As time goes by, the request content size m of the CP client always increases, while the cache capacity of the CDN cache node is stable in the long term, so it can be considered that the new film rate ε new and cold film rate ε cold It is entirely determined by the traffic characteristics of the CP and has nothing to do with the CDN.

[0078] For example, Figure 4 FIG. 1 is a schematic diagram of a composition of a back-to-source bandwidth provided in an embodiment of the present application. Figure 4 , the back-to-source bandwidth y can be split into new back-to-source y new And cold film back to source cold There are two parts. Among them, the new film returns to the source new and the new film rate ε new Related, cold film back to source cold and cold film rate ε cold Correlation can be understood as y = y new +y cold =mε new +mε cold It should be noted that the traffic for updating cache content to the superior when the cache expires is counted as the new film return source, and the traffic for only extending the expiration time without updating the cache content after update detection is ignored. Figure 4 The MISS in the code means miss.

[0079] Continue to see Figure 3 The requested content size can be represented by the symbol m, with the unit of MB / s. The requested content size can also be referred to as the total amount of content requested per unit time, which is used to indicate the size of the CDN cache storage space (also referred to as cache capacity) occupied by the content requested by the user.

[0080] The total throughput bandwidth can be represented by the symbol z, in Gbps. The total throughput bandwidth can also be called the total throughput bandwidth per unit time. The bandwidth hit rate can be represented by the symbol β, in %. The request hit rate can be represented by the symbol a, in %, which indicates the ratio of local hit requests (without back-to-source) to the number of service requests.

[0081] The number of service requests can be represented by the symbol u, and the unit is query per second (QPS). The number of service requests can also be called the number of client requests per unit time, which is used to represent the number of requests initiated by the client per unit time.

[0082] The average service segment size can be represented by the symbol a, in KB, which is used to represent the average size of each service request from the client. The average back-to-origin segment size can be represented by the symbol b, in KB, which is used to represent the average size of the back-to-origin request.

[0083] Content concentration can be represented by the symbol δ, with the unit of %. Content concentration can also be called content aggregation, which is used to represent the ratio of the requested content size to the service bandwidth per unit time.

[0084] The number of session transactions can be represented by the symbol s, and the unit is the number of transactions per second (TPS). The number of session transactions can also be called the number of client session transactions initiated per unit time, where a session transaction refers to multiple consecutive hypertext transfer protocol (HTTP) accesses initiated by a client to the same resource. Since the CDN cache node side usually cannot save the complete client session state, the number of session transactions can be approximated by the joint deduplication count of the interconnection protocol (terminal internet protocol, terminal-IP) and content identifier (cache-key) between client networks per unit time that meets the time order constraint.

[0085] The average number of transactions per session can be represented by the symbol p, with the unit of Q / T. The average number of transactions per session can also be called the average number of shards per session or the average number of requests per session, which is used to represent the average number of shards accessed by each session (including the number of requests). T is used to represent the time period. For example, if T is 1 second (S), it represents the average number of transactions per second in a single session, and Q is the total number of transactions in a single session.

[0086] The average service traffic of a single session can be represented by the symbol c, with the unit of MB / T. The average service traffic of a single session can also be called the average session service traffic, which is used to represent the average service traffic generated by a single session, where M represents the single session service traffic, B is bits, and T is the time period.

[0087] The average back-to-source traffic of a single session can be represented by the symbol d, and the unit is MB / T. When T is seconds, the unit of the average back-to-source traffic of a single session can be bps.

[0088] It should be noted that, in the above embodiments, the symbols corresponding to each type of characteristic data represent the corresponding characteristic data only for example, and those skilled in the art may change the symbols corresponding to each type of characteristic data according to the actual application scenario, and there is no limitation on this.

[0089] In some embodiments, each type of feature data in the feature data of N time periods is obtained from statistics in the service log and / or is determined based on a preset formula.

[0090] As an example, each type of characteristic data including service bandwidth, back-to-source bandwidth, number of service requests, number of back-to-source requests, average service segment size, average back-to-source segment size, request content size, number of session transactions, average number of transactions per session, average service traffic per session, and average back-to-source traffic per session is obtained from statistics of service logs.

[0091] As an example, each type of characteristic data in total throughput bandwidth, back-to-source service ratio, bandwidth hit rate, request hit rate, new film rate, cold film rate and content concentration is determined based on a preset formula.

[0092] For example, the total throughput bandwidth z=x+y, where x is the service bandwidth and y is the back-to-source bandwidth. Bandwidth Hit Ratio Request hit rate v is the number of back-to-origin requests, the new film rate m new The number of cache misses for the first access, the cold slice rate m cold The number of cache misses for non-first access, content concentration

[0093] In some embodiments, the back-to-source bandwidth, average back-to-source shard size, bandwidth hit rate, back-to-source service ratio, single-session average back-to-source traffic, single-session average service traffic, and single-session average transaction number can also be obtained based on a preset formula, for example, the back-to-source bandwidth Average back-to-origin shard size Or, b = avg(b n ). Among them, avg means average, b n Indicates the size of each back-to-origin request. Bandwidth hit rate Back-to-source service ratio The average back-to-source traffic per session is d = p(1-a)b, the average service traffic per session is c = p*a, and the average number of transactions per session is

[0094] based on Figure 2The illustrated embodiment brings at least the following beneficial effects: after feature extraction of the service log of the content distribution network, feature data of N time periods are obtained. Since the feature data of one time period includes at least one of the types such as traffic feature data, cache content feature data, and session feature data, the comprehensiveness of feature extraction of the content distribution network is improved, so that the operation changes of the content distribution network can be fully reflected. In addition, a feature extraction method provided in the embodiment of the present application provides a method for determining each type of feature data, which helps to improve the standardization of feature data and the accuracy of feature extraction of feature data.

[0095] In some embodiments, Figure 5 As shown, after step S102, the method may further include the following steps:

[0096] S201, extracting feature data of a target time period from feature data of N time periods.

[0097] In some embodiments, in order to better reflect the operation changes of the content distribution network, feature data of a target time period may be extracted from feature data of N time periods, wherein the target time period is the time period corresponding to the extreme value moment of the feature data of the target type.

[0098] In some embodiments, the characteristic data of the target type is any one of the following types: back-to-source service ratio, request hit rate, content concentration, and number of session transactions. The extreme value moment of the characteristic data of the target type is any one of the following: the maximum value moment of the back-to-source service ratio, the minimum value moment of the request hit rate, the minimum value moment of the content concentration, and the maximum value moment of the number of session transactions.

[0099] It should be understood that the target time period is the time period corresponding to the extreme moment of the characteristic data of the target type, and the extreme moment of the characteristic data of the target type can fully reflect the critical performance of each device in the content distribution network. Therefore, the characteristic data of the target time period is extracted from the characteristic data of N time periods, so that the user behavior model or the traffic model of the content distribution network cache node constructed based on at least one item of the characteristic data of the target time period can more accurately reflect the operation changes of the content distribution network.

[0100] S202: Building a user behavior model or a traffic model of a content distribution network cache node based on at least one item of the characteristic data of the target time period.

[0101] Among them, the user behavior model is used to analyze user behavior, and the traffic model of the content distribution network cache node is used to characterize the traffic changes of the content distribution network.

[0102] In some embodiments, the feature extraction device may construct a user behavior model based on feature data related to user behavior in the target time period. As an example, the feature extraction device may construct a user behavior model based on service bandwidth, request content size, number of service requests, request hit rate, content concentration, number of session transactions, average number of shards per session, and average service quality per session in the target time period.

[0103] For example, Figure 6 FIG. 1 is a schematic diagram of a process of extracting modeling data for a user behavior model provided in an embodiment of the present application. Figure 6 First, extract the user behavior characteristics, including determining the total service traffic requested in the time period, that is, determining the service bandwidth x, and then determining the total number of requests, that is, determining the number of service requests u, and then based on the service bandwidth x, the number of service requests u and the preset formula, determine the average service fragmentation number a, Further, the user behavior characteristics are drilled down to add content attributes, including determining the total amount of content requested, that is, determining the requested content size m, and then determining the content concentration δ based on the requested content size m, the service bandwidth x and a preset formula. Further, the user behavior characteristics are drilled down to add session attributes, including determining the total number of sessions initiated, that is, determining the number of session transactions s, and then determining the average number of transactions per session p based on the number of service requests u, the number of session transactions s and a preset formula. The average service flow c of a single session is determined according to the average number of transactions p of a single session and the average service slice size a, where c=p*a. In this way, the acquisition of modeling data of the user behavior model is completed.

[0104] In some embodiments, the feature extraction device can construct a traffic model of a content distribution network cache node based on feature data related to traffic changes in a target time period. As an example, the feature extraction device can construct a traffic model of a content distribution network cache node based on the back-to-source bandwidth, total throughput bandwidth, number of back-to-source requests, average back-to-source shard size, average service shard size, bandwidth hit rate, back-to-source service ratio, average back-to-source traffic per session, new shard rate, and cold shard rate in the target time period.

[0105] For example, Figure 7 FIG. 1 is a schematic diagram of a data extraction process for a traffic model of a content distribution network cache node provided by an embodiment of the present application. Figure 7First, extract basic traffic features, including determining the total throughput within the time period, that is, determining the total throughput bandwidth z. When determining the total throughput bandwidth z, it is necessary to determine how much back-to-source traffic is generated within the time period and how much service traffic is requested within the time period, that is, determine the back-to-source bandwidth y and the service bandwidth x. Then, based on the back-to-source bandwidth y, the service bandwidth x and the preset formula, determine the total throughput bandwidth z, z=x+y. After determining the total throughput bandwidth z, the bandwidth hit rate β can be determined based on the total throughput bandwidth z, the service bandwidth x and the preset formula. And based on the back-to-source bandwidth y, the service bandwidth x and the preset formula, the back-to-source service ratio γ is determined, Then, we can determine the total number of back-to-source requests, that is, the number of back-to-source requests v. Then, we can determine the average back-to-source shard size b based on the number of back-to-source requests v, the back-to-source bandwidth y, and the preset formula. It is also possible to determine the total number of service requests, that is, to determine the number of service requests u, and then determine the average service slice size a based on the number of service requests u, the service bandwidth x and a preset formula. Then, based on the number of back-to-origin requests v, the number of service requests u, and the preset formula, the request hit rate a is determined. In this way, the extraction of basic traffic features is completed. Then, the basic traffic features are drilled down to add content attributes, including determining the total amount of content requested, that is, determining the requested content size m, and then determining the content concentration δ based on the requested content size m, the service bandwidth x and the preset formula. Further, the basic traffic characteristics are drilled down to add session attributes, including determining the total number of sessions initiated, that is, determining the number of session transactions s, and then determining the average number of transactions per session p based on the number of service requests u, the number of session transactions s and a preset formula. Then, based on the average number of transactions per session p, the average service shard size a and the preset formula, the average service traffic c of a single session is determined, c = p*a; and based on the average number of transactions per session p, the average service shard size a, the average back-to-source shard size b and the preset formula, the average back-to-source traffic d of a single session is determined, d = p(1-a)b.

[0106] As an example, Figure 8 FIG. 1 is a schematic diagram showing a summary of characteristic data of a traffic model of a content distribution network cache node provided by an embodiment of the present application. Figure 8The characteristic data of the traffic model of the content distribution network cache node can be horizontally divided into traffic characteristic data, basic characteristic data, cache content characteristic data and session characteristic data. It can be vertically divided into service traffic, back-to-source traffic and total throughput traffic. Among them, the traffic characteristic data includes service bandwidth x, back-to-source bandwidth y, total throughput bandwidth z and bandwidth hit rate β, the basic characteristic parameters include the number of service requests u, average service fragment size a, back-to-source request number v, average back-to-source fragment size b, request hit rate a, and back-to-source service ratio γ, and the cache content characteristic data includes request content size m and content concentration δ. The session characteristic data includes the number of session transactions s, the average number of transactions per session p, the average service traffic per session c and the average back-to-source traffic d for the session.

[0107] As a possible example, taking the characteristic data of the target type as the back-to-source service ratio, after determining that the characteristic data of the target type is the back-to-source service ratio, the time period in which the maximum value of the back-to-source service ratio is located in N time periods can be determined, and then the time period in which the maximum value of the back-to-source service ratio is located in the N time periods can be used as the target time period, and then based on at least one item of the characteristic data of the target time period, a user behavior model or a traffic model of the content distribution network cache node is constructed. For example, assuming that the time period in which the maximum value of the back-to-source service ratio occurs is the second time period among N time periods, the second time period can be used as the target time period, and then based on the service bandwidth, request content size, number of service requests, request hit rate, content concentration, number of session transactions, average number of shards per session, and average service quality per session in the second time period, a user behavior model is constructed; based on the back-to-source bandwidth, total throughput bandwidth, number of back-to-source requests, average back-to-source shard size, average service shard size, bandwidth hit rate, back-to-source service ratio, average back-to-source traffic per session, new shard rate, and cold shard rate in the second time period, a traffic model of the content distribution network cache node is constructed.

[0108] It should be understood that the feature extraction method provided by the present application is more comprehensive than the feature extraction of the content distribution network in the related art, and the user behavior model constructed based on at least one item of the feature data of the target time period can more accurately, comprehensively and completely reflect the user behavior, and the traffic model of the content distribution network cache node constructed can more accurately, comprehensively and completely reflect the traffic changes of the content distribution network.

[0109] In some embodiments, the feature extraction device is a feature extraction device with a display function. After constructing the user behavior model and the traffic model of the content distribution network cache node, the feature extraction device can also display the user behavior model and the traffic model of the content distribution network cache node.

[0110] In some embodiments, after the feature extraction device extracts the feature data of the target time period from the feature data of N time periods, the feature extraction device may display all or part of the feature data of the target time period.

[0111] As an example, taking the target type's characteristic data as the back-to-source service ratio as an example, the characteristic extraction device can extract the service bandwidth, total throughput bandwidth and bandwidth hit rate in the time period corresponding to the maximum moment of the back-to-source service ratio, and then display the service bandwidth, total throughput bandwidth and bandwidth hit rate in the time period corresponding to the maximum moment of the back-to-source service ratio.

[0112] As another example, taking the request hit rate as the characteristic data of the target type, the feature extraction device can extract the number of service requests and the number of back-to-source requests in the time period corresponding to the minimum moment of the request hit rate, and then display the number of service requests and the number of back-to-source requests in the time period corresponding to the minimum moment of the request hit rate.

[0113] As another example, taking the characteristic data of the target type as content concentration, the characteristic extraction device can extract the service bandwidth and requested content size in the time period corresponding to the moment of the minimum content concentration, and then display the service bandwidth and requested content size in the time period corresponding to the moment of the minimum content concentration.

[0114] As an example, taking the target type's characteristic data as the number of session transactions, the feature extraction device can extract the service bandwidth, back-to-source bandwidth, and back-to-source service ratio in the time period corresponding to the maximum moment of the number of session transactions, and then display the service bandwidth, back-to-source bandwidth, and back-to-source service ratio in the time period corresponding to the maximum moment of the number of session transactions.

[0115] As another example, the feature extraction device can also extract the service bandwidth, total throughput bandwidth and bandwidth hit rate in the time period corresponding to the maximum moment of the back-to-source service ratio, as well as the number of service requests and the number of back-to-source requests in the time period corresponding to the minimum moment of the request hit rate, as well as the service bandwidth and request content size in the time period corresponding to the minimum moment of the content concentration, and the service bandwidth, back-to-source bandwidth and back-to-source service ratio in the time period corresponding to the maximum moment of the number of session transactions, and then display all the extracted feature data.

[0116] For example, Fig. 9 FIG. 1 is a schematic diagram showing a display of characteristic data provided by an embodiment of the present application. Fig. 9The feature data extracted from the time period corresponding to the extreme moment of the feature data of the target type are divided into three categories: traffic feature data, cache content feature data, and session feature data. The feature data corresponding to the maximum moment of the back-to-source service ratio include the back-to-source service ratio, service bandwidth, back-to-source bandwidth, total throughput bandwidth, and bandwidth hit rate. The feature data corresponding to the minimum moment of the request hit rate include the request hit rate, the number of service requests, and the number of back-to-source requests. The feature data corresponding to the minimum moment of the content concentration include content concentration, service bandwidth, and request content size. The feature data corresponding to the maximum moment of the number of session transactions include the number of session transactions, service bandwidth, back-to-source bandwidth, and back-to-source service ratio.

[0117] It should be understood that one or more feature data corresponding to the extreme value moment of the feature data of the target type can fully reflect the critical performance of each device in the content distribution network. The feature extraction device displays one or more feature data corresponding to the extreme value moment of the feature data of the target type, so that network operation and maintenance personnel can determine the operation status of the content distribution network according to the content displayed by the feature extraction device, and then quickly locate the problem and optimize the performance in a targeted manner, which helps to improve the efficiency of locating problems in the content distribution network and the accuracy of performance optimization.

[0118] The above mainly introduces the solution provided by the present application from the perspective of the method. It can be understood that in order to realize the above functions, the feature extraction device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0119] Fig.10 FIG. 1 is a schematic diagram showing the composition of a feature extraction device provided in an embodiment of the present application. Fig.10 As shown, the feature extraction device 30 includes an acquisition unit 301 and a processing unit 302. In some embodiments, the feature extraction device 30 may further include a display unit 303.

[0120] The feature extraction device 30 may be the CDN cache node or a chip in the CDN cache node. When the feature extraction device 30 is used to implement the function of the CDN cache node in the above embodiment, each unit is specifically used to implement the following functions.

[0121] The acquisition unit 301 is used to acquire a service log of a content distribution network.

[0122] The processing unit 302 is used to extract features from the service log to obtain feature data of N time periods; N is a positive integer, and the feature data of one time period includes at least one of the following types: traffic feature data, cache content feature data, and session feature data. The feature data of one time period is used to characterize the operation changes of the content distribution network within the time period.

[0123] In some embodiments, the traffic feature data includes at least one of the following types: service bandwidth, back-to-source bandwidth, total throughput bandwidth, back-to-source service ratio, bandwidth hit rate, request hit rate, number of service requests, number of back-to-source requests, average service shard size, average back-to-source shard size, new shard rate, and cold shard rate.

[0124] In some embodiments, the cache content feature data includes at least one of the following types: requested content size, content concentration; the requested content size is used to indicate the size of the cache storage space of the content distribution network occupied by the content requested by the user; the content concentration is obtained based on the ratio of the requested content size to the service bandwidth.

[0125] In some embodiments, the session feature data includes at least one of the following types: the number of session transactions, the average number of shards per session, the average service traffic per session, and the average back-to-source traffic per session; the number of session transactions is the number of client session transactions initiated per unit time, the average number of shards per session is the average number of shards accessed by each session, and the average service traffic per session is the average service traffic generated by each session.

[0126] In some embodiments, the processing unit 302 is further configured to: extract feature data of a target time period from the feature data of N time periods, where the target time period is a time period corresponding to an extreme value moment of the feature data of the target type;

[0127] Based on at least one item of the characteristic data of the target time period, a user behavior model or a traffic model of a content distribution network cache node is constructed. The user behavior model is used to analyze user behavior, and the traffic model of a content distribution network cache node is used to characterize traffic changes in the content distribution network.

[0128] In some embodiments, the characteristic data of the target type is any one of the following types: back-to-source service ratio, request hit rate, content concentration, and number of session transactions; wherein the back-to-source service ratio is the ratio of the back-to-source bandwidth to the service bandwidth; the request hit rate is obtained according to the ratio of the number of back-to-source requests to the number of service requests; the content concentration is obtained according to the ratio of the request content size to the service bandwidth; the number of session transactions is the number of client session transactions initiated per unit time;

[0129] The extreme value moment of the characteristic data of the target type is any one of the following: the maximum value moment of the back-to-source service ratio, the minimum value moment of the request hit rate, the minimum value moment of the content concentration, and the maximum value moment of the number of session transactions.

[0130] In some embodiments, the processing unit 302 is specifically used to build a user behavior model based on the service bandwidth, request content size, number of service requests, request hit rate, content concentration, number of session transactions, average number of shards per session, and average service quality per session in a target time period.

[0131] In some embodiments, the processing unit 302 is specifically used to build a traffic model of the content distribution network cache node based on the back-to-source bandwidth, total throughput bandwidth, number of back-to-source requests, average back-to-source shard size, average service shard size, bandwidth hit rate, back-to-source service ratio, average back-to-source traffic per session, new shard rate, and cold shard rate of the target time period.

[0132] In some embodiments, the processing unit 302 is specifically used to: sort the service logs based on the time sequence; and extract features from the sorted service logs to obtain feature data of N time periods.

[0133] In some embodiments, each type of feature data in the feature data of N time periods is obtained from statistics in the service log and / or is determined based on a preset formula.

[0134] In some embodiments, the display unit 303 is used to display the user behavior model and the traffic model of the content distribution network cache node.

[0135] It should be noted that Fig.10 The units in can also be called modules, for example, the acquisition unit can be called an acquisition module. Fig.10 In the illustrated embodiment, the names of the various units may not be the names shown in the figure. For example, the acquisition unit may also be called a communication unit.

[0136] Fig.10If each unit in the embodiment is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The storage medium for storing computer software products includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0137] When the feature extraction device 30 implements the functions of the above-mentioned integrated modules in the form of hardware, the present application embodiment provides a structural schematic diagram of the feature extraction device. Fig.11 As shown, the feature extraction device 40 includes: a processor 402, a communication interface 403, and a bus 404. Optionally, the feature extraction device 40 may also include a memory 401.

[0138] The processor 402 may be a processor that implements or executes various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present application. The processor 402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present application. The processor 402 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0139] The communication interface 403 is used to connect with other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0140] The memory 401 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0141] As a possible implementation, the memory 401 may exist independently of the processor 402, and the memory 401 may be connected to the processor 402 via a bus 404 for storing instructions or program codes. When the processor 402 calls and executes the instructions or program codes stored in the memory 401, the feature extraction method provided in the embodiment of the present application can be implemented.

[0142] In another possible implementation, the memory 401 may also be integrated with the processor 402 .

[0143] The bus 404 may be an extended industry standard architecture (EISA) bus, etc. The bus 404 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0144] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal device can be divided into different functional modules to complete all or part of the functions described above.

[0145] The embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by computer instructions to instruct the relevant hardware, and the program can be stored in the above computer-readable storage medium. When the program is executed, it may include the processes of the above method embodiments. The computer-readable storage medium can be any of the above embodiments or memory. The above computer-readable storage medium can also be an external storage device of the above CDN cache node, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the above CDN cache node. Further, the above computer-readable storage medium can also include both the internal storage unit of the above CDN cache node and an external storage device. The above computer-readable storage medium is used to store the above computer program and other programs and data required by the above CDN cache node. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0146] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program product is run on a computer, the computer is enabled to execute any one of the feature extraction methods provided in the above embodiments.

[0147] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0148] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A feature extraction method, characterized in that: include: Get the content distribution network's service logs; Feature extraction is performed on the service log to obtain feature data of N time periods; N is a positive integer, and the feature data of one time period includes at least one of the following types: traffic feature data, cache content feature data, and session feature data. The feature data of one time period is used to characterize the operation changes of the content distribution network within the time period.

2. The method according to claim 1, characterized in that The traffic feature data includes at least one of the following types: service bandwidth, back-to-source bandwidth, total throughput bandwidth, back-to-source service ratio, bandwidth hit rate, request hit rate, number of service requests, number of back-to-source requests, average service fragment size, average back-to-source fragment size, new fragment rate, and cold fragment rate.

3. The method according to claim 1, characterized in that The cache content feature data includes at least one of the following types: requested content size, content concentration; The requested content size is used to indicate the size of the cache storage space of the content distribution network occupied by the content requested by the user; the content concentration is obtained according to the ratio of the requested content size to the service bandwidth.

4. The method according to claim 1, characterized in that: The session characteristic data includes at least one of the following types: number of session transactions, average number of shards per session, average service traffic per session, and average back-to-source traffic per session; The number of session transactions is the number of client session transactions initiated per unit time, the average number of shards per session is the average number of shards accessed by each session, and the average service flow per session is the average service flow generated by each session.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Extracting feature data of a target time period from the feature data of the N time periods, the target time period being a time period corresponding to an extreme value moment of the feature data of a target type; Based on at least one item of the characteristic data of the target time period, a user behavior model or a traffic model of a content distribution network cache node is constructed, wherein the user behavior model is used to analyze user behavior, and the traffic model of the content distribution network cache node is used to characterize traffic changes in the content distribution network.

6. The method according to claim 5, characterized in that The characteristic data of the target type is any one of the following types: back-to-source service ratio, request hit rate, content concentration, and number of session transactions; wherein the back-to-source service ratio is the ratio of back-to-source bandwidth to service bandwidth; the request hit rate is obtained according to the ratio of the number of back-to-source requests to the number of service requests; the content concentration is obtained according to the ratio of the request content size to the service bandwidth; the number of session transactions is the number of client session transactions initiated per unit time; The extreme value moment of the characteristic data of the target type is any one of the following: the maximum value moment of the back-to-source service ratio, the minimum value moment of the request hit rate, the minimum value moment of the content concentration, and the maximum value moment of the number of session transactions.

7. The method according to claim 6, characterized in that The constructing of a user behavior model based on at least one item of the characteristic data of the target time period includes: The user behavior model is constructed based on the service bandwidth, request content size, number of service requests, request hit rate, content concentration, number of session transactions, average number of shards per session, and average service quality per session in the target time period.

8. The method according to claim 6, characterized in that The constructing a traffic model of a content distribution network cache node based on at least one item of the characteristic data of the target time period includes: Based on the back-to-source bandwidth, total throughput bandwidth, number of back-to-source requests, average back-to-source shard size, average service shard size, bandwidth hit rate, back-to-source service ratio, average back-to-source traffic per session, new slice rate, and cold slice rate of the target time period, a traffic model of the content distribution network cache node is constructed.

9. The method according to claim 1, characterized in that: The feature extraction of the service log to obtain feature data of N time periods includes: Sorting the service logs based on chronological order; Feature extraction is performed on the sorted service logs to obtain feature data of the N time periods.

10. The method according to claim 1, characterized in that Each type of characteristic data in the characteristic data of the N time periods is obtained from statistics of the service log and / or is determined based on a preset formula.

11. A feature extraction device, characterized in that: include: Processor and memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the feature extraction device implements the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 10.

Images