Log persistence method and apparatus for CDN, medium, and electronic device

Abstract

The present disclosure relates to a log persistence method and apparatus for a CDN, a medium, and an electronic device, which ensure high-performance log persistence without affecting the data throughput and performance of a system that generates logs. The log persistence method for a CDN comprises: transmitting, to a predetermined buffer, log information generated by operation of a system applied to a CDN, wherein the predetermined buffer is decoupled from the system applied to the CDN; reading the log information from the predetermined buffer; and writing the read log information into a disk.

Description

Method, device, medium, and electronic device for CDN log storage

[0001] This application claims priority to Chinese Patent Application No. 202410115507.5 filed on January 26, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The present disclosure relates to a method, device, medium, and electronic device for storing logs on a CDN. Background Art

[0003] A Content Delivery Network (CDN) is a distributed network built on top of the existing network to speed up network access. It relies on server clusters deployed at edge nodes around the world. Through load balancing, content publishing, content management, and content storage, the CDN server cluster shares the pressure of the source site server cluster, allowing users to obtain cached access resources nearby, avoiding network congestion and speeding up access.

[0004] The Domain Name System (DNS) used in CDNs typically uses the User Datagram Protocol (UDP) as a transport layer protocol. UDP servers are vulnerable to attacks, so the DNS service software on the data plane of a DNS system typically requires high performance. Common examples of high-performance software include the Data Plane Development Kit (DPDK) and the eXpress Data Path (XDP). These high-performance software packages can typically achieve a UDP DNS system throughput of 10 million queries per second (QPS). Such a high-performance data plane system can protect against UDP traffic attacks.

[0005] DNS logging systems are systems that need to record DNS request logs. These logs typically need to be stored on disk for troubleshooting and data analysis. Due to the performance limitations of disk input / output (I / O) interfaces, DNS logging systems are often unable to keep up with the throughput of DNS systems. Enabling DNS logging may result in a decrease in DNS system throughput. Summary of the Invention

[0006] This section is provided to briefly introduce the concepts that will be described in detail in the detailed description below. This section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] In a first aspect, the present disclosure provides a log storage method for a CDN, comprising: transmitting log information generated by the operation of a system applied to the CDN to a predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; reading the log information from the predetermined buffer; and writing the read log information to a disk.

[0008] In a second aspect, the present disclosure provides a log-writing device for CDN, comprising a transmission module, a predetermined buffer, and a log-writing module, wherein: the transmission module is used to transmit log information generated by the operation of a system applied to the CDN to the predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; the predetermined buffer is used to cache the transmitted log information; and the log-writing module is used to read the log information from the predetermined buffer and write the read log information to the disk.

[0009] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the log storage method for CDN described in any one of the first aspects.

[0010] In a fourth aspect, the present disclosure provides an electronic device comprising: a storage device storing at least one computer program; and at least one processing device for executing the at least one computer program in the storage device to implement the steps of the log storage method for CDN described in any one of the first aspects.

[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0013] FIG1 is a flowchart of a method for storing logs on a CDN according to an embodiment of the present disclosure.

[0014] FIG2 is a flowchart of another method for storing logs on a CDN according to an embodiment of the present disclosure.

[0015] FIG3 is a flowchart of a method for writing cached log information to a disk together when the cached log information meets a preset writing condition according to an embodiment of the present disclosure.

[0016] FIG4 is a schematic block diagram of a log storage device for CDN according to an embodiment of the present disclosure.

[0017] FIG5 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0019] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0020] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] All actions of acquiring signals, information or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0025] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0026] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0029] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0030] Figure 1 is a flow chart of a method for storing logs on a CDN according to an embodiment of the present disclosure. This method is applicable to storing log information generated by various systems used in the CDN, such as the DNS system used in the CDN.

[0031] As shown in FIG1 , the log storage method for CDN according to an embodiment of the present disclosure may include the following steps S11 to S13 .

[0032] In step S11 , log information generated by the operation of the system applied to the CDN is transmitted to a predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN.

[0033] Log information refers to the log information generated by the system that needs to record the log. Taking the DNS system used in CDN as an example, the log information can be the log information generated by the DNS system during operation.

[0034] A system applied to a CDN can be a component of the CDN, or it can be used to provide bypass services (e.g., traffic scheduling) to the CDN. For example, a DNS system can be a component of the CDN, or it can be used to provide domain name scheduling as a bypass service to the CDN.

[0035] Decoupling means that the predetermined buffer and the system applied to the CDN may be located in different physical locations, or although the predetermined buffer and the system applied to the CDN are located in the same physical location, the two share memory.

[0036] Furthermore, the predetermined buffer area may be a buffer area located at any physical location.

[0037] Log information can be transferred to a predetermined buffer in a multi-threaded manner.

[0038] In addition, log information can be transmitted to a predetermined buffer asynchronously. That is, the transmission of log information to the predetermined buffer is an asynchronous non-blocking operation. After the log information is transmitted, the CDN system will immediately enter the process of processing other messages.

[0039] In some embodiments, the log information generated by the operation of the system applied to the CDN is transmitted to the predetermined buffer, which can be achieved as follows. First, the log information is encoded using the buffer protocol, and then the encoded log information is transmitted to the predetermined buffer. For example, the encoded log information can be transmitted to the predetermined buffer in a multi-threaded manner. By encoding the log information using the buffer protocol, structured data can be serialized, and it is independent of the language and platform. By transmitting the encoded log information to the predetermined buffer in a multi-threaded manner, the encoded log information can be quickly transmitted to the predetermined buffer.

[0040] In some embodiments, transmitting log information generated by the operation of a CDN system to a predetermined buffer can include transmitting the log information to the predetermined buffer in the form of a lock-free queue. For example, the encoded log information mentioned above can be transmitted to the predetermined buffer in the form of a lock-free queue, and the transmission can be performed in a multi-threaded manner. For example, if the system applied to the CDN is a DNS system, since the DNS system generally uses multi-threaded high-performance software for data processing, the DNS system as a whole is a multi-producer and multi-consumer system. Therefore, transmitting the log information generated by the DNS system to the predetermined buffer in the form of a lock-free queue can improve transmission efficiency.

[0041] In step S12, log information is read from a predetermined buffer.

[0042] In some embodiments, log information stored in a predetermined buffer can be read from the predetermined buffer in a multi-threaded manner, which can speed up the reading of log information, enable the log information to be quickly written to the disk, and improve the efficiency and performance of writing to the disk.

[0043] In step S13, the read log information is written into the disk.

[0044] By adopting the above technical solution, the log information generated by the operation of the CDN system can be transmitted to a predetermined buffer decoupled from the CDN system that generates the log information, the log information can be read from the predetermined buffer, and the read log information can be written to the disk. In this way, the execution process of the CDN system that generates the log information can be decoupled from the disk-writing process of the generated log information. That is, the disk-writing operation of the generated log information will not affect the execution process of the CDN system that generates the log information. Therefore, the disk-writing operation will not block the execution process of the CDN system that generates the log information, and will not affect the data throughput (for example, query rate per second) and performance of the CDN system that generates the log information (that is, the main processing flow of the CDN system itself remains unchanged), and high-performance log-writing is also ensured. Taking the DNS system as an example, a DNS system with a QPS performance of 10 million in the related art usually only has tens of thousands of QPS due to the drag of existing log storage technology. However, by adopting the log storage method for CDN according to the embodiment of the present disclosure, the QPS performance will not be affected, and the DNS system can continue to use high-performance service frameworks such as DPDK and XDP for network acceleration. Moreover, the main processing flow of the DNS system itself remains unchanged.

[0045] Figure 2 is a flow chart of another method for storing logs on a CDN according to an embodiment of the present disclosure. As shown in Figure 2, the method for storing logs on a CDN may include the following steps S21 to S24.

[0046] In step S21, log information generated by the operation of the CDN system is transmitted to a predetermined buffer, wherein the predetermined buffer is decoupled from the CDN system. The implementation of the operation performed in step S21 has been described in detail in step S11 above and will not be repeated here.

[0047] In step S22, the log information is read from the predetermined buffer. The implementation of the operation performed in step S22 has been described in detail in step S12 described above and will not be repeated here.

[0048] In step S23, the read log information is cached.

[0049] Here, the buffer area for caching the read log information is a different buffer area from the aforementioned preset buffer area.

[0050] In step S24 , when the cached log information meets a preset writing condition, the cached log information is written to the disk together.

[0051] By adopting the above technical solution, log information generated by the operation of the CDN system can be transmitted to a predetermined buffer decoupled from the CDN system generating the log information, the log information can be read from the predetermined buffer, the read log information can be cached, and when the cached log information meets a preset write condition, the cached log information can be written to the disk together. In this way, on the one hand, the execution process of the CDN system generating the log information can be decoupled from the disk writing operation of the generated log information. That is, the disk writing operation of the generated log information will not affect the execution process of the CDN system generating the log information. Therefore, the disk writing operation will not block the execution process of the CDN system generating the log information, and will not affect the data throughput (for example, query rate per second) and performance of the CDN system generating the log information (that is, the main processing flow of the CDN system generating the log information itself remains unchanged). On the other hand, the log information can be written to the disk in batches, realizing batch disk writing of the log information and ensuring high-performance log disk writing.

[0052] FIG3 is a flowchart of a method for writing cached log information to a disk when the cached log information meets a preset write condition according to an embodiment of the present disclosure. As shown in FIG3 , the method may include the following steps S241 to S243 .

[0053] In step S241 , during a preset period, it is monitored whether the number of cached log information reaches a preset number.

[0054] In step S242 , if the number of cached log information items reaches a preset number during the preset period, the cached log information items are written to the disk together after the number of cached log information items reaches the preset number.

[0055] For example, the preset number of log messages may be N. If, during a preset period, that is, at any time within the preset period, the number of cached log messages reaches N, then these N cached log messages are merged and written to the disk together, regardless of whether the preset period has expired. In other words, as long as the cached log messages reach the preset number, even if the preset period has not expired, the preset number of cached log messages will be written to the disk together, without having to wait until the preset period expires to write.

[0056] In step S243 , if the number of cached log information does not reach the preset number when the preset period expires, the cached log information is written to the disk together when the preset period expires.

[0057] In some scenarios, the amount of log information generated may be very small. In this case, if the write is still performed when the cached log information reaches the preset number, the log information will not be written to the disk in time. However, through the technical solution according to the embodiment of the present disclosure, the writing of log information to the disk can be triggered regularly. That is, even if the actual amount of log information generated is very small, if the time for regular log writing to the disk arrives, the log will still be written to the disk, thereby ensuring that the generated log information is written to the disk in time.

[0058] The above technical solution can, on the one hand, realize batch storage of log information to disk and improve storage performance; on the other hand, ensure timely storage of log information when the actual amount of log information generated is small.

[0059] In some embodiments, the log information is generated by a domain name system used in a CDN. Therefore, before transferring the log information generated by the operation of the CDN system to a predetermined buffer, the log storage method for a CDN according to an embodiment of the present disclosure may further include: determining the current stage of the domain name system; extracting information that needs to be recorded in log form for the current stage of the domain name system; and inserting the extracted information into the log information in a predetermined format.

[0060] The predetermined format can be any predefined log information storage format. For example, the following is an exemplary predetermined format:

[0061] That is, based on the current stage of the domain name system used in the CDN, information related to socket_family and query_address can be extracted and added to the log information in the preset format. In the above exemplary preset format, the "1" in "socket_family=1" and the "3" in "query_address=3" are the relevant information extracted and added to the log information based on the current stage of the domain name system used in the CDN.

[0062] In addition, extracting information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN may include at least one of the following:

[0063] If the domain name system applied to the CDN is currently in the DNS request phase, extract at least one of the following as information that needs to be recorded in log form: the source address of the DNS request (query_address & socket_family), the request port of the DNS request (query_port), the request time of the DNS request (query_time_sec), the size of the data packet requested by the DNS request (query_size), the requested domain name of the DNS request (qname), the request type of the DNS request (qtype), the request class of the DNS request (qclass), and the address of the Elastic Compute Service (ECS) requested by the DNS request (ecs_address & ecs_socket_family);

[0064] If the domain name system applied to the CDN is currently in the query phase of the DNS request, extract at least one of the following as information to be recorded in log form: the query matching zone (query_zone) of the DNS response, the client IP line information (geo_info), and the request matching line (line);

[0065] If the domain name system applied to the CDN is currently in the DNS response phase, at least one of the following is extracted as information that needs to be recorded in the form of a log: a response status code (rcode) and answers.

[0066] By adopting the above technical solution, when the domain name system applied to the CDN has logs to be recorded, it is possible to extract information such as the client IP, DNS extension protocol (Extension Mechanisms for DNS, EDNS) IP, DNS request type, DNS response type, DNS response content, DNS query / matching line, etc., and fill the extracted information into the log information in a preset format to form log information that needs to be transmitted to a preset buffer.

[0067] In some embodiments, the log storage method for CDN according to the embodiments of the present disclosure may further include: determining whether a predetermined buffer is full; if it is determined that the predetermined buffer is full, discarding newly generated log information.

[0068] By adopting the above technical solution, when the generated log information is transmitted to the predetermined buffer, if the predetermined buffer is full and the log information cannot be further transmitted to the predetermined buffer, the newly generated log information can be directly discarded without waiting for storage space to appear in the predetermined buffer. In other words, the main process of the system applied to the CDN that generates the log information is not blocked, thereby ensuring the data throughput of the system applied to the CDN that generates the log information (for example, the DNS system applied to the CDN), and avoiding the impact of log storage on the main process service of the system applied to the CDN that generates the log information.

[0069] Figure 4 is a schematic block diagram of a log storage device for a CDN according to an embodiment of the present disclosure. As shown in Figure 4, the log storage device for a CDN according to an embodiment of the present disclosure includes a transmission module 41, a predetermined buffer 42, and a log storage module 43. The transmission module 41 is configured to transmit log information generated by the operation of a CDN system to the predetermined buffer 42, wherein the predetermined buffer 42 is decoupled from the CDN system; the predetermined buffer 42 is configured to cache the transmitted log information; and the log storage module 43 is configured to read the log information from the predetermined buffer 42 and write the read log information to disk.

[0070] By adopting the above technical solution, the log information generated by the operation of the CDN system can be transmitted to a predetermined buffer decoupled from the CDN system generating the log information, the log information can be read from the predetermined buffer, and the read log information can be written to the disk. In this way, the execution process of the CDN system generating the log information can be decoupled from the disk-writing operation of the generated log information. That is, the disk-writing operation of the generated log information will not affect the execution process of the CDN system generating the log information. Therefore, the disk-writing operation will not block the execution of the CDN system generating the log information, and will not affect the data throughput (for example, query rate per second) and performance of the CDN system generating the log information (that is, the main processing flow of the CDN system generating the log information itself remains unchanged), and high-performance log-writing can be ensured.

[0071] Optionally, the transmission module 41 transmits log information generated by the operation of the CDN system to a predetermined buffer, including: encoding the log information using a buffer protocol; and transmitting the encoded log information to the predetermined buffer.

[0072] Optionally, the transmission module 41 transmits the log information generated by the operation of the system applied to the CDN to a predetermined buffer, including: transmitting the log information to the predetermined buffer in the form of a lock-free queue.

[0073] Optionally, the log writing module 43 writes the read log information to the disk, including: caching the read log information; and when the cached log information meets a preset write condition, writing the cached log information to the disk. For example, a buffer zone is provided in the log writing module 43 to cache the log information read from the predetermined buffer zone 42 by the log writing module 43.

[0074] Optionally, when the cached log information meets a preset writing condition, the log writing module 43 writes the cached log information to the disk, including:

[0075] During a preset period, monitoring whether the number of cached log information has reached a preset number;

[0076] If the number of cached log information during the preset period reaches the preset number, after the number of cached log information reaches the preset number, the cached log information is written to the disk together;

[0077] If the number of cached log information does not reach the preset number when the preset period expires, the cached log information is written to the disk together when the preset period expires.

[0078] Optionally, the log information is log information generated by a domain name system applied to a CDN, and the log storage device for the CDN further includes an extraction module, which is used to: determine the current stage of the domain name system applied to the CDN; extract information that needs to be recorded in the form of a log at the current stage of the domain name system applied to the CDN; and fill the extracted information into the log information in a predetermined format before the transmission module 41 transmits the log information generated by the operation of the domain name system applied to the CDN to a predetermined buffer.

[0079] Optionally, the extraction module extracts information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN in the following manner:

[0080] If the domain name system applied to the CDN is currently in the DNS request stage, extract at least one of the following as information that needs to be recorded in the form of a log: the source address of the DNS request, the request port of the DNS request, the request time of the DNS request, the size of the data packet requested by the DNS request, the requested domain name of the DNS request, the request type of the DNS request, the request category of the DNS request, and the address of the request ECS of the DNS request;

[0081] If the domain name system applied to the CDN is currently in the query phase of the DNS request, extract at least one of the following as information that needs to be recorded in the form of a log: the query matching zone of the DNS response, the client IP line information, and the request matching line;

[0082] If the domain name system applied to the CDN is currently in the DNS response stage, at least one of the following is extracted as information that needs to be recorded in the form of a log: a response status code, and a reply.

[0083] Optionally, the transmission module 41 is further configured to: determine whether the predetermined buffer is full; and if it is determined that the predetermined buffer is full, discard the newly generated log information.

[0084] The specific implementation of the operations performed by each module in the log storage device for CDN according to the embodiment of the present disclosure has been described in detail in the log storage method for CDN, and will not be repeated here.

[0085] The present disclosure further provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of any one of the log storage methods for a CDN in the present disclosure.

[0086] The present disclosure also provides an electronic device, comprising: a storage device storing at least one computer program; and at least one processing device for executing at least one computer program in the storage device to implement any step of the log storage method for CDN in the present disclosure.

[0087] Reference is now made to FIG5 , which illustrates a schematic diagram of the structure of an electronic device 600 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device illustrated in FIG5 is merely an example and should not limit the functionality or scope of use of embodiments of the present disclosure.

[0088] As shown in Figure 5, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0089] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 5 shows the electronic device 600 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may be implemented or present instead.

[0090] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the log storage method for CDN in the embodiment of the present disclosure are performed.

[0091] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0092] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0093] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0094] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device is caused to: transmit log information generated by the operation of the system applied to the CDN to a predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; read the log information from the predetermined buffer; and write the read log information to a disk.

[0095] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not limit the module itself. For example, a transmission module may be described as "a module for transmitting log information generated by the operation of a CDN system to the predetermined buffer."

[0098] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0099] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] According to one or more embodiments of the present disclosure, Example 1 provides a log storage method for a CDN, comprising: transmitting log information generated by the operation of a system applied to the CDN to a predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; reading the log information from the predetermined buffer; and writing the read log information to a disk.

[0101] According to one or more embodiments of the present disclosure, Example 2 provides the log storage method for a CDN of Example 1, wherein the method of transferring log information generated by the operation of a system applied to the CDN to a predetermined buffer includes:

[0102] Encoding the log information using a buffer protocol;

[0103] The encoded log information is transmitted to the predetermined buffer zone.

[0104] According to one or more embodiments of the present disclosure, Example 3 provides the log storage method for a CDN of Example 1, wherein the method of transferring log information generated by the operation of a system applied to the CDN to a predetermined buffer includes:

[0105] The log information is transmitted to the predetermined buffer in the form of a lock-free queue.

[0106] According to one or more embodiments of the present disclosure, Example 4 provides the log storage method for CDN of Example 1, wherein writing the read log information to the disk includes:

[0107] Cache the read log information;

[0108] When the cached log information meets a preset writing condition, the cached log information is written into the disk together.

[0109] According to one or more embodiments of the present disclosure, Example 5 provides the log writing method for CDN of Example 4, wherein when the cached log information meets a preset writing condition, the cached log information is written to the disk together, including:

[0110] During a preset period, monitoring whether the number of cached log information has reached a preset number;

[0111] If the number of cached log information during the preset period reaches the preset number, after the number of cached log information reaches the preset number, the cached log information is written to the disk together;

[0112] If the number of cached log information does not reach the preset number when the preset period expires, the cached log information is written to the disk together when the preset period expires.

[0113] According to one or more embodiments of the present disclosure, Example 6 provides the log storage method for a CDN of Example 1, wherein the log information is log information generated by a domain name system applied to the CDN. Before transmitting the log information generated by the operation of the CDN system to a predetermined buffer, the log storage method for the CDN further includes:

[0114] Determining the current stage of the domain name system applied to the CDN;

[0115] Extracting information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN;

[0116] The extracted information is filled into the log information in a predetermined format.

[0117] According to one or more embodiments of the present disclosure, Example 7 provides the log storage method for a CDN according to Example 6, wherein extracting information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN includes:

[0118] If the domain name system applied to the CDN is currently in the DNS request stage, extract at least one of the following as information that needs to be recorded in the form of a log: the source address of the DNS request, the request port of the DNS request, the request time of the DNS request, the size of the data packet requested by the DNS request, the requested domain name of the DNS request, the request type of the DNS request, the request category of the DNS request, and the address of the request ECS of the DNS request;

[0119] If the domain name system applied to the CDN is currently in the query phase of the DNS request, extract at least one of the following as information that needs to be recorded in the form of a log: the query matching zone of the DNS response, the client IP line information, and the request matching line;

[0120] If the domain name system applied to the CDN is currently in the DNS response stage, at least one of the following is extracted as information that needs to be recorded in the form of a log: a response status code, and a reply.

[0121] According to one or more embodiments of the present disclosure, Example 8 provides the log storage method for a CDN according to any one of Examples 1 to 7, wherein the log storage method for a CDN further includes:

[0122] Determining whether the predetermined buffer is full;

[0123] If it is determined that the predetermined buffer area is full, the newly generated log information is discarded.

[0124] According to one or more embodiments of the present disclosure, Example 9 provides a log storage device for a CDN, comprising a transmission module, a predetermined buffer, and a log storage module, wherein:

[0125] The transmission module is configured to transmit log information generated by the operation of the system applied to the CDN to the predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN;

[0126] The predetermined buffer is used to cache the transmitted log information;

[0127] The log writing module is used to read the log information from the predetermined buffer and write the read log information to the disk.

[0128] According to one or more embodiments of the present disclosure, Example 10 provides the log storage device for a CDN of Example 9, wherein the transmission module transmits log information generated by the operation of a system applied to the CDN to a predetermined buffer, including:

[0129] Encoding the log information using a buffer protocol;

[0130] The encoded log information is transmitted to the predetermined buffer zone.

[0131] According to one or more embodiments of the present disclosure, Example 11 provides the log storage device for a CDN of Example 9, wherein the transmission module transmits log information generated by the operation of a system applied to the CDN to a predetermined buffer, including:

[0132] The log information is transmitted to the predetermined buffer in the form of a lock-free queue.

[0133] According to one or more embodiments of the present disclosure, Example 12 provides the log storage device for a CDN of Example 9, wherein the log storage module writes the read log information to the disk, including:

[0134] Cache the read log information;

[0135] When the cached log information meets a preset writing condition, the cached log information is written into the disk together.

[0136] According to one or more embodiments of the present disclosure, Example 13 provides a log write-to-disk device for a CDN according to Example 12, wherein the log write-to-disk module writes the cached log information to the disk when the cached log information meets a preset write condition, including:

[0137] During a preset period, monitoring whether the number of cached log information has reached a preset number;

[0138] If the number of cached log information during the preset period reaches the preset number, after the number of cached log information reaches the preset number, the cached log information is written to the disk together;

[0139] If the number of cached log information does not reach the preset number when the preset period expires, the cached log information is written to the disk together when the preset period expires.

[0140] According to one or more embodiments of the present disclosure, Example 14 provides the log storage device for a CDN of Example 9, wherein the log information is log information generated by a domain name system applied to the CDN. The log storage device for the CDN further includes an extraction module configured to, before the transmission module transmits the log information generated by the system operation applied to the CDN to a predetermined buffer:

[0141] Determining the current stage of the domain name system applied to the CDN;

[0142] Extracting information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN;

[0143] The extracted information is filled into the log information in a predetermined format.

[0144] According to one or more embodiments of the present disclosure, Example 15 provides the log storage device for a CDN of Example 14, wherein the extraction module extracts information that needs to be recorded in log form at the current stage of the domain name system applied to the CDN in the following manner:

[0145] If the domain name system applied to the CDN is currently in the DNS request stage, extract at least one of the following as information that needs to be recorded in the form of a log: the source address of the DNS request, the request port of the DNS request, the request time of the DNS request, the size of the data packet requested by the DNS request, the requested domain name of the DNS request, the request type of the DNS request, the request category of the DNS request, and the address of the request ECS of the DNS request;

[0146] If the domain name system applied to the CDN is currently in the query phase of the DNS request, extract at least one of the following as information that needs to be recorded in the form of a log: the query matching zone of the DNS response, the client IP line information, and the request matching line;

[0147] If the domain name system applied to the CDN is currently in the DNS response stage, at least one of the following is extracted as information that needs to be recorded in the form of a log: a response status code, and a reply.

[0148] According to one or more embodiments of the present disclosure, Example 16 provides the log storage device for a CDN according to any one of Examples 9 to 15, wherein the transmission module is further configured to:

[0149] Determining whether the predetermined buffer is full;

[0150] If it is determined that the predetermined buffer area is full, the newly generated log information is discarded.

[0151] According to one or more embodiments of the present disclosure, Example 17 provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the log storage method for CDN described in any one of Examples 1-8.

[0152] According to one or more embodiments of the present disclosure, Example 18 provides an electronic device, comprising: a storage device on which at least one computer program is stored; and at least one processing device for executing the at least one computer program in the storage device to implement the steps of the log storage method for CDN described in any one of Examples 1-8.

[0153] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0154] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0155] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A method for logging CDN data to disk, comprising: Transmitting log information generated by the operation of the system applied to the CDN to a predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; Reading the log information from the predetermined buffer; and Writing the read log information to a disk.

2. The log disk dropping method for CDN according to claim 1, wherein, The transmitting log information generated by the operation of the system applied to the CDN to a predetermined buffer includes: Encoding the log information using a buffer protocol; Transmitting the encoded log information to the predetermined buffer.

3. The log disk writing method for CDN according to claim 1, wherein, The transmitting log information generated by the operation of the system applied to the CDN to a predetermined buffer includes: Transmitting the log information to the predetermined buffer in the form of a lock-free queue.

4. The log disk writing method for CDN according to any one of claims 1-3, wherein, The writing the read log information to a disk includes: Caching the read log information; When the cached log information meets a preset writing condition, writing the cached log information to the disk together.

5. The log disk writing method for CDN according to claim 4, wherein, The writing the cached log information to the disk together when the cached log information meets a preset writing condition includes: During a preset period, monitoring whether the number of the cached log information reaches a preset number; If the number of the cached log information reaches the preset number during the preset period, then after the number of the cached log information reaches the preset number, writing the cached log information to the disk together; If the number of the cached log information does not reach the preset number when the preset period expires, then writing the cached log information to the disk together when the preset period expires.

6. The log disk writing method for CDN according to any one of claims 1-5, wherein, If the log information is the log information generated by the domain name system applied to the CDN, then before transmitting the log information generated by the operation of the system applied to the CDN to a predetermined buffer, the method for logging CDN data to disk further includes: Determining the current stage of the domain name system; Extracting the information that needs to be recorded in the form of logs at the current stage of the domain name system; Filling the extracted information into the log information in a predetermined format.

7. The log disk dropping method for CDN according to claim 6, wherein, The extracting the information that needs to be recorded in the form of logs at the current stage of the domain name system includes: If the current stage of the domain name system is the DNS request stage, extracting at least one of the following as the information that needs to be recorded in the form of logs: the source address of the DNS request, the request port of the DNS request, the request time of the DNS request, the size of the data packet requested by the DNS request, the requested domain name of the DNS request, the request type of the DNS request, the request classification of the DNS request, the address of the request ecs of the DNS request; If the current stage of the domain name system is the query stage of the DNS request, extracting at least one of the following as the information that needs to be recorded in the form of logs: the query matching area of the DNS response, the client IP line information, the request matching line; If the current stage of the domain name system is the DNS response stage, at least one of the following is extracted as the information to be logged: the response status code, the reply.

8. The log disk writing method for CDN according to any one of claims 1-7 further includes: Determining whether the predetermined buffer is full; If it is determined that the predetermined buffer is full, the newly generated log information is discarded.

9. A log disk writing device for CDN includes a transmission module, a predetermined buffer, and a log disk writing module, wherein The transmission module is configured to transmit the log information generated by the system operation applied to the CDN to the predetermined buffer, wherein the predetermined buffer is decoupled from the system applied to the CDN; The predetermined buffer is configured to cache the transmitted log information; The log disk writing module is configured to read the log information from the predetermined buffer and write the read log information to the disk.

10. A computer-readable medium stores a computer program, wherein, When the computer program is executed by the processing device, it implements the log disk writing method for CDN according to any one of claims 1-8.

11. An electronic device includes: A storage device storing at least one computer program; And At least one processing device configured to execute the at least one computer program in the storage device to implement the log disk writing method for CDN according to any one of claims 1-8.

Images