Memory index synchronization method and device, computer equipment and readable storage medium

Abstract

The invention relates to a memory index synchronization method and device, computer equipment and a readable storage medium. The method comprises the following steps: registering an exit signal processing function under the condition that a program is started; generating a program exit signal based on the registered exit signal processing function under the condition that the exit demand occurs; in response to the program exit signal, threads corresponding to the number of the disks in the CDN are started, so that the memory index is synchronized to the corresponding disk based on a multi-thread mode, and program exit is executed after synchronization of the memory index is completed. According to the memory index synchronization method, through a multi-thread memory index synchronization mode, the time for synchronizing the memory index to the corresponding disk can be greatly shortened, and the synchronization rate is increased.

Description

Technical Field

[0001] The present application relates to the technical field of memory index synchronization, and in particular to a memory index synchronization method, apparatus, computer equipment and readable storage medium. Background Art

[0002] In the CDN (Content Delivery Network) cache, multiple disks are used, each of which independently stores http (Hypertext Transfer Protocol) data, and each disk has a one-to-one corresponding independent index.

[0003] When the program starts, it reads the index from the head of each disk and loads it into the memory. Correspondingly, when the index in the memory changes, the index in the disk is also updated accordingly. However, the rate of synchronizing the memory index to the disk is slow, while the rate of writing HTTP data to the disk is fast, which leads to inconsistency between the memory index and the HTTP data. Summary of the invention

[0004] Based on this, it is necessary to provide a memory index synchronization method, device, computer equipment and readable storage medium that can quickly synchronize the memory index to the disk to ensure that the memory index is synchronized with the http data in response to the above technical problems.

[0005] In a first aspect, the present application provides a memory index synchronization method, which is applied in a content distribution network, and the method includes:

[0006] When the program starts, register the exit signal processing function;

[0007] In the event of an exit requirement, generating a program exit signal based on the exit signal processing function;

[0008] In response to the program exit signal, starting threads corresponding to the number of disks;

[0009] Based on the threads corresponding to the disks, synchronizing the memory index to the corresponding disks;

[0010] Exit the program in question.

[0011] In one embodiment, after synchronizing the memory index to the corresponding disk based on the thread corresponding to each disk, the method further includes:

[0012] An identifier is written in each disk, wherein the identifier is used to represent the corresponding relationship between each memory index and each disk.

[0013] In one embodiment, the method further comprises:

[0014] When the program is started, check whether there is a corresponding identifier in each disk;

[0015] In the case that at least one disk lacks a corresponding identifier, verification synchronization of the memory index and http data in the target disk is performed; wherein the target disk is the disk lacking the corresponding identifier.

[0016] In one embodiment, the step of generating a program exit signal based on the exit signal processing function includes:

[0017] When the program is executed, a normal exit signal is output based on the normal exit signal processing function;

[0018] In the case of abnormal program interruption, an abnormal exit signal is output based on the abnormal exit signal processing function.

[0019] In one embodiment, the step of determining abnormal interruption of a program includes:

[0020] Check the amount of memory currently occupied by the cache service;

[0021] When the amount of memory exceeds a preset memory threshold, the program is judged to be abnormally interrupted.

[0022] In one embodiment, the method further comprises:

[0023] In response to the abnormal exit signal generating operation, a program exit signal is generated.

[0024] In one embodiment, based on the threads corresponding to each disk, the step of synchronizing the memory index to the corresponding disk includes:

[0025] Based on the threads corresponding to each disk, the memory index is synchronized to the corresponding disk in the form of data blocks.

[0026] In a second aspect, the present application further provides a memory index synchronization device, including:

[0027] Function registration module, used to register the exit signal processing function when the program is started;

[0028] A program exit signal generation module is used to generate a program exit signal based on an exit signal processing function when an exit requirement occurs;

[0029] A multi-threaded start module, used to start threads corresponding to the number of disks in response to a program exit signal;

[0030] The synchronization module is used to synchronize the memory index to the corresponding disk based on the thread corresponding to each disk;

[0031] Exit module, used to exit the program.

[0032] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any step in the above-mentioned memory index synchronization method when executing the computer program.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step in the above-mentioned memory index synchronization method.

[0034] The above-mentioned memory index synchronization method, device, computer equipment and readable storage medium register an exit signal processing function when the program is started; when an exit requirement occurs, a program exit signal is generated based on the registered exit signal processing function; in response to the program exit signal, a thread corresponding to the number of disks in the CDN is started, thereby synchronizing the memory index to the corresponding disk based on a multi-threaded manner, and then executing the program exit after the memory index synchronization is completed. The above-mentioned memory index synchronization method can greatly reduce the time for the memory index to be synchronized to the corresponding disk through a multi-threaded memory index synchronization method, thereby improving the synchronization rate; further, synchronizing the memory index to the corresponding disk before exiting the program can ensure that the disk index and the disk http data remain consistent, thereby solving the problem of long-term cache unavailability due to inconsistency between the disk index and the disk http data after the program is restarted, and greatly ensuring the cache service quality of the CDN. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 An application environment diagram of a memory index synchronization method in an embodiment;

[0037] Figure 2 A schematic diagram of a flow chart of a memory index synchronization method in one embodiment;

[0038] Figure 3 is a structural block diagram of a memory index synchronization device in one embodiment;

[0039] Figure 4FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] The memory index synchronization method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the data storage system 102 can store data such as signal processing functions that the server 104 needs to process. The data storage system 102 can be integrated on the server 104, or it can be placed on the cloud or other network servers. Specifically, when the program is started, the exit signal processing function is registered; when an exit requirement occurs, a program exit signal is generated based on the registered exit signal processing function; in response to the program exit signal, a thread corresponding to the number of disks in the CDN is started, so as to synchronize the memory index to the corresponding disk based on a multi-threaded manner, and then execute the program exit after the memory index synchronization is completed.

[0042] In an exemplary embodiment, Figure 2 As shown, a memory index synchronization method is provided, which is applied in a content distribution network, and the method includes:

[0043] S202, when the program is started, register an exit signal processing function.

[0044] The program may be any program that is burned into the content distribution network and implements a preset function based on the data in each disk in the content distribution network.

[0045] S204: When an exit requirement occurs, a program exit signal is generated based on an exit signal processing function.

[0046] The exit signal processing function is a function that generates a program exit signal when an exit demand occurs in the content distribution network. The exit demand includes situations where the program needs to be exited, such as the completion of program execution, abnormal interruption of the program, and manual interruption by staff.

[0047] S206, in response to the program exit signal, start threads corresponding to the number of disks.

[0048] S208, based on the threads corresponding to each disk, synchronize the memory index to the corresponding disk.

[0049] Since the content distribution network includes multiple disks, starting threads corresponding to the number of disks to simultaneously write each memory index to the corresponding disk can greatly improve the synchronization rate of the memory index. For example, in the case of dozens of disks, in response to the program exit signal, starting a corresponding number of threads can synchronize all memory indexes to the corresponding disk within 5 seconds. Compared with the minute-dimensional time data consumed by a single thread to synchronize the index corresponding to each disk in sequence, the speed of the multi-threaded synchronization method is greatly improved.

[0050] S210, exit the program.

[0051] Exiting the program after the memory index synchronization is completed can ensure that the corresponding index on the disk is consistent with the memory index. Since the memory index is the latest index, the disk index is also updated to the latest index. Therefore, after reopening the program, there is no need to verify and synchronize the disk index with the disk http data.

[0052] The above-mentioned memory index synchronization method, when the program is started, registers an exit signal processing function; when an exit requirement occurs, generates a program exit signal based on the registered exit signal processing function; responds to the program exit signal to start a thread corresponding to the number of disks in the CDN, thereby synchronizing the memory index to the corresponding disk based on a multi-threaded manner, and then executing the program exit after the memory index synchronization is completed. The above-mentioned memory index synchronization method can greatly reduce the time for the memory index to be synchronized to the corresponding disk through a multi-threaded memory index synchronization method, thereby improving the synchronization rate; further, synchronizing the memory index to the corresponding disk before exiting the program can ensure that the disk index and the disk http data remain consistent, thereby solving the problem of the cache being unavailable for a long time due to the inconsistency between the disk index and the disk http data after the program is restarted, and greatly ensuring the cache service quality of the CDN.

[0053] In an exemplary embodiment, after the step of synchronizing the memory index to the corresponding disk based on the thread corresponding to each disk, the method further includes:

[0054] An identifier is written in each disk, wherein the identifier is used to represent the corresponding relationship between each memory index and each disk.

[0055] Since the identifier establishes a corresponding relationship between the memory index and the disk, and the identifier is written after the step of "synchronizing the memory index to the corresponding disk", the disk has the latest memory index according to the index corresponding to the identifier. Therefore, the latest memory index corresponds to the http data written to the disk, thereby ensuring the consistency between the index and the http data.

[0056] In an exemplary embodiment, the method further includes:

[0057] When the program is started, it is detected whether there is a corresponding identifier in each disk.

[0058] In the case that at least one disk lacks a corresponding identifier, verification synchronization of the memory index and http data in the target disk is performed; wherein the target disk is the disk lacking the corresponding identifier.

[0059] If the corresponding identifier exists on the disk, it means that the corresponding index on the disk has been updated to the latest memory index. At this time, the disk http data also corresponds to the latest memory index. Therefore, there is no need to verify and synchronize the memory index and http data on the disk.

[0060] If at least one disk in the disk lacks the corresponding identifier, it means that the index corresponding to the target disk has not been updated to the latest memory index, so there is inconsistency between the index and the http data. At this time, the memory index and http data in the target disk can be verified and synchronized based on the traditional method.

[0061] Specifically, the above traditional method is: after the program is started, the index is read from the head of the target disk, and the redundant http data in the target disk is cleaned up, and then the memory index is synchronized to the target disk in a timed and quantitative manner, and the synchronization rate is about 2MB / s; and the http data is synchronized to the target disk in real-time synchronization, so as to ensure the consistency between the index on the disk and the http data.

[0062] In an exemplary embodiment, the step of generating a program exit signal based on the exit signal processing function includes:

[0063] When the program is executed, a normal exit signal is output based on the normal exit signal processing function.

[0064] In the case of abnormal program interruption, an abnormal exit signal is output based on the abnormal exit signal processing function.

[0065] Program exception interrupts include exception interrupts caused by functions corresponding to signals such as SIGBUS (Bus Error), SIGSEGV (Segmentation Violation), SIGILL (Illegal Instruction), SIGTRAP (Trap Condition), SIGFPE (Floating Point Exception), and SIGABRT (Aborted).

[0066] Among them, SIGBUS is a program exit signal sent when a program attempts to access unaligned memory or an invalid memory address. SIGSEGV is a program exit signal sent when a program attempts to access unallocated memory or an invalid memory segment. SIGILL is a program exit signal sent when a program attempts to execute an instruction that is illegal or undefined in the current environment. SIGTRAP is a program exit signal sent when a program executes a trap instruction, usually used for debugging tools. SIGFPE is a program exit signal sent when a program performs an illegal floating-point operation (such as division by zero). SIGABRT is a program exit signal sent when a program calls the abort() function or is forcibly terminated due to some unexpected situation.

[0067] In an exemplary embodiment, the step of determining whether a program is abnormally interrupted includes:

[0068] Check the amount of memory currently used by the cache service.

[0069] When the amount of memory exceeds a preset memory threshold, the program is judged to be abnormally interrupted.

[0070] Because a program consumes too many memory resources during runtime, causing the amount of memory to exceed the preset memory threshold, the operating system actively terminates the program in order to protect the stability and performance of the system, and is unable to output an abnormal exit signal based on the abnormal exit signal processing function. Correspondingly, the amount of memory currently occupied by the cache service can be detected. When the amount of memory exceeds the preset memory threshold, it can be actively judged as an abnormal program interruption, thereby outputting an abnormal exit signal.

[0071] Specifically, the detection of the amount of memory currently occupied by the cache service can be performed once every preset time, so as to monitor the amount of memory in a timely manner and improve the reliability of the memory index synchronization method. The preset time can be 1 second, or other values.

[0072] Furthermore, this step can be implemented by an additionally deployed monitoring cache script, or it can be directly burned into the CDN.

[0073] In an exemplary embodiment, the method further includes:

[0074] In response to the abnormal exit signal generating operation, the program exit signal is generated.

[0075] The abnormal exit signal generation operation is based on the staff's active intention and is achieved through buttons or touch of the visual interface, so that the program is interrupted at the time the staff needs it, so that the staff can perform maintenance, improvement and other operations.

[0076] In an exemplary embodiment, based on the threads corresponding to each disk, the step of synchronizing the memory index to the corresponding disk includes:

[0077] Based on the threads corresponding to each disk, the memory index is synchronized to the corresponding disk in the form of data blocks.

[0078] Synchronizing the memory index to the corresponding disk in the form of data blocks can avoid a reduction in the write rate caused by too much data being written to the disk at one time, thereby increasing the synchronization rate. For example, the size of the data block can be 8MB.

[0079] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0080] Based on the same inventive concept, the embodiment of the present application also provides a memory index synchronization device for implementing the memory index synchronization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more memory index synchronization device embodiments provided below can refer to the limitations of the memory index synchronization method above, and will not be repeated here.

[0081] In an exemplary embodiment, Figure 3 As shown, a memory index synchronization device 300 is provided, including: a function registration module 302, a program exit signal generation module 304, a multi-threaded start module 306, a synchronization module 308 and an exit module 310, wherein:

[0082] The function registration module 302 is used to register the exit signal processing function when the program is started.

[0083] The program exit signal generating module 304 is used to generate a program exit signal based on an exit signal processing function when an exit requirement occurs.

[0084] The multi-threaded start module 306 is used to start threads corresponding to the number of disks in response to a program exit signal.

[0085] The synchronization module 308 is used to synchronize the memory index to the corresponding disk based on the thread corresponding to each disk.

[0086] The exit module 310 is used to exit the program.

[0087] In an exemplary embodiment, the memory index synchronization device 300 further includes: an identification writing module.

[0088] The identifier writing module is used to write identifiers in each disk; wherein the identifier is used to represent the corresponding relationship between each memory index and each disk.

[0089] In an exemplary embodiment, the memory index synchronization device 300 further includes: an identification detection module and a verification synchronization module.

[0090] The identification detection module is used to detect whether there is a corresponding identification in each disk when the program is started.

[0091] The verification synchronization module is used to perform verification synchronization of the memory index and http data in the target disk when at least one disk lacks a corresponding identifier; wherein the target disk is a disk lacking a corresponding identifier.

[0092] In an exemplary embodiment, the program exit signal generating module 304 includes: a first module and a second module.

[0093] The first module is used to output a normal exit signal based on a normal exit signal processing function when the program is executed.

[0094] The second module is used to output an abnormal exit signal based on the abnormal exit signal processing function when the program is abnormally interrupted.

[0095] In an exemplary embodiment, the memory index synchronization device 300 further includes: a memory quantity detection module and a judgment module.

[0096] The memory amount detection module is used to detect the amount of memory currently occupied by the cache service.

[0097] The judgment module is used to judge that the program is abnormally interrupted when the amount of memory exceeds a preset memory threshold.

[0098] In an exemplary embodiment, the memory index synchronization device 300 further includes: an external control module.

[0099] The external control module is used to generate a program exit signal in response to an abnormal exit signal generation operation.

[0100] In an exemplary embodiment, the synchronization module 308 includes: a synchronization submodule.

[0101] The synchronization submodule is used to synchronize the memory index to the corresponding disk in the form of data blocks based on the threads corresponding to each disk.

[0102] Each module in the above-mentioned memory index synchronization device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0103] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as exit signal processing functions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a memory index synchronization method is implemented.

[0104] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the steps of any one of the above-mentioned memory index synchronization methods.

[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any method of the above-mentioned memory index synchronization method are implemented.

[0107] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of any one of the above-mentioned memory index synchronization methods when executed by a processor.

[0108] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0109] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0110] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A memory index synchronization method, characterized in that: Applied in a content distribution network, the method comprises: When the program starts, register the exit signal processing function; In the event of an exit requirement, generating a program exit signal based on the exit signal processing function; In response to the program exit signal, starting threads corresponding to the number of disks; Based on the threads corresponding to the disks, synchronizing the memory index to the corresponding disks; Exit the program in question.

2. The method according to claim 1, characterized in that After the step of synchronizing the memory index to the corresponding disk based on the thread corresponding to each disk, the method further includes: An identifier is written in each of the disks; wherein the identifier is used to represent the corresponding relationship between each of the memory indexes and each of the disks.

3. The method according to claim 2, characterized in that The method further comprises: When the program is started, detecting whether there is a corresponding identifier in each of the disks; In the case that at least one of the disks lacks a corresponding identifier, verification synchronization of the memory index and the http data in the target disk is performed; wherein the target disk is a disk lacking a corresponding identifier.

4. The method according to claim 1, characterized in that: The generating a program exit signal based on the exit signal processing function comprises: When the program is executed, the normal exit signal is output based on the normal exit signal processing function; When the program is abnormally interrupted, the abnormal exit signal is output based on the abnormal exit signal processing function.

5. The method according to claim 4, characterized in that The step of determining abnormal interruption of the program comprises: Check the amount of memory currently occupied by the cache service; When the amount of memory exceeds a preset memory threshold, it is determined that the program is abnormally interrupted.

6. The method according to claim 1, characterized in that The method further comprises: In response to the abnormal exit signal generating operation, the program exit signal is generated.

7. The method according to claim 1, characterized in that The step of synchronizing the memory index to the corresponding disk based on the thread corresponding to each disk includes: Based on the threads corresponding to the disks, the memory index is synchronized to the corresponding disks in the form of data blocks.

8. A memory index synchronization device, characterized in that: The device comprises: Function registration module, used to register the exit signal processing function when the program starts; A program exit signal generating module, used for generating a program exit signal based on the exit signal processing function when an exit requirement occurs; A multi-threaded start module, used for starting threads corresponding to the number of disks in response to the program exit signal; A synchronization module, used for synchronizing the memory index to the corresponding disk based on the thread corresponding to each disk; The exit module is used to exit the program.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images