Log data processing method, device and equipment for memory access node

Abstract

The invention provides a log data processing method, device and equipment for a memory access node. The processing method comprises the following steps: acquiring a plurality of log threads which are started at present; dividing the plurality of log threads into N log thread groups according to the number N of non-uniform memory access nodes; obtaining a first preset number of target log file data to be operated by at least one log thread of the N consistent memory access nodes; distributing a global log sequence number for the first preset number of target log file data, wherein the global log sequence number is a log sequence number generated according to all log data in the N log thread groups; and according to a global log sequence number corresponding to the target log file data, writing a first preset quantity of target log file data to be operated by the at least one log thread into a local memory of the non-uniform memory access node. According to the embodiment of the invention, cross-node remote access can be avoided; and the memory access delay is obviously reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of database management, and in particular, to a method, device, and equipment for processing log data of a memory access node. Background Art

[0002] With the improvement of CPU (Central Processing Unit) performance starting to shift from increasing the main frequency to increasing the number of CPUs (multi-core, multi-CPU), the contention of more and more CPUs for the front-side bus has caused the traditional UMA (Uniform Memory Access) access to reach a bottleneck. To overcome the drawbacks brought by this architecture, the NUMA (Non-Uniform Memory Access) architecture emerged as the times require. Each NUMA node contains at least one set of CPU cores and local memory, and the memory access speed within the node (local access) is much faster than the cross-node access (remote access).

[0003] In a memory database system, a large amount of Redo log (redo log) reading and writing is involved. According to the conventional method of globally reading and writing Redo logs, it is inevitably necessary to perform remote memory access across NUMA nodes. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a method, device, and equipment for processing log data of a memory access node, which can enable each log thread group to write the target log file data into the local memory of the corresponding non-uniform memory access node, avoiding cross-node remote access; and significantly reducing the memory access latency.

[0005] To solve the above technical problem, the technical solution of the embodiments of the present invention is as follows:

[0006] A method for processing log data of a memory access node includes:

[0007] Obtain a plurality of currently started log threads;

[0008] Divide the plurality of log threads into N log thread groups according to the number N of non-uniform memory access nodes, where at least one log thread is provided for one non-uniform memory access node; where N is a positive integer;

[0009] Obtain at least a first preset number of target log file data to be operated by at least one log thread of N uniform memory access nodes;

[0010] Allocate a global log sequence number to each of the first preset number of target log file data, where the global log sequence number is the log sequence number generated according to all the log data in the N log thread groups;

[0011] Write the first preset number of target log file data to be operated by each of the at least one log thread to the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data.

[0012] Optionally, allocating a global log sequence number for each of the first preset number of target log file data includes:

[0013] Allocating consecutive global log sequence numbers for each of the first preset number of target log file data; wherein, among the global log sequence numbers of two adjacent target log file data, the global log sequence number of the next target log file data is obtained by adding the memory end address of the current target log sequence number and the memory address offset of the next target log file data.

[0014] Optionally, the memory address offset is equal to the size of the next target log file data.

[0015] Optionally, writing the first preset number of target log file data to be operated by each of the at least one log thread to the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data includes:

[0016] Write the preset number of target log file data to be operated by each of the at least one log thread to the local memory of the non-uniform memory access node in sequence according to the global log sequence number corresponding to the target log file data.

[0017] Optionally, N log thread groups have consecutive thread group numbers.

[0018] Optionally, this processing method further includes:

[0019] Parallelly read the second preset number of target log data from the local memories of multiple coherent memory access nodes;

[0020] Merge the second preset number of target log data according to the global log sequence number to obtain a merged target log file.

[0021] Optionally, this processing method further includes:

[0022] Send the merged target log file to a remote server.

[0023] An embodiment of the present invention further provides a log data processing device for a memory access node, including:

[0024] An acquisition module, configured to acquire a plurality of currently started log threads and acquire the first preset number of target log file data to be operated by at least one log thread of N coherent memory access nodes;

[0025] A processing module, configured to divide the multiple log threads into N log thread groups according to the number N of non-uniform memory access nodes, where at least one log thread is associated with one non-uniform memory access node; N is a positive integer; allocate a global log sequence number for each of the first preset number of target log file data, where the global log sequence number is the log sequence number generated according to all the log data in the N log thread groups; and write the first preset number of target log file data to be operated on by the at least one log thread to the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data.

[0026] An embodiment of the present invention further provides a computing device, including:

[0027] One or more processors;

[0028] A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0029] An embodiment of the present invention further provides a computer-readable storage medium for a computing device, where a program is stored in the computer-readable storage medium for a computing device, and when the program is executed by a processor, the method as described above is implemented.

[0030] The above solution of the embodiment of the present invention has at least the following beneficial effects:

[0031] In the above solution of the embodiment of the present invention, the log threads are divided into log thread groups with the same number as the non-uniform memory access nodes, so that each log thread group writes the target log file data to the local memory of the corresponding non-uniform memory access node, avoiding cross-node remote access; significantly reducing the memory access latency, improving the read and write efficiency of the target log file data in the memory database, and giving full play to the advantage of the fast local memory access speed of the non-uniform memory access architecture.

[0032] Grouping the log threads according to the number of non-uniform memory access nodes realizes parallel writing of the target log file data of multiple non-uniform memory access nodes to the local memory. Multiple non-uniform memory access nodes can perform the write operation of the target log file data simultaneously, improving the parallelism of log processing, and thus enhancing the throughput of the entire memory database system.

[0033] Allocating a globally unique and continuous sequence number for the target log file data ensures the sequentiality of the logs. In the case of parallel writing of the target log file data, the sequentiality of the logs can still be ensured through the global sequence number, enabling multi-way merging of the logs by sequence during database recovery and sending the logs in sequence during master-slave replication, ensuring the accuracy of data recovery and master-slave replication.

[0034] Group and bind the log threads to the non-uniform memory access (NUMA) nodes, and the log thread groups of each NUMA node process logs in the local memory. The CPU cores of each NUMA node can focus on processing the read and write of the local target log file data, reducing the contention of the CPU for the front-side bus, making full use of the CPU resources, and improving the overall performance of the system.

[0035] By means of trading space for performance, reasonably increase the memory and hard disk space occupancy. At the cost of reasonable storage resource consumption, trade for the high performance of local writing of the target log file data by multiple NUMA nodes, achieving the balance between space and performance, and making the system more reasonable in resource utilization. Brief Description of the Drawings

[0036] Figure 1 is a flowchart of the method for processing log data of a memory access node provided by an embodiment of the present invention.

[0037] Figure 2 is a flowchart of globally numbering the target log file data of Instance 1 provided by an embodiment of the present invention.

[0038] Figure 3 is a flowchart of writing the target log file data of Instance 1 into the local memory of a non-uniform memory access node provided by an embodiment of the present invention.

[0039] Figure 4 is a schematic diagram of the modules of the device for processing log data of a memory access node provided by an embodiment of the present invention. Detailed Embodiments

[0040] The exemplary embodiments of the present invention will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0041] As Figure 1 shown, an embodiment of the present invention provides a method for processing log data of a memory access node, including:

[0042] Step 11, obtain multiple currently started log threads; specifically, the log threads may be the currently started log threads in a non-uniform memory access (NUMA) architecture;

[0043] Step 12: Divide the multiple log threads into N log thread groups according to the number N of non-uniform memory access nodes, where at least one log thread is associated with one non-uniform memory access node; N is a positive integer. Specifically, the non-uniform memory access architecture includes N non-uniform memory access nodes. The multiple log threads are divided into N log thread groups such that each non-uniform memory access node has one log thread group, and each log thread group has at least one log thread.

[0044] Step 13: Obtain the first preset quantity of target log file data to be operated on by at least one log thread of the N coherent memory access nodes; the target log file data can be Redo logs.

[0045] Step 14: Allocate a global log sequence number to each of the first preset quantity of target log file data. The global log sequence number is the sequence number generated according to all the log data in the N log thread groups.

[0046] Step 15: Write the first preset quantity of target log file data to be operated on by at least one log thread respectively into the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data.

[0047] Specifically, the N log thread groups have consecutive thread group numbers.

[0048] In this embodiment, the log threads are divided into log thread groups with the same number as the non-uniform memory access nodes, so that each log thread group writes the target log file data into the local memory of the corresponding non-uniform memory access node, avoiding cross-node remote access; significantly reducing the memory access latency, improving the read and write efficiency of the target log file data in the memory database, and giving full play to the advantage of the fast local memory access speed of the non-uniform memory access architecture.

[0049] Grouping the log threads according to the number of non-uniform memory access nodes enables parallel writing of the target log file data of multiple non-uniform memory access nodes into the local memory. Multiple non-uniform memory access nodes can perform the write operation of the target log file data simultaneously, improving the parallelism of log processing, and thus enhancing the throughput of the entire memory database system.

[0050] Allocating a globally unique and consecutive sequence number to the target log file data ensures the order of the logs. In the case of parallel writing of the target log file data, the order of the logs can still be guaranteed through the global sequence number, enabling multi-way merging of the logs by sequence during database recovery and sending the logs in order during master-slave replication, ensuring the accuracy of data recovery and master-slave replication.

[0051] Group the log threads and bind them to the non-uniform memory access nodes. The thread groups of each non-uniform memory access node process the logs in the local memory. The CPU cores of each non-uniform memory access node can focus on processing the read and write of the local target log file data, reducing the contention of the CPU for the front-side bus, making full use of the CPU resources, and improving the overall performance of the system.

[0052] By the way of trading space for performance, reasonably increase the occupation of memory and hard disk space. At the cost of reasonable storage resource consumption, trade for the high performance of local writing of the target log file data on multiple non-uniform memory access nodes, achieving the balance between space and performance, and making the system more reasonable in resource utilization.

[0053] In an optional embodiment of the present invention, in step 14, allocating a global log sequence number for each of the first preset number of target log file data includes:

[0054] Step 141, allocating consecutive global log sequence numbers for each of the first preset number of target log file data; wherein, among the global log sequence numbers of two adjacent target log file data, the global log sequence number of the next target log file data is obtained by adding the memory end address of the current target log sequence number and the memory address offset of the next target log file data; specifically, use the system sequence number manager to allocate consecutive global log sequence numbers for each of the first preset number of target log file data, and the system sequence number manager uses atomic operations to maintain the global sequence number.

[0055] Specifically, the memory address offset is equal to the size of the next target log file data.

[0056] In this embodiment, the system sequence number manager maintains the global sequence number through atomic operations, avoiding sequence conflicts during multi-threaded concurrent access. Atomic operations are indivisible, ensuring the uniqueness of sequence allocation in a multi-threaded environment and preventing the situation of duplicate sequence numbers for different log data.

[0057] The sequence numbers of adjacent log data are generated according to the rule of the current end address + the size of the next data, making the global sequence number form a continuous sequence.

[0058] The continuous global sequence number provides a clear order basis for database recovery and master-slave replication. During recovery, the logs of multiple non-uniform memory access nodes can be merged in multiple ways according to the sequence number to ensure that the logs are applied in the operation order; during master-slave replication, the master end traverses and sends the logs according to the sequence number, and the slave end receives them in order to ensure data consistency.

[0059] If the sequence numbers are not continuous, it may lead to incorrect order during log merging or transmission, affecting the accuracy of data recovery and the reliability of master-slave replication. Continuous sequence numbers fundamentally eliminate such risks.

[0060] Although the log threads of each non-uniform memory access node write to local memory in parallel, the global consecutive sequence numbers keep the log data of different nodes in a strictly logical order.

[0061] The sequence number generation rule is directly related to the data size, and the memory occupancy and storage location of each log data can be accurately calculated.

[0062] When a new non-uniform memory access node is added to the system, only a sequence number segment needs to be allocated to the log thread of the new node according to the rule, without modifying the existing sequence number generation logic.

[0063] The combination of the global sequence number and local memory writing enables the log data of each non-uniform memory access node to be stored locally in order.

[0064] In an optional embodiment of the present invention, in step 15, according to the global log sequence number corresponding to the target log file data, writing the first preset number of target log file data to be operated by each of the at least one log thread into the local memory of the non-uniform memory access node includes:

[0065] Step 151, according to the global log sequence number corresponding to the target log file data, writing the preset number of target log file data to be operated by each of the at least one log thread into the local memory of the non-uniform memory access node in order.

[0066] In this embodiment, when writing log data into the local memory of the corresponding non-uniform memory access node, the high-speed connection between the CPU and memory within the node is directly utilized, completely eliminating the latency increment caused by cross-node remote access.

[0067] The local memory bandwidth of each non-uniform memory access node is independent of other nodes. Writing into the local memory in order can make full use of the memory bandwidth of this node. When multiple threads write to the local memory of their respective nodes in parallel, the bandwidth resources of each node do not preempt each other, and the overall throughput linearly increases with the number of nodes.

[0068] Writing into the local memory in the order of the global sequence number makes the log data strictly follow the operation logic order in physical storage.

[0069] The thread groups of different non-uniform memory access nodes can write to their respective local memories in parallel, but the write operations of all nodes are executed in the order of the global sequence number.

[0070] Writing into the local memory in the order of the sequence number can achieve continuous storage of log data and reduce memory fragmentation. Continuous storage is convenient for memory pre-allocation and release, improving memory management efficiency.

[0071] Given the global sequence number, the storage location of the data in the local memory can be quickly calculated through address offset without traversing and searching.

[0072] When the database is restored, the logs of each non-uniform memory access node have been stored in local order. Only by performing multi-way merge according to the global sequence number can a complete and ordered log stream be generated. Storing in local order reduces the comparison and rearrangement operations during merging, improving the restoration efficiency.

[0073] The master server reads the local logs of each node in the order of the global sequence number and sends them to the slave. The slave applies the logs in order to ensure the consistency of the master-slave data. Sequential writing avoids replication errors caused by out-of-order logs, especially in high-concurrency scenarios.

[0074] In an optional embodiment of the present invention, the processing method further includes:

[0075] Step 16, parallelly read a second preset number of target log data from the local memories of multiple coherent memory access nodes;

[0076] Step 17, merge the second preset number of target log data according to the global log sequence number to obtain a merged target log file.

[0077] In this embodiment, log data is parallelly read from the local memories of multiple non-uniform memory access nodes, utilizing the independent memory bandwidth and CPU resources of each node.

[0078] Merge the data according to the global log sequence number to ensure that the log data of different nodes is integrated in the operation order.

[0079] Parallelly read the log data of multiple nodes without having to read them sequentially according to the node order.

[0080] When writing, store locally according to the global sequence number. When reading, obtain and merge in parallel according to the sequence number to ensure the consistency of the write order - storage order - read order.

[0081] In an optional embodiment of the present invention, the processing method further includes:

[0082] Step 18, send the merged target log file to a remote server.

[0083] In this embodiment, the merged log file has been sorted according to the global sequence number. When sending it to the remote server, directly transmit it in order to ensure that the log order received by the slave is exactly the same as the master operation order.

[0084] Send the log data continuously according to the sequence number, which meets the requirements of network transmission orderliness, avoids the retransmission or recombination overhead caused by out-of-order data packets, and improves the utilization rate of network bandwidth.

[0085] The merged log has been sorted according to the sequence number. When sending, there is no need to sort again or perform sequence verification, reducing the consumption of CPU and memory resources.

[0086] Example 1

[0087] As Figure 2 , 3 shown, Example 1 of the present invention provides a method for processing log data of a memory access node, including:

[0088] Step 21, obtain multiple log threads currently started, where the log threads include NUMA1 thread... NUMAn thread;

[0089] Step 22, divide the multiple log threads into N log thread groups according to the number N of non-uniform memory access nodes, where at least one log thread is associated with one non-uniform memory access node; where N is a positive integer; taking N as 3 as an example, the non-uniform memory access nodes include a first non-uniform memory access node, a second non-uniform memory access node, and a third non-uniform memory access node; each non-uniform memory access node has a log thread group, and each log thread group has at least one log thread;

[0090] Step 23, obtain the first preset number of target log file data (Redo logs) to be operated by at least one log thread of N coherent memory access nodes; for example, the first preset number of the Redo logs is 5, and the byte sizes are 200 bytes, 100 bytes, 400 bytes, 200 bytes, and 100 bytes respectively;

[0091] Step 24, allocate a global log sequence number to each of the 5 Redo logs. For example, the global log sequence number of the first one is redo01+200, the global log sequence number of the second one is redo11+300, the global log sequence number of the third one is redo21+700, the global log sequence number of the fourth one is redo01+900, and the global log sequence number of the fifth one is redo11+1000;

[0092] Step 25, the disk memory of the first non-uniform memory access node includes redo01 file, redo02 file,... redo0N file, the disk memory of the second non-uniform memory access node includes redo11 file, redo12 file,... redo1N file, and the disk memory of the third non-uniform memory access node includes redo21 file, redo22 file,... redo2N file;

[0093] Write the first Redo log to the range of offset 0 - 200 in the redo01 file, the second Redo log to the range of offset 200 - 300 in the redo11 file, the third Redo log to the range of offset 300 - 700 in the redo21 file, the fourth Redo log to the range of offset 700 - 900 in the redo01 file, and the fifth Redo log to the range of offset 900 - 1000 in the redo11 file according to the global log sequence number corresponding to the Redo log.

[0094] Step 26, read the first Redo log,... the fifth Redo log in parallel from the first non-uniform memory access node, the second non-uniform memory access node, and the third non-uniform memory access node.

[0095] Step 27, merge the first Redo log,... the fifth Redo log according to the global log sequence numbers of the first Redo log,... the fifth Redo log to obtain the merged target log file.

[0096] Step 28, send the merged target log file to the remote server.

[0097] After grouping the log threads by non-uniform memory access nodes in the present invention, each log thread group only writes the Redo log to the local memory of the corresponding node. By utilizing the local access speed advantage of the non-uniform memory access architecture, the latency increment caused by cross-node remote access is completely avoided, and the log read and write efficiency is significantly improved.

[0098] Grouping by the number of nodes enables parallel writing of logs on multiple non-uniform memory access nodes. Each node independently operates on its local memory, and the bandwidth resources do not preempt each other. The overall throughput increases linearly with the number of nodes, breaking through the parallelism limitation of the traditional global log mode.

[0099] The system sequence number manager generates continuous global sequence numbers through atomic operations to ensure that the logs written in parallel by multiple nodes are strictly ordered logically. The sequence numbers are allocated according to the rule of the current end address + data size, enabling the cross-node logs to be merged in order during merging, and ensuring the accuracy of database recovery and master-slave replication.

[0100] The sequence number is directly associated with the data size, and the storage location of the log in the local memory can be accurately deduced, avoiding traversal and search; continuous storage reduces memory fragmentation, facilitating memory pre-allocation and disk space management, and improving resource utilization.

[0101] After the log thread group is bound to the non-uniform memory access node, the CPU cores of each node focus on local log processing, reducing the contention of the front-side bus, significantly improving the utilization rate of multi-core resources, and adapting to the multi-core and multi-node server architecture.

[0102] Through the space-for-performance strategy, the log space is allowed to linearly expand with the number of nodes (N groups of nodes occupy N times the space), so as to obtain high-performance local reading and writing at a reasonable storage cost, and the resource utilization is more reasonable.

[0103] When adding non-uniform memory access nodes, only need to allocate serial number segments according to rules, without modifying the existing logic. The system can linearly improve the processing ability with the increase of the number of nodes, and adapt to large-scale data scenarios.

[0104] When writing, store locally according to the serial number. When reading, obtain in parallel and merge in order, and keep the order when sending to the remote end, ensuring the consistency of the full process logic of writing-storing-reading-copying, and reducing the risk of data inconsistency.

[0105] The logs of each node are stored in local order. Only need to perform multi-way merging according to the serial number to generate an ordered log stream; when master-slave replication, send the logs in order, and the slave end applies them in order, greatly shortening the recovery time and ensuring the reliability of replication.

[0106] The merged logs are sent continuously according to the serial number, meeting the requirements of network transmission orderliness, reducing the overhead of packet retransmission; no additional sorting or verification is required, saving CPU and memory resources, and improving the overall efficiency of the system.

[0107] As Figure 4 shown, an embodiment of the present invention further provides a log data processing device 40 for a memory access node, including:

[0108] An obtaining module 41, configured to obtain a plurality of currently started log threads and at least one log thread of N coherent memory access nodes respectively to operate on a first preset number of target log file data;

[0109] A processing module 42, configured to divide the plurality of log threads into N log thread groups according to the number N of non-uniform memory access nodes, where at least one log thread is associated with one non-uniform memory access node; where N is a positive integer; allocate a global log serial number for each of the first preset number of target log file data, and the global log serial number is the log serial number generated according to all log data in the N log thread groups; write the at least one log thread respectively to operate on the first preset number of target log file data into the local memory of the non-uniform memory access node according to the global log serial number corresponding to the target log file data.

[0110] Optionally, allocating a global log serial number for each of the first preset number of target log file data includes:

[0111] Allocate consecutive global log sequence numbers to the target log file data of the first preset quantity; among them, in the global log sequence numbers of two adjacent target log file data, the global log sequence number of the next target log file data is obtained by adding the memory end address of the current target log sequence number to the memory address offset of the next target log file data.

[0112] Optionally, the memory address offset is equal to the size of the next target log file data.

[0113] Optionally, write the target log file data of the first preset quantity that at least one log thread is to operate on, to the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data, including:

[0114] Write the target log file data of the preset quantity that at least one log thread is to operate on, to the local memory of the non-uniform memory access node in sequence according to the global log sequence number corresponding to the target log file data.

[0115] Optionally, N log thread groups have consecutive thread group numbers.

[0116] Optionally, this processing method further includes:

[0117] Read the target log data of the second preset quantity in parallel from the local memories of multiple coherent memory access nodes;

[0118] Merge the target log data of the second preset quantity according to the global log sequence number to obtain a merged target log file.

[0119] Optionally, this processing method further includes:

[0120] Send the merged target log file to a remote server.

[0121] It should be noted that this device corresponds to the above method, and all implementation manners in the above method embodiments are applicable to this embodiment and can also achieve the same technical effects.

[0122] An embodiment of the present invention further provides a computing device, including:

[0123] One or more processors;

[0124] A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method as described above.

[0125] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computing device, cause the computing device to perform the method described above. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0126] Those of ordinary skill in the art will appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or in a combination of computing device software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0127] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0128] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, the functional units in the various embodiments of the present invention can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0131] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a storage medium readable by a computing device. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The software product of the computing device is stored in a storage medium and includes several instructions for causing a computing device (which may be a personal computing device, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0132] In addition, it should be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to execute them in chronological order. Some steps can be executed in parallel or independently of each other. For those of ordinary skill in the art, it can be understood that all or any steps or components of the methods and devices of the present invention can be implemented in any computing device (including processors, storage media, etc.) or in a network of computing devices in the form of hardware, firmware, software, or a combination thereof, which can be achieved by those of ordinary skill in the art using basic programming skills after reading the description of the present invention.

[0133] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a well-known general-purpose device. Therefore, the object of the present invention can also be achieved only by providing a program product containing program codes for implementing the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to execute them in chronological order. Some steps can be executed in parallel or independently of each other.

[0134] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for processing log data of a memory access node, characterized in that, including: Obtain multiple currently started log threads; Divide the multiple log threads into N log thread groups according to the number N of non-uniform memory access nodes, where each non-uniform memory access node has at least one log thread; where N is a positive integer; Obtain the first preset quantity of target log file data to be operated by at least one log thread of the N coherent memory access nodes; Allocate a global log sequence number for each of the first preset quantity of target log file data, where the global log sequence number is the log sequence number generated according to all the log data in the N log thread groups; Write the first preset quantity of target log file data to be operated by at least one log thread respectively into the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data.

2. The method for processing log data of a memory access node according to claim 1, wherein Allocating a global log sequence number for each of the first preset quantity of target log file data includes: Allocate consecutive global log sequence numbers for each of the first preset quantity of target log file data; where, among the global log sequence numbers of two adjacent target log file data, the global log sequence number of the next target log file data is obtained by adding the memory end address of the current target log sequence number to the memory address offset of the next target log file data.

3. The method for processing log data of a memory access node according to claim 2, wherein, The memory address offset is equal to the size of the next target log file data.

4. The method for processing log data of a memory access node according to claim 1, wherein Writing the first preset quantity of target log file data to be operated by at least one log thread respectively into the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data includes: Write the first preset quantity of target log file data to be operated by at least one log thread respectively into the local memory of the non-uniform memory access node in order according to the global log sequence number corresponding to the target log file data.

5. The method for processing log data of a memory access node according to claim 1, wherein The N log thread groups have consecutive thread group numbers.

6. The method for processing log data of a memory access node according to claim 1, wherein It also includes: Parallelly read the second preset quantity of target log data from the local memories of multiple coherent memory access nodes; Merge the second preset quantity of target log data according to the global log sequence number to obtain a merged target log file.

7. The method for processing log data of a memory access node according to claim 6, wherein It also includes: Send the merged target log file to a remote server.

8. A log data processing device for a memory access node, characterized in that, including: An obtaining module, configured to obtain multiple currently started log threads and the first preset quantity of target log file data to be operated by at least one log thread of the N coherent memory access nodes; A processing module, configured to divide the multiple log threads into N log thread groups according to the number N of non-uniform memory access nodes, where each non-uniform memory access node has at least one log thread; where N is a positive integer; allocate a global log sequence number for each of the first preset quantity of target log file data, where the global log sequence number is the log sequence number generated according to all the log data in the N log thread groups; write the first preset quantity of target log file data to be operated by at least one log thread respectively into the local memory of the non-uniform memory access node according to the global log sequence number corresponding to the target log file data.

9. A computing device, characterized in that, including: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors cause the one or more processors to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that A program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images