Data processing method and device and computing equipment

Abstract

The invention discloses a data processing method and device and computing equipment. The method is executed by a data processing system, after the data processing system obtains the cache data, under the condition that the cache data volume of the cache data is determined to be larger than the available memory space capacity of the memory of the locally deployed server cluster, part of the cache data in the cache data can be automatically overflowed into an external memory, and the cache data volume of the cache data is more than the available memory space capacity of the memory of the locally deployed server cluster. The data processing system can be prevented from frequently reporting OOM errors, so that the stability of the data processing system is improved.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on December 5, 2023, with application number 202311677930.6 and application name “A kind of internal and external memory data processing system for big data”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present invention relates to the field of data processing technology, and in particular to a data processing method, device and computing equipment. Background Art

[0003] The memory wall refers to the speed mismatch between the processor and memory of a computer system. As the computing speed of the processor continues to increase, the processor can execute multiple instructions in one clock cycle. However, the processor needs to wait for multiple clock cycles to read data from the memory, making the processor's computing speed much faster than the memory reading speed, resulting in the processor being unable to fully utilize its own computing power, limiting the processor's performance.

[0004] On the one hand, due to the limitation of memory resources, the CPU vacancy rate is generally high in the distributed cluster of big data processing systems, which leads to insufficient task concurrency and performance bottlenecks in computing tasks. On the other hand, in the process of executing computing tasks, out of memory (OOM) failures often occur in big data processing systems, which leads to the failure of computing tasks and seriously affects the stability of production operations of big data processing systems. Taking the spark system as an example, it is difficult for the spark system to accurately estimate the memory usage of executing spark jobs, so it is necessary to determine the memory usage of executing spark jobs by rerunning them multiple times. However, the cost of failure or retry of spark jobs is becoming increasingly unbearable. As the amount of data in the production environment of the spark system increases day by day, the estimated memory content soon encounters the OOM problem and needs to be re-evaluated. In addition, the structured query language (SQL) statements in the production environment of the spark system are becoming more and more complex, resulting in more execution stages and longer execution time of spark jobs, making the cost of multiple reruns increasingly high. Summary of the invention

[0005] In order to solve the above problems, a data processing method is provided in an embodiment of the present application. When the memory resources of the server or server cluster deployed by the data processing system are insufficient, the pre-stored cache data can be automatically overflowed to the external memory, which can avoid the data processing system from frequently reporting OOM errors, so as to improve the stability of the data processing system. In addition, the present application also provides a data processing device and a computing device corresponding to the data processing method.

[0006] To this end, the following technical solutions are adopted in the embodiments of the present application:

[0007] In a first aspect, a data processing method is provided in an embodiment of the present application, wherein a data processing system is deployed in a server cluster, the server cluster includes multiple computing devices and at least one memory, and the method is executed by any one or more computing devices among the multiple computing devices, including: obtaining cache data; the cache data refers to data that needs to be temporarily stored during the operation of the data processing system; when the amount of cached data of the cached data is greater than the available memory space capacity of the local memory, sending part of the cached data in the cached data to the at least one memory, so that the at least one memory stores part of the cached data in the cached data.

[0008] In this embodiment, after obtaining the cached data, if the data processing system determines that the cached data amount is greater than the available memory space capacity of the memory of the locally deployed server cluster, part of the cached data can be automatically overflowed to the external memory, thereby avoiding the data processing system from frequently reporting OOM errors and improving the stability of the data processing system.

[0009] In another possible embodiment, sending part of the cache data in the cache data to the at least one memory specifically includes: obtaining code of a memory allocation interface, and replacing the memory allocation interface with a custom memory allocation interface; the custom memory allocation interface is used to allocate storage space in the local memory and / or the at least one memory; and storing part of the cache data in the allocated storage space in the at least one memory through the custom memory allocation interface.

[0010] In this embodiment, the data processing system can obtain the code about the memory allocation interface in the source code, replace the code about the memory allocation interface in the source code with the custom memory allocation interface, and use the static call method to replace the memory allocation interface in the source code. The custom memory allocation interface can allocate storage space in the local memory or external memory, so that the CPU can access the data in the memory space, and can effectively control the interface interception range to avoid affecting other unrelated components in the data processing system.

[0011] In another possible embodiment, when the amount of cached data in the cached data is greater than the available memory space capacity of the local memory, before sending part of the cached data to the at least one memory, the method further includes: configuring a transparent memory for each executor; the computing device includes at least one executor, and the computing device uses the executor to store the cached data in the transparent memory configured by the executor, and the transparent memory includes a first set size of the available memory space capacity of the local memory and a second set size of the storage space of the at least one memory.

[0012] In this embodiment, the data processing system can configure a transparent memory including storage space of an external memory for each executor, so that when the local memory of the data processing system is insufficient, the executor can store part of the cached data in the external memory in the transparent memory, thereby realizing automatic overflow of the cached data to the external memory.

[0013] In another possible embodiment, storing part of the cache data in the cache data in the storage space allocated in the at least one memory through the custom memory allocation interface specifically includes: sending a memory allocation request to the at least one memory when the amount of cached data of the cache data is greater than the available memory space capacity of the local memory; the memory allocation request is used to allocate a target storage space for storing the cache data; determining through the custom memory allocation interface whether the target storage space requested to be allocated by the memory allocation request is greater than a set memory threshold; when the target storage space requested to be allocated by the memory allocation request is greater than the set memory threshold, allocating storage space in the transparent memory configured by the executor, and storing part of the cache data in the target storage space.

[0014] In another possible embodiment, the method further includes: sending a first request instruction to the at least one memory through the custom memory allocation interface; the first request instruction is used to request the at least one memory to allocate free storage space after the target storage space.

[0015] In this embodiment, the data processing system can send a first request instruction to the external memory through a custom memory allocation interface, so that after the external memory allocates storage space according to the memory allocation request, it can allocate free storage space after the allocated storage space according to the first request instruction, so that when the data processing system allocates storage space in the external memory for the second time, it can directly use the free storage space allocated based on the first request instruction, without copying the data of the memory block allocated last time, thereby realizing zero copy of cached data.

[0016] In another possible embodiment, when the amount of cached data in the cached data is greater than the available memory space capacity of the local memory, part of the cached data in the cached data is sent to the at least one external memory, specifically including: detecting the number of times each data in the cached data is accessed and / or stored; sending first cached data to the at least one external memory; the first cached data refers to data in the cached data whose number of accesses and / or storages is less than a set number of times.

[0017] In this embodiment, the data processing system can detect the number of times the cache data is accessed and the number of times it is stored, and determine the hotness and coldness of each data in the cache data. The data processing system can place the hot data in the memory and the cold data in the external memory, which can avoid reducing the memory access performance of the data processing system.

[0018] In another possible embodiment, the method also includes: obtaining a memory access request instruction; the memory access request instruction is used to read or write specified data, and the memory access request instruction includes a virtual memory address of the specified data; querying a locally stored address mapping table to determine a physical storage address of the specified data; the address mapping table records a mapping relationship between a virtual memory address and a physical storage address; the physical storage address refers to a physical memory address of the local memory and a physical address of the at least one memory; detecting whether the physical storage address of the specified data is the physical memory address of the local memory; if the physical storage address of the specified data is the physical memory address of the local memory, reading the data stored in the physical memory address of the specified data, or writing the specified data to the corresponding physical memory address.

[0019] In this embodiment, when the data processing system accesses the specified data, it can find the physical storage address mapped by the virtual memory address in the address mapping table according to the virtual memory address carried by the memory access request instruction. The data processing system determines that the physical storage address of the specified data is the physical memory address of the local memory, and directly reads the data from the corresponding physical memory address in the local memory, so that the data processing system can successfully read the data stored in the local memory.

[0020] In another possible embodiment, the method also includes: when the physical storage address of the specified data is not the physical memory address of the local memory, allocating a specified storage space in the local memory; forwarding the memory access request instruction to the at least one memory; the memory access request instruction is used to request the at least one memory to read the data stored at the physical storage address corresponding to the virtual memory address of the specified data; receiving the specified data sent by the at least one memory, and storing the specified data in the specified storage space.

[0021] In this embodiment, the data processing system determines that the physical storage address of the specified data is not the physical memory address of the local memory, and can call the cache replacement algorithm to select a cold data block from the physical memory space. The data processing system can send an IO request to the external memory to write the cold data block to the external memory, or read the accessed data to the selected cold data block position. After the data processing system determines that the IO operation is completed, it can return the physical memory address corresponding to the accessed data, and record the mapping relationship between the virtual memory address and the physical memory address of the accessed data, and between the virtual address of the cold data block and the physical address of the external memory in the address mapping table, so that the data processing system can successfully access the data stored in the external memory.

[0022] In another possible embodiment, before obtaining the memory access request instruction, the method also includes: obtaining the code of the external memory access interface, replacing the external memory access interface with a custom external memory access interface; the custom external memory access interface is used to read the data stored in the at least one memory.

[0023] In this embodiment, the data processing system can obtain the code about the external memory access interface in the source code, replace the code about the external memory access interface in the source code with the custom external memory access interface, and use the static calling method to replace the external memory access interface in the source code. The custom external memory access interface is used to obtain the data stored in the external memory, so that the CPU can access the data in the storage space of the external memory.

[0024] In a second aspect, a data processing device is provided in an embodiment of the present application, comprising: a transceiver unit for obtaining cache data; the cache data refers to data that needs to be temporarily stored during the operation of the data processing system; a processing unit for sending part of the cache data to the at least one memory when the amount of cache data of the cache data is greater than the available memory space capacity of the local memory, so that the at least one memory stores part of the cache data.

[0025] In another possible embodiment, the transceiver unit is further used to obtain the code of the memory allocation interface; the processing unit is further used to replace the memory allocation interface with a custom memory allocation interface; the custom memory allocation interface is used to allocate storage space in the local memory and / or the at least one memory; and part of the cache data in the cache data is stored in the allocated storage space in the at least one memory through the custom memory allocation interface.

[0026] In another possible embodiment, the processing unit is also used to configure a transparent memory for each executor; the computing device includes at least one executor, and the computing device uses the executor to store the cache data in the transparent memory configured by the executor, and the transparent memory includes an available memory space capacity of the local memory of a first set size and a storage space of the at least one memory of a second set size.

[0027] In another possible embodiment, the processing unit is specifically used to send a memory allocation request to the at least one memory when the amount of cached data of the cached data is greater than the available memory space capacity of the local memory; the memory allocation request is used to allocate a target storage space for storing the cached data; determine through the custom memory allocation interface whether the target storage space allocated by the memory allocation request is greater than a set memory threshold; when the target storage space allocated by the memory allocation request is greater than the set memory threshold, allocate storage space in the transparent memory configured by the executor, and store part of the cached data in the target storage space.

[0028] In another possible embodiment, the processing unit is further used to send a first request instruction to the at least one memory through the custom memory allocation interface; the first request instruction is used to request the at least one memory to allocate free storage space after the target storage space.

[0029] In another possible embodiment, the processing unit is specifically used to detect the number of times each data in the cache data is accessed and / or stored; and send the first cache data to the at least one memory; the first cache data refers to data in the cache data that is accessed and / or stored less than a set number of times.

[0030] In another possible embodiment, the processing unit is also used to obtain a memory access request instruction; the memory access request instruction is used to read or write specified data, and the memory access request instruction includes a virtual memory address of the specified data; query a locally stored address mapping table to determine a physical storage address mapped to the virtual memory address of the specified data; the address mapping table records a mapping relationship between a virtual memory address and a physical storage address; the physical storage address refers to a physical memory address of the local memory and a physical address of the at least one memory; detect whether the physical storage address of the specified data is the physical memory address of the local memory; if the physical storage address of the specified data is the physical memory address of the local memory, read the data stored in the physical memory address of the specified data, or write the specified data to the corresponding physical memory address.

[0031] In another possible embodiment, the processing unit is also used to allocate a designated storage space in the local memory when the physical storage address of the designated data is not the physical memory address of the local memory; forward the memory access request instruction to the at least one memory; the memory access request instruction is used to request the at least one memory to read the data stored at the physical storage address corresponding to the virtual memory address of the designated data; receive the designated data sent by the at least one memory, and store the designated data in the designated storage space.

[0032] In another possible embodiment, the processing unit is further used to obtain the code of the external memory access interface and replace the external memory access interface with a custom external memory access interface; the custom external memory access interface is used to read the data stored in the at least one memory.

[0033] In a third aspect, an embodiment of the present application provides a computing device, comprising: at least one memory; and at least one processor, wherein the processor is used to execute instructions stored in the memory so that the computing device executes various possible implementations of the first aspect.

[0034] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present application, comprising computer program instructions. When the computer program instructions are executed by the computing device, the computing device executes various possible implementations of the first aspect.

[0035] In a fifth aspect, a computer program product comprising instructions is provided in an embodiment of the present application, characterized in that the computer program product stores instructions, and when the instructions are executed by the computing device, the computing device implements each possible implementation embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0037] Figure 1 A schematic diagram of the hardware architecture of a server cluster deployed in a data processing system provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the software architecture of a data processing system provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of a process for selecting a memory allocator by a scheduling module provided in an embodiment of the present application;

[0040] Figure 4 (a) is a schematic diagram of the process of extending data blocks of the mremap interface in the related art;

[0041] Figure 4 (b) is a schematic diagram of the process of extending the data block of the mremap interface provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the memory of the spark system managed by the cache layer provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a process for modifying the source code of ClickHouse at the interface layer of the scheduling module provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of a process for allocating memory for a cache layer provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of a process of querying the physical memory address of access data at the cache layer provided in an embodiment of the present application;

[0046] Fig. 9 A schematic diagram of a flow chart of a cache layer monitoring memory access behavior provided in an embodiment of the present application;

[0047] Fig.10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0048] Fig.11 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0049] Fig.12 A schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application;

[0050] Fig.13A schematic diagram of the architecture of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0052] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0053] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0056] Before introducing the technical solution protected by this application, several terms appearing in the technical solution protected by this application are introduced, namely:

[0057] Spark is an open source big data processing framework, the full name is Apache spark. Spark is mainly used for distributed computing of large-scale data sets and usually runs in a cluster environment. Spark supports a variety of data sources, such as Hadoop distributed file system (HDFS), Apache Cassandra, ApacheHBase, etc. Spark uses in-memory computing technology, which can load data into memory for operation, thereby greatly accelerating the data processing speed.

[0058] Spark jobs are the units of execution of Spark applications, which are used to describe and execute data processing and analysis tasks. Spark jobs use Spark's distributed computing capabilities and optimization strategies to achieve high-performance, high-concurrency data processing.

[0059] Off-heap refers to a way of allocating memory outside the Java heap. In a Java application, the Java heap can be used to allocate memory space for objects, while off-heap memory is allocated in an unmanaged memory area outside the Java heap.

[0060] The natural language acceleration engine (native engine) vectorized acceleration system refers to a system that uses the natural language acceleration engine and vectorization technology to improve the performance of intensive applications. The natural language acceleration engine uses a specific vectorized instruction set and optimization algorithm to decompose the computing task into multiple subtasks and use vector registers to process multiple subtasks simultaneously.

[0061] A data processing system refers to a software system or a collection of tools used to process massive amounts of data. Data processing systems can effectively store, process, analyze, and manage large amounts of structured, semi-structured, and unstructured data to extract valuable information and insights from them. Common data processing systems can include Apache Hadoop, Apache spark, Apache flink, etc., to provide core components such as distributed storage, distributed computing frameworks, and data processing engines to support the processing and analysis needs of large-scale data. Data processing systems can be deployed on large-scale server clusters to complete data processing tasks, providing high-performance, high-reliability, and high-scalability data processing capabilities.

[0062] When processing large-scale data, the data processing system needs to call on the memory resources of the server cluster to store cached data. As the cached data increases, the required memory resources also increase. If the data processing system cannot effectively manage and process these large-scale data, it will lead to a shortage of memory resources. If the memory resources cannot store all the cached data, the data processing system will frequently report OOM errors, which will reduce the stability of the data processing system.

[0063] In order to solve the defects existing in the related technology, an embodiment of the present application provides a data processing system. When the memory resources of the server or server cluster deployed by the data processing system are insufficient, the pre-stored cache data can be automatically overflowed to the external memory, thereby avoiding the data processing system from frequently reporting OOM errors and improving the stability of the data processing system.

[0064] Figure 1The hardware architecture diagram of a server cluster deployed in a data processing system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the server cluster 100 may include multiple computing devices 110, a memory 120, and a communication bus 130. The multiple computing devices 110 may include a computing device 110-1, a computing device 110-2, a computing device 110-3, etc. The multiple computing devices 110 and the memory 120 may establish a communication connection through the communication bus 130 for data transmission.

[0065] The computing device 110 may be a server, a computer, a smart phone, or other devices. The multiple computing devices 110 may be a single device or a combination of multiple devices. The multiple computing devices 110 may be deployed with applications for running a data processing system to perform functions of the data processing system, such as data cleaning, data storage and management, data conversion and integration, data analysis and mining, real-time data processing, security and privacy protection, etc.

[0066] The computing device 110 may be provided with a storage unit, which may be a cache, a dynamic random access memory (DRAM), a static random access memory (SRAM), or other storage units for temporarily storing data, and is used to store data that needs to be temporarily stored during the operation of the data processing system for access or operation by the processor or other hardware. In an embodiment of the present application, the storage unit may store data corresponding to the computing tasks to be executed by the data processing system, data corresponding to the computing tasks that have not been executed in the interrupted computing tasks, intermediate data during the processing, processed data, and other cached data.

[0067] The computing device 110 may be provided with a communication interface, which may be connected to the communication bus 130 to realize data transmission with other computing devices, storage devices and other devices. The communication interface may be a wired transmission interface, such as a compute express link (CXL) interface, a peripheral component interconnect express (PCIe) interface, a universal serial bus (USB) interface, etc. The communication interface may be a wireless transmission interface, such as a Bluetooth (BT) module, a wireless fidelity (WI-FI) module, a wireless communication module, etc.

[0068] The memory 120 may be an independent storage device or a memory of the computing device 110. The memory 120 may be a hard disk drive (HDD), a solid state drive (SSD), a NAND flash memory, a disk, etc., and is used to store the overflow cache data when multiple computing devices 110 run the data processing system, as well as other data, such as the program running the data processing system. It should be noted that Figure 1 The connection between the memory 120 and the computing device 110 - 3 is not limited to the memory 120 establishing a communication connection with the computing device 110 - 3, but also means that the memory 120 establishes a communication connection with the data processing system deployed by the computing device 110 - 3, so that the memory 120 can establish a communication connection with any computing device 110 .

[0069] The memory 120 may be provided with a communication interface, which may be connected to the communication bus 130 to implement data transmission with multiple computing devices 110 and other devices. The communication interface may be a wired transmission interface, such as a CXL interface, a PCIe interface, a USB interface, etc. The communication interface may be a wireless transmission interface, such as a BT module, a WI-FI module, a wireless communication module, etc.

[0070] The communication bus 130 may refer to communication hardware such as cables, optical fibers, routers, base stations, etc., which are respectively connected to the communication interfaces of the computing device 110 and the communication interfaces of the memory 120, so that communication connections are established between multiple computing devices 110 and the memory 120 for data forwarding.

[0071] Figure 2 Schematic diagram of the software architecture of a data processing system provided in an embodiment of the present application. Figure 2 As shown, the data processing system 200 can be divided into a front end 210, a back end 220 and an operating system (OS) 230 from a software perspective.

[0072] The front end 210 is responsible for the development of user interfaces and applications. In the embodiment of the present application, in the data processing system 200, the front end 210 can be written in a development language such as Java or Scala. The front end 210 can input data, query and display results, as well as perform scheduling and monitoring functions.

[0073] The back end 220 is the core part of the data processing system 200, which is responsible for the processing, calculation and analysis of data. The back end 220 can be written in natural language (native code) (such as C language, C++ language, assembly language or other similar languages). The back end 220 is written in Java language, which can ensure the ease of use of the data processing system 200. The back end 220 is written in C++ language, which can improve the efficiency of the execution instructions of the data processing system 200. Natural language generally refers to C language, C++ language, assembly language or other similar languages, which can interact more directly with computer hardware and OS230. In the embodiment of the present application, the back end 220 can compile C programming language (C programming language, C) library and / or C++ library into a shared library and load it together with libjvm.so to call the memory allocation interface in the C library to allocate the storage space of the external memory. The memory allocation interface may be a memory allocation (malloc) interface, a clear allocation (calloc) interface, a reallocation (reallocate) interface, a memory mapped file (mmap) interface, and other interfaces. The external memory read and write interface may be a read interface, a write interface, and the like.

[0074] OS 230 is the underlying foundation of data processing system 200 and is responsible for managing and controlling hardware resources. OS 230 can provide basic functions such as process scheduling, memory management, file system and network connection, so that front end 210 and back end 220 can run correctly and work together on hardware.

[0075] In the embodiment of the present application, the data processing system 200 can add a scheduling module 240 between the OS 230 and the front end 210 and the back end 220. From a software perspective, the scheduling module 240 can be divided into an interface layer 241, a cache layer 242 and a driver layer 243.

[0076] The interface layer 241 is mainly responsible for docking the C library interface called by the underlying native code of the backend 220. In the embodiment of the present application, after the interface layer 241 docks with the interface of the backend 220, a custom memory allocation interface 2411, a custom external memory access interface 2412 and a custom memory access interface 2413 are generated.

[0077] Typically, the data processing system 200 has a memory allocation interface, which may be a malloc interface, a calloc interface, a realloc interface, a new interface, a new[] interface, an mmap interface, a memory remap (memory remap, mremap) interface, a delete interface, a delete[] interface, a free interface, and the like.

[0078] The Malloc interface is used to dynamically allocate a memory block of a specified size and return a pointer to the allocated memory. The Calloc interface is used to dynamically allocate a memory block of a specified number and size and initialize the allocated memory block to zero. The Realloc interface is used to reallocate the size of an allocated memory block and can increase or decrease the size of a memory block. The New interface is used to dynamically allocate memory for a single object and call the constructor for initialization. The New interface returns a pointer to the newly created object. The New[] interface is also used in C++ to dynamically allocate memory for an array of objects and call the constructor of each object for initialization. The Mmap interface is a system-level interface used to map files in memory or allocate anonymous memory areas. The Mremap interface is used to remap an allocated memory area in memory and adjust the size and position of the memory area. The Delete interface is used to release the memory of a single object created by the new operator and call the object's destructor for cleanup. The Delete[] interface is used to release the memory of an array of objects created by the new[] operator and call the destructor of each object for cleanup. The Free interface is used to release a memory block allocated by the malloc interface, calloc interface, or realloc interface.

[0079] In an embodiment of the present application, the interface layer 241 can obtain the code about the memory allocation interface in the source code, replace the code about the memory allocation interface in the source code with a custom memory allocation interface 2411, and use a static call method to replace the memory allocation interface in the source code. The custom memory allocation interface 2411 is used to allocate storage space in local memory or external memory so that the CPU can access the data in the memory space. For example, the malloc interface is replaced with a transparent memory allocation (transparent_malloc) interface. The interface layer 241 uses a static call method and does not use a dynamic interception method for replacement, which can effectively control the interface interception range and avoid the impact on other unrelated components in the data processing system 200. The source code refers to the code corresponding to the application program running the data processing system 200.

[0080] The interface layer 241 uses low-level virtual machine (LLVM) plug-in technology to replace all memory access instructions (such as load instructions, store instructions, memmove instructions, etc.) in the source code during the code compilation stage to achieve transparency of the application.

[0081] Normally, most of the memory allocation requests in the source code are responsible for allocating temporary data. The amount of temporary data occupying memory is relatively small, and it is released soon after use. If temporary data is overflowed to the external memory, the performance of the data processing system 200 will be lost. In addition, a memory allocator is integrated in the shared library, which can intercept the memory allocation interface in the program code that calls the shared library. Since the memory access instruction in the program code that calls the shared library has not been replaced, it will cause errors in the operation of the application program, so the interface layer 241 cannot be compatible with the Java virtual machine (Java virtualmachine, JVM) plug-in mechanism, and cannot run simultaneously with other memory allocators in the data processing system 200.

[0082] In order to solve the defects of the interface layer 241 using the LLVM plug-in technology, the interface layer 241 can identify the memory allocation interface in the source code that is prone to OOM and the memory access instructions in the related code, which are called "target memory allocation interface" and "target memory access instruction". The target memory allocation interface usually allocates large blocks of memory used for a long time. Then, the interface layer 241 can use an automated analysis tool to identify the target memory allocation interface and target memory access instruction that are prone to OOM from the Spark system source code. Use a custom memory allocation interface 2411 to replace the target memory allocation interface. The interface layer 241 can use the LLVM plug-in technology to only replace the target memory access request in the associated source code fragment. The interface layer 241 can request that the memory allocation request for a large block of memory overflow to the external memory to avoid the performance loss of the data processing system 200. The interface layer 241 uses a static call algorithm to replace the memory allocation interface that requests a large block of memory, and will not intercept the memory allocation interface that calls the shared library program, and will not cause errors in the operation of related applications.

[0083] Typically, the data processing system 200 has an external memory access interface, which may be a read interface, a write interface, and the like.

[0084] In the embodiment of the present application, the interface layer 241 can obtain the code about the external memory access interface in the source code, replace the code about the external memory access interface in the source code with the custom external memory access interface 2412, and use the static call method to replace the external memory access interface in the source code. The custom external memory access interface 2412 is used to obtain the data stored in the external memory, so that the CPU can access the data in the storage space of the external memory. For example, the read interface is replaced with a transparent read (transparent_read) interface.

[0085] Typically, the data processing system 200 has a memory access interface, which may be a load interface, a store interface, a memory copy (memcpy) interface, a memory move (memove) interface, a string copy (strcpy) interface, a string concatenation (strcat) interface, etc.

[0086] The Load interface is used to read data from the memory to the specified register or variable, usually provided by instructions or functions similar to ld or load. The Store interface is used to write data from a register or variable to a specified location in the memory, usually provided by instructions or functions similar to st or store. The Memcpy interface is used to copy data from one block of memory to another block of memory. It is usually used to handle data copy operations between memory blocks and provides efficient and reliable memory copy functions. The Memmove interface is similar to the memcpy interface and is used for data copy operations between memory blocks, but is safer when handling overlapping memory and can handle situations where the source memory block and the target memory block partially or completely overlap. The Strcpy interface is used to copy a string from the source memory to the target memory until a null character '\0' is encountered. To use the strcpy interface, it is usually necessary to ensure that the target memory has enough space to accommodate the copied string. The Strcat interface is used to append a string to the end of another string, and it is also necessary to ensure that the target string has enough space to accommodate the result of the append.

[0087] In the embodiment of the present application, the interface layer 241 can obtain the code about the memory access interface in the source code, compile the code about the memory access interface in the source code into the target code, and use the static call method to replace the memory access interface in the source code to obtain the customized memory access interface 2413. The customized memory access interface 2413 is used to access data in the local memory and the external memory. For example, the load interface is replaced with a transparent load (transparent_load) interface.

[0088] Traditional memory allocation interfaces (such as malloc interface, mmap interface, etc.) generally allocate a continuous virtual memory address from the user's virtual memory space, and then the CPU uses memory access instructions (such as load instructions, store instructions, etc.) to access the data in the virtual memory address. However, under the control and management of the OS and hardware, the virtual memory address will be automatically converted to a physical memory address through the address mapping of the translation lookaside buffer (TLB) or page table entry (PTE), so that the CPU can obtain cache data from the physical memory.

[0089] In the embodiment of the present application, the custom memory access interface 2413 can access data in the local memory and data in the external memory. The custom memory access interface 2413 can map the virtual memory address to the physical memory and map the virtual memory address to the external memory, so that the CPU can use the custom memory access interface 2413 to obtain data from the local memory and obtain data from the external memory.

[0090] During the compilation phase, the interface layer 241 can use LLVM static calls to replace the memory allocation interface 2411. Then, the interface layer 241 uses automated analysis tools to accurately control the replacement range of memory allocation requests (such as load instructions, store instructions, etc.). The interface layer 241 detects whether the upstream memory allocation interface has been replaced, and after the upstream memory allocation interface has been replaced, it can detect the memory access instructions related to the replaced memory allocation interface, and only replace these related memory access instructions with the custom memory access interface 2413, and do not replace irrelevant memory access instructions, thereby avoiding more serious performance losses.

[0091] Since the memory allocator in the related art may have problems such as slow allocation speed, many fragments, memory leaks, etc., the interface layer 241 can use lightweight multi-memory allocator management technology and be deployed locally. After the interface layer 241 receives the memory allocation request at the customized memory allocation interface 2411, it can first use the cache management algorithm for processing, and then choose to use a user-specified memory allocator for memory allocation. The current mainstream memory allocators include thread-specific memory allocators (per thread memory allocator, PTMALLOC), Jemma memory allocator (Jason Evans Memory Allocator, JEMALLOC), Microsoft memory allocator (Microsoft Allocator, MIMALLOC), etc. Different types of memory allocators have their own advantages in different scenarios, so the interface layer 241 can flexibly select different types of memory allocators according to demand.

[0092] For example, Figure 3 The following is a flow chart of the process of selecting a memory allocator by the scheduling module provided in the embodiment of the present application. Figure 3 As shown, the process of selecting the memory allocator is performed by the interface layer 241, and the implementation process is specifically as follows:

[0093] Step S301 : the interface layer 241 calls the user-defined memory allocation interface 2411 , and receives a memory allocation request through the user-defined memory allocation interface 2411 .

[0094] Step S302: before the application is started, the interface layer 241 imports the "determination condition" and the "selected memory allocator" from the configuration parameters.

[0095] Step S303, the interface layer 241 determines whether the memory allocation request meets the determination condition. In one case, the interface layer 241 determines that the memory allocation request does not meet the determination condition, and executes step S304. In another case, the interface layer 241 determines that the memory allocation request meets the determination condition, and executes step S305. The determination condition may be that the memory block requested to be allocated by the memory allocation request is larger than the set memory.

[0096] Step S304: the interface layer 241 puts the memory allocation request into the OS memory management mechanism for management.

[0097] Step S305: the interface layer 241 puts the memory allocation request into the transparent memory for management.

[0098] Transparent memory refers to an unused virtual memory space reserved in the OS. Taking the spark system as an example, each computing device 110 can run at least one executor, and the computing device 110 can store data in the transparent memory configured by the executor. The transparent memory includes the available memory space capacity of the local memory of a first set size and the storage space of the external memory of a second set size.

[0099] Exemplarily, taking the data processing system 200 spark system as an example, when the spark system is running, the interface layer 241 can set the judgment condition that the memory block requested to be allocated by the memory allocation request is greater than 1MB. If the interface layer 241 determines that the memory block requested to be allocated by the memory allocation request is less than or equal to 1MB, the memory allocation request can be put into the OS memory management mechanism for management. If the interface layer 241 determines that the memory block requested to be allocated by the memory allocation request is greater than 1MB, the memory allocation request can be put into the transparent memory for management.

[0100] Step S306, the interface layer 241 selects a corresponding memory allocator. The memory allocators selected by the interface layer 241 may include PTMALLOC, JEMALLOC, MIMALLOC, etc. For example, if the interface layer 241 determines that the memory block requested to be allocated by the memory allocation request is greater than 1MB, JEMALLOC may be selected as the memory allocator.

[0101] Normally, data processing system 200 performs mremap and realloc operations to virtual memory space, which can cause the copy of cached data. Therefore, interface layer 241 can statically call self-defined memory allocation interface 2411, such as transparent_mremap interface, transparent_realloc interface etc., and self-defined memory allocation interface 2411 is deployed locally. Interface layer 241 can utilize self-defined memory allocation interface 2411, and reserve enough virtual memory spaces when distributing for the first time, for future expansion. When interface layer 241 distributes again at a subsequent moment, after receiving mremap instruction, it can expand in the reserved virtual memory space, expand the memory block distributed last time into the memory block of larger internal memory, and does not need to copy the data of the memory block distributed last time, and realizes the zero copy of cached data. Zero copy refers to that the data in the original memory block do not need to be copied when extending the memory block.

[0102] In one embodiment, Figure 4(a) shows, mremap interface of the correlation technology carries out mmap operation at T1 moment, distributes a plurality of memory blocks continuously in virtual memory space, and the size of each memory block is 16KB.At T2 moment, when mremap interface of the correlation technology carries out mremap operation, a plurality of memory blocks need to be expanded into the memory blocks of 2 times of size.At this moment, mremap interface of the correlation technology distributes a 32KB memory block in the virtual memory space behind a plurality of memory blocks, and then copies the data inside the memory block of first 16KB, and stores the copied data in the memory block of 32KB.

[0103] like Figure 4 (b) Shown, the application's self-defined mremap interface performs mmap operation at T1 moment, and multiple memory blocks can be allocated in virtual memory space, and the free storage space of setting size is reserved between each memory block. The reserved free storage space is greater than 16KB. At T2 moment, when the application's self-defined mremap interface performs mremap operation, it is necessary to expand multiple memory blocks into memory blocks of 2 times of size. Now, the application's self-defined mremap interface can allocate a 16KB storage space in the reserved storage space between two memory blocks, and merge this storage space with the first memory block, and realize that the size of the first memory block is expanded to 32KB from 16KB.

[0104] Optionally, the customized memory allocation interface 2413 may send a first request instruction to the external memory. After allocating storage space according to the memory allocation request, the external memory may allocate free storage space after the allocated storage space according to the first request instruction.

[0105] The cache layer 242 can manage local memory and external memory. Take the data processing system 200 as a spark system as an example. Figure 5 As shown, when the CPU executes a spark job, after specifying a certain capacity of off-heap memory space for each executor, a part of the memory space is allocated from the off-heap memory space. Each executor can combine the allocated storage space of the local memory and the storage space of the external memory to form a transparent memory. Among them, each computing device 110 can run at least one executor, and the computing device 110 can store data in the transparent memory configured by the executor. Local memory refers to the storage unit of each computing device 110, and external memory refers to the storage unit of the memory 120. Transparent memory can record the access frequency of data in the cache, and manage hot and cold data in the data.

[0106] The cache layer 242 can store part of the data in the external memory in the transparent memory to avoid frequent OOM error reporting when the memory resources of multiple computing devices 110 deploying the data processing system 200 are insufficient. Preferably, the cache layer 242 can detect the hotness of the cached data, put the hot data in the local memory, and put the cold data in the external memory, so as to avoid reducing the memory access performance of the data processing system 200.

[0107] The cache layer 242 can manage the mapping of virtual memory addresses to physical memory addresses. The cache layer 242 can store an address mapping table locally, and the address mapping table records the mapping relationship between the virtual memory address and the physical storage address. The address mapping table in the related art can only map the virtual memory address to the physical memory address, and cannot be mapped to the external storage. The address mapping table protected by this application can not only map the virtual memory address to the local physical memory address, but also map it to the physical address of the external memory. The physical storage address refers to the physical memory address of the local memory and the physical address of the external memory.

[0108] Take the data processing system 200 as a spark system as an example. Figure 5 As shown, when the CPU executes a spark job, it can call the custom memory allocation interface 2411 provided by the interface layer 241 to allocate a virtual memory address from the storage space of the transparent memory. When the data storage system 200 executes an application, the cache layer 242 can use the custom memory access interface 2413 to access data in the transparent memory. For example, when the CPU uses a load instruction or a store instruction to access data in the memory, the custom memory access interface 2413 can be used to replace the load instruction or the store instruction.

[0109] After receiving the memory access request instruction, the custom memory access interface 2413 can parse out the virtual memory address carried by the memory access request instruction. The memory access request instruction is used to read the specified data, and the virtual memory address of the memory access request instruction refers to the virtual address of the accessed data. The custom memory access interface 2413 queries the address mapping table of the local storage to obtain the physical storage address mapped by the virtual memory address. The custom memory access interface 2413 can detect whether the physical storage address of the specified data is the physical memory address of the local memory. In one case, the custom memory access interface 2413 determines that the physical storage address of the specified data is the physical memory address of the local memory, and returns the physical memory address to the cache layer 242.

[0110] In another case, the custom memory access interface 2413 determines that the physical storage address of the specified data is not the physical memory address of the local memory, and can call the cache replacement algorithm to allocate a storage space of a set size in the local memory, and forward the memory access request instruction to the external memory. After the external memory obtains the virtual memory address of the memory access request instruction, it sends the data stored in the physical storage address corresponding to the virtual memory address of the memory access request instruction to a computing device 110, so that the computing device 110 caches the read data in the local memory allocated to the storage space. After the custom memory access interface 2413 determines that the external memory caches the accessed data in the local memory allocated to the storage space, it returns the physical memory address cached in the local memory to the cache layer 242. The custom memory access interface 2413 can record the mapping relationship between the virtual memory address of the accessed data and the physical memory address corresponding to the storage space allocated to the local memory in the address mapping table.

[0111] The cache layer 242 can monitor the behavior of accessing memory. The cache layer 242 can set a counter in the custom memory access interface 2413 to record the number of times each memory block is accessed during the running of the application program, and divide different memory blocks into hot and cold levels as the basis for cache replacement. The cache layer 242 can record the maximum value of the storage space of the data overflowing to the external memory, and add the maximum value of the storage space of the external memory to the local memory outside the heap, so as to accurately estimate the memory usage required for the task to run. The cache layer 242 uses the custom memory allocation interface 2411 in the interface layer 241 to easily transmit software information, and combines the memory access behavior of the custom memory access interface 2413 to monitor the memory access behavior, so as to track the object memory access behavior of the spark job at the software layer. The cache layer 242 uses the analysis algorithm to easily summarize the memory access characteristics of the spark job, such as streaming memory access, cyclic memory access, random memory access, etc. The cache layer 242 uses the summarized memory access characteristics to accurately predict the future and adaptively adjust the cache management algorithm. The cache layer 242 can obtain the optimal cache hit rate to improve the performance of the entire cache layer 242.

[0112] The access behavior of the application program at each stage is constantly changing, so there are random access, streaming access, cyclic access and other methods. If the access behavior of the application program changes and the cache management algorithm is not adjusted accordingly, the cache hit rate will be reduced, and the performance of the data processing system 200 will be reduced. The cache layer 242 tracks the memory access behavior of the application program, and can observe and predict the changes in the memory access behavior of the application program. For example, the cache layer 242 determines that the memory access behavior of the application program changes from random access to streaming access, and can adjust the cache management algorithm at the same time to ensure that the cache hit rate is always maintained at a high level, avoiding the loss of memory access performance of the data processing system 200. The data processing system of the related art can only try to increase the memory usage after the spark job encounters OOM failure through multiple reruns. And after the spark job of the related art succeeds, you can try to reduce the memory usage. After repeated retries, the data processing system of the related art can estimate the accurate memory usage for a spark job. The cache layer 242 in the embodiment of the present application records the maximum value of the storage space of the data overflow to the external memory, so that the accurate memory usage can be estimated with only one successful run, thereby reducing the time cost of manual tuning.

[0113] The driver layer 243 can deploy the input output (IO) driver of the user state of OS230, such as asynchronous IO (AIO), IO ring (IO ring), storage performance development kit (SPDK), etc. After the user state of OS230 completes most of the IO operations through the driver layer 243, the number of user state and kernel state switching in the traditional IO stack can be reduced, avoiding the performance overhead caused by frequent context switching.

[0114] The following takes the data processing system 200 as a spark system as an example to introduce the implementation process of the technical solution protected by this application.

[0115] The technical solution protected by this application can be applied to the spark system, and can be applied to other systems with insufficient stability due to OOM. When the Spark system executes a spark job, when memory resources are insufficient, the data cached in the memory can be automatically overflowed to the external memory. Compared with independent operation of the memory, the data cached in the memory automatically overflows to the external memory, and the performance of the data processing system 200 is reduced. When the data processing system 200 needs to use the external memory, it is necessary to configure the parameters of the external memory in advance, such as specifying the storage directory of the external memory. The cache layer 242 in the big data storage system 200 counts some advanced memory optimization features, such as lightweight multi-memory allocator management, custom mremap interface, cache replacement algorithm, data prefetching algorithm, etc., and configuration parameters need to be set in advance.

[0116] The technical solution protected by this application can be deployed in a natural language acceleration engine vectorization acceleration system for the spark system. The natural language acceleration engine vectorization acceleration system can be an open source tool used in combination with gluten and clickhouse. Gluten is a Java-based middleware for streaming and processing data. Clickhouse is a column-oriented distributed database management system with high scalability and high performance, and can better handle massive amounts of data. The natural language acceleration engine vectorization acceleration system combines gluten and clickhouse, and can load data into memory through gluten, and then use clickhouse for efficient data query and processing, which can improve response time, processing efficiency, and reduce processing costs and energy consumption.

[0117] Compared with the related art spark system that manages memory space in JVM, the natural language acceleration engine vectorized acceleration system uses the spark plug-in mechanism to intercept the spark query plan in the middle layer gluten, and send the spark query plan to the low-level vectorized engine clickhouse, which executes the query task and can skip the inefficient execution path of the native spark system. Gluten directly calls the natural language (native) code in the spark executor task thread in the form of Java native interface (JNI) calling shared libraries, without introducing a complex thread model.

[0118] For example, Figure 6 A schematic diagram of the process of modifying the source code of clickhouse at the interface layer of the scheduling module provided in the embodiment of the present application. Figure 6 As shown, the process is performed by the interface layer 241 of the scheduling module 240, and the specific implementation process is as follows:

[0119] Step S601, the interface layer 241 obtains the source code of clickhouse.

[0120] Step S602 : the interface layer 241 uses the user-defined memory allocation interface 2411 to replace the memory allocation interface in the C library.

[0121] Step S603 , the interface layer 241 determines the upstream and downstream dependent libraries of the custom memory allocation interface 2411 .

[0122] Step S604: the interface layer 241 replaces the memory access interface in the affected dependent library during the compilation phase.

[0123] Step S605 , the interface layer 241 generates a shared library libch.so.

[0124] In an embodiment of the present application, the interface layer 241 of the scheduling module 240 is connected to the vectorization engine clickhouse, and can be used as a component of clickhouse to participate in compilation. The interface layer 241 can obtain the source code of clickhouse to call the interface of the C library. The interface layer 241 can use the custom memory allocation interface 2411 to replace the memory allocation interface in the C library, and replace it with malloc interface, calloc interface, realloc interface, mmap interface, mremap interface, new interface, new[] interface, delete interface, delete[] interface, free interface or other interface. The interface layer 241 uses an automated analysis tool to clarify the upstream and downstream libraries of the custom memory allocation interface 2411. When the interface layer 241 uses the LLVM plug-in for the compilation phase, it can replace the memory allocation requests in the dependent library affected by the compilation, such as load, store, memcpy, memset and other instructions. After compilation, the interface layer 241 generates a shared libch.so and provides it to gluten for loading and use.

[0125] For example, Figure 7 A schematic diagram of a process for memory allocation of a cache layer provided in an embodiment of the present application. Figure 7 As shown, the process is performed by the cache layer 242 of the scheduling module 240, and the specific implementation process is as follows:

[0126] Step S701: the cache layer 242 uses the JNI of the JVM to call the shared database.

[0127] Step S702 : the cache layer 242 obtains the natural language code in the shared database.

[0128] Step S703: the cache layer 242 uses the custom memory allocation interface 2411 to obtain the memory corresponding to the memory allocation request.

[0129] In step S704, the cache layer 242 detects whether the memory corresponding to the acquired memory allocation request is greater than the determination condition. In one case, the cache layer 242 determines that the memory corresponding to the acquired memory allocation request is greater than the determination condition, and executes step S705. In another case, the cache layer 242 determines that the memory corresponding to the acquired memory allocation request is less than or equal to the determination condition, and executes step S706.

[0130] Step S705: the cache layer 242 obtains the user's virtual memory space.

[0131] Step S706: the cache layer 242 obtains the virtual memory space of the transparent memory.

[0132] In the embodiment of the present application, the cache layer 242 is responsible for processing the allocation request issued by the interface layer 241. Before executing the spark job, the CPU will set a judgment condition. The judgment condition can be set to put the memory allocation request exceeding 1MB into the transparent memory address space for management. After receiving the allocation request issued by the interface layer 241, the cache layer 242 will make a selection based on the judgment condition. In one case, the cache layer 242 determines that the allocation request does not meet the judgment condition and can be managed in the traditional user's virtual memory space. In another case, the cache layer 242 determines that the allocation request meets the judgment condition and can be put into the virtual memory space of the transparent memory for management.

[0133] For example, Figure 8 A schematic diagram of a process of querying the physical memory address of access data at the cache layer provided in an embodiment of the present application. Figure 8 As shown, the process is performed by the cache layer 242 of the scheduling module 240, and the specific implementation process is as follows:

[0134] Step S801: the cache layer 242 receives a memory access request instruction.

[0135] Step S802: After the cache layer 242 determines that the custom memory access interface 2413 receives a memory access request instruction, it queries the address mapping table.

[0136] In step S803, the cache layer 242 determines whether the physical storage address is the physical memory address of the local memory. In one case, the cache layer 242 determines that the physical storage address is not the physical memory address of the local memory, and executes step S804. In another case, the cache layer 242 determines that the physical storage address is the physical memory address of the local memory, and executes step S805.

[0137] Step S804 , the cache layer 242 uses a cache replacement algorithm to swap the data in the external memory into the local memory through the driver layer 243 .

[0138] Step S805: the cache layer 242 returns the physical memory address.

[0139] In an embodiment of the present application, the cache layer 242 is responsible for processing the memory access request instruction issued by the interface layer 220. After the cache layer 242 determines that the custom memory access interface 2413 receives the memory access request instruction, it will query the address mapping table to obtain the physical storage address mapped to the virtual memory address. The custom memory access interface 2413 can detect whether the physical storage address of the specified data is the physical memory address of the local memory. In one case, the custom memory access interface 2413 determines that the physical storage address of the accessed data is the physical memory address of the local memory, and returns the physical memory address to the cache layer 242.

[0140] In another case, the custom memory access interface 2413 determines that the physical storage address of the specified data is not the physical memory address of the local memory, and can call the cache replacement algorithm to select a cold data block from the physical memory space. The cache layer 242 can send an IO request to the external memory to write the cold data block to the external memory, or read the accessed data to the selected cold data block location. After the cache layer 242 determines that the IO operation is completed, it can return the physical memory address corresponding to the accessed data, and record the mapping relationship between the virtual memory address and the physical memory address of the accessed data, and between the virtual address of the cold data block and the physical address of the external memory in the address mapping table, so that the data processing system can successfully access the data stored in the external memory.

[0141] For example, Fig. 9 A schematic diagram of a process flow of monitoring memory access behavior of a cache layer provided in an embodiment of the present application. Fig. 9 As shown, the process is performed by the cache layer 242 of the scheduling module 240, and the specific implementation process is as follows:

[0142] Step S901: the cache layer 242 creates an object information table.

[0143] Step S902: the cache layer 242 analyzes memory access characteristics in real time according to the object information table.

[0144] Step S903: the cache layer 242 selects an appropriate management strategy according to the memory access characteristics.

[0145] In an embodiment of the present application, the cache layer 242 is responsible for monitoring the memory access behavior of the spark job. After the cache layer 242 determines that the custom memory allocation interface 2411 receives a memory allocation request, a record can be created in the object information table. The record includes information such as the size of the allocated memory, the physical memory address of the allocated memory, and the memory access frequency. When the cache layer 242 detects that the custom memory access interface 2413 is called, the memory access frequency information in the object information table can be updated in real time. The cache layer 242 can use a local built-in analysis algorithm to analyze memory access characteristics in real time, such as streaming access, sequential access, random access, etc. The cache layer 242 can select appropriate cache replacement algorithms and data prefetching algorithms based on the memory access characteristics.

[0146] Fig.10 Schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Fig.10 As shown, the data processing device 1000 can be divided into a transceiver unit 1010 and a processing unit 1020 according to the execution function. The specific implementation process of the data processing device 1000 is as follows:

[0147] The transceiver unit 1010 is used to obtain cache data. The cache data refers to data that needs to be temporarily stored during the operation of the data processing system. The processing unit 1020 is used to send part of the cache data to at least one memory when the amount of cache data of the cache data is greater than the available memory space capacity of the local memory, so that the at least one memory stores part of the cache data.

[0148] In another possible implementation, the transceiver unit 1010 is further used to obtain the code of the memory allocation interface. The processing unit 1020 is further used to replace the memory allocation interface with a custom memory allocation interface. The custom memory allocation interface is used to allocate storage space in the local memory and / or at least one memory. The processing unit 1020 is also used to store part of the cached data in the storage space allocated in the at least one memory through the custom memory allocation interface.

[0149] In another possible implementation, the processing unit 1020 is further configured to configure a transparent memory for each executor. The computing device includes at least one executor. The computing device uses the executor to store cache data in the transparent memory configured by the executor. The transparent memory includes an available memory space capacity of a local memory of a first set size and a storage space of at least one memory of a second set size.

[0150] In another possible implementation, the processing unit 1020 is specifically used to send a memory allocation request to at least one memory when the amount of cached data of the cached data is greater than the available memory space capacity of the local memory. The memory allocation request is used to allocate a target storage space for storing the cached data. The processing unit 1020 is specifically used to determine whether the target storage space requested to be allocated by the memory allocation request is greater than a set memory threshold through a custom memory allocation interface. The processing unit 1020 is specifically used to allocate storage space in the transparent memory configured by the executor when the target storage space requested to be allocated by the memory allocation request is greater than the set memory threshold, and store part of the cached data in the cached data in the target storage space.

[0151] In another possible implementation, the processing unit 1020 is further configured to send a first request instruction to at least one memory through a custom memory allocation interface. The first request instruction is used to request at least one memory to allocate free storage space after the target storage space.

[0152] In another possible implementation, the processing unit 1020 is specifically configured to detect the number of times each data in the cache data is accessed and / or stored. The processing unit 1020 is specifically configured to send the first cache data to at least one memory. The first cache data refers to data in the cache data that is accessed and / or stored less than a set number of times.

[0153] In another possible embodiment, the processing unit 1020 is also used to obtain a memory access request instruction. The memory access request instruction is used to read or write specified data, and the memory access request instruction includes a virtual memory address of the specified data. The processing unit 1020 is also used to query the address mapping table stored locally to determine the physical storage address mapped to the virtual memory address of the specified data. The address mapping table records the mapping relationship between the virtual memory address and the physical storage address. The physical storage address refers to the physical memory address of the local memory and the physical address of at least one memory. The processing unit 1020 is also used to detect whether the physical storage address of the specified data is the physical memory address of the local memory. The processing unit 1020 is also used to read the data stored in the physical memory address of the specified data, or write the specified data to the corresponding physical memory address when the physical storage address of the specified data is the physical memory address of the local memory.

[0154] In another possible implementation, the processing unit 1020 is further configured to allocate a designated storage space in the local memory when the physical storage address of the designated data is not the physical memory address of the local memory. The processing unit 1020 is further configured to forward the memory access request instruction to at least one memory. The memory access request instruction is used to request at least one memory to read the data stored at the physical memory address of the designated data. The processing unit 1020 is further configured to receive the designated data sent by at least one memory, and store the designated data in the designated storage space.

[0155] In another possible implementation, the transceiver unit 1010 is further used to obtain the code of the external memory access interface. The processing unit 1020 is further used to replace the external memory access interface with a custom external memory access interface. The custom external memory access interface is used to read data stored in at least one memory.

[0156] Fig.11 Schematic diagram of a computing device provided in an embodiment of the present application. Fig.11 As shown, the computing device 1100 includes a bus 1110, a processor 1120, a memory 1130, and a communication interface 1140. The processor 1120, the memory 1130, and the communication interface 1140 communicate with each other through the bus 1110. The computing device 1100 may be a server, a computer, a portable notebook, etc. It should be understood that the present application does not limit the number of processors and memories in the computing device 1100.

[0157] The bus 1110 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 The bus 1110 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 1110 may include a path for transmitting information between various components of the computing device 1100 (for example, the processor 1120, the memory 1130, and the communication interface 1140).

[0158] The processor 1120 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0159] The memory 1130 may include a volatile memory, such as a random access memory (RAM). The memory 1130 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0160] The memory 1130 stores executable program codes, and the processor 1120 executes the executable program codes to respectively implement the functions of the aforementioned multiple modules, such as the interface layer 241, the cache layer 242, the driver layer 243, etc., so as to implement the data processing method. That is, the memory 1130 stores instructions for executing the data processing method.

[0161] Alternatively, the memory 1130 stores executable codes, and the processor 1120 executes the executable codes to respectively implement the functions of the aforementioned modules, thereby implementing the data processing method. That is, the memory 1130 stores instructions for executing the data processing method.

[0162] The communication interface 1140 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0163] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0164] like Fig.12 As shown, the computing device cluster includes at least one computing device 1100. The memory 1130 in one or more computing devices 1100 in the computing device cluster may store the same instructions for executing the data processing method.

[0165] In some possible implementations, the memory 1130 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the data processing method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the data processing method.

[0166] It should be noted that the memory 1130 in different computing devices 1100 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the above-mentioned multiple modules. That is, the instructions stored in the memory 1130 in different computing devices 1100 can implement the functions of one or more modules in the above-mentioned multiple modules.

[0167] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.13 A possible implementation is shown. Fig.13 As shown, two computing devices are connected via a network, namely computing device 1100A and computing device 1100B. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1130 in the computing device 1100A stores instructions for executing the functions of some of the above-mentioned multiple modules. At the same time, the memory 1130 in the computing device 1100B stores instructions for executing the functions of another part of the above-mentioned multiple modules.

[0168] Fig.13 The connection method between the computing device clusters shown may be based on the consideration that the data processing method provided in the present application requires a large amount of data storage, and therefore the functions implemented by another part of the above-mentioned multiple modules may be handed over to the computing device 1100B for execution.

[0169] It should be understood that Fig.13 The functions of the computing device 1100A shown in FIG. 1100A may also be completed by multiple computing devices 1100. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices 1100.

[0170] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.11 and Fig.12 The connection mode of the computing device cluster is different in that the memory 1130 in one or more computing devices 1100 in the computing device cluster may store the same instructions for executing the data processing method.

[0171] In some possible implementations, the memory 1130 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the data processing method. In other words, the combination of one or more computing devices 1100 may jointly execute instructions for executing the data processing method.

[0172] It should be noted that the memory 1130 in different computing devices 1100 in the computing device cluster may store different instructions for executing partial functions of the computing device 1100. That is, the instructions stored in the memory 1130 in different computing devices 1100 may implement the functions of one or more of the above multiple modules.

[0173] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the data processing method.

[0174] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the data processing method.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method, characterized in that: The data processing system is deployed in a server cluster, the server cluster includes multiple computing devices and at least one memory, and the method is executed by any one or more computing devices among the multiple computing devices, including: Acquire cache data; the cache data refers to data that needs to be temporarily stored during the operation of the data processing system; When the cached data amount of the cached data is greater than the available memory space capacity of the local memory, part of the cached data is sent to the at least one memory so that the at least one memory stores part of the cached data.

2. The method according to claim 1, characterized in that The sending part of the cached data to the at least one memory specifically includes: Obtaining a code of a memory allocation interface, and replacing the memory allocation interface with a custom memory allocation interface; the custom memory allocation interface is used to allocate storage space in the local memory and / or the at least one memory; Part of the cache data is stored in the storage space allocated in the at least one memory through the customized memory allocation interface.

3. The method according to claim 1 or 2, characterized in that: In a case where the cached data amount of the cached data is greater than the available memory space capacity of the local memory, before sending part of the cached data to the at least one memory, the method further includes: A transparent memory is configured for each executor; the computing device includes at least one executor, and the computing device uses the executor to store the cache data in the transparent memory configured by the executor, wherein the transparent memory includes the available memory space capacity of the local memory of a first set size and the storage space of the at least one memory of a second set size.

4. The method according to claim 3, characterized in that The storing part of the cache data in the storage space allocated in the at least one memory through the customized memory allocation interface specifically includes: When the cached data amount of the cached data is greater than the available memory space capacity of the local memory, a memory allocation request is sent to the at least one memory; the memory allocation request is used to allocate a target storage space for storing the cached data; Determining, through the custom memory allocation interface, whether the target storage space allocated by the memory allocation request is greater than a set memory threshold; When the target storage space allocated by the memory allocation request is greater than the set memory threshold, storage space is allocated in the transparent memory configured by the executor, and part of the cache data is stored in the target storage space.

5. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: A first request instruction is sent to the at least one memory through the custom memory allocation interface; the first request instruction is used to request the at least one memory to allocate free storage space after the target storage space.

6. The method according to any one of claims 1 to 5, characterized in that: When the amount of cached data of the cached data is greater than the available memory space capacity of the local memory, sending part of the cached data to the at least one memory specifically includes: Detecting the number of times each data in the cache data is accessed and / or stored; Sending first cache data to the at least one memory; the first cache data refers to data in the cache data whose number of accesses and / or storages is less than a set number of times.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtaining a memory access request instruction; the memory access request instruction is used to read or write specified data, and the memory access request instruction includes a virtual memory address of the specified data; Querying a locally stored address mapping table to determine a physical storage address of a virtual memory address mapping of the specified data; the address mapping table records a mapping relationship between a virtual memory address and a physical storage address; the physical storage address refers to a physical memory address of the local memory and a physical address of the at least one memory; Detecting whether the physical storage address of the specified data is the physical memory address of the local memory; In the case where the physical storage address of the designated data is the physical memory address of the local memory, the data stored in the physical memory address of the designated data is read, or the designated data is written to the corresponding physical memory address.

8. The method according to claim 7, characterized in that The method further comprises: In a case where the physical storage address of the designated data is not the physical memory address of the local memory, allocating designated storage space in the local memory; forwarding the memory access request instruction to the at least one memory; the memory access request instruction is used to request the at least one memory to read the data stored at the physical storage address of the specified data; The designated data is received from the at least one memory, and the designated data is stored in the designated storage space.

9. The method according to claim 7 or 8, characterized in that: Before obtaining the memory access request instruction, the method further includes: Obtain the code of the external memory access interface, and replace the external memory access interface with a custom external memory access interface; the custom external memory access interface is used to read the data stored in the at least one memory.

10. A data processing device, characterized in that: include: A transceiver unit, used for obtaining cache data; The cache data refers to data that needs to be temporarily stored during the operation of the data processing system; The processing unit is used to send part of the cached data to the at least one memory when the cached data amount of the cached data is greater than the available memory space capacity of the local memory, so that the at least one memory stores part of the cached data.

11. The device according to claim 10, characterized in that The transceiver unit is also used to obtain the code of the memory allocation interface; The processing unit is further used to replace the memory allocation interface with a custom memory allocation interface; the custom memory allocation interface is used to allocate storage space in the local memory and / or the at least one memory; Part of the cache data is stored in the storage space allocated in the at least one memory through the customized memory allocation interface.

12. The device according to claim 10 or 11, characterized in that The processing unit is also used to configure a transparent memory for each executor; the computing device includes at least one executor, and the computing device uses the executor to store the cache data in the transparent memory configured by the executor, and the transparent memory includes an available memory space capacity of the local memory of a first set size and a storage space of the at least one memory of a second set size.

13. The device according to any one of claims 10 to 12, characterized in that: The processing unit is specifically configured to send a memory allocation request to the at least one memory when the cached data amount of the cached data is greater than the available memory space capacity of the local memory; The memory allocation request is used to allocate a target storage space for storing the cache data; Determining, through the custom memory allocation interface, whether the target storage space allocated by the memory allocation request is greater than a set memory threshold; When the target storage space allocated by the memory allocation request is greater than the set memory threshold, storage space is allocated in the transparent memory configured by the executor, and part of the cache data is stored in the target storage space.

14. The device according to any one of claims 11 to 13, characterized in that: The processing unit is further used to send a first request instruction to the at least one memory through the custom memory allocation interface; the first request instruction is used to request the at least one memory to allocate free storage space after the target storage space.

15. The device according to any one of claims 10 to 14, characterized in that: The processing unit is specifically used to detect the number of times each data in the cache data is accessed and / or stored; Sending first cache data to the at least one memory; the first cache data refers to data in the cache data whose number of accesses and / or storages is less than a set number of times.

16. The device according to any one of claims 10 to 15, characterized in that: The processing unit is further used to obtain a memory access request instruction; the memory access request instruction is used to read or write specified data, and the memory access request instruction includes a virtual memory address of the specified data; Querying a locally stored address mapping table to determine a physical storage address of a virtual memory address mapping of the specified data; the address mapping table records a mapping relationship between a virtual memory address and a physical storage address; the physical storage address refers to a physical memory address of the local memory and a physical address of the at least one memory; Detecting whether the physical storage address of the specified data is the physical memory address of the local memory; In the case where the physical storage address of the designated data is the physical memory address of the local memory, the data stored in the physical memory address of the designated data is read, or the designated data is written to the corresponding physical memory address.

17. The device according to claim 16, characterized in that The processing unit is further configured to allocate a designated storage space in the local memory when the physical storage address of the designated data is not a physical memory address of the local memory; forwarding the memory access request instruction to the at least one memory; The memory access request instruction is used to request the at least one memory to read the data stored in the physical storage address of the specified data; The designated data is received from the at least one memory, and the designated data is stored in the designated storage space.

18. The device according to claim 16 or 17, characterized in that The transceiver unit is also used to obtain the code of the external memory access interface; The processing unit is further used to replace the external memory access interface with a custom external memory access interface; the custom external memory access interface is used to read the data stored in the at least one memory.

19. A computing device, characterized in that include: at least one memory; At least one processor, wherein the processor is configured to execute instructions stored in the memory so that the computing device executes the method according to any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 9.

21. A computer program product comprising instructions, characterized in that The computer program product stores instructions, which, when executed by a computing device, enable the computing device to implement the method according to any one of claims 1 to 9.

Images