Database system, task processing method and device, equipment, medium and product
By binding thread groups to specific hardware resource nodes in the database system, the performance loss problem of the database in a multi-hardware resource node system is solved, and high-efficiency database operation performance is achieved.
Patent Information
- Application Number
- CN202610039783.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-13
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, databases in CPU systems with multiple hardware resource nodes experience performance degradation when threads access memory across resource sets, failing to meet high-performance operation requirements.
By binding the thread groups of the database front-end and back-end modules to hardware resource nodes under a specific non-consistent memory access architecture, it is ensured that each thread group runs only on the hardware resource node it is bound to, avoiding cross-node access.
It improves database performance in multi-NUMA node CPU systems, reduces performance loss caused by cross-node access by threads, increases CPU utilization and system throughput, and ensures performance stability under high concurrency.
Smart Images
Figure CN121919199A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of database technology, and in particular to a database system, task processing method, apparatus, device, medium and product. Background Technology
[0002] Currently, databases run on multiple hardware resource nodes, which can cause threads to access memory across resource sets, resulting in performance degradation. This problem is particularly severe in CPU systems with a large number of hardware resource nodes.
[0003] However, existing task processing methods cannot control which hardware resource nodes the database thread runs on, thus failing to meet the requirements for high-performance database operation. Summary of the Invention
[0004] This invention provides a database system, task processing method, apparatus, device, medium, and product to solve the problem that current task processing methods cannot meet the performance requirements of database operation. By allocating independent hardware resource nodes to thread groups executing different tasks, the system can control the thread groups to run on specific hardware resource nodes, thereby improving database operation performance.
[0005] In a first aspect, embodiments of the present invention also provide a database system, the system comprising:
[0006] Database front-end module and database back-end module;
[0007] The database front-end module is used to handle client task requests;
[0008] The database backend module is used for database system maintenance;
[0009] Both the database front-end module and the database back-end module include at least one thread group. Each thread group corresponds to a different database running task. Each thread group is bound to a hardware resource node so that each thread group runs only on the hardware resource node bound to the thread group. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture.
[0010] Secondly, embodiments of the present invention provide a task processing method, the method comprising:
[0011] The client thread in the client thread group obtains the first task request sent by the client and sends the first task request to the syntax parsing thread group through the first interface.
[0012] The syntax parsing thread of the syntax parsing thread group performs syntax parsing on the first task statement corresponding to the first task request, obtains the syntax parsing result, generates a task processing plan based on the syntax parsing result, and sends the task processing plan to the execution thread group through the second interface.
[0013] The task processing plan is executed by the execution thread of the execution thread group to obtain the first task processing result of the first task request;
[0014] Among them, the client thread group, the syntax parsing thread group, and the execution thread group are all bound to hardware resource nodes, which are hardware resource nodes under the preset non-uniform memory access architecture.
[0015] Thirdly, embodiments of the present invention also provide a task processing apparatus, the apparatus comprising:
[0016] The task request and response module is used to obtain the first task request sent by the client through the client thread in the client thread group, and send the first task request to the syntax parsing thread group through the first interface.
[0017] The syntax parsing module is used to perform syntax parsing on the first task statement corresponding to the first task request through the syntax parsing thread of the syntax parsing thread group, obtain the syntax parsing result, generate a task processing plan based on the syntax parsing result, and send the task processing plan to the execution thread group through the second interface.
[0018] The task execution module is used to execute the task processing plan through the execution threads of the execution thread group and obtain the first task processing result of the first task request.
[0019] The client thread group, syntax parsing thread group, and execution thread group are all bound to hardware resource nodes, which are hardware resource nodes under the preset non-uniform memory access architecture.
[0020] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device comprising:
[0021] At least one processor; and
[0022] A memory communicatively connected to the at least one processor; wherein,
[0023] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method described in any embodiment of the present invention.
[0024] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the task processing method described in any embodiment of the present invention.
[0025] Sixthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the task processing method as described in any of the embodiments of the present invention.
[0026] In this embodiment of the invention, a database front-end module and a database back-end module are used. The database front-end module handles client task requests, while the database back-end module performs database system maintenance. Both the front-end and back-end modules include at least one thread group. Each thread group corresponds to a different database execution task, and each thread group is bound to a hardware resource node, ensuring that each thread group runs only on the hardware resource node bound to it. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture. This embodiment of the invention solves the problem that current task processing cannot meet the performance requirements of database operation. By independently allocating hardware resource nodes to thread groups executing different tasks, the execution of thread groups on specific hardware resource nodes can be controlled, thereby improving database operation performance. Attached Figure Description
[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 A schematic diagram of the structure of a database system provided in an embodiment of the present invention;
[0029] Figure 2 A schematic diagram of the structure of another database system provided in an embodiment of the present invention.
[0030] Figure 3 A flowchart of a task processing method provided in an embodiment of the present invention;
[0031] Figure 4 This is a schematic diagram of the structure of a task processing device provided in an embodiment of the present invention;
[0032] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0034] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.
[0035] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the relevant content of the solution.
[0036] Figure 1 This is a schematic diagram of the structure of a database system provided in an embodiment of the present invention.
[0037] like Figure 1 As shown, the database system includes:
[0038] The database front-end module 110 and the database back-end module 120 are used to handle client task requests. The database back-end module 120 is used to perform database system maintenance. Both the database front-end module 110 and the database back-end module 120 include at least one thread group. Each thread group corresponds to a different database running task. Each thread group is bound to a hardware resource node so that each thread group runs only on the hardware resource node bound to the thread group. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture.
[0039] The database front-end module 110 serves as the interaction entry point between the database system and the client. It receives and processes client task requests initiated by the client. Client task requests can be business-related task requests, including data operation requests such as adding, querying, modifying, and deleting data in the database.
[0040] The database backend module 120 handles the database's backend processing, including system operation and maintenance tasks, as well as database data reading and writing, logical calculations, and data processing tasks corresponding to frontend business requests. In this embodiment, the database backend module 120 is not directly reflected in the client request response, but it ensures the stable operation of the database and remains a crucial module of the database system.
[0041] Both the database front-end module 110 and the database back-end module 120 implement task scheduling and execution based on thread groups. Each module is configured with at least one functionally independent thread group. Each thread group is assigned a specific task and independently executes a specific type of database task. There is no hierarchical relationship between the thread groups. They only complete the overall operation process of the database through task collaboration. For example, the thread groups in the database front-end module 110 transfer processed data through a data interface to collaboratively complete client tasks.
[0042] In this embodiment, to adapt to the hardware characteristics of Non-Uniform Memory Access (NUMA) architecture, reduce hardware resource access latency, and improve task execution efficiency, each thread group is pre-bound to a hardware resource node under the NUMA architecture, namely the NUMA node.
[0043] The binding relationship between NUMA nodes and thread groups is a hard constraint, which means that any thread group can only be scheduled and executed on the hardware resource node it is bound to during the entire process. That is, it can only occupy the local hardware resources such as CPU and memory of the bound node and cannot drift to other hardware resource nodes to run. This avoids threads accessing memory across NUMA nodes, thereby realizing localized scheduling and efficient utilization of hardware resources for database thread groups and improving the performance of the database in multi-NUMA node CPU systems.
[0044] In this embodiment, thread groups with different functions are independently bound to NUMA nodes. Compared to single-task, single-thread groups, this allows for precise binding to matching NUMA nodes based on the resource characteristics of each functional thread group, achieving low-latency access to local resources and completely avoiding cross-node performance overhead. Simultaneously, resident functional thread groups avoid the overhead of frequent creation and destruction of single-task thread groups. Furthermore, functional isolation and independent binding achieve resource isolation, reducing contention, improving CPU utilization and system throughput, and ensuring stable, uninterrupted performance under high concurrency. Different functional thread groups are bound to specific NUMA nodes, with threads prioritizing their own NUMA nodes and not preempting other functions. This reduces the risk of a single function under heavy load preempting other NUMA node resources, which could impact the operation of other functional threads and significantly reduce the overall performance of the database system.
[0045] The technical solution of this embodiment utilizes a database front-end module and a database back-end module. The database front-end module handles client task requests, while the database back-end module performs database system maintenance. Both the front-end and back-end modules include at least one thread group, each corresponding to a different database execution task. Each thread group is bound to a hardware resource node, ensuring that each thread group runs only on the hardware resource node bound to it. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture. This embodiment solves the problem that current task processing cannot meet the performance requirements of database operation. By independently allocating hardware resource nodes to thread groups executing different tasks, the execution of thread groups on specific hardware resource nodes can be controlled, thereby improving database performance.
[0046] Figure 2 This is a schematic diagram of the structure of a database system provided in an embodiment of the present invention.
[0047] like Figure 2 As shown, the database system includes:
[0048] The system comprises a database front-end module 210, a database back-end module 220, and a buffer pool management module 230. The database front-end module 210 handles client task requests. The database back-end module 220 performs database system maintenance. The database front-end module 210 includes at least one of the following: a client thread group, a syntax parsing thread group, an execution thread group, and a network communication thread group. The database back-end module 220 includes at least one of the following: an input / output thread group, a dirty file flushing thread group, a log thread group, and a master-slave thread group. Each thread group in both the database front-end module 210 and the database back-end module 220 corresponds to a different database running task. Each thread group is bound to a hardware resource node, ensuring that each thread group runs only on the hardware resource node bound to it. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture. The buffer pool management module 230 includes a buffer pool management thread group, used for allocating memory for the buffer pool and handling thread access to data in the buffer pool. The buffer pool management thread group is bound to a hardware resource node, ensuring that the buffer pool management thread group runs only on the hardware resource node bound to it.
[0049] A foreground thread pool is created for client requests, SQL syntax parsing, analysis, and optimization, and for generating specific execution plans for SQL requests. SQL parsing and buffer pool access are separated into thread groups, with a dedicated buffer pool management thread group.
[0050] An abstract front-end message interface is constructed to serve as the message communication medium between the SQL front-end thread and the execution thread, that is, the message communication medium between the syntax parsing thread group and the execution thread group.
[0051] The database front-end module 210 serves as the interaction module between the client and the database, as well as the business request processing layer. Its thread groups are all focused on the access and processing of client business requests, and at least one of the following is covered: client thread group, syntax parsing thread group, execution thread group, and network communication thread group.
[0052] The network communication thread group is responsible for listening to ports, establishing TCP connections, and sending and receiving network data packets, serving as the entry point for access requests. The client thread group takes over requests forwarded by the network communication thread group and maintains the client session state. The syntax parsing thread group performs lexical and syntactic analysis on the SQL statements in the request, generating an abstract syntax tree to prepare for subsequent execution. The execution thread group receives the parsed task instructions and completes core business logic processing such as data querying and calculation.
[0053] The database backend module 220 focuses on database operation and maintenance, as well as data persistence assurance. It includes at least one of the following: an input / output (IO) thread group, a dirty data flushing thread group, a log thread group, and a master-slave thread group. The IO thread group handles disk data read / write, providing data support for the foreground execution thread group. The dirty data flushing thread group asynchronously flushes dirty data in memory to disk, ensuring data consistency. The log thread group is responsible for writing redo / undo logs, providing a basis for transaction rollback and fault recovery. The master-slave thread group is responsible for data synchronization between the master and slave databases, ensuring high system availability. The thread groups of both types of modules work collaboratively and can be independently bound to NUMA nodes, improving file IO speed and balancing business processing efficiency with system stability.
[0054] In this embodiment, the foreground and background threads are separated. A foreground message interface is constructed to handle message communication between the foreground and background threads. The foreground message interface is responsible for passing messages to the background thread, and the foreground and background threads do not need to consider message synchronization time. The foreground thread can immediately execute the next client request after publishing a message to improve response efficiency.
[0055] The buffer pool requires a large block of memory. In this embodiment, a buffer pool management thread group is added to handle memory allocation and control other threads' access to the buffer pool data. The buffer pool management thread is bound to a separate NUMA node, controlling memory allocation within a designated NUMA node.
[0056] A NUMA node can be bound to multiple thread groups as needed, for example, based on the scope of business interactions among the thread groups. For instance, if other threads also utilize bufferpool thread management, and the I / O threads in the background thread group's input / output (IO) thread group frequently access the bufferpool, they can be bound to the same NUMA node. By clearly defining the interaction scope of thread groups, thread groups can be effectively bound, reducing the need for cross-NUMA node resource access.
[0057] In one optional implementation, the system further includes: a hardware resource node binding module, configured to, in response to a hardware resource node binding operation, determine the hardware resource node to be bound and at least one thread group to be bound corresponding to the hardware resource node binding operation; and bind the hardware resource node to be bound and at least one thread group to be bound.
[0058] For example, the hardware resource node binding module responds to the binding operation by identifying the hardware resource node to be bound as NUMA node 1, and the thread groups to be bound as the foreground network communication thread group and the background input / output thread group, and then completing the binding of NUMA node 1 with these two thread groups. It can also identify the hardware resource node to be bound as NUMA node 2, and the thread groups to be bound as the syntax parsing thread group and the dirty flushing thread group, completing the binding adaptation for both. Different thread groups can be bound to the same or different hardware resource nodes as needed, adapting to the running task characteristics and hardware resource requirements of various thread groups.
[0059] The technical solution of this embodiment includes a database front-end module and a database back-end module. The database front-end module is used to handle client task requests, and the database back-end module is used to perform database system maintenance. Both the database front-end module and the database back-end module include at least one thread group. Each thread group corresponds to a different database running task, and each thread group is bound to a hardware resource node so that each thread group runs only on the hardware resource node bound to it. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture. The database front-end module includes at least one of the following: client thread group, syntax parsing thread group, execution thread group, and network communication thread group. The database back-end module includes at least one of the following: input / output thread group, dirty flushing thread group, log thread group, and master / slave thread group. The database system also includes a buffer pool management module, which includes a buffer pool management thread group for allocating memory for the buffer pool and handling thread access to data in the buffer pool. The buffer pool management thread group is bound to a hardware resource node so that the buffer pool management thread group runs only on the hardware resource node bound to it. The technical solution of this invention solves the problem that current task processing cannot meet the performance requirements of database operation. It can improve database operation performance by allocating independent hardware resource nodes to thread groups that execute different tasks, controlling the thread groups to run on specific hardware resource nodes, and by clearly dividing the thread groups according to their functions, avoiding thread groups running across nodes, and independently binding the buffer pool management thread to nodes.
[0060] Figure 3 This is a flowchart illustrating a task processing method provided in an embodiment of the present invention. This embodiment is applicable to task processing scenarios. The method can be executed by a task processing device, which can be implemented in software and / or hardware and integrated into an electronic device.
[0061] like Figure 3 As shown, the task processing method includes the following steps:
[0062] S310. Obtain the first task request sent by the client through the client thread in the client thread group, and send the first task request to the syntax parsing thread group through the first interface.
[0063] The client thread in the client thread group listens for and obtains the first task request initiated by the client in real time. This task request can be a business request for adding, deleting, querying and modifying data in response to the database, which includes SQL instructions to be executed.
[0064] After receiving and validating the first task request, the complete first task request is forwarded to the syntax parsing thread group in a standardized manner through the preset first interface. The first interface is a dedicated communication link between the client thread group and the syntax parsing thread group, ensuring the efficiency and stability of request transmission, and enabling the syntax parsing thread group to perform lexical and syntax parsing processing on the SQL statements in the request in a timely manner, realizing the division of labor and task flow between thread groups.
[0065] S320. The syntax parsing thread of the syntax parsing thread group performs syntax parsing on the first task statement corresponding to the first task request, obtains the syntax parsing result, generates a task processing plan based on the syntax parsing result, and sends the task processing plan to the execution thread group through the second interface.
[0066] The syntax parsing thread in the syntax parsing thread group receives the first task statement corresponding to the first task request. This statement is an SQL statement initiated by the client to perform CRUD operations. The syntax parsing thread first performs a complete lexical and syntactic analysis on the first task statement to verify the syntactic compliance of the SQL statement. Simultaneously, it performs semantic analysis to verify the legality of the data and the rationality of the business logic, forming a standard syntax parsing result.
[0067] The database can have a built-in metadata statistics module that can provide real-time statistics on thread resource distribution, database data distribution, and hardware node load status, providing a core basis for optimizing computation. Based on the syntax parsing results and various information from the metadata statistics module, the optimizer accurately assesses the execution cost of data retrieval and processing, optimizes the execution path, and ultimately generates the optimal task processing plan.
[0068] After the plan is generated, the task processing plan is forwarded to the execution thread group in a standardized manner through the preset second interface. This interface is a dedicated communication link between the syntax parsing thread group and the execution thread group, ensuring efficient and stable plan transmission. The execution thread group takes over and executes the corresponding SQL query and data processing operations. The entire process relies on the front-end thread group to complete the response and parsing of client requests.
[0069] S330. The task processing plan is executed by the execution thread of the execution thread group to obtain the first task processing result of the first task request.
[0070] Among them, the client thread group, the syntax parsing thread group, and the execution thread group are all bound to hardware resource nodes, which are hardware resource nodes under the preset non-uniform memory access architecture.
[0071] The execution thread of the execution thread group undertakes and executes the task processing plan. During execution, it can coordinate with the background input / output thread group to complete data reading and writing based on task requirements. Relying on the local computing power of the bound hardware resource node, it completes core data processing operations such as calculation, filtering, association, and sorting corresponding to SQL. It can also call the resource information of the metadata statistics module throughout the process to ensure execution efficiency. The execution thread strictly follows the optimized task processing plan to execute logical instructions step by step. After completing the entire data processing process, it integrates to form the first task processing result corresponding to the first task request. This result is a complete data feedback that satisfies the client's CRUD (Create, Read, Update, Delete) requirements. Subsequently, the first task processing result can be sent back to the client to complete the first task processing process.
[0072] The technical solution of this embodiment involves obtaining a first task request sent by a client through a client thread in a client thread group, and sending the first task request to a syntax parsing thread group through a first interface. The syntax parsing thread in the syntax parsing thread group performs syntax parsing on the first task statement corresponding to the first task request, obtains the syntax parsing result, generates a task processing plan based on the syntax parsing result, and sends the task processing plan to the execution thread group through a second interface. The execution thread group executes the task processing plan to obtain the first task processing result of the first task request. Each of the client thread group, syntax parsing thread group, and execution thread group is bound to a hardware resource node, which is a hardware resource node under a preset non-uniform memory access architecture. This embodiment of the invention solves the problem that current task processing cannot meet the performance requirements of database operation. By independently allocating hardware resource nodes to thread groups executing different parts of the client task, the thread groups can be controlled to run on specific hardware resource nodes, thereby improving database operation performance.
[0073] Figure 4 This is a schematic diagram of the structure of a task processing device provided in an embodiment of the present invention. This embodiment is applicable to task processing scenarios. The device can be implemented by software and / or hardware and integrated into an electronic device.
[0074] like Figure 4 As shown, the task processing device includes: a task request response module 410, a syntax parsing module 420, and a task execution module 430.
[0075] The task request response module 410 is used to obtain the first task request sent by the client through the client thread in the client thread group, and send the first task request to the syntax parsing thread group through the first interface; the syntax parsing module 420 is used to perform syntax parsing on the first task statement corresponding to the first task request through the syntax parsing thread of the syntax parsing thread group, obtain the syntax parsing result, generate a task processing plan based on the syntax parsing result, and send the task processing plan to the execution thread group through the second interface; the task execution module 430 is used to execute the task processing plan through the execution thread of the execution thread group to obtain the first task processing result of the first task request; wherein, the client thread group, the syntax parsing thread group and the execution thread group are all bound to hardware resource nodes, and the hardware resource nodes are hardware resource nodes under a preset non-uniform memory access architecture.
[0076] The technical solution of this invention involves obtaining a first task request sent by a client through a client thread in a client thread group, and sending the first task request to a syntax parsing thread group through a first interface. The syntax parsing thread in the syntax parsing thread group performs syntax parsing on the first task statement corresponding to the first task request, obtains the syntax parsing result, generates a task processing plan based on the syntax parsing result, and sends the task processing plan to an execution thread group through a second interface. The execution thread group executes the task processing plan to obtain the first task processing result of the first task request. Each of the client thread group, syntax parsing thread group, and execution thread group is bound to a hardware resource node, which is a hardware resource node under a preset non-consistent memory access architecture. This technical solution solves the problem that current task processing cannot meet the performance requirements of database operation. By independently allocating hardware resource nodes to thread groups executing different parts of the client task, the thread groups can be controlled to run on specific hardware resource nodes, thereby improving database operation performance.
[0077] The task processing device provided in the embodiments of the present invention can execute the task processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.
[0078] Figure 5A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0079] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0080] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0081] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as task processing methods.
[0082] In some embodiments, the task processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the task processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the task processing method by any other suitable means (e.g., by means of firmware).
[0083] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0084] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0085] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0086] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0087] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0088] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0089] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0090] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the task processing method provided in any embodiment of this application.
[0091] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0092] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0093] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A database system, characterized in that, include: Database front-end module and database back-end module; The database front-end module is used to process client task requests; The database backend module is used for database system maintenance; Both the database front-end module and the database back-end module include at least one thread group. Each thread group corresponds to a different database running task. Each thread group is bound to a hardware resource node so that each thread group runs only on the hardware resource node bound to the thread group. The hardware resource node is a hardware resource node under a preset non-consistent memory access architecture.
2. The system according to claim 1, characterized in that, Also includes: The hardware resource node binding module is used to respond to the hardware resource node binding operation by determining the hardware resource node to be bound and at least one thread group to be bound corresponding to the hardware resource node binding operation. Bind the hardware resource node to be bound to the at least one thread group to be bound.
3. The system according to claim 1, characterized in that, The database front-end module includes at least one of the following: client thread group, syntax parsing thread group, execution thread group, and network communication thread group.
4. The system according to claim 1, characterized in that, The database backend module includes at least one of the following: input / output thread group, dirty file flushing thread group, log thread group, and master / slave thread group.
5. The system according to claim 1, characterized in that, The database system also includes a buffer pool management module, which includes a buffer pool management thread group for allocating memory for the buffer pool and processing thread access to data in the buffer pool. The buffer pool management thread group is bound to a hardware resource node so that the buffer pool management thread group only runs on the hardware resource node to which the buffer pool management thread group is bound.
6. A task processing method, characterized in that, include: The client thread in the client thread group obtains the first task request sent by the client and sends the first task request to the syntax parsing thread group through the first interface. The syntax parsing thread of the syntax parsing thread group performs syntax parsing on the first task statement corresponding to the first task request, obtains the syntax parsing result, generates a task processing plan based on the syntax parsing result, and sends the task processing plan to the execution thread group through the second interface. The task processing plan is executed by the execution thread of the execution thread group to obtain the first task processing result of the first task request; The client thread group, the syntax parsing thread group, and the execution thread group are all bound to hardware resource nodes, which are hardware resource nodes under a preset non-uniform memory access architecture.
7. A task processing device, characterized in that, include: The task request response module is used to obtain the first task request sent by the client through the client thread in the client thread group, and send the first task request to the syntax parsing thread group through the first interface; The syntax parsing module is used to perform syntax parsing on the first task statement corresponding to the first task request through the syntax parsing thread of the syntax parsing thread group, obtain the syntax parsing result, generate a task processing plan based on the syntax parsing result, and send the task processing plan to the execution thread group through the second interface. The task execution module is used to execute the task processing plan through the execution threads of the execution thread group to obtain the first task processing result of the first task request; The client thread group, the syntax parsing thread group, and the execution thread group are all bound to hardware resource nodes, which are hardware resource nodes under a preset non-uniform memory access architecture.
8. An electronic device, characterized in that, The device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method of claim 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the task processing method of claim 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the task processing method as described in claim 6.