Database operation method, electronic device, storage medium, and program product

Abstract

The present disclosure provides a database running method, an electronic device, a readable storage medium and a program product. The database running method comprises: obtaining a data service request; forwarding the data service request to one or more corresponding service nodes to obtain a request task according to the type of the data service request; and forwarding the data service request to one or more corresponding processing engines to process the request task by the processing engines to obtain a processing result.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a database operation method, electronic device, storage medium, and program product. Background Technology

[0002] With the widespread application of vector retrieval technology, database devices are often required to provide corresponding vector database services.

[0003] In related technologies, Milvus, as a cloud-native vector data management system, employs a four-layer architecture: proxy layer, coordinator layer, worker node layer, and storage layer, to provide services such as DQL, DML, and DDL for vector databases. Specifically, Milvus's architecture configures four types of coordinators—query coordinator, data coordinator, index coordinator, and root coordinator—in the coordinator layer, acting as the system's brain and distributing service tasks to the corresponding worker nodes. However, the Milvus architecture has relatively complex components with strong dependencies between them, making error localization difficult and increasing system maintenance costs. Summary of the Invention

[0004] This disclosure provides a database operation method, electronic device, storage medium, and program product.

[0005] This disclosure provides a database operation method, comprising: acquiring a data service request; forwarding the data service request to one or more corresponding service nodes to acquire a request task according to the type of the data service request; and forwarding the data service request to one or more corresponding processing engines to process the request task through the processing engines to acquire a processing result. In some embodiments, generating semantic vector data based on log entry data includes: performing semantic vectorization processing on the log entry data to obtain multiple semantic vectors; and generating the semantic vector data based on the multiple semantic vectors.

[0006] In some implementations, the service node includes a definition node, the request task includes a definition task and an import task, and forwarding the data service request to one or more corresponding service nodes to obtain the request task according to the type of the data service request includes: if the type of the data service request is a definition service, forwarding the data service request to the definition node to obtain the definition task; or if the type of the data service request is an import service, forwarding the data service request to the definition node to obtain the import task.

[0007] In some implementations, the processing engine includes a definition engine and an import engine. Forwarding the data service request to one or more corresponding processing engines, and processing the request task through the processing engines to obtain a processing result, includes: starting the import engine based on the import task, forwarding the data service request to the import engine, and granting the import engine data source permissions; using the import engine to obtain internal data from the data source according to the import task, and storing the internal data in a storage node to obtain the processing result; destroying the import engine when the processing result returns to the definition node; forwarding the data service request to the definition engine based on the definition task; using the definition engine to convert the data service request into a metadata change request according to the definition task, and forwarding the metadata change request to the storage node; and processing the target stored data at the storage node according to the metadata change request to obtain the processing result.

[0008] In some implementations, the service node includes multiple operation nodes, the request task includes an operation task, and forwarding the data service request to one or more corresponding service nodes to obtain the request task according to the type of the data service request includes: when the type of the data service request is an operation service, obtaining the operation-affected data table corresponding to the operation service; determining the target operation node corresponding to the operation-affected data table; and forwarding the data service request to the target operation node to obtain the operation task.

[0009] In some implementations, the processing engine includes multiple operation engines, each corresponding to a data table. Forwarding the data service request to one or more corresponding processing engines and processing the request task through the processing engines to obtain a processing result includes: forwarding the data service request to the operation engine corresponding to the operation table based on the operation task; using the operation engine to process the operation table in memory according to the operation task to obtain the processing result; and storing the data in memory to a storage node.

[0010] In some implementations, the service node includes multiple query nodes, the request task includes a query task, and forwarding the data service request to one or more corresponding service nodes to obtain the request task according to the type of the data service request includes: when the type of the data service request is a query service, obtaining the query data table corresponding to the query service; determining the target query node corresponding to the query data table; and forwarding the data service request to the target query node to obtain the query task.

[0011] In some implementations, the processing engine includes multiple query engines, each corresponding to a data table. Forwarding the data service request to one or more corresponding processing engines and processing the request task through the processing engines to obtain a processing result includes: based on the query task, forwarding the data service request to the query engine corresponding to the queried data table; using the query engine to perform a data query in the video memory according to the query task and the queried data table to obtain the processing result; and loading the data in the video memory to the storage node.

[0012] A second aspect of this disclosure provides an electronic device, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform a database operation method according to any embodiment of this disclosure.

[0013] A third aspect of this disclosure provides a readable storage medium storing executable instructions, which, when executed by a processor, are used to implement the database operation method described in any embodiment of this disclosure.

[0014] The fourth aspect of this disclosure provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the database operation method described in any embodiment of this disclosure. Attached Figure Description

[0015] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0016] Figure 1 This is a flowchart illustrating a database operation method according to some embodiments of this disclosure.

[0017] Figure 2 This is a schematic diagram of the database structure of some embodiments of this disclosure.

[0018] Figure 3 This is a schematic diagram illustrating the process of forwarding data service requests to service nodes in some embodiments of this disclosure.

[0019] Figure 4 This is a timing diagram of database processing DDL service requests according to some embodiments of this disclosure.

[0020] Figure 5 This is a schematic diagram illustrating the process of processing a request task through a processing engine in some embodiments of this disclosure.

[0021] Figure 6 This is a timing diagram of database processing DML service requests according to some embodiments of this disclosure.

[0022] Figure 7 This is a schematic diagram illustrating the process of forwarding data service requests to service nodes in some embodiments of this disclosure.

[0023] Figure 8 This is a schematic diagram illustrating the process of processing a request task through a processing engine in some embodiments of this disclosure.

[0024] Figure 9 This is a timing diagram of database processing DQL service requests according to some embodiments of this disclosure.

[0025] Figure 10 This is a schematic diagram illustrating the process of forwarding data service requests to service nodes in some embodiments of this disclosure.

[0026] Figure 11 This is a schematic diagram illustrating the process of processing a request task through a processing engine in some embodiments of this disclosure.

[0027] Figure 12 This is a schematic block diagram of a database operation device using a processor-based hardware implementation according to one embodiment of the present disclosure. Detailed Implementation

[0028] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0029] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways in which the technical concepts of this disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of this disclosure.

[0031] The use of crosshairs and / or shading in the accompanying drawings is generally used to clarify the boundaries between adjacent components. Thus, unless otherwise stated, the presence or absence of crosshairs or shading does not convey or indicate any preference or requirement for the specific material, material properties, dimensions, proportions, commonalities between the illustrated components, or any other characteristics, properties, etc., of the components. Furthermore, in the accompanying drawings, the dimensions and relative dimensions of components may be exaggerated for clarity and / or descriptive purposes. When exemplary embodiments can be implemented differently, a specific process sequence may be performed in a different order than that described. For example, two consecutively described processes may be performed substantially simultaneously or in the reverse order of their description. Furthermore, the same reference numerals denote the same components.

[0032] When a component is referred to as being "on" or "above" another component, "connected to" or "joined to" another component, the component may be directly on, directly connected to, or directly joined to the other component, or there may be intermediate components. However, when a component is referred to as being "directly on" another component, "directly connected to," or "directly joined to" another component, there are no intermediate components. Therefore, the term "connection" can refer to physical connection, electrical connection, etc., and may or may not have intermediate components.

[0033] The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “(the)” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0034] The database operation method disclosed herein can be applied to the database operation apparatus disclosed herein, which can be configured on an electronic device. The electronic device can be a server or a terminal device, and the terminal device can be a mobile terminal, such as a mobile phone, tablet computer, personal digital assistant, or other hardware device with various operating systems.

[0035] The following text combines Figures 1 to 12 The database operation method and database operation device disclosed herein are described in detail.

[0036] Figure 1This is a flowchart illustrating some embodiments of the database operation method disclosed herein. Please refer to... Figure 1 The database operation method M100 provided in this disclosure may include steps S102, S104 and S106.

[0037] S102: Request to obtain data service.

[0038] S104: Based on the type of data service request, forward the data service request to one or more corresponding service nodes to obtain the request task.

[0039] S106: Forward the data service request to one or more corresponding processing engines, and process the request task through the processing engine to obtain the processing result.

[0040] The database operation method disclosed herein utilizes service nodes to assign corresponding request tasks and distributes data service requests to corresponding processing engines for data processing according to the request tasks. In this database architecture, service nodes handling the same type of service requests have a corresponding relationship with processing engines, while the dependency between service nodes handling different types of service requests and processing engines is weak. This makes it easier to locate errors in the database system and reduces maintenance costs. Compared to the Milvus system in related technologies, the database operation method disclosed herein does not require configuring a coordinator to manage service nodes in the database, making the database architecture more lightweight and efficient. Furthermore, the database operation method disclosed herein supports data processing using multiple service nodes and multiple processing engines, thus improving data processing efficiency through horizontal scaling of service nodes.

[0041] Optionally, data service requests may include Data Definition Language (DDL) service requests, Data Manipulation Language (DML) service requests, and Data Query Language (DQL) service requests.

[0042] For example, please combine Figure 2 , Figure 2This is a schematic diagram of the database structure constructed using the database operation method provided in this disclosure. The database includes a gateway layer, a service layer, an engine layer, and a storage layer. The service layer includes three types of service nodes: definition nodes, operation nodes, and query nodes. Definition nodes handle DDL service requests, operation nodes handle DML service requests, and query nodes handle DQL service requests. The engine layer includes four types of processing engines: import engine, definition engine, operation engine, and query engine. The import engine corresponds to the definition node and is used to handle DDL service requests; the definition engine corresponds to the definition node and is used to handle DML service requests; the operation engine corresponds to the operation node and is used to handle DML service requests; and the query engine corresponds to the query node and is used to handle DQL service requests. Thus, the dependencies between the different types of components handling DDL, DML, and DQL service requests are relatively low, making it easier to locate the corresponding component causing database operation errors and reducing the cost of database maintenance.

[0043] The gateway layer includes gateways, which are used to obtain data service requests and, based on the type of the data service request, forward the data service request to the corresponding service node, and then process the request task through the corresponding service node.

[0044] The storage layer includes storage nodes, which are used to store data. The four types of processing engines in the engine layer can read data from the storage nodes or store data into the storage nodes according to the requested task.

[0045] Please see Figure 3 In some implementations, the service node includes a definition node. The request task includes a definition task and an import task. Step S104 includes steps S1041 and S1042.

[0046] S1041: If the data service request type is definition service, forward the data service request to the definition node to obtain the definition task.

[0047] S1042: If the data service request type is import service, forward the data service request to the definition node to obtain the import task.

[0048] For example, DDL service requests include definition services and import services. Definition services refer to the creation, deletion, or modification of data tables. Import services refer to the import of data from other data sources into the database. Both definition and import services obtained by the gateway are forwarded to a unified definition node for processing. In this way, DDL service requests can be handled using relatively lightweight service nodes and processing engine components.

[0049] For example, a definition task is used to assign a corresponding processing engine (i.e., the definition engine) to handle specific tasks, such as requests to create, delete, or modify data tables. An import task is used to assign a corresponding processing engine (i.e., the import engine) to handle specific tasks, such as processing tasks to store internal data from other data sources into storage nodes. The definition node can obtain data service requests (DDL service requests) forwarded by the gateway and distinguish whether the data service request is a definition service or an import service, thereby obtaining the corresponding definition task or import task. Based on the definition task or import task, the node can forward the data service request to the corresponding category of processing engine.

[0050] Please see Figure 4 , Figure 4 This is a sequence diagram of how the database processes DDL service requests. In some implementations, the processing engine includes a definition engine and an import engine. The database launches a unified definition engine to handle requests for creating, deleting, or modifying data tables.

[0051] Please combine Figure 5 In some implementations, step S106 includes steps S0601, S0602, S0603, S0604, S0605 and S0606.

[0052] S0601: Start the import engine based on the import task, forward the data service request to the import engine, and grant the import engine data source permissions.

[0053] For example, the database will not start the import engine unless it receives an import service request, to ensure high database operating efficiency. When the definition node obtains an import task, the import engine is started and granted data source permissions, enabling it to read the data source's internal data.

[0054] Optionally, the import task is used to instruct the import engine to accept data provided by the corresponding data source, and also to grant the import engine data source permissions.

[0055] S0602: The import engine is used to obtain the internal data of the data source according to the import task, and the internal data is stored in the storage node to obtain the processing results.

[0056] For example, storing the internal data of the data source to a storage node includes sending a corresponding read / write request to the storage node based on the internal data of the data source, and the storage node processing the internal data of the data source according to the specific read / write request and generating a processing result.

[0057] S0603: Destroy the import engine when the processing result is returned to the defining node.

[0058] For example, after the storage node stores the internal data of the data source, the generated processing results are returned sequentially to the import engine and the definition node. The definition node can then return the processing results to the requesting client through the gateway. After the processing results are returned to the definition node, the import engine container is automatically destroyed to release certain resources and ensure the cleanliness and efficiency of the database.

[0059] S0604: Based on the defined task, forward the data service request to the definition engine.

[0060] For example, the database will launch a unified definition engine to handle data service requests related to definition tasks, without requiring definition nodes to start every time a data service request is received.

[0061] S0605: The definition engine is used to convert data service requests into metadata change requests based on the defined tasks, and then forward the metadata change requests to the storage nodes.

[0062] For example, defining a task is used to assign the definition engine to perform specific processing such as creating, deleting, or modifying a data table. The definition engine will convert the corresponding creation, deletion, or modification processing request into a specific metadata change request based on the definition task, and forward it to the storage node for corresponding data read and write processing.

[0063] S0606: The target stored data is processed on the storage node according to the metadata change request to obtain the processing result.

[0064] For example, a metadata change request corresponds to a specific definition task (the creation, deletion, or modification of a data table) and a read / write request. The storage node processes the internal data of the data source according to the specific read / write request and generates a processing result. The processing result generated by the storage node in processing the import task is returned to the definition engine and the definition node in sequence. The definition node can then return the processing result to the requesting client through the gateway.

[0065] The database operation method described above defines that the node forwards the data service request to the import engine or definition engine for processing according to the specific import service and definition service of the DDL service request. The component architecture for processing DDL service requests is relatively lightweight and simple.

[0066] Please see Figure 6 , Figure 6This is a sequence diagram of the database processing DML service requests. In some implementations, the processing engine includes multiple operation engines, the service node includes multiple operation nodes, and the request task includes an operation task. The operation task may specifically include operations such as inserting data into a table, updating data elements, and deleting data from a data source. The operation engine can perform corresponding operation processing on the data tables stored in the storage node according to the specific operation task. In one example, the vector or scalar data of the data tables in the storage node is preloaded into memory for the operation engine to perform corresponding operation processing. Different operation engines will be activated for operation processing on different data tables. In one example, the same data table uniquely corresponds to one operation engine to avoid interference between operation processing on different data tables.

[0067] Optionally, the data in the data tables stored in the storage nodes may include vector data and scalar data, without any restrictions.

[0068] Please combine Figure 7 In some implementations, step S104 includes steps S1043, S1044 and S1045.

[0069] S1043: When the data service request type is operation service, obtain the operation data table corresponding to the operation service.

[0070] For example, the data service request type is an operation service, that is, a DML service request. The object data table of the operation such as inserting data into the data table, updating data elements, and deleting data sources performed by the DML service request is the data table being operated on by the operation service.

[0071] S1044: Determine the target operation node corresponding to the data table to be operated on.

[0072] For example, there is a correspondence between operation nodes and operation engines. An operation node can assign operation tasks to the corresponding operation engine to process the data table to be operated on. Each data table uniquely corresponds to one operation engine. Therefore, if the operation engine corresponding to a certain operation node is the same as the operation engine corresponding to the data table to be operated on, then that operation node is the target operation node.

[0073] Optionally, in one embodiment, each operation node uniquely corresponds to one operation engine. In this way, one operation node manages the data operation services for one data table, the component relationship between the operation node and the operation engine is simple, and it is easy to locate the error.

[0074] Optionally, in another embodiment, the same operation node can correspond to multiple operation engines. Thus, one operation node can manage data operation services for multiple data tables. Based on this, operation engines corresponding to data tables with similar categories can be assigned to the same operation node, facilitating the management of multiple data tables with similar categories using a single operation node and reducing resource consumption.

[0075] S1045: Forward the data service request to the target operation node to obtain the operation task.

[0076] Optionally, the operation task is used to assign the operation engine corresponding to the operation table to perform operation processing on the operation table, and to assign the specific task to be executed by the operation engine, including operation tasks such as data table insertion, data element update, and data source deletion.

[0077] In the database operation method described above, when the data service request is a DML service request, the operation node for processing the DML service request is determined based on the data table to be processed by the DML service request, so as to accurately forward the DML service request to the corresponding operation node for processing.

[0078] Please combine Figure 8 In some implementations, step S106 includes steps S0607, S0608 and S0609.

[0079] S0607: Based on the operation task, forward the data service request to the operation engine corresponding to the operation table.

[0080] Optionally, when the operation engine corresponding to the data table being operated on is started, the operation engine can preload the vector or scalar data in the data table being operated on through the shared storage service of the storage node. In this way, when a data service request (DML service request) is forwarded to the operation engine, the operation engine can quickly respond to the data service request and process the corresponding data.

[0081] S0608: The operation engine processes the operated data table in memory according to the operation task to obtain the processing result.

[0082] For example, memory refers to the CPU's RAM. Operation tasks assign specific operations such as inserting data into a table, updating data elements, and deleting data from a data source, as well as the order in which these operations are performed. Processing the operated data table in memory includes processing the specific operation tasks sequentially. In one example, the operation engine processes the related operations of inserting data into a table, updating data elements, and deleting data from a data source in memory using the "append," "update," and "delete" operations of the dataframe data structure and the ndarray data structure, respectively.

[0083] S0609: Storing data in memory to a storage node.

[0084] For example, at preset time intervals, the operation engine automatically stores the vector or scalar data cached in memory to the storage node, completing the persistent data storage. In one example, the preset time interval ranges from [6s, 14s], such as 6s, 7s, 8s, 9s, 10s, 11s, 12s, 13s, 14s, etc., which will not be listed here. In this way, the frequency of automatic saving will not be too high, thus avoiding excessive resource consumption, nor will the risk of data loss increase due to too low a saving frequency.

[0085] Please see Figure 9 , Figure 9 This is a sequence diagram of the database processing DQL service requests. In some implementations, the processing engine includes multiple query engines, the service node includes multiple query nodes, and the request task includes a query task. A query task may specifically include querying data based on query conditions. The query engine can perform corresponding query processing on the data tables stored in the storage node according to the specific query task. In one example, the vector or scalar data of the data tables in the storage node is preloaded into the GPU's video memory for the query engine to perform corresponding query processing. Different query engines will be activated for querying different data tables.

[0086] Optionally, the data in the data tables stored in the storage nodes includes both vector data and scalar data, without any restrictions. The query engine supports queries for both scalar and vector data.

[0087] Please combine Figure 10 In some implementations, step S104 includes steps S1046, S1047 and S1048.

[0088] S1046: If the data service request type is query service, obtain the query data table corresponding to the query service.

[0089] For example, the data service request type is a query service, that is, a DQL service request. A DQL service request queries a specific data table for data that meets the specified conditions, and this specific data table is the queried data table.

[0090] S1047: Determine the target query node corresponding to the data table being queried.

[0091] For example, there is a correspondence between query nodes and query engines. A query node can assign operation tasks to the corresponding query engine to query data in the data table to which the query service is located. Different data tables correspond to different query engines. Therefore, if the query engine corresponding to a certain query node is the same as the query engine corresponding to the data table being queried, then that query node is the target query node.

[0092] Optionally, in one embodiment, each query node uniquely corresponds to one query engine. In this way, one query node manages the data query service for one data table, the component relationship between the query node and the query engine is simple, and it is easy to locate the error.

[0093] Optionally, in another embodiment, the same query node can correspond to multiple query engines. Thus, one query node can manage data query services for multiple data tables. Based on this, query engines corresponding to data tables with similar categories can be assigned to the same query node, facilitating the management of multiple data tables with similar categories using a single query node and reducing resource consumption.

[0094] Optionally, a single DQL service request can query a table that corresponds to multiple query nodes. For example, assuming each query node can process 100 vector data queries per second, if a DQL service request needs to query data with a processing speed of 1000 vector data queries per second, then the DQL service request can be forwarded to 10 query nodes corresponding to the query table for processing. In this way, query nodes support horizontal scaling, which can improve query efficiency.

[0095] S1048: Forward the data service request to the target query node to obtain the query task.

[0096] Optionally, the query task is used to assign the query engine corresponding to the queried data table to query the data in the queried data table, and to provide the query conditions in the DQL service request.

[0097] The database operation method described above, when the data service request is a DQL service request, determines the query node for processing the DQL service request based on the queried data table being processed by the DQL service request. The query node for processing the DQL service request can be a single node or can be expanded to multiple query nodes as needed to improve query efficiency. Furthermore, the architecture of the components related to the DQL service request is simple, and the query node does not interfere with other types of service nodes or processing engines, making the expansion of the query node easy to implement.

[0098] Please combine Figure 11 In some implementations, step S106 includes steps S0610, S0611 and S0612.

[0099] S0610: Based on the query task, forward the data service request to the query engine corresponding to the queried data table.

[0100] Optionally, when the query engine corresponding to the queried data table is launched, the query engine can preload the vector or scalar data from the queried data table into the GPU's video memory through the shared storage service of the storage node. This allows the query engine to utilize the GPU to accelerate the query process.

[0101] S0611: The query engine is used to perform data queries in the video memory based on the query task and the data table being queried in order to obtain the processing results.

[0102] For example, the query engine performs data query calculations in the GPU's video memory. This leverages the GPU's powerful parallel processing capabilities to accelerate data query computations. When the target query nodes are expanded to multiple, query speed can be improved by utilizing the GPU to process the query tasks of multiple target query nodes in parallel.

[0103] S0612: Load data from memory into the storage node.

[0104] For example, at preset time intervals, the operation engine automatically loads the vector or scalar data cached in the video memory to the storage node via shared storage service, thus refreshing the queried data. In one example, the preset time interval ranges from [6s, 14s], such as 6s, 7s, 8s, 9s, 10s, 11s, 12s, 13s, 14s, etc., which are not listed here. In this way, the frequency of automatic refresh will not be too high, thus avoiding excessive resource consumption, nor will the frequency of refresh be too low, resulting in untimely data updates.

[0105] Figure 12 This is a schematic block diagram of a database operation device using a processor-based hardware implementation according to one embodiment of the present disclosure.

[0106] The database operating apparatus may include corresponding modules that perform one or more steps in the flowchart described above. Therefore, each or more steps in the flowchart can be performed by a corresponding module, and the apparatus may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform a corresponding step, or implemented by a processor configured to perform a corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented through some combination thereof.

[0107] The hardware architecture of the database operating device disclosed herein can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0108] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0109] Any process or method description in the flowcharts or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain. The processor performs the various methods and processes described above. For example, the method embodiments of this disclosure may be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some embodiments, part or all of the software program may be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).

[0110] The logic and / or steps represented in the flowchart or otherwise described herein may be specifically implemented in any readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0111] For the purposes of this specification, a "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM). Furthermore, a readable storage medium can even be paper or other suitable media on which a program can be printed, since a program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in memory.

[0112] It should be understood that various parts of this disclosure can be implemented in hardware, software, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0113] Those skilled in the art will understand that all or part of the steps of the methods described above can be implemented by a program instructing related hardware. The program can be stored in a readable storage medium, and when executed, the program includes one or a combination of the steps of the method implementation.

[0114] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0115] refer to Figure 12 According to one embodiment of the present disclosure, the database operation device 1000 of the present disclosure includes an acquisition module 1002, a gateway module 1004, and a service module 1006.

[0116] The acquisition module 1002 is used to acquire data service requests. The gateway module 1004 is used to forward data service requests to one or more corresponding service nodes to obtain request tasks based on the type of the data service request. The service module 1006 is used to forward data service requests to one or more corresponding processing engines, which process the request tasks to obtain processing results.

[0117] This disclosure also provides an electronic device, including: a memory storing execution instructions; and a processor or other hardware module executing the execution instructions stored in the memory, causing the processor or other hardware module to perform the above-described method.

[0118] This disclosure also provides a readable storage medium storing executable instructions, which, when executed by a processor, are used to implement the methods described above.

[0119] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the methods described above.

[0120] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0121] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0122] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A database operation method, characterized in that, include: Get data service request; Based on the type of the data service request, the data service request is forwarded to one or more corresponding service nodes to obtain the request task; as well as The data service request is forwarded to one or more corresponding processing engines, which then process the request task to obtain the processing result. The service node includes multiple operation nodes, the request task includes an operation task, and the acquisition of the request task includes: when the data service request type is an operation service, acquiring the operation-affected data table corresponding to the operation service, determining the target operation node corresponding to the operation-affected data table, and forwarding the data service request to the target operation node to acquire the operation task; the processing engine includes multiple operation engines, each operation engine corresponding to a data table, and the acquisition of the processing result includes: based on the operation task, forwarding the data service request to the operation engine corresponding to the operation-affected data table, using the operation engine to process the operation-affected data table in memory according to the operation task to acquire the processing result, and storing the data in memory to a storage node.

2. The database operation method according to claim 1, characterized in that, The service node further includes a definition node, and the request task further includes a definition task and an import task. The step of forwarding the data service request to one or more corresponding service nodes to obtain the request task, based on the type of the data service request, further includes: If the data service request type is a definition service, the data service request is forwarded to the definition node to obtain the definition task; or If the data service request type is import service, the data service request is forwarded to the defined node to obtain the import task.

3. The database operation method according to claim 2, characterized in that, The processing engine further includes a definition engine and an import engine. Forwarding the data service request to one or more corresponding processing engines, and processing the request task through the processing engines to obtain a processing result, also includes: The import engine is started based on the import task, the data service request is forwarded to the import engine, and the import engine is granted data source permissions. The import engine is used to obtain the internal data of the data source according to the import task, and the internal data is stored in the storage node to obtain the processing result; The import engine is destroyed if the processing result is returned to the defined node. Based on the defined task, the data service request is forwarded to the definition engine; The definition engine converts the data service request into a metadata change request based on the defined task, and then forwards the metadata change request to the storage node; and The storage node processes the target stored data according to the metadata change request to obtain the processing result.

4. The database operation method according to claim 1, characterized in that, The service node further includes multiple query nodes, and the request task further includes a query task. The step of forwarding the data service request to one or more corresponding service nodes to obtain the request task, based on the type of the data service request, further includes: If the data service request type is a query service, obtain the query data table corresponding to the query service; Determine the target query node corresponding to the queried data table; and The data service request is forwarded to the target query node to obtain the query task.

5. The database operation method according to claim 4, characterized in that, The processing engine further includes multiple query engines, each corresponding to a data table. Forwarding the data service request to one or more corresponding processing engines, and processing the request task through the processing engines to obtain a processing result, also includes: Based on the query task, the data service request is forwarded to the query engine corresponding to the queried data table; The query engine performs a data query in video memory based on the query task and the queried data table to obtain the processing result; and Load the data in the video memory into the storage node.

6. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the database operation method according to any one of claims 1 to 5.

7. A readable storage medium, characterized in that, The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the database operation method according to any one of claims 1 to 5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the database operation method according to any one of claims 1 to 5.

Images