Method for realizing two-dimensional set data type based on database extension head structure
By constructing a nestable storage structure and a context transfer assignment mechanism at the database kernel level, the problem that relational databases cannot directly support multidimensional nested data is solved, enabling efficient storage and operation of two-dimensional set data and improving data processing efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-24
- Publication Date
- 2026-03-27
AI Technical Summary
Existing relational databases such as openGauss cannot directly support multidimensional nested data structures, resulting in low efficiency in processing complex data. Existing solutions increase the complexity of data modeling and query parsing overhead.
Design a two-dimensional collection data type based on an extended header structure. By constructing a nested storage structure, context transfer assignment, serialization to disk, and deserialization reading, achieve efficient storage and operation of two-dimensional collection data.
It improves the assignment efficiency and storage performance of multidimensional data structures, reduces the query complexity of complex data models, and enhances the database's ability to directly express two-dimensional and higher-dimensional data.
Smart Images
Figure CN121743330A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data type processing technology, and in particular to a method, system, computer-readable storage medium, and electronic device for implementing a two-dimensional collection data type based on a database extension header structure. Background Technology
[0002] In existing relational databases (such as openGauss), the native processing capabilities for complex data types have significant limitations. Taking the Expanded Array Header (EAH) mechanism provided by openGauss as an example, while this mechanism can handle extended objects such as arrays and sets, its design has inherent constraints: the traditional EAH mechanism only supports storing specific data elements and prohibits element nesting; that is, one EAH structure cannot directly contain another EAH structure. This limitation results in the database kernel lacking the ability to directly store and manipulate multi-dimensional nested data structures (such as two-dimensional arrays and nested sets), making it difficult to meet the needs of complex business scenarios.
[0003] In practical applications, users often need to handle complex data structures with multi-dimensional nesting characteristics (such as two-dimensional arrays, hierarchical collections, etc.). However, existing solutions for these needs have significant drawbacks: they typically require indirect storage through multi-table join mapping or complex structure serialization (such as converting to strings). These methods not only introduce additional data modeling complexity and query parsing overhead, but also reduce the overall system performance due to data transformation and join operations.
[0004] In summary, existing technologies have not yet provided a mechanism to directly support nested data types of two-dimensional Collections at the database kernel level. This has become a key bottleneck restricting the efficiency of complex data processing and urgently needs to be overcome by those skilled in the art. Summary of the Invention
[0005] To overcome the aforementioned deficiencies in the existing technology, this application proposes a new method for implementing two-dimensional collection data types based on a database extension header structure.
[0006] This invention aims to provide a method and system for implementing two-dimensional set data types in relational databases. Based on existing extended tuple header mechanisms, a nestable two-dimensional set data storage structure is designed to effectively represent data in the form of two-dimensional arrays. By extending data types, designing serialization and deserialization methods, and adapting database kernel operators and functions, efficient storage, retrieval, and complex calculations of two-dimensional set data are achieved, solving the structural deficiencies of existing one-dimensional array types in expressing two-dimensional and multi-dimensional data models.
[0007] To achieve the above objectives, this application provides the following technical solution: The first aspect of this application provides a method for implementing a two-dimensional collection data type based on a database extension header structure, the method comprising: S1. Construct a nestable storage structure: In the relational database kernel, a two-dimensional collection storage structure that supports recursive nesting is constructed based on the extended header structure EAH. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. S2, Context Transfer Assignment: When assigning values to elements of a two-dimensional set, memory lifetime is transferred through the eoh_context of the source EAH and the target EAH, and the reference of the nested EAH is directly bound to the corresponding index of the dvalues of the target root EAH, thus completing the zero-copy assignment. S3, Serialization to disk: During persistent storage, the root EAH header information, nested EAH header sequence, and specific element data in the last-level dvalues are filled in sequentially from top to bottom to construct a complete nested varlena binary object and write it to the storage medium. S4. Deserialization Reading: During data access, the varlena object on the disk is parsed to reconstruct the root EAH header. The nested EAH sequence stored in the dvalues of the root EAH is traversed based on the index, and the element values of the last dvalues data area are read recursively.
[0008] Furthermore, in the method of this application, in the step of constructing a nestable storage structure: each index position in the dvalues data area of the root EAH stores a nested EAH head pointer, and the nested EAH itself constitutes a complete varlena object, and its dvalues data area stores specific data elements in sequence, thereby realizing the hierarchical expression of the two-dimensional set.
[0009] Furthermore, in the method of this application, during the context transfer assignment step: when performing the initialization assignment of the two-dimensional set, the rvalue EAH object generated by the constructor adjusts the memory context pointed to by its eoh_context to the memory context where the lvalue variable is located, so that the life cycle of the rvalue EAH object is synchronized with the lvalue variable, thus avoiding deep copy operations of the EAH structure and dvalues data.
[0010] Furthermore, in the method of this application, the serialization and disk writing step specifically includes: Calculate the total length of the root EAH header, all nested EAH headers, and the last element data, and allocate storage space for varlena. Sequentially serialize the root EAH metadata to the buffer header; Iterate through the dvalues of the root EAH and append the data of each nested EAH header and its dvalues to the buffer in turn. Write the complete buffer as an atomic varlena object to the storage engine.
[0011] Furthermore, in the method of this application, in the deserialization reading step: For one-dimensional access operations on a two-dimensional set, the corresponding nested EAH head in the dvalues of the root EAH is located directly by the index and the nested EAH object is returned. For two-dimensional access operations on a two-dimensional set, first perform a one-dimensional access to obtain the nested EAH object, and then locate the corresponding element value in the dvalues of the nested EAH according to the second-dimensional index and return it.
[0012] Furthermore, the method of this application also includes: Type system extension: Define nested collection type metadata in the database type system, register nested EAH types as legal element subtypes of the root EAH type, and declare dimension recursion rules.
[0013] Furthermore, the method of this application also includes: Operator adaptation: Overload the database kernel's index access operators, assignment operators, and comparison operators to enable them to recognize nested EAH structures and recursively perform element-level operations on the dvalues data area.
[0014] A second aspect of this application provides a system for implementing a two-dimensional collection data type based on a database extension header structure. The system, when running, implements the steps of the aforementioned method for implementing a two-dimensional collection data type based on a database extension header structure. The system includes: The storage structure building module is used to build a two-dimensional collection storage structure that supports recursive nesting based on the extended header structure EAH in the relational database kernel. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. The assignment processing module is used to assign values to elements of a two-dimensional set. It performs memory lifetime transfer through the eoh_context of the source EAH and the target EAH, and directly binds the reference of the nested EAH to the corresponding index of the dvalues of the target root EAH to complete the zero-copy assignment. The serialization module is used to fill in the root EAH header information, nested EAH header sequence and specific element data in the last layer dvalues in a top-down manner during persistent storage, construct a complete nested varlena binary object and write it to the storage medium. The deserialization module is used to parse the varlena object on disk to reconstruct the root EAH header during data access, traverse the nested EAH sequence stored in the dvalues of the root EAH based on the index, and recursively read the element values of the last-level dvalues data area.
[0015] A third aspect of this application provides an electronic device, including: a memory and a processor; Memory: Used to store computer programs; Processor: Used to execute the computer program to implement the steps of the aforementioned method for implementing a two-dimensional collection data type based on a database extended header structure.
[0016] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for implementing a two-dimensional collection data type based on a database extension header structure.
[0017] In summary, this invention achieves native support for two-dimensional collection data types at the database kernel level through a nested EAH hierarchical varlena storage architecture and the eoh_context context transfer mechanism. Compared with traditional one-dimensional arrays, it significantly improves the assignment efficiency and storage performance of multi-dimensional data structures, reduces the implementation complexity of complex data model queries, and enhances the database's ability to directly express two-dimensional and higher-dimensional data.
[0018] Other features and advantages of this application will be set forth in detail in the following description, or will become apparent through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and means explicitly pointed out in the description, claims, and drawings, and will be obtained through the implementation of these technical contents. Attached Figure Description
[0019] To more clearly illustrate the technical solutions of this application, the accompanying drawings involved in the description of the embodiments will be briefly introduced below. It should be noted that the drawings only show some embodiments of this application. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the overall implementation process of the method for implementing a two-dimensional collection data type based on a database extended header structure, as described in this application.
[0021] Figure 2 This is a schematic diagram of the existing EAH memory structure.
[0022] Figure 3This is a schematic diagram of the memory structure of EAH after the extension of the present invention.
[0023] Figure 4 This is a structural diagram of the system that implements a two-dimensional collection data type based on the database extension header structure in this application.
[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.
[0026] In this document, the term "comprising" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not exhaustive and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors. The term "one embodiment" should be understood as "at least one embodiment," meaning that the described embodiment is not the only possible implementation, and other similar embodiments may exist.
[0027] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."
[0028] Figure 1 The diagram shows the overall implementation flow of the method for implementing a two-dimensional collection data type based on a database extended header structure provided in this application, including the following steps: S1. Construct a nestable storage structure: In the relational database kernel, a two-dimensional collection storage structure that supports recursive nesting is constructed based on the extended header structure EAH. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. S2, Context Transfer Assignment: When assigning values to elements of a two-dimensional set, memory lifetime is transferred through the eoh_context of the source EAH and the target EAH, and the reference of the nested EAH is directly bound to the corresponding index of the dvalues of the target root EAH, thus completing the zero-copy assignment. S3, Serialization to disk: During persistent storage, the root EAH header information, nested EAH header sequence, and specific element data in the last-level dvalues are filled in sequentially from top to bottom to construct a complete nested varlena binary object and write it to the storage medium. S4. Deserialization Reading: During data access, the varlena object on the disk is parsed to reconstruct the root EAH header. The nested EAH sequence stored in the dvalues of the root EAH is traversed based on the index, and the element values of the last dvalues data area are read recursively.
[0029] The core of this invention is the design of a nestable storage structure based on EAH, which organizes a two-dimensional collection into a hierarchical varlena object.
[0030] The existing memory structure of EAH is as follows: Figure 2 As shown, in the EAH structure, eoh_context records the current context information, and dvalues stores the actual data, which can be directly modified and accessed based on the index. For example... Figure 3 As shown, this invention extends the dvalues data area. The root EAH serves as the entry point for the entire two-dimensional set. It is a standard varlena header structure. In its dvalues, the EAH headers of nested collections are stored, and the specific elements are stored in the dvalues data area of each nested collection.
[0031] When it is necessary to assign a value to a two-dimensional collection element, the target EAH is directly transferred according to the eoh_context on the EAH structure. This avoids the need to reconstruct a new array and completes the array assignment by modifying the memory lifetime, which greatly improves the assignment efficiency.
[0032] When data needs to be written to disk, the root EAH, nested EAHs, and element data are sequentially filled according to the storage structure described above, ultimately constructing a complete varlena object, which is then written to disk. Simultaneously, when output or computation is required, the database kernel parses the root EAH header, then traverses the nested EAH sequence, reading data line by line and element by element to complete deserialization.
[0033] To more clearly illustrate the technical solution of this application, the following will provide further explanation through embodiments of actual use scenarios.
[0034] DECLARE type arr1 is a table of int; Type arr1_nested is table of arr1; var1_nested arr1_nested; BEGIN var1_nested := arr1_nested(arr1(1,2), arr1(3,4)); raise info '%', var1_nested(1); raise info '%', var1_nested(1)(1); END; / In the example above, `var1_nested arr1_nested := arr1_nested(arr1(1,2), arr1(3,4))` uses a constructor to generate the rvalue of the expression, initializing the two-dimensional collection. During assignment, the lifecycle of the rvalue is modified to directly assign the value to the lvalue, avoiding array copying. In `raise info '%',var1_nested(1);`, an element of a specific dimension in the two-dimensional collection is retrieved directly based on its index, obtaining the dvalues data corresponding to the root EAH. In `raise info '%', var1_nested(1)(1);`, after retrieving the two-dimensional collection by index, the specific data is retrieved from the dvalues data area of the two-dimensional collection based on the index and returned.
[0035] Figure 4 The figure shown is a system for implementing a two-dimensional collection data type based on a database extended header structure, as proposed in this application. The system includes: The storage structure building module is used to build a two-dimensional collection storage structure that supports recursive nesting based on the extended header structure EAH in the relational database kernel. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. The assignment processing module is used to assign values to elements of a two-dimensional set. It performs memory lifetime transfer through the eoh_context of the source EAH and the target EAH, and directly binds the reference of the nested EAH to the corresponding index of the dvalues of the target root EAH to complete the zero-copy assignment. The serialization module is used to fill in the root EAH header information, nested EAH header sequence and specific element data in the last layer dvalues in a top-down manner during persistent storage, construct a complete nested varlena binary object and write it to the storage medium. The deserialization module is used to parse the varlena object on disk to reconstruct the root EAH header during data access, traverse the nested EAH sequence stored in the dvalues of the root EAH based on the index, and recursively read the element values of the last-level dvalues data area.
[0036] The above system implements the steps of the method for implementing a two-dimensional collection data type based on a database extension header structure disclosed in this application when running.
[0037] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of systems, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.
[0038] like Figure 5 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-described method for implementing a two-dimensional collection data type based on a database extended header structure.
[0039] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).
[0040] The operations described in this application can be implemented by writing computer program code using one or more programming languages or a combination thereof. The programming languages include, but are not limited to, the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc. Conventional procedural programming languages, such as "C" or similar programming languages.
[0041] The execution methods of program code include, but are not limited to: It runs entirely on the user's computer; Part of it executes on the user's computer, and part of it executes on a remote computer; Execute as a standalone software package; It is executed entirely on a remote computer or server.
[0042] In scenarios involving remote computers, the remote computer can connect to the user's computer via any type of network, including but not limited to local area networks (LANs) or wide area networks (WANs). Furthermore, the remote computer can also connect to external computers through an internet service provider, for example, by utilizing the internet for connection.
[0043] Furthermore, this application also discloses a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the various steps of the method for implementing a two-dimensional set data type based on a database extension header structure disclosed in this application.
[0044] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: such as floppy disks and other removable magnetic storage media.
[0045] (2) Hard disk: including mechanical hard disks and solid-state hard disks and other fixed storage devices.
[0046] (3) Random Access Memory (RAM): A volatile storage medium used for temporary storage of data and program code.
[0047] (4) Read-only memory (ROM): a non-volatile storage medium used to store fixed programs and data.
[0048] (5) Erasable programmable read-only memory (EPROM) or flash memory: non-volatile storage media that supports multiple erasures and reprogrammings.
[0049] (6) Fiber optic storage devices: storage media based on fiber optic technology.
[0050] (7) Portable compact disc read-only memory (CD-ROM): a read-only medium that stores data in the form of an optical disc.
[0051] (8) Optical storage devices: such as DVDs, Blu-ray discs and other storage media based on optical principles.
[0052] (9) Magnetic storage devices: such as magnetic tapes, disks and other storage media based on magnetic principles.
[0053] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage needs.
[0054] These computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.
[0055] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program includes program code for executing the method disclosed in this application for implementing a two-dimensional collection data type based on a database extended header structure. When this computer program is executed by a processing system, it can achieve the functions defined in the embodiments of this application.
[0056] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.
[0057] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for implementing a two-dimensional collection data type based on a database extended header structure, characterized in that, The method includes: S1. Construct a nestable storage structure: In the relational database kernel, a two-dimensional collection storage structure that supports recursive nesting is constructed based on the extended header structure EAH. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. S2, Context Transfer Assignment: When assigning values to elements of a two-dimensional set, memory lifetime is transferred through the eoh_context of the source EAH and the target EAH, and the reference of the nested EAH is directly bound to the corresponding index of the dvalues of the target root EAH, thus completing the zero-copy assignment. S3, Serialization to disk: During persistent storage, the root EAH header information, nested EAH header sequence, and specific element data in the last-level dvalues are filled in sequentially from top to bottom to construct a complete nested varlena binary object and write it to the storage medium. S4. Deserialization Reading: During data access, the varlena object on the disk is parsed to reconstruct the root EAH header. The nested EAH sequence stored in the dvalues of the root EAH is traversed based on the index, and the element values of the last dvalues data area are read recursively.
2. The method according to claim 1, characterized in that, In the step of constructing a nested storage structure: each index position in the dvalues data area of the root EAH stores a nested EAH head pointer. The nested EAH itself constitutes a complete varlena object, and its dvalues data area stores specific data elements in sequence, thereby realizing the hierarchical expression of the two-dimensional set.
3. The method according to claim 1, characterized in that, In the context transfer assignment step: when the two-dimensional set initialization assignment is performed, the rvalue EAH object generated by the constructor adjusts the memory context pointed to by its eoh_context to the memory context where the lvalue variable is located, so that the life cycle of the rvalue EAH object is synchronized with the lvalue variable, avoiding deep copy operation of EAH structure and dvalues data.
4. The method according to claim 1, characterized in that, The serialization and disk writing step specifically includes: Calculate the total length of the root EAH header, all nested EAH headers, and the last element data, and allocate storage space for varlena. Sequentially serialize the root EAH metadata to the buffer header; Iterate through the dvalues of the root EAH and append the data of each nested EAH header and its dvalues to the buffer in turn. Write the complete buffer as an atomic varlena object to the storage engine.
5. The method according to claim 1, characterized in that, In the deserialization reading step: For one-dimensional access operations on a two-dimensional set, the corresponding nested EAH head in the dvalues of the root EAH is located directly by the index and the nested EAH object is returned. For two-dimensional access operations on a two-dimensional set, first perform a one-dimensional access to obtain the nested EAH object, and then locate the corresponding element value in the dvalues of the nested EAH according to the second-dimensional index and return it.
6. The method according to claim 1, characterized in that, The method further includes: Type system extension: Define nested collection type metadata in the database type system, register nested EAH types as legal element subtypes of the root EAH type, and declare dimension recursion rules.
7. The method according to claim 1, characterized in that, The method further includes: Operator adaptation: Overload the database kernel's index access operators, assignment operators, and comparison operators to enable them to recognize nested EAH structures and recursively perform element-level operations on the dvalues data area.
8. A system for implementing a two-dimensional set data type based on a database extended header structure, characterized in that, The system, when running, implements the steps of the method for implementing a two-dimensional collection data type based on a database extension header structure as described in any one of claims 1-7, the system comprising: The storage structure building module is used to build a two-dimensional collection storage structure that supports recursive nesting based on the extended header structure EAH in the relational database kernel. The root EAH is configured as a varlena header object, and the dvalues data area is set to store the EAH header of the nested collection instead of the direct elements, forming a hierarchical varlena object. The assignment processing module is used to assign values to elements of a two-dimensional set. It performs memory lifetime transfer through the eoh_context of the source EAH and the target EAH, and directly binds the reference of the nested EAH to the corresponding index of the dvalues of the target root EAH to complete the zero-copy assignment. The serialization module is used to fill in the root EAH header information, nested EAH header sequence and specific element data in the last layer dvalues in a top-down manner during persistent storage, construct a complete nested varlena binary object and write it to the storage medium. The deserialization module is used to parse the varlena object on disk to reconstruct the root EAH header during data access, traverse the nested EAH sequence stored in the dvalues of the root EAH based on the index, and recursively read the element values of the last-level dvalues data area.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for implementing a two-dimensional collection data type based on a database extension header structure as described in any one of claims 1-7.
10. An electronic device, characterized in that, include: Memory and processor; Memory: Used to store computer programs; Processor: for executing the computer program to implement the steps of the method for implementing a two-dimensional collection data type based on a database extended header structure as described in any one of claims 1-7.