A method for processing database tuples and related products

Abstract

The application provides a database tuple processing method and related products. The method comprises the following steps: dividing the tuple into a tuple header, a column data area and a column offset table area which are arranged continuously in an address space; for tuple writing of a target data table, starting from a position which is offset from a tuple start address by a length of the tuple header, each column data is written into the column data area in sequence, and the column offset of each column data relative to the tuple start address is recorded; after all the column data is written, the column offset is arranged into a column offset table and written into the column offset table area, the length value of the tuple header is determined as a first offset, the sum of the length of the tuple header and the length of the column data area is determined as a second offset, and the first offset and the second offset are stored into the tuple header. The scheme of the application realizes direct addressing of any column data, eliminates the dependence of column access on the previous sequence analysis result, and eliminates the problem that the tuple deformation time linearly increases with the number of columns.

Description

Technical Field

[0001] This invention relates to database technology, and in particular to a method for processing database tuples and related products. Background Technology

[0002] Row-oriented database tuples are stored contiguously, consisting of a tuple header and a fixed-length / variable-length data area. When executing a query, the tuple must be transformed into a logical representation operable by the executor (e.g., a logical format of a datum array + isnull bitmap). In practice, tuple transformation primarily employs sequential parsing or partially variable-length offset table parsing. This parsing method relies on the parsing results of the preceding sequence for accessing any column, causing the tuple transformation time to increase linearly with the number of columns in the table. This becomes a performance bottleneck in wide-table scenarios. Furthermore, when the same tuple is accessed multiple times, the exact same offset calculation and prefix read must be repeated each time, resulting in wasted CPU resources. Summary of the Invention

[0003] One objective of this invention is to improve the processing performance of row-oriented database tuple transformations and overcome the performance bottleneck caused by sequential dependencies.

[0004] A further objective of this invention is to eliminate the overhead of redundant calculations when the same tuple is accessed multiple times, thereby enabling the reuse of offset calculation results.

[0005] Specifically, the present invention provides a method for processing database tuples, comprising:

[0006] The tuple is divided into a tuple header, column data area, and column offset table area that are arranged consecutively in the address space;

[0007] In response to a tuple write to the target data table, starting from the position offset from the tuple start address by the length of the tuple header, each column of data is sequentially written into the column data area, and the column offset of each column of data relative to the tuple start address is recorded.

[0008] After all column data has been written, the column offsets are organized into a column offset table and written into the column offset table area. The length value of the tuple header is determined as the first offset, and the sum of the tuple header length and the column data area length is determined as the second offset. The first offset and the second offset are stored in the tuple header.

[0009] Optionally, the step of sequentially writing each column of data into the column data area includes:

[0010] Maintain a write pointer, which is initialized to point to the beginning of the column data area before performing a tuple write operation;

[0011] If the column to be written is a fixed-length data column and is not empty, the data of the fixed-length data column is written to the memory address pointed to by the write pointer according to the preset byte length corresponding to the fixed-length data column;

[0012] If the column to be written is a variable-length data column and is not null, the valid data of the variable-length data column is written to the memory address pointed to by the write pointer after removing the prefix field, based on the valid data length indicated by the prefix field of the variable-length data column.

[0013] Optionally, the step of sequentially writing each column of data into the column data area further includes:

[0014] If the column to be written is a fixed-length data column and is empty, no data is written to the column data area, and the write pointer remains unchanged; or, according to the preset byte length corresponding to the fixed-length data column, preset empty value fill data is written to the memory address pointed to by the write pointer, and the write pointer is moved backward by the preset byte length.

[0015] If the column to be written is a variable-length data column and is empty, the write operation of the variable-length data column is not performed, and the write pointer remains unchanged.

[0016] Optionally, the step of recording the column offset of each column of data relative to the starting address of the tuple includes:

[0017] A column offset table is configured at the head of the tuple. The number of offset fields contained in the column offset table is the same as the number of columns contained in the tuple to be written, and each offset field corresponds to one column.

[0018] When writing the first column of data into the column data area, the column offset of the first column of data is determined based on the difference between the initial value of the write pointer and the starting address of the tuple to be written, and written into the offset field corresponding to the first column in the column offset table;

[0019] Before writing each subsequent column of data into the column data area, the column offset of the current column to be written is determined based on the difference between the current value of the write pointer and the starting address of the tuple to be written, and written into the offset field corresponding to the current column to be written in the column offset table.

[0020] Optionally, the method for processing the database tuples further includes:

[0021] In response to a read request for a target tuple in a target data table, the first offset and the second offset are obtained from the tuple header of the target tuple, the read request including the target column number;

[0022] The column offset table area is located according to the second offset, and the column offset corresponding to the target column in the target tuple is read from the column offset table area according to the target column number;

[0023] Verify that the column offset is greater than or equal to the first offset; if the verification is successful, locate and extract the data of the target column from the starting address of the target tuple based on the column offset; otherwise, return a data corruption error.

[0024] Optionally, after returning a data corruption error, the following may also be included:

[0025] Obtain all column offsets recorded in the column offset table area of ​​the target tuple to form a set of column offsets to be verified.

[0026] Verify one by one whether each column offset in the set of column offsets to be verified meets the preset validity condition, wherein the validity condition is that the column offset is greater than or equal to the first offset and less than or equal to the second offset;

[0027] If any column offset does not meet the validity condition, the target tuple is determined to have suffered storage structure damage. The complete data of the target tuple is read from the backup medium according to the preset recovery strategy, and the complete data is written sequentially from the starting address of the target tuple to overwrite the damaged data.

[0028] Optionally, the method for processing the database tuples further includes:

[0029] In response to a table structure change operation targeting a target data table, each tuple in the target data table is traversed, where the table structure change operation includes any one of adding a column, deleting a column, or adjusting the column order.

[0030] For the current tuple, the column offset of each column in the column data area is re-determined according to the changed column order;

[0031] Write the redefined column offset into the column offset table area;

[0032] The first offset is re-determined based on the updated actual length of the column offset table area, and the second offset is re-determined based on the updated first offset and the actual length.

[0033] The updated first and second offsets are stored in the tuple header of the current tuple.

[0034] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the database tuple processing method described above.

[0035] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of the database tuple processing method described above.

[0036] According to another aspect of the present invention, a computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any database tuple processing method.

[0037] The database tuple processing method of this invention provides a unified address space basis for rapid column data location by storing each column of data contiguously in the column data area and centrally recording the offset of each column in the column offset table area, thereby improving the regularity and addressability of tuple storage. During writing, the column offset relative to the tuple's starting address is recorded column by column. Combined with the subsequently organized column offset table, direct addressing of any column data is achieved, eliminating the dependence of column access on the parsing results of the preceding sequence and eliminating the problem of tuple transformation time increasing linearly with the number of columns. The column offset table is stored independently in the column offset table area, and the first and second offsets are fixed in the tuple header, enabling rapid location of the boundary between the column data area and the column offset table area. This significantly simplifies the reading and addressing logic of column data and the column offset table, improving the execution efficiency of tuple transformation. The recording of column offsets and the construction of the column offset table are completed in one go during the writing phase. Subsequent accesses to the tuple can directly reuse this offset information without repeatedly performing column offset calculations and variable-length column prefix reading operations, reducing CPU resource consumption.

[0038] Furthermore, the database tuple processing method of the present invention can directly locate the column offset table area based on the first offset and second offset pre-stored in the tuple header during reading, and quickly obtain the column offset by combining the target column number, realizing O(1) level addressing of the target column data, eliminating the dependence on the parsing results of the previous sequence, and significantly reducing the time consumption of tuple deformation in wide table scenarios. The addition of a validity verification step for the column offset and the first offset in the reading process can promptly identify offset errors caused by storage anomalies, table structure changes, etc., avoid reading invalid or corrupted data, and improve the accuracy and reliability of tuple data reading.

[0039] Furthermore, the database tuple processing method of this invention supports all types of table structure change operations, such as adding columns, deleting columns, and adjusting column order. It eliminates the need to reconstruct the overall tuple storage layout; only column offsets and offset markers in the tuple header are updated to adapt to structure changes, significantly reducing the execution cost and time consumption of table structure changes. For table structure changes, the method traverses the tuples and recalculates column offsets to ensure that the offset markers of each column data accurately match the physical storage location after the change. This guarantees the accuracy of column data addressing during tuple reading and avoids data reading errors caused by structure changes. After the change, only the column offset table area and the first and second offsets in the tuple header are updated; the original data in the column data area does not need to be modified, reducing data rewriting and storage overhead, and adapting to structure change scenarios for wide tables and large-volume data tables.

[0040] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0041] The following sections will describe some specific embodiments of the invention in a detailed manner by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0042] Figure 1 This is a schematic flowchart of a database tuple processing method according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic flowchart illustrating the writing of column data into the column data area in a database tuple processing method according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic flowchart illustrating the record column offset in a database tuple processing method according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic flowchart illustrating a read request in a database tuple processing method according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic flowchart illustrating a database tuple processing method according to an embodiment of the present invention that returns a data corruption error;

[0047] Figure 6 This is a schematic flowchart of a table structure change operation in a database tuple processing method according to an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and

[0050] Figure 9 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0051] In existing row-oriented databases, tuples employ a physically contiguous storage layout, consisting of a tuple header, a fixed-length data area, and a variable-length data area arranged sequentially. The tuple header stores metadata such as version information and NULL bitmaps. Fixed-length data columns are compactly stored in the fixed-length data area according to their declaration order, while variable-length data columns such as VARCHAR and TEXT are stored centrally in the variable-length data area. This layout naturally suits OLTP (Online Transaction Processing) transaction operations such as single-row primary key lookups and single-row updates, making it a classic storage format for row-oriented databases.

[0052] When performing various query operations in a database, the database executor cannot directly process the raw byte data in the physical storage format. It must first transform the physical tuples into a standardized logical representation that the executor can directly recognize and manipulate. Currently, the commonly used logical representation in the industry is a combination of datum arrays and isnull bitmaps. This form decouples the physical storage details from the logical execution layer, allowing the executor's computational logic to be unconcerned about the specific format of the underlying storage, thereby ensuring the modular design of the database kernel layer.

[0053] In practical database applications, the transformation of physical tuples into the aforementioned logical representation is typically accomplished using sequential parsing or parsing based on partially variable-length offset tables. Both of these parsing methods suffer from significant order dependency: parsing, locating, and accessing any target column requires first calculating the length of all preceding columns, accumulating offsets, or performing prefix parsing, making random access to column data impossible. This causes the overall processing time for tuple transformation to increase linearly with the number of columns in the table. In wide tables with over a hundred columns, the time consumed by tuple transformation increases dramatically, becoming a core bottleneck restricting the overall query performance of the database.

[0054] In real-world business scenarios with high-concurrency OLTP and HTAP (Hybrid Transactional / Analytical Processing) workloads, the same physical tuple is often repeatedly accessed due to MVCC (Multiversion Concurrency Control) visibility checks, multiple table lookups, and hot row cache hits. However, with existing parsing methods, each access to this tuple requires starting from the beginning and repeatedly performing the exact same column offset calculations and variable-length column prefix reads. Previously completed calculations cannot be effectively cached and reused. This large amount of deterministic, repetitive calculations with no real business value not only consumes a significant number of CPU cycles, severely depleting computing resources, but also reduces the database's concurrent processing capabilities, further amplifying performance issues in wide table scenarios.

[0055] This embodiment provides a solution for processing database tuples. While maintaining the original physical storage layout of the row-oriented database, it achieves O(1) time addressing and zero-decoding access for arbitrary columns in the tuple.

[0056] Figure 1 This is a schematic flowchart of a database tuple processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the processing method for this database tuple generally includes:

[0057] Step S101: Divide the tuple into a tuple header, column data area and column offset table area that are arranged continuously in the address space;

[0058] Step S102: In response to the tuple writing to the target data table, starting from the position offset from the tuple start address by the length of the tuple header, the data of each column is written continuously into the column data area, and the column offset of each column relative to the tuple start address is recorded.

[0059] Step S103: After all column data is written, organize the column offsets into a column offset table and write it into the column offset table area. Then, determine the length value of the tuple header as the first offset, and determine the sum of the tuple header length and the column data area length as the second offset. Store the first offset and the second offset in the tuple header.

[0060] In this embodiment, based on the address continuity requirements of row-based databases for physical storage, the physical address space of a single tuple is structurally divided into three continuous and non-overlapping address segments: the tuple header, the column data area, and the column offset table area. The tuple header is a fixed first address segment used to store tuple metadata and offset location identifiers; the column data area is the middle core address space used to continuously store the original data of all columns; and the column offset table area is the last address segment used to store the offset information of each column's data. These three address segments are sequentially connected in physical storage without any address gaps.

[0061] In this embodiment, when the database receives a tuple write instruction (including INSERT, UPDATE, etc.) for the target data table, it first obtains the preset fixed length of the tuple header. Based on the physical starting address of the tuple to be written, it offsets the header length backward to obtain the starting address of the column data area, which serves as the starting position for writing the column data. Then, according to the column declaration order of the target data table, the data of each column is written sequentially and continuously into the column data area without gaps. Both fixed-length and variable-length data columns are stored in the original data format (the length prefix is ​​removed from the variable-length data columns). After each column of data is written, the offset of the starting address of the column data relative to the physical starting address of the tuple is calculated and recorded in real time, forming a column offset set. The calculation precision of the offset is consistent with the address granularity stored in the database.

[0062] In this embodiment, after all column data of the target tuple has been written and the physical address space of the column data area has been determined, the set of column offsets recorded in step S102 is organized according to the column declaration order to form a column offset table corresponding one-to-one with the number of columns. This column offset table is then completely written to the starting address of the column offset table area, completing the physical storage of the offset table. Simultaneously, the actual physical length of the tuple header is assigned as the first offset, serving as the starting positioning identifier of the column data area. The second offset is obtained by calculating the sum of the tuple header length and the actual physical length of the column data area, serving as the starting positioning identifier of the column offset table area. Finally, the first and second offsets are written into a preset field in the tuple header in a fixed format, completing the solidification of the offset positioning information throughout the tuple writing process.

[0063] Based on the above steps, by storing each column of data contiguously in the column data area and centrally recording the offsets of each column in the column offset table area, a unified address space foundation is provided for rapid column data location, improving the regularity and addressability of tuple storage. During writing, the column offset relative to the tuple's starting address is recorded column by column. Combined with the subsequently organized column offset table, direct addressing of any column data is achieved, eliminating the dependence of column access on the previous sequence parsing results and eliminating the problem of tuple transformation time increasing linearly with the number of columns. The column offset table is stored independently in the column offset table area, and the first and second offsets are fixed in the tuple header, enabling rapid location of the boundary between the column data area and the column offset table area. This significantly simplifies the reading and addressing logic of column data and the column offset table, improving the execution efficiency of tuple transformation. The recording of column offsets and the construction of the column offset table are completed in one go during the writing phase. Subsequent accesses to the tuple can directly reuse this offset information without repeatedly performing column offset calculations and variable-length column prefix reading operations, reducing CPU resource consumption.

[0064] Figure 2 This is a schematic flowchart illustrating the writing of column data to the column data area in a database tuple processing method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the steps for sequentially writing data from each column into the column data area include:

[0065] Step S201: Maintain the write pointer, which is initialized to point to the beginning of the column data area before the tuple write operation is performed;

[0066] Step S202: If the column to be written is a fixed-length data column and is not empty, write the data of the fixed-length data column to the memory address pointed to by the write pointer according to the preset byte length corresponding to the fixed-length data column.

[0067] Step S203: If the column to be written is a variable-length data column and is not null, according to the effective data length indicated by the prefix field of the variable-length data column, after removing the prefix field, the effective data of the variable-length data column is written to the memory address pointed to by the write pointer.

[0068] In this embodiment, after triggering a tuple write operation in the target data table, a write pointer is constructed and maintained for the column data area. This write pointer is a memory address addressing identifier, and its addressing granularity is consistent with the byte granularity stored in the database. Before performing any column data write operations, this write pointer is initialized so that it precisely points to the physical starting memory address of the column data area, serving as the starting address for all column data writes. During subsequent column data writes, the pointer is dynamically updated as the data write progresses, always pointing to the next free memory address in the column data area.

[0069] In this embodiment, the column to be written is traversed and the column attributes and value status are determined. If the current column to be written is a fixed-length data column and the value of the column is not null, the fixed-length data column is first retrieved from the target data table of the database with a predefined fixed byte length (i.e., preset byte length). Starting from the memory address currently pointed to by the write pointer, the original data of the fixed-length data column is written completely and continuously into the corresponding memory address range according to the preset byte length. After the writing is completed, the writing pointer is synchronously updated to jump to the address of the next byte after the end of the data written this time.

[0070] In this embodiment, if the column to be written is a variable-length data column and the value of the column is not null, the prefix field of the original data of the variable-length data column is first parsed to obtain the actual effective data length of the column data. Then, the original data is preprocessed to remove the length prefix field it carries, and only the effective data part is retained. Then, with the memory address currently pointed to by the write pointer as the starting position, the effective data of the variable-length data column is continuously written to the corresponding memory address range according to the parsed effective data length. After the writing is completed, the write pointer is updated to jump to the address of the next byte after the end of the current effective data writing.

[0071] It should be noted that the steps of sequentially writing data from each column to the column data area also include:

[0072] If the column to be written is a fixed-length data column and is empty, no data is written to the column data area, and the write pointer remains unchanged; or, according to the preset byte length corresponding to the fixed-length data column, the preset empty value is filled with data and written to the memory address pointed to by the write pointer, and the write pointer is moved backward by the preset byte length; if the column to be written is a variable-length data column and is empty, the write operation of the variable-length data column is not performed, and the write pointer remains unchanged.

[0073] Based on the above steps, differentiated adaptation solutions are designed for null value scenarios in fixed-length and variable-length data columns through null value / non-null value write processing logic. This retains the flexibility of null value handling while strictly adhering to the core rule of continuous writing to the column data area, ensuring the regularity and consistency of the physical storage layout of tuples. For fixed-length data columns with null values, dual modes are supported: "no write + pointer unchanged" or "preset padding + pointer offset." This adapts to different database storage design requirements. The no-write mode saves storage space in the column data area and is suitable for wide tables with multiple null values. The padding mode ensures byte alignment of fixed-length column storage and is compatible with the underlying database memory addressing and data reading specifications. For variable-length null values, the write is skipped directly and the pointer remains unchanged, eliminating the need for prefix parsing and storage address allocation. This eliminates the unnecessary processing overhead when variable-length data columns have null values ​​and avoids interference from null values ​​on column offset records, ensuring continuous storage of valid data in variable-length columns.

[0074] Figure 3 This is a schematic flowchart illustrating the record column offset in a database tuple processing method according to an embodiment of the present invention, as shown below. Figure 3 As shown, the steps for recording the column offset of each column of data relative to the starting address of the tuple include:

[0075] Step S301: Configure a column offset table in the tuple header. The number of offset fields in the column offset table is the same as the number of columns in the tuple to be written, and each offset field corresponds to one column.

[0076] Step S302: When writing the first column of data into the column data area, the column offset of the first column of data is determined based on the difference between the initial value of the write pointer and the starting address of the tuple to be written, and written into the offset field corresponding to the first column in the column offset table.

[0077] Step S303: Before writing each subsequent column data into the column data area, determine the column offset of the current column data to be written based on the difference between the current value of the write pointer and the starting address of the tuple to be written, and write it into the offset field corresponding to the current column to be written in the column offset table.

[0078] In this embodiment, a column offset table is constructed within a preset storage area in the tuple header, which is completely matched with the number of columns in the target data table. This column offset table is a fixed-length structured data unit, and the number of offset fields it contains corresponds one-to-one with the total number of columns in the tuple to be written. Each offset field is pre-allocated with a fixed byte length of storage space, and the field order is consistent with the column declaration order of the target data table. A single field is dedicated to storing the offset information of the corresponding column data relative to the starting address of the tuple.

[0079] In this embodiment, when the operation of writing the first column of data to the column data area is triggered, the difference between the initial value of the write pointer (the starting address of the column data area) and the physical starting address of the tuple to be written is first obtained. This difference is the column offset of the first column of data. Then, the column offset is written into the offset field corresponding to the first column in the column offset table according to the preset byte format, and the solidification record of the first column offset is completed.

[0080] In this embodiment, for the second column and all subsequent columns to be written, in the pre-stage of writing the data of that column to the column data area, the current memory address value of the write pointer is read in real time, the difference between the value and the starting address of the tuple is calculated, and the column offset of the current column to be written is obtained; then the offset is written to the corresponding offset field in the column offset table in column order, and the recording of the current column offset is completed; after each column offset is recorded, the process waits for the data of that column to be written to be completed and the write pointer is updated, and then the above process is repeated to process the next column until all column offsets are recorded.

[0081] Based on the above steps, by pre-setting a column offset table corresponding one-to-one with the column number in the tuple header, the offset and column are precisely bound, avoiding chaotic column offset storage, providing a standardized data carrier for O(1) level column addressing, and ensuring the consistency of offset parsing when reading tuples. The column offset of the first column of data is calculated based on the difference between the initial value of the write pointer and the starting address of the tuple, directly anchoring the starting position of the column data area, ensuring the accuracy of the first column offset from the root, and laying a reliable benchmark for the chained recording of subsequent column offsets. The column offset is calculated and stored before each column of data is written, rather than being calculated backtracked after writing, avoiding column offset calculation errors caused by address changes due to data writing, while ensuring that the column offset recording and data writing process are parallelized, improving the overall writing efficiency of tuples.

[0082] Figure 4 This is a schematic flowchart illustrating a read request in a database tuple processing method according to an embodiment of the present invention, as shown below. Figure 4 As shown, the processing method for this database tuple also includes:

[0083] Step S401: In response to a read request for a target tuple in the target data table, obtain a first offset and a second offset from the tuple header of the target tuple. The read request includes the target column number.

[0084] Step S402: Locate the column offset table area according to the second offset, and read the column offset corresponding to the target column in the target tuple from the column offset table area according to the target column number;

[0085] Step S403: Verify that the column offset is greater than or equal to the first offset; if the verification passes, locate and extract the data of the target column from the starting address of the target tuple based on the column offset; otherwise, return a data corruption error.

[0086] Based on the above steps, the column offset table area can be directly located by using the first and second offsets pre-stored in the tuple header during reading. Combined with the target column number, the column offset can be quickly obtained, achieving O(1) level addressing of the target column data. This eliminates the dependence on the parsing results of the previous sequence and significantly reduces the time consumption for tuple deformation in wide table scenarios. Adding a validity verification step for the column offset and the first offset to the reading process can promptly identify offset errors caused by storage anomalies, table structure changes, etc., avoiding the reading of invalid or corrupted data and improving the accuracy and reliability of tuple data reading.

[0087] Figure 5 This is a schematic flowchart illustrating a database tuple processing method according to an embodiment of the present invention, in which a data corruption error is returned. Figure 5 As shown, after returning a data corruption error, it also includes:

[0088] Step S501: Obtain all column offsets recorded in the column offset table area of ​​the target tuple to form a set of column offsets to be verified;

[0089] Step S502: Verify one by one whether each column offset in the set of column offsets to be verified meets the preset validity conditions. The validity conditions are that the column offset is greater than or equal to the first offset and less than or equal to the second offset.

[0090] Step S503: If any column offset does not meet the validity condition, it is determined that the target tuple has suffered storage structure damage. According to the preset recovery strategy, the complete data of the target tuple is read from the backup medium and the complete data is written sequentially from the starting address of the target tuple to overwrite the damaged data.

[0091] Based on the above steps, by extracting all column offsets and verifying the dual conditions of "greater than or equal to the first offset and less than or equal to the second offset" one by one, misjudgments caused by single column offset anomalies are avoided. This accurately locates tuple storage structure corruption issues, distinguishes between "single column offset anomalies" and "overall storage structure corruption," and improves the accuracy of anomaly detection. After determining that the tuple storage structure is corrupted, a preset recovery strategy is used to read complete data from the backup medium and overwrite the damaged parts. This quickly repairs the corrupted tuples, preventing tuple data loss or unavailability due to storage structure corruption and ensuring the integrity and consistency of database data.

[0092] Figure 6 This is a schematic flowchart illustrating the table structure modification operation in a database tuple processing method according to an embodiment of the present invention, such as... Figure 6 As shown, the processing method for this database tuple also includes:

[0093] Step S601: In response to a table structure change operation for the target data table, traverse each tuple in the target data table. The table structure change operation includes any one of adding a column, deleting a column, and adjusting the column order.

[0094] Step S602: For the current tuple, redetermine the column offset of each column in the column data area according to the changed column order;

[0095] Step S603: Write the newly determined column offset into the column offset table area;

[0096] Step S604: Determine the first offset based on the updated actual length of the column offset table area, and determine the second offset based on the updated first offset and the actual length.

[0097] Step S605: Store the updated first offset and second offset into the tuple header of the current tuple.

[0098] Based on the above steps, this system supports all types of table structure changes, including adding, deleting, and rearranging columns. It eliminates the need to reconstruct the overall tuple storage layout; only column offsets and offset markers in the tuple header are updated to adapt to the structure change, significantly reducing the execution cost and time associated with table structure changes. For table structure changes, the system traverses tuples and recalculates column offsets to ensure that the offset markers of each column's data accurately match their physical storage location after the change. This guarantees the accuracy of column data addressing during tuple reading and avoids data reading errors caused by structure changes. After the change, only the column offset table area and the first and second offsets in the tuple header are updated; the original data in the column data area does not need to be modified, reducing data rewriting and storage overhead, and adapting to structure change scenarios for wide tables and large-volume data tables.

[0099] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0100] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0101] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 7 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 8 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 9 This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. Figure 7 As shown, the computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described database tuple processing methods. For example... Figure 8 As shown, a computer-readable storage medium 20 stores the aforementioned computer program 11, which, when executed by the processor 32, implements the steps of the database tuple processing method of any of the above embodiments. Figure 9 As shown, the computer device 30 may include a memory 31, a processor 32, and a computer program 11 stored on the memory 31 and running on the processor 32.

[0102] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer. In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing status information of the computer-readable program instructions.

[0103] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0104] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0105] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0106] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0107] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0108] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for processing database tuples, characterized in that, include: The tuple is divided into a tuple header, column data area, and column offset table area that are arranged consecutively in the address space; In response to a tuple write to the target data table, starting from the position offset from the tuple start address by the length of the tuple header, each column of data is sequentially written into the column data area, and the column offset of each column of data relative to the tuple start address is recorded. After all column data has been written, the column offsets are organized into a column offset table and written into the column offset table area. The length value of the tuple header is determined as the first offset, and the sum of the tuple header length and the column data area length is determined as the second offset. The first offset and the second offset are stored in the tuple header.

2. The database tuple processing method according to claim 1, characterized in that, The steps of sequentially writing each column of data into the column data area include: Maintain a write pointer, which is initialized to point to the beginning of the column data area before performing a tuple write operation; If the column to be written is a fixed-length data column and is not empty, the data of the fixed-length data column is written to the memory address pointed to by the write pointer according to the preset byte length corresponding to the fixed-length data column; If the column to be written is a variable-length data column and is not null, the valid data of the variable-length data column is written to the memory address pointed to by the write pointer after removing the prefix field, based on the valid data length indicated by the prefix field of the variable-length data column.

3. The database tuple processing method according to claim 2, characterized in that, The step of sequentially writing each column of data into the column data area further includes: If the column to be written is a fixed-length data column and is empty, no data is written to the column data area, and the write pointer remains unchanged; or, according to the preset byte length corresponding to the fixed-length data column, preset empty value fill data is written to the memory address pointed to by the write pointer, and the write pointer is moved backward by the preset byte length. If the column to be written is a variable-length data column and is empty, the write operation of the variable-length data column is not performed, and the write pointer remains unchanged.

4. The database tuple processing method according to claim 1, characterized in that, The step of recording the column offset of each column of data relative to the starting address of the tuple includes: A column offset table is configured at the head of the tuple. The number of offset fields contained in the column offset table is the same as the number of columns contained in the tuple to be written, and each offset field corresponds to one column. When writing the first column of data into the column data area, the column offset of the first column of data is determined based on the difference between the initial value of the write pointer and the starting address of the tuple to be written, and written into the offset field corresponding to the first column in the column offset table; Before writing each subsequent column of data into the column data area, the column offset of the current column to be written is determined based on the difference between the current value of the write pointer and the starting address of the tuple to be written, and written into the offset field corresponding to the current column to be written in the column offset table.

5. The method for processing database tuples according to claim 1, characterized in that, Also includes: In response to a read request for a target tuple in a target data table, the first offset and the second offset are obtained from the tuple header of the target tuple, the read request including the target column number; The column offset table area is located according to the second offset, and the column offset corresponding to the target column in the target tuple is read from the column offset table area according to the target column number; Verify that the column offset is greater than or equal to the first offset; if the verification is successful, locate and extract the data of the target column from the starting address of the target tuple based on the column offset; otherwise, return a data corruption error.

6. The database tuple processing method according to claim 5, characterized in that, After returning a data corruption error, it also includes: Obtain all column offsets recorded in the column offset table area of ​​the target tuple to form a set of column offsets to be verified. Verify one by one whether each column offset in the set of column offsets to be verified meets the preset validity condition, wherein the validity condition is that the column offset is greater than or equal to the first offset and less than or equal to the second offset; If any column offset does not meet the validity condition, the target tuple is determined to have suffered storage structure damage. The complete data of the target tuple is read from the backup medium according to the preset recovery strategy, and the complete data is written sequentially from the starting address of the target tuple to overwrite the damaged data.

7. The method for processing database tuples according to claim 1, characterized in that, Also includes: In response to a table structure change operation targeting a target data table, each tuple in the target data table is traversed, where the table structure change operation includes any one of adding a column, deleting a column, or adjusting the column order. For the current tuple, the column offset of each column in the column data area is re-determined according to the changed column order; Write the redefined column offset into the column offset table area; The first offset is re-determined based on the updated actual length of the column offset table area, and the second offset is re-determined based on the updated first offset and the actual length. The updated first and second offsets are stored in the tuple header of the current tuple.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the database tuple processing method according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method for processing database tuples as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for processing database tuples according to any one of claims 1 to 7.

Images