A database serialization and version compatibility method and system based on binary stream version

Abstract

The application discloses a database serialization and version compatibility method and system based on a binary stream version, which comprises an encoding process and a decoding process: in the encoding process, a plurality of Boolean or enumeration attributes in an object are combined into a single integer flag bit through a bit field mask; variable length coding is adopted for integer types, and signed integers are coded after Zigzag mapping; data is organized into a key-value pair sequence, and a key is generated through variable length coding after a field number and a cable type are logically ORed; a composite type field is coded by using a length prefix or a start / end marker nested message; and object references are replaced by integer indexes; in the decoding process, a field number and a cable type are parsed from a tag value read from a byte stream, corresponding values are read according to the cable type, and assignment is distributed through a lookup table; a flag bit field is unpacked to restore a plurality of attributes through a bit field mask; and integer indexes are replaced by actual object references. The application has high compression rate, high parsing efficiency and robust cross-version compatibility.

Description

Technical Field

[0001] This invention relates to the field of electronic design automation, specifically to a database serialization and version compatibility method and system based on binary stream versions. Background Technology

[0002] In the field of Electronic Design Automation (EDA), the design database is a fundamental component of the integrated circuit design flow. Across different design flows, it is typically necessary to frequently store and exchange information such as ultra-large-scale netlists, physical information, layout information, timing constraints, and place-and-route data. With advancements in process nodes and the increasing size of netlists, a single chip design may contain hundreds of millions of nets, cells, pins, and other objects, each carrying dozens of Boolean flags, enumerated states, and complex object reference relationships. The choice of serialization method directly determines data storage costs, tool loading speed, and the reliability of cross-version collaboration, becoming a key bottleneck in EDA infrastructure.

[0003] Currently, various serialization schemes exist in industry and academia, and their design concepts and technical characteristics can be summarized into the following categories:

[0004] 1. Text Format Serialization: Text formats such as XML, JSON, and YAML are widely used for configuration and lightweight data exchange due to their strong self-descriptiveness and readability. In the EDA field, netlist formats are also divided into Verilog and SPICE formats, which essentially fall under the category of text serialization. However, text formats require storing redundant structural tags and attribute names, holding a large number of repetitive text objects. This results in data volumes that are typically 3-10 times larger than binary formats. Parsing requires extensive string matching and lexical analysis; parsing millions of objects can take minutes, making it unsuitable for the high-throughput requirements of large-scale database designs.

[0005] 2. Self-describing binary serialization: Google's Protocol Buffers (protobuf) and Apache Thrift are representative of cross-language self-describing binary serialization. They describe data structures using predefined schemas, encoding fields as key-value pairs, where the key contains the field number and cable type. Values ​​use compact formats such as variable-length integers (varint), fixed-length encoding, and length prefixes, significantly reducing size and improving parsing speed. Protobuf has become the industry standard for microservice communication and data persistence. However, when faced with a large number of attributes in EDA objects, these general frameworks still assign a separate field number to each attribute and encode it as an independent key-value pair. For example, a wire mesh object with 20 attributes will generate 20 key-value pairs, causing unnecessary storage overhead. Furthermore, their version compatibility mainly relies on field obsolescence and optional flags, lacking optimization for "frequently occurring typical values ​​that can be completely excluded from the stream."

[0006] 3. Zero-copy / zero-parse binary serialization formats like FlatBuffers and Cap'n Proto, which prioritize extreme performance, organize data into a structure that can be directly accessed in memory, eliminating decoding overhead. This makes them extremely suitable for fixed-length fields and scenarios like games and real-time systems. However, in EDA databases, there is a large amount of variable-length data (such as string names and nested messages), dynamically sized arrays, and combinations of variable-width flags. Zero-copy formats require indirect references to variable-length parts using offsets and length tables, increasing implementation complexity and pointer repair costs. Furthermore, their support for version evolution is far less flexible than protobuf, making it difficult to dynamically handle skipping unknown fields.

[0007] In summary, existing serialization methods struggle to simultaneously meet the demands of high compression rates, high parsing efficiency, and robust cross-version compatibility when dealing with the large scale, dense flags, rapid version iterations, and complex object references inherent in EDA design databases. Therefore, it is necessary to innovate database serialization and version compatibility technologies to overcome these shortcomings. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a database serialization and version compatibility method and system based on binary stream versions. This invention achieves high compression ratio, high parsing efficiency, and robust cross-version data exchange through bit-field mask merging, variable-length encoding, object reference indexing, and a multi-layer version compatibility mechanism.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a database serialization and version compatibility method based on binary stream version, including an encoding process and a decoding process;

[0010] The encoding process includes:

[0011] Step A: Traverse the database object to be serialized, extract multiple Boolean or enumeration attributes from the object's metadata, and merge the multiple Boolean or enumeration attributes into a single unsigned integer flag bit by bit mapping according to the predefined bit field mask enumeration of the object's type; wherein, Boolean attributes occupy one bit, and enumeration attributes occupy multiple consecutive bits, and the bit field mask enumeration defines a set of non-overlapping bit positions and attribute semantics correspondences for each object type;

[0012] Step B: For the unsigned integer flag bit and other unsigned integer type fields generated in Step A above, use variable length encoding; during encoding, divide the binary representation of the integer to be encoded into a data unit of 7 bits each, starting from the least significant bit. Fill each data unit into the lower 7 bits of a byte, and set the highest bit of the byte as the continuation flag; if the remaining bits of the integer are non-zero, the continuation flag is set to 1, otherwise it is set to 0; output byte by byte in this manner until all valid data units have been processed.

[0013] Step C: For signed integer type fields, first apply Zigzag mapping to transform the signed integer into an unsigned integer. The specific mapping rule is: for a 32-bit signed integer n, its mapping value m = (n << 1) ^ (n >> 31), where >> is an arithmetic right shift; finally, perform the variable-length encoding described in step B on the obtained unsigned integer m.

[0014] Step D: Organize the encoded binary data into a key-value pair sequence; where each key generates a tag value by left-shifting the field number by 3 bits and performing a bitwise OR operation with the cable type identifier, and then writes the tag value using the variable length encoding in Step B; the cable type identifier is used to indicate the encoding format of the subsequent value, and the cable type includes: variable length integer type, 32-bit fixed length type, 64-bit fixed length type, length prefix type, nested message start type, and nested message end type;

[0015] Step E: For the two types of composite type fields, the nested message encoding method is adopted: If the length prefix format is adopted, all subfields of the composite type field are first serialized into a continuous byte stream according to the rules of this method, the length of the byte stream is calculated, the length value is first written in variable length encoding, and then the byte stream is written; if the start / end marker format is adopted, the nested message start tag is first written, then the key-value pair sequence of the subfields is written, and finally the nested message end tag is written.

[0016] Step F: Replace direct pointer references or object identifiers between database objects with integer indices, including: traversing the object graph before encoding, assigning a unique integer index value to each referenced object, and establishing a mapping table from objects to indices; when encoding referenced fields, writing the corresponding index value, encoded in a fixed 32-bit integer format;

[0017] The decoding process includes:

[0018] Step G: Read a variable-length integer from the input binary byte stream as a tag value according to the decoding rules in step B, and parse out the field number and cable type from it; the field number is the tag value shifted right by 3 bits, and the cable type is the lower 3 bits of the tag value;

[0019] Step H: Read the subsequent value according to the cable type: If it is a variable-length integer type, continue decoding according to the variable length to obtain an integer; if it is a 32-bit fixed-length type, read the next 4 bytes; if it is a 64-bit fixed-length type, read the next 8 bytes; if it is a length prefix type, first decode a variable-length integer as the length L, and then read the next L bytes as the value data; if it is a nested message start type, recursively enter the nested message decoding until the corresponding nested message end tag is encountered; if it is a nested message end type, exit the current nesting level.

[0020] Step 1: Based on the parsed field number, dispatch to the corresponding field processing function by looking up a table; assign the decoded integer, byte sequence, or sub-message to the corresponding field of the current object; if an unrecognized field number is parsed, skip the parsing of that field;

[0021] Step J: For integers marked as flag fields, apply the bit field mask enumeration of the corresponding object type, extract each Boolean or enumeration attribute value through bitwise AND and shift operations, and assign them to the corresponding multiple member variables respectively;

[0022] Step K: Use the index mapping table that has been restored in advance during the decoding process to replace the integer index field with the actual object reference; if the complete mapping table has not yet been established during decoding, the index is temporarily stored and referenced and linked uniformly after all objects are instantiated.

[0023] Furthermore, the pre-defined bit-field mask enumeration is defined separately for different object types in the design data of electronic design automation. Among them, the mask enumeration defined for net objects contains at least 10 composite flag bits, each corresponding to a Boolean semantic, including at least one of the following: whether it is a power net, whether it is a ground net, whether it is a clock net, whether it is a constant net, and whether it is a tri-state drive. For example, bit 0 indicates whether it is a power net, bit 1 indicates whether it is a ground net, bit 2 indicates whether it is a clock net, bit 3 indicates whether it is a constant net, bit 4 indicates whether it is a tri-state drive, etc. The mask enumeration defined for port objects contains at least 8 composite flag bits, including port direction attribute combinations (such as input, output, bidirectional occupying 2 bits) and electrical characteristic flags, etc.

[0024] Furthermore, mechanisms for achieving version compatibility include:

[0025] Unknown field adaptive skipping mechanism: During decoding, when the parsed field number does not exist in the current version of the object schema, the boundary of the value is determined and skipped based on the cable type carried by the key: If the cable type is a variable-length integer, bytes are read in a loop until a byte with the highest bit being 0 is encountered, thus skipping the entire variable-length integer; if it is a 32-bit fixed-length, 4 bytes are skipped; if it is a 64-bit fixed-length, 8 bytes are skipped; if it is a length prefix, the length L is decoded first, and then L bytes are skipped; if it is a nested message start type, the sub-message is recursively entered and all internal fields are skipped in pairs until the nested message end type is reached; through this mechanism, fields added to the data in the new version of the tool will not cause parsing errors or aborts in the old version of the tool;

[0026] Expected default value embedding and skipping mechanism: An expected default value is additionally stored in the pattern description of the field; during encoding, if the actual value of the field is equal to the expected default value, the field key is skipped along with it, meaning that no trace of the field appears in the serialized stream; during decoding, for fields that exist in the pattern but are missing in the stream, the expected default value is used to fill them; this mechanism can further reduce data redundancy for frequently occurring specific enumeration values ​​or typical size values.

[0027] Furthermore, the encoding algorithm for the variable-length encoding is as follows: Input an unsigned integer value, and repeatedly execute the following sub-steps:

[0028] Sub-step A: Take the lower 7 bits of value and store them in the lower 7 bits of the current byte;

[0029] Sub-step B: Shift the value logically to the right by 7 bits;

[0030] Sub-step C: If the value is not 0 after shifting, set the highest bit of the byte to 1, output the byte, and continue generating the next byte; if the value is 0 after shifting, set the highest bit of the byte to 0, output the byte, and end the encoding.

[0031] The decoding algorithm for the variable-length encoding is as follows: input a byte sequence, set the result to 0, and the shift value to 0; repeat the following steps for each read byte:

[0032] Sub-step A: Take the lower 7 bits of the byte, shift it left by shift bits, and then perform a bitwise OR operation with the result to update the result;

[0033] Sub-step B: If the highest bit of the byte is 1, increment shift by 7 and continue reading the next byte; if the highest bit is 0, end the decoding and return result.

[0034] This encoding method is particularly efficient for integers with small numerical ranges, occupying only one byte when the value is less than 128.

[0035] Furthermore, the encoding format of the key-value pair is varint_encode((field_number << 3) | wire_type), where field_number is the field number, such as 1 for the net name field and 2 for the net integer flag. wire_type is the cable type, which includes variable-length integer varint, 32-bit fixed-length data type, 64-bit fixed-length data type, length prefix data type, nested message start encoding, and nested message end encoding.

[0036] Furthermore, for string type field encoding, a length prefix encoding format is adopted, specifically: first, the string is converted into a byte sequence, and then the byte length len of the byte sequence is calculated; then, len is written to the output stream using the variable length encoding; subsequently, the entire byte sequence is written to the output stream as is; during decoding, the length len is first read according to the variable length decoding rules, and then len bytes are accurately read from the stream to reconstruct the string object according to the predetermined character encoding.

[0037] Furthermore, the step of replacing direct pointer references or object identifiers between database objects with integer indices specifically includes:

[0038] During the serialization preparation phase, each object to be persisted is traversed, and each object is assigned an integer value that starts from 1 and increments continuously or starts from 0 as its index, and a two-way mapping table from object pointers to indices is established.

[0039] During the encoding phase, when writing to a reference type field, the index value of the target object is encoded as a fixed 32-bit integer.

[0040] During the decoding phase, an index mapping is established when an object is read. In subsequent readings of other types of objects, if the object is referenced, the object pointer is directly extracted from the index table to establish the index.

[0041] Furthermore, the decoding process employs a two-pass reading strategy, including the following steps:

[0042] First pass: Read the module interface section in the design file, create or find the corresponding design unit for each module interface message, read its port declaration information, and complete the skeleton construction of all modules;

[0043] The second pass involves reading the library files to obtain the port definitions of all leaf units, then reading the module body section of the design file to read instances, nets, assignment statements, and port connection relationships, and filling them into the port skeleton established in the first pass to complete the complete reconstruction of the database.

[0044] This invention also provides a database serialization and version compatibility system based on binary stream versions, comprising:

[0045] The serialization writer is used to perform the encoding process as described in claim 1, encoding the database object in memory into a binary stream;

[0046] The serialization input is used to perform the decoding process as described in claim 1, decoding the binary stream and reconstructing it into a memory data object.

[0047] Furthermore, the serialization writing end includes:

[0048] The cable definition layer is used to encode field numbers and cable types into tag values;

[0049] The abstract write layer provides an abstract interface for writing fields, encapsulating variable-length encoding, Zigzag mapping encoding, fixed-length encoding, and length prefix encoding.

[0050] The nested message RAII wrapper layer is used to encode nested messages for composite type fields using a start / end tag format. It saves the field key and parent stream reference during construction, automatically writes the nested message start tag when the child field is first written, and automatically writes the nested message end tag during destruction.

[0051] The serialization read-in terminal includes:

[0052] The abstract reading layer provides an abstract interface for field decoding. It reads tag values ​​from the input byte stream and parses out the field number and cable type. It then reads subsequent values ​​based on the cable type and performs table lookup, distribution, and assignment based on the field number.

[0053] The RAII nested message reading layer is used to read nested messages. During construction, it enters the scope of the nested message and intercepts the nested message end marker to determine the boundary. During destruction, it automatically skips the remaining bytes in the current nested message that have not yet been read.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] In terms of runtime performance, this invention significantly reduces the time required for serialization and deserialization through the synergistic effect of multiple technical means.

[0056] First, by merging bit-field masks, multiple scattered Boolean or enumerated attributes on a single object are aggregated into a single flag integer for encoding. This method combines the multiple operations required in traditional schemes to generate key-value pairs for each attribute into a single operation requiring only one key-value pair, thus significantly reducing the frequency of label encoding, field iteration, and function calls, effectively reducing processor branch and instruction overhead. On the decoding side, correspondingly, all attributes can be recovered with only one bitmask extraction, eliminating multiple label parsing and field distribution processes, greatly mitigating pipeline stalls caused by branch prediction failures.

[0057] Secondly, by embedding and skipping expected default values, a large number of optional fields that retain default values ​​are completely omitted in the encoding stream, thereby avoiding a massive amount of redundant encoding and decoding operations in large-scale object processing scenarios.

[0058] Furthermore, by using integer indices instead of strings or names for object references, array subscripts can be accessed directly using the index value when resolving references. This transforms the high-overhead operation of hash-based or name-based lookup in traditional solutions into a low-overhead operation with constant time, significantly improving the speed of reference resolution.

[0059] In summary, the above measures effectively reduce the encoding and decoding time of this invention by orders of magnitude under the same EDA database load.

[0060] This invention also achieves significant optimizations in terms of memory and storage usage.

[0061] In terms of serialized data volume, by merging bit-field masks, multiple flag bits that originally required multiple sets of key-value pairs are compressed into a very small number of bytes of flag bit encoding. Combined with the elimination of redundant data by the expected default value skipping mechanism, and the compact representation of common small integers by variable-length encoding and Zigzag mapping, the overall volume of the serialized data stream is compressed from multiple dimensions.

[0062] In terms of runtime memory usage, flag merging allows each object to store all its boolean states with only a small amount of contiguous memory, resulting in a compact memory layout that is highly friendly to the CPU cache. The unknown field skipping mechanism operates directly on the input stream without allocating temporary runtime objects, effectively suppressing memory fragmentation. The object reference index map uses a contiguous array to store pointers, which substantially reduces the memory overhead of each reference compared to a map based on string names, thus avoiding memory bloat in large-scale object scenarios.

[0063] Therefore, the present invention can effectively reduce the storage space and transmission bandwidth requirements of serialized data, while significantly reducing the peak runtime memory when processing ultra-large-scale designs.

[0064] Through the combined advantages in performance and storage, this invention provides a more efficient and lightweight technical solution for the persistence and cross-version collaboration of ultra-large-scale design databases in the field of electronic design automation. Attached Figure Description

[0065] Figure 1 This is a flowchart of the encoding process in an embodiment of the present invention, used to show the actual calling sequence of the system encoding process.

[0066] Figure 2 This is a flowchart of the decoding process in an embodiment of the present invention, used to show the actual calling sequence of the system decoding process. Detailed Implementation

[0067] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0068] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0069] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0070] This embodiment provides a database serialization and version compatibility method and system based on binary stream versions, which is implemented through a layered architecture and includes a serialization writing end and a serialization reading end.

[0071] The serialization writer employs a layered writer hierarchy to encode EDA database objects in memory into binary streams. The serialization writer includes a cable definition layer, an abstract writer layer, and a nested message RAII wrapper layer. These layers work together to implement the encoding process, such as... Figure 1 As shown.

[0072] (1) Cable Definition Layer and Key Generation: The cable definition layer consists of multiple static field key instances, each corresponding to a field to be serialized. This layer is responsible for encoding the field number (field_number) and cable type (wire_type) into tag values. Specifically, the field number is shifted left by 3 bits and then bitwise ORed with the cable type identifier, followed by variable-length encoding, i.e., varint_encode((field_number << 3) | wire_type). Among them, the cable type includes variable-length integer (varint), 32-bit fixed-length, 64-bit fixed-length, length prefix, nested message start, and nested message end types. This layer provides a unified key generation capability for the upper layer, enabling the abstract writing layer to generate and output tags by obtaining the corresponding field key instance when writing any field.

[0073] (2) Abstract Write Layer: Provides an abstract entry point for field writing, encapsulating the various encoding algorithms defined in the claims into callable methods, mainly including:

[0074] Writing Variable-Length Integers: When this method is called to write an unsigned integer `value`, an internal loop executes: the lower 7 bits of `value` are stored in the lower 7 bits of the current byte, and `value` is logically shifted right by 7 bits; if `value` is non-zero after the shift, the highest bit of the current byte is set to 1 and output, and the process continues to the next byte; if it is zero, the highest bit is set to 0 and output, and the encoding ends. This method uses only one byte for integers less than 128, significantly improving the encoding efficiency for small values.

[0075] Writing signed integers: When writing a 32-bit signed integer n, first apply a Zigzag mapping to transform it into an unsigned integer m, with the mapping rule m = (n << 1) ^ (n >> 31), where the right shift is an arithmetic right shift. Then, use m as a parameter to call the above method for writing variable-length integers to complete the compact encoding of the signed integer.

[0076] Write fixed-length integer: Supports writing 32-bit or 64-bit integers directly to a fixed 4 bytes or 8 bytes in little-endian or big-endian order, corresponding to the 32-bit fixed-length and 64-bit fixed-length types in cable types.

[0077] Write a length-prefixed string: First, convert the string to a byte sequence according to a predefined character encoding and calculate the byte length len; then call the write variable-length integer method to write len to the output stream; finally, write the entire byte sequence as is.

[0078] Write message field key: Call the field key instance provided by the cable definition layer and write its pre-encoded tag value directly to the output stream.

[0079] Write floating-point numbers and other types: Provides support for writing fixed-length types such as double-precision floating-point numbers, and can write 8 bytes in IEEE754 format.

[0080] (3) Nested Message RAII Wrapper Layer: For composite type fields, this embodiment uses a start / end tag format for nested message encoding. This layer employs a Resource Acquisition and Initialization (RAII) mechanism, saving only the field key and parent stream reference during construction, without immediately writing any data. When a subfield is written through this wrapper layer for the first time, a nested message start tag (i.e., the tag corresponding to the field key, with cable type "nested message start") is automatically written to the parent stream, allowing sequential writing of the key-value pair sequence of the subfield. During destruction, this layer automatically writes a nested message end tag (with cable type "nested message end") to the parent stream, thus ensuring strict pairing of start and end tags.

[0081] (4) Object Reference Indexing: During the serialization preparation phase, the encoder traverses the object graph, assigns a unique integer index value (incrementing continuously from 1 or 0) to each referenced object, and establishes a bidirectional mapping table from object pointers to indices. When processing reference type fields in the abstract write layer, the index value of the target object is directly queried from the mapping table, and the index is encoded and output by calling the write fixed-length integer method (in fixed 32-bit integer form). This indexing process not only compresses the data volume but also provides a foundation for the decoder to quickly recover the reference.

[0082] Through the collaboration of these layers, the writing end can efficiently encode database objects into binary streams. During actual writing, it can be divided into design files and library files as needed: design files contain port information and module body content for each module, while library files contain port definitions and other information for library units.

[0083] The serialization read end employs a hierarchical reader structure symmetrical to the write end, used to decode the binary stream and reconstruct it into in-memory data objects. The serialization read end mainly includes an abstract read layer and a nested message RAII read layer; each layer works together to implement the decoding process, such as... Figure 2 As shown.

[0084] (1) Abstract Reading Layer: Provides an abstract interface for field decoding. The main methods include: Reading Field Key: Calls the read variable-length integer method from the input byte stream to read an unsigned integer as the tag value, and then parses out the field number and cable type: the field number is the tag value shifted right by 3 bits, and the cable type is the lower 3 bits of the tag value. According to the parsed cable type, the subsequent value is read in the corresponding way: if it is a variable-length integer type, the read variable-length integer method is called to get an integer; if it is a 32-bit fixed-length type, the next 4 bytes are read; if it is a 64-bit fixed-length type, the next 8 bytes are read; if it is a length prefix type, a variable-length integer is first decoded as the length L, and then the next L bytes are read as the value data; if it is a nested message start type, the nested message decoding is recursively entered; if it is a nested message end type, the current nested level is exited.

[0085] Reading Variable-Length Integers: Implement the decoding algorithm described in claim 4. Input a byte sequence, set the result (result) to 0, and the shift (shift) to 0. For each read byte, take the lower 7 bits, shift them left by shift bits, and then perform a bitwise OR operation with result. If the highest bit of the byte is 1, increment shift by 7 and continue reading; if it is 0, end the decoding and return result. This method can correctly decode the variable-length integer generated at the writing end.

[0086] Reading signed integers: First, obtain an unsigned integer m by reading a variable-length integer, then perform a Zigzag inverse mapping n = (m >>> 1) ^ -(m & 1) to restore it to a signed integer.

[0087] Reading a string: First, obtain the length len using the read variable-length integer method, then accurately read len bytes from the stream, and reconstruct the string object according to the predefined character encoding.

[0088] Skip Current Field: When the parsed field number does not exist in the current version of the object pattern, the corresponding bytes are automatically skipped according to the cable type, realizing a version-compatible adaptive skipping mechanism for unknown fields. Specifically: if it is a variable-length integer, read bytes in a loop until the highest bit is 0; if it is a 32-bit fixed-length integer, skip 4 bytes; if it is a 64-bit fixed-length integer, skip 8 bytes; if it is a length prefix, decode the length L first and then skip L bytes; if it is the beginning of a nested message, recursively enter the sub-message and skip all internal fields in pairs until the end type.

[0089] Field dispatch and assignment: Based on the parsed field number, the decoded integer, byte sequence or sub-message is dispatched to the field processing function corresponding to the current object by looking up a table to complete the assignment.

[0090] (2) Nested Message RAII Read Layer: This layer is used to read nested messages and corresponds to the RAII wrapper layer on the writing side. During construction, it enters the scope of a nested message and intercepts the nested message end marker to determine the boundary. When this layer object is destructed, it automatically skips any remaining bytes not yet read in the current nested message, thus ensuring that the read position is accurately aligned to the next field of the outer message. This mechanism guarantees that even if some fields within the nested message are ignored due to version differences, the overall parsing process will not be disrupted.

[0091] (3) Unpacking of flag fields: For unsigned integers marked as flag fields, after reading, the bit field mask enumeration definition of the corresponding object type is called. Through bitwise AND and shift operations, the values ​​of each Boolean or enumeration attribute are extracted and assigned to multiple member variables of the current object. For example, for a net object, all flags such as power net, ground net, and clock net can be recovered from a single integer at once based on its mask enumeration, without the need to parse each flag separately.

[0092] (4) Object reference recovery: During the decoding process, each time a new object is read, a mapping from the index to the object pointer is established based on its allocated integer index. When a reference type field is parsed, after reading the index value, if the target object has been instantiated, the pointer is directly extracted from the index mapping table to complete the link; if it has not yet been instantiated, the index is temporarily stored, and the reference link is repaired uniformly after all objects are instantiated.

[0093] When reading EDA design files, this invention preferably employs a two-pass read strategy to efficiently reconstruct the complete database. Its specific implementation is as follows:

[0094] The first pass reads the module interface section of the design file. At this point, the higher-level messages in the input stream contain a sequence of key-value pairs for the module interface, such as the module name and port declaration list. The abstract reading layer parses each field sequentially, entering the module scope when it encounters a nesting start marker in the module message. For each module, it creates or finds the corresponding design unit object using its name and other identifiers, and reads its port declaration information: including attribute fields such as port name, direction, and bit width, which exist as their respective key-value pairs. After reading all port fields, the nested message RAII reading layer automatically skips unrecognized fields when exiting the module scope. After this pass, the skeleton of all design units (i.e., the module and its port list) is complete, but the specific implementations within the modules (such as instances, nets, and connections) are not yet filled in.

[0095] The second pass: First, the library file is read to obtain the port definitions of all leaf units. The message format of each library unit in the library file is similar to that of the module, containing a name and port field. After reading, a library unit index is created. Then, the module body of the design file is read again. At this time, the instance field, net field, assign statement field, and port connection relationship field inside the module message are parsed. These fields are each encoded with predefined field numbers and cable types. For example, the instance field may contain the instance name, the referenced library unit index, the port connection list, etc.; the net field contains the net name, flag bits (i.e., the integer after merging bit field masks), etc. When the abstract reading layer reads the instance field, it parses the referenced library unit index and obtains the port definition through the established library unit mapping table; when parsing the port connection relationship, it associates the instance port with the net name. After decoding, the net object is unpacked according to its flag bits to obtain various Boolean attributes. Since the module skeleton has been established in the first pass, these instances and net objects can now be directly filled into the internal data structure of the corresponding module to complete the complete reconstruction of the module body.

[0096] By using these two strategies, the module interface and implementation can be separated first, and the design hierarchy can be established in advance. This allows subsequent instantiation and connection filling to quickly locate the target module, and also facilitates handling scenarios of incremental loading and partial updates.

[0097] The system possesses forward and backward compatibility, primarily achieved through the following two mechanisms:

[0098] (1) Unknown Field Adaptive Skip Mechanism: When the new version of the tool adds a field to the data (such as adding a new tag field to a wire mesh object), and the old version of the reader parses the field, its field number cannot match any known field key in the current version's schema description. At this time, the field dispatch logic of the abstract reading layer will call the skip current field method. This method determines the boundary of the value based on the cable type carried in the tag and skips it, as described above. This mechanism ensures that the old version of the tool will not parse incorrectly or abort when encountering an unknown field, thus safely ignoring the added content.

[0099] (2) Expected Default Value Embedding and Skipping Mechanism: An expected default value is additionally stored in the pattern description of a field. During the encoding phase, before writing a field, the writer checks whether the actual value of the field is equal to the expected default value. If they are equal, the field's key and value are skipped, meaning the field does not appear in the serialization stream at all. During the decoding phase, after reading all fields of an object, for fields that exist in the pattern but are missing in the stream, their corresponding expected default values ​​will be used to fill them. This mechanism can further compress data redundancy for frequently occurring specific enumeration values ​​or typical size values, while maintaining the simplicity of the architecture.

[0100] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0101] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A database serialization and version compatibility method based on binary stream versions, characterized in that, Includes the encoding and decoding processes; The encoding process includes: Step A: Traverse the database object to be serialized, extract multiple Boolean or enumeration attributes from the object's metadata, and merge the multiple Boolean or enumeration attributes into a single unsigned integer flag bit by bit mapping according to the predefined bit field mask enumeration of the object's type; wherein, Boolean attributes occupy one bit, and enumeration attributes occupy multiple consecutive bits, and the bit field mask enumeration defines a set of non-overlapping bit positions and attribute semantics correspondences for each object type; Step B: For the unsigned integer flag bit and other unsigned integer type fields generated in Step A above, use variable length encoding; during encoding, divide the binary representation of the integer to be encoded into a data unit of 7 bits each, starting from the least significant bit. Fill each data unit into the lower 7 bits of a byte, and set the highest bit of the byte as the continuation flag; if the remaining bits of the integer are non-zero, the continuation flag is set to 1, otherwise it is set to 0; output byte by byte in this manner until all valid data units have been processed. Step C: For signed integer type fields, first apply Zigzag mapping to transform the signed integer into an unsigned integer. The specific mapping rule is: for a 32-bit signed integer n, its mapping value m = (n << 1) ^ (n >> 31), where >> is an arithmetic right shift; finally, perform the variable-length encoding described in step B on the obtained unsigned integer m. Step D: Organize the encoded binary data into a key-value pair sequence; where each key generates a tag value by left-shifting the field number by 3 bits and performing a bitwise OR operation with the cable type identifier, and then writes the tag value using the variable length encoding in Step B; the cable type identifier is used to indicate the encoding format of the subsequent value, and the cable type includes: variable length integer type, 32-bit fixed length type, 64-bit fixed length type, length prefix type, nested message start type, and nested message end type; Step E: For the two types of composite type fields, the nested message encoding method is adopted: If the length prefix format is adopted, all subfields of the composite type field are first serialized into a continuous byte stream according to the rules of this method, the length of the byte stream is calculated, the length value is first written in variable length encoding, and then the byte stream is written; if the start / end marker format is adopted, the nested message start tag is first written, then the key-value pair sequence of the subfields is written, and finally the nested message end tag is written. Step F: Replace direct pointer references or object identifiers between database objects with integer indices, including: traversing the object graph before encoding, assigning a unique integer index value to each referenced object, and establishing a mapping table from objects to indexes; when encoding referenced fields, writing the corresponding index value. The decoding process includes: Step G: Read a variable-length integer from the input binary byte stream as a tag value according to the decoding rules in step B, and parse out the field number and cable type from it; the field number is the tag value shifted right by 3 bits, and the cable type is the lower 3 bits of the tag value; Step H: Read the subsequent value according to the cable type: If it is a variable-length integer type, continue decoding according to the variable length to obtain an integer; if it is a 32-bit fixed-length type, read the next 4 bytes; if it is a 64-bit fixed-length type, read the next 8 bytes; if it is a length prefix type, first decode a variable-length integer as the length L, and then read the next L bytes as the value data; if it is a nested message start type, recursively enter the nested message decoding until the corresponding nested message end tag is encountered; if it is a nested message end type, exit the current nesting level. Step 1: Based on the parsed field number, dispatch to the corresponding field processing function by looking up a table; assign the decoded integer, byte sequence, or sub-message to the corresponding field of the current object; if an unrecognized field number is parsed, skip the parsing of that field; Step J: For integers marked as flag fields, apply the bit field mask enumeration of the corresponding object type, extract each Boolean or enumeration attribute value through bitwise AND and shift operations, and assign them to the corresponding multiple member variables respectively; Step K: Use the index mapping table that has been restored in advance during the decoding process to replace the integer index field with the actual object reference; if the complete mapping table has not yet been established during decoding, the index is temporarily stored and referenced and linked uniformly after all objects are instantiated.

2. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, The pre-defined bit field mask enumeration is defined separately for different object types in the design data of electronic design automation. Among them, the mask enumeration defined for net objects contains at least 10 composite flag bits, each corresponding to a Boolean semantic; the mask enumeration defined for port objects contains at least 8 composite flag bits, including port direction attribute combinations and electrical characteristic flags.

3. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, Mechanisms for achieving version compatibility include: Unknown field adaptive skipping mechanism: During decoding, when the parsed field number does not exist in the current version of the object schema, the boundary of the value is determined and skipped according to the cable type carried by the key: If the cable type is a variable-length integer, the bytes are read in a loop until the byte with the highest bit being 0 is encountered, thus skipping the entire variable-length integer; if it is a 32-bit fixed-length, 4 bytes are skipped; if it is a 64-bit fixed-length, 8 bytes are skipped; if it is a length prefix, the length L is decoded first, and then L bytes are skipped; if it is a nested message start type, the sub-message is recursively entered and all internal fields are skipped in pairs until the nested message end type is reached. Expected default value embedding and skipping mechanism: An expected default value is stored in the pattern description of the field; during encoding, if the actual value of the field is equal to the expected default value, the field key is skipped along with it; during decoding, for fields that exist in the pattern but are missing in the stream, the expected default value is used to fill them.

4. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, The variable-length encoding algorithm is as follows: Input an unsigned integer value, and repeatedly execute the following sub-steps: Sub-step A: Take the lower 7 bits of value and store them in the lower 7 bits of the current byte; Sub-step B: Shift the value logically to the right by 7 bits; Sub-step C: If the value is not 0 after shifting, set the highest bit of the byte to 1, output the byte, and continue generating the next byte; if the value is 0 after shifting, set the highest bit of the byte to 0, output the byte, and end the encoding. The decoding algorithm for the variable-length encoding is as follows: input a byte sequence, set the result to 0, and the shift value to 0; repeat the following steps for each read byte: Sub-step A: Take the lower 7 bits of the byte, shift it left by shift bits, and then perform a bitwise OR operation with the result to update the result; Sub-step B: If the highest bit of the byte is 1, increment shift by 7 and continue reading the next byte; if the highest bit is 0, end the decoding and return result.

5. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, The encoding format for key-value pairs is varint_encode((field_number << 3) | wire_type), where field_number is the field number and wire_type is the cable type. The cable type includes variable-length integer varint, 32-bit fixed-length data type, 64-bit fixed-length data type, length prefix data type, nested message start encoding, and nested message end encoding.

6. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, For string field encoding, a length prefix encoding format is used, specifically: first, the string is converted into a byte sequence, and then the byte length len of the byte sequence is calculated; then, len is written to the output stream using the variable length encoding; subsequently, the entire byte sequence is written to the output stream as is; during decoding, the length len is read first according to the variable length decoding rules, and then len bytes are read from the stream to reconstruct the string object according to the predetermined character encoding.

7. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, The step of replacing direct pointer references or object identifiers between database objects with integer indexes specifically includes: During the serialization preparation phase, each object to be persisted is traversed, and each object is assigned an integer value that starts from 1 and increments continuously or starts from 0 as its index, and a two-way mapping table from object pointers to indices is established. During the encoding phase, when writing to a reference type field, the index value of the target object is encoded as a fixed 32-bit integer. During the decoding phase, an index mapping is established when an object is read. In subsequent readings of other types of objects, if the object is referenced, the object pointer is directly extracted from the index table to establish the index.

8. The database serialization and version compatibility method based on binary stream versioning according to claim 1, characterized in that, The decoding process employs a two-pass read strategy and includes the following steps: First pass: Read the module interface section in the design file, create or find the corresponding design unit for each module interface message, read its port declaration information, and complete the skeleton construction of all modules; The second pass involves reading the library files to obtain the port definitions of all leaf units, then reading the module body section of the design file to read instances, nets, assignment statements, and port connection relationships, and filling them into the port skeleton established in the first pass to complete the complete reconstruction of the database.

9. A database serialization and version compatibility system based on binary stream versions, characterized in that, include: The serialization writer is used to perform the encoding process as described in claim 1, encoding the database object in memory into a binary stream; The serialization input is used to perform the decoding process as described in claim 1, decoding the binary stream and reconstructing it into a memory data object.

10. The database serialization and version compatibility system based on binary stream versioning according to claim 9, characterized in that, The serialization writing end includes: The cable definition layer is used to encode field numbers and cable types into tag values; The abstract write layer provides an abstract interface for writing fields, encapsulating variable-length encoding, Zigzag mapping encoding, fixed-length encoding, and length prefix encoding. The nested message RAII wrapper layer is used to encode nested messages for composite type fields using a start / end tag format. It saves the field key and parent stream reference during construction, automatically writes the nested message start tag when the child field is first written, and automatically writes the nested message end tag during destruction. The serialization read-in terminal includes: The abstract reading layer provides an abstract interface for field decoding. It reads tag values ​​from the input byte stream and parses out the field number and cable type. It then reads subsequent values ​​based on the cable type and performs table lookup, distribution, and assignment based on the field number. The RAII nested message reading layer is used to read nested messages. During construction, it enters the scope of the nested message and intercepts the nested message end marker to determine the boundary. During destruction, it automatically skips the remaining bytes in the current nested message that have not yet been read.

Images