A data persistence system and method based on dynamic interpreter

Abstract

This invention discloses a data persistence system and method based on a dynamic interpreter in the field of interpreter technology, aiming to solve the problems of poor reusability and scalability of existing interpreter data persistence modules, as well as poor backward compatibility with new versions. It includes a storage layer and an interpretation layer; the storage layer stores raw data; the interpretation layer includes an interpreter library, an interpreter matching module, a dynamic adaptation module, and a data parsing module; the interpreter library stores pre-registered interpreters and new interpreter templates generated by the dynamic adaptation module; this invention improves the reusability and scalability of the data persistence module through a layered architecture design of data storage, storage layer, and interpretation layer; and achieves data updates by generating new interpreter templates through the dynamic adaptation module, improving the compatibility of data persistence between new and old versions of the system, providing a highly compatible solution for data parsing under the iterative development of industrial automation equipment.

Description

Technical Field

[0001] This invention relates to a data persistence system and method based on a dynamic interpreter, belonging to the field of interpreter technology. Background Technology

[0002] Against the backdrop of the national drive for industrial transformation and upgrading, industrial automation equipment is rapidly entering a period of replacement. The persistence of data from industrial automation equipment plays a crucial role in its operation. As an important component of new-quality productivity, it places high demands on the persistent storage and use of equipment data.

[0003] In traditional data persistence systems, the storage of different data types is bound to a single interpreter, resulting in a high degree of binding between data storage and parsing. This leads to poor reusability and scalability of the data persistence module. Data parsing relies on a fixed format, and adding or deleting data fields results in poor backward compatibility with new versions.

[0004] In summary, the existing dynamic interpreter's data persistence module has poor reusability and scalability, and the new version has poor backward compatibility, which affects the actual use of the interpreter. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a data persistence system and method based on a dynamic interpreter. This system utilizes an interpreter library to store the interpreter, and eliminates the high degree of binding between data storage and parsing through a storage layer and a data parsing module. A layered architecture design of data storage, storage layer, and interpretation layer improves the reusability and scalability of the data persistence module. Furthermore, a dynamic adaptation module generates new interpreter templates to update data, enhancing the compatibility of data persistence between new and old versions of the system. This provides a highly compatible solution for data parsing under the iterative development of industrial automation equipment.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a data persistence system based on a dynamic interpreter, including a storage layer and an interpretation layer;

[0008] The storage layer is used to store the raw data;

[0009] The interpretation layer includes an interpreter library, an interpreter matching module, a dynamic adaptation module, and a data parsing module;

[0010] The interpreter library is used to store pre-registered interpreters and new interpreter templates generated by the dynamic adaptation module;

[0011] The interpreter matching module is used to match the original data with the corresponding interpreter according to the version mapping table and the interpreter library. If the match is successful, the interpreter type of the corresponding interpreter is sent to the data parsing module.

[0012] The dynamic adaptation module is used to generate a new interpreter template based on the original data when the interpreter matching module fails to match, store the new interpreter template in the interpreter library, and send the new interpreter template type corresponding to the new interpreter template to the data parsing module.

[0013] The data parsing module is used to call the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and to perform field rule validation and data parsing on the original data according to the final interpreter to obtain the parsing result of the original data.

[0014] Furthermore, the original data includes multiple data blocks, each data block is used to store one type of data, the header of the data contains a general feature identifier, the data contains multiple fields, and the header of each field contains a field feature identifier.

[0015] Furthermore, the overall feature identifier includes a version number and a first data type; the field feature identifier includes a field number, a second data type, and a field length;

[0016] Both the first data type and the second data type include 8-bit unsigned integer, 32-bit integer, floating-point number, string, and boolean.

[0017] Furthermore, the data parsing module includes a field rule validation unit and a data parsing unit;

[0018] The field rule verification unit is used to perform field rule verification;

[0019] The data parsing unit is used to perform data parsing.

[0020] Secondly, the present invention provides a data persistence method based on a dynamic interpreter, and a data persistence system based on a dynamic interpreter as described in the first aspect, comprising:

[0021] Step a: The interpreter matching module retrieves data from the storage layer and parses the data to obtain the version number and the first data type, then proceeds to step b;

[0022] Step b: The interpreter matching module queries the version mapping table based on the version number and the first data type, matches the data with the corresponding interpreter, and obtains the matching result;

[0023] If the first data type does not exist in the version mapping table, the matching result is a failure and the work ends; if both the first data type and the version number exist in the version mapping table, the matching result is a success, the interpreter type of the corresponding interpreter is sent to the data parsing module, and step d is executed; if the first data type exists in the version mapping table, but the version number does not exist in the version mapping table, the matching result is a failure, and step c is executed.

[0024] The version mapping table is obtained from the interpreter library;

[0025] Step c: The dynamic adaptation module parses the field feature identifiers of each field in the data;

[0026] The dynamic adaptation module obtains the current version, determines the reference interpreter in real time based on the current version, and obtains the field feature identifier of the reference interpreter;

[0027] The dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, stores the new interpreter template in the interpreter library, and sends the new interpreter template type of the new interpreter template to the data parsing module, and executes step d;

[0028] Step d: The data parsing module calls the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and performs field rule validation and data parsing on the original data according to the final interpreter to obtain the data parsing result.

[0029] Furthermore, the execution of field rule validation and data parsing specifically includes:

[0030] The field rule validation includes data type validation and structural integrity checks.

[0031] If both data type validation and structural integrity check pass, the data parsing unit completes data parsing based on the final interpreter.

[0032] Otherwise, discard the data and terminate its parsing; the task is complete.

[0033] Furthermore, the field feature identifiers of each field in the parsed data specifically include:

[0034] First, parse the first N bytes of the data to obtain the field feature identifier of the first field, and infer the starting position of the next field based on the first field feature identifier. Then, parse the field feature identifier of the second field, and so on, until the parsing of the field feature identifiers of each field is completed.

[0035] Where N is the preset number of bytes.

[0036] Furthermore, the dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, specifically including:

[0037] Based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, each field is classified to obtain the field type;

[0038] Generate a new interpreter template based on the field type;

[0039] The field feature identifier includes the field number, the second data type, and the field length.

[0040] Furthermore, the specific types of fields obtained include identical fields, similar fields, different fields, redundant fields, and missing fields;

[0041] The same field: This field has the same field number, second data type, and field length as the reference interpreter; however, if the field and the second data type of the reference interpreter are both strings, then the field length does not need to be the same.

[0042] The similar field: This field has the same field number as the reference interpreter, and this field conforms to the mapping rules with the second data type of the reference interpreter;

[0043] The different fields: the field has the same field number as the reference interpreter, and the field does not conform to the mapping rules of the second data type of the reference interpreter;

[0044] The redundant fields: the field feature identifier of each field is compared with the field feature identifier of the reference interpreter, and the field number of the field is missing in the reference interpreter;

[0045] The missing field: When the field feature identifier of each field is compared with the field feature identifier of the reference interpreter, the field number of that field is missing in each field.

[0046] Furthermore, the new interpreter template specifically includes:

[0047] When the field types are the same, the new interpreter template parses the data for that field based on the corresponding field in the reference interpreter.

[0048] When the field type is similar, the new interpreter template parses the data based on that field and converts the second data type of that field in the data according to the mapping rules;

[0049] When the field types are different, the new interpreter template abandons data parsing for that field in the data;

[0050] When a field is a redundant field, the new interpreter template abandons data parsing for that field in the data;

[0051] When a field is a missing field, the new interpreter template creates a new field based on the extra field in the reference interpreter and fills it with a default safe value; wherein the default safe value is determined according to the field type of the newly created field.

[0052] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0053] This data persistence system based on a dynamic interpreter implements interpreter storage through an interpreter library. By eliminating the high degree of binding between data storage and parsing through the storage layer and data parsing module, the system improves the reusability and scalability of the data persistence module through a layered architecture design of data storage, storage layer, and interpretation layer. The system generates new interpreter templates through a dynamic adaptation module to realize data updates, improves the compatibility of data persistence between new and old versions of the system, and provides a highly compatible solution for data parsing under the iteration of industrial automation equipment. Attached Figure Description

[0054] Figure 1 This is a system diagram of a data persistence system based on a dynamic interpreter according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the workflow of the interpretation layer provided in an embodiment of the present invention. Detailed Implementation

[0056] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0057] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Example

[0058] like Figures 1-2 As shown, the present invention provides a data persistence system based on a dynamic interpreter, including a storage layer and an interpretation layer;

[0059] The storage layer is used to store the raw data;

[0060] The interpretation layer includes an interpreter library, an interpreter matching module, a dynamic adaptation module, and a data parsing module;

[0061] The interpreter library is used to store pre-registered interpreters and new interpreter templates generated by the dynamic adaptation module;

[0062] The interpreter matching module is used to match the original data with the corresponding interpreter according to the version mapping table and the interpreter library. If the match is successful, the interpreter type of the corresponding interpreter is sent to the data parsing module.

[0063] The dynamic adaptation module is used to generate a new interpreter template based on the original data when the interpreter matching module fails to match, store the new interpreter template in the interpreter library, and send the new interpreter template type corresponding to the new interpreter template to the data parsing module.

[0064] The data parsing module is used to call the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and to perform field rule validation and data parsing on the original data according to the final interpreter to obtain the parsing result of the original data.

[0065] The data is binary data without semantic meaning. The storage layer contains different types of data blocks. Each piece of data in the data block is stored in a binary form without semantic meaning. The header of a binary data block stores a total feature identifier, and each field in the data stores a field feature identifier.

[0066] The interpreter library is used to store predefined interpreters. Each data type corresponds to one interpreter, that is, each interpreter has a unique ID corresponding to a data type. The interpreter defines the parsing rules for each piece of binary data.

[0067] The interpreter matching module parses the total feature identifier in the binary data header and prioritizes matching pre-registered interpreters from the version relationship mapping table. If the match fails, the dynamic adaptation module is called.

[0068] The dynamic adaptation module is used to receive the adaptation request from the interpreter matching module when the pre-registered interpreter matching fails. It extracts the field feature identifiers of each field header of the binary data, compares and analyzes the mapping rules of each field with the reference interpreter, adapts and generates a new interpreter template, adds the new interpreter to the interpreter library and updates the version relationship mapping table.

[0069] Specifically, the storage layer is built using an SQLite database (a lightweight database) and stores binary data in BLOB format (a container for storing binary data). Structured data storage is achieved through a multi-table structure, including a system settings table, a pressure plate information table, a user information table, an operation record table, and a log record table. Each record in each table includes two columns: "Key ID" and "Detailed Information." The Key ID is only used for sorting and retrieval within the table. All valid information is stored in BLOB format in the "Detailed Information" column. The header stores a general feature identifier, and each field header stores a field feature identifier. The general feature identifier includes a version number (version) and a first data type (data_Type). The version number occupies the first and second bytes, and the data type occupies the third byte.

[0070] Data parsing module: From the interpreter library, it uses field rule validation and data parsing to validate and parse each field of the raw binary data, converting the raw data into agreed-upon information.

[0071] In this embodiment, the original data includes multiple data blocks, each data block is used to store a type of data, the header of the data contains a general feature identifier, the data contains multiple fields, and the header of each field contains a field feature identifier.

[0072] The overall feature identifier includes a version number and a first data type; the field feature identifier includes a field number, a second data type, and a field length.

[0073] Both the first data type and the second data type include 8-bit unsigned integer, 32-bit integer, floating-point number, string, and boolean.

[0074] Specifically, a BLOB data entry (i.e., data) contains multiple fields. Each field header contains a field feature identifier area with a fixed length of 4 bytes, specifically including: field number, stored as a uint8_t (eight-bit unsigned integer) type, occupying 1 byte; data type, stored as a uint8_t type, occupying 1 byte; and field length, stored as a uint16_t (sixteen-bit unsigned integer) type, occupying 2 bytes. The field number is a unique identifier for that field within the BLOB data type. During interpreter iterations, field numbers can be expanded or discarded, but old field numbers cannot be reused for new fields. The field feature identifier table is shown in Table 1.

[0075] Table 1. Field Feature Identifier Table

[0076]

[0077] The data types are identified as 1, 2, 3, 4, and 5, which correspond to five data types respectively: 8-bit unsigned integer uint8_t, 32-bit integer int32, floating-point number float, string, and boolean bool.

[0078] In this embodiment, the interpreter matching module performs the following:

[0079] Parse the header of the data to obtain the version number and type.

[0080] The version mapping table matches the interpreter for the corresponding data type based on the version number and the first data type. If the interpreter matches successfully, the selected interpreter is directly notified to the data parsing module. If the match fails, the interpreter for the current version and the corresponding type is obtained as the reference interpreter based on "current version number + first data type", and the dynamic adaptation module is notified. The version mapping table is created during the system initialization phase and pre-stores the mapping relationship between all interpreters and their version numbers and types under the latest version. Each set of data in the mapping table includes the version number, the first data type, and the interpreter ID (i.e., the interpreter type). The version mapping table is used to match the corresponding interpreter ID for this data.

[0081] Specifically, such as Figure 2 The diagram shows the overall workflow of the interpretation layer. After obtaining BLOB data from the storage layer, the interpretation layer parses the data's version number and first data type identifier. First, it matches the first data type. If this type does not exist in the version relationship mapping table, the parsing of the binary data is terminated. It then determines if the version number is new, including: if a set of "version number + first data type" in the version relationship mapping table completely matches the overall characteristics of this binary data, then the version number is new and belongs to the new version data; the match is successful, and the corresponding interpreter ID is notified to the data parsing module. If the first data type exists in the mapping table, but the version number of this binary data does not match the version number in the mapping table, then the version number is old and belongs to the old version data; the match fails. Using "current version number + first data type of this binary data" as the matching condition, the interpretation layer obtains the interpreter ID of the corresponding type in the current version, which serves as the reference interpreter for the dynamic adapter. The interpreter library is updated using the new interpreter template, and the data parsing module parses the data, ultimately generating a new interpreter template and its corresponding unique ID.

[0082] The version mapping table includes a general feature identifier and an interpreter ID; it supports establishing mapping relationships between different versions, first data types, and interpreters based on the general feature identifier; during system initialization, it pre-stores all current version interpreters and their version numbers, as well as the mapping relationships of the first data types; it is updated in real time during system operation, enabling interpreter matching for all types of data, including historical versions.

[0083] The dynamic adaptation module dynamically generates new interpreter templates for binary data that cannot be matched with the interpreter. The specific process includes extracting feature identifiers for each field, analyzing field mappings, generating interpreter templates, and updating the interpreter library and version mapping manager.

[0084] Extraction of feature identifiers for each field: When the interpreter matching module fails to match, extract the feature identifiers of each field of the binary data (field number, secondary data type, field length).

[0085] Field mapping rule analysis: Combining the feature identifiers of each field of binary data with the rules of each field of the reference interpreter, define the parsing rules of each field of the data and generate a new interpreter template;

[0086] Update the interpreter library and version mapping table, store the new interpreter template in the interpreter library, and update the version mapping table with "version number, type, new interpreter template ID (i.e., new interpreter template type)" as a set of mapping data.

[0087] In some possible embodiments, the interpretation layer reads a BLOB data from the data table from the storage layer, first parsing the header 3 bytes of information to obtain the data version number (version) and the first data type (data_Type); for example, version=2.0 and data_Type=3 represent operation record data in version 2.0.

[0088] The version relationship mapping table stores version number, data type, and interpreter ID. Within the table, a set of "version number + first data type" corresponds to a unique interpreter. The "version number + first data type" is used to match the interpreter ID to prevent the same type of data from corresponding to the same interpreter ID in new and old versions. The interpreter ID is the number of the interpreter in the interpreter library, and each ID corresponds to a unique interpreter.

[0089] The version relation mapping table is used to match the interpreter: First, it matches the first data type in the total feature identifier. If the first data type does not exist in the mapping table, the parsing of the BLOB data is terminated.

[0090] If the first data type exists in the mapping table, match the unique interpreter ID using "BLOB version number + BLOB first data type" as the condition. If the match is successful, pass the interpreter ID to the parsing module.

[0091] If a match fails, the interpreter for the BLOB of the current version number and the first data type of the BLOB is obtained using the condition "current new version number + BLOB first data type". This interpreter is then passed to the dynamic adaptation module as a reference interpreter. For example, if the header type is version=1.1 and data_Type=3, and the current version is 2.0, the interpreter corresponding to version=2.0 and data_Type=3 is selected as the reference interpreter.

[0092] In this embodiment, the field mapping analysis analyzes each field by comparing the current binary data and referring to the field feature identifiers of the interpreter, and divides the fields into five types: same field, similar field, different field, missing field, and redundant field.

[0093] Furthermore, a new interpreter template is generated based on the field type. Specifically: for identical fields, the new interpreter template fully adopts the parsing rules for that field in the reference interpreter; for similar fields, the second data type of the field is converted without losing data precision to achieve partial data type compatibility; for different fields, the field is directly discarded; for missing fields that are present in the new version but not in the old version, the new interpreter template fills the field with a default safe value; for redundant fields that are present in the old version but have been deleted in the new version, the new interpreter template skips the parsing of that field.

[0094] The parsing process for the feature identifiers of each field in BLOB data is as follows: First, the first four bytes are parsed to obtain the feature identifier of the first field. The starting position of the next field is inferred based on the field length. The feature identifier of the second field is then parsed, and so on, until the parsing of all fields is completed.

[0095] Specifically, for the same fields: the field number is the same, the second data type is the same, and the data length is the same (for string types, data length is not compared), the new interpreter template completely uses the parsing rules for that field in the reference interpreter;

[0096] Similar fields with identical field numbers must conform to the mapping rules of the reference interpreter's second data type. Specifically, if the second data type of the BLOB is uint8_t, and the reference interpreter classifies this field as int, the new interpreter template parses the BLOB field according to the uint8_t type and converts it to int; conversely, if the second data type of the BLOB is uint8_t, and the reference interpreter classifies this field as int, the new interpreter template parses the BLOB field according to the uint8_t type and converts it to float.

[0097] Different fields: If the field numbers are the same, but the field does not conform to the mapping rules of the second data type of the reference interpreter, then the new interpreter template will abandon the parsing of that field;

[0098] Redundant fields: If a field exists in the BLOB data and there is no corresponding number field in the reference interpreter, then the field is a redundant field, and the new version of the interpreter template will not parse this field;

[0099] Missing field: If a field exists in the reference interpreter but there is no corresponding number field in the BLOB data, then the field is a newly added field, and the new version of the interpreter fills it with a default safe value according to the data type of the field;

[0100] The specific mapping rules are shown in Table 2:

[0101] Table 2. Mapping Rule Table

[0102] Default security values ​​are shown in Table 3:

[0103] Table 3. Default Security Values ​​Table

[0104]

[0105] Specifically, the dynamic adaptation module generates a new interpreter template through feature vector calculation and field rule mapping analysis, and stores the new interpreter template in the interpreter library, updating the version mapping table with the content of "version number + first data type + new interpreter template ID"; furthermore, the new interpreter template ID is passed to the data parsing module.

[0106] In this embodiment, the data parsing module includes a field rule verification unit and a data parsing unit;

[0107] The field rule verification unit is used to perform field rule verification;

[0108] The data parsing unit is used to perform data parsing.

[0109] The data parsing module receives the BLOB data to be parsed and the interpreter ID or new interpreter template ID, calls the final interpreter from the interpreter library, and performs field rule validation and data parsing.

[0110] The field rule verification includes integrity verification and version verification. Integrity verification ensures that the data packet is not corrupted through CRC check. Version verification verifies whether the data version is within the range supported by the version mapping table by obtaining the data version of the BLOB and the first data type. If the verification is successful, data parsing is performed; if the verification fails, the data parsing is abandoned.

[0111] Specifically, field rule validation includes: verifying whether the parsed data type conforms to the interpreter definition (such as integer range, string length) based on the final interpreter; checking whether the required fields required by the new version exist; marking redundant fields and recording their position information; and abandoning the parsing of the binary data if the validation fails.

[0112] After the field rules pass validation, the data parsing execution includes: performing data parsing using the final interpreter.

[0113] In some possible implementations, taking the pressure plate information table as an example, there are cases where the new interpreter template has added fields, but the BLOB data has missing fields, i.e., missing fields.

[0114] The fields and feature identifiers of BLOB data are shown in Table 4:

[0115] Table 4. First Field and Feature Recognition Identifier Table

[0116]

[0117] The rules for each field of the interpreter are shown in Table 5:

[0118] Table 5. First Reference Interpreter Field Rules Table

[0119]

[0120] For the 0th field in the BLOB data, i.e. the field with field number 0, compare it with the corresponding field in the reference interpreter: the field numbers are the same, the data types are the same (string type), they are the same field, and the new version of the interpreter template uses the reference interpreter rules to parse this field;

[0121] For the first field in the BLOB data, compare it with the corresponding field in the reference interpreter: if the field number, data type, and field length are the same, it is the same field, and the new version of the interpreter template uses the reference interpreter rules to parse this field.

[0122] For the second field of the reference interpreter, there is no field with field number 2 in the BLOB data, so it is a missing field. Create this field and fill the "Pressure Plate ID" field with the default value of 0 in the new version of the interpreter template.

[0123] Furthermore, the parsing results of the new interpreter template for this BLOB data are shown in Table 6:

[0124] Table 6. First Analysis Results Table

[0125]

[0126] In some possible embodiments, taking the pressure plate information table as an example, for cases where the reference interpreter has a deleted field, and the field is redundant in the BLOB data, i.e., a redundant field;

[0127] The fields and feature identifiers of BLOB data are shown in Table 7:

[0128] Table 7. Second Field and Feature Identification Table

[0129]

[0130] The rules for each field of the reference interpreter are shown in Table 8:

[0131] Table 8. Second Reference Interpreter Field Rules Table

[0132]

[0133] For the 0th field in the BLOB data, compare it with the corresponding field in the reference interpreter: the field numbers are the same, the data types are both string, so they are the same field; the new interpreter template uses the reference interpreter rules to parse this field.

[0134] For the second field in the BLOB data, compare it with the corresponding field in the reference interpreter: the field number, data type, and field length are the same, so it is the same field; the new version of the interpreter template uses the reference interpreter rules to parse this field;

[0135] For the first field in the BLOB data, the field with field number 1 does not exist in the interpreter, so this field is redundant; the new version of the interpreter template abandons the parsing of this field.

[0136] Furthermore, the parsing results of the new interpreter template for this BLOB data are shown in Table 9:

[0137] Table 9. Second Analysis Results Table

[0138]

[0139] In some possible implementations, taking a log information recording table as an example, this addresses the situation where the reference interpreter has a deleted field, and the field is redundant in the BLOB data;

[0140] The fields and feature identifiers of BLOB data are shown in Table 10:

[0141] Table 10. Third Field and Feature Identification Table

[0142]

[0143] The rules for each field of the interpreter are shown in Table 11:

[0144] Table 11. Third Reference Interpreter Field Rules Table

[0145]

[0146] For the 0th field in the BLOB data, compare it with the corresponding field in the reference interpreter: the field numbers are the same, but the data types are different, so they are similar fields; the new version of the interpreter template parses this field according to uint8_t and converts the value to int type;

[0147] For the first field in the BLOB data, compare it with the corresponding field in the reference interpreter: the field numbers are the same, but the data types are different, so they are similar fields; the new version of the interpreter template parses the field according to uint8_t and converts the value to float type;

[0148] For the second field in the BLOB data, comparing it with the corresponding field in the reference interpreter: the field numbers are the same, but the data types and field lengths are different, so they are different fields; the new version of the interpreter template abandons parsing this field;

[0149] Furthermore, the parsing results of the new interpreter template for this BLOB data are shown in Table 12:

[0150] Table 12, Third Analysis Results Table

[0151]

[0152] This invention implements interpreter storage through an interpreter library, and eliminates the high degree of binding between data storage and parsing by using a storage layer and a data parsing module. Through a layered architecture design of data storage, storage layer, and interpretation layer, the reusability and scalability of the data persistence module are improved. By generating new interpreter templates through a dynamic adaptation module, data updates are achieved, improving the compatibility of data persistence between new and old versions of the system, and providing a highly compatible solution for data parsing under the iteration of industrial automation equipment. Example

[0153] This invention provides a data persistence method based on a dynamic interpreter, and a data persistence system based on a dynamic interpreter as described in Embodiment 1, comprising:

[0154] Store binary data, add a general feature identifier (version number, first data type) to the header of each data entry, and store field feature identifiers in the header of each field in the data;

[0155] Step a: The interpreter matching module retrieves data from the storage layer and parses the data to obtain the version number and the first data type, then proceeds to step b;

[0156] Step b: The interpreter matching module queries the version mapping table based on the version number and the first data type, matches the data with the corresponding interpreter, and obtains the matching result;

[0157] If the first data type does not exist in the version mapping table, the matching result is a failure, the data is discarded and the parsing of the data is terminated, and the work ends; if both the first data type and the version number exist in the version mapping table, the matching result is a success, the interpreter type of the corresponding interpreter is sent to the data parsing module, and step d is executed; if the first data type exists in the version mapping table, but the version number does not exist in the version mapping table, the matching result is a failure, and step c is executed.

[0158] The version mapping table is obtained from the interpreter library;

[0159] Step c: The dynamic adaptation module parses the field feature identifiers of each field in the data;

[0160] The dynamic adaptation module obtains the current version, determines the reference interpreter in real time based on the current version, and obtains the field feature identifier of the reference interpreter;

[0161] The dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, stores the new interpreter template in the interpreter library, and sends the new interpreter template type of the new interpreter template to the data parsing module, and executes step d;

[0162] Step d: The data parsing module calls the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and performs field rule validation and data parsing on the original data according to the final interpreter to obtain the data parsing result.

[0163] In this embodiment, the execution of field rule validation and data parsing specifically includes:

[0164] The field rule validation includes data type validation and structural integrity checks.

[0165] If both data type validation and structural integrity check pass, the data parsing unit completes data parsing based on the final interpreter.

[0166] Otherwise, discard the data and terminate its parsing; the task is complete.

[0167] The field feature identifiers of each field in the parsed data specifically include:

[0168] First, parse the first N bytes of the data to obtain the field feature identifier of the first field, and infer the starting position of the next field based on the first field feature identifier. Then, parse the field feature identifier of the second field, and so on, until the parsing of the field feature identifiers of each field is completed.

[0169] Where N is the preset number of bytes, which is optional, and N equals 4.

[0170] The dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, specifically including:

[0171] Based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, each field is classified to obtain the field type;

[0172] Generate a new interpreter template based on the field type;

[0173] The field feature identifier includes the field number, the second data type, and the field length.

[0174] In this embodiment, the field types obtained specifically include identical fields, similar fields, different fields, redundant fields, and missing fields;

[0175] The same field: This field has the same field number, second data type, and field length as the reference interpreter; however, if the field and the second data type of the reference interpreter are both strings, then the field length does not need to be the same.

[0176] The similar field: This field has the same field number as the reference interpreter, and this field conforms to the mapping rules with the second data type of the reference interpreter;

[0177] The different fields: the field has the same field number as the reference interpreter, and the field does not conform to the mapping rules of the second data type of the reference interpreter;

[0178] The redundant fields: the field feature identifier of each field is compared with the field feature identifier of the reference interpreter, and the field number of the field is missing in the reference interpreter;

[0179] The missing field: When the field feature identifier of each field is compared with the field feature identifier of the reference interpreter, the field number of that field is missing in each field.

[0180] The new interpreter template specifically includes:

[0181] When the field types are the same, the new interpreter template parses the data for that field based on the corresponding field in the reference interpreter.

[0182] When the field type is similar, the new interpreter template parses the data based on that field and converts the second data type of that field in the data according to the mapping rules;

[0183] When the field types are different, the new interpreter template abandons data parsing for that field in the data;

[0184] When a field is a redundant field, the new interpreter template abandons data parsing for that field in the data;

[0185] When a field is a missing field, the new interpreter template creates a new field based on the extra field in the reference interpreter and fills it with a default safe value; wherein the default safe value is determined according to the field type of the newly created field.

[0186] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data persistence system based on a dynamic interpreter, characterized in that, Includes a storage layer and an interpretation layer; The storage layer is used to store the raw data; The interpretation layer includes an interpreter library, an interpreter matching module, a dynamic adaptation module, and a data parsing module; The interpreter library is used to store pre-registered interpreters and new interpreter templates generated by the dynamic adaptation module; The interpreter matching module is used to match the original data with the corresponding interpreter according to the version mapping table and the interpreter library. If the match is successful, the interpreter type of the corresponding interpreter is sent to the data parsing module. The dynamic adaptation module is used to generate a new interpreter template based on the original data when the interpreter matching module fails to match, store the new interpreter template in the interpreter library, and send the new interpreter template type corresponding to the new interpreter template to the data parsing module. The data parsing module is used to call the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and to perform field rule validation and data parsing on the original data according to the final interpreter to obtain the parsing result of the original data; The original data includes multiple data blocks, each data block is used to store one type of data, the header of the data contains a general feature identifier, the data contains multiple fields, and the header of each field contains a field feature identifier. The overall feature identifier includes a version number and a first data type; the field feature identifier includes a field number, a second data type, and a field length. Both the first data type and the second data type include 8-bit unsigned integers, 32-bit integers, floating-point numbers, strings, and booleans; The interpreter matching module retrieves data from the storage layer and parses the data to obtain the version number and the first data type; The interpreter matching module queries the version mapping table based on the version number and the first data type, matches the data with the corresponding interpreter, and obtains the matching result; If the first data type does not exist in the version mapping table, the matching result is a failure, and the work ends. If both the first data type and the version number exist in the version mapping table, the matching result is a success, and the interpreter type of the corresponding interpreter is sent to the data parsing module. If the first data type exists in the version mapping table, but the version number does not exist in the version mapping table, the matching result is unsuccessful, and the dynamic adaptation module parses the field feature identifiers of each field in the data. The dynamic adaptation module obtains the current version, obtains the current version number based on the current version, determines the reference interpreter in real time based on the current version number and the first data type, and obtains the field feature identifiers of the reference interpreter. The dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, stores the new interpreter template in the interpreter library, and sends the new interpreter template type of the new interpreter template to the data parsing module. The version mapping table is obtained from the interpreter library; The data parsing module calls the final interpreter from the interpreter library according to the interpreter type or the new interpreter template type, and performs field rule validation and data parsing on the original data according to the final interpreter to obtain the data parsing result; The field feature identifiers of each field in the parsed data specifically include: First, parse the first N bytes of the data to obtain the field feature identifier of the first field, and infer the starting position of the next field based on the first field feature identifier. Then, parse the field feature identifier of the second field, and so on, until the parsing of the field feature identifiers of each field is completed. Where N is the preset number of bytes; The dynamic adaptation module generates a new interpreter template based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, specifically including: Based on the field feature identifiers of each field and the field feature identifiers of the reference interpreter, each field is classified to obtain the field type; Generate a new interpreter template based on the field type; The field feature identifier includes the field number, the second data type, and the field length; The specific field types include identical fields, similar fields, different fields, redundant fields, and missing fields; The new interpreter template specifically includes: When the field types are the same, the new interpreter template parses the data for that field type based on the corresponding field in the reference interpreter. When the field type is similar, the new interpreter template parses the data based on the field of that type in the data, and converts the second data type of the field of that type in the data according to the mapping rules; When the field types are different, the new interpreter template abandons data parsing for fields of that type in the data; When a field is a redundant field, the new interpreter template abandons data parsing for fields of that type in the data; When a field is of type missing, the new interpreter template creates a new field based on the field of that type in the reference interpreter and fills it with a default safe value; wherein the default safe value is determined based on the field type of the newly created field.

2. The data persistence system based on a dynamic interpreter according to claim 1, characterized in that, The data parsing module includes a field rule validation unit and a data parsing unit; The field rule verification unit is used to perform field rule verification; The data parsing unit is used to perform data parsing.

3. The data persistence system based on a dynamic interpreter according to claim 1, characterized in that, The execution field rule validation and data parsing specifically include: The field rule validation includes data type validation and structural integrity checks. If both data type validation and structural integrity check pass, the data parsing unit completes data parsing based on the final interpreter. Otherwise, discard the data and terminate its parsing; the task is complete.

4. The data persistence system based on a dynamic interpreter according to claim 1, characterized in that, If a field is of the same type as the aforementioned field, then the field number, second data type, and field length are all the same as those of the reference interpreter; however, if the field and the second data type of the reference interpreter are both strings, then the field length does not need to be the same. If a field is of the type of the similar field, then the field has the same field number as the field number of the reference interpreter, and the field conforms to the mapping rules with the second data type of the reference interpreter; If a field is of type different from the above fields, then the field has the same field number as the field number of the reference interpreter, and the field does not conform to the mapping rules of the second data type of the reference interpreter; If a field is of the type of the redundant field, then the field feature identifier of the field is compared with the field feature identifier of the reference interpreter, and the field number of the field is missing in the reference interpreter; If a field is of the type of the missing field, then the field feature identifier of the field is compared with the field feature identifier of the reference interpreter, and the field number of the field is missing in the data.

Images