Method and device for storing and reading data by using Protobuf

Abstract

The invention discloses a data storage method and device and a data reading method and device using Probuf, and relates to the technical field of data storage, the data storage method using Probuf comprises the following steps: constructing a Probuf message instance, storing data to be stored in the Probuf message instance, and serializing the Probuf message instance into binary data; structural data used for self-describing the Protobuf message instance are constructed, the structural data comprise serialized binary data and self-description information, and the self-description information comprises a message instance name used for representing the type of the serialized binary data and a check code of the structural data; and writing the structural data into the binary data file. According to the method, the problems of data format compatibility and difficulty in expansion can be solved.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a data storage and reading method and device using Protobuf. Background Art

[0002] Protobuf is an efficient and lightweight information description format designed by Google. It is language-neutral, platform-neutral, highly efficient, and scalable, making it ideal for data storage and exchange. Compared to protocols like JSON and XML that use text encoding rules, Protobuf uses binary encoding rules, resulting in shorter encoded data and higher storage efficiency.

[0003] Typically, in traditional binary data storage formats, fields are identified by byte offsets (i.e., field A has an offset of 0 in the file, field B has an offset of 4, field N has an offset of 256, and so on). This approach presents the following problems: on the one hand, encoding is required even when a field parameter is the default value or does not exist, resulting in reduced encoding efficiency; on the other hand, adding new fields to the stored data can cause incompatibility between old and new versions of data and applications.

[0004] However, each field in the Protobuf protocol has a unique number. When stored, the number is stored together with the field, and the field is identified by the field number. This method has the following advantages: fields that do not need to be stored can be excluded from encoding and do not affect the decoding of subsequent data; when adding new fields, as long as the field encoding is not repeated, data compatibility can be guaranteed. Adding new fields does not cause any changes to already released programs, and the data structure can be upgraded without destroying deployed programs that rely on the old data format. In this way, devices, software, or platforms do not need to worry about large-scale code reconstruction or migration issues caused by changes in data structure.

[0005] In the prior art, the proto source file of the Protobuf protocol defines the data format, that is, it defines the field name, field encoding and data type of the data. The proto source file is compiled with the Protobuf compilation tool to generate a program source file of the corresponding programming language. A Protobuf message instance can be constructed through the program source file. When the message instance is to be stored, the message instance in the memory needs to be serialized into binary data. The serialized binary data does not contain a message type identifier, so a method is needed to identify the message type of the encoded binary data. The usual method is to define a root type message, and all other message types are used as fields of the root message. This method has many limitations. For example, when there are many message types, the root message becomes very complicated, and when a new message type is added, the root category also has to increase the field accordingly to store the message type, and the scalability is limited.

[0006] As can be seen, in the prior art, when storing or reading data from a message instance, a root message needs to be constructed, the message instance is added to the root message, and the message instance is serialized into binary data. When a new message type is added, the root message also needs to add corresponding fields. When there are multiple parties maintaining and updating the proto source file of the root message, conflicts may occur in the root proto file, requiring manual maintenance of the proto file consistency, increasing communication costs and resulting in higher maintenance costs. Summary of the Invention

[0007] The present application provides a data storage and reading method and device using Protobuf, which can solve the problems of data format compatibility and difficulty in expansion.

[0008] In a first aspect, an embodiment of the present application provides a data storage method using Protobuf, the data storage method using Protobuf comprising:

[0009] Construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data;

[0010] Constructing structure data for a self-describing Protobuf message instance, the structure data including serialized binary data and self-describing information, the self-describing information including a message instance name for characterizing the type of the serialized binary data and a check code for the structure data;

[0011] The structure data is written into a binary data file.

[0012] In conjunction with the first aspect, in one embodiment, constructing a Protobuf message instance, storing the data to be stored in the Protobuf message instance, and serializing the Protobuf message instance into binary data includes:

[0013] Construct a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance;

[0014] Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

[0015] In conjunction with the first aspect, in one embodiment, the constructing of structural data for self-describing a Protobuf message instance, the structural data including serialized binary data and self-describing information, the self-describing information including a message instance name for characterizing the type of the serialized binary data and a check code for the structural data, includes:

[0016] Build the structure data and write the message instance name into the message name field;

[0017] Fill the serialized binary data into the application layer field;

[0018] Calculates the checksum of the structure data based on the serialized binary data and fills the fields.

[0019] In combination with the first aspect, in one implementation, the message instance name includes a package name of application layer data and a message name.

[0020] In a second aspect, an embodiment of the present application provides a data storage device using Protobuf, wherein the data storage device using Protobuf includes:

[0021] A data acquisition module, which is used to acquire data to be stored;

[0022] A first message instance construction module is configured to: construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data;

[0023] A structure data construction module is used to construct structure data for a self-describing Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data;

[0024] The data storage module is used to write the structure data into a binary data file.

[0025] In combination with the second aspect, in one embodiment, the first message instance construction module constructs a Protobuf message instance, stores the data to be stored in the Protobuf message instance, and serializes the Protobuf message instance into binary data, including:

[0026] Construct a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance;

[0027] Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

[0028] In combination with the second aspect, in one embodiment, the structure data construction module constructs structure data for self-describing a Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, and the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data, including:

[0029] Build the structure data and write the message instance name into the message name field;

[0030] Fill the serialized binary data into the application layer field;

[0031] Calculates the checksum of the structure data based on the serialized binary data and fills the fields.

[0032] In combination with the second aspect, in one embodiment, the message instance name includes a package name of application layer data and a message name.

[0033] In a third aspect, an embodiment of the present application provides a method for reading data using Protobuf, the method comprising:

[0034] Loading a binary data file to obtain structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing a type of the serialized binary data and a check code for the structure data;

[0035] Parsing the serialized binary data, the message instance name, and the checksum from the structure data, calculating the checksum of the serialized binary data and comparing it with the checksum;

[0036] Construct a Protobuf message instance based on the message instance name, call the deserialization API of the Protobuf message instance to deserialize the serialized binary data, and restore the stored data.

[0037] In a fourth aspect, an embodiment of the present application provides a data reading device using Protobuf, wherein the data reading device using Protobuf includes:

[0038] A data loading module, which is used to load a binary data file and obtain structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data;

[0039] a structure data parsing module, which is used to parse the serialized binary data, the message instance name and the check code from the structure data, calculate the check code of the serialized binary data and compare it with the check code;

[0040] The second message instance construction module constructs a Protobuf message instance according to the message instance name;

[0041] The data deserialization module is used to call the deserialization API of the Protobuf message instance to deserialize the serialized binary data and restore the stored data.

[0042] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0043] The data storage method using Protobuf in the present application constructs a Protobuf message instance, stores the data to be stored in the Protobuf message instance, and serializes the Protobuf message instance into binary data; constructs structural data for self-describing the Protobuf message instance, the structural data including the serialized binary data and self-describing information, the self-describing information including a message instance name for characterizing the type of the serialized binary data and a check code for the structural data; and writes the structural data into a binary data file.

[0044] Since the structured data carries self-describing information, application software and display platforms can automatically parse data files of different formats based on this information. The specific fields of the data are defined using the Protobuf format. After the software is upgraded, it can still read the data of the old version, and the old version of the software can also read the data files of the new version, achieving compatibility and scalability of the data format. It has the characteristics of two-way compatibility and effectively reduces the cost of product upgrades and maintenance. Furthermore, when defining or updating the data format, there is no need to manually write the encoding and decoding program code of the corresponding data format. Instead, the proto file compilation tool can be used to conveniently generate parsing code in different languages (C / C++, Java, Python, C#, JavaScript, etc.), thereby improving development efficiency and being applicable to different development platforms. In addition, the structured data itself also contains verification information. By comparing the verification code generated based on the structured data with the verification information, incorrect data reading is prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flowchart of an embodiment of a data storage method using Protobuf in this application;

[0046] Figure 2 Schematic diagram of the composition of the data header;

[0047] Figure 3 This is a schematic diagram of the structure of serialized binary data;

[0048] Figure 4 This is a flowchart of an embodiment of a method for reading data using Protobuf in this application;

[0049] Figure 5 This is a structural block diagram of an embodiment of a data storage device using Protobuf in this application;

[0050] Figure 6 This is a structural block diagram of an embodiment of a data reading device using Protobuf in this application. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] It is worth noting that as application scenarios expand, monitoring content and parameters also expand, and the types, parameters, and fields of data generated by equipment operation also become more complex. There are not only collected raw data, but also data filtered by algorithms, and data after feature extraction by algorithms or result data of algorithm application. This also increases the difficulty of data format design, storage, and reading.

[0053] As more and more devices are installed and deployed, the data generated during the operation of the devices plays an important role in the training of application algorithms. It is also a very important data accumulation and asset. When upgrading and maintaining the equipment, the old version of the data should maintain the greatest compatibility. Traditional data storage solutions have version compatibility issues, the upgrade and maintenance costs increase, and the possibility of problems also increases. With the upgrade and update of application software or display platforms, there will also be problems with failure to read the original data.

[0054] At the same time, data will flow between collection software and application algorithms, display platforms and other systems. Different systems may be implemented using different programming languages and development platforms, and complex data formats will reduce development efficiency.

[0055] Therefore, the present application provides a binary storage and reading solution that is bidirectionally compatible, scalable, easy to implement, and efficient.

[0056] It should be noted that the data storage method using Protobuf provided in the embodiments of the present application can be applied to operating platforms such as Linux and Windows, and can be applied to programming languages or development platforms including but not limited to C / C++, Java, Python, C#, JavaScript, etc. In addition, in order to be applicable to a specific operating system or a specific language environment, the program code implementing the data storage method using Protobuf provided in this embodiment can be written according to the characteristics of the operating system or language environment.

[0057] Serialization refers to the conversion of structured data or memory objects into a format that can be stored and transmitted, while ensuring that the serialized result can be reconstructed through deserialization operations to the original structured data or memory objects. Taking the Protobuf protocol as an example, a Protobuf message instance can be serialized using the serialization API provided by the Protobuf language library to obtain the serialized binary data of the message instance; at the same time, the serialized binary data can also be deserialized and restored to the original Protobuf message instance using the deserialization API provided by the Protobuf language library.

[0058] Generally speaking, traditional data formats define devices in the following ways:

[0059]

[0060] When equipment or software is upgraded, fields need to be added to the data file to record the added parameters, such as adding a filter coefficient field. In this case, the upgraded software cannot open the previous data, or the old version of the software cannot open the new version of the data, causing incompatibility and increasing upgrade and maintenance costs.

[0061] If you use the data file format defined by proto:

[0062]

[0063]

[0064] To add new fields, such as adding new field filter coefficients, you only need to update the proto file:

[0065]

[0066] Then use protoc to compile the new proto file to generate a new corresponding programming language source file. Based on the characteristics of the Protobuf format, the data format has bidirectional compatibility at this time, that is, the upgraded software can open the old version of the data, and the old version of the software can also read the new version of the data.

[0067] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0068] In a first aspect, an embodiment of the present application provides a data storage method using Protobuf.

[0069] In one embodiment, referring to Figure 1 As shown, Figure 1 This is a flow chart of an embodiment of a data storage method using Protobuf in this application. Figure 1 As shown, the data storage methods using Protobuf include:

[0070] S1. Construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data;

[0071] It is worth noting that in this embodiment, it is necessary to first obtain the data to be stored. In actual applications, the data to be stored can be any type of data, such as integer, floating point, byte, string, etc. At the same time, the data to be stored can be one or more data of the same type (array or list), or a combination of multiple different types of data.

[0072] In specific implementation, step S1 includes:

[0073] S11. Build a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance;

[0074] S12. Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

[0075] S2. Constructing structural data for a self-describing Protobuf message instance, wherein the structural data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structural data;

[0076] In specific implementation, step S2 includes:

[0077] S21. Build structure data and write the message instance name into the message name field;

[0078] In this embodiment, the message instance name includes the package name of the application layer data and the message name.

[0079] S22. Fill the serialized binary data into the application layer field;

[0080] S23. Calculate the check code of the structure data according to the serialized binary data and fill in the fields.

[0081] S3. Writing the structure data into a binary data file.

[0082] The scheme in this application divides the data into a data header and an application layer. The data header consists of a frame header, a frame length, a data payload, and a checksum field. Figure 2 As shown in the figure, the application layer is divided into two fields: type name and application layer data. The type name consists of two parts: the first part is the package name of the Protobuf message, and the second part is the name of the message. The application layer data is the binary data serialized by Protobuf. Figure 3 .

[0083] The following is an example to illustrate this. There is a proto file:

[0084]

[0085] Taking the compilation into C++ programming language as an example, use the proto source file compilation tool to compile the above proto file. The generated C++ source file will contain the C++ class: DTS::RawData.

[0086] To save the original data, follow these steps:

[0087] Get the original data (data to be stored), including sampling parameters such as sampleRate, sampleLength, sampleTime, sampleCount, and the original data array datas;

[0088] Construct a DTS::RawData instance object, fill the corresponding fields of the instance object with the above data, and call the SerializeToArray interface of the message instance to serialize the instance object into binary data;

[0089] Construct structure data and fill in the corresponding fields. For example, fill the message name field with "DTS.RawData" and the application layer field with the serialized data of the above DTS::RawData instance object. Calculate the checksum of the structure data and fill in the fields.

[0090] The data storage step is completed by saving the above structure data to a binary data file.

[0091] To sum up, the data storage method using Protobuf in the present application constructs a Protobuf message instance, stores the data to be stored in the Protobuf message instance, and serializes the Protobuf message instance into binary data; constructs structural data for self-describing the Protobuf message instance, the structural data including serialized binary data and self-describing information, the self-describing information including a message instance name for characterizing the type of serialized binary data and a check code for the structural data; and writes the structural data into a binary data file.

[0092] Since the structured data carries self-describing information, application software and display platforms can automatically parse data files of different formats based on this information. The specific fields of the data are defined using the Protobuf format. After the software is upgraded, it can still read the data of the old version, and the old version of the software can also read the data files of the new version, achieving compatibility and scalability of the data format. It has the characteristics of two-way compatibility and effectively reduces the cost of product upgrades and maintenance. Furthermore, when defining or updating the data format, there is no need to manually write the encoding and decoding program code of the corresponding data format. Instead, the proto file compilation tool can be used to conveniently generate parsing code in different languages (C / C++, Java, Python, C#, JavaScript, etc.), thereby improving development efficiency and being applicable to different development platforms. In addition, the structured data itself also contains verification information. By comparing the verification code generated based on the structured data with the verification information, incorrect data reading is prevented.

[0093] In a second aspect, an embodiment of the present application provides a data reading method using Protobuf.

[0094] In one embodiment, referring to Figure 4 As shown, Figure 4 This is a flow chart of an embodiment of a method for reading data using Protobuf. Figure 4 As shown, the data reading method using Protobuf includes:

[0095] S1 , , loading a binary data file, obtaining structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing a type of the serialized binary data and a check code for the structure data;

[0096] S2 , , parsing the serialized binary data, the message instance name and the check code from the structure data, calculating the check code of the serialized binary data and comparing it with the check code;

[0097] S3 , , build a Protobuf message instance according to the message instance name, call the deserialization API of the Protobuf message instance to deserialize the serialized binary data and restore the stored data.

[0098] It should be noted that the data reading method using Protobuf provided in this embodiment can be applied to operating systems such as Linux and Windows, and can also be applied to programming languages or development platforms including but not limited to C / C++, Java, Python, C#, JavaScript, etc. In addition, in order to be applicable to a specific operating system or a specific language environment, the program code implementing the data reading method using Protobuf provided in this embodiment can be written according to the characteristics of the operating system or the language environment.

[0099] Based on the above-mentioned Protobuf data storage method, the stored binary file contains self-describing information, so the data reading program can obtain the package name and message name of the application layer data field from the file without knowing the relevant information of the binary file in advance, and then use the package name and message name to construct a Protobuf message instance, and then use the message instance to deserialize the application layer data and restore the stored structured data or memory object in memory.

[0100] The following is an example to illustrate this. There is a proto file:

[0101]

[0102] Taking the compilation into C++ programming language as an example, use the proto source file compilation tool to compile the above proto file. The generated C++ source file will contain the C++ class: DTS::TempData. Use the above storage method to save a data file. Now you need to read the contents of the binary file using the following steps:

[0103] Read all the data in the file;

[0104] Extract self-describing information from the data, mainly including the package name and message name of the application layer data, and calculate and verify the checksum;

[0105] The package name of the application layer data is "DTS", and the message name is "TempData". "DTS.TempData" is passed to the Protobuf language library to dynamically construct a Protobuf message instance;

[0106] Call the deserialization interface ParseFromArray of the above message instance to deserialize the above Protobuf message binary data and restore the binary data into structured data or memory objects.

[0107] In a third aspect, an embodiment of the present application provides a data storage device using Protobuf.

[0108] In one embodiment, referring to Figure 5 As shown, Figure 5 This is a structural block diagram of an embodiment of a data storage device using Protobuf in this application. Figure 5 As shown, the data storage device using Protobuf includes:

[0109] A data acquisition module, which is used to acquire data to be stored;

[0110] A first message instance construction module is configured to: construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data;

[0111] A structure data construction module is used to construct structure data for a self-describing Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data;

[0112] The data storage module is used to write the structure data into a binary data file.

[0113] In this embodiment, the Protobuf language library is used to support the reflection mechanism, and the Protobuf message instance can be dynamically generated by the name of the message instance. Self-describing information is inserted when storing data. When the software needs to store multiple data format types, the same storage solution can be used without worrying about confusion and conflicts caused by the large number of data formats. This also speeds up development efficiency and reduces maintenance costs.

[0114] Furthermore, in one embodiment, the first message instance construction module constructs a Protobuf message instance, stores the data to be stored in the Protobuf message instance, and serializes the Protobuf message instance into binary data, including:

[0115] Construct a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance;

[0116] Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

[0117] Furthermore, in one embodiment, the structure data construction module constructs structure data for self-describing a Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a checksum of the structure data, including:

[0118] Build the structure data and write the message instance name into the message name field;

[0119] Fill the serialized binary data into the application layer field;

[0120] Calculates the checksum of the structure data based on the serialized binary data and fills the fields.

[0121] Furthermore, in one embodiment, the message instance name includes a package name of application layer data and a message name.

[0122] Among them, the functional implementation of each module in the above-mentioned data storage device using Protobuf corresponds to the steps in the above-mentioned data storage method embodiment using Protobuf, and their functions and implementation processes are no longer repeated here.

[0123] In a fourth aspect, an embodiment of the present application provides a data reading device using Protobuf.

[0124] In one embodiment, referring to Figure 6 As shown, Figure 6 This is a structural block diagram of an embodiment of a data reading device using Protobuf in this application. Figure 6 As shown, the data reading device using Protobuf includes:

[0125] A data loading module, which is used to load a binary data file and obtain structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data;

[0126] a structure data parsing module, which is used to parse the serialized binary data, the message instance name and the check code from the structure data, calculate the check code of the serialized binary data and compare it with the check code;

[0127] The second message instance construction module constructs a Protobuf message instance according to the message instance name;

[0128] The data deserialization module is used to call the deserialization API of the Protobuf message instance to deserialize the serialized binary data and restore the stored data.

[0129] In this embodiment, the Protobuf language library supports reflection, and a Protobuf message instance can be dynamically generated based on the message instance name. When reading data, the package name and message name of the application-layer Protobuf binary data in the data file are obtained, and a Protobuf message instance is dynamically constructed. The application-layer data is then deserialized using this message instance to restore the stored structured data.

[0130] Among them, the functional implementation of each module in the above-mentioned data reading device using Protobuf corresponds to the various steps in the above-mentioned data reading method embodiment using Protobuf, and their functions and implementation processes are no longer repeated here.

[0131] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0132] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0133] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0134] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0135] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0137] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data storage method using Protobuf, characterized in that: The data storage method using Protobuf includes: Construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data; Constructing structure data for a self-describing Protobuf message instance, the structure data including serialized binary data and self-describing information, the self-describing information including a message instance name for characterizing the type of the serialized binary data and a check code for the structure data; The structure data is written into a binary data file.

2. The data storage method using Protobuf according to claim 1, wherein: The step of constructing a Protobuf message instance, storing the data to be stored in the Protobuf message instance, and serializing the Protobuf message instance into binary data includes: Construct a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance; Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

3. The data storage method using Protobuf according to claim 1, wherein: The structure data for constructing a self-describing Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data, includes: Build the structure data and write the message instance name into the message name field; Fill the serialized binary data into the application layer field; Calculates the checksum of the structure data based on the serialized binary data and fills the fields.

4. The data storage method using Protobuf as claimed in claim 3, characterized in that: The message instance name includes the package name of the application layer data and the message name.

5. A data storage device using Protobuf, characterized in that: The data storage device utilizing Protobuf includes: A data acquisition module, which is used to acquire data to be stored; A first message instance construction module is configured to: construct a Protobuf message instance, store the data to be stored in the Protobuf message instance, and serialize the Protobuf message instance into binary data; A structure data construction module is used to construct structure data for a self-describing Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data; The data storage module is used to write the structure data into a binary data file.

6. The data storage device using Protobuf as claimed in claim 5, wherein: The first message instance construction module constructs a Protobuf message instance, stores the data to be stored in the Protobuf message instance, and serializes the Protobuf message instance into binary data, including: Construct a Protobuf message instance and fill the data to be stored into the corresponding fields of the Protobuf message instance; Call the serialization API of the Protobuf message instance to serialize the Protobuf message instance into binary data.

7. The data storage device using Protobuf as claimed in claim 5, characterized in that: The structure data construction module constructs structure data for self-describing a Protobuf message instance, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data, including: Build the structure data and write the message instance name into the message name field; Fill the serialized binary data into the application layer field; Calculates the checksum of the structure data based on the serialized binary data and fills the fields.

8. The data storage device using Protobuf as claimed in claim 7, wherein: The message instance name includes the package name of the application layer data and the message name.

9. A data reading method using Protobuf, characterized in that: The data reading method using Protobuf includes: Loading a binary data file to obtain structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing a type of the serialized binary data and a check code for the structure data; Parsing the serialized binary data, the message instance name, and the checksum from the structure data, calculating the checksum of the serialized binary data and comparing it with the checksum; Construct a Protobuf message instance based on the message instance name, call the deserialization API of the Protobuf message instance to deserialize the serialized binary data, and restore the stored data.

10. A data reading device using Protobuf, characterized in that: The data reading device utilizing Protobuf includes: A data loading module, which is used to load a binary data file and obtain structure data, wherein the structure data includes serialized binary data and self-describing information, wherein the self-describing information includes a message instance name for characterizing the type of the serialized binary data and a check code for the structure data; a structure data parsing module, which is used to parse the serialized binary data, the message instance name and the check code from the structure data, calculate the check code of the serialized binary data and compare it with the check code; The second message instance construction module constructs a Protobuf message instance according to the message instance name; The data deserialization module is used to call the deserialization API of the Protobuf message instance to deserialize the serialized binary data and restore the stored data.

Images