Real-time monitoring platform and cross-language operation method of real-time database of real-time monitoring platform
By defining messages in the .proto file and generating a file with the suffix, and combining protobuf serialization and deserialization functions, the problems of low efficiency and memory consumption in cross-language operations of custom structures are solved, achieving efficient data storage and parsing.
Patent Information
- Application Number
- CN202411271874.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies suffer from low efficiency and additional memory overhead when storing and parsing custom structures, especially when performing cross-language operations.
By defining messages in the .proto file and using the protoc compiler to generate corresponding suffix files, the storage and retrieval ends call protobuf serialization and deserialization functions respectively to serialize the data into protobuf strings and store and retrieve them in the Redis database. A unified naming rule is used to name the keys.
It achieves efficient storage and parsing for cross-language operations, reduces memory overhead, and improves data conversion efficiency.
Smart Images
Figure CN121658113A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a real-time monitoring platform and its real-time database, and its cross-language operation method, belonging to the field of data storage and parsing. Background Technology
[0002] Real-time databases are database systems designed specifically for real-time data processing. They can handle real-time data from various data sources and provide real-time data access, real-time analysis, and real-time data update and synchronization functions.
[0003] In real-time monitoring platforms, operations on real-time databases typically involve storing and retrieving real-time data in key-value pair format, with the value often being a simple data type. For value data structures in the form of custom structs, there are currently two parsing methods: the first is based on Qt's QByteArray type conversion for storage and parsing, and the second is based on JSON string conversion for storage and parsing. However, while the first method is highly efficient, it does not meet the requirements for cross-language data interaction; the second method, while overcoming the interaction barriers between different languages, requires additional computing resources and time when serializing or deserializing JSON strings back to the custom struct, and the JSON strings contain extra characters, incurring additional memory overhead. Summary of the Invention
[0004] The purpose of this invention is to provide a cross-language operation method for a real-time monitoring platform and its real-time database, in order to solve the problems of low efficiency and additional overhead when storing and parsing custom structures.
[0005] To achieve the above objectives, the present invention includes: This invention provides a cross-language operation method for real-time databases, which is applicable when performing cross-programming language operations on key-value data: Add a message to the .proto file, where the field types in the message correspond one-to-one with the field types in the value. Use the protoc compiler to compile the .proto file and generate corresponding suffix files according to the programming languages used by the storage and retrieval ends, respectively. After importing the corresponding file extension file into the storage terminal, set the value of the value field to be stored, call the protobuf serialization function to serialize the file extension file into a protobuf string, and call the database storage interface function to store the protobuf string into the database; The extraction end calls the database read interface function to read the protobuf string, and then calls the protobuf deserialization function to deserialize the protobuf string into a structure corresponding to the programming language used by the extraction end.
[0006] Furthermore, the key is named using a predefined, unified naming rule.
[0007] Furthermore, the database is Redis.
[0008] Furthermore, the programming languages used for both storage and retrieval include C++ and Java.
[0009] A real-time monitoring platform performs the following steps when operating on key-value data across programming languages: Add a message to the .proto file, where the field types in the message correspond one-to-one with the field types in the value. Use the protoc compiler to compile the .proto file and generate corresponding suffix files according to the programming languages used by the storage and retrieval ends, respectively. After importing the corresponding file extension file into the storage terminal, set the value of the value field to be stored, call the protobuf serialization function to serialize the file extension file into a protobuf string, and call the database storage interface function to store the protobuf string into the database; The extraction end calls the database read interface function to read the protobuf string, and then calls the protobuf deserialization function to deserialize the protobuf string into a structure corresponding to the programming language used by the extraction end.
[0010] Furthermore, the key is named using a predefined, unified naming rule.
[0011] Furthermore, the database is Redis.
[0012] Furthermore, the programming languages used for both storage and retrieval include C++ and Java.
[0013] The beneficial effects of this invention are as follows: This invention is a pioneering invention. When performing cross-programming language operations on key-value data, this invention defines messages in a .proto file and compiles the .proto file to generate corresponding suffix files for the programming languages used by the storage and retrieval ends. The storage end calls a serialization function to serialize the value into string data and stores it in a real-time database. The retrieval end calls a deserialization function to deserialize the string data in the real-time database into a structure in the corresponding programming language. This invention effectively solves the problems of low efficiency and additional overhead when storing custom structures. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the overall process of storing and parsing a real-time database according to the present invention. Detailed Implementation
[0015] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0016] The concept of this invention is that when the storage end stores a custom structure to be stored, it first serializes the custom structure into a string, and then stores the serialized string into the real-time database; when the reading end reads the real-time database, it first deserializes the string into the corresponding structure, and then performs the corresponding operations.
[0017] Method Implementation Examples: This embodiment provides a cross-language operation method for real-time databases. When storing and parsing key-value data in a real-time database, in order to balance the needs of cross-language compatibility, conversion efficiency, and reduced memory overhead, a unified naming rule is adopted when naming the key, and the value is stored and parsed in the form of proto stream serialization.
[0018] This invention creates a custom structure for the real-time database value to be stored and parsed based on business requirements. This custom structure contains multiple business attribute fields of different data types and supports nested structures. A custom .proto file is created based on this custom structure. This file includes a custom message, with each message containing field types that correspond one-to-one with the field types in the custom structure. The .proto file is compiled using the protoc compiler to generate a file with the corresponding extension for the desired programming language. This includes extensions for the programming languages used on both the storage and retrieval sides. The storage side imports the corresponding extension and uses the protobuf serialization function to serialize the structure from the source code into a protobuf string. The serialized string data is then stored in the real-time database through the storage interface. The retrieval side imports the corresponding extension and uses the real-time database's read interface to read the serialized string data. The protobuf deserialization function then deserializes the read string data into the structure corresponding to the programming language used, thus completing cross-language data reading from the real-time database. QuestDB, MongoDB, and Redis, which are used in real-time data processing scenarios, will be used as real-time databases.
[0019] Specifically, such as Figure 1 As shown in the diagram, C++ and Java are used as the programming languages for storing and retrieving values. Thick arrows indicate C++ storage and Java retrieval; thin arrows indicate Java storage and C++ retrieval. Taking the storage and retrieval of data in a substation, using Redis as the real-time database, and C++ as the programming language for storage and Java for retrieval as an example, the process of this invention is illustrated, specifically including the following steps: S1. Create a custom .proto file and customize the messages within it, for example: message Status{ string position = 1; bool is_active=2; } message Breaker{ int32 id=1; string name = 2; Status status=3; } Here, Breaker represents the circuit breaker, and Status represents the device status.
[0020] S2. Use the protoc compiler to compile the .proto file to generate the corresponding .h and .cpp files for C++, and the corresponding .java file for Java.
[0021] S3. In the C++ implementation, reference the .h file and create an object corresponding to the message in step S1. Set the values of the corresponding fields in this object, then call the protobuf serialization function (SerializeToString) to serialize the object into a protobuf string. Finally, call the storage interface function provided by Redis to store the protobuf string in Redis. Before storage, name the key according to the unified naming rule "tablename_fieldname:objected". Here, tablename represents the table name stored in Redis, fieldname represents the field in the table, and objected represents the equipment in the substation.
[0022] S4. In the Java side, reference the .Java file, call the read interface function provided by Redis to read the protobuf string, and then call the protobuf deserialization function (ParseFromString) to deserialize the serialized protobuf string into the corresponding Java language structure. Then, in the Java side, use the corresponding field values in the structure set in the C++ side.
[0023] Platform Implementation Examples: This embodiment provides a real-time monitoring platform. When storing and retrieving data from a real-time database, the platform employs a cross-language operation method for real-time databases as described in the method embodiment. Since the method is already clearly described, it will not be repeated here.
[0024] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A cross-language operation method for a real-time database of a real-time monitoring platform, characterized in that, When performing cross-programming language operations on key-value data: Add a message to the .proto file, where the field types in the message correspond one-to-one with the field types in the value. Compile the .proto file using the protoc compiler to generate corresponding suffix files based on the programming languages used by the storage and retrieval ends, respectively. After importing the corresponding file extension file into the storage terminal, the value field to be stored is set, the protobuf serialization function is called to serialize the file extension file into a protobuf string, and the database storage interface function is called to store the protobuf string into the database. The extraction end calls the database reading interface function to read the protobuf string, and then calls the protobuf deserialization function to deserialize the protobuf string into a structure corresponding to the programming language used by the extraction end.
2. The cross-language operation method for the real-time database of the real-time monitoring platform according to claim 1, characterized in that, The key is named using a predefined, uniform naming rule.
3. The cross-language operation method for the real-time database of the real-time monitoring platform according to claim 1, characterized in that, The database is Redis.
4. The cross-language operation method for the real-time database of the real-time monitoring platform according to claim 1, characterized in that, The programming languages used in the storage and retrieval ends include C++ and Java.
5. A real-time monitoring platform, characterized in that, When performing cross-programming language operations on key-value data, the platform executes the following steps: Add a message to the .proto file, where the field types in the message correspond one-to-one with the field types in the value. Compile the .proto file using the protoc compiler to generate corresponding suffix files based on the programming languages used by the storage and retrieval ends, respectively. After importing the corresponding file extension file into the storage terminal, the value of the value field to be stored is set, the protobuf serialization function is called to serialize the value into a protobuf string, and the database storage interface function is called to store the protobuf string into the database; After the extraction end imports the corresponding file extension, it calls the database reading interface function to read the protobuf string, and then calls the protobuf deserialization function to deserialize the protobuf string into a structure corresponding to the programming language used by the extraction end.
6. The real-time monitoring platform according to claim 5, characterized in that, The key is named using a predefined, uniform naming rule.
7. The real-time monitoring platform according to claim 5, characterized in that, The database is Redis.
8. The real-time monitoring platform according to claim 5, characterized in that, The programming languages used in the storage and retrieval ends include C++ and Java.