DDS persistent data storage management method and system

Abstract

This invention relates to the field of DDS communication and proposes a method and system for managing persistent data storage in DDS. The method includes the following steps: obtaining the target persistence level declared by the application during the data subscription / publishing process through a persistence service quality policy, where the persistence level includes at least transient persistence and persistent persistence; during data communication, using a protocol engine; based on the persistence level, dispatching persistent operation tasks to storage backends through an operation dispatch table, whereby the storage backends perform persistent storage; the operation dispatch table defines a unified persistent operation interface in the form of function pointers; the storage backends include at least transient persistence backends and persistent persistence backends, with each storage backend registering its respective implementation path in the operation dispatch table. This invention solves the problem of increased management costs caused by the differentiated management of transient and persistent persistence levels.

Description

Technical Field

[0001] This invention relates to the field of DDS communication, and more particularly to a method and system for managing persistent DDS data. Background Technology

[0002] Data Distribution Service (DDS) controls data persistence through Durability Quality of Service (QoS) policies. The transient level requires data to be managed by a separate Durability Service process, while the persistent level requires data to be persisted to non-volatile storage media. The DDS specification only defines semantic requirements but does not specify the concrete implementation methods for transient and persistent persistence. Existing technologies mainly suffer from the following problems regarding persistence:

[0003] First, while transient and persistent persistence are highly consistent in terms of data model and operational semantics, they are currently typically managed separately, resulting in fragmented implementation of the underlying system code. This not only causes a large amount of code redundancy and redundant development, increasing the cost of later software maintenance and vulnerability patching, but also easily leads to conflicts in system operation behavior due to inconsistencies in boundary conditions or exception handling between the two independent management logics.

[0004] Secondly, current solutions typically employ a tightly coupled, hard-coded approach. This means that the specific implementation logic for transient and persistent persistence is scattered and intertwined within the DDS protocol engine. When adjustments to the storage backend are needed, it requires delving into the core logic of DDS communication, extensively modifying and refactoring the core communication code, significantly increasing later maintenance costs. Furthermore, the same dilemma arises when the system needs optimization or the addition of a new storage backend, greatly increasing the development difficulty of the middleware software. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a DDS persistent data storage management method and system to address the shortcomings of existing technologies and solve the problem of increased management costs caused by distinguishing between transient-level persistence and persistent-level persistence.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0007] Firstly, a persistent data storage management method for DDS is proposed, including the following steps:

[0008] Obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence.

[0009] During data communication, the protocol engine triggers corresponding persistence operation tasks at preset times. The corresponding persistence operation tasks triggered at preset times include at least the following: triggering persistent writing after the write end successfully sends data, triggering historical data recovery when the write end is created, triggering linked deletion when the write end's memory historical cache evicts data, triggering reading consumption progress when the write end and the read end successfully match, and triggering recording the latest consumption progress when the read end successfully receives data.

[0010] Based on the persistence level, persistent operation tasks are dispatched to the storage backend through an operation dispatch table, and the storage backend performs persistent storage. The operation dispatch table defines a unified persistent operation interface in the form of function pointers. The storage backend includes at least transient-level persistent backends and persistent-level persistent backends, and each storage backend registers its own implementation path in the operation dispatch table.

[0011] In one implementation, a condition check is performed before the persistent operation task is dispatched to the storage backend, including at least the reliability quality of service policy being reliable and the entity name being non-empty.

[0012] In one implementation, for a persistence operation task assigned to a transient-level persistence backend, the implementation path of the transient-level persistence backend is to communicate with the persistence service process via a socket through the persistence service's private protocol.

[0013] The persistent service private protocol uses a binary message format. The message content includes at least a message header and a payload. The message header includes at least a fixed value for protocol identification and verification, the protocol version number, the message type, the message sequence number, and the number of bytes of payload data. The message type includes at least insert write history, query write history, delete write history, query read progress, and update read progress, which are mapped to the persistent operation interfaces defined in the operation dispatch table.

[0014] In one implementation, for a persistent write task assigned to a transient-level persistent backend, after the write end publishes the data, it constructs a request message for inserting the history type of the write end, uses the data sample as the payload, and pushes it to the persistent service process through a socket; after the server verifies the fixed value of the message header and the protocol version number field, it stores the data sample in the memory of the persistent service process.

[0015] For historical data recovery tasks assigned to transient-level persistent backends, when the writer process restarts and creates a writer with the same entity name as before the restart, a request message is constructed to query the writer's history type, querying the persistent service process for all historical data under that entity name. The server responds by returning all historical data samples. After receiving the response, the client calls a preset historical data query callback function to re-inject the historical data samples into the writer's memory history cache. When a new reader matches the writer, the protocol engine delivers the data in the writer's memory history cache to the new reader.

[0016] For linked deletion tasks assigned to transient-level persistent backends, when the memory history cache of the write end evicts data, a message of deleting the history type of the write end is constructed, and the entity name and sequence number of the write end are sent to the persistent service process as the payload. The server deletes the corresponding record from the memory of the persistent service process.

[0017] For read consumption progress tasks assigned to transient-level persistent backends, when the read-end process restarts and re-matches with the write-end, a request message of the read-end progress type is constructed. The query is performed in the memory of the persistent service process based on the write-end entity name and the read-end entity name. If the last confirmed sequence number is found, the historical data after the last confirmed sequence number is delivered from the write-end's memory history cache, skipping the part already confirmed by the read-end. If the last confirmed sequence number is not found, all historical data in the write-end's memory history cache is delivered.

[0018] For tasks that record the latest consumption progress and are assigned to the transient-level persistent backend, whenever the reader confirms that it has received data, a request message to update the reader's progress type is constructed, and the latest confirmation sequence number of the reader to the current writer is written to the persistent service process.

[0019] In one implementation, for persistence operation tasks dispatched to the persistence level persistence backend, the implementation path of the persistence level persistence backend is to access the local SQLite database through a pre-compiled SQL statement call.

[0020] The pre-compiled SQL statements include at least querying the writer history, inserting or updating the writer history, deleting a specified writer history, querying the last confirmation sequence number of the reader, and updating the confirmation sequence number of the reader, which are respectively mapped to the persistent operation interfaces defined in the operation dispatch table;

[0021] An SQLite database contains at least a writer history table and a reader status tracking table. The writer history table contains the writer entity name, data sequence number, serialized data payload, data source timestamp, and a composite primary key consisting of the writer entity name and data sequence number. The reader status tracking table contains the reader entity name, the corresponding writer entity name, the last confirmed sequence number, and a composite primary key consisting of the reader entity name and the corresponding writer entity name.

[0022] In one implementation, for persistent write tasks assigned to persistent level backends, after the write end publishes data, data samples are stored in the SQLite database through pre-compiled insert or update write end history statements.

[0023] For historical data recovery tasks assigned to persistent backends, when the write process restarts and creates a write with the same entity name as before the restart, it accesses the SQLite database through a pre-compiled query statement to retrieve all historical data under that entity name. The server responds by returning a sample of all historical data. After receiving the response, the client calls a preset historical data query callback function to re-inject the historical data sample into the write's in-memory history cache. When a new read matches the write, the protocol engine delivers the data from the write's in-memory history cache to the new read.

[0024] For linked deletion tasks assigned to persistent backends, when the memory history cache of the write end evicts data, the entity name and sequence number of the evicted data are obtained, and the record with the primary key of the entity name and sequence number is deleted from the write end history data table through the pre-compiled delete statement specifying the write end history.

[0025] For read consumption progress tasks assigned to persistent backends, when the read process restarts and re-matches with the write process, a pre-compiled query statement for the last confirmed sequence number is used to retrieve the last confirmed sequence number from the read status tracking table by composite primary key. If a record is found, historical data after the last confirmed sequence number is delivered from the write's memory history cache, skipping the portion already confirmed by the read. If no record is found, all historical data in the write's memory history cache is delivered.

[0026] For tasks that record the latest consumption progress and are assigned to persistent backends, whenever the reader confirms that it has received data, a pre-compiled statement to update the reader confirmation sequence number is called to write the latest confirmation sequence number of the reader to the current writer into the reader status tracking table.

[0027] In one implementation, the transient-level persistence backend and the persistent-level persistence backend use a unified data model to store data. The data model includes at least historical data records on the write end and consumption status records on the read end.

[0028] Each record in the historical data record of the write end represents a data sample published by the write end, including the write end entity name, sequence number, serialized data payload pointer, payload size, and source timestamp when the data was written. The write end entity name and sequence number serve as a combined unique identifier.

[0029] Each record in the read-end consumption status record represents the consumption progress of a read-write pair, including the read-end entity name, the corresponding write-end entity name, and the sequence number of the last confirmation by the read-end. The read-end entity name and the corresponding write-end entity name serve as a combined unique identifier.

[0030] In one implementation, when a new storage backend is added, during the initialization phase, the new storage backend registers its implementation path in the operation dispatch table.

[0031] Secondly, a DDS persistent data storage management system is proposed, including:

[0032] The application layer module is used to obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence.

[0033] The protocol engine module is used to trigger corresponding persistent operation tasks at preset times during data communication. The corresponding persistent operation tasks triggered at preset times include at least the following: triggering persistent writing after the write end successfully sends data, triggering historical data recovery when the write end is created, triggering linked deletion when the write end's memory historical cache evicts data, triggering reading consumption progress when the write end and the read end successfully match, and triggering recording the latest consumption progress when the read end successfully receives data.

[0034] The unified persistence interface layer module is used to dispatch persistence operation tasks to the storage backend based on the persistence level through an operation dispatch table; the operation dispatch table defines a unified persistence operation interface in the form of function pointers.

[0035] The storage backend module is used to perform persistent storage through the storage backend; the storage backend module includes at least transient level persistent backend units and persistent level persistent backend units, and each storage backend registers its own implementation path in the operation dispatch table.

[0036] In one implementation, the transient-level persistence backend unit is used for persistent operation tasks assigned to the transient-level persistence backend. The implementation path of the transient-level persistence backend is to communicate with the persistence service process via a socket through a private persistence service protocol.

[0037] The persistent-level persistence backend unit is used for persistent write tasks dispatched to the persistent-level persistence backend. After the write end publishes data, the data sample is stored in the SQLite database through pre-compiled insert or update write end history statements.

[0038] The beneficial effects of this invention are:

[0039] 1. The two persistence levels, Transient and Persistent, are managed based on a unified architecture, logically sharing the same data model, unified operation interface, and callback mechanism, differing only in the implementation path of the storage backend. Furthermore, for core processes such as write-side data persistence, historical data recovery, read-side consumption progress tracking, and WHC-linked cleanup, the same logical control framework is reused across different levels, effectively reducing the complexity of system development and maintenance, and ensuring a high degree of consistency in communication behavior across different persistence levels from the architectural root.

[0040] 2. By introducing a unified persistence interface layer, the direct coupling between the upper-layer protocol engine and the lower-layer specific storage backend is severed in the control flow. Callers uniformly execute persistence operations through standard function pointer signatures in the operation dispatch table, shielding the differences between the underlying physical storage media in network communication and local file read / write. When introducing new storage backends, only the standard operation interface needs to be implemented according to the contract and dynamically registered; no modifications to the existing write path, clear path, and restore path code are required. This architecture naturally supports parallel routing of multiple backends and simulated backend testing, reserving standardized expansion space for the introduction of new storage media such as remote databases or distributed caches.

[0041] 3. The implementation path of the transient-level persistence backend is through the DurabilityService Protocol (DURS), which communicates with the persistence service process via sockets. The DURS protocol has a concise message header, with significantly less message overhead than the standard RTPS protocol. Message types cover all persistence operation requirements, and the message payload uses CDR encoding to ensure cross-platform compatibility. This DURS protocol design ensures highly reliable data transmission while reducing the consumption of computing power and network bandwidth, making it suitable for resource-constrained scenarios such as embedded systems and automotive applications.

[0042] 4. The persistent backend is implemented by accessing the local SQLite database through pre-compiled SQL statements. The SQLite embedded database eliminates the need to deploy and maintain an external database server, making maintenance more convenient. The SQLite database includes a historical data table for the write end and a status tracking table for the read end. The composite primary key in the historical data table ensures that data from the same write end is stored in sequence, facilitating range queries and sequential recovery, and guaranteeing that historical data is not lost after the write end restarts. The status tracking table for the read end independently tracks the consumption progress of each read-write relationship, ensuring that consumption progress is not rolled back after the read end restarts. The combined use of these two tables ensures precise breakpoint resumption even if either or both ends restart simultaneously.

[0043] 5. When the Writer History Cache (WHC) on the write end evicts old samples, the corresponding record in the persistent storage is also cleared. This ensures that the persistent layer always maintains a consistent cache range with memory, preventing old data from accumulating indefinitely in the persistent layer, and ensuring that the data is completely consistent with the QoS policy configured in the application when the write end restarts and recovers. Attached Figure Description

[0044] The invention will now be further described with reference to the accompanying drawings.

[0045] Figure 1 This is an architecture diagram of the DDS-based persistent data storage management method according to an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the structure of the private protocol message header of the persistent service based on the DDS persistent data storage management method in an embodiment of the present invention. Detailed Implementation

[0047] like Figure 1 As shown in the figure, this invention proposes a DDS persistent data storage management method, including the following steps:

[0048] S1, obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence.

[0049] Specifically, applications publish and subscribe to data through the DDS standard API and declare the required persistence level through a Durability QoS policy, without needing to be aware of the underlying implementation. DDS controls data persistence behavior through the Durability QoS policy, where the Transient level requires data to be managed by a separate persistence service, and the Persistent level requires data to be persisted to non-volatile storage media. The DDS specification only defines semantic requirements but does not specify concrete implementation methods. Although the persistence levels differ, their data management requirements are highly structurally consistent, mainly including the following three aspects:

[0050] (1) Data storage. After completing the RTPS (standard DDS protocol) network transmission, the write end must synchronously write the data sample to persistent storage so that the data can still be accessed by subsequent read ends after the write end is destroyed or the system is restarted. Therefore, both levels need to provide the operation of inserting historical data into the write end.

[0051] (2) Data Recovery. The core value of persistent storage lies in supporting the recovery of historical data after the writer restarts. When the writer process exits and restarts, the system must query all historical data previously saved by the writer from persistent storage and re-inject it into the memory history cache (WHC), restoring the writer to its state before the restart. Subsequently, when a reader (including a late-joiner) matches the writer, the historical data in the WHC will be automatically delivered to the reader. Therefore, both levels need to provide the operation of querying the writer's historical data. In addition, to help determine which sequence number should be used for delivery, two operations are also needed: querying the reader's consumption progress and updating the reader's consumption progress.

[0052] (3) Data cleanup. When a data record is evicted due to the Quality of Service (QoS) policy, the Write History Cache (WHC) on the write end will clear the record. At this time, the corresponding record in persistent storage should also be deleted synchronously to ensure that the data in WHC and persistent storage are consistent. Therefore, both levels need to provide the operation of deleting historical data on the write end, which is linked to the confirmation and clearing process of WHC.

[0053] Based on the above analysis, transient persistence and persistent persistence have completely consistent structures in terms of stored content (entity identifier, sequence number, serialized payload, source timestamp), operational semantics (five operations), recovery requirements, cleanup mechanisms, and persistence determination logic. Therefore, the core design idea of ​​this embodiment is to build a unified data model and operation interface for transient persistence and persistent persistence, so that the two differ only in the storage backend (process memory or embedded database), while maintaining consistency in data management semantics.

[0054] S2, during data communication, triggers corresponding persistence operation tasks at preset times through the protocol engine; the corresponding persistence operation tasks triggered at preset times include at least the following: triggering persistent writing after the write end successfully sends data, triggering historical data recovery when the write end is created, triggering linked deletion when the write end's memory historical cache evicts data, triggering reading consumption progress when the write end and the read end successfully match, and triggering recording the latest consumption progress when the read end successfully receives data.

[0055] Specifically, the protocol engine is responsible for data serialization (CDR encoding), memory history cache (WHC) management, and real-time publish-subscribe wire protocol (RTPS) network transmission.

[0056] S3, based on persistence level, dispatches persistence operation tasks to the storage backend through an operation dispatch table; the operation dispatch table defines a unified persistence operation interface in the form of function pointers;

[0057] By abstracting persistent operations into a unified interface specification, the upper-layer protocol engine and the lower-layer storage backend are completely decoupled, and the same process framework can drive different storage backends without any modification.

[0058] In one implementation, when a new storage backend is added, its implementation path is registered in the operation dispatch table during the initialization phase. The backend implementation can be developed and replaced independently.

[0059] Specifically, the operation dispatch table defines a unified interface in the form of a function pointer table, and each storage backend registers its own implementation during initialization. Upper-layer callers only need to hold a pointer to the dispatch table and call functions using a unified function signature, without needing to know whether the underlying communication is network communication or a local database. Five unified operations cover the complete lifecycle of persistent data management—data insertion (insert_writer_history), data retrieval (query_writer_history), data cleanup (remove_writer_history), progress query (query_reader_seq), and progress recording (update_reader_seq). The parameter signatures and return value semantics for these five operations are completely consistent between the two backends; only the underlying implementation differs.

[0060] The unified operation interface is defined as follows:

[0061] / **

[0062] * Persistent storage backend operation interface (operation dispatch table)

[0063] *The transient-level persistence backend and the persistent-level persistence backend each provide a specific implementation of this interface:

[0064] * - TRANSIENT implementation: Communicates with the Durability Service process via the DURS protocol.

[0065] * - PERSISTENT implementation: Access the local SQLite database via SQL calls

[0066] * Callers operate uniformly through this interface, without needing to know whether the underlying communication is network communication or a local database.

[0067] /

[0068] typedef struct ddsi_durability_backend {

[0069] / **

[0070] / **

[0071] * Insert historical data at the write end

[0072] * Called after each successful data transmission on the writing end to synchronously write data samples to persistent storage.

[0073] * If (entity_name, seq_num) already exists, the old value is overwritten (idempotent semantics).

[0074] /

[0075] int (*insert_writer_history)(

[0076] void *backend_data,

[0077] const char *entity_name,

[0078] uint64_t seq_num,

[0079] const void *payload,

[0080] uint32_t payload_size,

[0081] int64_t source_timestamp );

[0083] / **

[0084] * Query historical data on the write end

[0085] * Called upon creation after a restart of the write end to restore historical data from persistent storage.

[0086] * Retrieve all historical records by entity_name, and call the callback function one by one in ascending order of sequence number.

[0087] The callback function re-injects each record into the WHC, restoring the Writer to its state before the restart.

[0088] * The iteration will terminate early if the callback returns a non-zero value.

[0089] /

[0090] int (*query_writer_history)(

[0091] void *backend_data,

[0092] const char *entity_name,

[0093] ddsi_writer_history_callback_t callback,

[0094] void *userdata );

[0096] The historical data query callback interface is defined as follows:

[0097] / **

[0098] * Historical data query callback function types

[0099] When querying the write history, the storage backend calls this callback sequentially, one record at a time, in ascending order of sequence number.

[0100] * Return historical data to the caller for processing (e.g., re-inject WHC or deliver it to the application).

[0101] * The callback design avoids loading all history into memory at once, making it suitable for scenarios involving the recovery of large amounts of historical data.

[0102] * @param seq_num Data sequence number

[0103] * @param payload Serializable data payload pointer

[0104] * @param payload_size Payload size (bytes)

[0105] * @param source_timestamp Data source timestamp

[0106] * @param userdata Context pointer passed in by the caller

[0107] * @return 0 Continue iterating to the next item; non-zero value terminates the iteration.

[0108] /

[0109] typedef int (*ddsi_writer_history_callback_t)(

[0110] uint64_t seq_num,

[0111] const void *payload,

[0112] uint32_t payload_size,

[0113] int64_t source_timestamp,

[0114] void *userdata );

[0116] / **

[0117] * Delete the history of the specified writer end

[0118] This operation is invoked in conjunction with the data in the WHC when it is evicted and cleaned up.

[0119] * Clear the corresponding record in persistent storage to ensure data consistency.

[0120] /

[0121] int (*remove_writer_history)(

[0122] void *backend_data,

[0123] const char *entity_name,

[0124] uint64_t seq_num );

[0126] / **

[0127] * Query the final confirmed serial number

[0128] * Called during the endpoint matching phase to retrieve the sequence number last consumed by the remote reader.

[0129] * Used to determine from which sequence number historical data should be delivered, avoiding duplicate transmissions.

[0130] * If there are no records (first connection), max_seq returns 0.

[0131] /

[0132] int (*query_reader_seq)(

[0133] void *backend_data,

[0134] const char *reader_entity_name,

[0135] const char *writer_entity_name,

[0136] uint64_t *max_seq );

[0138] / **

[0139] * Update the serial number on the reader

[0140] * During normal operation of the reading end, this operation is continuously called to record the latest confirmed serial number.

[0141] * If (reader_entity_name, writer_entity_name) does not exist, it will be created; otherwise, it will be overwritten.

[0142] /

[0143] int (*update_reader_seq)(

[0144] void *backend_data,

[0145] const char *reader_entity_name,

[0146] const char *writer_entity_name,

[0147] uint64_t seq_num );

[0149] In one implementation, a condition check is performed before the persistent operation task is dispatched to the storage backend, including at least the reliability quality of service policy being reliable and the entity name being non-empty.

[0150] Specifically,

[0151] / **

[0152] * Persistence determination

[0153] * Determine whether persistence should be performed by this backend based on the entity's QoS configuration.

[0154] * The criteria for judgment are a triple AND: Durability level matches + RELIABLE + Entity Name is not empty.

[0155] /

[0156] bool (*should_persist)(

[0157] const struct dds_qos *qos );

[0159] void *backend_data; / * Backend private data (such as database handles, Socket connections, etc.) * /

[0160] } ddsi_durability_backend_t;

[0161] S4, the storage backend performs persistent storage. The storage backend includes at least a transient level persistent backend and a persistent level persistent backend. Each storage backend registers its own implementation path in the operation dispatch table.

[0162] During system initialization, specific backend implementations are instantiated and registered based on compilation configuration and runtime requirements:

[0163] / **

[0164] * Backend registration example (dDS domain initialization)

[0165] /

[0166] void ddsi_durability_init(struct ddsi_domaingv *gv)

[0167] {

[0168] / * TRANSIENT backend: Initializes DURS Socket connection and populates operation dispatch table * /

[0169] if (gv->config.durability & TRANSIENT) {

[0170] ddsi_durability_backend_t *backend = gv->durability_backend;

[0171] backend->insert_writer_history = durs_insert_writer_history;

[0172] backend->query_writer_history = durs_query_writer_history;

[0173] backend->remove_writer_history = durs_remove_writer_history;

[0174] backend->query_reader_seq = durs_query_reader_seq;

[0175] backend->update_reader_seq = durs_update_reader_seq;

[0176] backend->should_persist = durs_should_persist;

[0177] backend->backend_data = durs_connection_init(gv);

[0178] }

[0179] / * PERSISTENT backend: Initialize the SQLite database and populate the operation dispatch table * /

[0180] if (gv->config.durability & PERSISTENT) {

[0181] ddsi_durability_backend_t *backend = gv->durability_backend;

[0182] backend->insert_writer_history = sqlite_insert_writer_history;

[0183] backend->query_writer_history = sqlite_query_writer_history;

[0184] backend->remove_writer_history = sqlite_remove_writer_history;

[0185] backend->query_reader_seq = sqlite_query_reader_seq;

[0186] backend->update_reader_seq = sqlite_update_reader_seq;

[0187] backend->should_persist = sqlite_should_persist;

[0188] backend->backend_data = sqlite_db_init(gv);

[0189] }

[0190] }

[0191] The caller doesn't need to worry about the backend type; it can directly call the registered backend interface (taking data writing as an example):

[0192] void persist_after_transmit(

[0193] struct ddsi_domaingv *gv,

[0194] const struct dds_qos *qos,

[0195] const char *entity_name,

[0196] uint64_t seq_num,

[0197] const void *payload,

[0198] uint32_t payload_size,

[0199] int64_t timestamp)

[0200] {

[0201] ddsi_durability_backend_t *backend = gv->durability_backend;

[0202] if (backend != NULL && backend->should_persist != NULL && backend->should_persist(qos)) {

[0203] backend->insert_writer_history(

[0204] backend->backend_data, entity_name, seq_num,

[0205] payload, payload_size, timestamp);

[0206] }

[0207] }

[0208] In one implementation, for a persistence operation task assigned to a transient-level persistence backend, the implementation path of the transient-level persistence backend is to communicate with the persistence service process via a socket through the persistence service's private protocol.

[0209] The persistent service private protocol uses a binary message format. The message content includes at least a message header and a payload. The message header includes at least a fixed value for protocol identification and verification, the protocol version number, the message type, the message sequence number, and the number of bytes of payload data. The message type includes at least insert write history, query write history, delete write history, query read progress, and update read progress, which are mapped to the persistent operation interfaces defined in the operation dispatch table.

[0210] Specifically, a proprietary protocol was designed instead of reusing the standard RTPS protocol because the communication pattern in persistent scenarios is a simple request-response (insert, query, or delete), while the RTPS protocol is designed for complex publish-subscribe scenarios. Its message header (at least 20 bytes RTPS Header + Submessage Header), discovery process, and reliability handshake mechanism are unnecessary overheads for persistent communication. For transient-level persistent backends, data is stored in the memory of the Durability Service process. After the write end is destroyed, the data is still retained by the service and can be retrieved by subsequently added read ends, but it does not cross system restarts.

[0211] In this embodiment, the message header is fixed at 12 bytes, and its structure is as follows: Figure 2 As shown, Figure 2 The first row of numbers 0 to 3 is used to identify the byte offset within each 4-byte data line, where "0" represents the 1st byte, "1" represents the 2nd byte, "2" represents the 3rd byte, and "3" represents the 4th byte. The numbers 0 to 7 below each byte represent the bit sequence number within that byte. Detailed explanations of each field are as follows:

[0212] (a) Magic (4 bytes): Fixed value 0x44555253 (ASCII "DURS"), used for protocol identification and verification. The receiver must verify this field to filter illegal messages;

[0213] (b) Version (1 byte): Protocol version number, currently 1, reserving version negotiation capability for future protocol extensions;

[0214] (c) Msg Type (1 byte): Message type, indicating the behavior of the current message;

[0215] (d) Msg ID (4 bytes): Message sequence number, generated by a global atomic counter, monotonically increasing, used for log tracing and request-response correlation; Figure 2 In the text, due to the cross-line layout, it is split into two 2-byte segments for display. The first half is identified as Msg ID, and the second half is identified as Msg ID (continued).

[0216] (e) Payload Size (2 bytes): The number of bytes of subsequent payload data.

[0217] The DURS protocol defines eight message types, divided into client requests (5 types) and server responses (3 types).

[0218] Client request messages (type codes 1-5):

[0219] Table 1 Message Type Table for Client Request Messages

[0220]

[0221] Server response message (type code 128-131):

[0222] Table 2 Message Type Table for Server Response Messages

[0223]

[0224] The five types of client request messages correspond one-to-one with the five unified operations defined above. The request type code uses the low-order bits (1-5), and the response type code uses the high-order bits (128+). The direction of the request and response can be distinguished by the highest-order bit.

[0225] The payload of the ACK response message carries a 32-bit error code, defined as follows:

[0226] Table 3 Error Code Types for ACK Acknowledgment Messages

[0227]

[0228] After receiving the ACK response, the client must check if the error code is SUCCESS; otherwise, the operation is considered a failure.

[0229] The fields in the message payload use the following encoding rules: string fields use CDR encoding (4-byte length prefix + string content + 4-byte alignment padding), integer fields are stored directly in native byte order (8 bytes for uint64, 4 bytes for uint32), and binary data is embedded using a length prefix. The entire payload is aligned to 4-byte boundaries. The complete message structure is illustrated using the example of a QUERY_WRITER_HISTORY request-response for restoring historical data after a writer restart:

[0230] The request message structure is shown in the table below (to query all historical data where entity_name is "sensor / lidar"):

[0231] Table 4. Request message structure table for querying the history of the writing end.

[0232]

[0233] The response message structure is shown in the table below (returning two history records):

[0234] Table 5. Response message structure for querying the history of the writing end.

[0235]

[0236] The payload structure for other message types follows the same encoding rules, and the field combinations for each type are as follows:

[0237] An INSERT_WRITER_HISTORY request contains entity_name + seq_num + timestamp + payload; a DELETE_WRITER_HISTORY request contains entity_name + seq_num; a QUERY_READER_MAX_SEQ request contains reader_entity_name + writer_entity_name; and an UPDATE_READER_SEQ request appends seq_num to these. An ACK response contains only a 4-byte error_code, and a SEQ_RESPONSE response contains only an 8-byte max_seq_num.

[0238] In one implementation,

[0239] For persistent write tasks dispatched to transient-level persistent backends, after the write end publishes data, it constructs a request message of the type to insert the write end's history, uses a data sample as the payload, and pushes it to the persistent service process via a socket. The server verifies the fixed values ​​in the message header and the protocol version number field, then stores the data sample in the persistent service process's memory. Specifically, the server also returns an ACK response. This stage ensures that each successfully sent data item retains a copy in the Durability Service of the persistent service process, so that the data remains held by the service process even if the write end process exits.

[0240] For historical data recovery tasks assigned to transient-level persistent backends, when the writer process restarts and creates a writer with the same entity name as before the restart, a request message is constructed to query the writer's history type, retrieving all historical data under that entity name from the persistent service process. The server responds by returning all historical data samples. After receiving the response, the client calls a preset historical data query callback function to re-inject the historical data samples into the writer's in-memory history cache. When a new reader matches the writer, the protocol engine delivers the data from the writer's in-memory history cache to the new reader. Specifically, after retrieving the matching records, the server returns all historical samples with a DATA_RESPONSE response. After receiving the response, the client parses and generates records, calling the recovery callback function one by one to re-inject the historical data into the WHC. After recovery, the state of the WHC is equivalent to that before the writer restarts.

[0241] For linked deletion tasks assigned to transient-level persistent backends, when the memory history cache of the write end evicts data, a message of deleting the history type of the write end is constructed, and the entity name and sequence number of the write end are sent to the persistent service process as the payload. The server deletes the corresponding record from the memory of the persistent service process.

[0242] The cache capacity of WHC is determined by the user-configured QoS policy. When History QoS is set to KEEP_LAST(depth), WHC retains at most depth samples for each data instance; when a new sample is written, causing the cache to exceed the depth limit, the oldest sample will be evicted and deleted. Similarly, the max_samples and max_samples_per_instance of Resource Limits QoS will also trigger the eviction of old samples. After a sample is deleted in WHC, a copy of that sample is still retained in persistent storage. If not cleared synchronously, the data in the persistent layer will be inconsistent with WHC—old data that has been evicted by WHC will continue to accumulate in the persistent layer, not only wasting storage space but also causing the evicted old data to be re-injected into WHC when the write end restarts and recovers, violating the caching policy configured by the application through QoS. To solve this problem, this embodiment embeds a linkage clearing callback of the persistent layer in the node deletion process of WHC. When WHC deletes a cache node due to QoS policy constraints, the system extracts the entity_name and seq_num of the node and calls the corresponding operation of the corresponding backend according to the Durability QoS level of the write end. The linked deletion task ensures that when the write end restarts and recovers, the historical data retrieved from the persistence layer completely matches the QoS configuration caching policy. A corresponding linked deletion task also exists for the persistent backend.

[0243] For read consumption progress tasks assigned to transient-level persistent backends, when the read-end process restarts and re-matches with the write-end, a request message of the read-end progress type is constructed. The query is performed in the memory of the persistent service process based on the write-end entity name and the read-end entity name. If the last confirmed sequence number is found, the historical data after the last confirmed sequence number is delivered from the write-end's memory history cache, skipping the part already confirmed by the read-end. If the last confirmed sequence number is not found, all historical data in the write-end's memory history cache is delivered.

[0244] For tasks that record the latest consumption progress and are assigned to the transient-level persistent backend, whenever the reader confirms that it has received data, a request message to update the reader's progress type is constructed, and the latest confirmation sequence number of the reader to the current writer is written to the persistent service process.

[0245] In one implementation, for persistence operation tasks assigned to the persistent level persistence backend, the implementation path of the persistent level persistence backend is to access the local SQLite database through a pre-compiled SQL statement call; specifically, the database file is named {process name}_{domain ID}.db and stored on the local disk, which can be recovered across system restarts and power outages.

[0246] The pre-compiled SQL statements include at least querying the writer history, inserting or updating the writer history, deleting a specified writer history, querying the last confirmation sequence number of the reader, and updating the confirmation sequence number of the reader, which are respectively mapped to the persistent operation interfaces defined in the operation dispatch table;

[0247] Specifically, five SQL statements are pre-compiled during initialization:

[0248] -- Operation 1: Query the write history (sorted by sequence number in ascending order)

[0249] SELECT seq_num, payload, source_timestamp

[0250] FROM writers_histories

[0251] WHERE entity_name = ?

[0252] ORDER BY seq_num;

[0253] -- Operation 2: Insert or update writer history

[0254] INSERT OR REPLACE INTO writers_histories

[0255] (entity_name, seq_num, payload, source_timestamp)

[0256] VALUES (?, ?, ?, ?);

[0257] -- Operation 3: Delete the history of the specified writer end

[0258] DELETE FROM writers_histories

[0259] WHERE entity_name = ? AND seq_num = ?;

[0260] -- Operation 4: Query the last confirmed serial number on the reading end

[0261] SELECT last_seq_num FROM readers

[0262] WHERE reader_entity_name = ? AND writer_entity_name = ?;

[0263] -- Operation 5: Update the serial number on the reader

[0264] INSERT OR REPLACE INTO readers

[0265] (reader_entity_name, writer_entity_name, last_seq_num)

[0266] VALUES (?, ?, ?);

[0267] The prepared statements use the persistent prepared statement flag (SQLITE_PREPARE_PERSISTENT), indicating that SQLite statements will be executed repeatedly. SQLite will then perform long-term caching optimization on its internal data structures, avoiding re-parsing and recompiling the SQL text each time it is executed. Operations 2 and 5 both use INSERT OR REPLACE semantics: if the primary key already exists, the old value is overwritten; otherwise, a new row is inserted. This avoids the two-step operation of querying first and then deciding on the INSERT / UPDATE, reducing one database access.

[0268] An SQLite database contains at least a writer history table and a reader status tracking table. The writer history table contains the writer entity name, data sequence number, serialized data payload, data source timestamp, and a composite primary key consisting of the writer entity name and data sequence number. The reader status tracking table contains the reader entity name, the corresponding writer entity name, the last confirmed sequence number, and a composite primary key consisting of the reader entity name and the corresponding writer entity name.

[0269] Specifically, the structures of the writer-side historical data table `writers_histories` and the reader-side status tracking table `readers` are as follows:

[0270] CREATE TABLE IF NOT EXISTS writers_histories(

[0271] entity_name TEXT, -- The name of the entity being written (unique identifier)

[0272] seq_num INTEGER -- Data sequence number (>0)

[0273] CHECK(seq_num > 0),

[0274] payload BLOB, -- serialized data payload

[0275] source_timestamp INTEGER, -- Data source timestamp

[0276] PRIMARY KEY(entity_name, seq_num) -- Composite primary key

[0277] WITHOUT ROWID; -- No RowID optimization, reducing storage overhead.

[0278] CREATE TABLE IF NOT EXISTS readers(

[0279] reader_entity_name TEXT, -- The name of the entity being read

[0280] writer_entity_name TEXT, -- Corresponding entity name on the writing end

[0281] last_seq_num INTEGER -- The last confirmed sequence number

[0282] CHECK(last_seq_num > 0),

[0283] PRIMARY KEY(reader_entity_name, -- composite primary key)

[0284] writer_entity_name)

[0285] WITHOUT ROWID;

[0286] In one implementation, for persistent write tasks assigned to persistent level backends, after the write end publishes data, data samples are stored in the SQLite database through pre-compiled insert or update write end history statements.

[0287] For historical data recovery tasks assigned to persistent backends, when the write process restarts and creates a write with the same entity name as before the restart, it accesses the SQLite database through a pre-compiled query statement to retrieve all historical data under that entity name. The server responds by returning a sample of all historical data. After receiving the response, the client calls a preset historical data query callback function to re-inject the historical data sample into the write's in-memory history cache. When a new read matches the write, the protocol engine delivers the data from the write's in-memory history cache to the new read.

[0288] For linked deletion tasks assigned to persistent backends, when the memory history cache of the write end evicts data, the entity name and sequence number of the evicted data are obtained, and the record with the primary key of the entity name and sequence number is deleted from the write end history data table through the pre-compiled delete statement specifying the write end history.

[0289] For read consumption progress tasks assigned to persistent backends, when the read process restarts and re-matches with the write process, a pre-compiled query statement for the last confirmed sequence number is used to retrieve the last confirmed sequence number from the read status tracking table by composite primary key. If a record is found, historical data after the last confirmed sequence number is delivered from the write's memory history cache, skipping the portion already confirmed by the read. If no record is found, all historical data in the write's memory history cache is delivered. Specifically, if a record is found (last_seq_num=N), the write process delivers subsequent historical data starting from sequence number N+1 in the WHC.

[0290] For tasks that record the latest consumption progress and are assigned to persistent backends, whenever the reader confirms that it has received data, a pre-compiled statement to update the reader confirmation sequence number is called to write the latest confirmation sequence number of the reader to the current writer into the reader status tracking table.

[0291] In fact, the process of writing and restoring historical data on the writing end is exactly the same for both persistence levels, only the underlying calls are different (DURS protocol requests or SQL statement execution); the process of updating and restoring the consumption progress on the reading end is also exactly the same for both levels.

[0292] In one implementation, the transient-level persistence backend and the persistent-level persistence backend use a unified data model to store data. The data model includes at least historical data records on the write end and consumption status records on the read end.

[0293] Each record in the historical data record of the write end represents a data sample published by the write end, including the write end entity name, sequence number, serialized data payload pointer, payload size, and source timestamp when the data was written. The write end entity name and sequence number serve as a combined unique identifier.

[0294] Each record in the read-end consumption status record represents the consumption progress of a read-write pair, including the read-end entity name, the corresponding write-end entity name, and the sequence number of the last confirmation by the read-end. The read-end entity name and the corresponding write-end entity name serve as a combined unique identifier.

[0295] Specifically, the data model is defined as follows:

[0296] / **

[0297] * Historical data records on the write end

[0298] Each record represents a data sample published by the writer and is uniquely identified by the entity name (entity_name) and sequence number (seq_num).

[0299] * The transient-level persistence backend maintains a collection of this structure in the memory of the persistence service process;

[0300] *The persistent level backend maps it to a row in the historical data table on the write end.

[0301] /

[0302] typedef struct ddsi_durability_history_entry {

[0303] const char *entity_name; / * Entity name at the writing end, a globally unique identifier within the DDS domain * /

[0304] uint64_t seq_num; / * RTPS sequence number, monotonically increasing, indicating the data order * /

[0305] const void *payload; / * Pointer to the serialized data payload (CDR format) * /

[0306] uint32_t payload_size; / * Payload size (bytes) * /

[0307] int64_t source_timestamp; / * Source timestamp (nanosecond precision) when data was written * /

[0308] } ddsi_durability_history_entry_t;

[0309] / **

[0310] * Reader consumption status record

[0311] * Tracks the consumption progress of a read-write pair, uniquely identified by the combined name of the read entity and the corresponding write entity (reader_entity_name + writer_entity_name).

[0312] * Used for endpoint matching to determine which sequence number should be used to start delivering historical data, avoiding duplicate transmissions.

[0313] /

[0314] typedef struct ddsi_durability_reader_state {

[0315] const char *reader_entity_name; / * Name of the entity being read * /

[0316] const char *writer_entity_name; / * Corresponding entity name on the writing end * /

[0317] uint64_t last_seq_num; / * The last confirmed sequence number of this reader * /

[0318] } ddsi_durability_reader_state_t;

[0319] The entity name (entity_name) is configured by the application through the Entity Name QoS policy and serves as a unique identifier for the write end in persistent storage. The entity name (entity_name) remains unchanged across processes and restarts, forming the basis for persistent storage addressing. The sequence number (seq_num) uses a 64-bit sequence number from the RTPS protocol, monotonically increasing from 1 to ensure total order of data. The composite key (entity_name, seq_num) is globally unique throughout the persistent storage. The read-end state model allows each read end to independently track its consumption progress across multiple write ends and also supports concurrent consumption of the same write end by multiple read ends.

[0320] This embodiment also proposes a DDS persistent data storage management system, including:

[0321] The application layer module is used to obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence.

[0322] The protocol engine module is used to trigger corresponding persistent operation tasks at preset times during data communication. The corresponding persistent operation tasks triggered at preset times include at least the following: triggering persistent writing after the write end successfully sends data, triggering historical data recovery when the write end is created, triggering linked deletion when the write end's memory historical cache evicts data, triggering reading consumption progress when the write end and the read end successfully match, and triggering recording the latest consumption progress when the read end successfully receives data.

[0323] The unified persistence interface layer module is used to dispatch persistence operation tasks to the storage backend based on the persistence level through an operation dispatch table; the operation dispatch table defines a unified persistence operation interface in the form of function pointers.

[0324] The storage backend module is used to perform persistent storage through the storage backend; the storage backend module includes at least transient level persistent backend units and persistent level persistent backend units, and each storage backend registers its own implementation path in the operation dispatch table.

[0325] In one implementation, a transient-level persistence backend unit is used for persistence operation tasks assigned to the transient-level persistence backend. The implementation path of the transient-level persistence backend is to communicate with the persistence service process via a socket through a private persistence service protocol. A persistent-level persistence backend unit is used for persistent write tasks assigned to the persistent-level persistence backend. After the write end publishes data, it stores the data sample into the SQLite database through pre-compiled insert or update write end history statements.

Claims

1. A DDS persistent data storage management method, characterized in that, Includes the following steps: Obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence. During data communication, the protocol engine triggers corresponding persistent operation tasks at preset times; The preset timing triggers the corresponding persistence operation tasks, including at least the following: triggering persistent writing after the write end successfully sends data; triggering historical data recovery when the write end is created; triggering linked deletion when the write end's memory historical cache evicts data; triggering read consumption progress when the write end and read end successfully match; and triggering recording the latest consumption progress when the read end successfully receives data. Based on the persistence level, persistent operation tasks are dispatched to the storage backend through an operation dispatch table, and the storage backend performs persistent storage. The operation dispatch table defines a unified persistent operation interface in the form of function pointers. The storage backend includes at least transient-level persistent backends and persistent-level persistent backends, and each storage backend registers its own implementation path in the operation dispatch table.

2. The DDS persistent data storage management method according to claim 1, characterized in that: Before dispatching persistent operation tasks to the storage backend, perform condition checks, including at least ensuring that the reliability quality of service policy is reliable and the entity name is not empty.

3. The DDS persistent data storage management method according to claim 1, characterized in that: For persistence operation tasks assigned to transient-level persistence backends, the implementation path of the transient-level persistence backend is to communicate with the persistence service process via sockets through the persistence service's private protocol. The persistent service private protocol uses a binary message format. The message content must include at least a message header and a payload. The message header must include at least a fixed value for protocol identification and verification, the protocol version number, the message type, the message sequence number, and the number of bytes of payload data. The message types include at least inserting write history, querying write history, deleting write history, querying read progress, and updating read progress, which are mapped to the persistent operation interfaces defined in the operation dispatch table.

4. The DDS persistent data storage management method according to claim 3, characterized in that: For persistent write tasks assigned to transient-level persistent backends, after the write end publishes data, it constructs a request message of inserting the history type of the write end, uses the data sample as the payload, and pushes it to the persistent service process through the socket; after the server verifies the fixed value of the message header and the protocol version number field, it stores the data sample in the memory of the persistent service process. For historical data recovery tasks assigned to transient-level persistent backends, when the write end process restarts and creates a write end with the same entity name as before the restart, a request message for querying the write end's historical type is constructed to query the persistent service process for all historical data under that entity name. The server responds by returning all historical data samples. After receiving the response, the client calls the preset historical data query callback function to re-inject the historical data samples into the write end's memory historical cache. When a new read end matches the write end, the data in the write end's memory history cache is delivered to the new read end through the protocol engine; For linked deletion tasks assigned to transient-level persistent backends, when the memory history cache of the write end evicts data, a message of deleting the history type of the write end is constructed, and the entity name and sequence number of the write end are sent to the persistent service process as the payload. The server deletes the corresponding record from the memory of the persistent service process. For read consumption progress tasks assigned to transient-level persistent backends, when the read-end process restarts and re-matches with the write-end, a request message of the read-end progress type is constructed. The query is performed in the memory of the persistent service process based on the write-end entity name and the read-end entity name. If the last confirmed sequence number is found, the historical data after the last confirmed sequence number is delivered from the write-end's memory history cache, skipping the part already confirmed by the read-end. If the last confirmed sequence number is not found, all historical data in the write-end's memory history cache is delivered. For tasks that record the latest consumption progress and are assigned to the transient-level persistent backend, whenever the reader confirms that it has received data, a request message to update the reader's progress type is constructed, and the latest confirmation sequence number of the reader to the current writer is written to the persistent service process.

5. The DDS persistent data storage management method according to claim 1, characterized in that: For persistence operation tasks dispatched to the persistent level persistence backend, the implementation path of the persistent level persistence backend is to access the local SQLite database through pre-compiled SQL statements. The pre-compiled SQL statements include at least querying the writer history, inserting or updating the writer history, deleting a specified writer history, querying the last confirmation sequence number of the reader, and updating the confirmation sequence number of the reader, which are respectively mapped to the persistent operation interfaces defined in the operation dispatch table; An SQLite database must contain at least a historical data table for the write end and a status tracking table for the read end; The historical data table for the write end includes the write end entity name, data sequence number, serialized data payload, data source timestamp, and a composite primary key consisting of the write end entity name and data sequence number; The read-end status tracking table contains the read-end entity name, the corresponding write-end entity name, the last confirmed sequence number, and a composite primary key consisting of the read-end entity name and the corresponding write-end entity name.

6. The DDS persistent data storage management method according to claim 5, characterized in that: For persistent write tasks assigned to persistent backends, after the write end publishes data, data samples are stored in the SQLite database through pre-compiled insert or update write end historical statements. For historical data recovery tasks assigned to persistent backends, when the write process restarts and creates a write end with the same entity name as before the restart, it accesses the SQLite database through a pre-compiled query write end history statement to query all historical data under that entity name. The server responds by returning all historical data samples. After receiving the response, the client calls the preset historical data query callback function to re-inject the historical data samples into the write end's memory history cache. When a new read end matches the write end, the protocol engine delivers the data from the write end's memory history cache to the new read end; For linked deletion tasks assigned to persistent backends, when the memory history cache of the write end evicts data, the entity name and sequence number of the evicted data are obtained, and the record with the primary key of the entity name and sequence number is deleted from the write end history data table through a pre-compiled delete statement specifying the write end history. For read consumption progress tasks assigned to persistent backends, when the read process restarts and re-matches with the write process, the last confirmed sequence number is retrieved from the read status tracking table by composite primary key using a pre-compiled query statement to retrieve the last confirmed sequence number of the read process. If a record is found, the historical data after the last confirmed sequence number is delivered from the memory history cache of the write end, skipping the part already confirmed by the read end; if no record is found, all historical data in the memory history cache of the write end is delivered. For tasks that record the latest consumption progress and are assigned to persistent backends, whenever the read end confirms that it has received data, a pre-compiled statement to update the read end confirmation sequence number is called to write the latest confirmation sequence number of the read end to the current write end into the read end status tracking table.

7. The DDS persistent data storage management method according to claim 1, characterized in that: Transient-level persistence backend and persistent-level persistence backend use a unified data model to store data. The data model includes at least historical data records on the write end and consumption status records on the read end. Each record in the historical data record of the write end represents a data sample published by the write end, including the write end entity name, sequence number, serialized data payload pointer, payload size, and source timestamp when the data was written. The write end entity name and sequence number serve as a combined unique identifier. Each record in the read-end consumption status record represents the consumption progress of a read-write pair, including the read-end entity name, the corresponding write-end entity name, and the sequence number of the last confirmation by the read-end. The read-end entity name and the corresponding write-end entity name serve as a combined unique identifier.

8. The DDS persistent data storage management method according to claim 1, characterized in that: When a new storage backend is added, during the initialization phase, the new storage backend registers its implementation path in the operation dispatch table.

9. A DDS persistent data storage management system, characterized in that, include: The application layer module is used to obtain the target persistence level declared by the application through the persistence service quality policy during the data subscription or publication process. The persistence level includes at least transient level persistence and persistent level persistence. The protocol engine module is used to trigger corresponding persistent operation tasks at preset times during data communication. The preset timing triggers the corresponding persistence operation tasks, including at least the following: triggering persistent writing after the write end successfully sends data; triggering historical data recovery when the write end is created; triggering linked deletion when the write end's memory historical cache evicts data; triggering read consumption progress when the write end and read end successfully match; and triggering recording the latest consumption progress when the read end successfully receives data. The unified persistence interface layer module is used to dispatch persistence operation tasks to the storage backend based on the persistence level through an operation dispatch table; the operation dispatch table defines a unified persistence operation interface in the form of function pointers. The storage backend module is used to perform persistent storage through the storage backend; the storage backend module includes at least transient level persistent backend units and persistent level persistent backend units, and each storage backend registers its own implementation path in the operation dispatch table.

10. A DDS persistent data storage management system according to claim 9, characterized in that: The transient-level persistence backend unit is used for persistence operation tasks assigned to the transient-level persistence backend. The implementation path of the transient-level persistence backend is to communicate with the persistence service process via socket through the persistence service private protocol. The persistent-level persistence backend unit is used for persistent write tasks dispatched to the persistent-level persistence backend. After the write end publishes data, the data sample is stored in the SQLite database through pre-compiled insert or update write end history statements.

Images