Data access method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data access method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of loading a pre-constructed interface library in response to a received access request for real-time data in a shared memory; wherein the real-time data in the shared memory is stored by the real-time operating system according to a target data structure, and the interface library is constructed based on an intermediate language between a programming language of the non-real-time operating system and a programming language of the real-time operating system; based on a first application interface in the interface library, establishing an address mapping relationship between the interface library and the shared memory, and obtaining an address pointer of the shared memory; accessing the real-time data in the shared memory based on a second application interface, the address mapping relationship and the address pointer in the interface library; wherein the non-real-time operating system is used for storing the accessed real-time data according to the target data structure. By adopting the method, cross-language data access can be realized, and the real-time performance is ensured.

Description

Technical Field

[0001] This application relates to the field of cross-language data access technology, and in particular to a data access method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] In fields such as industrial control, robotics, and autonomous driving, where timeliness is critical, systems typically employ a layered architecture, specifically a real-time control layer and an upper application layer. The real-time control layer, built using C / C++ and a real-time kernel, is required to achieve microsecond-level deterministic responses, ensuring the precise execution of critical tasks such as equipment control and data acquisition. The upper application layer, developed using modern programming languages ​​such as Rust, Python, Go, and Java, is used to implement functions such as monitoring interface display, data analysis and processing, and business logic implementation. Its normal operation depends on efficient access to data from the real-time control layer.

[0003] In traditional technologies, three main solutions are used to achieve cross-language data sharing between the upper application layer and the real-time control layer: First, network communication solutions based on Socket or REST APIs, which complete cross-layer data interaction through data serialization and network transmission; second, middleware solutions based on message queues, which achieve asynchronous data transmission through queue caching; and third, solutions based on ordinary shared memory, which enable direct data reading and writing by allocating shared memory areas between processes.

[0004] However, network communication solutions based on Socket or REST APIs suffer from high data transmission latency due to high serialization overhead, and middleware solutions based on message queues incur additional overhead from data copying, making it difficult to meet the microsecond-level response requirements of the real-time control layer. While ordinary shared memory solutions offer lower latency, they lack adaptation to the characteristics of real-time kernels and cannot guarantee deterministic responses from the real-time control layer. Therefore, there is an urgent need for a data access method that can achieve cross-language data access while ensuring real-time performance. Summary of the Invention

[0005] Therefore, it is necessary to provide a data access method, apparatus, computer device, computer-readable storage medium, and computer program product that can achieve cross-language data access and ensure real-time performance in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a data access method, including:

[0007] In response to receiving a request to access real-time data in shared memory, a pre-built interface library is loaded; wherein the real-time data in shared memory is stored by a real-time operating system according to a target data structure, and the interface library is built based on an intermediate language between the programming language of the non-real-time operating system and the programming language of the real-time operating system.

[0008] Based on the first application interface in the interface library, establish an address mapping relationship between the interface library and the shared memory, and obtain the address pointer of the shared memory;

[0009] Based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the real-time data in the shared memory is accessed; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

[0010] In one embodiment, after accessing real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the process includes:

[0011] Based on the third application interface in the interface library, the address mapping relationship between the interface library and the shared memory is released, and the resources occupied during the process of accessing real-time data in the shared memory are released.

[0012] In one embodiment, the real-time operating system stores real-time data in shared memory based on a target data structure and removes compiler-filled padding bytes between fields of the real-time data; after accessing the real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the process includes:

[0013] Based on a data structure with the same memory layout as the target data structure, the accessed real-time data is stored, and the padding bytes automatically filled by the compiler between the fields of the accessed real-time data are deleted.

[0014] In one embodiment, the real-time operating system is also used to define the target data structure as having a fixed-size data type and / or to prohibit the use of non-serializable elements.

[0015] In one embodiment, before storing the accessed real-time data based on a data structure consistent with the memory layout of the target data structure, and deleting the padding bytes automatically filled by the compiler between the fields of the accessed real-time data, the following steps are included:

[0016] When the target language code used to access real-time data in shared memory is compiled or run, the consistency results of the field memory offset, field data type byte size and / or total byte length of the data structure used to store the accessed real-time data with the target data structure are determined.

[0017] The method of storing the accessed real-time data based on a data structure consistent with the memory layout of the target data structure, and deleting the padding bytes automatically filled by the compiler between the fields of the accessed real-time data, includes:

[0018] If the consistency result is consistent, the real-time data obtained from the access is stored based on the data structure, and redundant bytes filled between the fields of the real-time data obtained from the access are deleted.

[0019] In one embodiment, the real-time operating system is used to run at least one real-time task, and the non-real-time operating system is used to run at least one non-real-time task, wherein the scheduling priority of the real-time task is higher than that of the non-real-time task.

[0020] Secondly, this application also provides a data access device, comprising:

[0021] An interface library loading module is used to load a pre-built interface library in response to a received request to access real-time data in shared memory; wherein the real-time data in shared memory is stored by a real-time operating system according to a target data structure, and the interface library is built based on an intermediate language between the programming language of a non-real-time operating system and the programming language of the real-time operating system.

[0022] A shared memory connection module is used to establish an address mapping relationship between the interface library and the shared memory based on a first application interface in the interface library, and to obtain the address pointer of the shared memory;

[0023] The data access module is used to access real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

[0024] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above-mentioned embodiments.

[0025] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0026] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0027] The aforementioned data access methods, devices, computer equipment, computer-readable storage media, and computer program products, through a pre-built interface library of an intermediate language between programming languages ​​based on non-real-time operating systems and programming languages ​​based on real-time operating systems, form an intermediate layer adapted to both types of operating systems. This allows non-real-time operating systems to effectively access real-time shared memory by calling the intermediate layer interface, achieving cross-language real-time data sharing. Furthermore, by establishing an address mapping relationship between the intermediate layer and shared memory and obtaining address pointers, real-time data in shared memory can be directly located and read based on this address mapping relationship and address pointers, avoiding the additional data required in traditional solutions. According to the copy operation; at the same time, since the real-time data in the shared memory is stored by the real-time operating system according to the target data structure, and the non-real-time operating system also stores the data according to the target data structure after access, there is no need to convert the data format, avoiding time-consuming intermediate links such as serialization, improving the timeliness and transmission efficiency of cross-language data access, and controlling the cross-language data access latency to the microsecond level; in addition, the access behavior of the non-real-time operating system only interacts with the shared memory through the intermediate layer interface, without occupying the computing resources of the real-time kernel, effectively ensuring the priority scheduling characteristics and execution determinism of the real-time kernel, and ensuring that the normal operation of real-time tasks is not affected. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is an application environment diagram of a data access method in one embodiment;

[0030] Figure 2 This is a flowchart illustrating a data access method in one embodiment;

[0031] Figure 3 This is a flowchart illustrating the data access method in another embodiment;

[0032] Figure 4 This is a schematic diagram of the architecture of a data access method in one embodiment;

[0033] Figure 5This is a flowchart illustrating the data access method in another embodiment;

[0034] Figure 6 This is a schematic diagram of the data structures in C and Rust languages ​​in one embodiment;

[0035] Figure 7 This is a flowchart illustrating the data access method in another embodiment;

[0036] Figure 8 This is a flowchart illustrating the data access method in another embodiment;

[0037] Figure 9 This is a structural block diagram of a data access device in one embodiment;

[0038] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0041] In related technologies, the real-time control layer can use real-time shared memory for inter-task communication, but real-time shared memory cannot be directly accessed by non-real-time languages.

[0042] Based on this, embodiments of this application provide a data access method that can be applied to, for example... Figure 1 The application environment shown. In this environment, the non-real-time operating system serves as the upper application layer, through... oftenModern programming languages ​​(e.g., Rust, Python, Go, Java, etc.) are used to implement non-real-time monitoring or analysis applications. These applications need to access real-time data (e.g., real-time device status data, real-time sensor data, etc.) from the real-time operating system (i.e., the real-time control layer). This real-time data is typically updated periodically to shared memory (e.g., Xenomai RT_HEAP, POSIX real-time extensions, etc.) by the underlying real-time kernel (e.g., Xenomai, RT-PREEMPT, QNX, etc.) using system programming languages ​​(e.g., C / C++). Here, shared memory is a contiguous physical memory region managed by the real-time kernel, used for high-speed data interaction between real-time tasks or between real-time tasks and external applications. As the real-time control layer, the real-time operating system requires deterministic responses, typically on the order of microseconds. Therefore, shared memory is often difficult for non-real-time modern programming languages ​​to access directly and securely. Based on this, this application provides a data access method. An intermediate language is constructed between the programming language of a non-real-time operating system and the programming language of a real-time operating system to form an interface library, creating a data access intermediate layer. This allows the non-real-time operating system to load the pre-built interface library in response to a request to access real-time data in shared memory. Based on a first application interface in the interface library, an address mapping relationship between the interface library and shared memory is established, and an address pointer to the shared memory is obtained. Based on a second application interface in the interface library, the address mapping relationship, and the address pointer, the real-time data in shared memory is accessed. The real-time data in shared memory is stored by the real-time operating system according to a target data structure, and the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

[0043] In one exemplary embodiment, such as Figure 2 As shown, a data access method is provided, which can be applied to... Figure 1 Taking a non-real-time operating system as an example, the explanation includes the following steps S202 to S206. Wherein:

[0044] S202, in response to receiving a request to access real-time data in shared memory, a pre-built interface library is loaded; wherein, the real-time data in shared memory is stored by the real-time operating system according to the target data structure, and the interface library is built based on an intermediate language between the programming language of the non-real-time operating system and the programming language of the real-time operating system.

[0045] In this context, the programming language for the non-real-time operating system (RTOS) can be a modern programming language with a higher abstraction level than the ROS's programming language (e.g., the system programming language C / C++). Non-RTOS programming languages ​​typically cannot directly access shared memory managed by the ROS (i.e., real-time shared memory). In one possible implementation, an intermediate language between the non-RTOS and ROS programming languages ​​can directly call user-space APIs provided by the real-time kernel. This intermediate language could be C.

[0046] For example, the pre-built interface library can be a standard C language interface library conforming to the C language ABI (Application Binary Interface), compiled into a dynamic link library (e.g., a .so file on Linux, a .dll file on Windows). Furthermore, step S202 above can include: in response to receiving a request to access real-time data in shared memory, loading the pre-built interface library through the Foreign Function Interface (FFI). FFI is compatible with programming languages ​​of various non-real-time operating systems. Specifically, for Rust, the interface library can be linked using #[link(name="interface library name")] and the interface library functions can be declared with extern "C"; for Python, the interface library can be loaded using ctypes.CDLL and the parameter types and return type of the functions can be declared.

[0047] S204: Based on the first application interface in the interface library, establish the address mapping relationship between the interface library and shared memory, and obtain the address pointer of shared memory.

[0048] The first application interface can be encapsulated by an initialization function, which establishes the connection between the data access middleware and the shared memory. For example, the first application interface can be used to call the user-space API of the real-time kernel to map the physical address of the shared memory managed by the real-time kernel to the virtual address space of the process where the interface library resides, thus establishing the address mapping relationship between the interface library and the shared memory. The first application interface can also be used to extract the virtual address corresponding to the shared memory from the mapping result returned by the real-time kernel, and assign this virtual address to a global pointer variable as the address pointer of the shared memory (e.g., denoted as g_shared_ptr).

[0049] Specifically, the first application interface (rt_data_init) can be called to establish the address mapping relationship between the interface library and real-time shared memory, and obtain the shared memory access entry point, including: receiving the unique identifier of the real-time shared memory (such as the name "rt_mem_name"), calling the user-space API provided by the real-time kernel (such as bind_rt_shared_memory); mapping the physical address of the shared memory managed by the real-time kernel to the virtual address space of the process where the interface library is located through the API, establishing a one-to-one correspondence between the virtual address and the physical address; extracting the virtual address of the shared memory from the mapping result returned by the real-time kernel, assigning it to the global pointer variable (g_shared_ptr), as the fixed entry address for accessing the shared memory; returning a success flag (0) after initialization, and returning a non-zero error code if binding fails, and feeding back the connection result to the non-real-time operating system. The first application interface can adapt to the calling logic of different non-real-time languages: Rust calls it in the unsafe code block, Python calls it directly and verifies the return value, and Go calls it in the form of C.function_name.

[0050] S206 accesses real-time data in shared memory based on the second application interface, address mapping relationship, and address pointer in the interface library; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

[0051] The second application interface can be obtained by encapsulating a data read function, which can be used for zero-copy reading of shared memory data. The address mapping relationship can include a one-to-one correspondence between the interface library's virtual address and the shared memory's physical address, ensuring that the interface library's read and write operations on virtual addresses directly affect the physical storage area of ​​the real-time shared memory. Address pointers are used to provide the entry address for accessing shared memory, avoiding the need to repeatedly establish address mappings for each data read, thus reducing access overhead.

[0052] Specifically, the second application interface (rt_data_read) can be called to directly access shared memory without data copying / serialization operations. This includes: receiving the index of the target data block in shared memory from the non-real-time operating system, which is used to locate the specific data in a multi-data-block scenario; and storing the target structure pointer (out) of the read result, which points to the area in the non-real-time application memory used to store data; verifying that the virtual address pointed to by the global pointer (g_shared_ptr) can be correctly mapped to the physical address of the real-time shared memory to ensure the validity of the access; using g_shared_ptr as the base address, combined with the index and the structure byte length, calculating the physical address offset of the data block to be read in shared memory; directly reading real-time data fields (such as sensor_value, device_status) from this physical address and writing the field values ​​to the memory area pointed to by the out pointer; returning a success flag (0) after reading is completed; and returning a non-zero error code if reading fails, and feeding back the result to the non-real-time operating system.

[0053] Understandably, since real-time data is stored in a shared memory physical address, the interface library does not need to copy the real-time data to its own cache. Instead, it can directly write the data in the physical address to the memory pointed to by the data target address through the address pointer, thus achieving zero-copy access.

[0054] Optionally, after step S206, the data access method may further include: storing the accessed real-time data according to the target data structure. For example, the accessed real-time data may be stored based on a data structure with a memory layout consistent with the target data structure of the real-time data in shared memory, ensuring memory layout consistency across language data structures. In one possible implementation, the target data structure may be used to ensure that the memory layout is uniquely determined by field types and order.

[0055] Understandably, since real-time data always exists in a consistent memory layout, it doesn't need to be converted to any intermediate format, thus avoiding serialization. Here, serialization refers to the process of converting structured data in memory (such as C structs, Python classes) into transmittable formats such as strings (JSON) or binary streams (Protobuf / MessagePack). Traditional technologies based on Socket / REST APIs and message queues rely on serialization.

[0056] In one possible implementation, before the non-real-time operating system loads a pre-built interface library in response to a request for access to real-time data in shared memory, the real-time operating system can first be initialized. This includes: the real-time operating system starting a real-time task, creating or binding a shared memory region through the real-time kernel API; defining the data structure of this shared memory region as the target data structure; and periodically updating the real-time data in the shared memory region. Then, a data access middleware layer is constructed, including: building an interface library based on an intermediate language between the programming language of the non-real-time operating system and the programming language of the real-time operating system; this interface library provides multiple application interfaces by encapsulating data access functions. Next, cross-language data structure mapping is performed, including: defining the data structure of the target language as the target data structure. Here, the target language can be a modern programming language that needs to access real-time shared memory. Alternatively, the real-time operating system can store the metadata of the target data structure (field name, field type, field offset, total byte length, and / or alignment rules) in a fixed address area of ​​shared memory; before the non-real-time operating system accesses the real-time data, it can read the metadata and dynamically generate a temporary data structure consistent with the memory layout of the target data structure, and store the accessed real-time data based on this temporary data structure.

[0057] In the aforementioned data access method, an intermediate layer adapted to both operating systems is formed by pre-building an interface library in an intermediate language between programming languages ​​of non-real-time operating systems and programming languages ​​of real-time operating systems. This allows non-real-time operating systems to effectively access real-time shared memory by calling the intermediate layer interface, achieving cross-language real-time data sharing. Furthermore, by establishing an address mapping relationship between the intermediate layer and shared memory and obtaining address pointers, the real-time data in shared memory can be directly located and read based on this address mapping relationship and address pointers, avoiding the additional data copying operations in traditional solutions. Simultaneously, since the real-time data in shared memory is stored by the real-time operating system according to the target data structure, and the non-real-time operating system also stores the data according to the target data structure after access, there is no need to convert the data format, avoiding time-consuming intermediate steps such as serialization, improving the timeliness and transmission efficiency of cross-language data access, and controlling the cross-language data access latency to the microsecond level (e.g., within 10 microseconds). In addition, the access behavior of the non-real-time operating system only interacts with shared memory through the intermediate layer interface, without occupying the computing resources of the real-time kernel, effectively ensuring the priority scheduling characteristics and execution determinism of the real-time kernel, and ensuring that the normal operation of real-time tasks is not affected.

[0058] In one exemplary embodiment, such as Figure 3 As shown, the above data access method may also include:

[0059] S208, based on the third application interface in the interface library, removes the address mapping relationship between the interface library and shared memory, and releases the resources occupied during the process of accessing real-time data in shared memory.

[0060] The third application interface can be obtained by encapsulating a resource release function. This resource release function can be used to disconnect the interface library from the real-time shared memory and release the memory resources occupied by the interface library during access to shared memory. For example, the third application interface can be used to verify the validity of the shared memory address pointer (g_shared_ptr): if the pointer is not null, it is determined that an address mapping relationship has been established and subsequent release operations are performed; if the pointer is null, it is determined that there is no valid mapping relationship and a success flag (0) is returned directly; the user-space API of the real-time kernel is called to initiate an address mapping release request to the real-time kernel to release the one-to-one correspondence between the virtual address of the interface library and the physical address of the shared memory; the address pointer (g_shared_ptr) is set to null, and the global memory space used to store the pointer in the interface library process is released; the auxiliary resources occupied by the interface library during access to shared memory are released; the resource release result is fed back to the non-real-time operating system: 0 is returned for successful release, and an error code (non-zero) is returned for failure.

[0061] Specifically, for Rust, non-real-time applications can call the resource release function `rt_data_close()` corresponding to the third-party application interface within an `unsafe` block. No parameters are needed when calling the function; the function call is executed directly. The return value is checked: a return value of 0 indicates successful resource release, and the pointer to the local structure pointing to the shared memory data in the Rust application is set to null; a non-zero return value logs an error. After successful release, the structure instance storing real-time data in the Rust application is destroyed, triggering memory reclamation. For Python, non-real-time applications can call the resource release function `lib.rt_data_close()` in the interface library via `ctypes`. No parameters are needed when calling the function; the function call is executed directly. The return value is checked: a return value of 0 indicates successful resource release, and the structure instance used to store real-time data in Python is destroyed, releasing the memory it occupies; a non-zero return value triggers a runtime exception. After successful release, the associated interface library object in the Python application is set to null, triggering resource reclamation. In Go, non-real-time applications can call the resource release function via cgo using the form C.rt_data_close(). No parameters are required when calling the function; the function call is executed directly. The function's return value is checked: if it returns 0, the memory occupied by the struct instance storing real-time data in the Go application is released; if it returns a non-zero value, a program exception is triggered. After successful release, the variable storing the shared memory address pointer in the Go application is set to zero, thus severing the association with the real-time shared memory.

[0062] In one possible implementation, please refer to Figure 4 , Figure 4 This is a schematic diagram of the architecture of a data access method in implementation. Specifically, the real-time control layer runs on a real-time kernel such as Xenomai and includes real-time task A (sensor data acquisition) and real-time task B (motion control). These two tasks write the acquired real-time data to real-time shared memory (RTHEAP or POSIXSHM) at high frequency to ensure data timeliness. The C language middleware layer (following the standard C language ABI) is used to bind the real-time shared memory through the bind_rt_shared_memory() interface (i.e., the first application interface), establish a mapping relationship between virtual addresses and physical addresses, obtain the address pointer of the shared memory, and achieve zero-copy reading through the rt_data_read() interface (i.e., the second application interface), directly obtaining data from the physical address of the shared memory without additional data copying or serialization. The non-real-time application layer (ordinary operating system processes) calls the interface of the C language middleware layer through FFI (such as libffi, cbindgen) to load and call the functions of the middleware layer. Non-real-time applications can be web monitoring interfaces developed in Rust, data analysis / machine learning programs developed in Python, or microservice APIs developed in Go. They access data in real-time shared memory through the middleware interface in a zero-copy manner. After data access is complete, the non-real-time application calls the rt_data_close() interface (i.e., the third-party application interface) to release the address mapping between the middleware and the shared memory, thus releasing the occupied resources.

[0063] In this embodiment, by releasing the relevant resources after the non-real-time operating system accesses real-time data, it is possible to avoid the decline in system operating efficiency caused by resource accumulation, and at the same time, avoid a single process occupying shared memory access permissions for a long time, thereby improving the resource utilization of real-time shared memory.

[0064] In one exemplary embodiment, the aforementioned real-time operating system is used to store real-time data in shared memory based on the target data structure and remove padding bytes automatically filled by the compiler between fields of the real-time data, such as... Figure 5 As shown, the above data access method may also include:

[0065] S210, based on a data structure consistent with the memory layout of the target data structure, stores the accessed real-time data and removes the padding bytes automatically filled by the compiler between the fields of the accessed real-time data.

[0066] For example, when defining a target data structure, the compiler can force the 1-byte alignment rule through compiler-related rules, eliminating the compiler's automatic padding bytes between fields and ensuring that the fields of the target data structure are stored continuously according to their actual byte length without redundant padding bytes. When constructing a data structure with the same memory layout as the target data structure, 1-byte alignment can be achieved through the native syntax of the target language, eliminating compiler padding bytes. Specifically, for real-time operating system code written in C, the `#pragmapack(1)` directive can be added before the target data structure definition, and the `#pragmapack()` directive can be added after the definition to restore the default alignment rule and eliminate the compiler's automatic padding bytes between fields. For Rust, the `#[repr(C,packed)]` attribute can be added to the structure definition to ensure that the structure follows the C language field order rules and forces 1-byte alignment, eliminating compiler padding bytes. For Python, when defining a data structure based on `ctypes.Structure`, the class attribute `_pack_=1` can be set to force the structure fields to be stored with 1-byte alignment. For Go, the compiler's automatic padding bytes can be avoided by manually calculating the field offsets.

[0067] In one possible implementation, please refer to Figure 6 , Figure 6 This is a schematic diagram of a C language data structure and a Rust language data structure in one embodiment. Specifically, the C language structure uses the #pragmapack(1) compiler directive to enable 1-byte alignment, eliminating automatic padding bytes between fields. It contains two fields: sensor_value (64-bit floating-point, 8 bytes) with an offset of 0; and device_status (32-bit integer, 4 bytes) with an offset of 8. The total size of the structure is 12 bytes, with no redundant padding bytes. Correspondingly, the Rust language structure is modified by the #[repr(C,packed)] attribute, which ensures that the field order is completely consistent with the C language structure and forces 1-byte alignment, eliminating automatic padding bytes. It contains fields that correspond one-to-one with the C language structure: sensor_value (64-bit floating-point) and device_status (32-bit integer). The total size of the structure is also 12 bytes, with no redundant padding bytes. Understandably, since the memory layouts of the two are consistent, when a non-real-time Rust application calls the C language middleware through FFI, it can directly map the pointer of shared memory to the Rust structure and correctly read real-time data without data copying or serialization.

[0068] In this way, by eliminating the padding bytes automatically filled by the compiler, the field offsets and total lengths of data structures in non-real-time languages ​​and real-time languages ​​are completely consistent in memory. This avoids field misalignment problems caused by padding bytes when reading data across languages, while reducing the memory footprint of data storage and improving the resource utilization of shared memory.

[0069] Optionally, the aforementioned real-time operating system is also used to define the target data structure as having a fixed-size data type, and / or to prohibit the use of non-serializable elements.

[0070] For example, the target data structure can use only basic data types with defined byte lengths that do not change with the compiler, operating system, or hardware platform, avoiding the use of general-purpose integer types such as int / long, and general-purpose floating-point types with unspecified lengths. For instance, fixed-length integer types such as int8_t, int16_t, uint8_t, and uint16_t can be preferred over general-purpose integer types like int / long whose lengths vary depending on the platform / compiler. Floating-point types can use only standard fixed-length floating-point types such as float (4 bytes) and double (8 bytes). Character types can use only fixed-length character types such as char (1 byte).

[0071] For example, the target data structure can contain only the aforementioned fixed-size basic data types, and it is prohibited to introduce non-serializable elements such as pointers, virtual function tables, dynamic arrays, and reference types, which are only valid within a single process / language environment and cannot be directly parsed across languages. Specifically, it is prohibited to define pointer fields in the target data structure; it is prohibited to use C++ classes containing virtual functions to define the target data structure; and it is prohibited to use dynamically length arrays to ensure that the total byte length of the target data structure is fixed and predictable.

[0072] In this way, by limiting the usage rules of the target data structure type, we can ensure that the memory layout of the data structure has cross-platform and cross-language determinism, avoid data parsing failures caused by changes in data type length or non-serializable elements, and reduce the adaptation cost of cross-language data access.

[0073] Optionally, consistency checks on memory layout may include verifying whether attributes such as field memory offsets, field data type size in bytes, and total byte length are consistent. For example... Figure 7 As shown, the above data access method may also include:

[0074] S209, when the target language code used to access real-time data in shared memory is compiled or executed, determine the consistency results between the data structure used to store the accessed real-time data and the target data structure in terms of field memory offsets, field data type byte size, and / or total byte length.

[0075] Furthermore, step S208 may include: if the consistency result is consistent, storing the accessed real-time data based on the data structure, and deleting redundant bytes that fill the fields between the accessed real-time data.

[0076] For example, for different target languages, memory layout consistency verification can be achieved through compile-time assertions (such as Rustsize_of check) or runtime checks (such as hash value comparison). For instance, based on the cross-language data type correspondence defined in a general mapping protocol, constant assertions can be used during the Rust code compilation phase to verify the field memory offset, field byte size, and total byte length of the data structure. If any of the above assertions are false, the code compilation fails, and a compilation error indicating that the data structure layout is inconsistent with the target data structure is output. If all assertions pass, the consistency result is determined to be consistent, and real-time data is stored based on this structure, with redundant padding bytes between fields removed.

[0077] Alternatively, based on the cross-language data type correspondence defined in the universal mapping protocol, the ctypes module can be used to obtain the layout information of the custom data structure before Python code accesses shared memory. This information is then compared with the target data structure metadata pre-stored at a fixed address in shared memory by the real-time operating system. If the runtime verification result is consistent, the accessed real-time data is stored based on this structure. If the verification result is inconsistent, data storage is terminated and an exception is displayed. The cross-language universal mapping protocol can be found in Table 1.

[0078] Table 1

[0079]

[0080] In one possible implementation, the target data structure is a 1-byte aligned structure using fixed-size basic types (int32_t, double, uint8_t) and without non-serializable elements such as pointers / virtual function tables. The non-real-time operating system constructs an adaptive data structure within its own programming language, ensuring that the field order, type, and size of this adaptive data structure are completely identical to the target data structure. The compiler padding bytes for the adaptive data structure are eliminated through the programming language's native memory layout control mechanism (Rust's #[repr(C,packed)], Python's ctypes.Structure._pack_=1). During the compilation phase (Rust) or runtime phase (Python / Go), the consistency between the field memory offsets and total byte length of the adaptive data structure and the target data structure is verified. Once the verification passes, real-time data is stored.

[0081] In this way, by verifying the consistency of data structures during the compilation or runtime phase, problems with memory layout mismatch can be detected in advance, avoiding anomalies caused by invalid data access. At the same time, it ensures that there are no redundant bytes during data storage, further improving the accuracy and efficiency of data access.

[0082] In one exemplary embodiment, the real-time operating system is used to run at least one real-time task, and the non-real-time operating system is used to run at least one non-real-time task, wherein the scheduling priority of the real-time task is higher than that of the non-real-time task.

[0083] For example, a non-real-time operating system can run non-real-time tasks (lower priority) with low time determinism requirements through its general-purpose scheduler, while a real-time operating system can run real-time tasks (higher priority) that need to meet time limits through its real-time kernel scheduler. The two communicate with each other through shared memory, achieving spatial and temporal isolation. Therefore, the delays that may occur in non-real-time tasks will not interfere with the deterministic execution of real-time tasks.

[0084] In one exemplary embodiment, please refer to Figure 8 , Figure 8 This is a flowchart illustrating a data access method in one embodiment. Specifically: The application layer calls the second application interface rt_data_read(index) in the interface library through FFI, where index is the index of the target data block in shared memory, and rt_data_read is the second application interface encapsulating zero-copy read logic; the interface library (C language intermediate layer) calculates the offset offset(index) of the data block based on the established address mapping relationship, using base_ptr (shared memory address pointer) as the base address, and combining index with the byte length of the target data structure (first appearance: the physical address offset of the target data block calculated based on the index), thus locating the physical address of the target data block; the interface library directly reads real-time data fields (such as sensor_value, device_status) from the located physical address, and writes the data to the memory area pointed to by the target structure pointer passed by the application layer, without memcpy or serialization operations, achieving zero-copy access; the interface library encapsulates the read real-time data into a Result. <datablock>The returned data is a structure whose memory layout is consistent with the target data structure; the application layer stores and processes real-time data based on the DataBlock structure.

[0085] In practical applications, the above data access method can be applied to industrial automation control systems. This system includes a real-time controller running a real-time kernel (real-time operating system side) and a web monitoring system written in Rust (non-real-time operating system side). The real-time controller needs to frequently share real-time device status data (including temperature, speed, status codes, and error flags) to the web monitoring system. This involves: an intermediate language (C language) between the real-time OS's C++ and the non-real-time OS's Rust, constructing a pre-compiled interface library (following CABI) containing rt_data_init / rt_data_read / rt_data_close; the non-real-time OS (Rust application layer) responds to data access requests, loading the C language interface library via the FFI mechanism (#[link(name="rtdata")]+extern"C"); and calling rt_data... The `_init` function binds the shared memory `rt_device_data` using `bind_rt_shared_memory`, establishing a mapping between the interface library's virtual address and the shared memory's physical address. It then assigns the shared memory address pointer to the global variable `g_state.shm_ptr`, converting it to `DeviceData*data_array` as the shared memory address pointer. The `rt_data_read` function, based on the established address mapping and `data_array` (the address pointer), directly reads the `DeviceData` block at index 0 in shared memory using zero-copy, writing fields such as temperature and speed into a structure pointer passed from Rust. The non-real-time OS (Rust) defines a local structure completely identical to the real-time OS's `DeviceData` (target data structure), storing the read real-time data in this structure. The real-time OS (C++) forces 1-byte alignment using `#pragmapack(push,1) / #pragmapack(pop)`, removing compiler padding bytes between `DeviceData` fields, and storing the real-time data organized according to this structure in shared memory. In non-real-time operating systems (Rust), the DeviceData structure is defined with the #[repr(C,packed)] modifier. This ensures that the field order, type, and byte length are completely consistent with the target data structure of the real-time operating system. #[repr(C,packed)] also forces 1-byte alignment and removes compiler padding bytes between fields in the native structure. The target data structure DeviceData uses only types with fixed byte lengths (double / 8 bytes, int32_t / 4 bytes, uint8_t / 1 byte), and does not use general-purpose integer types such as int / long whose lengths vary with the platform.The target data structure contains only the aforementioned fixed-size basic types and does not introduce non-serializable elements that cannot be parsed across languages, such as pointers, dynamic arrays, virtual function tables, and reference types, ensuring that the total byte length of the structure is fixed at 21 bytes.

[0086] Compared to traditional REST API-based solutions, the solution provided in this application significantly improves performance in industrial automation control system scenarios: read latency is reduced from 2000 microseconds to 8 microseconds; CPU utilization is reduced from 20% to <1%; and sampling frequency is increased from 500Hz to >100kHz.

[0087] The data access method described above can also be applied to Python data analysis scenarios in industrial automation control systems. This system includes a real-time controller (real-time operating system side) running a real-time kernel (such as Xenomai, RT-PREEMPT) and a data analysis application written in Python (non-real-time operating system side). The real-time controller needs to frequently share real-time data of device status to the Python data analysis application for non-real-time tasks such as offline analysis of device operating status and anomaly detection.

[0088] In summary, the above data access method utilizes a pre-built interface library in an intermediate language that bridges the gap between programming languages ​​used in non-real-time operating systems and those used in real-time operating systems. This creates an intermediate layer that adapts to both types of operating systems, enabling non-real-time operating systems to effectively access real-time shared memory by calling this intermediate layer interface, thus achieving cross-language real-time data sharing. Furthermore, by establishing an address mapping relationship between the intermediate layer and shared memory and obtaining address pointers, the method directly locates and reads real-time data from shared memory based on this address mapping and pointers, avoiding the additional data copying operations found in traditional solutions. Simultaneously, because shared memory... Real-time data is stored by the real-time operating system according to the target data structure, and non-real-time operating systems also store data according to the same target data structure after access, eliminating the need for data format conversion and avoiding time-consuming intermediate steps such as serialization. This improves the timeliness and transmission efficiency of cross-language data access, keeping cross-language data access latency to the microsecond level (e.g., within 10 microseconds). Furthermore, the access behavior of non-real-time operating systems only interacts with shared memory through an intermediate layer interface, without consuming the computing resources of the real-time kernel. This effectively guarantees the priority scheduling characteristics and execution determinism of the real-time kernel, ensuring that the normal operation of real-time tasks is unaffected. In practical applications, compared to traditional technologies based on REST... The API solution provided in this application improves upon traditional solutions in terms of data read latency, CPU resource consumption, data access throughput, and memory copy overhead. Traditional solutions have read latency of 1000 to 5000 microseconds, while the solution provided in this application, through address mapping and zero-copy logic using direct pointer access, controls the latency to within 10 microseconds. Traditional solutions require time-consuming operations such as data copying and serialization / deserialization, resulting in CPU utilization of 15% to 30%. The solution provided in this application, because it eliminates the need for additional data processing steps, controls CPU utilization to within 1%. This significantly reduces system operating overhead, reserving more computing resources for other business tasks. Traditional solutions have a throughput of less than 1,000 times per second, which is insufficient to meet the needs of high-frequency real-time data access. The solution provided in this application, through zero-copy and simplified interface call logic, can achieve a throughput of 100,000 times per second, supporting large-scale, high-frequency cross-language real-time data access scenarios. Traditional solutions require 3 to 5 memory copies during data transmission, which not only increases latency but also wastes memory bandwidth. The solution provided in this application achieves zero-copy access through direct address mapping between the intermediate layer and shared memory.

[0089] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0090] Based on the same inventive concept, this application also provides a data access device for implementing the data access method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more data access device embodiments provided below can be found in the limitations of the data access method described above, and will not be repeated here.

[0091] In one exemplary embodiment, such as Figure 9 As shown, a data access device 300 is provided, including: an interface library loading module 301, a shared memory connection module 302, and a data access module 303, wherein:

[0092] The interface library loading module 301 is used to load a pre-built interface library in response to a received request to access real-time data in shared memory. The real-time data in shared memory is stored by the real-time operating system according to the target data structure, and the interface library is built based on an intermediate language between the programming language of the non-real-time operating system and the programming language of the real-time operating system.

[0093] The shared memory connection module 302 is used to establish an address mapping relationship between the interface library and the shared memory based on the first application interface in the interface library, and to obtain the address pointer of the shared memory.

[0094] The data access module 303 is used to access real-time data in shared memory based on the second application interface, address mapping relationship and address pointer in the interface library; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

[0095] In one embodiment, the data access device 300 further includes:

[0096] The resource release module is used to release the address mapping relationship between the interface library and shared memory based on the third application interface in the interface library, and release the resources occupied during the process of accessing real-time data in shared memory.

[0097] In one embodiment, the real-time operating system is used to store real-time data in shared memory based on a target data structure and remove padding bytes automatically filled by the compiler between fields of the real-time data. The data access device 300 further includes:

[0098] The data storage module is used to store the accessed real-time data based on a data structure with the same memory layout as the target data structure, and to remove the padding bytes automatically filled by the compiler between the fields of the accessed real-time data.

[0099] In one embodiment, the aforementioned real-time operating system is further configured to define the target data structure as using a fixed-size data type, and / or to prohibit the use of non-serializable elements.

[0100] In one embodiment, the data access device 300 further includes:

[0101] The consistency verification module is used to determine the consistency between the data structure used to store the accessed real-time data and the target data structure in terms of field memory offsets, field data type byte size, and / or total byte length, when the target language code used to access real-time data in shared memory is compiled or run.

[0102] The aforementioned data storage module is also used to store the real-time data obtained from the access based on the data structure, and to delete redundant bytes that fill the fields between the fields of the real-time data obtained from the access, provided that the consistency result is consistent.

[0103] In one embodiment, the real-time operating system is used to run at least one real-time task, and the non-real-time operating system is used to run at least one non-real-time task, wherein the scheduling priority of the real-time task is higher than that of the non-real-time task.

[0104] Each module in the aforementioned data access device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0105] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data access method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0106] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0107] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0109] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0112] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / datablock>

Claims

1. A data access method, characterized in that, Applied to non-real-time operating systems, the methods include: In response to receiving a request to access real-time data in shared memory, a pre-built interface library is loaded; wherein the real-time data in shared memory is stored by a real-time operating system according to a target data structure, and the interface library is built based on an intermediate language between the programming language of the non-real-time operating system and the programming language of the real-time operating system. Based on the first application interface in the interface library, establish an address mapping relationship between the interface library and the shared memory, and obtain the address pointer of the shared memory; Based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the real-time data in the shared memory is accessed; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

2. The method according to claim 1, characterized in that, After accessing real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the process includes: Based on the third application interface in the interface library, the address mapping relationship between the interface library and the shared memory is released, and the resources occupied during the process of accessing real-time data in the shared memory are released.

3. The method according to claim 1, characterized in that, The real-time operating system is used to store real-time data in shared memory based on the target data structure and to remove padding bytes automatically filled by the compiler between fields of the real-time data. After accessing real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer, the process includes: Based on a data structure with the same memory layout as the target data structure, the accessed real-time data is stored, and the padding bytes automatically filled by the compiler between the fields of the accessed real-time data are deleted.

4. The method according to claim 3, characterized in that, The real-time operating system is also used to define target data structures using fixed-size data types and / or to prohibit the use of non-serializable elements.

5. The method according to claim 3, characterized in that, Before storing the accessed real-time data in the data structure based on the memory layout consistent with the target data structure, and deleting the padding bytes automatically filled by the compiler between the fields of the accessed real-time data, the following steps are included: When the target language code used to access real-time data in shared memory is compiled or run, the consistency results of the field memory offset, field data type byte size and / or total byte length of the data structure used to store the accessed real-time data with the target data structure are determined. The method of storing the accessed real-time data based on a data structure consistent with the memory layout of the target data structure, and deleting the padding bytes automatically filled by the compiler between the fields of the accessed real-time data, includes: If the consistency result is consistent, the real-time data obtained from the access is stored based on the data structure, and redundant bytes filled between the fields of the real-time data obtained from the access are deleted.

6. The method according to claim 1, characterized in that, The real-time operating system is used to run at least one real-time task, and the non-real-time operating system is used to run at least one non-real-time task, wherein the scheduling priority of the real-time task is higher than that of the non-real-time task.

7. A data access device, characterized in that, The device includes: An interface library loading module is used to load a pre-built interface library in response to a received request to access real-time data in shared memory; wherein the real-time data in shared memory is stored by a real-time operating system according to a target data structure, and the interface library is built based on an intermediate language between the programming language of a non-real-time operating system and the programming language of the real-time operating system. A shared memory connection module is used to establish an address mapping relationship between the interface library and the shared memory based on the first application interface in the interface library, and to obtain the address pointer of the shared memory; The data access module is used to access real-time data in the shared memory based on the second application interface in the interface library, the address mapping relationship, and the address pointer; wherein, the non-real-time operating system is used to store the accessed real-time data according to the target data structure.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When a computer program is executed by a processor, it implements the steps of the method of any one of claims 1 to 6.

Images