Method for compatible running of Android camera HAL in container based on memory access virtualization

Abstract

The application discloses a container-in-Android camera HAL compatible running method based on memory access virtualization, takes a DMA-Buf memory heap of a Linux kernel host system as a target memory heap, takes an ION memory heap of an Android container system as a source memory heap, creates a process-dedicated memory management context, an FD cache table and a synchronization barrier pool, completes kernel registration and node binding of a virtual ION device, first memory pool pre-allocation and kernel layer device access hook registration, checks legality of a container process and initializes context binding to a file descriptor when an ION file descriptor is acquired by a HAL process; intercepts a memory allocation request of the HAL process, parses and adapts parameters, and preferentially reuses first memory pool resources to obtain an ION handle; when a data sharing request is processed, an FD cache is matched or a new FD is generated through the ION handle; when memory is released by the HAL process, resources are recycled according to the memory source, and when the HAL process exits, the context is cached or destroyed, and cross-system memory operation compatibility is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer software development, and particularly relates to a container-in Android camera HAL compatible running method based on memory access virtualization. BACKGROUND

[0002] When an Android application is run in a container manner in a host system based on a Linux kernel, camera function adaptation still faces core technical obstacles. The camera hardware abstraction layer (Hardware Abstraction Layer, HAL) of the Android system in the container is a non-modifiable closed-source component, which relies on the ION memory management mechanism to complete frame data allocation, sharing, mapping and synchronization, while the Linux kernel used on the host system side only supports the DMA-Buf (Direct Memory Access Buffer) general memory sharing framework, and there are significant differences in interface semantics, memory management granularity and synchronization mechanism between the two.

[0003] For example, when an Android application is run in a container manner in a host system based on a Hongmeng system, camera function adaptation still faces core technical obstacles. The camera hardware abstraction layer (Hardware Abstraction Layer, HAL) of the Android system in the container is a non-modifiable closed-source component, which relies on the ION memory management mechanism to complete frame data allocation, sharing, mapping and synchronization, while the Linux kernel used on the host system side only supports the DMA-Buf (Direct Memory Access Buffer) general memory sharing framework, and there are significant differences in interface semantics, memory management granularity and synchronization mechanism between the two.

[0004] Specifically, the ION memory management mechanism and the DMA-Buf general memory sharing framework mainly have the following core differences: first, the Heap management mode, ION adopts a vendor-customized Heap classification, and realizes memory operation through a fixed ioctl command set, while DMA-Buf adopts a standardized Heap registration mechanism, supports dynamic expansion and uniform interfaces; second, the synchronization mechanism, ION relies on user-mode manual management of synchronization barriers, which is prone to data competition, while DMA-Buf integrates dma_fence kernel-mode automatic synchronization, which can realize cross-device operation sequence control; third, the memory mapping logic, ION needs to manually trigger mapping in combination with handles and cache strategies, while DMA-Buf automatically adapts hardware cache attributes through a standardized mmap interface. Both support file descriptor (FD) cross-process sharing, but the underlying implementation logic difference causes the Android camera HAL to be unable to run directly on the Hongmeng system kernel.

[0005] The existing compatible solution mainly adopts a user state conversion layer such as libion_emu, and has the following defects: 1. large performance loss, and conversion delay exceeding 10 ms cannot meet the real-time requirement of 30 fps of a camera; 2. insufficient compatibility, and cannot cover the manufacturer-defined ioctl command of ION; and 3. low safety, and the user state layer is difficult to adapt to the security enhanced Linux access control of the Hongmeng system. SUMMARY

[0006] Therefore, the application provides a container-in-Android camera HAL compatible running method based on memory access virtualization, which realizes efficient simulation of the ION mechanism through a special kernel module without modifying the closed-source HAL and the host kernel body.

[0007] The container-in-Android camera HAL compatible running method based on memory access virtualization provided by the application specifically comprises the following steps:

[0008] Taking a Linux kernel-based system as a host system, taking a DMA-Buf memory heap as a target memory heap, taking an Android system as a container system, taking an ION memory heap as a source memory heap, and constructing a kernel state conversion module in the host system;

[0009] The conversion module establishes a first structure body to save the exclusive memory management context of an Android camera HAL process, establishes an FD cache table to save the mapping relationship between an ION handle and a DMA-Buf file descriptor FD, creates a set number of synchronization barrier pools to store synchronization barriers to be reused, registers a virtual ION device during kernel initialization of the host system to create a virtual ION device node, and allocates a memory block set as a first memory pool; and binds the virtual ION device node with an ION path in the container when the container is started.

[0010] The conversion module monitors the HAL process, allocates the first structure body and initializes it when the HAL process acquires the FD of the ION memory allocator, binds the pointer of the first structure body to the ION file descriptor thereof, determines the target memory heap type, memory size, alignment and flag bit related to the DMA-Buf according to the first structure body when the HAL process initiates a memory allocation request, returns the memory pool pointer and the ION handle acquired from the first memory pool, returns the FD corresponding to the ION handle in the sharing parameter determined according to the FD cache table when the HAL process shares the ION memory with other components, acquires the DMA-Buf pointer corresponding to the FD according to the FD when the HAL process imports the ION handle of another process, generates and returns the imported ION handle, adds the mapping relationship to the first structure body, determines the release mode according to the DMA-Buf pointer when the HAL process releases the memory, releases the spin lock and restores the interrupt, and marks the orphan FD when the reference count of the first structure body is greater than zero when the HAL process exits, and registers a delayed callback function.

[0011] Further, the first structure includes a first hash table, a memory heap mapping table, a reference count array, a security label, a memory pointer, and an import handle and FD mapping table, wherein the first hash table is a mapping table between an ION handle and a DMA-Buf pointer, the memory heap mapping table is a mapping table of a source memory heap type to a target memory heap type, the reference count array is a reference count array of a shared FD, the security label is a SELinux label of an Android HAL process, and the memory pointer is a DMA-Buf pointer associated with an ION handle.

[0012] Further, a way of creating a virtual ION device node when a host system kernel is initialized is as follows: a primary and secondary device number of the virtual ION device is applied and bound as a dedicated device number through a Linux character device number registration mechanism; a set of custom device operation interfaces is initialized, and the interface set is associated to the dedicated device number; a virtual ION device node is created under a corresponding directory of the host system; when a container is started, the virtual device node is bound to a standard ION path in the container, and a device access hook of a virtual memory file system is registered.

[0013] Further, a way of allocating a first structure and initializing the first structure when the ION memory allocator acquires an FD of the ION memory allocator is as follows: an open function of a HAL process triggers an open callback function of a conversion module; the callback function first checks whether the process is a container process by comparing a process namespace identifier; if the process is not a container process, a denied access is returned; if the process is a container process, a first structure memory is allocated, and core fields of the first structure are initialized.

[0014] Further, when a memory allocation request is initiated, a target memory heap type, a memory size, an alignment, and a flag bit related to a DMA-Buf are determined according to the first structure; if the memory size and a source memory heap type of the memory allocation request are matched with a memory block in a first memory pool, a corresponding memory pool pointer and a pre-generated ION handle are obtained from the first memory pool; otherwise, a full allocation process of the target memory heap is performed to obtain a memory pointer and an ION handle.

[0015] Further, the full allocation process of the target memory heap is as follows: a conversion module calls a dma_heap_alloc to allocate a target memory heap and obtain a buffer pointer; a unique ION handle is generated in a manner that a high 8-bit is a target memory heap type and a low 24-bit is a global self-incrementing serial number; a mapping relationship between the ION handle and the DMA-Buf pointer is added to the first structure through hash_add_rcu, and an RCU mechanism is enabled to guarantee safety of a read operation.

[0016] Further, the conversion module monitors the HAL process, when it maps ION memory to user space, parses the command to extract the mapped memory size, access permission, mapping type and ION handle, determines the DMA-Buf pointer corresponding to the ION handle according to the first hash table, converts the access permission into the permission supported by the DMA-Buf, sets the attribute of the mapping page table according to the attribute of the DMA-Buf pointer, maps the physical memory determined by the DMA-Buf pointer to the virtual address of the user space by using the message of the DMA-Buf for mapping memory to the user space, and returns the start address of the virtual address.

[0017] Further, the conversion module monitors the HAL process, when it synchronizes ION memory data between the CPU and the hardware device, parses the synchronization parameters to obtain the ION handle and the synchronization direction, determines the DMA-Buf pointer corresponding to the ION handle according to the first hash table; checks the synchronization record in the private data of the DMA-Buf pointer, if the difference between the current time and the timestamp of the last record is less than a threshold value, the current synchronization operation is ignored and a synchronization success message is returned; otherwise, the synchronization direction from the CPU to the device is converted into the synchronization write of the DMA-Buf, the synchronization direction from the device to the CPU is converted into the synchronization read of the DMA-Buf, the bidirectional synchronization is converted into the synchronization read and write of the DMA-Buf, the corresponding DMA-Buf synchronization interface is called according to the hardware architecture, the bottom cache operation instruction is triggered, and the timestamp and the synchronization direction of this synchronization are saved to the private data of the DMA-Buf pointer.

[0018] Further, the conversion module monitors the HAL process, when it creates an ION synchronization barrier, if there is a free DMA synchronization barrier in the synchronization barrier pool, the DMA synchronization barrier is obtained, otherwise the DMA synchronization barrier is created and added to the synchronization barrier pool; the initialization of the DMA synchronization barrier is completed, the barrier FD associated with the DMA synchronization barrier is generated, and the barrier FD is returned; when the HAL process sets the blocking waiting, the conversion module waits for the DMA synchronization barrier trigger in a loop until the barrier trigger or the process is interrupted.

[0019] Further, the conversion module monitors the HAL process, when it delivers the FD, parses the message to obtain the FD, checks whether it exists in the FD cache table, if it exists, creates a new FD same as the FD in the PID namespace of the receiving process for the receiving process, and increments the reference count of the FD; otherwise, the message is ignored; when it receives the FD, allocates the container FD for it, and returns it to the HAL process. Advantageous effects

[0020] The application takes the DMA-Buf memory heap of a Linux kernel host system as a target memory heap, takes the ION memory heap of an Android container system as a source memory heap, creates a process-specific memory management context, an FD cache table and a synchronization barrier pool, completes kernel registration and node binding of a virtual ION device, first memory pool pre-allocation and kernel layer device access hook registration, checks legality of a container process when an ION file descriptor is acquired by a HAL process and initializes context binding to the file descriptor; then intercepts a memory allocation request of the HAL process, parses and adapts parameters, and obtains an ION handle by preferentially reusing first memory pool resources; when a data sharing request is processed, an FD cache is matched or a new FD is generated through the ION handle, and an effective FD is reconstructed for a receiving process when data is transmitted across namespaces; meanwhile, ION handle import, memory mapping to a user space, CPU and hardware data synchronization, synchronization barrier creation and other operations are adapted, and ION related requests are converted into DMA-Buf corresponding logic; finally, resources are recycled according to a memory source when memory is released by the HAL process, and context is cached or destroyed when the HAL process exits, thereby realizing cross-system memory operation compatibility. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a container-in-Android camera HAL compatible running method based on memory access virtualization is provided in the application. DETAILED DESCRIPTION

[0022] The application is described in detail below in combination with the drawings.

[0023] The container-in-Android camera HAL compatible running method based on memory access virtualization provided by the application has the following core idea: taking the DMA-Buf memory heap of a Linux kernel host system as a target memory heap and taking the ION memory heap of an Android container system as a source memory heap, a process-specific memory management context, an FD cache table and a synchronization barrier pool are created, kernel registration and node binding of a virtual ION device are completed, first memory pool pre-allocation and kernel layer device access hook registration are completed, legality of a container process is checked when an ION file descriptor is acquired by a HAL process and context is initialized and bound to the file descriptor; then a memory allocation request of the HAL process is intercepted, parameters are parsed and adapted, and an ION handle is obtained by preferentially reusing first memory pool resources; when a data sharing request is processed, an FD cache is matched or a new FD is generated through the ION handle, and an effective FD is reconstructed for a receiving process when data is transmitted across namespaces; meanwhile, ION handle import, memory mapping to a user space, CPU and hardware data synchronization, synchronization barrier creation and other operations are adapted, and ION related requests are converted into DMA-Buf corresponding logic; finally, resources are recycled according to a memory source when memory is released by the HAL process, and context is cached or destroyed when the HAL process exits, thereby realizing cross-system memory operation compatibility.

[0024] The application provides a container-in-Android camera HAL compatible running method based on memory access virtualization, and a flowchart is shown in Figure 1 The method comprises the following steps:

[0025] Step 1, taking a Linux kernel-based system as a host system, taking a DMA-Buf memory heap as a target memory heap, taking an Android system as a container system, taking an ION memory heap as a source memory heap, and constructing a kernel-mode conversion module in the host system;

[0026] The conversion module establishes a first structure body for saving a dedicated memory management context of an Android camera HAL process, establishes an FD cache table for saving a mapping relationship between an ION handle and a DMA-Buf file descriptor FD, and creates a set number of synchronization barrier pools for storing synchronization barriers to be reused; kernel-level registration of a virtual ION device is completed when the host system kernel is initialized, a virtual ION device node is created, and a set of fixed-size memory blocks in the target memory heap are pre-allocated as a first memory pool; the virtual ION device node on the host side is bound to a standard ION path in the container when the container is started, and a temporary file system device access hook is registered in the kernel layer;

[0027] When the HAL process acquires a file descriptor of an ION memory allocator, the conversion module detects whether the current HAL process is a container-in process, and returns a permission denial error if the current HAL process is not a container-in process, otherwise, the first structure body is allocated to the current HAL process and field initialization is completed; the pointer of the first structure body is bound to the ION file descriptor of the current HAL process.

[0028] The first structure body is used for storing the mapping relationship of the process ION, the security context and the resource state, and comprises a first hash table, a memory heap mapping table, a reference count array, a security label, a memory pointer and an import handle and an FD mapping table, specifically, the first hash table is a mapping table between the ION handle and the DMA-Buf pointer, the memory heap mapping table is a mapping table of the source memory heap type to the target memory heap type, the reference count array is a reference count array of the shared FD, the security label is a SELinux label of the Android HAL process, and the memory pointer is a DMA-Buf pointer associated with the ION handle, so as to ensure process-level resource isolation and state tracking.

[0029] Further, the kernel-level registration of the virtual ION device is specifically as follows:

[0030] By the Linux character device number registration mechanism, the primary and secondary device numbers of the virtual ION device are applied and bound as exclusive device numbers; a self-defined device operation interface set is initialized, and the interface set is associated to the exclusive device numbers, and a virtual ION device node is created under the host system / dev / ion directory; when the container is started, the virtual device node is bound to the standard ION path in the container, and the device access hook of the virtual memory file system is registered, so that all access requests pointing to / dev / ion are routed to the self-defined operation interface of the conversion module instead of the original ION driver.

[0031] Further, the current HAL process creates and initializes an exclusive memory management context, specifically:

[0032] The open function call of the current HAL process triggers the open callback function of the conversion module, which first checks whether it is a container process by comparing the process namespace identifier, and returns access denied for non-container processes, and allocates first structure memory for container processes, initializes its core fields, including enabling RCU mechanism, first hash table based on jhash algorithm, dynamically updateable memory heap mapping table, memory heap mapping table recording FD reference count, SELinux security context of HAL process, and first memory pool pre-allocated with memory blocks according to common specifications; finally, the pointer to the initialized first structure is bound to the private data of the ION file descriptor of the current process, providing context support for subsequent operations.

[0033] Further, the conversion module maintains a global context cache pool that can cache up to 16 free first structures. When the HAL process calls the open function again, the conversion module preferentially reuses the free context in the cache pool, only updating the associated process ID of the SELinux security context and the memory pool pointer; if there is no free context, a new one is created and added to the cache pool, and the oldest free context is discarded when the cache pool is full. This significantly reduces the open operation delay.

[0034] Step 2, the conversion module intercepts the current HAL process's memory allocation request, denoted as the source memory allocation request, parses the memory allocation parameters including source memory heap type, memory size, alignment and flag bit, extracts the source memory heap type with the highest priority, and converts it to the corresponding target memory heap type according to the memory heap mapping table in the first structure; if the memory size is an integer multiple of the minimum granularity supported by the target memory heap, it remains unchanged, otherwise the memory size is rounded up; if the alignment is zero, the default alignment of the target memory heap is used, if the alignment is greater than the default alignment, the alignment is used as the parameter for the target memory heap allocation; the flag bit is converted to the attribute of DMA-Buf;

[0035] If the memory size of the memory allocation request and the source memory heap type both match the memory block in the first memory pool, the corresponding memory pool pointer and the pre-generated ION handle are obtained from the first memory pool; otherwise, the full allocation process of the target memory heap is performed to obtain the memory pointer and the ION handle; and the ION handle is taken as the handle of the source memory allocation request.

[0036] In the full process target memory heap allocation, the conversion module calls dma_heap_alloc to allocate the target memory heap and obtain a buffer pointer, generates a unique ION handle in the manner of taking the high 8 bits as the target memory heap type and the low 24 bits as the global incremental serial number, adds the mapping relationship between the ION handle and the DMA-Buf pointer to the first hash table of the first structure body by means of hash_add_rcu, and enables the RCU mechanism to guarantee the safety of read operation.

[0037] In step 3, when the conversion module monitors the source data sharing request of the ION memory shared with other components by the current HAL process, the conversion module obtains the ION handle from the sharing parameter, looks up the ION handle in the first hash table of the first structure body, returns the error information of the native ION if the ION handle does not exist, looks up the FD cache table to check whether the FD corresponding to the ION handle exists if the ION handle exists, obtains the FD and increments the reference count of the FD if the FD exists, performs the full generation process of the FD to obtain the FD and saves the corresponding relationship between the FD and the ION handle to the FD cache table if the FD does not exist, and takes the obtained FD as the FD of the source data sharing request.

[0038] When the conversion module monitors the transfer FD message sent by the current HAL process, the conversion module parses the message to obtain the FD, looks up whether the FD exists in the FD cache table, creates a new FD identical to the FD in the PID namespace of the receiving process and increments the reference count of the FD if the FD exists, and ignores the message if the FD does not exist.

[0039] When the conversion module monitors the receive FD message sent by the current HAL process, the conversion module returns the container-side FD allocated for the receiving process to the current HAL process.

[0040] In the full generation process of the FD, the conversion module generates the FD with the O_CLOEXEC and O_RDWR flags, associates the callback functions of mmap and release, maps the security label of the container side to the label of the host side based on the SELinux context in the first structure body, sets the label for the FD to prevent unauthorized access, stores the FD into the reference count array and initializes the reference count to 1, and finally adds the mapping relationship between the ION handle and the FD to the FD cache table.

[0041] Step 4, when the conversion module monitors that the current HAL process imports other process ION handle, the command is parsed to obtain FD, and the corresponding DMA-Buf pointer is obtained according to the private data of the FD related file; the target memory heap type is determined by the DMA-Buf pointer, the corresponding source memory heap type is determined according to the memory heap mapping table, the import ION handle is generated, and the mapping relationship between the import ION handle and the DMA-Buf pointer is added to the import ION handle and the FD mapping table, and the generated import ION handle is returned to the current HAL process.

[0042] Step 5, when the conversion module monitors that the current HAL process maps ION memory to the user space, the command is parsed to extract the mapping memory size, access permission, mapping type and ION handle, the DMA-Buf pointer corresponding to the ION handle is determined according to the first hash table, the access permission is converted into the permission supported by the DMA-Buf, the mapping page table attribute is set according to the DMA-Buf pointer attribute, the physical memory determined by the DMA-Buf pointer is mapped as a virtual address of the user space by using the DMA-Buf message for mapping the memory to the user space, and the obtained virtual starting address of the user space is returned to the current HAL process.

[0043] Step 6, when the conversion module monitors that the current HAL process synchronizes ION memory data between the CPU and the hardware device, the synchronization parameters are parsed to obtain the ION handle and the synchronization direction, and the DMA-Buf pointer corresponding to the ION handle is determined according to the first hash table; the synchronization record in the private data of the DMA-Buf pointer is checked, if the difference between the current time and the timestamp of the last record is less than a threshold value, the current synchronization operation is ignored and a synchronization success message is returned; otherwise, the synchronization direction from the CPU to the device is converted into the synchronization write of the DMA-Buf, the synchronization direction from the device to the CPU is converted into the synchronization read of the DMA-Buf, the bidirectional synchronization is converted into the synchronization read and write of the DMA-Buf, the corresponding DMA-Buf synchronization interface is called according to the hardware architecture, the bottom cache operation instruction is triggered, and the timestamp and the synchronization direction of this synchronization are saved in the private data of the DMA-Buf pointer.

[0044] Step 7, when the conversion module monitors that the current HAL process creates an ION synchronization barrier for adjusting the operation sequence of multiple devices, if there is a free DMA synchronization barrier in the synchronization barrier pool, the DMA synchronization barrier is obtained, otherwise the DMA synchronization barrier is created and added to the synchronization barrier pool; the initialization of the DMA synchronization barrier is completed, the barrier FD associated with the DMA synchronization barrier is generated, and the barrier FD is returned to the current HAL process; when the current HAL process is set to block and wait, the conversion module waits for the DMA synchronization barrier trigger in a loop until the barrier trigger or the process is interrupted.

[0045] Step 8, when the conversion module monitors the current HAL process to release memory, the command is parsed to obtain a handle array and a handle quantity, the handle array is traversed to determine the DMA-Buf pointer corresponding to the ION handle according to the first hash table, if the DMA-Buf pointer is natively allocated and comes from the first memory pool, the memory block corresponding to the DMA-Buf pointer is saved to the free list of the first memory pool, and the corresponding entry is deleted from the first hash table; if the DMA-Buf pointer is natively allocated and comes from a non-first memory pool, the reference count of the memory heap pointed to by the DMA-Buf pointer is decremented, and when the decrement is zero, the underlying physical memory is released; if the DMA-Buf pointer is an imported handle, the reference count of the memory heap pointed to by the DMA-Buf pointer is decremented, and the handle related entry is deleted from the imported handle and the FD mapping table, and the reference count of the related FD is decremented; the spin lock is released and the interrupt is restored;

[0046] When the conversion module monitors the current HAL process to exit, the reference count array of the first structure is traversed, the FDs with reference counts greater than zero are marked as orphan FDs, and a delayed recovery callback function is registered;

[0047] If the global context cache pool is not full, the first hash table is emptied, the reference count array is reset, the imported handle and the FD mapping table are emptied, the first structure is marked as idle and added to the cache pool; if the global context cache pool is full, the first hash table is destroyed, the SELinux context string is released, and the memory of the first structure is released. Embodiment

[0048] In this embodiment, taking OpenHarmony as the host system as an example, the method for compatible running of an Android camera HAL in a container based on memory access virtualization is adopted, the ION-DMA-Buf conversion acceleration layer and the refined process adaptation are adopted, efficient running of the camera HAL based on the ION mechanism in the Android system in the container is realized, and the specific process includes:

[0049] S1, virtual adaptation and context initialization of an Android camera HAL to open an ION device.

[0050] When the Android camera HAL is started, the open(" / dev / ion", O_RDWR) system call is first executed to try to open the ION device node. The conversion module realizes the pseudo existence of the ION device through virtual device mapping and process-specific context initialization, specifically:

[0051] S1.1, creation and mapping of a virtual ION device node.

[0052] The conversion module registers the character device number including the major device number 240 and the minor device number 0 in the OpenHarmony kernel initialization stage through register_chrdev_region(), and associates the custom file_operations structure including the open, ioctl, mmap, and release callbacks to the device through cdev_init() and cdev_add(), to finally create a virtual device node / dev / ion_emu in the / dev directory.

[0053] Through the device mapping configuration of the container runtime such as the ohos_container tool of OpenHarmony, the mount --bind / dev / ion_emu / dev / ion command is executed when the Android container is started, and the virtual node / dev / ion_emu of the host side is bound to the standard ION access / dev / ion in the container. When the kernel automatically routes the request to the file_operations of the conversion module instead of the original ION driver.

[0054] S1.2, initialization of the process-specific ion_dma_ctx context, i.e., the first structure.

[0055] When the HAL process calls the open(" / dev / ion", O_RDWR) function, the open callback function of the conversion module is triggered. The function first checks whether the current process belongs to an Android container by reading the nsproxy->mnt_ns namespace identifier of the process and comparing it with the namespace of the container. If not, it returns EPERM to reject access by non-container processes. If yes, it continues initialization.

[0056] ion_dma_ctx memory is allocated by calling kzalloc(sizeof(struct ion_dma_ctx), GFP_KERNEL), and the core fields are initialized:

[0057] ion_handle_map: The hash table is initialized by hash_init(), and the hash function jhash algorithm is used for lookup. The number of hash tables is set to 256 to balance memory occupancy and lookup efficiency, and the RCU (Read-Copy-Update) mechanism is enabled by hash_rcu_enable() to allow read operations to be executed without locks, and only write operations such as adding or deleting entries need to be locked to improve multi-threaded concurrent access performance.

[0058] dma_heap_map: Load the preset ION heap and DMA-Buf heap mapping table stored in the / etc / ion_dma_heap.map configuration file. For example, the mapping relationship includes: ION_HEAP_TYPE_SYSTEM to "system" (system heap of DMA-Buf), ION_HEAP_TYPE_CARVEOUT to "reserved" (reserved memory heap of DMA-Buf), ION_HEAP_TYPE_QCOM_ADSP to "reserved" (Qualcomm ADSP heap mapped to reserved heap), and ION_HEAP_TYPE_MEDIATEK_M4U to "system_contig" (MediaTek M4U heap mapped to contiguous system heap). This mapping table supports dynamic updates through the sysfs interface / sys / ion_emu / heap_mapping, such as echo "ION_HEAP_TYPE_CUSTOM=custom_heap" > / sys / ion_emu / heap_mapping.

[0059] fd_refcount: Initialize the reference count array atomic_t fd_refcount

[1024] with a size of 1024, all elements initialized to 0, to record the reference count of each DMA-Buf shared FD, to avoid FD leakage caused by abnormal process exit.

[0060] security_ctx: Obtain the SELinux security ID of the current HAL process through security_task_getsecid(current, &secid), and then convert it to a SELinux context string such as "u:r:camera_hal:s0" through security_secid_to_secctx(secid, &ctx, &ctx_len), and store it in the security_ctx field for subsequent permission mapping of DMA-Buf.

[0061] prealloc_pool: call ion_prealloc_pool_init() to initialize process-level pre-allocated memory pool, read configuration from / sys / ion_emu / prealloc_sizes according to common specifications of camera HAL, which contains 4MB, 8MB, 16MB by default, pre-allocate 5 memory blocks / specification to corresponding DMA-Buf heap, pre-generate ION handle for each memory block, high 8 bits are heap type identifier, low 24 bits are incremental serial number, store it together with DMA-Buf pointer in the linked list of prealloc_pool.

[0062] Associate the initialized ion_dma_ctx structure pointer to the file descriptor (ion_fd) of the current process through file->private_data, to ensure that subsequent ioctl, mmap, etc. operations can directly obtain the context.

[0063] S1.3, Context cache pool optimization.

[0064] The conversion module maintains a global context cache pool struct ion_dma_ctx *ctx_cache_pool

[16] , which can cache up to 16 idle ion_dma_ctx structures, covering the typical number of processes of Android camera HAL.

[0065] When HAL calls open(" / dev / ion", O_RDWR) again, the conversion module first checks if there is an idle context in the cache pool: if there is, it is directly reused, only updating the associated process ID of security_ctx and prealloc_pool, skipping the structure reallocation and hash table initialization steps, reducing the open operation delay from 100us to 10us; if not, a new context is created and added to the cache pool, and when the cache pool is full, the oldest idle context is discarded.

[0066] S2, Parameter analysis and DMA-Buf conversion when camera HAL allocates ION memory.

[0067] When HAL needs to allocate camera frame buffers, such as allocating 3 1080P frame buffers when initializing, it will call ioctl(ion_fd, ION_IOC_ALLOC, &alloc_data), where alloc_data is a struct ion_allocation_data structure containing heap_id_mask, size, align, flags, and handle fields. The conversion module implements ION memory allocation simulation through refined parameter analysis and DMA-Buf allocation, as follows:

[0068] S2.1, Full analysis of ION allocation parameters.

[0069] heap_id_mask parsing: heap_id_mask is a bitmap, 0x1 means ION_HEAP_TYPE_SYSTEM, 0x2 means ION_HEAP_TYPE_CARVEOUT, the conversion module extracts the highest priority ION heap type (ION supports multiple heap masks, the first matching heap is selected according to priority) by find_first_bit(&alloc_data->heap_id_mask, 32), and then queries ion_dma_ctx->dma_heap_map to get the corresponding DMA-Buf heap name, such as ION_HEAP_TYPE_CARVEOUT to "reserved".

[0070] For vendor-defined heaps, such as ION_HEAP_TYPE_QCOM_ADSP of Qualcomm, mask 0x1000, the conversion module stores the vendor alias in the extended field of dma_heap_map, maps it to the "reserved" heap of DMA-Buf, and appends the DMA_BUF_ATTR_QCOM_ADSP_COMPAT attribute when allocating to inform the DMA-Buf heap to enable ADSP hardware compatibility mode.

[0071] size and align adaptation:

[0072] size processing: first check if alloc_data->size is an integer multiple of the minimum granularity supported by the DMA-Buf heap, such as system heap minimum granularity of 4KB, reserved heap minimum granularity of 2MB, if not, automatically round up, such as 3MB request rounded to 4MB, to avoid memory fragmentation. At the same time, if the size exceeds the maximum supported capacity of the DMA-Buf heap, such as reserved heap maximum of 64MB, return ENOMEM to simulate ION memory shortage error.

[0073] align processing: if alloc_data->align is 0, it means no alignment is specified, then use the default alignment of DMA-Buf heap, which is 4KB; if align is greater than the default alignment, such as 16KB, then pass it as the align parameter of dma_heap_alloc to ensure that the allocated memory meets the hardware alignment requirements, such as camera ISP requires 16KB alignment.

[0074] Flag semantics conversion: convert the flags of ION to the attributes of DMA-Buf, ensure that the cache strategy, security attributes and access permissions of the memory match, and the specific mapping relationship is as follows:

[0075] ION_FLAG_CACHED to DMA_BUF_ATTR_CACHED: enable CPU cache, suitable for memory frequently read and written by CPU, such as camera preview frame;

[0076] ION_FLAG_UNCACHED to DMA_BUF_ATTR_UNCACHED: disable CPU cache, suitable for memory directly accessed by hardware devices such as ISP, to avoid cache consistency problems;

[0077] ION_FLAG_SECURE to DMA_BUF_ATTR_SECURE: enable the security isolation of OpenHarmony TEE trusted execution environment, trigger TEE memory allocation through the flags parameter of dma_heap_alloc, and ensure that the memory can only be accessed by authorized hardware or processes, such as encrypted camera data;

[0078] ION_FLAG_CONTIGUOUS to DMA_HEAP_ALLOC_CONTIG: force to allocate physically contiguous memory, implemented through the CMA (Contiguous Memory Allocator) backend of DMA-Buf, and the physical start address is stored in the private data dma_buf->priv->phys_addr of dma_buf after allocation, which is directly used by hardware devices such as camera sensors that require physically contiguous addresses;

[0079] ION_FLAG_DMA_BUF to ignore the native support of FD sharing of DMA-Buf, no additional flags are needed.

[0080] S2.2, DMA-Buf memory allocation and ION handle mapping.

[0081] Pre-allocated pool matching: first check whether the size and heap type of alloc_data match the memory blocks in ION_dma_ctx->prealloc_pool, with an error tolerance of ±0.5MB, if they match, directly get the corresponding DMA-Buf pointer and pre-generated ION handle from the pool, skip the subsequent allocation process, and call list_del() to move the memory block from the free list to the allocated list; if they do not match, perform the full process allocation.

[0082] Full-process DMA-Buf allocation:

[0083] Call dma_heap_get(alloc_data->dma_heap_name) to get the struct dma_heap * pointer of the target DMA-Buf heap, return EINVAL if the heap does not exist (such as configuration error);

[0084] Call dma_heap_alloc(heap, size, align, flags, GFP_KERNEL) to allocate DMA-Buf memory, where flags contains the converted DMA-Buf attributes such as DMA_BUF_ATTR_CONTIGUOUS, and this function returns the DMA-Buf buffer pointer struct dma_buf *dma_buf_ptr;

[0085] Generate a unique ION handle: generate a handle in the format of high 8 bits heap type and low 24 bits self-incrementing sequence number, such as ION_HEAP_TYPE_SYSTEM (0x01) corresponding to sequence number 100, handle 0x01000064, where the self-incrementing sequence number is obtained through the global atomic variable atomic_inc_return(&ion_handle_seq), ensuring that the handle is globally unique;

[0086] Hash table mapping: call hash_add_rcu(ion_handle_map, &entry, handle) to add the mapping relationship between the ION handle and dma_buf_ptr to the ion_handle_map hash table, where entry is a struct hash_entry structure that stores handle and dma_buf_ptr, and the RCU mechanism is enabled to ensure the safety of read operations during the addition process.

[0087] Result return: assign the generated ION handle to alloc_data->handle, and return 0 to indicate success, and the HAL can perform sharing, mapping and other operations through the handle subsequently.

[0088] S3, when the camera HAL shares ION memory, FD conversion and cross-container delivery.

[0089] When HAL needs to share the allocated frame buffer to other components, such as sharing the preview frame to the GPU for rendering, or sharing to the ISP for image processing, ioctl(ION_fd, ION_IOC_SHARE, &share_data) is called, where share_data is a struct ion_share_data structure containing the handle to be shared ION handle and the output shared FD fd field. The conversion module realizes the sharing of ION memory through DMA-Buf FD generation and cross-namespace delivery, which is specifically:

[0090] S3.1, ION sharing parameter verification and DMA-Buf matching.

[0091] Handle verification: Call hash_for_each_possible_rcu(ion_handle_map, entry, node, handle) to find the dma_buf_ptr corresponding to share_data->handle from ion_dma_ctx->ion_handle_map. If not found, such as invalid handle or released, return EINVAL; if found, continue the following process.

[0092] FD cache table matching: Check if there is a mapping entry of the ION handle in the global fd_cache_table (FD cache table). If it exists, directly get the corresponding DMA-Buf FD, call atomic_inc(&fd_refcount[fd]) to increment the reference count, and assign the FD to share_data->fd, skip the following FD generation process; if it does not exist, execute the FD generation.

[0093] S3.2, MA-Buf shared FD generation and permission adaptation.

[0094] Call dma_buf_export(dma_buf_ptr, O_CLOEXEC | O_RDWR, &dma_buf_ops) to generate DMA-Buf shared FD, where dma_buf_ops is the DMA-Buf operation structure defined by the conversion module, including mmap, release callback, which is used for subsequent mapping and resource recycling, and the O_CLOEXEC flag ensures that the FD is automatically closed when the process forks to avoid leakage.

[0095] SELinux permission mapping: According to ion_dma_ctx->security_ctx like "u:r:camera_hal:s0", call security_dma_buf_set_ctx(dma_buf_ptr, security_ctx) to set the SELinux security label of OpenHarmony side for the DMA-Buf FD. For example, map the camera_hal label of Android to the ohos_camera_device label of OpenHarmony, to ensure that the SELinux policy of the host side, such as allowing the display controller to access the memory of the ohos_camera_device label, can take effect, preventing unauthorized access.

[0096] Reference count initialization: Store the generated FD into the ion_dma_ctx->fd_refcount array, and call atomic_set(&fd_refcount[fd], 1) to initialize the reference count to 1, indicating that the HAL process itself holds the FD.

[0097] FD cache table addition: Call hash_add(&fd_cache_table, &cache_entry, handle) to add the mapping relationship between the ION handle and the FD to the global fd_cache_table. The cache table entry contains handle, fd, dma_buf_ptr, and expire_time. The expire time is 5 minutes by default. When the entry expires and the reference count is 0, it is automatically deleted from the cache table to avoid memory occupation.

[0098] S3.3, Cross-container FD transfer mechanism.

[0099] Since the Android container and the OpenHarmony host belong to different process IDs, namely, PID namespaces and mount namespaces, the directly transferred FD may be invalid in the container, that is, the host FD and the container FD numbers do not correspond. The conversion module implements cross-namespace adaptation of the FD by hooking the sys_sendmsg system call processing logic of the kernel:

[0100] When the HAL process such as the camera_hal process sends a message containing the SCM_RIGHTS control information for transferring the FD through sendmsg, the sendmsg hook function of the conversion module is triggered. First, check whether the FD in the message is the DMA-Buf FD generated by the conversion module, and verify it by querying the fd_cache_table.

[0101] If the FD is generated by the conversion module, get_task_struct(recv_pid) is called to obtain the task_struct of the receiving process, such as the GPU process, and fd_install(recv_fd, file) is called to create a new FD for the receiving process in its PID namespace, the number of which is determined by the file descriptor table of the process within the container, and the new FD is associated with the file structure of the original DMA-Buf.

[0102] atomic_inc(&fd_refcount[original FD]) is called to increment the reference count of the original FD, ensuring that the DMA-Buf memory is not released during the receiving process's holding of the FD.

[0103] If the receiving process calls recvmsg within the container to receive the FD, the container FD allocated by the conversion module is directly obtained, and subsequent mmap, ioctl, and other operations can be performed through this FD, consistent with the native ION shared experience.

[0104] S4, camera HAL imports external ION handle conversion adaptation.

[0105] When the HAL process needs to import an ION handle shared by other processes, such as a video encoding process, such as encoded frame data, ioctl(ION_fd, ION_IOC_IMPORT, &import_data) is called, where import_data is a structure_import_data structure containing external shared FD and output imported ION handle handle fields. The conversion module imports and reconstructs the ION handle through the FD of the DMA-Buf, achieving compatibility with external ION handles, specifically:

[0106] S4.1, external FD verification and DMA-Buf import.

[0107] FD legality check: First, check if import_data->fd is a valid file descriptor. Call fget(import_data->fd) to obtain the corresponding struct file *file. If file is NULL or file->f_op does not match the DMA-Buf operation structure, return EINVAL indicating that the FD is not of DMA-Buf type.

[0108] DMA-Buf pointer extraction: Get the externally shared DMA-Buf pointer struct dma_buf *dma_buf_ptr through file->private_data, and call dma_buf_get(dma_buf_ptr) to increment the reference count of the DMA-Buf, ensuring that the DMA-Buf is not released during import.

[0109] Permission check: Call security_dma_buf_check_access(dma_buf_ptr, current, MAY_READ | MAY_WRITE) to check whether the current HAL process has the right to access the DMA-Buf based on the SELinux label. If not, return EPERM to ensure the safety of memory access.

[0110] S4.2, ION handle reconstruction and mapping.

[0111] Import handle generation: Generate an ION handle dedicated to import in the format of high 8-bit heap type and low 24-bit self-incrementing sequence number. The heap type is obtained through dma_buf_ptr->heap->name reverse lookup dma_heap_map, such as the reserved heap of DMA-Buf corresponding to ION_HEAP_TYPE_CARVEOUT.

[0112] Hash table mapping addition: Call hash_add_rcu(ion_handle_map, &entry, import_handle) to add the mapping relationship between the import handle and dma_buf_ptr to ion_dma_ctx->ion_handle_map, and mark the entry as imported by setting entry->is_imported = true, which is convenient for distinguishing between native allocation and import when releasing.

[0113] FD reference count management: Increment the reference count of import_data->fd by 1, and record the association between the FD and the import handle in the import_fd_map linked list of ion_dma_ctx, which is convenient for synchronous cleaning when releasing.

[0114] Result return: Assign the generated import handle to import_data->handle, and return 0. The HAL can subsequently perform mapping, cache synchronization, and other operations on the imported memory through the handle, which is consistent with the native ION import logic.

[0115] S5, Camera HAL maps ION memory to user space.

[0116] When HAL needs to map allocated or imported ION memory to user space for direct read / write frame data for image pre-processing, mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, ION_fd, handle) is called. The conversion module adapts the mmap interface of DMA-Buf to achieve user space mapping of ION memory. The specific process is as follows:

[0117] S5.1, ION mapping parameter analysis and DMA-Buf matching.

[0118] Parameter analysis: Extract the mapping size size, access permission prot such as PROT_READ | PROT_WRITE, mapping type flags such as MAP_SHARED, and ION handle handle from the mmap call, where ion_fd is the opened file descriptor, and the ion_dma_ctx context is obtained through file->private_data.

[0119] DMA-Buf pointer lookup: Call hash_for_each_possible_rcu(ion_handle_map, entry, node, handle) to find the corresponding dma_buf_ptr from ion_dma_ctx->Ion_handle_map. If not found or size exceeds the actual size of DMA-Buf, return EINVAL.

[0120] S5.2, DMA-Buf mapping and cache attribute adaptation.

[0121] Mapping permission conversion: Convert the prot parameter of mmap to the permission supported by DMA-Buf, for example: PROT_READ to DMA_BUF_ACCESS_READ, PROT_WRITE to DMA_BUF_ACCESS_WRITE, and PROT_EXEC to EPERM, which prohibits execution permission and complies with the memory safety policy.

[0122] Cache attribute adaptation: Set the page table attribute of mmap according to the attribute of dma_buf_ptr, such as DMA_BUF_ATTR_UNCACHED.

[0123] If the attribute is UNCACHED, pgprot_noncached(PAGE_KERNEL) is called to set the mapping page table to non-cached mode, ensuring that user space read and write directly operate physical memory without CPU cache; if the attribute is CACHED, the default page table attribute PAGE_KERNEL is used, enabling CPU cache to improve read and write performance.

[0124] Call the mmap interface of DMA-Buf: call dma_buf_mmap(dma_buf_ptr, vma), where vma is a struct vm_area_struct structure containing the mapped virtual address range, permissions, and other information. This function maps the DMA-Buf physical memory to the user space virtual address and associates the vma vm_ops containing fault, page_mkwrite, and other callbacks to handle page faults and write protection.

[0125] Mapping address return: After dma_buf_mmap executes successfully, the virtual start address of the user space is returned. HAL can directly read and write ION memory, which is actually DMA-Buf memory, through this address, consistent with the original ION mmap experience.

[0126] S5.3, mapping performance optimization.

[0127] Large page mapping: If the DMA-Buf memory is allocated with 2MB large pages, enable large page mapping when mmap by vma->vm_flags |= VM_HUGETLB, reduce the number of page table entries, such as 1 2MB large page corresponding to 1 page table entry instead of 512 4KB small pages, reduce the TLB (Translation Lookaside Buffer) miss rate, and improve memory access speed.

[0128] Pre-mapping: For memory specifications frequently mapped by HAL, such as 4MB, the conversion module completes the kernel mapping of DMA-Buf in advance through dma_buf_kmap(dma_buf_ptr, 0) when initializing the pre-allocated pool. When HAL calls mmap, directly map the kernel mapping address to user space, reducing the mapping delay, such as from 50μs to 10μs.

[0129] S6, ION cache synchronization conversion during camera data transmission.

[0130] When HAL needs to synchronize ION memory data between CPU and hardware devices such as camera ISP, GPU, such as CPU writes a preview frame and notifies ISP to read, or ISP processing is completed and notifies CPU to read, ioctl(ION_fd, ION_IOC_SYNC, &sync_data) is called, where sync_data is a struct ion_sync_data structure containing ION handle handle and synchronization direction direction such as ION_SYNC_FOR_DEVICE / ION_SYNC_FOR_CPUde and other fields. The conversion module realizes the compatibility of ION cache synchronization through the cache synchronization interface of DMA-Buf, and the specific process is as follows:

[0131] S6.1, ION synchronization parameter analysis and DMA-Buf matching.

[0132] Handle and direction analysis: Extract handle and direction from sync_data, call hash_for_each_possible_rcu(ion_handle_map, entry, node, handle) to find the corresponding dma_buf_ptr, if not found, return EINVAL.

[0133] Synchronization direction verification: Check if the direction is of the supported type ION_SYNC_FOR_DEVICE / ION_SYNC_FOR_CPU / ION_SYNC_FOR_DEVICE_WRITE, if not, return EINVAL.

[0134] S6.2, DMA-Buf cache synchronization and hardware instruction adaptation.

[0135] Synchronization direction conversion: Convert the synchronization direction of ION to the synchronization type of DMA-Buf:

[0136] ION_SYNC_FOR_DEVICE is converted to DMA_BUF_SYNC_WRITE: Ensure that the data in the CPU cache has been written to the physical memory, and the device can read the latest data.

[0137] ION_SYNC_FOR_CPU is converted to DMA_BUF_SYNC_READ: Ensure that the latest data written by the device has been synchronized to the physical memory, and the CPU reads without old data in the cache.

[0138] ION_SYNC_FOR_DEVICE_WRITE bidirectional synchronization to DMA_BUF_SYNC_RW: read and write synchronization are performed at the same time, which is suitable for the scenario of bidirectional interaction between CPU and device.

[0139] Hardware architecture adaptation: according to the current hardware architecture such as ARMv8-A and x86_64, the corresponding DMA-Buf synchronization interface is called to trigger the underlying cache operation instruction:

[0140] For the ARMv8-A architecture, dma_buf_sync(dma_buf_ptr, 0, size, sync_type) is called, which will trigger the cache operation in the ARM instruction set at the bottom layer:

[0141] DMA_BUF_SYNC_WRITE to DC CVAC (Data Cache Clean by Virtual Address to PoC): clean the CPU cache and write data to the physical memory; DMA_BUF_SYNC_READ to DC CIVAC (Data Cache Clean and Invalidate by Virtual Address to PoC): clean and invalidate the CPU cache to ensure that the latest data in the physical memory is read.

[0142] Multi-core synchronization: if the system is a symmetric multiprocessor (SMP) architecture such as an eight-core CPU, smp_call_function_single(cpu, cache_sync_func, data, 1) is called to distribute the cache synchronization operation to all CPU cores, ensuring multi-core cache consistency and avoiding reading old data by core B when the cache data of core A is not synchronized.

[0143] Synchronization state recording: record the time current jiffies timestamp and direction of this synchronization to the private data dma_buf->priv->last_sync of dma_buf_ptr, which is used for subsequent synchronization operation merging.

[0144] S6.3, synchronization operation merging optimization.

[0145] Merging condition judgment: check whether the difference between dma_buf_ptr->priv->last_sync.time and the current time is less than 1ms, which can be configured through the sysfs interface / sys / ion_emu / sync_merge_ms, and the synchronization direction is consistent with this time.

[0146] Merge execution: if the merge condition is met, return 0 directly to simulate synchronization success, and do not perform the actual cache operation; if not, perform the synchronization process of step 6.2, and update the time and direction of last_sync. This mechanism can reduce the overhead of repeated synchronization in a short time, for example, when the camera preview is 30fps, only 1 synchronization is required per frame instead of multiple times, to reduce the bus bandwidth occupation.

[0147] S7, ION fence to dma_fence conversion when multiple devices cooperate.

[0148] When the HAL process needs to manage the operation sequence of multiple device cooperation, such as the frame data pipeline of camera sensor to ISP to GPU, it needs to ensure that the GPU renders after the ISP processing is completed, and the ION synchronization fence is created through the sync_fence mechanism of Android. The conversion module realizes the compatibility of ION fence through the dma_fence kernel synchronization mechanism, specifically:

[0149] S7.1, dma_fence conversion of ION fence creation.

[0150] Fence creation interception: when the HAL process calls sync_file_create("ion_fence") to create an ION fence, the conversion module intercepts the request by hooking the sys_sync_file_create system call, and judges whether the process belongs to the Android container and the fence name contains ion_fence.

[0151] dma_fence object acquisition: get the idle struct dma_fence *fence from the fence pooling object fence_pool, if there are not enough idle objects in the pool, call dma_fence_alloc(sizeof(struct dma_fence), GFP_KERNEL) to create a new object and add it to the pool.

[0152] dma_fence initialization:

[0153] context field: call dma_fence_context_alloc(1) to get an independent context ID from the global context pool, each conversion module instance corresponds to a context, ensuring that the fence is globally unique;

[0154] seqno field: call the global atomic variable atomic_inc_return(&fence_seq) to get the incremental sequence number, to avoid sequence number duplication in concurrent scenarios;

[0155] ops field: associated with a custom struct dma_fence_ops structure, which contains three core callbacks: wait, signal, and release, which trigger the wait fence, mark the fence completion, and release the fence resource, respectively.

[0156] wait callback: based on wait_queue_head_t, when the process calls poll to wait for the fence, it calls prepare_to_wait(&fence->wq, &wait, TASK_INTERRUPTIBLE) to add the process to the waiting queue and set the process state to interruptible wait; if the fence has been triggered fence->flags & FENCE_FLAG_SIGNALED, return 0 directly.

[0157] signal callback: after the hardware completes the operation, dma_fence_signal(fence) is called, which sets fence->flags |= FENCE_FLAG_SIGNALED and calls wake_up_all(&fence->wq) to wake up all processes in the waiting queue, notifying the fence completion.

[0158] release callback: kfree(fence) is called to release the fence memory and mark it as free, returning it to fence_pool.

[0159] Fence FD generation: dma_fence_fd_get(fence) is called to generate a file descriptor associated with dma_fence, fence FD, and return it to HAL, which can subsequently perform wait or signal operations through the FD, consistent with the original ION fence FD.

[0160] S7.2, conversion of ION fence wait and signal.

[0161] Fence wait conversion: when HAL calls poll(fence_fd, 1, -1) to wait for the fence trigger, the poll callback function of the conversion module is triggered, calling dma_fence_wait(fence, false), which indicates non-blocking wait. If the fence is not triggered, return EAGAIN, and if it is triggered, return 0. If HAL sets the blocking wait timeout to -1, loop calls dma_fence_wait until the fence is triggered or the process is interrupted.

[0162] Fence signal conversion: When the hardware device such as ISP completes data processing, it will notify HAL to trigger the fence through the driver callback. The conversion module intercepts the notification through the completion callback of the hardware driver, calls dma_fence_signal(fence) to mark the corresponding dma_fence as triggered, and automatically wakes up the waiting process.

[0163] S7.3, cross-device synchronization chain construction.

[0164] When cross-container or cross-device synchronization is required, such as the display controller on the OpenHarmony side waiting for the GPU fence on the Android side, the conversion module constructs a synchronization chain through the dma_fence_chain mechanism:

[0165] dma_fence_chain_alloc(fence_android, fence_ohos) is called to create a synchronization chain, where fence_android is the dma_fence corresponding to the ION fence on the Android side, and fence_ohos is the dma_fence of the display controller on the OpenHarmony side.

[0166] The synchronization chain ensures that fence_ohos is only executed after fence_android is triggered, avoiding the display controller from reading frame data that has not been processed.

[0167] S8, camera HAL resource release and cleaning of ION device shutdown.

[0168] When the HAL process stops using the camera, such as when the application exits, it will execute ION_IOC_FREE to release the ION handle, close(ION_fd) to close the ION device. The conversion module associates resource recycling with context cleaning to achieve resource management without leakage, and the specific process is as follows:

[0169] S8.1, DMA-Buf resource recycling for ION handle release.

[0170] Release parameter analysis: When HAL calls ioctl(ION_fd, ION_IOC_FREE, &free_data), free_data contains a handle array handles and the number of handles num_handles, and the conversion module obtains ion_dma_ctx from file->private_data and traverses the handle array.

[0171] Batch hash table deletion:

[0172] spin_lock_irqsave(&ion_handle_map_lock, flags) is called to acquire the spinlock, hold it for a short time suitable for batch operations, and disable local interrupts to prevent concurrent access to the hash table by interrupt handlers.

[0173] The handle array is traversed, and for each handle:

[0174] The mapping entry in ion_handle_map is looked up, and the dma_buf_ptr and entry type (native allocation and import;

[0175] If it is a native allocation and comes from the preallocated pool, the memory block corresponding to dma_buf_ptr is put back into the free list of the preallocated pool, and hash_del_rcu(&entry->node) is called to delete the entry from the hash table;

[0176] If it is a native allocation and not from the preallocated pool, dma_buf_put(dma_buf_ptr) is called to decrement the reference count of the DMA-Buf, and when the count is 0, the underlying physical memory is automatically released.

[0177] If it is an import handle, dma_buf_put(dma_buf_ptr) is called to decrement the reference count, and the FD association of this handle is removed from the import_fd_map linked list, and atomic_dec(&fd_refcount[import_fd]) is called to decrement the FD reference count.

[0178] spin_unlock_irqrestore(&ion_handle_map_lock, flags) is called to release the spinlock and restore interrupts.

[0179] S8.2, Context cleanup when ION device is closed.

[0180] Reference count check: When the HAL process calls close(ION_fd), the release callback function of the conversion module is triggered, which first traverses the ion_dma_ctx->fd_refcount array to check if there are any FDs with reference count greater than 0, indicating that the shared FD is not closed. If so, mark these FDs as orphan FDs, and register the delayed recycling callback through call_rcu(&rcu_head, orphan_fd_cleanup), which will forcibly close the FD and release the associated resources after the RCU grace period, which is 100ms by default.

[0181] Context cache and destruction:

[0182] Check if the global context cache pool is not full, i.e. if there are less than 16 free contexts, if not, then:

[0183] Empty all entries of ion_handle_map by batch deletion with hash_for_each_safe; reset fd_refcount array to 0, empty import_fd_map linked list; mark ion_dma_ctx as free and add to cache pool;

[0184] If the cache pool is full, then:

[0185] Call hash_destroy(ion_handle_map) to destroy the hash table and release the bucket memory; call kfree(ion_dma_ctx->security_ctx) to release the SELinux context string; call kfree_rcu(ion_dma_ctx, rcu_head) to release the context structure memory after RCU grace period, ensuring safe concurrent access.

[0186] Asynchronous cleanup submission: encapsulate the above cleanup tasks as struct work_struct, and submit them to the custom high-priority work queue ion_cleanup_wq through queue_work(ion_cleanup_wq, &work). The high-priority work queue is created by alloc_workqueue("ion_cleanup_wq", WQ_HIGHPRI | WQ_UNBOUND, 0), ensuring that the cleanup tasks are executed in priority, avoiding long-term memory occupation.

[0187] The related terms involved in this embodiment are as follows:

[0188] ION-DMA-Buf conversion acceleration layer: refers to a modular architecture implemented in the kernel mode, which is used to intercept the requests of processes in the Android container to the ION interface, such as open, ioctl, mmap, and convert them into standardized operations of DMA-Buf. This layer contains three sub-layers: interface adaptation sub-layer for intercepting system calls, parameter conversion sub-layer for parsing ION parameters and mapping them into DMA-Buf attributes, and performance optimization sub-layer for pre-allocated pool, cache reuse, etc. The overall architecture is a kernel mode direct connection architecture without user mode intermediate layer.

[0189] ion_dma_ctx context structure: a dedicated data structure created by the conversion module for each Android process accessing / dev / ion, used to store the process's ION-DMA-Buf mapping relationship, security context, and resource state. Each structure contains five core fields: the hash table ion_handle_map of ION handles and DMA-Buf pointers, the mapping table dma_heap_map of ION heap and DMA-Buf heap, the reference count array fd_refcount of shared FDs, the SELinux label security_ctx of the Android process, and the process-level pre-allocated memory pool pointer prealloc_pool, ensuring process-level resource isolation and state tracking.

[0190] ION-specific pre-allocated memory pool: when initializing, the conversion module pre-allocates a fixed-size memory block set to the DMA-Buf heap for commonly used memory specifications by the camera HAL, such as 4MB for 1080P frame buffers and 16MB for 4K frame buffers. Each memory block is associated with a pre-generated ION handle and DMA-Buf pointer. When the HAL requests memory matching the specifications, it directly allocates from the pool, without triggering the full-flow allocation of DMA-Buf, reducing the allocation delay from 200μs to 20μs.

[0191] FD cache table: a global hash table maintained by the conversion module, used to store the mapping relationship between ION handles and generated DMA-Buf shared FDs. When the HAL initiates multiple sharing requests for the same ION handle, it directly reuses the FD in the cache table, only incrementing the reference count, avoiding repeated calls to the dma_buf_export interface and reducing FD generation overhead.

[0192] Synchronization operation merging mechanism: the conversion module implements an optimization strategy for repeated cache synchronization requests for the same ION handle within a short period of time (default 1ms), such as consecutive calls to ION_IOC_SYNC, by only executing the first operation and marking subsequent requests as "merged". By checking the last_sync_time and last_sync_dir fields in the dma_buf private data, it can reduce the number of cache synchronization operations by 60% and reduce the bus bandwidth occupancy.

[0193] fence pooling: the conversion module pre-creates a fixed number (default 128) of dma_fence object pools to store the synchronization barriers to be reused. When the HAL creates an ION barrier, it directly obtains the idle dma_fence from the pool to update the context and seqno fields; when the barrier is completed, it is reset and put back into the pool, avoiding the memory overhead of frequently creating and destroying dma_fence, reducing the barrier creation delay from 50us to 5us.

[0194] ION handle import conversion: when the HAL imports the ION handle shared by other processes through the ION_IOC_IMPORT interface, the conversion module converts it into the process of importing the DMA-Buf FD. This process needs to verify the legality of the imported handle, i.e., whether it is generated by the conversion module, and reconstruct the mapping relationship between the ION handle and the DMA-Buf pointer to ensure the compatibility of cross-process sharing.

[0195] The present application realizes efficient simulation of ION to DMA-Buf through the kernel-level ION-DMA-Buf conversion acceleration layer, full-quantity ION interface coverage and multi-level performance optimization mechanism, and experimental verification shows that:

[0196] Through pre-allocated pool, FD cache, synchronization merging and other optimizations, the memory allocation delay is reduced to 20us, the sharing delay is reduced to 10us, and the overall performance loss is < 5%, meeting the real-time requirements of 30fps / 60fps of the camera, significantly better than the user-mode conversion solution; the full-quantity ION interface coverage includes 12 types of ioctl commands such as ION_IOC_ALLOC, SHARE, IMPORT, SYNC, and supports camera HAL from Android 7.0 to 14.0 without modifying the closed-source components, including Qualcomm ADSP, Mediatek M4U, etc.; through SELinux context mapping, from Android tag to OpenHarmony tag, FD reference count management, orphan FD delay recycling, the permission control of cross-container memory access is realized, meeting the security architecture requirements of OpenHarmony; modular design is adopted, supporting dynamic configuration, which can adjust parameters in real time through the sysfs interface, without restarting the kernel to adapt to different hardware platforms such as mobile phones, tablets, smart cockpits, etc.

[0197] The present application fully utilizes the similarity of ION and DMA-Buf in the FD sharing mode, solves the interface difference between the two through the kernel-level direct conversion architecture, and provides a high-performance and high-reliability solution for memory management compatibility in the heterogeneous OS containerization scenario.

[0198] To sum up, the above is only the preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A container-based Android camera HAL-compatible operation method based on memory access virtualization, characterized in that, Specifically, the following steps are included: Using a Linux kernel-based system as the host system, the DMA-Buf memory heap is designated as the target memory heap; using an Android system as the container system, the ION memory heap is designated as the source memory heap; a kernel-mode conversion module is built in the host system. The kernel-mode transition module establishes a first structure to store the dedicated memory management context of the Android camera HAL process, establishes an FD cache table to store the mapping relationship between ION handles and DMA-Buf file descriptors (FD), and creates a set number of synchronization fence pools to store synchronization fences to be reused. During the initialization of the host system kernel, it registers virtual ION devices, creates virtual ION device nodes, and allocates a set of memory blocks, which is recorded as the first memory pool. When the container starts, it binds the virtual ION device nodes to the ION paths within the container. The conversion module monitors the HAL process. When it acquires the FD of the ION memory allocator, it allocates and initializes the first structure and binds the pointer of the first structure to its ION file descriptor. When it initiates a memory allocation request, it determines the target memory heap type, memory size, alignment, and flag bits related to the DMA-Buf based on the first structure, and returns the memory pool pointer and ION handle obtained from the first memory pool. When it shares ION memory with other components, it returns the FD corresponding to the ION handle in the sharing parameters determined by the FD cache table. When it imports an ION handle from another process, it obtains the corresponding DMA-Buf pointer based on the FD, generates and returns the imported ION handle, and adds the mapping relationship to the first structure; when it releases memory, it determines the release method based on the DMA-Buf pointer, releases the spinlock and resumes the interrupt; when it exits, it traverses the reference count array of the first structure, marks FDs with reference counts greater than zero as orphan FDs, and registers a delayed garbage collection callback function. The first structure includes a first hash table, a memory heap mapping table, a reference count array, security tags, memory pointers, import handles, and an FD mapping table. The first hash table is a mapping table between ION handles and DMA-Buf pointers. The memory heap mapping table is a mapping table from the source memory heap type to the target memory heap type. The reference count array is a reference count array of shared FDs. The security tags are the SELinux tags of the Android HAL process. The memory pointers are DMA-Buf pointers associated with the ION handle. The method for registering and creating virtual ION device nodes during host system kernel initialization is as follows: The primary and secondary device numbers of the virtual ION device are applied for and bound as exclusive device numbers through the Linux character device number registration mechanism; a custom device operation interface set is initialized and associated with the exclusive device number; a virtual ION device node is created in the corresponding directory of the host system; when the container starts, the virtual ION device node is bound to the standard ION path within the container, and a device access hook for the virtual memory file system is registered.

2. The method for HAL-compatible operation of an Android camera inside a container according to claim 1, characterized in that, The method for allocating and initializing the first structure when it obtains the FD of the ION memory allocator is as follows: the HAL process calls the open function to trigger the open callback function of the conversion module. The callback function first checks whether it is a process in the container by comparing the process namespace identifier. If it is not a container process, it returns access denied. If it is a container process, it allocates memory for the first structure and initializes its core fields.

3. The method for HAL-compatible operation of an Android camera inside a container according to claim 1, characterized in that, When a memory allocation request is initiated, the target memory heap type, memory size, alignment, and flag bits related to the DMA-Buf are determined according to the first structure. If the memory size and source memory heap type of the memory allocation request match the memory block in the first memory pool, the corresponding memory pool pointer and pre-generated ION handle are obtained from the first memory pool; otherwise, the entire allocation process of the target memory heap is executed to obtain the memory pointer and ION handle.

4. The method for HAL-compatible operation of an Android camera within a container according to claim 3, characterized in that, The entire process of allocating the target memory heap is as follows: the conversion module calls dma_heap_alloc to allocate the target memory heap and obtain the buffer pointer. Then, it generates a unique ION handle by using the high 8 bits as the target memory heap type and the low 24 bits as the global auto-incrementing sequence number. The mapping relationship between the ION handle and the DMA-Buf pointer is added to the first structure through hash_add_rcu, and the RCU mechanism is enabled to ensure the safety of read operations.

5. The method for HAL-compatible operation of an Android camera within a container according to claim 1, characterized in that, The translation module monitors the HAL process. When it maps ION memory to user space, it parses the command to extract the mapped memory size, access permissions, mapping type, and ION handle. Based on the first hash table, it determines the DMA-Buf pointer corresponding to the ION handle, converts the access permissions to those supported by the DMA-Buf, sets the attributes of the mapping page table according to the DMA-Buf pointer attributes, and uses the DMA-Buf's message to map memory to user space to map the physical memory determined by the DMA-Buf pointer to a virtual address in user space, returning the starting address of the virtual address.

6. The method for HAL-compatible operation of an Android camera inside a container according to claim 1, characterized in that, The conversion module monitors the HAL process. When it synchronizes ION memory data between the CPU and hardware devices, it parses the synchronization parameters to obtain the ION handle and synchronization direction. Based on the first hash table, it determines the DMA-Buf pointer corresponding to the ION handle. It checks the synchronization record in the private data of the DMA-Buf pointer. If the difference between the current time and the timestamp of the last record is less than a threshold, it ignores the current synchronization operation and returns a synchronization success message. Otherwise, it converts the synchronization direction from the CPU to the device to a synchronous write to the DMA-Buf, the synchronization direction from the device to the CPU to a synchronous read to the DMA-Buf, and the bidirectional synchronization to a synchronous read and write to the DMA-Buf. Based on the hardware architecture, it calls the corresponding DMA-Buf synchronization interface, triggers the underlying cache operation instruction, and then saves the timestamp and synchronization direction of this synchronization to the private data of the DMA-Buf pointer.

7. The method for HAL-compatible operation of an Android camera inside a container according to claim 1, characterized in that, The conversion module monitors the HAL process. When it creates an ION synchronization barrier, if there is an idle DMA synchronization barrier in the synchronization barrier pool, it acquires the DMA synchronization barrier; otherwise, it creates the DMA synchronization barrier and adds it to the synchronization barrier pool. After completing the initialization of the DMA synchronization barrier, it generates the barrier FD associated with the DMA synchronization barrier and returns the barrier FD. When the HAL process is set to block and wait, the conversion module waits in a loop for the DMA synchronization barrier to be triggered until the barrier is triggered or the process is interrupted.

8. The method for HAL-compatible operation of an Android camera inside a container according to claim 1, characterized in that, The conversion module monitors the HAL process. When the HAL process transmits a File Controller (FD), it parses the message to obtain the FD, checks if the FD exists in the FD cache table, and if it does, it creates a new FD with the same namespace in the receiving process's PID and increments the reference count of the FD; otherwise, it ignores the message. When it receives a FD, it allocates the FD within the container and returns it to the HAL process.

Images