A context-aware based harmonious containerized tls compatible method

Abstract

The application discloses a context-aware-based hyperjump containerized TLS compatible method, which comprises the following steps: distinguishing simple TLS variables and complex TLS variables in a source hyperjump system, and establishing TLS variable source features; constructing a dynamic scalable TLS memory pool matching variable access authority in a target system, extending a kernel state TLS segment descriptor, constructing a bidirectional mapping table of musl TLS offset and memory pool address, and optimizing hot spot variable to construct a cluster structure body aligned with CPU cache lines to improve access efficiency; recording a TLS variable access track through a dynamic binary translation engine, updating a TLS segment descriptor, reading the descriptor to recover a TLS context when thread switching, refreshing a cache to guarantee consistency, and converting addresses and offsets according to target system alignment requirements in combination with source features and the bidirectional mapping table for TLS variables of nested structures, pointers and array types, so as to complete dynamic conversion and execution of a source system application in the target system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer software development, and particularly relates to a context-aware Harmony containerized TLS compatible method. BACKGROUND

[0002] When the user state program of the Harmony operating system runs in the standard Linux operating system container environment, variable access conflicts, memory layout disorder, thread switching exceptions and complex type adaptation failures may be caused by the underlying differences in thread local storage (TLS) implementation mechanisms.

[0003] TLS, as the core mechanism for storing private data of threads in multi-threaded programs, is a key technology for ensuring thread independence and data security. OpenHarmony selects musl as the default C library to adapt to embedded devices and distributed scenarios, and its TLS implementation adopts the design concept of compact layout. The glibc commonly used in standard Linux systems pursues full functionality and compatibility, and adopts a segmented storage and dynamic expansion TLS architecture. The fundamental differences between the two not only exist in the surface variable address calculation, but also extend to the underlying levels of memory allocation strategies, thread context management, CPU cache utilization and complex type layout, which brings multiple challenges for cross-environment compatibility.

[0004] Specifically, there are differences in memory allocation and layout. Fixed-size contiguous pages are allocated through mmap, and musl arranges all TLS variables in a single contiguous memory area, supporting direct access through the base address and fixed offset. The glibc divides the TLS variables into static TLS and dynamic TLS, and the TLS memory space of different threads may be scattered in multiple non-contiguous pages. This causes the large-size TLS variables in the OpenHarmony program to access the glibc environment, triggering memory out-of-bounds errors due to non-contiguous page tables.

[0005] The context synchronization defect during thread switching. Musl only needs to save the TLS base address register during thread switching, without actively flushing the cache. The glibc relies on the set_thread_area system call in the kernel mode to update the TLS descriptor, and synchronously flushes the thread context. This difference causes the OpenHarmony program to appear cache dirty reading phenomenon in the high-concurrency thread switching scenario, that is, the thread reads the old data of other threads after recovery, which destroys the data consistency.

[0006] Layout conflict of complex type variable, TLS variable containing pointer, nested structure or array is compactly arranged according to natural alignment rule in musl, while glibc may insert additional padding bytes between structure fields for optimizing access performance. Thus, pointer points to wrong address in OpenHarmony program, causing program crash.

[0007] Low utilization efficiency of CPU cache, traditional compatible scheme only focuses on address conversion, without considering CPU cache characteristics, causing frequent cache invalidation when accessing TLS variable. For example, TLS variable access across cache line needs multiple cache loading, and performance loss is more than 50% in high-frequency access scenario; write-intensive variable does not use targeted cache optimization strategy, causing cache pollution, further reducing program running efficiency.

[0008] However, existing compatible schemes, such as static recompilation, simple address mapping, user-mode thread interception, have significant limitations: static recompilation needs to modify program source code, which cannot adapt to closed-source binary programs; simple address mapping only supports basic type variables, and cannot handle complex nested structures; user-mode thread interception is difficult to cover kernel-level thread switching scenarios, and the reliability of context synchronization is insufficient; and none of them optimize CPU cache characteristics, and performance cannot meet high-frequency access requirements.

[0009] In summary, it is urgent to implement a scheme without modifying source code, supporting all types of variables, considering compatibility and high performance, to solve the technical problems caused by the difference between musl and glibc TLS mechanism. SUMMARY

[0010] Therefore, the application provides a context-aware Harmony containerized TLS compatibility method, which realizes zero-aware compatibility of TLS variable and efficient and stable running of Harmony application in high-performance access scenario.

[0011] The context-aware Harmony containerized TLS compatibility method provided by the application specifically includes the following steps:

[0012] Step 1, the Harmony system is recorded as a source system, the Linux system is recorded as a target system, and the Harmony application is recorded as a first application; the basic attributes and runtime access mode of the TLS variable in the first application in the source system are extracted, and the structure relationship of the complex TLS variable is constructed, and the source characteristics of the TLS variable are composed of the basic attributes, the structure relationship and the access mode;

[0013] Step 2, allocate the TLS memory pool composed of base page and extension page in the target system, the access permission is the same as the TLS variable, the virtual address of the base page is unchanged; extend the kernel TLS segment descriptor, add the TLS variable context including the TLS memory pool base address, memory pool size and base address register value; form the first bidirectional mapping table from the mapping relationship between the source system TLS offset and the TLS memory pool address;

[0014] Step 3, the first application initializes the registration of the TLS memory pool to the kernel in the container, and obtains the cache line size of the target system; according to the basic attribute and the access mode, the TLS variable with access frequency and read-write ratio greater than the corresponding threshold and size not greater than the cache line is selected as the hot variable, the hot variable set is composed of the hot variable with access frequency greater than the threshold and read-write ratio difference less than the threshold and satisfying the total size not greater than the cache line, and the padding byte is inserted into the cache line aligned cluster structure of the hot variable set;

[0015] Step 4, the dynamic binary translation engine executes the access track of the first application record TLS variable, obtains the TLS memory pool base address, memory pool size and base address register value of the current thread, and updates the TLS segment descriptor; when the thread switches, the TLS segment descriptor is read, and the TLS variable context of the thread to be executed after switching is obtained and restored; the conversion of the TLS variable and the access instruction of the TLS variable is completed.

[0016] Further, the basic attribute includes segment information, symbol characteristics and storage attribute for simple TLS variable, and includes field composition, type, offset and alignment requirement for complex TLS variable.

[0017] Further, the structure relationship is constructed by the target type, size, alignment rule and access permission of the complex TLS variable pointer field, and the belonging and association relationship between fields.

[0018] Further, the access mode includes access frequency, access granularity and read-write ratio, the access frequency is the read-write times of each TLS variable in a single thread life cycle, the access granularity includes byte, field and block access, and the read-write ratio is the ratio of read times to write times.

[0019] Further, the updating mode of the first bidirectional mapping table is that when the first application dynamically creates a TLS variable, the extension page memory is automatically allocated, and the mapping table entry is updated; when the dynamic TLS variable is released, the corresponding mapping entry is marked as invalid, and the extension page is recycled after all variables in the extension page are released.

[0020] Further, the hot variable set is formed by preferentially selecting the hot variables belonging to the same functional module from the hot variables with access frequency greater than the threshold and read-write ratio difference less than the threshold and satisfying the total size not greater than the cache line.

[0021] Further, the hot spot variables in the cluster structure of the hot spot variable set are arranged in descending order of access frequency, and the starting address of the cluster structure is forced to be aligned to the cache line boundary.

[0022] Further, the dynamic binary translation engine executes the first application, and when the execution instruction is an access instruction of a TLS variable, only when the TLS variable belongs to the hot spot variable set, the access is located in a loop, the access time interval is less than a threshold, and the access type is reading, a preloading instruction is inserted according to the cache level corresponding to the TLS variable to realize instruction pipeline cooperation.

[0023] Further, the dynamic binary translation engine executes the first application, and when the TLS variable to be converted is a nested structure, the TLS variable is converted into a corresponding variable in the target system, and the parent field, the child field and the respective offset are obtained from the basic attribute and the structural relationship, the actual offset of the TLS variable in the target system is calculated layer by layer according to the alignment requirement of the target system, and the difference between the actual offset and the offset is filled.

[0024] When the TLS variable to be converted is a pointer type, the pointer target address of the TLS variable is extracted from the basic attribute, if the pointer target address is in the address space corresponding to the TLS memory pool, the pointer variable is converted into a corresponding variable in the target system, and the offset of the target address is calculated according to the starting address of the pointer variable, and then the address of the pointer variable in the target system is obtained from the first bidirectional mapping table, and the conversion of the pointer variable and the corresponding address of the target field is completed; otherwise, only the pointer variable is converted into a corresponding variable in the target system.

[0025] When the TLS variable to be converted is an array type, the TLS variable is converted into a corresponding variable in the target system, and the array base address of the TLS variable is extracted from the basic attribute, the target array base address in the corresponding target system is obtained from the first bidirectional mapping table, and the difference between the array base address and the target array base address is calculated, and when the address is not aligned, padding bytes are inserted according to the alignment requirement of the target system; all elements in the array are traversed, and the address conversion is completed.

[0026] Further, the preloading instruction is inserted according to the cache level corresponding to the TLS variable, specifically: when the TLS variable is loaded into the L1 cache, the prefetcht0 instruction is inserted; when the TLS variable is only loaded into the L2 or L3 cache, the prefetchnta instruction is inserted.

[0027] Advantages:

[0028] The application distinguishes simple TLS variables and complex TLS variables in a source hyper system, extracts basic attributes such as segment information, symbol features and storage attributes of simple variables, analyzes the structural relationship of complex variables and the runtime access mode of all TLS variables to form source features of the TLS variables, constructs a dynamic scalable TLS memory pool matching variable access permissions in a target system, extends the kernel state TLS segment descriptor, constructs a bidirectional mapping table of the musl TLS offset and the memory pool address, and optimizes the construction of the cluster structure aligned with the CPU cache line of the hotspot variable to improve the access efficiency, records the TLS variable access track through the dynamic binary translation engine, updates the TLS segment descriptor, reads the descriptor to restore the TLS context and refreshes the cache to guarantee consistency when switching threads, and completes the address and offset conversion according to the alignment requirements of the target system in combination with the source features and the bidirectional mapping table for the TLS variables of the nested structure, pointer and array type, to complete the dynamic conversion execution of the application program in the source system in the target system. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A flowchart of a hyper containerized TLS compatible method based on context perception provided by the application is shown. DETAILED DESCRIPTION

[0030] The application will be described in detail below in combination with the embodiments and the drawings.

[0031] The core idea of the hyper containerized TLS compatible method based on context perception provided by the application is that simple TLS variables and complex TLS variables are distinguished in a source hyper system, basic attributes such as segment information, symbol features and storage attributes of simple variables are extracted, the structural relationship of complex variables is analyzed, and the runtime access mode of all TLS variables is analyzed to form source features of the TLS variables, a dynamic scalable TLS memory pool matching variable access permissions is constructed in a target system, the kernel state TLS segment descriptor is extended, a bidirectional mapping table of the musl TLS offset and the memory pool address is constructed, and the construction of the cluster structure aligned with the CPU cache line of the hotspot variable is optimized to improve the access efficiency, the TLS variable access track is recorded through the dynamic binary translation engine, the TLS segment descriptor is updated, the descriptor is read to restore the TLS context and refresh the cache to guarantee consistency when switching threads, and the address and offset conversion is completed according to the alignment requirements of the target system in combination with the source features and the bidirectional mapping table for the TLS variables of the nested structure, pointer and array type, to realize the dynamic conversion execution of the application program in the source system in the target system.

[0032] The hyper containerized TLS compatible method based on context perception provided by the application has the flow as shown in Figure 1 The specific steps include the following steps:

[0033] Step 1, the source system is recorded as the source system, the Linux system is recorded as the target system, and the musl TLS-based Hyper system is recorded as the first application; in the source system, the basic attributes of the simple TLS variable in the first application are extracted based on ELF file analysis and source code analysis, including segment information, symbol characteristics and storage attributes; for complex TLS variables, the variable structure is analyzed according to program debugging information and static code analysis, and information such as field composition, type, offset and alignment requirement of complex types is extracted, the target type, size, alignment rule and access permission of the pointer field are identified, the ownership and association relationship between variable fields are determined, and the structure relationship of the variable is established accordingly; the access mode of the TLS variable at runtime is extracted by analyzing the intermediate code, including access frequency, access granularity and read-write ratio; the source characteristics of the TLS variable are composed of basic attributes, structure relationship and access mode.

[0034] Among them, the target type of the pointer field is the specific type of the data in the memory area pointed to by the pointer, the size of the pointer field is the total number of bytes occupied by the data corresponding to the target type in the memory, the alignment rule of the pointer field is the byte alignment requirement that the starting address of the data corresponding to the target type needs to meet, and the access permission of the pointer field is the read-write execution permission of the memory area corresponding to the target type. The simple TLS variable refers to the TLS variable without nested structure, pointer or array structure. The complex TLS variable refers to the TLS variable containing nested structure, pointer or array. The segment information includes the starting address, size, permission flag and alignment requirement of the segment. The symbol characteristics include variable name, size, fixed offset in the musl environment and initialization value. The storage attributes include inter-thread read-only variables, read-write variables and temporary variables.

[0035] Based on program debugging information and static code analysis, the relationship between fields in the complex TLS variable is determined to form the structure relationship of the variable, including the ownership relationship, hierarchical relationship, dependency relationship and attribute association. Specifically, the ownership relationship refers to the subordinate relationship between the field and the parent structure or array, the hierarchical relationship refers to the nested hierarchical path of the field in the complex structure, the dependency relationship refers to the pointing association between the pointer field and the target type, and the association formed between the fields due to memory alignment or offset, and the attribute association refers to the mapping relationship between the field offset, type, size, alignment rule and other attributes.

[0036] Among them, the way to analyze the intermediate code to extract the access characteristics of the TLS variable at runtime is: first, simulate the execution of the first application through static instrumentation, count the read and write times of each TLS variable in a single-threaded life cycle, and distinguish high-frequency and low-frequency variables; the variable whose read and write times exceed the threshold is a high-frequency variable, and the variable whose read and write times are below the threshold is a low-frequency variable; second, identify the variable access granularity and determine it as byte, field and block access; finally, calculate the read-write ratio of the variable, that is, the ratio of the number of reads to the number of writes, to distinguish three types of read more than write, write more than read and balanced read and write.

[0037] Step 2, allocate memory pages for storing small size TLS variables with high access frequency in the target system, mark the base page as the base page, the capacity of the base page is the same as the default size of the musl TLS memory page, the virtual address of the base page remains unchanged, when the total size of the TLS variable obtained in step 1 exceeds the capacity of the base page, allocate an extension page, and map the virtual address of the extension page to the continuous address space of the base page, set the same access permission for the base page and the extension page as the TLS variable, and form a dynamically scalable TLS memory pool by the base page and the extension page; extend the kernel state TLS segment descriptor, and increase the context segment for storing the TLS variable, the TLS variable context includes the TLS memory pool base address, the memory pool size and the base address register value, etc.;

[0038] The mapping relationship between the musl TLS offset and the TLS memory pool address forms a first bidirectional mapping table, including the musl TLS offset, the memory pool address, the page type, the variable size, the access frequency, etc., wherein the page type includes the base page and the extension page; the mapping entry corresponding to the high frequency TLS variable is stored in the continuous memory area.

[0039] Further, the first bidirectional mapping table is dynamically updated during the running of the first application, specifically including: when the first application dynamically creates a TLS variable, automatically allocating extension page memory and updating the mapping table entry; when the dynamic TLS variable is released, the corresponding mapping entry is marked as invalid, and after all variables in the extension page are released, the extension page is recycled, realizing the on-demand release of memory.

[0040] In addition, the present application realizes the adaptive expansion and contraction of the TLS memory pool, specifically: when the thread is created after the first application is started, the base page and the extension page are pre-allocated according to the total size of the TLS variable extracted in step 1; if new dynamic TLS variables are added during running, resulting in insufficient memory, a new extension page is allocated and the page table mapping is updated; the access state of the extension page is checked regularly by a timer, for the extension page that has not been accessed for a continuous time, the valid variables in the extension page are first migrated to other extension pages, and then the physical memory of the extension page is released, the virtual address range of the released extension page is recorded, and the virtual address range is preferentially reused when new variables are added subsequently, avoiding frequent memory allocation and release overhead.

[0041] In the kernel state, the implementation of TLS is usually related to the thread local storage segment TLS segment. The Linux kernel creates or revokes the TLS segment for a process through set_thread_area() and get_thread_area() system calls. These system calls involve modifying the segment descriptor in the global descriptor table GDT or the local descriptor table LDT to define a thread-specific memory segment.

[0042] TLS segment descriptors are a type of system segment descriptor, following the standard segment descriptor format, but specific fields can vary by kernel version or architecture. Generally, segment descriptors contain the following key fields: BASE, the base address of the segment (linear address), LIMIT, the size limit of the segment (in bytes or 4KB), G, the granularity flag (G=1 indicates 4KB units), S, the segment type flag (S=0 indicates a system segment, such as a TLS segment), Type, which specifies the type and access permissions of the segment, DPL, the descriptor privilege level, P, the present flag, etc. In kernel code, TLS segment descriptors are usually indirectly referenced through macro definitions, but the specific structure definitions can vary by kernel version. In actual implementation, the kernel maintains a TLS segment selector for each thread and updates the segment registers to point to the thread-specific storage area during context switching.

[0043] Step 3, when the first application is initialized in the container, register the information of the TLS memory pool to the kernel, and obtain the cache line size of the CPU cache of the target system; according to the basic properties and access mode of the TLS variable obtained in step 1, select the hot variable whose access frequency and read-write ratio are greater than the corresponding threshold and whose size is not greater than the cache line; select the hot variable whose access frequency is greater than the threshold and whose read-write ratio difference is less than the threshold, and preferentially select the hot variable belonging to the same functional module from them, and then select the hot variable set from the hot variable set whose total size is not greater than the cache line.

[0044] A clustering structure is defined for the hot variable set, and the hot variables in the clustering structure are arranged in descending order of access frequency, the starting address of the clustering structure is aligned to the cache line boundary, and padding bytes are inserted to ensure that the hot variable set does not cross the cache line.

[0045] The application can register the TLS memory pool related information in the kernel state TLS segment descriptor to the kernel by establishing a custom kernel to user state communication interface, and the kernel will check the legality of the memory address and the reasonableness of the memory pool size during the registration stage, avoiding the risk of illegal memory access.

[0046] Step 4, the dynamic binary translation engine executes the first application, and records the access trajectory of the TLS variable, including: access instruction address, access type, whether in a loop, and access time interval; the TLS memory pool base address, memory pool size, and base address register value of the current thread are obtained by using the lightweight probe provided by the kernel, and the TLS segment descriptor is updated.

[0047] Step 5, when thread switching is performed, the TLS segment descriptor is read, the TLS variable context of the thread to be executed after switching is obtained and restored, and the cache corresponding to the TLS memory pool is flushed;

[0048] When the execution instruction is an access instruction of a TLS variable, only when the TLS variable belongs to the hot spot variable set, the access is located in a loop, the access time interval is less than a threshold, and the access type is reading, a preload instruction is inserted according to the corresponding cache level of the TLS variable to realize instruction pipeline cooperation;

[0049] When the to-be-converted TLS variable is a nested structure, the TLS variable is converted into a corresponding variable in the target system, and a parent field, a child field and respective offsets are obtained from basic attributes and structural relationships of a source feature of the TLS variable; actual offsets of the TLS variable in the target system are calculated layer by layer according to alignment requirements of the target system, and a difference between the actual offsets and the offsets is filled in.

[0050] When the to-be-converted TLS variable is a pointer type, a pointer target address of the TLS variable is extracted from basic attributes of a source feature of the TLS variable; if the pointer target address is in an address space corresponding to a TLS memory pool, the pointer variable is converted into a corresponding variable in the target system, and an offset of the target address is calculated according to a start address of the pointer variable, and an address of the pointer variable in the target system is obtained from a first bidirectional mapping table, and conversion of the pointer variable and a corresponding address of a target field of the pointer variable is completed; otherwise, only the pointer variable is converted into a corresponding variable in the target system.

[0051] When the to-be-converted TLS variable is an array type, the TLS variable is converted into a corresponding variable in the target system, and an array base address of the TLS variable is extracted from basic attributes of a source feature of the TLS variable; a target array base address in the target system is obtained from the first bidirectional mapping table, and a difference between the array base address and the target array base address is calculated; when the addresses are not aligned, padding bytes are inserted according to alignment requirements of the target system; all elements in the array are traversed, and address conversion is completed.

[0052] The preload instruction is inserted according to the corresponding cache level of the TLS variable, specifically, when the TLS variable is loaded into an L1 cache, a prefetcht0 instruction is inserted to realize first access; when the TLS variable is only loaded into an L2 or L3 cache, a prefetchnta instruction is inserted to realize repeated access.

[0053] Further, for a TLS variable with a read-write ratio less than a threshold, that is, a variable that needs to be frequently modified, a non-temporary storage instruction is used to write the variable into a write-merge buffer of a CPU. For a large-size TLS buffer, a cache line is actively released after use to avoid occupying cache resources and causing cache invalidation of other variables. In addition, for continuous small-size writing, dynamic binary translation is used to combine the writing into a single block writing to reduce the number of cache modifications.

[0054] Further, to solve the integrity damage problem caused by compiler optimization, memory tampering and the like after the TLS structure conversion, the application verifies whether the actual offset of the key field of the structure is consistent with the expected value to identify the alignment adjustment error problem; and calculates the CRC32 value of the converted structure, compares it with the reference CRC value pre-calculated in the source system, and detects the field data error.

[0055] When the actual offset of the key field of the structure is different from the expected value and the CRC32 value after the conversion is different from the reference CRC value, the structure is reinitialized, the data is restored based on the value extracted in the source feature, and the conversion process is re-executed to complete the repair; when the source feature array is missing, the repair fails, and then the compatible mode is switched to run.

[0056] Embodiment:

[0057] This embodiment takes an OpenHarmony application as an example, and uses a context-aware Harmony containerized TLS compatible method provided by the application to realize zero-aware compatibility and high-performance access of the OpenHarmony application when running in a Linux container. The method is suitable for distributed task scheduling, high-frequency log output, multi-thread concurrent computing and other core business scenarios with strong TLS dependency, and the specific process includes:

[0058] S1, hierarchical feature analysis and structure modeling of musl TLS variable. Through a three-level analysis mechanism, multi-dimensional features of the TLS variable are accurately extracted from the OpenHarmony program, and a complete structure model is constructed to provide accurate basis for subsequent adaptation and break through the limitation of traditional schemes that only extract basic field information.

[0059] S1.1, basic feature extraction by first-level analysis. Based on ELF file analysis and musl source code analysis, the underlying attributes of the TLS variable are extracted.

[0060] Segment information extraction: the.tdata and.tbss segments of the initialized TLS variable of the program are analyzed by the readelf tool to obtain the starting address, size, permission flag and alignment requirement of the segment, so that the memory attributes remain consistent when mapping in the glibc environment; symbol feature extraction: the symbols of the type STT_TLS are identified by traversing the ELF symbol table, and the variable name, size, fixed offset in the musl environment and initialization value are recorded; storage attribute classification: according to the declaration attribute and musl extension mark of the variable, the inter-thread read-only variable, read-write variable and temporary variable are distinguished to provide a basis for subsequent memory allocation and cache optimization. For example, the inter-thread read-only variable can share the physical memory page in the glibc environment to reduce redundant allocation.

[0061] S1.2, Secondary resolution structure nesting relationship resolution. For complex TLS variables containing nested structures, pointers or arrays, combined with program debugging information and static code analysis, recursive structure resolution is implemented:

[0062] Debugging information resolution: read the.debug_info section of the ELF file, extract the definition information of complex types, such as the composition of nested structures, field types, offsets and alignment requirements; pointer target identification: through symbol table association and type inference, determine the target type of the pointer field, such as pointing to the internal variable of TLS, heap memory or global variable, record the size, alignment rule and access permission of the target type; structure dependency graph construction: taking parent structure and child field as the core, construct multi-level dependency graph, clearly define the ownership and association of each field. For example, for struct ohm_dist_task_t, the graph will record the hierarchical relationship of task_id with offset 0, attr.flags with offset 8, attr.data_ptr with offset 12, and name with offset 24, ensuring that no field is missed during subsequent conversion.

[0063] S1.3, Third-level resolution access mode resolution. Based on LLVM IR intermediate code analysis, extract the runtime access characteristics of TLS variables:

[0064] Access frequency statistics: simulate program execution flow through static instrumentation, count the read and write times of each TLS variable in single-thread life cycle, and distinguish high-frequency variables with access frequency ≥100 times / second and low-frequency variables; access granularity identification: judge whether the variable access mode is byte access, such as buf[0] = 1, field access, such as task->task_id = 5, or block access, such as memcpy(buf, data, 32); read-write ratio calculation: calculate the proportion of read operation of variable, distinguish read more and write less, such as read ratio ≥80% configure cache, write more and read less, such as write ratio ≥80% statistics counter, read-write ratio close to read-write balance, provide basis for cache optimization strategy.

[0065] S2, Dynamic scalable TLS memory pool construction and mapping. For the layout difference between musl continuous memory and glibc discrete memory, construct a dynamic memory pool to realize virtual continuous mapping and on-demand scaling of TLS space, balance compatibility and memory utilization.

[0066] S2.1, Hierarchical design of memory pool.

[0067] Base page: allocate 1 4KB memory page, consistent with musl default TLS page size, used to store high-frequency access small size variables such as log_level, task_id. When allocating through mmap, set the MAP_FIXED flag to fix the virtual address of the base page, simulate the continuous memory layout of musl, and ensure fast access to basic variables; Extended page: when the total size of the TLS variable exceeds the capacity of the base page, such as large size buffer, structure array, automatically allocate an extended page, each extended page size is 4KB or 8KB, dynamically adjusted according to variable size. Through page table entry modification, the virtual address of the extended page is mapped to the continuous address space of the base page, such as the base page ending at 0x7f000000, and the extended page is mapped to 0x7f001000-0x7f002000, so that the program perceives as continuous memory, solving the problem of cross-page access out-of-bounds; Memory permission control: according to the extracted variable permission characteristics, set the corresponding memory permissions for the base page and the extended page, such as setting PROT_READ for read-only variable pages, and setting PROT_READ|PROT_WRITE for read-write variables, to ensure memory access security.

[0068] S2.2, Construction and management of bidirectional mapping table. Construct a bidirectional mapping table of musl offset to memory pool address, realize address association and fast query of two kinds of TLS model:

[0069] Mapping table structure design: each entry of the mapping table contains musl offset, memory pool address, page type (base page / extended page), variable size, access frequency, etc. It supports fast lookup of corresponding memory pool address according to musl offset, and supports reverse lookup of musl offset according to memory pool address, meeting the needs of bidirectional conversion; Dynamic update mechanism: when the program runs through pthread_key_create to dynamically create TLS variables, automatically allocate extended page memory and update mapping table entries; When the dynamic variable is released through pthread_key_delete, mark the corresponding mapping entry as invalid, and after all variables in the extended page are released, recycle the extended page through munmap, realize the on-demand release of memory; Cache optimization: store the mapping entries of high-frequency variables in the CPU cache-friendly continuous memory area, reduce cache invalidation when querying, and control the mapping table query time within 10ns.

[0070] S2.3, Adaptive scaling strategy of memory pool.

[0071] Expansion mechanism: when a thread is created, base pages and necessary extension pages are pre-allocated according to the total size of the extracted TLS variables; if new dynamic TLS variables are added during running and cause insufficient memory, new extension pages are automatically allocated and the page table mapping is updated to ensure that the program is not aware; contraction mechanism: through a timer, the access state of the extension page is checked regularly, and for the extension page that has not been accessed for 5 minutes, the valid variables in the extension page are first migrated to other extension pages, and then the physical memory of the extension page is released; at the same time, the released virtual address range is recorded, and the newly added variables are preferentially reused in the future to avoid frequent memory allocation and release overhead.

[0072] S3, CPU cache line-based access acceleration mechanism.

[0073] CPU cache is the core hardware resource for improving program performance, and the access efficiency of TLS variables directly depends on the utilization of the cache. If the TLS adaptation of musl and glibc ignores the cache characteristics, the cache invalidation rate will increase sharply in high-frequency access scenarios.

[0074] S3.1, cache line-aligned variable clustering and layout optimization. CPU cache loads and stores data in fixed-size cache lines, with x86_64 architecture defaulting to 64 bytes and ARM architecture defaulting to 32 bytes. If TLS variables are stored across cache lines, a single access will trigger 2 to 3 cache loads, i.e., cache line splitting, which will increase the delay by 2 to 3 times. For example, if the 4-byte log_level and 8-byte log_timestamp of the OpenHarmony log module are located in two cache lines, reading these two variables requires two cache loads, and the total delay increases from 5ns to 12ns.

[0075] S3.1.1, cache line parameter adaptive detection. The cache line size of different CPU architectures is different, and it needs to be accurately detected during the initialization of the container to provide a benchmark for subsequent layout optimization. The cache configuration is obtained through the CPUID instruction or reading the system register. The detection result is usually 64 bytes, stored as a global constant CACHE_LINE_SIZE, which serves as a benchmark for subsequent layout optimization.

[0076] S3.1.2, hot variable clustering.

[0077] The core of hot variable clustering is to combine high-frequency access, read-write, and strongly correlated variables into a compact variable set based on the extracted TLS variable access characteristics, and to align the layout according to the CPU cache line size, so that a single cache load covers multiple variables. Specifically:

[0078] The pre-preparation of data clustering is completed by relying on the extracted access mode. The premise of hot variable clustering is to obtain the access behavior portrait of TLS variables to ensure that the clustering has a clear basis.

[0079] Access frequency data, total number of variable read and write in single thread life cycle, such as sched_task_id access 200 times / s, sched_flags 180 times / s; read-write ratio data, read operation proportion of variable, such as sched_timestamp read proportion 95%, sched_task_id read proportion 90%; variable basic attribute, variable type, size, such as sched_task_id is int type 4 bytes, sched_timestamp is uint64_t type 8 bytes.

[0080] The above data is obtained through static instrumentation and ELF symbol analysis. For example, for the TLS variable of the OpenHarmony distributed scheduling module kernel / linux / include / linux / sched.h, the output raw data is as follows:

[0081] Variable Name Type Size (bytes) Access Frequency (times / sec) Read Percentage Belonging Function Module sched_task_id int 4 200 90% Task Scheduling sched_flags uint32_t 4 180 85% Task Scheduling sched_timestamp uint64_t 8 150 95% Task Scheduling log_level int 4 120 88% Log Output temp_buf char

[32] 32 50 60% Temporary Buffer

[0082] The selection condition of the hot variable is defined. To ensure that the clustered variables can maximize the use of the cache, variables that meet the characteristics of high frequency, read more, and small size need to be selected first. The specific selection rules based on CPU cache characteristics and TLS access scene optimization include: frequency threshold, single-thread access frequency is not less than 100 times / s, and the variable below this frequency has less than 10% cache optimization benefit, and does not need to be clustered; read-write ratio threshold, read operation proportion is not less than 80%; size limit, single variable size is not greater than CPU cache line size, such as x86_64 architecture CACHE_LINE_SIZE=64 bytes, to avoid single variable crossing cache line; functional relevance, preferentially selecting variables of the same functional module, such as 3 variables of the scheduling module and log_level of the log module, to avoid cache pollution caused by mixed clustering of cross-module variables.

[0083] Based on the above rules, the candidate hot variables that meet the conditions after screening the above raw data are as follows:

[0084] Variable Name Type Size (bytes) Access Frequency (times / sec) Read Percentage Belonging Function Module sched_task_id int 4 200 90% Task Scheduling sched_flags uint32_t 4 180 85% Task Scheduling sched_timestamp uint64_t 8 150 95% Task Scheduling log_level int 4 120 88% Log Output

[0085] Clustering combination strategy: after selecting the candidate variables, the combination needs to be performed according to the principles of total size not greater than cache line, functional relevance priority, and high-frequency variable in the front, to avoid the clustered variable set crossing the cache line, and to improve the access locality.

[0086] Combination principle: total size control, the total size of the clustered variable set including padding bytes is not greater than CACHE_LINE_SIZE, CACHE_LINE_SIZE is 64 bytes, ensuring that a single cache load can cover it; function association priority, variables in the same function module are preferentially clustered, such as the combination of 3 variables in the scheduling module, and the log_level in the log module is clustered alone or combined with other variables in the same module; high-frequency variables are placed at the beginning of the variable set, close to the cache line header, reducing the address offset calculation overhead when loading the cache; type compatibility, avoid mixing variables that are easy to modify with pure read-only variables, and reduce cache pollution caused by write operations on read-only variables.

[0087] Take the OpenHarmony scheduling module as an example of actual combination, and combine the 3 candidate variables of the scheduling module:

[0088] Calculate the total size: sched_task_id (4) + sched_flags (4) + sched_timestamp (8) = 16 bytes, which is not greater than 64 bytes, meeting the size requirement; sort by access frequency: sched_task_id (200 times / s), sched_flags (180 times / s), and sched_timestamp (150 times / s), ensuring that high-frequency variables are placed at the beginning; supplement padding bytes: 16 bytes need to be filled with 48 bytes (64-16=48), so that the total size is equal to the cache line size, avoiding the destruction of alignment when new variables are added later.

[0089] The final combination result of the hot variable set of the scheduling module is:

[0090] Variable Name Offset (bytes) in Cluster Type Size (bytes) Access Frequency (times / sec) sched_task_id 0 int 4 200 sched_flags 4 uint32_t 4 180 sched_timestamp 8 uint64_t 8 150 Padding Bytes 16~63 char 48 -

[0091] Cluster layout implementation in the glibc environment: after determining the cluster combination, a compact alignment layout needs to be implemented in the glibc static TLS region. The core is to use the aligned attribute of GCC to force cache line alignment, ensuring that the starting address of the variable set falls on the cache line boundary. The specific operation is as follows:

[0092] Define the cluster structure, based on the above combination result, define the cluster structure in the adaptation header file in the glibc environment, such as ohm_tls_glibc.h. Refer to the TLS layout logic of musl to ensure compatibility with the access logic of OpenHarmony variables:

[0093] / / Hot variable cluster structure of OpenHarmony scheduling module (glibc environment)

[0094] #define CACHE_LINE_SIZE 64

[0095] __thread struct {

[0096] / / Hotspot variables in descending order of access frequency

[0097] int sched_task_id; / / Offset 0, high-frequency core variable (task ID)

[0098] uint32_t sched_flags; / / Offset 4, scheduling status flag

[0099] uint64_t sched_timestamp; / / Offset 8, task timestamp

[0100] char _pad

[48] ; / / Offset 16~63, padding to 64 bytes (cache line size)

[0101] } ohm_sched_tls __attribute__(

[0102] (aligned(CACHE_LINE_SIZE), / / Force the starting address to align to 64 bytes (cache line boundary)

[0103] tls_model("initial-exec")) / / Use static TLS model to avoid dynamic resolution overhead );

[0105] Alignment verification ensures layout validity. After compilation, use the objdump tool to verify the memory layout of the cluster structure body to ensure that the starting address is aligned and the total size meets the cache line requirements: Execute the command: objdump -s -j.tbss ohm_sched_module.o, assuming the compiled object file is ohm_sched_module.o; Verification result: The starting address of the structure body ohm_sched_tls is offset by 0x0 or an integer multiple of 64, the total size is 64 bytes, and the variable offset is consistent with the expected value, such as sched_task_id at offset 0x0, sched_timestamp at offset 0x8, padding bytes covering 0x10~0x3F, and 64 bytes at 0x40.

[0106] Cluster effect verification: After the implementation of cluster layout, the optimization effect is confirmed through dynamic monitoring to ensure that the cache hit rate and access delay meet the expectations. The specific verification method is as follows: cache hit rate monitoring: use the perf tool of Linux to count the cache loading of cluster variables; expected result: the cache hit rate before clustering is about 60%, and after clustering, it is improved to more than 99.5%; access delay test: use the rdtsc instruction to count the CPU cycle number of variable access, and convert it to delay, which is about 1 ns for 1 GHz CPU of x86_64 or ARM architecture. Expected result: before clustering, accessing 3 variables requires 12 to 15 cycles, about 12 to 15 ns, and after clustering, only 5 to 6 cycles, about 5 to 6 ns, the delay is reduced by more than 50%.

[0107] Cluster adjustment for special scenarios: If the candidate variables have special situations such as size exceeding expectations, cross-module association, etc., the clustering strategy needs to be adjusted: variable total size exceeds cache line, such as 4 variables with a total size of 72 bytes greater than 64 bytes, need to be split into clusters, such as the first 3 variables with a size of 16 bytes, and the 4th variable with a separate cluster, to avoid crossing the cache line; cross-module strongly associated variables: such as log_level in the log module and sch ed_task_id in the scheduling module need to be accessed continuously, such as log printing needs to include task ID, which can break the functional module limit and combine the two with log_timestamp, with a total size of 4+4+8=16 bytes, to improve access locality; frequently written variables: such as sch ed_flags, although the read proportion is 85%, but it needs to be written 1000 times a day, which can be split from the cluster set and laid out separately to avoid cache invalidation caused by write operations.

[0108] S3.1.3, forced alignment and compact layout in glibc environment, when allocating memory for the hot variable set in the glibc static TLS region, use __attribute__((aligned(CACHE_LINE_SIZE))) to force the starting address to align to the cache line boundary, and insert padding bytes to ensure that the variable set does not cross the cache line.

[0109] S3.1.4, intelligent insertion and timing control of preloading instructions, the prefetch series of preloading instructions can load TLS variables into CPU cache in advance, but blind insertion will increase the instruction overhead by 10% to 15%, such as preloading of non-hot variables will waste CPU resources.

[0110] Implementation logic of access trajectory tracking, in the dynamic binary translation engine, record the access trajectory of each TLS variable, including: access instruction address ip, access type including read and write, judge whether in loop through jmp or loop instruction, calculate the time interval of next access through instruction address difference.

[0111] The condition insertion rule of preloading instructions is to insert a preloading instruction before a TLS variable access instruction only when all the following conditions are met: the variable belongs to the hot variable set, the access is inside a loop, trace->is_in_loop = true; the next access interval is no more than 8 instructions, such as trace->next_access_interval <= 32 bytes, 4 bytes per instruction on x86_64; and the access type is read.

[0112] S3.2, write merge optimization and cache pollution control, write-intensive TLS variables such as statistical counters and temporary buffers, if frequently modified, will cause cache coherence traffic to surge, especially in multi-core scenarios. The traditional write throughput is only 50% of the native environment.

[0113] S3.2.1, application scenarios and code of non-temporary storage instructions

[0114] For write more read less (write proportion not less than 90%) and variable size not more than 8 bytes, such as statistical counters, use movnti (x86_64) or vstrnt.32 (ARM) non-temporary storage instructions to directly write into the CPU's write merge buffer (Write Combine Buffer), bypassing L1 / L2 cache and reducing cache coherence traffic. The advantage of non-temporary storage instructions is batch flushing.

[0115] S3.2.2, cache line exclusivity management and large buffer optimization, for large size TLS buffers such as __thread char temp_buf

[1024] , use clflushopt to actively release the cache line after use to avoid occupying cache resources and causing other variable cache invalidation. The optimized cache flush instruction clflushopt is 30% faster than the traditional clflush and supports parallel cleaning.

[0116] For example, 1024 bytes of temp_buf need to clean 16 cache lines (64 bytes per line), clflushopt instructions can be processed in parallel, the total cleaning time is about 160ns, which is much lower than the natural cache invalidation time (about 10μs). This operation improves the cache hit rate of other TLS variables by 15%.

[0117] S3.2.3, batch merge of continuous write operations, for continuous small size write, such as multiple updates of different fields in a structure, use dynamic binary translation to merge into a single block write such as memcpy, reducing the number of cache modifications.

[0118] S4, Kernel-state synchronization of TLS context during thread switching. In the multi-threading scenario, thread switching is the core pain point of TLS compatibility. musl only saves the TLS base address register, while glibc relies on the kernel-state set_thread_area to synchronize the complete context. If only intercept switching in user mode, kernel-level switching will be missed, resulting in cache dirty read.

[0119] S4.1, Kernel-state TLS descriptor extension and registration mechanism. The struct user_desc defined in linux / thread_info.h in the Linux kernel only supports glibc's TLS management. Its structure needs to be extended to carry the context information of musl TLS, while ensuring compatibility with the existing kernel logic, such as get_thread_area and set_thread_area.

[0120] Based on the extension descriptor's kernel-state definition of the Linux 5.15 kernel, the extended structure needs to have struct user_desc as the first member to ensure that the kernel function can normally access the original field, and the new field is used to store the core information of musl TLS:

[0121] / / Extended TLS descriptor in kernel module (compatible with Linux struct user_desc)

[0122] struct ohm_tls_desc {

[0123] struct user_desc glibc_desc; / / Original glibc TLS descriptor (first field)

[0124] void __user *musl_base; / / User-mode muslTLS memory pool base address (such as 0x7f000000)

[0125] size_t pool_size; / / Memory pool size (such as 16KB)

[0126] uint64_t tls_ptr; / / muslTLS base address register value (%fs user value)

[0127] struct mm_struct *mm; / / Associated memory space (for permission verification)

[0128] struct hlist_node node; / / Hash table node (fast lookup by pid)

[0129] };

[0130] The key fields of glibc_desc include entry_number (TLS segment selector), base_addr (glibc TLS base address), and limit (segment size), which are fully compatible with the kernel native logic; musl_base and tls_ptr are used to store the memory pool address and base address register value of musl TLS, which are the core data of context synchronization.

[0131] The implementation of the user-kernel registration interface provides an ioctl registration interface through the character device / dev / ohm_tls, and the user-kernel registration interface is called by the OpenHarmony program during initialization to register the musl TLS memory pool information to the kernel.

[0132] S4.2, TLS context interception and synchronization during thread switching, the kernel probe kprobe is used to intercept the thread scheduling core function defined in kernel / sched / core.c, that is, schedule, the TLS context of the current thread is saved before scheduling, and the context of the target thread is restored after scheduling, ensuring atomicity during switching.

[0133] Registration and implementation of kprobe hooks, kprobe is a lightweight probe mechanism provided by the Linux kernel, which can insert custom logic at the function entry and exit, and the entry and exit hooks of the schedule function are registered.

[0134] Context recovery and cache flushing, the exit hook is executed when the target thread is scheduled to the CPU, which restores its TLS context and flushes the cache, and the key function invalidate_tls_cache invalidates the page table entry corresponding to the TLS memory pool through the invlpg instruction, forcing the CPU to reload data from memory, eliminating cache consistency problems between multiple cores.

[0135] S4.3, user-kernel communication and exception handling mechanism, in extreme scenarios such as kernel module exceptions and memory tampering, the TLS context may be inconsistent, and a bidirectional communication and degradation mechanism needs to be designed to ensure that the program does not crash.

[0136] S4.3.1, Kernel-mode exception detection and signaling, the kernel module periodically checks the TLS state of the thread through the timer hrtimer, and the period is set to 10 ms to balance the detection accuracy and overhead; user-mode exception recovery and degradation, the OpenHarmony program registers the SIGUSR1 signal processing function, and after receiving the exception signal, reinitializes the TLS context. If multiple recovery failures occur (such as 3 times in a row), automatically switch to a compatible mode, and map the TLS variable to a thread-safe global variable. Although the compatible mode will lose some performance (mutex overhead is about 100 ns), it ensures that the program can run normally and meets the needs of high-availability scenarios.

[0137] The kernel-state auxiliary mechanism is introduced, the context atomic synchronization is realized by extending the TLS descriptor and intercepting thread scheduling, and the kernel-level thread switching problem that cannot be handled by user-mode interception is solved; the cache flushing mechanism significantly reduces the TLS access error rate after thread switching.

[0138] S5, recursive mapping and verification of complex type variables. The adaptation of basic type TLS variables only requires address conversion, while complex types containing nested structures, pointers, and arrays, such as OpenHarmony's struct ohm_dist_task_t, have different alignment and padding due to the layout differences between musl and glibc. Direct conversion will cause field offset errors.

[0139] S5.1, recursive mapping and alignment adaptation of nested structures, the difference in alignment strategies for nested structures between musl and glibc is the core problem. musl aligns according to natural alignment (field offset is an integer multiple of its own size), and glibc may insert additional padding bytes due to optimized alignment (align according to the maximum field size of the parent structure).

[0140] For example, in musl, the substructure attr has an 8-byte pointer, and its own alignment is 8 bytes, so the offset in the parent structure is adjusted from 4 bytes to 8 bytes; the name field has a maximum alignment of 8 bytes in the parent structure, so the offset is adjusted from 20 bytes to 24 bytes, and the total structure size is 24+32=56 bytes. In glibc (Ubuntu 22.04, 5.15 kernel), the compiler may adjust the attr offset to 16 bytes for optimized access, and the name offset to 16+12=28 (aligned to 8 bytes, which is 32 bytes), and the total structure size is 32+32=64 bytes. If you directly access attr.flags according to the musl offset (offset 8), you will read the wrong field in glibc, such as the low 4 bytes of attr.data_ptr.

[0141] S5.1.1, Core flow of recursive mapping and example code. The core function to calculate the actual offset in glibc, layer by layer, in the order of parent structure to child structure to leaf field, is as follows:

[0142] / / Recursively calculate the offset of a field in glibc (based on the structure characteristics in step 1)

[0143] size_t calc_glibc_field_offset(const struct tls_feature *feat,

[0144] const char *field_path) {

[0145] / / field_path format: "attr.flags" (parent field. child field)

[0146] char *parent_field = strtok((char *)field_path, ".");

[0147] char *child_field = strtok(NULL, ".");

[0148] / / 1. Find the parent structure feature

[0149] struct tls_nested_field *parent = find_nested_field(feat, parent_field);

[0150] if (!parent) return -1;

[0151] / / 2. Calculate the offset of the parent structure in glibc (musl offset + alignment adjustment value)

[0152] size_t parent_offset = parent->musl_offset +

[0153] (parent->glibc_align - parent->musl_align);

[0154] / / 3. If there is a child field, recursively calculate the offset of the child field

[0155] if (child_field) {

[0156] struct tls_feature *child_feat = parent->detail.nested_struct;

[0157] size_t child_offset = calc_glibc_field_offset(child_feat, child_field);

[0158] return parent_offset + child_offset;

[0159] }

[0160] return parent_offset;

[0161] }

[0162] For example, the offset of attr.flags in glibc is calculated as follows:

[0163] The musl offset of the parent field attr is 8, and the glibc alignment adjustment value is 8 (musl alignment 8, glibc alignment 16), so the parent offset = 8 + 8 = 16; the musl offset of the child field flags is 0, and the alignment adjustment value is 0, so the total offset = 16 + 0 = 16; finally, the offset of attr.flags in glibc is 16 bytes, which is different from the musl 8 bytes. This function automatically compensates for the difference.

[0164] S5.1.2, padding adaptation for alignment differences, for the padding bytes caused by alignment differences, zero padding is actively inserted in the glibc memory pool to ensure that the structure size is consistent with musl. For example, the total size of struct ohm_dist_task_t in glibc is 64 bytes, which needs to be padded with 8 bytes (64-56=8). The padding position is selected at the end of the structure, and after padding, the structure size in glibc is consistent with musl (56 bytes), avoiding out-of-bound access due to size mismatch.

[0165] S5.2, chain-like conversion of needle fields and target verification, pointers in complex structures may form a chain of TLS internal pointers, such as data_ptr pointing to the name field of the same structure. If only the pointer itself is converted without processing the target address, it may point to the wrong memory, such as pointing to the TLS space of musl, which does not exist in glibc.

[0166] S5.2.1, the identification logic of the pointer target type, based on the obtained structural feature analysis, the pointer target type is divided into three categories: TLS internal variable: the pointer points to other variables in the same TLS space, which needs to be converted; heap memory: the pointer points to the malloc allocated heap space, which does not need to be converted; global variable: the pointer points to the global static variable, which does not need to be converted.

[0167] The address of the TLS internal variable is between musl_base and musl_base + pool_size, which is determined by the target address range of the pointer field.

[0168] S5.2.2, the flow and code example of chain conversion:

[0169] Taking the data_ptr pointer of struct ohm_dist_task_t (pointing to the name field) as an example, the chain conversion steps are as follows:

[0170] Read the musl address of data_ptr: 0x7f000020 + 24 = 0x7f000038 (musl offset of name field is 24);

[0171] Calculate the musl offset of the name field: 0x7f000038 - 0x7f000020 = 24;

[0172] Convert to glibc offset through the mapping table in step 2: 24 + 8 = 32 (assuming musl_base = 0x7f000020, glibc_base = 0x7f100020);

[0173] Update data_ptr to glibc address: 0x7f100020 + 32 = 0x7f100040.

[0174] Core conversion function:

[0175] / / Pointer field chain conversion (supports multi-level pointers)

[0176] void translate_tls_ptr_chain(void ptr, void *musl_base, void *glibc_base) {

[0177] if (!is_tls_internal_ptr(*ptr, musl_base, POOL_SIZE))

[0178] return; / / Non-TLS internal pointer, no conversion needed

[0179] / / 1. Calculate musl offset

[0180] size_t musl_offset = (char *)*ptr - (char *)musl_base;

[0181] / / 2. Convert to glibc offset (based on mapping table from step 2)

[0182] size_t glibc_offset = get_glibc_offset(musl_offset);

[0183] / / 3. Update pointer to glibc address

[0184] *ptr = (char *)glibc_base + glibc_offset;

[0185] / / 4. If it's a multi-level pointer (like void), recursively convert

[0186] if (is_multi_level_ptr(ptr)) {

[0187] translate_tls_ptr_chain((void )*ptr, musl_base, glibc_base);

[0188] }

[0189] }

[0190] This function supports chain conversion for multi-level pointers (like void pptr), ensuring that all associated pointers point to the glibc TLS space.

[0191] S5.2.3, Boundary Check and Error Handling of Target Address, after conversion, the target address needs to be checked whether it is within the legal range of glibc TLS memory pool to avoid wild pointer access:

[0192] / / Boundary check of pointer target address

[0193] bool validate_ptr_target(void *ptr, void *glibc_base, size_t pool_size) {

[0194] if (!ptr) return false;

[0195] return (ptr >= glibc_base) && (ptr < (char *)glibc_base + pool_size);

[0196] }

[0197] / / Error handling: fix when pointer is out of bounds

[0198] void fix_invalid_ptr(void ptr, void *glibc_base) {

[0199] if (!validate_ptr_target(*ptr, glibc_base, POOL_SIZE)) {

[0200] *ptr = NULL; / / Set to NULL to avoid wild pointer access

[0201] log_error("TLS pointer out of bounds: %p", *ptr);

[0202] }

[0203] }

[0204] The test shows that this check can reduce the pointer out-of-bounds error rate from 5% to 0.1%. The main source of errors is illegal pointers, such as uninitialized wild pointers.

[0205] S5.3, Batch conversion of array type and element adaptation, arrays in TLS, especially structure arrays, need to process elements in batches and adapt to the array alignment difference between musl and glibc. musl aligns by element type, while glibc may align by the size of the entire array, resulting in different array base address offsets.

[0206] S5.3.1, Conversion and alignment adaptation of array base address, take the OpenHarmony __thread struct ohm_dist_task_t tasks[4] structure array as an example. In musl, the array base address is aligned to 8 bytes, and the element type is aligned to 8. In glibc, it may be aligned to 16 bytes, so the following steps are needed for adaptation:

[0207] Base address translation: convert musl array base address (e.g. 0x7f000040) to glibc address (e.g. 0x7f100040) according to mapping table in step 2; alignment adjustment: calculate the difference between glibc's alignment requirement (e.g. 16 bytes) and musl's alignment (8 bytes), if the base address is not aligned, insert padding bytes (e.g. 8 bytes) to ensure the array base address is aligned to glibc's requirement. For example, glibc base address 0x7f100040 (aligned 8) needs to be adjusted to 0x7f100048 (aligned 16), 8 bytes padding is inserted.

[0208] S5.3.2, Batch translation of array elements, the translation of array elements needs to traverse all elements, and perform nested structure translation and pointer translation for each element, the core code:

[0209] / / Batch translation of structure array

[0210] void translate_tls_struct_array(void *glibc_array,

[0211] void *musl_base,

[0212] size_t elem_count,

[0213] const struct tls_feature *elem_feat) {

[0214] size_t elem_size = elem_feat->size;

[0215] for (size_t i = 0; i < elem_count; i++) {

[0216] void *elem = (char *)glibc_array + i * elem_size;

[0217] / / 1. Translate the nested structure fields of the element

[0218] translate_nested_struct(elem, elem_feat);

[0219] / / 2. Translate the pointer fields in the element

[0220] translate_tls_ptr_fields(elem, elem_feat, musl_base, glibc_array);

[0221] }

[0222] }

[0223] S5.4 Multi-level verification and dynamic repair of structural integrity. Even after conversion, structural integrity may still be compromised due to compiler optimization, memory tampering, etc. A three-level mechanism of field offset verification, overall CRC verification, and dynamic repair is required to ensure correctness.

[0224] S5.4.1 Field Offset Validation: Verify whether the offset of key fields matches the expected extracted value. For example, verify whether the offset of attr.flags in glibc is 16 bytes.

[0225] / / Field offset validation

[0226] bool check_field_offset(void *glibc_struct, const struct tls_feature*feat,

[0227] const char *field_path, size_t expected_offset) {

[0228] size_t actual_offset = calc_glibc_field_offset(feat, field_path);

[0229] return actual_offset == expected_offset;

[0230] }

[0231] This check can detect alignment adjustment errors (such as offset calculation errors), with an error rate of approximately 0.05%.

[0232] S5.4.2 CRC Overall Verification: Calculate the CRC32 value of the entire structure after conversion and compare it with the pre-calculated baseline CRC in the musl environment. The baseline CRC is extracted during parsing.

[0233] / / Overall CRC check

[0234] bool check_struct_crc(void *glibc_struct, const struct tls_feature *feat) {

[0235] uint32_t current_crc = crc32(0, glibc_struct, feat->size);

[0236] return current_crc == feat->base_crc; / / base_crc is the reference value in musl environment

[0237] }

[0238] CRC check can find data errors in the field, such as byte loss during copying, with an error rate of about 0.03%.

[0239] S5.4.3, dynamic repair mechanism, if the first two levels of verification fail, trigger the structure to reinitialize, restore data based on the extracted initialization value:

[0240] / / Structure dynamic repair

[0241] void repair_struct(void *glibc_struct, const struct tls_feature *feat) {

[0242] / / 1. Re-copy initialization value (data from.tdata segment in step 1)

[0243] memcpy(glibc_struct, feat->init_value, feat->init_size);

[0244] / / 2. Re-execute the conversion process

[0245] translate_nested_struct(glibc_struct, feat);

[0246] translate_tls_ptr_fields(glibc_struct, feat, feat->musl_base, feat->glibc_base);

[0247] }

[0248] Repair success rate is about 99%, failure cases are mainly initialization value missing, such as uninitialized.tbss segment variables, at this time need to switch to compatibility mode, such as using default values.

[0249] Experiments show that the application has the following advantages in compatibility depth: all types of TLS variables in the OpenHarmony program are supported, including basic types, nested structures, pointer arrays and the like, and the compatibility scene is expanded from simple variables to core business modules such as distributed task scheduling and high-frequency logs, and the compatibility rate reaches 100%; the performance index method: the access delay of high-frequency TLS variables is reduced to 1.1 times of the original musl environment, the TLS synchronization time consumption when switching threads is controlled within 50ns, and the overall program performance is improved by more than 40% in a high-concurrency scene; in terms of memory efficiency: the dynamic scalable memory pool reduces the TLS memory occupation by 40%, and avoids the memory waste caused by fixed size mapping; the cache optimization strategy improves the cache hit rate of TLS variable access to 99.5%; in terms of reliability: the kernel context synchronization and multi-level check mechanism reduces the TLS related crash rate from 3.2% to 0.01%, completely eliminates the cache dirty read and wild pointer problems, and meets the demand of high-availability scenarios.

[0250] To sum up, the above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A context-aware HarmonyOS containerized TLS compatible method, characterized in that, Specifically, the following steps are included: Step 1: Denote the HarmonyOS system as the source system, the Linux system as the target system, and the HarmonyOS application as the first application; Extract the basic attributes and runtime access modes of the TLS variables in the first application from the source system, as well as the structural relationships of complex TLS variables. The source characteristics of TLS variables are composed of basic attributes, structural relationships, and access modes; The TLS variables in the first application include simple TLS variables and complex TLS variables. Step 2: Allocate a TLS memory pool consisting of base pages and extended pages in the target system. The access permissions are the same as those of the TLS variables in the first application, and the virtual address of the base page remains unchanged. Extend the kernel-mode TLS segment descriptor and add the TLS variable context from the first application, including the TLS memory pool base address, memory pool size, and base address register value. Form a first bidirectional mapping table based on the mapping relationship between the source system TLS offset and the TLS memory pool address. Step 3: The first application initializes in the container to register the TLS memory pool to the kernel and obtains the target system cache line size; based on basic attributes and access patterns, the TLS variables in the first application whose access frequency and read / write ratio are both greater than the corresponding threshold and whose size is not greater than the cache line are designated as hot variables. A hot variable set is formed by hot variables whose access frequency is greater than the threshold, whose read / write ratio difference is less than the threshold, and whose total size is not greater than the cache line. Padding bytes are inserted into the cache line aligned clustering structure of the hot variable set. Step 4: The dynamic binary translation engine executes the first application to record the access trajectory of TLS variables in the first application, obtains the TLS memory pool base address, memory pool size and base address register value of the current thread, and updates the TLS segment descriptor; when switching threads, it reads the TLS segment descriptor, obtains and restores the TLS variable context of the thread to be executed in the first application after the switch; and completes the conversion of TLS variables and their access instructions in the first application.

2. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The basic attributes, for simple TLS variables, include segment information, symbol characteristics, and storage attributes, while for complex TLS variables, they include field composition, type, offset, and alignment requirements.

3. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The structural relationships are constructed from the target type, size, alignment rules, and access permissions of complex TLS variable pointer fields, as well as the attribution and association relationships between fields.

4. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The access mode includes access frequency, access granularity, and read-write ratio. The access frequency is the number of times the TLS variable is read and written in each of the first applications within the lifecycle of a single thread. The access granularity includes access to bytes, fields, and blocks. The read-write ratio is the ratio of the number of reads to the number of writes.

5. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The first bidirectional mapping table is updated as follows: when the first application dynamically creates TLS variables in the first application, extended page memory is automatically allocated and the mapping table entries are updated; when the dynamic TLS variables in the first application are released, the corresponding mapping entries are marked as invalid, and the extended pages are reclaimed after all variables in the extended pages are released.

6. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The hotspot variable set is formed by prioritizing hotspot variables belonging to the same functional module from those hotspot variables whose access frequency is greater than a threshold, whose read-write ratio difference is less than a threshold, and whose total size is not greater than a cache line.

7. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The hot variables in the cluster structure of the hot variable set are arranged in descending order of access frequency, and the starting address of the cluster structure is forced to be aligned to the cache line boundary.

8. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The dynamic binary translation engine executes the first application. When the execution instruction is an access instruction for a TLS variable in the first application, it inserts a preload instruction according to the cache level corresponding to the TLS variable in the first application to achieve instruction pipeline coordination only when the TLS variable in the first application belongs to the hot variable set, the access is located in a loop, the access time interval is less than the threshold, and the access type is read.

9. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, The dynamic binary translation engine executes the first application. When the TLS variables in the first application to be converted are nested, the TLS variables in the first application are converted into the corresponding variables in the target system. At the same time, the parent field, child field and their respective offsets are obtained from the basic attributes and structural relationships. The actual offset in the target system is calculated layer by layer according to the alignment requirements of the target system, and then the difference between the actual offsets is filled. When the TLS variable in the first application to be converted is of pointer type, the pointer target address of the TLS variable in the first application is extracted from the basic attributes. If the pointer target address is within the address space corresponding to the TLS memory pool, the pointer variable is converted into the corresponding variable in the target system. At the same time, the offset of the target address is calculated based on the starting address of the pointer variable, and the address of the pointer variable in the target system is obtained from the first bidirectional mapping table, thus completing the conversion of the pointer variable and the address corresponding to its target field. Otherwise, only the pointer variable is converted into the corresponding variable in the target system. When the TLS variable in the first application to be converted is an array type, the TLS variable in the first application is converted into the corresponding variable in the target system. At the same time, the array base address of the TLS variable in the first application is extracted from the basic attributes. The target array base address in the target system is obtained according to the first bidirectional mapping table. The difference between the array base address and the target array base address is calculated. When the address is not aligned, padding bytes are inserted according to the alignment requirements of the target system. All elements in the array are traversed to complete the address conversion.

10. The HarmonyOS containerized TLS compatible method according to claim 1, characterized in that, Preload instructions are inserted according to the cache level corresponding to the TLS variable in the first application. Specifically, when the TLS variable in the first application is loaded into the L1 cache, the prefetcht0 instruction is inserted; when the TLS variable in the first application is only loaded into the L2 or L3 cache, the prefetchnta instruction is inserted.

Images