A file processing method and apparatus

Abstract

Embodiments of the present application provide a file processing method and device, relate to the technical field of computers, and improve the execution efficiency of an instruction stream of an application program. The specific scheme is as follows: a plurality of first library files are obtained, the plurality of first library files are mapped to a continuous virtual address space, a second library file is obtained, the second library file is rearranged, and a third library file is obtained. Embodiments of the present application are used in the process of global optimization of an instruction stream.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a file processing method and apparatus. Background Technology

[0002] With the development of software technology, the application loads of mobile phones, personal computers and servers are becoming increasingly complex, leading to bottlenecks in the microarchitecture pipeline of the central processing unit (CPU) and affecting the CPU's execution efficiency.

[0003] Currently, different software ecosystems in the computer field share common characteristics: complex application logic and the use of numerous middleware and system libraries (i.e., library files) during development. This improves application development efficiency but also leads to large application images. When an application runs, it may jump between different library files, frequently causing CPU issues such as cache hit failures, page table hit failures, and branch prediction failures, resulting in low execution efficiency of the application's instruction flow. Summary of the Invention

[0004] This application provides a file processing method and apparatus that improves the execution efficiency of application instruction streams.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions.

[0006] In a first aspect, embodiments of this application provide a file processing method, which includes: obtaining multiple first library files, mapping the multiple first library files to a continuous virtual address space to obtain second library files, and rearranging the second library files to obtain third library files.

[0007] Therefore, in the file processing method provided in this application embodiment, multiple first library files are mapped to a contiguous virtual address space. That is, multiple first library files with different random virtual address spaces are mapped to a contiguous virtual address space. This allows for global rearrangement of the second library files to optimize the instruction flow of the application. Compared to optimizing the instruction flow of a single library file, the file processing method provided in this application embodiment can optimize the instruction flow of multiple first library files from a global perspective. This means that optimization can be performed on a single library file as well as across library files, reducing the number of cache hit failures and branch prediction failures, thereby improving the execution efficiency of the application's instruction flow.

[0008] In one possible implementation, the second library file includes multiple first code segments. The second library file is rearranged to obtain a third file, which includes: obtaining the execution count of each first code segment in the second library file, arranging multiple first code segments with execution counts greater than a preset threshold consecutively, and arranging multiple first code segments with execution counts less than or equal to the preset threshold consecutively to obtain the third library file.

[0009] In this implementation, the first code segment whose execution count is greater than a preset threshold can be understood as a hot code segment, and the first code segment whose execution count is less than or equal to the preset threshold can be understood as a cold code segment. The file processing method provided in this application embodiment can realize the aggregation of global hot code segments, which can reduce the number of cache hit failures, branch jump failures and branch prediction failures, and improve the execution efficiency of instruction flow.

[0010] In one possible implementation, obtaining the execution count of each first code segment in the second library file includes: compiling the original code corresponding to each first code segment to obtain a relocation binary file, and obtaining the execution count of each first code segment based on the relocation binary file.

[0011] In one possible implementation, obtaining the execution count of each first code segment in the second library file includes: obtaining the original binary based on the original code corresponding to each first code segment, sampling the original binary, and obtaining the execution count of each first code segment.

[0012] In one possible implementation, the first library file includes a second data segment and a second code segment. Mapping multiple first library files to a contiguous virtual address space to obtain a second library file includes: merging multiple first library files to obtain a fourth library file. The fourth library file includes a third data segment, a third code segment, and a merged segment. The third data segment includes multiple second data segments, the third code segment includes multiple second code segments, and the merged segment includes the mapping relationship between the fourth library file and multiple first library files. Mapping the fourth library file to a contiguous virtual address space to obtain the second library file.

[0013] In this implementation, the file processing method can merge multiple first library files into a single fourth library file. The data and code segments within the fourth library file are recombined, and a merged segment is added, which includes the mapping relationships between the fourth library file and the multiple first library files. This file processing method can then perform global optimizations on the merged fourth library file to improve the execution efficiency of the application's instruction flow.

[0014] In one possible implementation, multiple first library files are mapped to a contiguous virtual address space to obtain a second library file, including: sequentially mapping multiple first library files to a contiguous virtual address space to obtain a second library file.

[0015] In this implementation, the file processing method can sequentially map multiple first library files into a contiguous virtual address space, so as to perform global optimization on the second library files and improve the execution efficiency of the application's instruction flow.

[0016] In one possible implementation, the method also includes mapping the virtual address space corresponding to the third library file to large page memory.

[0017] In this implementation, mapping the virtual address space corresponding to the third library file to large page memory can reduce the number of page table entries and improve the hit rate of the query cache.

[0018] Secondly, embodiments of this application provide a file processing apparatus, which includes an acquisition module, a processing module, and a rearrangement module. The acquisition module is used to acquire multiple first library files, the processing module is used to map the multiple first library files to a contiguous virtual address space to obtain second library files, and the rearrangement module is used to rearrange the second library files to obtain third library files.

[0019] In one possible implementation, the reordering module is specifically used to: obtain the execution count of each first code segment in the second library file, arrange multiple first code segments with execution counts greater than a preset threshold consecutively, and arrange multiple first code segments with execution counts less than or equal to the preset threshold consecutively to obtain the third library file.

[0020] In one possible implementation, the reordering module is specifically used to: compile the original code corresponding to each first code segment to obtain a relocation binary file, and based on the relocation binary file, obtain the execution count of each first code segment.

[0021] In one possible implementation, the reordering module is specifically used to: obtain the original binary based on the original code corresponding to each first code segment, sample the original binary, and obtain the execution count of each first code segment.

[0022] In one possible implementation, the first library file includes a second data segment and a second code segment. The processing module is specifically used to: merge multiple first library files to obtain a fourth library file; the fourth library file includes a third data segment, a third code segment, and a merged segment; the third data segment includes multiple second data segments; the third code segment includes multiple second code segments; and the merged segment includes the mapping relationship between the fourth library file and the multiple first library files. The fourth library file is then mapped to a contiguous virtual address space to obtain a second library file.

[0023] In one possible implementation, the processing module is specifically used to: map multiple first library files sequentially into a contiguous virtual address space, where the end address of the previous first library file and the start address of the next first library file are contiguous, to obtain a second library file.

[0024] In one possible implementation, the processing module is also used to: map the virtual address space corresponding to the third library file to large page memory.

[0025] Thirdly, embodiments of this application provide an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, which includes computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the file processing method in any of the possible implementations of the first aspect described above.

[0026] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform a file processing method in any of the possible implementations of the first aspect described above.

[0027] Fifthly, embodiments of this application provide a computer program product that, when run on a computer or processor, causes the computer or processor to execute the file processing method in any possible implementation of the first aspect described above.

[0028] It is understood that any of the document processing devices, electronic devices, computer-readable storage media or computer program products provided above can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0029] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0030] Figure 1 A schematic diagram of the architecture of a computer system provided in this application embodiment;

[0031] Figure 2 A flowchart illustrating a file processing method provided in this application embodiment;

[0032] Figure 3 A flowchart illustrating another document processing method provided in this application embodiment;

[0033] Figure 4 A flowchart illustrating the process of determining the number of executions is provided for an embodiment of this application;

[0034] Figure 5 A flowchart illustrating yet another document processing method provided in this application embodiment;

[0035] Figure 6 This application provides a flowchart illustrating the process of calling a library file.

[0036] Figure 7 A flowchart illustrating yet another document processing method provided in this application embodiment;

[0037] Figure 8 A schematic diagram illustrating a library file mapping provided in an embodiment of this application;

[0038] Figure 9 A schematic diagram illustrating the mapping between virtual addresses and physical memory provided in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of the structure of a document processing device provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0041] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0042] The file processing method provided in this application is applied to a computer system, which may be a terminal device, server, routing device, virtual machine (VM), or container, etc.

[0043] Terminal equipment, also known as user equipment (UE), includes consumer electronics, home electronics, automotive electronics, and financial electronics. Consumer electronics include mobile phones, tablets, laptops, e-readers, personal computers (PCs), personal digital assistants (PDAs), desktop monitors, smart wearables (e.g., smartwatches, smart bracelets), virtual reality (VR) devices, augmented reality (AR) devices, mixed reality (MR) devices, artificial intelligence (AI) devices, and drones. Home electronics include smart locks, televisions, refrigerators, and small rechargeable household appliances (e.g., soymilk makers, robot vacuums). Automotive electronics include car navigation systems and digital video discs (DVDs). Financial electronics include automated teller machines (ATMs) or self-service terminals. This application does not impose any special restrictions on the specific form of the terminal device described above.

[0044] like Figure 1 As shown, Figure 1 The diagram illustrates the architecture of a computer system, which includes a user layer, an operating system, and a hardware layer.

[0045] The user layer comprises multiple applications (apps), such as application 1 and application 2, etc., and the executable code implementing these applications is the application program. Each application can include one or more business microservices, and each business microservice can correspond to an operation. For example, an application involving a user login scenario might include two microservices. The first business microservice can handle user login, changing the user's state to logged-in. The second business microservice could be a traffic logging microservice, recording the traffic used to modify the user's state.

[0046] The operating system (OS) is used to load and run applications for each application. The OS consists of user mode and kernel mode. When the OS loads an application, it allocates memory for the application in kernel mode, loads the application's executable and linkable format (ELF) files, and then switches to user mode to execute the dynamic library interpreter's process of loading dynamic libraries. The dynamic library interpreter first maps the dynamic library to the virtual address space, and then loads the dynamic library into physical memory only when the application runs.

[0047] The hardware layer includes communication interfaces, processors, physical memory, a memory management unit (MMU), and buses. The communication interfaces, processors, physical memory, and MMU are connected via a bus. The processor can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can also be a microprocessor or any conventional processor.

[0048] Physical memory can be configured into multiple memory spaces, and can include storage and cache. Storage can be volatile or non-volatile, or a combination of both. Non-volatile storage can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile storage can be random access memory (RAM), which serves as an external cache.

[0049] Alternatively, the cache can be implemented using an N-way set association approach. The cache can include a tag array and a data array. The tag array stores information such as the address and status of cache blocks, while the data array stores the data of the cache blocks. The tag array can include N-way and M set indices, where N and M are positive integers. Each way can also store tag information, which may include a valid bit, a tag address, and "others".

[0050] When a user accesses the cache, the address carried in the user's access request can include the tag address, group index, and byte offset. Specifically, the cache access process can include: the cache simultaneously reads tag information from N paths based on the group index in the address; by comparing the tag address in the current address with the tag addresses from the N paths, it determines whether the desired data exists in the cache. If the desired data exists in the cache, it is considered a hit, and the corresponding cache block in the data array is read / written. If the desired data does not exist in the cache, it is considered a miss, and the data is read from memory and replaced using one of the N paths.

[0051] In addition, caches can also include two types of caches, such as i-cache and d-cache. i-cache is used to store instructions so that the processing module can quickly fetch and execute instructions, while d-cache is used to store data so that the processor can reduce the time spent reading data from memory when processing data.

[0052] The Memory Management Unit (MMU) is a piece of computer hardware responsible for handling memory access requests from the central processing unit (CPU). Its functions include virtual address to physical address translation, memory protection, and CPU cache control.

[0053] Currently, computer system applications are becoming increasingly large and complex, and software engineering has been developing towards modularity. This leads to a situation where an application needs to load hundreds of library files, and even a single thread may need to load hundreds of library files. These library files are randomly distributed in the virtual address space, and the application may jump between different library files during runtime. This often causes problems such as cache hit failures, page table hit failures, and branch prediction failures for the CPU, resulting in low efficiency of the application's instruction flow.

[0054] In one possible implementation, to improve the efficiency of the application's instruction flow, the computer system can merge multiple library files into a single library file. This library file includes pre-compiled code and data, which can be reused by multiple programs, saving library loading and linking time and reducing application startup time.

[0055] However, this method only merges library files, and multiple jumps are still required when executing the application. Cache line utilization and page utilization are not improved, and the efficiency of the application's instruction flow remains low.

[0056] In another possible implementation, to improve the efficiency of the application's instruction flow, the computer system rearranges hot and cold functions and data blocks in the binary image to improve cache line utilization and page utilization, reduce branch instruction jumps to reduce the number of branch prediction failures, and improve the execution efficiency of the application's instruction flow. However, most applications call a large number of library files and do not have obvious hot libraries. Hot functions and data blocks cannot be rearranged between multiple library images, resulting in lower benefits.

[0057] In another possible implementation, to improve the efficiency of program instruction flow, computer systems can use big page technology. Currently, operating systems typically use 4KB physical pages. Big page technology uses physical pages of 16KB, 64KB, or 2MB to reduce the number of page faults, reduce cache hits in the CPU microarchitecture, and improve the efficiency of application instruction flow. However, with a large number of library files and a small size for each file, using big page technology can lead to lower physical memory utilization.

[0058] Therefore, this application provides a file processing method that maps multiple first library files to a contiguous virtual address space. Specifically, it maps multiple first library files with different random virtual address spaces to a contiguous virtual address space. This allows for global rearrangement of the second library files to optimize the application's instruction flow. Compared to optimizing the instruction flow of a single library file, the file processing method provided in this application can optimize the instruction flow of multiple first library files globally. This means it can optimize both single and cross-library files, reducing cache hit failures and branch prediction failures, thereby improving the execution efficiency of the application's instruction flow.

[0059] The document processing method provided in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0060] This application provides a file processing method, such as... Figure 2 As shown, Figure 2 A flowchart of a file processing method is shown. This file processing method is applied to the aforementioned computer system. Of course, the entity executing the file processing method can also be a device / module in the computer system, such as a chip, processor, or processing unit in the computer system. This application embodiment does not limit this. The file processing method includes the following steps.

[0061] S201. Obtain multiple first library files.

[0062] For example, the first library file may include static libraries and dynamic libraries. A static library is a set of pre-compiled object files (files with the .o or .obj extension) that include the implementation of functions and classes. Static libraries exist as files, and during the linking phase, the linker completely copies the code from the static library that is called by the application into the final executable file.

[0063] Additionally, a dynamic library, also known as a dynamic link library (DLL), is a library containing code and data that can be used by multiple programs simultaneously. A dynamic library can also be understood as an executable and linkable format (ELF) file or function. On one operating system (such as Windows), dynamic libraries have the .dll extension, while on another operating system (Linux), they have the .so extension.

[0064] S202. Map multiple first library files to a contiguous virtual address space to obtain second library files.

[0065] For example, a virtual address space is a separate and contiguous range of addresses provided by the operating system for each process executing an application, allowing the process to believe it owns the entire system's memory resources. The virtual address space uses virtual addresses, which allow processes to use a larger address range than the actual physical memory and can dynamically swap data and code between physical memory and disk, improving system memory utilization and multitasking capabilities. Processes use virtual addresses to access memory without needing to consider the actual physical memory layout and allocation. Corresponding to virtual addresses are physical addresses, which are the hardware addresses where data is actually stored in memory. Physical addresses correspond to the location of storage units on the physical memory chips and are the addresses that memory devices actually use to read and write data. When a computer system needs to access data in memory, it must translate the virtual address into a physical address to locate the corresponding data in the actual physical memory.

[0066] For example, multiple first library files are distributed across different random virtual address spaces. Existing technologies can use instruction stream boost technology (ISBT) to optimize the instruction stream of a single first library file. Instruction stream optimization relies on the compiler and linker, whose scope is limited to individual binary files. For instance, profile-guided optimization (PGO) and binary optimization and layout tool (BOLT) techniques can both optimize instruction streams. These techniques can call instruction streams based on hot data blocks, thereby performing instruction layout optimization and instruction prefetching. However, existing technologies are limited to single library files. For cross-library file calls, due to the randomization of library file addresses, it is impossible to predict the library file addresses and achieve cross-library file layout optimization, thus failing to obtain the global benefits of instruction optimization.

[0067] In this embodiment, multiple first library files with different random virtual address spaces are mapped to consecutive virtual address spaces to obtain second library files. This allows for global instruction-related optimization techniques to be applied to the second library files, enabling optimization of both individual library files and cross-library files.

[0068] S203. Rearrange the second library file to obtain the third library file.

[0069] For example, a library file may include a data segment and a code segment. In one possible example, the code segment may include initialization code (with the suffix .init), exit code (with the suffix .fini), text code (with the suffix .text), and branch jump code (with the suffix .plt), etc., and the data segment may include initialization data (with the suffix .init.array) and exit data (with the suffix .fini.array), etc.

[0070] For example, the computer system may rearrange only the code segment of the second library file (i.e., rearrange it), or rearrange both the code segment and the data segment of the second library file at the same time. This application embodiment does not limit this.

[0071] Specifically, taking the reordering of code segments in the second library file as an example, a code segment can include multiple functions. The order in which these functions are defined affects their arrangement in the final library file. Placing code corresponding to functions that are typically called simultaneously in adjacent positions improves the flexibility of the code segment. Alternatively, the code can be modularized, encapsulating related functionalities into different modules. These modules are then linked in a specific order during linking, thus controlling the code arrangement in the library file. This reduces the number of cache hit failures and branch prediction failures, improving the execution efficiency of the application's instruction flow.

[0072] Optionally, the second library file includes multiple first code segments. S203 may include: obtaining the execution count of each first code segment in the second library file, arranging multiple first code segments with execution counts greater than a preset threshold consecutively, and arranging multiple first code segments with execution counts less than or equal to the preset threshold consecutively to obtain a third library file.

[0073] For example, the execution count refers to the call frequency of a code segment, the number of times it has been called during the previous operation of the application. A code segment with an execution count greater than a preset threshold can be considered a "hot" code segment, while a code segment with an execution count less than or equal to the preset threshold can be considered a "cold" code segment. The preset threshold can be set by those skilled in the art according to optimization needs, and this application embodiment does not limit it in this regard.

[0074] like Figure 3 As shown, Figure 3 The flowchart of another file processing method is shown in the figure. Figure 3 The document specifically illustrates multiple first library files, such as first library file _A and first library file _B. First library file _A and first library file _B each include multiple hot code segments and multiple cold code segments, which are randomly distributed. Mapping these multiple first library files to a contiguous virtual address space yields a second library file, where the hot and cold code segments are also randomly distributed. Rearranging the second library file yields a third library file; this rearrangement can be achieved by arranging the hot code segments consecutively or by arranging the cold code segments consecutively. Therefore, the file processing method provided in this application can achieve the aggregation of globally hot code segments, improve the limitation of code segments, reduce the number of cache hit failures, branch jump failures, and branch prediction failures, and improve the execution efficiency of the instruction flow.

[0075] Understandably, the second library file can include multiple first data segments, which can also be divided into hot data segments and cold data segments. This rearrangement method can also arrange multiple hot data segments consecutively, or multiple cold data segments consecutively, to achieve the aggregation of globally hot data segments, improve the limitation of data segments, reduce the number of cache hit failures, and improve the execution efficiency of the instruction stream.

[0076] Optional, such as Figure 4 As shown, Figure 4 The diagram illustrates a process for determining the number of executions. In one possible implementation, step S203 may specifically include: compiling the original code corresponding to each first code segment to obtain a relocation binary, and obtaining the number of executions for each first code segment based on the relocation binary.

[0077] For example, a computer system can use a compiler to compile raw code. The compiler's process can include: preprocessing, compilation, assembly, relocation information generation, and linking. Preprocessing involves processing preprocessor directives in the raw code to generate a preprocessed source file. Compilation converts the preprocessed source file into an assembly file. Assembly converts the assembly file into an object file, which is in binary format and contains unresolved symbols and relocation information, so it cannot be run directly. Relocation information generation records the location and type of symbol references (such as function calls and global variable references) in the object file for address correction in the subsequent linking stage. Linking combines multiple object files and library files to generate a relocated binary file. In this relocated binary file, symbol references have been replaced with actual addresses, allowing the application to load and run in the operating system.

[0078] For example, a computer system can use a static binary (SBI) instrumentation tool to analyze a relocation binary file. After running the relocation binary file, the SBI instrumentation tool collects relevant information and generates profile data. After obtaining the profile data, further analysis can be performed, such as identifying frequently called functions in the program, i.e., obtaining the execution count of the first code segment in this embodiment.

[0079] Continue reading Figure 4In another possible implementation, S203 may specifically include: obtaining the original binary based on the original code corresponding to each first code segment, sampling the original binary, and obtaining the execution count of each first code segment.

[0080] For example, a computer system can use a compiler to process the original code to obtain the raw binary. The compiler's processing flow can include: preprocessing, compilation, assembly, and linking. The compiler's processing flow to obtain the raw binary does not include the generation of relocation information. Other processing flows for obtaining the raw binary are similar to those for obtaining the relocated binary file, as explained above, and will not be repeated here.

[0081] For example, a computer system can perform binary rewriting based on execution (BRBE) on the original binary code to obtain analysis data. Specifically, the computer system can rewrite the binary code during the execution of the original binary code, inserting code for collecting runtime information, such as function call counts, instruction execution frequency, and memory access patterns—that is, analysis data. After obtaining the analysis data, further analysis can be performed, for example, identifying frequently called functions in the program, thus obtaining the execution count of the first code segment in this embodiment.

[0082] Optionally, the first library file includes a second data segment and a second code segment. S202 may include: merging multiple first library files to obtain a fourth library file, the fourth library file including a third data segment, a third code segment, and a merged segment. The third data segment includes multiple second data segments, the third code segment includes multiple second code segments, and the merged segment includes the mapping relationship between the fourth library file and the multiple first library files. The fourth library file is mapped to a contiguous virtual address space to obtain a second library file.

[0083] For example, such as Figure 5 As shown, Figure 5 The diagram shows a flowchart of yet another file processing method. Figure 5 The document specifically illustrates first library file _A and first library file _B. First library file _A and first library file _B each include multiple second code segments, such as .init, .fini, .text, and .plt, and multiple second data segments, such as .init.array and .fini.array. Additionally, Figure 5The document also shows a fourth library file, which is the merged version of the first library file _A and the second library file _B. This fourth library file includes multiple third code segments, such as .init_A, .fini_A, .text_A, .init_B, .fini_B, and .text_B. The code segments with the suffix 'A' originate from the code segments in the first library file _A, and the code segments with the suffix 'B' originate from the code segments in the first library file _B. The fourth library file also includes multiple third data segments, such as .init.array_A, .fini.array_A, .init.array_B, and .fini.array_B. The data segments with the suffix 'A' originate from the data segments in the first library file _A, and the data segments with the suffix 'B' originate from the data segments in the first library file _B. In addition, the fourth library file includes a merged segment with the suffix '.adlt'. '.adlt' can include a header and a binary blob. The header can include identifiers indicating the location of the data, and the binary blob can include array information, i.e., the location where the actual data is stored. The merged segment can include a mapping between the fourth library file and multiple first library files. This mapping can include the names and relocation entries of the first library files. By parsing the merged segment, the information of the original first library files can be reconstructed. Understandably, the fourth library file also includes merged code segments, such as .plt and .got.plt, which can be used for function calls.

[0084] Understandably, when merging multiple first library files, a library file merging tool of the computer system can be used. This library file merging tool can support user-unnoticed technology, debugging support technology, and component upgrade technology, meaning that the actions performed on the first library files in this embodiment are imperceptible to the developers of the first library files.

[0085] For example, due to the nature of dynamic loading, the existing technology for calling functions across library files is as follows: the original address of the function call initiation location in the first library file is usually the address of the procedure link table (PLT). Then, the call jumps to the PLT through the address of the PLT, and then jumps to the global offset table (GOT) through the PLT. The corresponding library file function is then called through the GOT. This two-step jump affects the processor's instruction execution efficiency.

[0086] In this embodiment of the application, multiple first library files are merged into a fourth library file, which allows for direct function calls, reduces the number of branch instructions, and improves instruction execution efficiency.

[0087] The above-mentioned process regarding the prior art can be found in [reference]. Figure 6Understand (a) in the text. For example... Figure 6 As shown in (a), the first library file _A may contain function _A, and the starting position of the call initiation location for function _A is .plt. When function _A is executed, it first jumps to PLT based on .plt. In PLT, .got.plt is found through .plt, and then jumps to GOT based on .got.plt. In GOT, the address of function _B in the first library file _B is found through .got.plt, and then function _B is called based on the address of function _B.

[0088] In the embodiments of this application, such as Figure 6 As shown in (b), the starting address of the function _A in the first library file _A is the address of the function _B in the first library file _B, so function _B can be called directly.

[0089] Optionally, S202 may include: mapping multiple first library files sequentially to a contiguous virtual address space, with the end address of the previous first library file and the start address of the next library file being contiguous, to obtain a second library file.

[0090] For example, such as Figure 7 As shown, Figure 7 The diagram shows flowcharts of various file processing methods. Figure 7 The document specifically illustrates first library file _A and first library file _B. First library file _A and first library file _B each include multiple second code segments, such as .init, .fini, .text, and .plt, and multiple second data segments, such as .init.array and .fini.array. Additionally, Figure 7 The document also shows a second library file, which is a contiguous mapping of the first library files _A and _B. The second library file includes adjacent first library files _A and _B, with the end address of first library file _A and the start address of first library file _B being consecutive. Understandably, the fourth library file also includes merged code segments, such as .plt and .got.plt, which can be used for function calls.

[0091] In one possible example, the end address of the first library file _A and the start address of the first library file _B can be in the same physical page. This contiguous mapping improves the utilization of the virtual address space. In another possible example, the end address of the first library file _A and the start address of the first library file _B can be in adjacent physical pages. This mapping according to integer multiples of physical pages improves the convenience of virtual address space management.

[0092] Optionally, the file processing method may also include mapping the virtual address space corresponding to the third library file to large page memory.

[0093] For example, a huge page, also known as a large page memory, is a physical page larger than 4KB, such as 16KB, 64KB, 1MB, or 2MB.

[0094] like Figure 8 As shown, Figure 8 The diagram illustrates a library file mapping. For example... Figure 8 As shown in (a), the hot and cold code segments in the first library file _A and the first library file _B are respectively mapped to 4KB physical pages. Figure 8 As shown in (b), hot code segments in the third library file can be mapped to large pages of memory, such as 16KB / 1MB / 2MB physical pages.

[0095] For example, such as Figure 9 As shown, Figure 9 The diagram illustrates a mapping between virtual addresses and physical memory. Figure 9 The diagram illustrates a four-level page table structure. A virtual address can include the Page Global Directory (PGD), Page Middle Directory (PMD), Page Table Entry (PTE), and Page Offset. The PGD is the first-level directory for virtual address translation, and can be considered the highest-level index. A PGD can include multiple entries (e.g., PGD_0 to PGD_511), each pointing to a PMD. The PMD is the intermediate level in the four-level page table structure, located between the PGD and PTE. A PMD can also include multiple entries (e.g., PMD_0 to PMD_511), each pointing to a PTE. The PTE is the crucial link in the virtual-to-physical address translation. It stores the mapping relationship between virtual and physical pages, as well as related control information, such as page access permissions (read, write, or execute permissions), whether the page is in memory (presence bit), and whether it has been modified (dirty bit). Page Entries (PTEs) can include multiple entries (e.g., PTE_0 to PTE_511). PTEs can accurately translate virtual addresses into corresponding physical addresses, enabling effective control and management of page access. Page offsets determine the specific location of a virtual address within a page. Since the page size is fixed, page offsets allow for precise location of specific bytes or data items within a defined physical page.

[0096] In the prior art, one PTE corresponds to one 4KB physical page, requiring 512 PTEs for a 2MB storage space. In this embodiment, one PTE corresponds to one 2MB physical page, requiring only one PMD for a 2MB storage space.

[0097] For example, in this embodiment of the application, for a third library file that merges multiple first library files, since the code segments are all clustered in a contiguous virtual address space, using big-page memory mapping for the third library file will not cause a significant waste of physical memory. Specifically, the number of big pages occupied by a library file of the same size is much less than the number of 4KB physical pages occupied. Once the page table entries of the big pages are cached in the translation lookup buffer (TLB), they will not be quickly deleted due to an excessive number of page table entries in the TLB. When the application or other applications call the library file, they can query the page table entries of the library file in the TLB, improving the TLB hit rate.

[0098] In this embodiment, the TLB includes an instruction translation lookaside buffer (iTLB) and a data translation lookaside buffer (dTLB). The dynamic library includes code segments and data segments. The page table entries of the code segments can be cached in the iTLB. Using large page memory to store the code segments can improve the hit rate of querying the iTLB. Similarly, the page table entries of the data segments can be cached in the dTLB. Using large page memory to store the code segments can improve the hit rate of querying the dTLB.

[0099] It is understood that, in order to achieve the above functions, the electronic device includes hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0100] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0101] When dividing each function into modules according to its corresponding function. Figure 10 A schematic diagram of a possible composition of the file processing apparatus 1000 involved in the above embodiments is shown, such as... Figure 10 As shown, the file processing device 1000 may include: an acquisition module 1001, a processing module 1002, and a rearrangement module 1003.

[0102] The acquisition module 1001 can be used to support the file processing device 1000 in performing the above-described steps S201, and / or other processes used in the technology described herein.

[0103] The processing module 1002 can be used to support the file processing device 1000 in performing the above-described steps S202, etc., and / or other processes used in the techniques described herein.

[0104] The rearrangement module 1003 can be used to support the file processing device 1000 in performing the above-described steps S203, and / or other processes for the techniques described herein.

[0105] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0106] The file processing device 1000 provided in this embodiment is used to execute the above-described file processing method, and therefore can achieve the same effect as the above-described implementation method.

[0107] When using integrated units, the file processing device 1000 may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the operations of the file processing device 1000, for example, to support the file processing device 1000 in executing the steps performed by the acquisition module 1001, processing module 1002, and rearrangement module 1003. The storage module can be used to support the file processing device 1000 in storing program code and data. The communication module can be used to support communication between the file processing device 1000 and other devices, such as communication with a wireless access device.

[0108] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0109] This application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the aforementioned method steps to implement the file processing method described above.

[0110] Embodiments of this application also provide a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the file processing method described above.

[0111] Embodiments of this application also provide a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the file processing method executed by the electronic device in the above embodiments.

[0112] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the file processing methods executed by the electronic devices in the above-described method embodiments.

[0113] In this embodiment, the file processing device, electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0114] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0118] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A file processing method, characterized in that, The method includes: Obtain multiple first library files; The multiple first library files are mapped to a contiguous virtual address space to obtain the second library files; The second library file is rearranged to obtain the third library file.

2. The method according to claim 1, characterized in that, The second library file includes multiple first code segments. The second library file is rearranged to obtain a third library file, which includes: Obtain the number of times each first code segment in the second library file is executed; The third library file is obtained by arranging multiple first code segments whose execution count is greater than a preset threshold consecutively, and by arranging multiple first code segments whose execution count is less than or equal to the preset threshold consecutively.

3. The method according to claim 2, characterized in that, Obtain the execution count of each first code segment in the second library file, including: The original code corresponding to each first code segment is compiled to obtain a relocation binary file; Based on the relocation binary file, the number of executions for each first code segment is obtained.

4. The method according to claim 2, characterized in that, Obtain the execution count of each first code segment in the second library file, including: The original binary is obtained based on the original code corresponding to each first code segment; The original binary data is sampled to obtain the number of times each first code segment is executed.

5. The method according to any one of claims 1-4, characterized in that, The first library file includes a second data segment and a second code segment. Mapping the plurality of first library files to a contiguous virtual address space yields a second library file, including: The multiple first library files are merged to obtain a fourth library file. The fourth library file includes a third data segment, a third code segment, and a merge segment. The third data segment includes multiple second data segments, the third code segment includes multiple second code segments, and the merge segment includes the mapping relationship between the fourth library file and the multiple first library files. The fourth library file is mapped to a contiguous virtual address space to obtain the second library file.

6. The method according to any one of claims 1-4, characterized in that, The plurality of first library files are mapped to a contiguous virtual address space to obtain second library files, including: The multiple first library files are sequentially mapped to a continuous virtual address space, with the end address of the previous first library file and the start address of the next first library file being continuous, to obtain the second library file.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Map the virtual address space corresponding to the third library file to large page memory.

8. A document processing device, characterized in that, include: The module includes an acquisition module, a processing module, and a rearrangement module. The acquisition module is used to acquire multiple first library files; The processing module is used to map the plurality of first library files to a contiguous virtual address space to obtain second library files; The rearrangement module is used to rearrange the second library file to obtain the third library file.

9. The apparatus according to claim 8, characterized in that, The rearrangement module is specifically used for: Obtain the number of times each first code segment in the second library file is executed; The third library file is obtained by arranging multiple first code segments whose execution count is greater than a preset threshold consecutively, and by arranging multiple first code segments whose execution count is less than or equal to the preset threshold consecutively.

10. The apparatus according to claim 9, characterized in that, The rearrangement module is specifically used for: The original code corresponding to each first code segment is compiled to obtain a relocation binary file; Based on the relocation binary file, the number of executions for each first code segment is obtained.

11. The apparatus according to claim 9, characterized in that, The rearrangement module is specifically used for: The original binary is obtained based on the original code corresponding to each first code segment; The original binary data is sampled to obtain the number of times each first code segment is executed.

12. The apparatus according to any one of claims 8-11, characterized in that, The first library file includes a second data segment and a second code segment. The processing module is specifically used for: The multiple first library files are merged to obtain a fourth library file. The fourth library file includes a third data segment, a third code segment, and a merge segment. The third data segment includes multiple second data segments, the third code segment includes multiple second code segments, and the merge segment includes the mapping relationship between the fourth library file and the multiple first library files. The fourth library file is mapped to a contiguous virtual address space to obtain the second library file.

13. The apparatus according to any one of claims 8-11, characterized in that, The processing module is specifically used for: The multiple first library files are sequentially mapped to a continuous virtual address space, with the end address of the previous first library file and the start address of the next first library file being continuous, to obtain the second library file.

14. The apparatus according to any one of claims 8-13, characterized in that, The processing module is further configured to: Map the virtual address space corresponding to the third library file to large page memory.

15. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any one of claims 1-7.

Images