File processing method, apparatus, system and related device

Abstract

Provided in the present application are a file processing method, an apparatus, a system and a related device. The method comprises: acquiring a file set, the file set comprising executable codes of a plurality of first operators and executable codes of a plurality of second operators; acquiring first identifiers of first operators required by a first processor and second identifiers of second operators required by a second processor (heterogeneous with the first processor); according to the first identifiers, filtering the executable codes of the plurality of first operators, so as to obtain a first file; according to the second identifiers, filtering the executable codes of the plurality of second operators, so as to obtain a second file; and using the first processor and the second processor to execute computational tasks according to the first file and the second file. In this way, the amount of executable code data required for storage and loading during execution of computational tasks can be effectively reduced, thus effectively reducing the storage space occupied during the execution of the computational tasks, and decreasing the consumption of storage resources.

Description

File processing method, device, system and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 30, 2023, with application number 202311110182.3 and application name “File processing methods, devices, systems and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a file processing method, apparatus, system and related equipment. Background Art

[0003] Heterogeneous computing refers to computing devices that utilize different types of processors to collaborate and complete computing tasks, thereby improving the device's processing efficiency. For example, a computing device might include both a central processing unit (CPU) and a graphics processing unit (GPU). When executing a computing task, the CPU can perform preprocessing on the data, such as cleaning and deduplication, while the GPU performs vector calculations on the preprocessed data (GPUs are more efficient at vector calculations). Finally, the CPU performs computations on the vector calculation results, such as converting them into scalar quantities. In this way, the computing device can leverage the GPU to accelerate the completion of the computing task and improve processing efficiency.

[0004] The operations performed by the computing device are referred to as operators. Furthermore, different processors in the computing device can execute different parts of the executable code of the operator to complete the data processing operations corresponding to the operator. The executable code can be, for example, binary code.

[0005] In actual applications, the heterogeneous processors on computing devices can be divided into host-side processors and device-side processors. Manufacturers usually integrate all operators that can be executed by the host-side processor and all operators that can be executed by the device-side processor, and centrally release the executable code of all operators. Accordingly, the computing device can store the executable code of all operators locally. In this way, the host-side processor and the device-side processor can perform corresponding data processing operations based on the locally stored executable code to complete the computing tasks generated by the application on the computing device.

[0006] However, the executable code of full operators is too large, which requires host-side and device-side processors to consume a large amount of storage resources when performing computing tasks based on this executable code. This limits the application scenarios in which heterogeneous computing can be applied. For example, computing devices in edge environments have limited storage space, so excessive storage resource consumption will hinder the deployment of heterogeneous computing on computing devices.

[0007] Summary of the Invention

[0008] In view of this, embodiments of the present application provide a file processing method for reducing the storage resources required by host-side processors and device-side processors in a computing device when performing computing tasks, thereby enriching the application scenarios applicable to heterogeneous computing. This application also provides corresponding apparatuses, systems, computing devices, computer-readable storage media, and computer program products.

[0009] In a first aspect, an embodiment of the present application provides a file processing method, which is applied to a computing device, wherein the computing device includes a first processor and a second processor, and the first processor and the second processor are heterogeneous, such as the first processor can be a CPU, the second processor can be a GPU or a DPU, etc.; in the process of executing file processing, a file set is first obtained, and the file set includes executable codes of multiple first operators and executable codes of multiple second operators. Usually, the operator that can be executed by the first processor is the first operator, and the operator that can be executed by the second processor is the second operator, and the first processor and the second processor can collaboratively execute the same operator, or the first processor can execute one alone. operator; then, obtaining a first identifier of a first operator required to be executed when the first processor performs a computing task, and a second identifier of a second operator required to be executed when the second processor performs the computing task, and filtering the executable codes of multiple first operators in the file set according to the first identifier to obtain a first file, the obtained first file including the executable code of the first operator indicated by the first identifier, and filtering the executable codes of multiple second operators in the file set according to the second identifier to obtain a second file, the obtained second file including the executable code of the second operator indicated by the second identifier; finally, using the first processor and the second processor, according to the filtered first file and the second file, execute the computing task.

[0010] In this way, before executing a computing task, the executable code including multiple operators is filtered according to the identifiers of the operators that each processor needs to execute, which can effectively reduce the amount of data of the executable code that needs to be stored and loaded during the execution of the computing task. That is, the executable code of the operators that each processor does not need to execute does not need to be stored locally on the computing device, which can effectively reduce the storage space occupied during the execution of the computing task, that is, reduce the consumption of storage resources, and reduce the requirements of heterogeneous computing for storage resources on the computing device, thereby enriching the application scenarios applicable to heterogeneous computing. In addition, when the number of first operators that the first processor needs to execute is multiple, or the number of second operators that the second processor needs to execute is multiple, the executable codes of the multiple first operators or the multiple second operators are packaged into one file, which enables the first processor to reduce the file loading delay when loading the packaged file (i.e., the first file or the second file) into the memory, thereby improving the execution efficiency of the computing task.

[0011] In one possible implementation, in the process of filtering the executable codes of multiple first operators according to a first identifier and obtaining a first file, the executable codes of the multiple first operators corresponding to the first identifier may be first filtered to obtain the executable code of the first operator indicated by the first identifier, and then the executable code of the first operator indicated by the first identifier may be packaged to obtain the first file. In this way, the packaged first file may include the executable code of one or more first operators indicated by the first identifier, so that when the first processor loads the packaged first file into memory, the file loading delay can be reduced, thereby improving the execution efficiency of the computing task.

[0012] Similarly, in the process of generating the second file, the executable codes of multiple second operators corresponding to the second identifier can be filtered first to obtain the executable code of the second operator indicated by the second identifier, and then the executable code of the second operator indicated by the second identifier can be packaged to obtain the second file.

[0013] In a possible embodiment, when executing a computing task according to the first file and the second file, the first file can be parsed first to obtain at least one first executable file, wherein each first executable file is used to record the executable code of a first operator, and the second file can be parsed to obtain at least one second executable file, wherein each second executable file is used to record the executable code of a second operator, so that the first processor can execute the at least one first executable file, and according to the execution order of the at least one first executable file, the second processor can be controlled to execute the at least one second executable file in sequence. In this way, the first processor and the second processor can realize the collaborative execution of computing tasks according to the executable code included in the files required by each of them, and the first file and the second file are packaged files, which can reduce the file loading delay when they are loaded into the memory, thereby improving the execution efficiency of the computing task.

[0014] In a possible implementation, the formats of the first file and the second file are binary file formats, such as ".bin" format.

[0015] In a possible implementation, the first executable file is a dynamic link library file, and each dynamic link library file records the binary code of a first operator, while the second executable file is an object file, and each object file records the binary code of a second operator.

[0016] In one possible implementation, during file processing, the lightweight first and second files may be stored in a file system on the computing device. This file system may be constructed, for example, using memory on the computing device. In this way, the computing device only needs to store the lightweight first and second files, eliminating the need for a heavyweight file collection. This reduces the storage resources required to execute the computing task.

[0017] In a possible implementation, during the file processing process, a third file and a fourth file can also be obtained from the file system. The third file and the fourth file can be obtained by filtering the file set according to the above process, for example. The third file includes executable files of multiple third operators required to be executed when the first processor executes the target model, and the fourth file includes executable files of at least one fourth operator required to be executed when the second processor executes the target model. The at least one fourth operator is a subset of the multiple third operators, that is, the fourth operator is an operator executed collaboratively by the second processor and the first processor; then, the third file is parsed to obtain multiple third executable files, each of the multiple third executable files is used to record the executable code (such as binary code, etc.) of a third operator, and the fourth file is parsed to obtain multiple fourth executable files, each of the fourth executable files is used to record the executable code of a fourth operator; finally, the executable code in the multiple third executable files and the executable code in the at least one fourth executable file are added to the file of the target model to obtain an offline model file, and the offline model file is output. In this way, after the offline model file is deployed to a computing device including multiple heterogeneous processors, due to the executable code of multiple operators integrated in the offline model file, the computing device does not need to obtain the file set including the executable code of the full set of operators released by the manufacturer. Instead, it can execute the multiple operators in the target model based on multiple heterogeneous processors by executing the executable code integrated in the offline model file, thereby realizing the running of the target model on the computing device and reducing the requirements for the storage resources of the computing device.

[0018] In one possible implementation, when obtaining the identifier of a first operator to be executed by a first processor and the second identifier of a second operator to be executed by a second processor, historical operation data may be first obtained, where the historical operation data includes data generated by the first and second processors when executing the computing task according to the file set. The first and second identifiers are then determined based on the historical operation data. In this way, the operators to be executed by the first and second processors are automatically detected based on the data records of the first and second processors' past execution of computing tasks.

[0019] In a possible implementation, the identifiers of the plurality of second operators match the identifiers of some of the plurality of first operators, wherein the operators with matching identifiers are the operators executed collaboratively by the first processor and the second processor.

[0020] In a second aspect, the present application provides a file processing device, which includes various modules for executing the file processing method in the first aspect or any possible implementation of the first aspect.

[0021] In a third aspect, the present application provides a file processing system, which includes a first processor, at least one second processor, a memory, and a file processing device. The first processor and the at least one second processor are heterogeneous, such as the first processor is a CPU, and the file system includes multiple processors such as a GPU and a DPU that are heterogeneous with the first processor; the file processing device is used to execute the method described in the first aspect or any implementation method of the first aspect; the functions of the first processor, at least one second processor, and the memory can be referred to the relevant description of the first aspect above.

[0022] In a fourth aspect, the present application provides a computing device comprising a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to execute the file processing method described in the first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or independent of the processor. The computing device may also include a bus. The processor is connected to the memory via the bus. The memory may include a readable memory and a random access memory.

[0023] In a fifth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on at least one computing device, the at least one computing device executes the method described in the first aspect or any implementation of the first aspect.

[0024] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0025] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1 is a schematic diagram of the structure of an exemplary computing device provided in an embodiment of the present application;

[0027] FIG2 is a schematic diagram of an operator package;

[0028] FIG3 is a flowchart of a file processing method provided in an embodiment of the present application;

[0029] FIG4 is a schematic diagram of executing a computing task according to a first file and a second file;

[0030] FIG5 is a flow chart of another file processing method provided in an embodiment of the present application;

[0031] FIG6 is a schematic structural diagram of a file processing device provided in an embodiment of the present application;

[0032] FIG7 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will describe the solutions in the embodiments provided in this application in conjunction with the drawings in this application.

[0034] Referring to Figure 1, a schematic diagram of the structure of a computing device is shown. As shown in Figure 1, the computing device 10 includes multiple processors, which are respectively located on the host side and the device side. For ease of understanding and explanation, Figure 1 takes the host side including the processor 101 and the device side including the processor 102 as an example for illustrative explanation. In actual application, the number of processors included in the host side and the device side can be one or more. The processor 101 and the processor 102 can be connected via a bus, such as via a Compute Express Link (CXL) bus. In addition, the computing device 10 also includes a memory 103 and a file processing device 104, as shown in Figure 1. Alternatively, the computing device 10 may also include other components.

[0035] The processor 101 on the host side is used to independently execute an operator, specifically to execute the executable code (such as binary code, etc.) corresponding to the operator, or to cooperate with the processor 102 on the device side to execute the same operator. For example, the processor 101 can be a general-purpose processor, such as a central processing unit (CPU), etc.

[0036] The processor 102 on the device side is used to execute the same operator in collaboration with the processor 101 on the host side. Specifically, the processor 101 may execute part of the executable code of the operator, and the processor 102 may execute another part of the executable code of the operator. Exemplarily, the processor 102 may be any type of processor such as a data processing unit (DPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), or other types of processors, and this is not limited. Moreover, the processor 102 and the processor 101 are heterogeneous processors. For example, when the processor 101 is a CPU, the processor 102 may be a DPU or a GPU, etc., and may execute the same operator in collaboration with the processor 101 to accelerate the processing of computing tasks.

[0037] Memory 103 is used to store executable code executed by processors 101 and 102, such as by recording the executable code in the form of files. In actual applications, a file system can be constructed based on memory 103 in computing device 10, and the file system can manage the executable files in memory 103 accordingly. For example, memory 103 can be a solid-state drive (SSD), a magnetic disk, or the like.

[0038] The file processing device 104 is used to perform corresponding processing on the executable codes of the operators executed by the processor 101 and the processor 102, so as to reduce the occupation of the storage resources of the computing device 10 during the execution of the computing task.

[0039] Specifically, because manufacturers usually produce operator packages as shown in Figure 2 in order to adapt to the task execution process under various application scenarios, the operator package includes the executable code of all operators that may be executed by the processors on the host side and the device side. The executable code can be, for example, binary code. Among them, the binary code of all operators that may be executed by the processor 101 on the host side will be placed in a larger executable file, while the binary code of all operators that may be executed by the processor 102 on the device side will be stored in a dispersed manner as multiple smaller executable files (such as dispersed storage in the form of hundreds of small files, etc.). Due to the large file size of the operator package, not only does the memory 103 need to consume a large storage space to store the operator package, but the processor 101 needs to occupy a large memory space when loading the operator package (loading it into the memory in the processor 101). In addition, since the operator package includes multiple small files (executable code of all operators that may be executed by the processor 102), it takes a long time for these small files to be loaded into the memory, thereby affecting the overall execution efficiency of the computing task.

[0040] Based on this, in the computing device 10 provided in this embodiment, the file processing device 104 first obtains a file set including a full set of operators (such as the above-mentioned operator package) released by the manufacturer, and obtains the first identifier of the first operator required to be executed when the processor 101 performs the computing task, and the second identifier of the second operator required to be executed when the processor 102 performs the computing task. Then, the file processing device 104 filters the executable codes of multiple first operators in the file set according to the first identifier to obtain file 1, which includes the executable code of the first operator indicated by the first identifier (that is, the executable code obtained by filtering according to the first identifier), and filters the executable codes of multiple second operators in the file set according to the second identifier to obtain file 2, which includes the executable code of the second operator indicated by the second identifier (that is, the executable code obtained by filtering according to the second identifier). In this way, the file processing device 104 can use the processor 101 and the processor 102 to perform the computing task based on the filtered files 1 and 2.

[0041] In this way, before executing the computing task, the file processing device 104 filters the executable code in the file collection according to the identification of the operator that each processor needs to execute. This can effectively reduce the amount of data of the executable code that needs to be stored and loaded in the process of executing the computing task. That is, the executable code of the operator that each processor does not need to execute does not need to be stored locally on the computing device, thereby effectively reducing the storage space occupied in the process of executing the computing task, that is, reducing the consumption of storage resources, and reducing the requirements of heterogeneous computing for storage resources on the computing device, thereby enriching the application scenarios applicable to heterogeneous computing.

[0042] Moreover, when the number of first operators that the processor 101 needs to execute is multiple, or the number of second operators that the processor 102 needs to execute is multiple, the executable codes of the multiple first operators or the multiple second operators can be packaged into one file, which enables the processor 101 to reduce the file loading delay when loading the packaged file (i.e., file 1 or file 2) into the memory on the host side, thereby improving the execution efficiency of the computing task.

[0043] Illustratively, the file processing device 104 shown in FIG. 1 may be implemented by software or hardware.

[0044] In one example, when implemented by software, the file processing device 104 may be code running on the computing device 10 including the processor 101 (and the processor 102 ), such as a computing engine, a plug-in, etc. on the computing device 10 .

[0045] In another example, when implemented by hardware, the file processing device 104 can be hardware with data processing capabilities, etc. For example, the hardware with data processing capabilities can be a CPU, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system on chip (SoC), a software-defined infrastructure (SDI) chip, an artificial intelligence (AI) chip, a data processing unit (DPU), or any combination thereof. In addition, the number of hardware with data processing capabilities included in the file processing device 104 can be one or more, and the type of hardware included can be one or more. The number and type of the hardware can be set according to the business requirements of the actual application, and this embodiment does not limit this.

[0046] It is worth noting that the computing device 10 shown in FIG1 is merely an example. In other possible implementations, the computing device 10 may include more types of components to support more functions of the computing device 10; or the computing device 10 may include a larger number of processors, etc. The specific structure of the computing device 10 is not limited in this application.

[0047] Next, various non-limiting specific implementations of the file processing process are described in detail.

[0048] Refer to FIG3 , which is a flowchart of a file processing method according to an embodiment of the present application. This method can be applied to the computing device 10 shown in FIG1 , or can also be applied to other possible computing devices. For ease of understanding, the following description will be based on the application to the computing device 10 shown in FIG1 . The file processing method shown in FIG3 may specifically include:

[0049] S301: The file processing device 104 obtains a file set, where the file set includes executable codes of multiple first operators and executable codes of multiple second operators.

[0050] Among them, the first operator is the operator executed by the processor 101 on the host side. Normally, the processor 101 can execute a first operator alone. Alternatively, the processor 101 can execute the same operator in collaboration with the processor 102. The operator is the first operator for the processor 101, and is referred to as the second operator for the processor 102. Normally, the processor 102 does not execute an operator alone; accordingly, the identifier of the second operator that the processor 102 can execute matches the identifier of some of the first operators among the multiple first operators that the processor 101 can execute, and the operators with matching identifiers are also the operators executed in collaboration by the processors 101 and 102.

[0051] For example, when processor 101 and processor 102 collaborate to execute the same operator, processor 101 may generally first execute the first calculation phase of the operator and control processor 102 to execute the second calculation phase of the operator. Finally, processor 101 may continue to execute the third calculation phase based on the result generated by processor 102 executing the second calculation phase, thereby completing the execution process of the operator. Each processor executes a corresponding calculation phase based on the executable code of a different part of the operator.

[0052] In this embodiment, the file processing device 104 may first obtain executable codes of multiple operators that may be executed by the processor 101 and the processor 102. The executable codes of the operators may be, for example, binary codes, assembly language codes, or other types of executable codes.

[0053] In a first implementation example, the file processing device 104 may first obtain a file set released by a manufacturer. The file set may be, for example, an operator package as shown in FIG2 , including a dynamic link library file and multiple object files.

[0054] The dynamic link library file records the binary codes of multiple first operators that can be executed by the processor 101 (such as all first operators that may be executed by the processor 101, etc.). For example, under the Linux system, the dynamic link library file can be, for example, a file in the ".so" format. The so file refers to a shared object file, which is a compiled binary file containing code and data that can be dynamically loaded by the program. Under the Windows system, the dynamic link library file can be, for example, a file in the ".dll" format. The dll file is a tightly integrated independent file that can be called by other programs, and the file may include functions, data, and resources.

[0055] Each object file may be used to record binary codes of one or more second operators that can be executed by the processor 102. For example, the object file may be a file in the ".obj" format.

[0056] In the second implementation example, the file set released by the manufacturer and obtained by the file processing device 104 may include multiple dynamic link library files and multiple object files.

[0057] Each dynamic link library records the binary code of a first operator, and the dynamic link library file can be, for example, a file in the ".so" format or a ".dll" format; each object file is used to record the binary code of a second operator, and the object file can be a file in the ".obj" format.

[0058] In a third implementation example, the file set published by the manufacturer and obtained by the file processing device 104 may include file 1 and file 2. File 1 records the binary codes of multiple first operators that can be executed by the processor 101. For example, file 1 may be obtained by packaging the binary codes of multiple first operators. File 2 records the binary codes of one or more second operators that can be executed by the processor 102. For example, file 2 may be obtained by packaging the binary codes of multiple second operators.

[0059] S302: The file processing device 104 obtains a first identifier of a first operator that the processor 101 needs to execute when executing a computing task, and a second identifier of a second operator that the processor 102 needs to execute when executing the computing task.

[0060] It is understood that the file collection obtained by the file processing device 104 includes not only the binary codes of the operators that the processors 101 and 102 need to execute, but also the binary codes of the operators that the two processors do not need to execute. For example, the file collection includes 100 binary codes of a first operator and 50 binary codes of a second operator, but the processor 101 may only need to execute 5 of the first operators and the processor 102 may only need to execute 3 of the second operators.

[0061] Therefore, the file processing device 104 can first determine the operators that the two processors need to execute when performing the computing task, so as to subsequently filter out the executable code of the operators that each processor does not need to execute. The computing task to be executed can be a task that the computing device 10 executes periodically, such as a model training task. Of course, the computing task can also be other types of non-periodic tasks, and this is not limited to this.

[0062] In one possible implementation, the processor 101 and the processor 102 can collaboratively perform computing tasks based on a file set within a historical time period (such as the past 24 hours, etc.). During this process, the processor 101 and the processor 102 complete the computing task by respectively executing the executable code of the first operator and the executable code of the second operator included in the file set, and generate corresponding historical operation data. Then, the file processing device 104 can obtain the historical operation data and determine which operators the processor 101 executes and which operators the processor 102 executes based on the historical operation data, that is, determine the first identifier of the first operator that the processor 101 needs to execute and the second identifier of the second operator that the processor 102 needs to execute. In this way, the file processing device 104 automatically senses the operators that the processor 101 and the processor 102 need to execute based on the data records of the processor 101 and the processor 102 executing computing tasks in the past.

[0063] In another possible implementation, the file processing device 104 can use corresponding tools to analyze the computing task and determine the identifiers of the operators that the processor 101 and the processor 102 need to execute when performing the computing task. For example, assuming that the computing task is a model training task, the file processing device 104 can compile and analyze the program file corresponding to the computing task through a compiler to extract attribute information such as the name of the operator included in the program file. In this way, the file processing device 104 can determine the name of the first operator that the processor 101 needs to execute (that is, the first identifier) ​​and the name of the second operator that the processor 102 needs to execute (that is, the second identifier) ​​based on the names of each operator (and other attribute information).

[0064] The number of the determined first identifiers may be one or more; the number of the determined second identifiers may be one or more.

[0065] S303: The file processing device 104 filters the executable codes of the plurality of first operators according to the first identifier to obtain a first file, where the first file includes the executable code of the first operator indicated by the first identifier.

[0066] S304: The file processing device 104 filters the executable codes of the plurality of second operators according to the second identifier to obtain a second file, where the second file includes the executable code of the second operator indicated by the second identifier.

[0067] In this embodiment, for operators that do not need to be executed when processor 101 and processor 102 perform computing tasks, the file processing device 104 does not need to save the executable code corresponding to the operator, so that the executable code in the file collection can be filtered according to the identification of the operator that each processor needs to execute.

[0068] As a first implementation example, the file collection may include a dynamic link library file and multiple object files. In this case, the file processing device 104 can disassemble the dynamic link library file to obtain executable files corresponding to multiple first operators, which can be binary files, etc. The executable file of each first operator is used to record the executable code of the first operator. Furthermore, by disassembling the dynamic link library file, a metadata file 1 can also be obtained. The metadata file 1 is used to record metadata. The metadata is used to describe the attribute information of the executable file of each first operator, such as the name and size of the executable file of the first operator. Similarly, the file processing device 104 can disassemble the object file to obtain a metadata file 2 and executable files corresponding to multiple second operators. The metadata file 2 is used to record the attribute information of the executable file of each second operator, and the executable file of each second operator is used to record the executable code of the second operator.

[0069] The file processing device 104 can then use the determined first identifier to filter the executable files corresponding to the multiple first operators obtained by disassembly. For example, the first identifier can be the name of the first operator. The file processing device 104 can then filter out executable files that do not match the name from the multiple executable files. In this way, the executable files obtained by filtering are the executable files of the first operator indicated by the first identifier.

[0070] Furthermore, for the one or more executable files obtained by filtering, the file processing device 104 can package them into a first file, such as using corresponding tools to compress the one or more executable files and obtain a compressed file, or using tools to copy all the binary codes in the one or more executable files to one file (that is, the first file), etc.

[0071] Similarly, the file processing device 104 may use the determined second identifier to filter multiple object files, and package the object files that obtain one or more second operators indicated by the second identifier through filtering to obtain a second file.

[0072] As a second implementation example, a file collection may include multiple dynamic link library files and multiple object files. In this case, the file processing device 104 can use the determined first identifier to filter the multiple dynamic link library files in the file collection and package the object files of the one or more first operators indicated by the first identifier to obtain the first file. Furthermore, the file processing device 104 can use the determined second identifier to filter the multiple object files and package the object files of the one or more second operators indicated by the second identifier to obtain the second file.

[0073] As a third implementation example, the file collection may include file 1 and file 2. In this case, the file processing device 104 may disassemble the file 1 to obtain dynamic link library files corresponding to multiple first operators, each of which is used to record the executable code of a first operator. Furthermore, by disassembling the dynamic link library file, a metadata file 1 may be obtained, which is used to record metadata, and the metadata is used to describe the attribute information of the dynamic link library file of each first operator. Similarly, the file processing device 104 may disassemble the file 2 to obtain multiple object files and a metadata file 2. Each object file is used to record the executable code of a second operator, and the metadata file 2 is used to record the attribute information of each object file.

[0074] Then, the file processing device 104 can use the determined first identifier to filter the multiple dynamic link library files obtained by disassembly, and package the filtered one or more dynamic link library files into a first file. Furthermore, the file processing device 104 can use the determined second identifier to filter the multiple object files, and package the filtered one or more object files of the second operator to obtain a second file.

[0075] Exemplarily, the first file and the second file generated by the file processing device 104 may both be in binary file format, for example, the first file and the second file may both be files with the suffix “.bin”.

[0076] In actual application, after generating the first file and the second file, the file processing device 104 can save them. For example, the file processing device 104 can save the first file and the second file in the memory 103. In this way, the memory 103 can save lightweight operator files without consuming large storage resources to save the file collection. In this way, each time the subsequent processor 101 and the processor 102 execute a computing task, they can execute the computing task based on the lightweight operator file, which can effectively reduce the consumption of storage resources during each execution of the computing task.

[0077] Furthermore, the computing device 10 may construct a file system based on the memory 103 , so that the file processing device 104 may store the first file and the second file in the file system, so as to manage and query the first file and the second file.

[0078] S305 : The file processing device 104 utilizes the processor 101 and the processor 102 to perform a computing task according to the first file and the second file.

[0079] During specific implementation, the file processing device 104 can instruct the processor 101 and the processor 102 to start executing the computing task. As shown in Figure 4, the processor 101 on the host side can load the first file and the second file into the memory of the host side, for example, the first file and the second file in the file system can be read into the memory. Then, the processor 101 can parse the first file in the memory to obtain at least one first executable file, each first executable file is used to record the executable code of a first operator that the processor 101 needs to execute to perform the computing task, and the first executable file can be, for example, a dynamic link library file, such as a file in the ".so" format. In addition, the processor 101 can also parse the second file in the memory to obtain at least one second executable file, each second executable file is used to record the executable code of a second operator that the processor 101 needs to execute to perform the computing task, and the second executable file can be, for example, an object file, such as a file in the ".obj" format.

[0080] Then, since the content recorded in the first executable file is executable code, the processor 101 can deserialize the executable code in each first executable file in memory to generate temporary operator objects, as shown in Figure 4. Each operator object corresponds to a first operator. For example, the processor 101 can use the dlopen function to open each first executable file and generate a temporary operator object based on the information recorded in the first executable file. The operator object has attribute information such as object name, object address, object type, and object size.

[0081] Next, the processor 101 can match each operator object with the second executable file to determine the first executable file (executed by the processor 101) and the second executable file (executed by the processor 102) for the same operator. For example, the name of the operator object (i.e., the object name) can be the name of the operator. In this case, the processor 101 can match the name of the operator object with the name of the second operator corresponding to the second executable file, and determine the second executable file that successfully matches as the second executable file corresponding to the operator object. In this way, the first executable file and the second executable file corresponding to the same operator object (i.e., the same operator) can be obtained.

[0082] Finally, processor 101 can execute the first executable file of each first operator based on the execution logic for the first operator defined in the computing task, and control processor 102 to sequentially execute the second executable file based on the execution order of the first executable files of each first operator. Specifically, the executable code in the second executable file can be serialized into the memory of processor 102. In the process of sequentially executing multiple first operators, processor 101 can determine whether the first operator is executed solely by processor 101 based on a temporary operator object, such as determining whether it corresponds to a second executable file based on the operator object. If so, processor 101 can directly execute the first executable file of the first operator. If not, indicating that the first operator is executed collaboratively by processor 101 and processor 102, processor 101 can launch the executable code in the second executable file corresponding to the first operator into the memory of processor 102 based on the operator object. In this way, by separately executing the executable files in their respective memories, processors 101 and 102 can achieve collaborative execution of the same operator by processors 101 and 102.

[0083] In this way, the file processing device 104 filters the executable code in the file collection according to the identifier of the operator to be executed by each processor, which can effectively reduce the amount of executable code data required to be stored and loaded in the process of executing the computing task, thereby effectively reducing the storage space occupied in the process of executing the computing task, that is, reducing the consumption of storage resources, reducing the requirements of heterogeneous computing for storage resources on the computing device, and thus enriching the application scenarios applicable to heterogeneous computing. In actual application, the computing tasks collaboratively executed by the processor 101 and the processor 102 can be determined by the application deployed on the computing device 10. Therefore, different computing tasks can be generated for applications deployed on different computing devices, so that the file processing device on each computing device can locally save the corresponding lightweight operator file for the computing task generated by the application, so that each computing device can perform the corresponding computing task based on the lightweight operator file, reducing the consumption of storage resources.

[0084] It is worth noting that the file processing method shown in FIG3 is only an exemplary illustration and is not intended to be limiting. In other embodiments, the method steps shown in FIG3 can be replaced accordingly. For example, after obtaining the file set, the file processing device 104 can directly instruct the processor 101 and the processor 102 to perform the computing task according to the file set, so as to determine the identifier of the operator required for the processor 101 and the processor 102 to collaboratively perform the computing task, and filter the executable code of the multiple operators in the file set according to the identifier to obtain a lightweight file. In this way, when the processor 101 and the processor 102 need to perform the computing task again, they can directly perform the computing task based on the lightweight file, thereby reducing the storage resources required to perform the computing task again, and also improving the efficiency of executing the computing task.

[0085] Furthermore, the file processing method shown in FIG3 is exemplified by applying it to the computing device 10 shown in FIG1 . In other possible embodiments, the file processing method can also be applied to a file processing system, which includes a first processor on the host side and multiple (or one) second processors on the device side, that is, the first processor can call multiple (or one) second processors on the device side to collaboratively execute corresponding operators to perform a computing task. Furthermore, the file processing system also includes a file processing device and a memory, wherein the file processing device is used to filter the executable code of the first operator to be executed by the first processor and the executable code of the second operator to be executed by multiple (or one) second processors from the file collection, and package them respectively to obtain a lightweight file 1 (including the executable code of the first operator to be executed by the first processor) required for the first processor to perform the computing task, and a lightweight file 2 (including the executable code of the second operator to be executed by each processor participating in the execution of the computing task) required by the multiple (or one) second processors to perform the computing task, so that the first processor and the multiple (or one) second processors respectively perform the computing task according to the lightweight file 1 and file 2. The memory in the file processing system is used to store the lightweight file 1 and the file 2.

[0086] In the embodiment shown in FIG3 , processors 101 and 102 are used as an example to illustrate the execution of computing tasks. In other possible embodiments, processors 101 and 102 may also generate offline model files based on lightweight files (including executable files of multiple operators). An offline model file refers to a model file stored locally (the model has not yet been run online).

[0087] Next, another file processing method provided by this embodiment will be described in conjunction with FIG5 . FIG5 shows a flow chart of another file processing method for generating an offline model file, wherein some operators included in the offline model file can be collaboratively executed by a processor on the host side and a processor on the device side. Specifically, the method shown in FIG5 may include:

[0088] S501: The file processing device 104 obtains a third file and a fourth file from the file system, wherein the third file includes executable files of multiple third operators required to be executed when the processor 101 executes the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed when the processor 102 executes the target model, and the at least one fourth operator is a subset of the multiple third operators.

[0089] The target model, for example, may be a deep neural network (DNN) model or a convolutional neural network (CNN), and may include multiple operators, which may specifically be multiple neurons (or nodes) included in a network layer in the target model. In actual applications, some of the multiple operators included in the target model may be executed independently by the processor 101 on the host side, while other operators may be executed collaboratively by the processor 101 and the processor 102 on the device side, so that the processor 102 can be used to accelerate the execution efficiency of the other operators.

[0090] Exemplarily, processor 101 and processor 102 may pre-execute the target model based on a file set, which may be, for example, an operator package containing executable code for a full set of operators released by a manufacturer. Thus, file processing device 104 may obtain the identifiers of the operators that processor 101 and processor 102 need to execute during the target model execution, i.e., determine the identifiers of multiple third operators (executed by processor 101) and the identifier of at least one fourth operator (executed by processor 102) in step S501.

[0091] In this way, the file processing device 104 can filter the executable codes of the multiple operators included in the file set based on the identifiers of the multiple third operators and the identifier of at least one fourth operator, thereby obtaining multiple executable files of the third operators, hereinafter referred to as third executable files, and obtaining at least one executable file of the fourth operator, hereinafter referred to as fourth executable file. Furthermore, the file processing device 104 can package the third executable files of the multiple third operators into a third file, and package the fourth executable file of at least one fourth operator into a fourth file, and save the third file and the fourth file to a file system constructed based on the memory 103. Regarding the specific implementation method of filtering the file set to generate the third file and the fourth file, please refer to the relevant description of the aforementioned embodiment, which will not be repeated here.

[0092] Exemplarily, the third file and the fourth file may both be in binary file format, such as a file with a suffix of "bin".

[0093] S502: The file processing device 104 parses the third file to obtain a plurality of third executable files, each of which is used to record the executable code of a third operator.

[0094] S503: The file processing device 104 parses the fourth file to obtain at least one fourth executable file, where each fourth executable file is used to record the executable code of a fourth operator.

[0095] It will be appreciated that, because the third file includes the executable code of at least one operator, the file processing device 104 may instruct the processor 101 to parse the third file to obtain the executable code of multiple third operators, with each executable file of the third operator being stored in a third executable file. The third executable file may be, for example, a dynamic link library file, such as a file in the ".so" format. Similarly, the file processing device 104 may instruct the processor 102 to parse the fourth file to obtain the executable code of at least one fourth operator, with each executable file of the fourth operator being stored in a fourth executable file. The fourth executable file may be, for example, an object file, such as a file in the ".obj" format.

[0096] S504: The file processing device 104 adds the executable codes in the plurality of third executable files and the executable code in at least one fourth executable file to the file of the target model to obtain an offline model file.

[0097] Because the two heterogeneous processors need to execute multiple operators in the target model based on the executable code in the multiple third executable files and the executable code in the at least one fourth executable file during the execution of the target model, the file processing device 104 can add the executable code in the executable files required to be executed by the two heterogeneous processors to the target model file. Specifically, this can be done by serializing the executable code in each executable file into the target model file. In this way, the executable code corresponding to multiple operators can be integrated into the target model file.

[0098] S505: The file processing device 104 outputs the offline model file.

[0099] In this way, after the offline model file is deployed to a computing device including multiple heterogeneous processors, due to the executable code of multiple operators integrated in the offline model file, the computing device does not need to obtain the file set including the executable code of the full set of operators released by the manufacturer. Instead, it can execute the multiple operators in the target model based on multiple heterogeneous processors by executing the executable code integrated in the offline model file, thereby realizing the running of the target model on the computing device and reducing the requirements for the storage resources of the computing device.

[0100] The above describes the file processing method provided in the embodiment of the present application in conjunction with Figures 1 to 5. Next, the functions of the file processing device provided in the embodiment of the present application and the computing device that implements the file processing device will be introduced in conjunction with the accompanying drawings.

[0101] 6 , which shows a schematic diagram of the structure of a file processing device. The file processing device 600 shown in FIG6 can be applied to a computing device, such as the computing device 10 shown in FIG1 . The computing device includes a first processor and a second processor, and the first processor and the second processor are heterogeneous. As shown in FIG6 , the file processing device 600 includes:

[0102] An acquisition module 601 is configured to acquire a file set, the file set including executable codes of multiple first operators and executable codes of multiple second operators; and acquire a first identifier of a first operator to be executed when a first processor performs a computing task, and a second identifier of a second operator to be executed when a second processor performs a computing task.

[0103] Filtering module 602 is configured to filter executable codes of a plurality of first operators according to a first identifier to obtain a first file, the first file including the executable code of the first operator indicated by the first identifier; and filter executable codes of a plurality of second operators according to a second identifier to obtain a second file, the second file including the executable code of the second operator indicated by the second identifier;

[0104] The execution module 603 is configured to utilize the first processor and the second processor to execute a computing task according to the first file and the second file.

[0105] In a possible implementation, the filtering module 602 is specifically configured to:

[0106] Filtering the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier;

[0107] The executable code of the first operator indicated by the first identifier is packaged to obtain a first file.

[0108] In a possible implementation, the execution module 603 is specifically configured to:

[0109] Parsing the first file to obtain at least one first executable file, each first executable file being used to record executable code of a first operator;

[0110] Parsing the second file to obtain at least one second executable file, each second executable file being used to record executable code of a second operator;

[0111] The first processor is used to execute at least one first executable file, and the second processor is controlled to sequentially execute at least one second executable file according to the execution order of the at least one first executable file.

[0112] In a possible implementation, the formats of the first file and the second file are binary file formats.

[0113] In a possible implementation, the first executable file is a dynamic link library file, and each dynamic link library file records a binary code of a first operator;

[0114] The second executable file is an object file, and each object file records the binary code of a second operator.

[0115] In a possible implementation, the file processing device 600 further includes:

[0116] The saving module 604 is configured to save the first file and the second file in a file system of the computing device.

[0117] In one possible implementation, the acquisition module 601 is further configured to acquire a third file and a fourth file from a file system, where the third file includes executable files of multiple third operators required to be executed when the first processor executes the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed when the second processor executes the target model, where the at least one fourth operator is a subset of the multiple third operators.

[0118] Then, the file processing device 600 may further include:

[0119] The parsing module 605 is configured to parse the third file to obtain a plurality of third executable files, each of which is configured to record the executable code of a third operator; parse the fourth file to obtain at least one fourth executable file, each of which is configured to record the executable code of a fourth operator;

[0120] An adding module 606 is configured to add the executable codes in the plurality of third executable files and the executable code in the at least one fourth executable file to a target model file to obtain an offline model file;

[0121] The output module 607 is used to output the offline model file.

[0122] In a possible implementation, the acquisition module 601 is specifically configured to:

[0123] Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute computing tasks according to the file set;

[0124] A first identifier and a second identifier are determined based on historical operation data.

[0125] In a possible implementation, the identifiers of the plurality of second operators match the identifiers of some of the plurality of first operators.

[0126] Since the file processing device 600 shown in Figure 6 corresponds to the file processing device 104 in the method embodiments shown in Figures 3 and 5, the specific implementation method of the file processing device 600 shown in Figure 6 and the technical effects it has can be found in the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0127] Figure 7 shows a structural diagram of a computing device, on which the above-mentioned file processing device 104 can be deployed. The computing device can be a separate device, or can be any computing device in a computing device cluster, and can be specifically used to implement the functions of the file processing device 104 in the embodiments shown in Figures 3 and 5 above.

[0128] As shown in Figure 7, the computing device 700 includes a processor 710, a memory 720, a communication interface 730 and a bus 740. The processor 710, the memory 720 and the communication interface 730 communicate with each other through the bus 740. The bus 740 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in Figure 7, but it does not mean that there is only one bus or one type of bus. The communication interface 730 is used to communicate with the outside, such as obtaining a file collection or outputting an offline model file.

[0129] The processor 710 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits. The processor 710 may also be an integrated circuit chip with signal processing capabilities. During implementation, the functions of the server 200 may be implemented by hardware integrated logic circuits or software instructions in the processor 710. The processor 710 may also be a general-purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor. The methods disclosed in the embodiments of the present application may be directly implemented as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 720 , and the processor 710 reads the information in the memory 720 and completes part or all of the functions of the server 200 in combination with its hardware.

[0130] The memory 720 may include a volatile memory, such as a random access memory (RAM), or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD, or an SSD.

[0131] The memory 720 stores executable codes, and the processor 710 executes the executable codes to perform the method performed by the aforementioned file processing device 104 .

[0132] Specifically, when implementing the embodiments shown in Figures 3 and 5, and when the file processing device 104 described in the embodiments shown in Figures 3 and 5 is implemented by software, the software or program code required to execute the functions of the file processing device 104 in Figures 3 and 5 is stored in the memory 720, and the interaction between the file processing device 104 and other devices is realized through the communication interface 730. The processor is used to execute the instructions in the memory 720 to implement the method executed by the file processing device 104.

[0133] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on one or more computing devices, the one or more computing devices execute the method executed by the file processing device 104 of the above embodiment.

[0134] In addition, embodiments of the present application further provide a computer program product. When the computer program product is executed by one or more computing devices, the one or more computing devices perform any of the aforementioned file processing methods. The computer program product may be a software installation package. When any of the aforementioned file processing methods is required, the computer program product may be downloaded and executed on a computer.

[0135] It should also be noted that the embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the embodiment drawings provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0136] The terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate. This is merely a way of distinguishing objects with the same attributes in the embodiments of this application.

[0137] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0138] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0139] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A file processing method, characterized in that: The method is applied to a computing device, the computing device includes a first processor and a second processor, the first processor and the second processor are heterogeneous, and the method includes: Acquire a file set, where the file set includes executable codes of multiple first operators and executable codes of multiple second operators; Obtaining a first identifier of a first operator that needs to be executed when the first processor performs a computing task, and a second identifier of a second operator that needs to be executed when the second processor performs the computing task; Filter the executable codes of the plurality of first operators according to the first identifier to obtain a first file, where the first file includes the executable code of the first operator indicated by the first identifier; Filter the executable codes of the plurality of second operators according to the second identifier to obtain a second file, where the second file includes the executable code of the second operator indicated by the second identifier; The computing task is executed using the first processor and the second processor according to the first file and the second file.

2. The method according to claim 1, characterized in that The filtering the executable codes of the plurality of first operators according to the first identifier to obtain a first file includes: Filter the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier; The executable code of the first operator indicated by the first identifier is packaged to obtain the first file.

3. The method according to claim 1 or 2, characterized in that: The using the first processor and the second processor to perform a computing task according to the first file and the second file includes: Parsing the first file to obtain at least one first executable file, each first executable file being used to record an executable code of a first operator; Parsing the second file to obtain at least one second executable file, each second executable file being used to record an executable code of a second operator; The at least one first executable file is executed by the first processor, and the second processor is controlled to execute the at least one second executable file in sequence according to the execution order of the at least one first executable file.

4. The method according to claim 3, characterized in that The formats of the first file and the second file are binary file formats.

5. The method according to claim 4, characterized in that The first executable file is a dynamic link library file, and each of the dynamic link library files records a binary code of a first operator; The second executable file is an object file, and each object file records the binary code of a second operator.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The first file and the second file are saved in a file system on the computing device.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a third file and a fourth file from a file system, wherein the third file includes executable files of multiple third operators required to be executed by the first processor when executing the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed by the second processor when executing the target model, and the at least one fourth operator is a subset of the multiple third operators; Parsing the third file to obtain a plurality of third executable files, each of the plurality of third executable files being used to record an executable code of a third operator; Parsing the fourth file to obtain at least one fourth executable file, each fourth executable file in the at least one fourth executable file is used to record an executable code of a fourth operator; Adding the executable codes in the plurality of third executable files and the executable code in the at least one fourth executable file to the file of the target model to obtain an offline model file; The offline model file is output.

8. The method according to any one of claims 1 to 7, characterized in that: The obtaining a first identifier of a first operator to be executed by the first processor and a second identifier of a second operator to be executed by the second processor includes: Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute the computing task according to the file set; The first identifier and the second identifier are determined according to the historical operation data.

9. The method according to any one of claims 1 to 8, characterized in that: The identifiers of the plurality of second operators match identifiers of some of the plurality of first operators.

10. A file processing device, characterized in that: The file processing apparatus is applied to a computing device, the computing device includes a first processor and a second processor, the first processor and the second processor are heterogeneous, and the file processing apparatus includes: an acquisition module, configured to acquire a file set, the file set comprising executable codes of a plurality of first operators and executable codes of a plurality of second operators; and to acquire a first identifier of a first operator to be executed when the first processor performs a computing task and a second identifier of a second operator to be executed when the second processor performs the computing task; a filtering module, configured to filter the executable codes of the plurality of first operators according to the first identifier to obtain a first file, wherein the first file includes the executable code of the first operator indicated by the first identifier; and filter the executable codes of the plurality of second operators according to the second identifier to obtain a second file, wherein the second file includes the executable code of the second operator indicated by the second identifier; An execution module is used to utilize the first processor and the second processor to execute the computing task according to the first file and the second file.

11. The device according to claim 10, characterized in that The filtering module is used for: Filter the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier; The executable code of the first operator indicated by the first identifier is packaged to obtain the first file.

12. The device according to claim 10 or 11, characterized in that The execution module is used to: Parsing the first file to obtain at least one first executable file, each first executable file being used to record an executable code of a first operator; Parsing the second file to obtain at least one second executable file, each second executable file being used to record an executable code of a second operator; The at least one first executable file is executed by the first processor, and the second processor is controlled to execute the at least one second executable file in sequence according to the execution order of the at least one first executable file.

13. The device according to claim 12, characterized in that The formats of the first file and the second file are binary file formats.

14. The device according to claim 13, characterized in that The first executable file is a dynamic link library file, and each of the dynamic link library files records a binary code of a first operator; The second executable file is an object file, and each object file records the binary code of a second operator.

15. The device according to any one of claims 10 to 14, characterized in that The file processing device also includes: A saving module is used to save the first file and the second file in a file system of the computing device.

16. The device according to any one of claims 10 to 15, characterized in that The acquisition module is further used to acquire a third file and a fourth file from the file system, wherein the third file includes executable files of multiple third operators required to be executed when the first processor executes the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed when the second processor executes the target model, and the at least one fourth operator is a subset of the multiple third operators; The file processing device also includes: a parsing module, configured to parse the third file to obtain a plurality of third executable files, each of the plurality of third executable files being used to record an executable code of a third operator; and parse the fourth file to obtain at least one fourth executable file, each of the at least one fourth executable file being used to record an executable code of a fourth operator; An adding module is used to convert the executable codes in the plurality of third executable files and the executable codes in the at least one fourth executable file into Execute the code and add it to the file of the target model to obtain an offline model file; An output module is used to output the offline model file.

17. The device according to any one of claims 10 to 16, characterized in that The acquisition module is used to: Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute the computing task according to the file set; The first identifier and the second identifier are determined according to the historical operation data.

18. The device according to any one of claims 10 to 17, characterized in that The identifiers of the plurality of second operators match identifiers of some of the plurality of first operators.

19. A file processing system, characterized in that: The file processing system comprises a first processor, at least one second processor, a memory and a file processing device, wherein the first processor and the at least one second processor are heterogeneous; The file processing device is used to execute the method according to any one of claims 1 to 9.

20. A computing device, characterized in that The computing device comprises a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the method according to any one of claims 1 to 9.

Images