File processing method, device and system and related equipment

Abstract

The invention provides a file processing method, device and system and related equipment. The method comprises the following steps: acquiring a file set which comprises executable codes of a plurality of first operators and executable codes of a plurality of second operators, and acquiring a first identifier of the first operator required by a first processor and a second identifier of the second operator required by a second processor (heterogeneous with the first processor), the method comprises the following steps: acquiring a first identifier of a plurality of first operators, filtering executable codes of the plurality of first operators according to the first identifier to obtain a first file, filtering executable codes of the plurality of second operators according to the second identifier to obtain a second file, and executing a calculation task by utilizing a first processor and a second processor according to the first file and the second file. Therefore, the data volume of the executable codes needing to be stored and loaded in the process of executing the calculation task can be effectively reduced, occupation of storage space in the process of executing the calculation task can be effectively reduced, and consumption of storage resources is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a file processing method, device, system and related equipment. Background Art

[0002] Heterogeneous computing refers to computing devices using different types of processors to collaborate in completing computing tasks in order to improve the processing efficiency of the computing device. For example, a computing device may include a central processing unit (CPU) and a graphics processing unit (GPU); when performing computing tasks, the CPU may be responsible for preprocessing data such as data cleaning and deduplication, and the GPU is responsible for vector calculations on the preprocessed data (the GPU is more efficient in vector calculations). Finally, the CPU calculates the vector calculation results, such as converting the vector calculation results into scalars. In this way, the computing device can use the GPU to accelerate the completion of the computing task and improve processing efficiency.

[0003] The operation performed by the computing device may be referred to as an operator. Moreover, different processors in the computing device may implement the data processing operation corresponding to the operator by executing different parts of the executable code of the operator. The executable code may be, for example, a binary code.

[0004] In actual application, the heterogeneous processors on the computing device can be divided into processors on the host side and processors on the device side. Manufacturers usually integrate all operators that can be executed by the processor on the host side and all operators that can be executed by the processor on the device side, and centrally publish the executable code of all operators. Accordingly, the computing device can save the executable code of all operators locally, so that the processor on the host side and the processor on the device side can perform corresponding data processing operations according to the locally saved executable code to complete the computing tasks generated by the application on the computing device.

[0005] However, the amount of data in the executable code of the full operator is too large, which requires the processors on the host side and the device side to occupy more storage resources when performing computing tasks based on the executable code, thus limiting the application scenarios applicable to heterogeneous computing. For example, for computing devices in edge environments, their storage space is limited, so excessive storage resource consumption will affect the deployment of heterogeneous computing on computing devices. Summary of the invention

[0006] In view of this, an embodiment of the present application provides a file processing method for reducing the storage resources required by the processor on the host side and the processor on the device side of the computing device when performing computing tasks, thereby enriching the application scenarios applicable to heterogeneous computing. The present application also provides corresponding devices, systems, computing devices, computer-readable storage media, and computer program products.

[0007] In a first aspect, an embodiment of the present application provides a file processing method, which is applied to a computing device, wherein the computing device includes a first processor and a second processor, and the first processor and the second processor are heterogeneous, such as the first processor can be a CPU, and the second processor can be a GPU or a DPU, etc.; in the process of executing file processing, a file set is first obtained, and the file set includes executable codes of multiple first operators and executable codes of multiple second operators. Usually, the operator that can be executed by the first processor is the first operator, and the operator that can be executed by the second processor is the second operator, and the first processor and the second processor can collaboratively execute the same operator, or the first processor can execute one alone. operator; then, obtaining a first identifier of a first operator to be executed when the first processor executes the computing task, and a second identifier of a second operator to be executed when the second processor executes the computing task, and filtering the executable codes of multiple first operators in the file set according to the first identifier to obtain a first file, the first file obtained includes the executable code of the first operator indicated by the first identifier, and filtering the executable codes of multiple second operators in the file set according to the second identifier to obtain a second file, the second file obtained includes the executable code of the second operator indicated by the second identifier; finally, using the first processor and the second processor, according to the filtered first file and the second file, execute the computing task.

[0008] In this way, before executing a computing task, the executable code including multiple operators is filtered according to the identifiers of the operators that each processor needs to execute, which can effectively reduce the amount of data of the executable code that needs to be stored and loaded during the execution of the computing task, that is, the executable code of the operators that each processor does not need to execute does not need to be stored locally on the computing device, which can effectively reduce the storage space occupied during the execution of the computing task, that is, reduce the consumption of storage resources, and reduce the requirements of heterogeneous computing for storage resources on the computing device, thereby enriching the application scenarios applicable to heterogeneous computing. In addition, when the number of first operators that the first processor needs to execute is multiple, or the number of second operators that the second processor needs to execute is multiple, the executable codes of the multiple first operators or the multiple second operators are packaged into one file, which enables the first processor to reduce the file loading delay when loading the packaged file (i.e., the first file or the second file) into the memory, thereby improving the execution efficiency of the computing task.

[0009] In a possible implementation, in the process of filtering the executable codes of multiple first operators according to the first identifier and obtaining the first file, the executable codes of multiple first operators corresponding to the first identifier may be first filtered to obtain the executable code of the first operator indicated by the first identifier, and then the executable code of the first operator indicated by the first identifier is packaged to obtain the first file. In this way, the packaged first file may include the executable code of one or more first operators indicated by the first identifier, so that when the first processor loads the packaged first file into the memory, the file loading delay can be reduced, thereby improving the execution efficiency of the computing task.

[0010] Similarly, in the process of generating the second file, the executable codes of multiple second operators corresponding to the second identifier can be filtered first to obtain the executable code of the second operator indicated by the second identifier, and then the executable code of the second operator indicated by the second identifier can be packaged to obtain the second file.

[0011] In a possible implementation, when executing a computing task according to a first file and a second file, the first file may be parsed first to obtain at least one first executable file, wherein each first executable file is used to record the executable code of a first operator, and the second file may be parsed to obtain at least one second executable file, wherein each second executable file is used to record the executable code of a second operator, so that the at least one first executable file is executed by the first processor, and the second processor is controlled to execute the at least one second executable file in sequence according to the execution order of the at least one first executable file. In this way, the first processor and the second processor can realize the collaborative execution of computing tasks according to the executable codes included in the files required by each, and the first file and the second file are packaged files, which can reduce the file loading delay when they are loaded into the memory, thereby improving the execution efficiency of the computing task.

[0012] In a possible implementation, the formats of the first file and the second file are binary file formats, such as ".bin" format.

[0013] In a possible implementation, the first executable file is a dynamic link library file, and each dynamic link library file records a binary code of a first operator, while the second executable file is an object file, and each object file records a binary code of a second operator.

[0014] In a possible implementation, during the file processing, the lightweight first file and the second file may also be stored in a file system on the computing device, and the file system may be constructed by, for example, a memory on the computing device. In this way, the computing device may only store the lightweight first file and the second file, without including a heavyweight file collection, thereby reducing the storage resources consumed by executing the computing task.

[0015] In a possible implementation, during the file processing process, a third file and a fourth file may also be obtained from the file system. The third file and the fourth file may be obtained by filtering the file set according to the above process, for example. The third file includes executable files of multiple third operators required to be executed when the first processor executes the target model. The fourth file includes executable files of at least one fourth operator required to be executed when the second processor executes the target model. The at least one fourth operator is a subset of the multiple third operators, that is, the fourth operator is an operator executed in collaboration between the second processor and the first processor. Then, the third file is parsed to obtain multiple third executable files, each of which is used to record an executable code (such as binary code, etc.) of a third operator. The fourth file is parsed to obtain multiple fourth executable files, each of which is used to record an executable code of a fourth operator. Finally, the executable code in the multiple third executable files and the executable code in at least one fourth executable file are added to the file of the target model to obtain an offline model file, and the offline model file is output. In this way, after the offline model file is deployed to a computing device including multiple heterogeneous processors, due to the executable code of multiple operators integrated in the offline model file, the computing device does not need to obtain the file set including the executable code of the full set of operators released by the manufacturer. Instead, it can execute the multiple operators in the target model based on multiple heterogeneous processors by executing the executable code integrated in the offline model file, thereby realizing the running of the target model on the computing device and reducing the requirements for the storage resources of the computing device.

[0016] In a possible implementation, when obtaining the identifier of the first operator to be executed by the first processor and the second identifier of the second operator to be executed by the second processor, specifically, historical operation data may be first obtained, the historical operation data including data generated when the first processor and the second processor execute the computing task according to the file set, and the first identifier and the second identifier are determined according to the historical operation data. In this way, the operator to be executed by the first processor and the second processor is automatically sensed based on the data records of the computing tasks executed by the first processor and the second processor in the past.

[0017] In a possible implementation, the identifiers of the plurality of second operators match the identifiers of some of the plurality of first operators, wherein the operators with matching identifiers are the operators executed collaboratively by the first processor and the second processor.

[0018] In a second aspect, the present application provides a file processing device, which includes various modules for executing the file processing method in the first aspect or any possible implementation of the first aspect.

[0019] In a third aspect, the present application provides a file processing system, which includes a first processor, at least one second processor, a memory, and a file processing device. The first processor and the at least one second processor are heterogeneous, such as the first processor is a CPU, and the file system includes multiple processors such as a GPU and a DPU that are heterogeneous with the first processor; the file processing device is used to execute the method described in the first aspect or any one of the implementation methods of the first aspect; the functions of the first processor, at least one second processor, and the memory can be referred to the relevant description of the first aspect above.

[0020] In a fourth aspect, the present application provides a computing device, the computing device including a processor and a memory, the memory storing program instructions, the processor running the program instructions to execute the file processing method in the first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The computing device can also include a bus. The processor is connected to the memory via the bus. The memory can include a readable memory and a random access memory.

[0021] In a fifth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on at least one computing device, the at least one computing device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0022] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0023] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the structure of an exemplary computing device provided in an embodiment of the present application;

[0025] Figure 2is a schematic diagram of an operator package;

[0026] Figure 3 A flowchart of a file processing method provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of executing a computing task according to a first file and a second file;

[0028] Figure 5 A flowchart of another file processing method provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the structure of a file processing device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The scheme in the embodiments provided in this application will be described below in conjunction with the drawings in this application.

[0032] See also Figure 1 , which shows a schematic diagram of the structure of a computing device. Figure 1 As shown, the computing device 10 includes multiple processors, which are respectively located on the host side and the device side. For ease of understanding and description, Figure 1 In the example, the host side includes a processor 101 and the device side includes a processor 102. In actual application, the number of processors included in the host side and the device side can be one or more. The processor 101 and the processor 102 can be connected through a bus, such as a Compute Express Link (CXL) bus. In addition, the computing device 10 also includes a memory 103 and a file processing device 104, such as Figure 1 Alternatively, the computing device 10 may further include other components.

[0033] The processor 101 on the host side is used to execute an operator independently, specifically to execute the executable code (such as binary code, etc.) corresponding to the operator, or to cooperate with the processor 102 on the device side to execute the same operator. Exemplarily, the processor 101 can be a general-purpose processor, such as a central processing unit (CPU), etc.

[0034] The processor 102 on the device side is used to execute the same operator in collaboration with the processor 101 on the host side. Specifically, the processor 101 may execute part of the executable code of the operator, and the processor 102 may execute another part of the executable code of the operator. Exemplarily, the processor 102 may be any type of processor such as a data processing unit (DPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), or may be other types of processors, which are not limited to this. Moreover, the processor 102 and the processor 101 are heterogeneous processors. For example, when the processor 101 is a CPU, the processor 102 may be a DPU or a GPU, etc., which may execute the same operator in collaboration with the processor 101 to realize accelerated processing of computing tasks.

[0035] The memory 103 is used to store the executable code executed by the processor 101 and the processor 102, such as recording the executable code in the form of a file. In actual application, a file system can be constructed based on the memory 103 in the computing device 10, and the file system can manage the executable files in the memory 103 accordingly. Exemplarily, the memory 103 can be a solid state disk (SSD), a magnetic disk, etc.

[0036] The file processing device 104 is used to perform corresponding processing on the executable codes of the operators executed by the processor 101 and the processor 102 to reduce the occupation of the storage resources of the computing device 10 during the execution of the computing task.

[0037] Specifically, manufacturers usually produce the following in order to adapt to the task execution process in various application scenarios: Figure 2The operator package shown includes the executable code of all operators that may be executed by the processors on the host side and the device side, and the executable code may be, for example, a binary code. Among them, the binary code of all operators that may be executed by the processor 101 on the host side will be placed in an executable file of a larger size, and the binary code of all operators that may be executed by the processor 102 on the device side will be stored in a dispersed manner in the form of multiple executable files of smaller sizes (such as dispersed storage in the form of hundreds of small files, etc.). Since the file size of the operator package is large, not only does the memory 103 need to consume a large storage space to store the operator package, but also the processor 101 needs to occupy a large memory space when loading the operator package (loaded into the memory in the processor 101). In addition, since the operator package includes multiple small files (executable codes of all operators that may be executed by the processor 102), it takes a long time for these small files to be loaded into the memory, thereby affecting the overall execution efficiency of the computing task.

[0038] Based on this, in the computing device 10 provided in this embodiment, the file processing device 104 first obtains a file set including a full set of operators (such as the above-mentioned operator package) released by the manufacturer, and obtains the first identifier of the first operator required to be executed when the processor 101 performs the computing task, and the second identifier of the second operator required to be executed when the processor 102 performs the computing task. Then, the file processing device 104 filters the executable codes of multiple first operators in the file set according to the first identifier to obtain file 1, which includes the executable code of the first operator indicated by the first identifier (that is, the executable code obtained by filtering according to the first identifier), and filters the executable codes of multiple second operators in the file set according to the second identifier to obtain file 2, which includes the executable code of the second operator indicated by the second identifier (that is, the executable code obtained by filtering according to the second identifier). In this way, the file processing device 104 can use the processor 101 and the processor 102 to perform the computing task according to the filtered files 1 and 2.

[0039] In this way, before executing the computing task, the file processing device 104 filters the executable code in the file collection according to the identification of the operators that each processor needs to execute, which can effectively reduce the amount of executable code data that needs to be stored and loaded in the process of executing the computing task, that is, the executable code of the operators that each processor does not need to execute does not need to be stored locally on the computing device, thereby effectively reducing the storage space occupied in the process of executing the computing task, that is, reducing the consumption of storage resources, reducing the requirements of heterogeneous computing for storage resources on the computing device, and thus enriching the application scenarios applicable to heterogeneous computing.

[0040] Moreover, when the number of first operators that the processor 101 needs to execute is multiple, or the number of second operators that the processor 102 needs to execute is multiple, the executable codes of the multiple first operators or the multiple second operators can be packaged into one file, which enables the processor 101 to reduce the file loading delay when loading the packaged file (i.e., file 1 or file 2) into the memory on the host side, thereby improving the execution efficiency of the computing task.

[0041] For example, the above Figure 1 The file processing device 104 shown may be implemented by software or by hardware.

[0042] In one example, when implemented by software, the file processing device 104 may be a code running on a computing device 10 including a processor 101 (and a processor 102 ), such as a computing engine, a plug-in, etc. on the computing device 10 .

[0043] In another example, when implemented by hardware, the file processing device 104 may be hardware with data processing capabilities, etc. Exemplarily, the hardware with data processing capabilities may be a CPU, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system on chip (SoC), a software-defined infrastructure (SDI) chip, an artificial intelligence (AI) chip, a data processing unit (DPU), or any combination thereof. In addition, the number of hardware with data processing capabilities included in the file processing device 104 may be one or more, and the type of hardware included may be one or more. Specifically, the number and type of the hardware may be set according to the business requirements of the actual application, and this embodiment does not limit this.

[0044] It is worth noting that the above Figure 1The computing device 10 shown is only used as an exemplary illustration. In other possible implementations, the computing device 10 may also include more types of components to support the computing device 10 to have more functions; or, the computing device 10 may include a larger number of processors, etc. The specific structure of the computing device 10 is not limited in this application.

[0045] Next, various non-limiting specific implementations of the file processing process are described in detail.

[0046] See also Figure 3 , is a flowchart of a file processing method in an embodiment of the present application. The method can be applied to the above Figure 1 The computing device 10 shown in FIG. 1 may also be applied to other possible computing devices. Figure 1 The computing device 10 shown is used as an example for illustration. Figure 3 The file processing method shown may specifically include:

[0047] S301: The file processing device 104 obtains a file set, where the file set includes executable codes of multiple first operators and executable codes of multiple second operators.

[0048] Among them, the first operator is the operator executed by the processor 101 on the host side. Normally, the processor 101 can execute a first operator alone. Alternatively, the processor 101 can execute the same operator in collaboration with the processor 102. The operator is the first operator for the processor 101, and is referred to as the second operator for the processor 102. Normally, the processor 102 does not execute an operator alone; accordingly, the identifier of the second operator that the processor 102 can execute matches the identifier of some of the first operators among the multiple first operators that the processor 101 can execute, and the operator with matching identifiers is also the operator executed by the processor 101 and the processor 102 in collaboration.

[0049] Exemplarily, when processor 101 and processor 102 cooperate to execute the same operator, processor 101 may generally first execute the first calculation stage of the operator, and control processor 102 to execute the second calculation stage of the operator, and finally, processor 101 may continue to execute the third calculation stage according to the result generated by processor 102 executing the second calculation stage, thereby completing the execution process of the operator. Each processor executes a corresponding calculation stage based on the executable code of different parts of the operator.

[0050] In this embodiment, the file processing device 104 may first obtain executable codes of multiple operators that may be executed by the processor 101 and the processor 102. The executable codes of the operators may be, for example, binary codes, assembly language codes, etc., or other types of executable codes.

[0051] In a first implementation example, the file processing device 104 may first obtain a file set released by a manufacturer. The file set may be, for example, Figure 2 The operator package shown includes a dynamic link library file and multiple object files.

[0052] The dynamic link library file records binary codes of multiple first operators that can be executed by the processor 101 (such as all first operators that may be executed by the processor 101, etc.). Exemplarily, under the Linux system, the dynamic link library file can be, for example, a file in the ".so" format. The so file refers to a shared object file, which is a compiled binary file containing code and data that can be dynamically loaded by a program. Under the Windows system, the dynamic link library file can be, for example, a file in the ".dll" format. The dll file is a tightly integrated independent file that can be called by other programs, and the file may include functions, data, and resources.

[0053] Each object file may be used to record binary codes of one or more second operators that can be executed by the processor 102. For example, the object file may be a file in the ".obj" format.

[0054] In the second implementation example, the file set released by the manufacturer and obtained by the file processing device 104 may include multiple dynamic link library files and multiple object files.

[0055] Each dynamic link library records the binary code of a first operator, and the dynamic link library file may be a file in the ".so" format or a file in the ".dll" format; each object file is used to record the binary code of a second operator, and the object file may be a file in the ".obj" format.

[0056] In the third implementation example, the file set released by the manufacturer obtained by the file processing device 104 may include file 1 and file 2. File 1 records the binary codes of multiple first operators that can be executed by the processor 101, for example, file 1 can be obtained by packaging the binary codes of multiple first operators, etc.; file 2 records the binary codes of one or more second operators that can be executed by the processor 102, for example, file 2 can be obtained by packaging the binary codes of multiple second operators, etc.

[0057] S302: The file processing device 104 obtains a first identifier of a first operator that needs to be executed when the processor 101 executes a computing task, and a second identifier of a second operator that needs to be executed when the processor 102 executes the computing task.

[0058] It can be understood that the file set acquired by the file processing device 104 includes not only the binary codes of the operators that the processors 101 and 102 need to execute, but also the binary codes of the operators that the two processors do not need to execute. For example, the file set includes 100 binary codes of the first operator and 50 binary codes of the second operator, but the processor 101 may only need to execute 5 first operators and the processor 102 may only need to execute 3 second operators.

[0059] Therefore, the file processing device 104 can first determine the operators that need to be executed when the two processors perform the computing tasks, so as to subsequently filter out the executable codes of the operators that each processor does not need to execute. The computing tasks that need to be executed can be tasks that are periodically executed by the computing device 10, such as model training tasks, etc. Of course, the computing tasks can also be other types of non-periodic tasks, which are not limited to this.

[0060] In a possible implementation, the processor 101 and the processor 102 can collaboratively perform a computing task according to a file set within a historical time period (such as the past 24 hours, etc.). In this process, the processor 101 and the processor 102 complete the computing task by respectively executing the executable code of the first operator and the executable code of the second operator included in the file set, and generate corresponding historical operation data. Then, the file processing device 104 can obtain the historical operation data, and determine which operators the processor 101 executes and which operators the processor 102 executes based on the historical operation data, that is, determine the first identifier of the first operator that the processor 101 needs to execute and the second identifier of the second operator that the processor 102 needs to execute. In this way, the file processing device 104 automatically senses the operators that the processor 101 and the processor 102 need to execute based on the data records of the processor 101 and the processor 102 executing computing tasks in the past.

[0061] In another possible implementation, the file processing device 104 can use corresponding tools to analyze the computing task and determine the identifiers of the operators that the processor 101 and the processor 102 need to execute when executing the computing task. For example, assuming that the computing task is a model training task, the file processing device 104 can compile and analyze the program file corresponding to the computing task through a compiler to extract the attribute information such as the name of the operator included in the program file. In this way, the file processing device 104 can determine the name of the first operator that the processor 101 needs to execute (that is, the first identifier) ​​and the name of the second operator that the processor 102 needs to execute (that is, the second identifier) ​​based on the names of each operator (and other attribute information).

[0062] The number of the determined first identifiers may be one or more; the number of the determined second identifiers may be one or more.

[0063] S303: The file processing device 104 filters the executable codes of the plurality of first operators according to the first identifier to obtain a first file, where the first file includes the executable code of the first operator indicated by the first identifier.

[0064] S304: The file processing device 104 filters the executable codes of the plurality of second operators according to the second identifier to obtain a second file, where the second file includes the executable code of the second operator indicated by the second identifier.

[0065] In this embodiment, for operators that do not need to be executed when processor 101 and processor 102 perform computing tasks, the file processing device 104 does not need to save the executable code corresponding to the operator, so that the executable code in the file collection can be filtered according to the identification of the operator that each processor needs to execute.

[0066] As a first implementation example, the file set may include a dynamic link library file and multiple object files. At this time, the file processing device 104 may disassemble the dynamic link library file to obtain executable files corresponding to multiple first operators, which may be binary files, etc. Among them, the executable file of each first operator is used to record the executable code of the first operator. Further, by disassembling the dynamic link library file, a metadata file 1 may be obtained, and the metadata file 1 is used to record metadata, which is used to describe the attribute information of the executable file of each first operator, such as the name and size of the executable file of the first operator. Similarly, the file processing device 104 may disassemble the object file to obtain a metadata file 2 and executable files corresponding to multiple second operators. Among them, the metadata file 2 is used to record the attribute information of the executable file of each second operator, and the executable file of each second operator is used to record the executable code of the second operator.

[0067] Then, the file processing device 104 can use the determined first identifier to filter the executable files corresponding to the multiple first operators obtained by disassembly. For example, the first identifier can be the name of the first operator, and the file processing device 104 can filter out the executable files that do not match the name from the multiple executable files. In this way, the executable file obtained by filtering is the executable file of the first operator indicated by the first identifier.

[0068] Furthermore, for the one or more executable files obtained by filtering, the file processing device 104 can package them into a first file, such as using corresponding tools to compress the one or more executable files and obtain a compressed file, or using tools to copy all binary codes in the one or more executable files to one file (that is, the first file), etc.

[0069] Similarly, the file processing device 104 may filter multiple object files using the determined second identifier, and pack the object files of the one or more second operators indicated by the second identifier obtained through filtering to obtain a second file.

[0070] As a second implementation example, the file set may include multiple dynamic link library files and multiple object files. In this case, the file processing device 104 may filter multiple dynamic link library files in the file set using the determined first identifier, and package the object files of one or more first operators indicated by the first identifier obtained by filtering to obtain the first file. In addition, the file processing device 104 may filter multiple object files using the determined second identifier, and package the object files of one or more second operators indicated by the second identifier obtained by filtering to obtain the second file.

[0071] As a third implementation example, the file set may include file 1 and file 2. At this time, the file processing device 104 may disassemble the file 1 to obtain dynamic link library files corresponding to multiple first operators, each of which is used to record the executable code of a first operator. Further, by disassembling the dynamic link library file, a metadata file 1 may be obtained, and the metadata file 1 is used to record metadata, which is used to describe the attribute information of the dynamic link library file of each first operator. Similarly, the file processing device 104 may disassemble the file 2 to obtain multiple object files and a metadata file 2. Each object file is used to record the executable code of a second operator, and the metadata file 2 is used to record the attribute information of each object file.

[0072] Then, the file processing device 104 can use the determined first identifier to filter the multiple dynamic link library files obtained by disassembly, and package the filtered one or more link library files into a first file. Also, the file processing device 104 can use the determined second identifier to filter the multiple object files, and package the filtered one or more object files of the second operator to obtain a second file.

[0073] Exemplarily, the first file and the second file generated by the file processing device 104 may both be in the format of binary files, for example, the first file and the second file may both be files with the suffix “.bin”.

[0074] In actual application, after the file processing device 104 generates the first file and the second file, it can save them. For example, the file processing device 104 can save the first file and the second file in the memory 103. In this way, the memory 103 can save lightweight operator files without consuming large storage resources to save the file collection. In this way, the subsequent processor 101 and the processor 102 can perform the computing task based on the lightweight operator file each time they perform the computing task, which can effectively reduce the consumption of storage resources in the process of executing the computing task each time.

[0075] Furthermore, the computing device 10 may construct a file system based on the memory 103 , so that the file processing device 104 may store the first file and the second file in the file system, so as to manage and query the first file and the second file.

[0076] S305: The file processing device 104 utilizes the processor 101 and the processor 102 to perform a computing task according to the first file and the second file.

[0077] In a specific implementation, the file processing device 104 may instruct the processor 101 and the processor 102 to start executing the computing task. Figure 4 As shown, the processor 101 on the host side can load the first file and the second file into the memory on the host side, for example, the first file and the second file in the file system can be read into the memory. Then, the processor 101 can parse the first file in the memory to obtain at least one first executable file, each of which is used to record the executable code of a first operator that the processor 101 needs to execute to perform the computing task. The first executable file can be, for example, a dynamic link library file, such as a file in the ".so" format. In addition, the processor 101 can also parse the second file in the memory to obtain at least one second executable file, each of which is used to record the executable code of a second operator that the processor 101 needs to execute to perform the computing task. The second executable file can be, for example, an object file, such as a file in the ".obj" format.

[0078] Then, since the content recorded in the first executable file is an executable code, the processor 101 can deserialize the executable code in each first executable file in the memory to generate a temporary operator object, such as Figure 4 As shown, each operator object corresponds to a first operator. For example, the processor 101 can use the dlopen function to open each first executable file, and generate a temporary operator object according to the information recorded in the first executable file. The operator object has attribute information such as object name, object address, object type, and object size.

[0079] Then, the processor 101 can match each operator object with the second executable file to determine the first executable file (executed by the processor 101) and the second executable file (executed by the processor 102) of the same operator. For example, the name of the operator object (i.e., the object name) can be the name of the operator, then the processor 101 can match the name of the operator object with the name of the second operator corresponding to the second executable file, and determine the second executable file that successfully matches as the second executable file corresponding to the operator object, so that the first executable file and the second executable file corresponding to the same operator object (i.e., the same operator) can be obtained.

[0080] Finally, the processor 101 can execute the first executable file of each first operator according to the execution logic of the first operator defined in the computing task, and control the processor 102 to execute the second executable file in sequence according to the execution order of the first executable files of each first operator, which can be specifically serialized into the memory of the processor 102. In the process of sequentially executing multiple first operators, the processor 101 can determine whether the first operator is executed by the processor 101 alone according to the temporary operator object, such as determining whether it corresponds to the second executable file according to the operator object. If yes, the processor 101 can directly execute the first executable file of the first operator. If not, it indicates that the first operator is executed by the processor 101 and the processor 102 in collaboration, and the processor 101 can start the executable code in the second executable file corresponding to the first operator into the memory of the processor 102 according to the operator object. In this way, the processor 101 and the processor 102 can realize the collaborative execution of the same operator by the processor 101 and the processor 102 by executing the executable files in their respective memories respectively.

[0081] In this way, the file processing device 104 filters the executable code in the file set according to the identifier of the operator to be executed by each processor, which can effectively reduce the amount of data of the executable code required to be stored and loaded in the process of executing the computing task, thereby effectively reducing the occupation of storage space in the process of executing the computing task, that is, reducing the consumption of storage resources, reducing the requirements of heterogeneous computing for storage resources on the computing device, and thus enriching the application scenarios applicable to heterogeneous computing. In actual application, the computing tasks collaboratively executed by the processor 101 and the processor 102 can be determined by the application deployed on the computing device 10. Therefore, different computing tasks can be generated for applications deployed on different computing devices, so that the file processing device on each computing device can locally save the corresponding lightweight operator file for the computing task generated by the application, so that each computing device can perform the corresponding computing task based on the lightweight operator file, reducing the consumption of storage resources.

[0082] It is worth noting that the above Figure 3 The file processing method shown is only an exemplary description and is not intended to be limiting. Figure 3The method steps shown are replaced accordingly. For example, after acquiring the file set, the file processing device 104 can directly instruct the processor 101 and the processor 102 to perform the computing task according to the file set, so as to determine the identifier of the operator that the processor 101 and the processor 102 need to execute in collaboration to perform the computing task, and filter the executable codes of the multiple operators in the file set according to the identifier to obtain a lightweight file. In this way, when the processor 101 and the processor 102 need to execute the computing task again, the computing task can be executed directly based on the lightweight file, thereby reducing the storage resources consumed by executing the computing task again, and also improving the efficiency of executing the computing task.

[0083] Furthermore, the above Figure 3 The file processing method shown is based on the application of Figure 1 The computing device 10 shown is used as an example for illustrative description. In other possible embodiments, the file processing method can also be applied to a file processing system, which includes a first processor on the host side and multiple (or one) second processors on the device side, that is, the first processor can call multiple (or one) second processors on the device side to collaboratively execute corresponding operators to perform a computing task. In addition, the file processing system also includes a file processing device and a memory, wherein the file processing device is used to filter and obtain the executable code of the first operator to be executed by the first processor and the executable code of the second operator to be executed by multiple (or one) second processors from the file collection, and pack them respectively to obtain the lightweight file 1 (including the executable code of the first operator to be executed by the first processor) required by the first processor when performing the computing task, and the lightweight file 2 (including the executable code of the second operator to be executed by each processor participating in the execution of the computing task) required by multiple (or one) second processors when performing the computing task, so that the first processor and multiple (or one) second processors respectively perform computing tasks according to the lightweight file 1 and file 2. The memory in the file processing system is used to store the lightweight file 1 and file 2.

[0084] Above Figure 3 In the illustrated embodiment, the processor 101 and the processor 102 are used as an example to perform a computing task. In other possible embodiments, the processor 101 and the processor 102 may also generate an offline model file based on a lightweight file (including executable files of multiple operators). The offline model file refers to a model file saved locally (the model has not yet been run online).

[0085] Next, combine Figure 5 , another file processing method provided by this embodiment is introduced. Figure 5, shows a flow chart of another file processing method, the method is used to generate an offline model file, and some operators included in the offline model file can be executed collaboratively by a processor on the host side and a processor on the device side. Specifically, Figure 5 The methods shown may include:

[0086] S501: The file processing device 104 obtains a third file and a fourth file from the file system, wherein the third file includes executable files of multiple third operators required to be executed when the processor 101 executes the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed when the processor 102 executes the target model, and the at least one fourth operator is a subset of the multiple third operators.

[0087] The target model, for example, may be a deep neural network (DNN) model, a convolutional neural network (CNN), etc., including multiple operators, which may specifically be multiple neurons (or nodes) included in the network layer in the target model. In actual application, among the multiple operators included in the target model, some operators may be executed by the processor 101 on the host side alone, and another part of the operators may be executed by the processor 101 in collaboration with the processor 102 on the device side, so as to utilize the processor 102 to accelerate the execution efficiency of the other part of the operators.

[0088] Exemplarily, the processor 101 and the processor 102 may execute the target model based on a file set in advance, and the file set may be, for example, an operator package including executable codes of all operators released by a manufacturer. In this way, the file processing device 104 can obtain the identifiers of the operators that the processor 101 and the processor 102 need to execute during the operation of the target model, that is, determine the identifiers of multiple third operators (executed by the processor 101) and the identifier of at least one fourth operator (executed by the processor 102) in step S501.

[0089] In this way, the file processing device 104 can filter the executable codes of the multiple operators included in the file set according to the identifiers of the multiple third operators and the identifier of at least one fourth operator, obtain the executable files of the multiple third operators, hereinafter referred to as the third executable files, and obtain the executable file of at least one fourth operator, hereinafter referred to as the fourth executable file. Further, the file processing device 104 can package the third executable files of the multiple third operators into a third file, and package the fourth executable file of at least one fourth operator into a fourth file, and save the third file and the fourth file to the file system constructed based on the memory 103. Among them, regarding the specific implementation method of filtering the file set to generate the third file and the fourth file, please refer to the relevant description of the aforementioned embodiment, which will not be repeated here.

[0090] Exemplarily, the third file and the fourth file may both be in binary file format, such as a file with a suffix of "bin".

[0091] S502: the file processing device 104 parses the third file to obtain a plurality of third executable files, each of which is used to record the executable code of a third operator.

[0092] S503: The file processing device 104 parses the fourth file to obtain at least one fourth executable file, each of which is used to record an executable code of a fourth operator.

[0093] It can be understood that, since the third file includes the executable code of at least one operator, the file processing device 104 can instruct the processor 101 to parse the executable code of multiple third operators from the third file, and each executable file of the third operator is saved in a third executable file, and the third executable file can be, for example, a dynamic link library file, such as a file in the ".so" format. Similarly, the file processing device 104 can instruct the processor 102 to parse the executable code of at least one fourth operator from the fourth file, and each executable file of the fourth operator is saved in a fourth executable file, and the fourth executable file can be, for example, an object file, such as a file in the ".obj" format.

[0094] S504: the file processing device 104 adds the executable codes in the plurality of third executable files and the executable code in at least one fourth executable file to the file of the target model to obtain an offline model file.

[0095] Since the two heterogeneous processors need to execute multiple operators in the target model according to the executable codes in the multiple third executable files and the executable codes in at least one fourth executable file during the execution of the target model, the file processing device 104 can add the executable codes in the executable files that the two heterogeneous processors need to execute to the file of the target model, specifically, the executable codes in each executable file can be serialized into the file of the target model. In this way, the executable codes corresponding to multiple operators can be integrated in the file of the target model.

[0096] S505: The file processing device 104 outputs the offline model file.

[0097] In this way, after the offline model file is deployed to a computing device including multiple heterogeneous processors, due to the executable code of multiple operators integrated in the offline model file, the computing device does not need to obtain the file set including the executable code of the full set of operators released by the manufacturer. Instead, it can execute the multiple operators in the target model based on multiple heterogeneous processors by executing the executable code integrated in the offline model file, thereby realizing the running of the target model on the computing device and reducing the requirements for the storage resources of the computing device.

[0098] Combination of the above Figures 1 to 5 The file processing method provided in the embodiment of the present application is introduced, and then the functions of the file processing device provided in the embodiment of the present application and the computing device that implements the file processing device are introduced in conjunction with the accompanying drawings.

[0099] See also Figure 6 , shows a schematic diagram of the structure of a file processing device. Figure 6 The file processing apparatus 600 shown can be applied to a computing device, such as the above-mentioned Figure 1 The computing device 10 shown in FIG. 1 includes a first processor and a second processor, and the first processor and the second processor are heterogeneous. Figure 6 As shown, the file processing device 600 includes:

[0100] The acquisition module 601 is used to acquire a file set, where the file set includes executable codes of multiple first operators and executable codes of multiple second operators; acquire a first identifier of a first operator to be executed when the first processor performs a computing task, and a second identifier of a second operator to be executed when the second processor performs a computing task;

[0101] A filtering module 602 is configured to filter the executable codes of the plurality of first operators according to the first identifier to obtain a first file, the first file including the executable code of the first operator indicated by the first identifier; and filter the executable codes of the plurality of second operators according to the second identifier to obtain a second file, the second file including the executable code of the second operator indicated by the second identifier;

[0102] The execution module 603 is used to use the first processor and the second processor to execute the computing task according to the first file and the second file.

[0103] In a possible implementation, the filtering module 602 is specifically configured to:

[0104] Filter the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier;

[0105] The executable code of the first operator indicated by the first identifier is packaged to obtain a first file.

[0106] In a possible implementation, the execution module 603 is specifically configured to:

[0107] Parsing the first file to obtain at least one first executable file, each first executable file being used to record an executable code of a first operator;

[0108] Parsing the second file to obtain at least one second executable file, each second executable file being used to record an executable code of a second operator;

[0109] The first processor is used to execute at least one first executable file, and the second processor is controlled to execute at least one second executable file in sequence according to the execution order of the at least one first executable file.

[0110] In a possible implementation manner, the formats of the first file and the second file are binary file formats.

[0111] In a possible implementation manner, the first executable file is a dynamic link library file, and each dynamic link library file records a binary code of a first operator;

[0112] The second executable file is an object file, and each object file records a binary code of a second operator.

[0113] In a possible implementation, the file processing device 600 further includes:

[0114] The saving module 604 is configured to save the first file and the second file in a file system of the computing device.

[0115] In a possible implementation, the acquisition module 601 is further used to acquire a third file and a fourth file from a file system, where the third file includes executable files of multiple third operators that need to be executed when the first processor executes the target model, and the fourth file includes an executable file of at least one fourth operator that needs to be executed when the second processor executes the target model, and the at least one fourth operator is a subset of the multiple third operators;

[0116] Then, the file processing device 600 may further include:

[0117] The parsing module 605 is used to parse the third file to obtain multiple third executable files, each of which is used to record the executable code of a third operator; parse the fourth file to obtain at least one fourth executable file, each of which is used to record the executable code of a fourth operator;

[0118] An adding module 606, configured to add executable codes in the plurality of third executable files and executable codes in at least one fourth executable file to a file of a target model to obtain an offline model file;

[0119] The output module 607 is used to output the offline model file.

[0120] In a possible implementation, the acquisition module 601 is specifically configured to:

[0121] Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute computing tasks according to the file set;

[0122] According to the historical operation data, a first identifier and a second identifier are determined.

[0123] In a possible implementation, the identifiers of the plurality of second operators match the identifiers of some of the plurality of first operators.

[0124] because Figure 6 The file processing device 600 shown corresponds to Figure 3 as well as Figure 5 The file processing device 104 in the method embodiment shown in FIG. Figure 6 The specific implementation method of the file processing device 600 shown and its technical effects can be found in the relevant description of the above embodiments, which will not be repeated here.

[0125] Figure 7The schematic diagram of the structure of a computing device is shown. The file processing device 104 can be deployed on the computing device. The computing device can be a separate device or any computing device in a computing device cluster. Specifically, it can be used to implement the above Figure 3 as well as Figure 5 The functions of the file processing device 104 in the illustrated embodiment.

[0126] like Figure 7 As shown, computing device 700 includes processor 710, memory 720, communication interface 730 and bus 740. Processor 710, memory 720 and communication interface 730 communicate through bus 740. Bus 740 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The communication interface 730 is used for external communication, such as obtaining a file set or outputting an offline model file.

[0127] Among them, the processor 710 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits. The processor 710 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, the functions of the server 200 can be completed by the hardware integrated logic circuit or software instructions in the processor 710. The processor 710 can also be a general processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. Among them, the general processor can be a microprocessor or the processor can also be any conventional processor, etc., and the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 720 , and the processor 710 reads the information in the memory 720 and completes part or all of the functions in the server 200 in combination with its hardware.

[0128] The memory 720 may include a volatile memory, such as a random access memory (RAM), or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD, or a SSD.

[0129] The memory 720 stores executable codes, and the processor 710 executes the executable codes to execute the method executed by the aforementioned file processing device 104 .

[0130] Specifically, in implementing Figure 3 as well as Figure 5 In the case of the embodiment shown, and Figure 3 as well as Figure 5 In the embodiment shown, the file processing device 104 is implemented by software. Figure 3 as well as Figure 5 The software or program code required for the functions of the file processing device 104 is stored in the memory 720. The interaction between the file processing device 104 and other devices is realized through the communication interface 730. The processor is used to execute the instructions in the memory 720 to implement the method executed by the file processing device 104.

[0131] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on one or more computing devices, the one or more computing devices execute the method executed by the file processing apparatus 104 in the above embodiment.

[0132] In addition, the embodiment of the present application further provides a computer program product, and when the computer program product is executed by one or more computing devices, the one or more computing devices execute any of the aforementioned file processing methods. The computer program product may be a software installation package, and when any of the aforementioned file processing methods is required, the computer program product may be downloaded and executed on a computer.

[0133] It should also be noted that the embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the embodiment drawings provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0134] The terms "first", "second", etc. in the specification, claims and the above drawings of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0135] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0136] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0137] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A file processing method, characterized in that: The method is applied to a computing device, the computing device includes a first processor and a second processor, the first processor and the second processor are heterogeneous, and the method includes: Acquire a file set, where the file set includes executable codes of multiple first operators and executable codes of multiple second operators; Obtaining a first identifier of a first operator that needs to be executed when the first processor performs a computing task, and a second identifier of a second operator that needs to be executed when the second processor performs the computing task; Filter the executable codes of the plurality of first operators according to the first identifier to obtain a first file, where the first file includes the executable code of the first operator indicated by the first identifier; Filter the executable codes of the plurality of second operators according to the second identifier to obtain a second file, where the second file includes the executable code of the second operator indicated by the second identifier; The computing task is executed using the first processor and the second processor according to the first file and the second file.

2. The method according to claim 1, characterized in that: The filtering the executable codes of the plurality of first operators according to the first identifier to obtain a first file includes: Filter the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier; The executable code of the first operator indicated by the first identifier is packaged to obtain the first file.

3. The method according to claim 1 or 2, characterized in that: The using the first processor and the second processor to perform a computing task according to the first file and the second file includes: Parsing the first file to obtain at least one first executable file, each first executable file being used to record an executable code of a first operator; Parsing the second file to obtain at least one second executable file, each second executable file being used to record an executable code of a second operator; The at least one first executable file is executed by the first processor, and the second processor is controlled to execute the at least one second executable file in sequence according to the execution order of the at least one first executable file.

4. The method according to claim 3, characterized in that The formats of the first file and the second file are binary file formats.

5. The method according to claim 4, characterized in that The first executable file is a dynamic link library file, and each of the dynamic link library files records a binary code of a first operator; The second executable file is an object file, and each object file records the binary code of a second operator.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The first file and the second file are saved in a file system on the computing device.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a third file and a fourth file from a file system, wherein the third file includes executable files of multiple third operators required to be executed by the first processor when executing the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed by the second processor when executing the target model, and the at least one fourth operator is a subset of the multiple third operators; Parsing the third file to obtain a plurality of third executable files, each of the plurality of third executable files being used to record an executable code of a third operator; Parsing the fourth file to obtain at least one fourth executable file, each fourth executable file in the at least one fourth executable file is used to record an executable code of a fourth operator; Adding the executable codes in the plurality of third executable files and the executable code in the at least one fourth executable file to the file of the target model to obtain an offline model file; The offline model file is output.

8. The method according to any one of claims 1 to 7, characterized in that: The obtaining a first identifier of a first operator to be executed by the first processor and a second identifier of a second operator to be executed by the second processor includes: Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute the computing task according to the file set; The first identifier and the second identifier are determined according to the historical operation data.

9. The method according to any one of claims 1 to 8, characterized in that: The identifiers of the plurality of second operators match identifiers of some of the plurality of first operators.

10. A file processing device, characterized in that: The file processing apparatus is applied to a computing device, the computing device includes a first processor and a second processor, the first processor and the second processor are heterogeneous, and the file processing apparatus includes: an acquisition module, configured to acquire a file set, the file set comprising executable codes of a plurality of first operators and executable codes of a plurality of second operators; and to acquire a first identifier of a first operator to be executed when the first processor performs a computing task and a second identifier of a second operator to be executed when the second processor performs the computing task; a filtering module, configured to filter the executable codes of the plurality of first operators according to the first identifier to obtain a first file, wherein the first file includes the executable code of the first operator indicated by the first identifier; and filter the executable codes of the plurality of second operators according to the second identifier to obtain a second file, wherein the second file includes the executable code of the second operator indicated by the second identifier; An execution module is used to utilize the first processor and the second processor to execute the computing task according to the first file and the second file.

11. The device according to claim 10, characterized in that The filtering module is used for: Filter the executable codes of the plurality of first operators according to the first identifier to obtain the executable code of the first operator indicated by the first identifier; The executable code of the first operator indicated by the first identifier is packaged to obtain the first file.

12. The device according to claim 10 or 11, characterized in that The execution module is used to: Parsing the first file to obtain at least one first executable file, each first executable file being used to record an executable code of a first operator; Parsing the second file to obtain at least one second executable file, each second executable file being used to record an executable code of a second operator; The at least one first executable file is executed by the first processor, and the second processor is controlled to execute the at least one second executable file in sequence according to the execution order of the at least one first executable file.

13. The device according to claim 12, characterized in that The formats of the first file and the second file are binary file formats.

14. The device according to claim 13, characterized in that The first executable file is a dynamic link library file, and each of the dynamic link library files records a binary code of a first operator; The second executable file is an object file, and each object file records the binary code of a second operator.

15. The device according to any one of claims 10 to 14, characterized in that The file processing device also includes: A saving module is used to save the first file and the second file in a file system of the computing device.

16. The device according to any one of claims 10 to 15, characterized in that The acquisition module is further used to acquire a third file and a fourth file from the file system, wherein the third file includes executable files of multiple third operators required to be executed when the first processor executes the target model, and the fourth file includes an executable file of at least one fourth operator required to be executed when the second processor executes the target model, and the at least one fourth operator is a subset of the multiple third operators; The file processing device also includes: a parsing module, configured to parse the third file to obtain a plurality of third executable files, each of the plurality of third executable files being used to record an executable code of a third operator; and parse the fourth file to obtain at least one fourth executable file, each of the at least one fourth executable file being used to record an executable code of a fourth operator; An adding module, used for adding the executable codes in the plurality of third executable files and the executable code in the at least one fourth executable file to the file of the target model to obtain an offline model file; An output module is used to output the offline model file.

17. The device according to any one of claims 10 to 16, characterized in that The acquisition module is used to: Acquire historical operation data, where the historical operation data includes data generated when the first processor and the second processor execute the computing task according to the file set; The first identifier and the second identifier are determined according to the historical operation data.

18. The device according to any one of claims 10 to 17, characterized in that The identifiers of the plurality of second operators match identifiers of some of the plurality of first operators.

19. A file processing system, characterized in that: The file processing system comprises a first processor, at least one second processor, a memory and a file processing device, wherein the first processor and the at least one second processor are heterogeneous; The file processing device is used to execute the method according to any one of claims 1 to 9.

20. A computing device, characterized in that The computing device comprises a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the method according to any one of claims 1 to 9.

Images