Executable file format setting method, data stream program migration method and device

Abstract

The application discloses an executable file format setting method, a data stream program migration method and device, wherein the executable file format setting method creates a node program topology graph formed based on a starting address list and a topology information list of each data stream node in the executable file, and when an operating system participates in data scheduling, data layout and data optimization in a data stream execution mode, the topology information and the starting address of each data stream node can be quickly obtained by the operating system, and the starting address of each data stream node is aligned at a page boundary, so that the subsequent node dynamic migration is facilitated, and the system running speed is improved. The data stream program migration method is based on the node program topology graph (DAG graph), and the mapping and unmapping of virtual and real pages can be performed with reference, so that the system running speed can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of executable file format setting, in particular to an executable file format setting method and a data flow program migration method and device. BACKGROUND

[0002] An executable file refers to a file that can be loaded and executed by an operating system, and the presentation of the executable file is different in different operating system environments. In a Windows operating system, the executable file can be an.exe file, and in a Linux operating system environment, the executable file is an ELF file (Executable Linkable Format).

[0003] A data flow program is executed in a data-driven manner, allocates data to be processed to each core, separates the calculation and communication of data, uses the parallel characteristics of software pipelining to fully exploit the potential parallelism in the flow program through task scheduling and allocation, and balances the load between the cores.

[0004] Since most current software and hardware platforms are based on control flow program design, the ELF file format is also matched with the mode of the control flow program, and the ELF file format in this mode cannot be directly applied to the ELF file of the data flow program. Therefore, in order to cooperate with the scheduling and execution of the operating system of the data flow program, the ELF file format of the data flow program also needs to be adapted and modified accordingly.

[0005] In the current mainstream computing system, for example, in the environment of a Linux operating system running on hardware platforms such as x86, ARM, and MIPS, the ELF file mainly records the content of the code and data, and the related information of the ELF file format is recorded in the application program. Therefore, when the ELF file of the data flow program is executed to run a task, the data flow execution mode is completely responsible by the application program itself, or is supported by the application program and the runtime library in the user mode, which causes the operating system to be unable to participate in the data scheduling, data layout, and data optimization in the data flow execution mode, and finally leads to slow data flow migration speed and affects system operation. SUMMARY

[0006] Therefore, the technical problem to be solved by the present application is to overcome the problem that the operating system in the prior art cannot participate in the data scheduling, data layout, and data optimization in the data flow execution mode, which finally leads to slow data flow migration speed and affects system operation, so as to provide an executable file format setting method, a data flow program migration method, and a device.

[0007] According to the first aspect, the embodiments of the present application further disclose a setting method of an executable file format based on a data flow program, comprising the following steps:

[0008] obtaining each data flow node of a function code;

[0009] creating a start address list and a topology information list of the each data flow node, wherein the topology information list comprises: a total number of the each data flow node and / or a topology relationship between the each data flow node and / or an execution order of the each data flow node and / or a current scheduling execution state of the each data flow node in dynamic migration;

[0010] creating a node program topology graph based on the start address list and the topology information list of the each data flow node;

[0011] arranging the start address list and the topology information list to a first program segment, and arranging the each data flow node to a second program segment;

[0012] forming a target executable file according to the content of the first program segment and the second program segment, and adding position identification information of the node program topology graph to the target executable file.

[0013] By executing the above embodiment, the node program topology graph based on the start address list and the topology information list of the each data flow node is created in the executable file, when the operating system participates in data scheduling, data layout and data optimization in the data flow execution mode, the topology information and the start address of the each data flow node are quickly obtained by the operating system, and the start of the each data flow node is aligned at the page boundary based on the start address of the each data flow node, so that the subsequent node dynamic migration is facilitated, and the system running speed is improved.

[0014] In combination with the first aspect, in an implementation manner of the first aspect, when the each data flow node is linked, the each data flow node is linked into PIC position independent code, and the PIC position independent code is independently formed into a section; when the each data flow node is linked, the start of the each data flow node is aligned at the page boundary based on the start address of the each data flow node.

[0015] By executing the above-mentioned embodiments, when compiling each data flow node, each data flow node is linked into PIC position-independent code, and the PIC position-independent code is independently segmented. This is conducive to normal operation in any stack space when each data flow node is called, ensures that each data flow node can operate normally at any position, does not need to modify the address information of each data flow node, and ultimately is conducive to improving the calling speed of each data flow node. At the same time, the starting point of each data flow node is aligned at the page boundary, which is conducive to subsequent migration of the executable file, allows the executable file to migrate in units of nodes, improves the possibility and flexibility of data flow node migration, and further improves the data flow running speed.

[0016] In combination with any embodiment of the first aspect, in another embodiment of the first aspect, when linking each data flow node, the address information of the input data of each data flow node is relocated to the target stack region, and the address information of the output data of each data flow node is not relocated.

[0017] By executing the above-mentioned embodiments, the process space can be migrated simply without the need for complex dynamic linking and relocation again, and only the starting address of the output data needs to be simply relocated. When linking each data flow node, the address information of the input data of each data flow node is loaded in the stack region, which is conducive to improving the data flow running speed when each data flow node is dynamically migrated.

[0018] According to the second aspect, an embodiment of the present application provides a migration method of an executable file data flow program, including the following steps:

[0019] In the target executable file, determine a data flow node to be migrated;

[0020] According to the node program topology graph in the executable file format setting method based on a data flow program according to the first aspect or any embodiment of the first aspect, the starting address of the data flow node to be migrated is relocated to the target stack region, and the current starting address of the target stack region is updated in the node program topology graph;

[0021] According to the node program topology graph, obtain the current scheduling running state of the data flow node to be migrated;

[0022] According to the current scheduling running state of the data flow node to be migrated, determine whether to map the data flow node to be migrated on the physical storage area.

[0023] In combination with the second aspect, in any implementation of the second aspect, the determining whether to map the to-be-migrated data flow node of the virtual storage area on the physical storage area according to the current scheduling running state of the to-be-migrated data flow node comprises:

[0024] if the current scheduling running state of the to-be-migrated data flow node is the executed state, the to-be-migrated data flow node of the virtual storage area is un-mapped on the physical storage area;

[0025] if the current scheduling running state of the to-be-migrated data flow node is the executing state, the to-be-migrated data flow node of the virtual storage area is locked on the physical storage area;

[0026] if the current scheduling running state of the to-be-migrated data flow node is the to-be-executed state, the to-be-migrated data flow node of the virtual storage area does not need to be mapped on the physical storage area.

[0027] By executing the above-mentioned implementation of the second aspect, since the node program topology graph-DAG graph in the implementation of the first aspect is used, the mapping and un-mapping of the virtual and real pages are performed with basis in the embodiment of the present application, instead of blindly using the LRU algorithm as the page replacement algorithm, so that the page fault rate can be significantly reduced, and the system running speed is improved.

[0028] In combination with any implementation of the second aspect, in another implementation of the second aspect, the migration method of the executable file data flow program further comprises: if the storage space of the physical storage area is less than or equal to a preset threshold, mapping the to-be-migrated data flow node of the virtual storage area on the physical storage area according to the priority execution order of the node program topology graph.

[0029] By executing the above-mentioned implementation, the address mapping of the "front" node is preferentially established, the mapping relationship between the virtual page and the real page can be flexibly established in combination with the actual situation of the physical storage area, and the migration efficiency of the data flow node program is improved, and the system running speed is improved.

[0030] In combination with any implementation of the second aspect, in another implementation of the second aspect, based on the node program topology graph, when the to-be-migrated data flow node is migrated, the input address of the to-be-migrated data flow node is relocated to the target stack area, and the output address of the to-be-migrated data flow node does not need to be relocated, the output data of the to-be-migrated data flow node is written into the stack area where the subsequent node is located, and the output data of the to-be-migrated data flow node is written into the stack area where the subsequent node is located.

[0031] In combination with any of the embodiments of the second aspect, in another embodiment of the second aspect, based on the node program topology graph, based on the node program topology graph, when migrating the to-be-migrated data flow node, the stack frame start address of the to-be-migrated data flow node executed on the same processor is the same, and when executing the to-be-migrated data flow node, the data memory address of the to-be-migrated data flow node is limited in the memory space of the node by using a page table address mechanism.

[0032] According to a third aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores computer instructions, the computer instructions are used to make the computer execute the setting method of the executable file format based on the data flow program in the first aspect or any of the embodiments of the first aspect; or, the migration method of the executable file data flow program in the second aspect or any of the embodiments of the second aspect.

[0033] According to a fourth aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor, which are in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions, thereby executing the setting method of the executable file format based on the data flow program in the first aspect or any of the embodiments of the first aspect; or, the migration method of the executable file data flow program in the second aspect or any of the embodiments of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0035] Figure 1 A flow chart of a specific example of the setting method of the executable file format based on the data flow program in the embodiments of the present application;

[0036] Figure 2 A node program topology graph in the embodiments of the present application;

[0037] Figure 3 A setting schematic diagram of the data structure in the executable file in the embodiments of the present application;

[0038] Figure 4A A loading schematic diagram of a traditional ELF executable file;

[0039] Figure 4BAn ELF executable file loading diagram in an embodiment of the present application;

[0040] Figure 5 A diagram for writing data from a predecessor node a to a successor node b in an embodiment of the present application;

[0041] Figure 6 A flow chart of a specific example of an executable file data flow program migration method in an embodiment of the present application;

[0042] Figure 7 An executable file running mode diagram in an embodiment of the present application;

[0043] Figure 8 An executable file data flow node program b migration diagram in an embodiment of the present application;

[0044] Figure 9 A current scheduling execution state diagram of each data flow node in an embodiment of the present application;

[0045] Figure 10 An executable file data flow node program migration diagram in an embodiment of the present application;

[0046] Figure 11 An executable file format setting and migration diagram based on data flow programs in an embodiment of the present application;

[0047] Figure 12 An executable file format setting device structure diagram based on data flow programs in an embodiment of the present application;

[0048] Figure 13 An executable file data flow program migration device in an embodiment of the present application;

[0049] Figure 14 A computer device hardware structure diagram in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0051] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0052] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or it can be the internal communication of two elements, it can be wireless connection, or wired connection. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0053] In addition, the technical features involved in the different embodiments of the application described below can be combined with each other as long as there is no conflict between them.

[0054] The embodiment of the present application discloses a setting method of executable file format based on data flow program, as shown in Figure 1 The method comprises the following steps:

[0055] Step S11: obtaining each data flow node of the function code.

[0056] Each data flow node of the function code is executed by a data-driven mode, the data to be processed is allocated to each core, the calculation and communication of the data are separated, the parallel characteristics of software flow are utilized to fully mine the potential parallelism in the flow program through task scheduling and allocation, and the load balancing between each core is realized. The connection operation of input and output is defined. Therefore, each data flow node does not adopt the command mode, the relevant operation is immediately executed as soon as the data is ready for input, so the data flow programming is essentially parallel, and can be well operated in a multi-core processor and a large distributed system. For example, the obtained each data flow node of the function code is a, b, c, d, e, f and g.

[0057] Step S12: creating a start address list and a topology information list of each data flow node, the topology information list comprising: the total number of each data flow node and / or the topological relationship between each data flow node and / or the execution sequence of each data flow node and / or the current scheduling execution state of each data flow node when dynamically migrating.

[0058] For example, the starting address list is used to record the starting position of each data flow node. This starting address list is equivalent to the starting address index list of each data flow node. For example, if the data flow nodes are a, b, c, d, e, f, and g, then the starting address list records the starting position of each data flow node a, b, c, d, e, f, and g.

[0059] For example: If the data flow nodes are a, b, c, d, e, f, and g, then the total number of data flow nodes in the topology information list is 7. For example: The topological relationships between the data flow nodes in the topology information list are the input-to-output relationships of any data flow node, that is, the predecessor-successor relationships of any data flow node. For example: If the data flow nodes are a, b, c, and d, and a is executed first, then b, then c and d, then the execution order in the topology information list is: a→b→c, d, where a is the predecessor node of b, and c and d are the successors nodes of b. For example: For the current scheduling execution states of data flow nodes a, b, and c, where a is the executed state, b is the currently executing state, and c is the unexecuted state, then the current scheduling execution states of each data flow node during dynamic migration include: executed state, currently executing state, and unexecuted state.

[0060] In one specific implementation, the topology information list of each data flow node further includes: the data size of the topology relationship between each data flow node and / or the entry function address of each data flow node and / or the stack frame size of the stack area to which each data flow node belongs.

[0061] For example: Figure 2 As shown, node e's topology input is node b, and its topology output is node g; node c's inputs are a and b, and its topology output is f. The data size of the topology relationship between data flow nodes is 2 bytes, and the data size of each data flow node is 1 byte. The entry function address of each data flow node and the stack frame size of the stack area to which each data flow node belongs are usually fixed values, for example, the entry function address of each data flow node and the stack frame size of the stack area to which each data flow node belongs are both 256 bytes.

[0062] Step S13: Create a node program topology graph based on the starting address list and topology information list of each data flow node.

[0063] For example, static instances of each data stream node can be described as a directed graph (DAG) based on a list of node start addresses and a list of node topology information, i.e., the node program topology graph mentioned above. Figure 2In the figure, the nodes represent the computing units, and the edges represent the data transmission paths. Data is transmitted between adjacent nodes through the edges. The nodes consume data for computation and output the generated data to the input / output sequence as the input of the next computing unit. The topology relationship between the data flow nodes is described by the node program topology graph, which can more intuitively express the input and output between the data flow nodes.

[0064] Step S14: arranging the start address list and the topology information list to the first program segment, and arranging each data flow node to the second program segment.

[0065] For example, as shown in FIG. 1, the first program segment records the start address list and the topology information list; and the second program segment records the data flow functions, i.e., the data flow nodes df1-dfn in the above embodiment. Figure 3

[0066] In a specific embodiment, the content data size of the first program segment is set as a preset byte value. For example, the data size of the start address list is M bytes, and the data size of the topology information list is N bytes. Therefore, the content data size of the first program segment pair is M+N bytes, and the preset byte value is M+N bytes. Of course, the preset byte value is not limited to this, and can be set according to the actual situation.

[0067] In a specific embodiment, the offset of the first program segment in the file is saved on the program header of the target executable file.

[0068] Step S15: forming the target executable file according to the content of the first program segment and the second program segment, and adding the position identification information of the node program topology graph to the target executable file.

[0069] In the executable file, there is only one segment, i.e., the content of the first program segment is used to record the data flow node topology information, and the above-mentioned position identification information is used to indicate the belonging position of the node program topology graph in the executable file. The position identification information indicates the start byte offset of the node program topology graph in the executable file. The position identification is beneficial to quickly retrieve the node program topology graph, i.e., the DAG graph in FIG. 1. Figure 2

[0070] In a specific embodiment, when linking the data flow nodes, the data flow nodes are linked into PIC position-independent code, and the PIC position-independent code is independently segmented; and when linking the data flow nodes, the start points of the data flow nodes are aligned at the page boundary to form a segment.

[0071] ​​When linking the data stream nodes, each data stream node is linked into PIC (Position-Independent Code) code, and this PIC code is then separated into independent sections. This ensures that each data stream node can run normally in any stack space when called, guaranteeing that each data stream node can run normally in any location without needing to modify the address information of each data stream node, ultimately improving the calling speed of each data stream node.

[0072] In one specific implementation, when linking the various data stream nodes, the starting points of each data stream node are aligned at the page boundary based on its starting address to form a segment. Aligning the starting points of each data stream node at the page boundary facilitates subsequent migration of the executable file, allowing the executable file to migrate in units of nodes, increasing the possibility and flexibility of data stream node migration, and thus improving the data stream execution speed.

[0073] like Figure 4A The diagram shown illustrates the loading of a traditional ELF executable file, which does not load memory in units of nodes. Figure 4B The diagram shown is a schematic of loading an ELF executable file in an embodiment of the present invention, with the starting points of each data stream node aligned with the page boundaries.

[0074] In one specific implementation, when linking the data stream nodes, the address information of the input data of each data stream node is limited to its respective stack area, but the address information of the output data of each data stream node is not relocated.

[0075] For example, when linking the functions of each data stream node, the input data of each data stream node is in its respective stack, while the address of its output data has not yet been relocated (it is within the stack frame of subsequent data stream nodes). Similar to the dynamic linking process of dynamic libraries, the difference is that the function code of these data stream nodes is restricted to not accessing global variables. Therefore, it can be easily migrated within the process space without undergoing complex dynamic linking and relocation again; only a simple relocation of the starting address of the output data is needed. It is evident that restricting the address information of the input data of each data stream node to its respective stack area during linking is beneficial for improving the data stream execution speed during subsequent dynamic migrations.

[0076] For example: Figure 5 The diagram illustrates how predecessor node a writes data to successor node b, and how data flow node a outputs data to data flow node b. Data flow node a is bound to thread 1 and runs, with its input data residing in thread 1's stack. However, its output data needs to be written to the stack of the task of data flow node program b. The address of its output data can only be determined after data flow node b is bound to a specific thread (in this case, thread 2) and its stack is known.

[0077] The section of the function of each data flow node can be migrated from the original NUMA node to another NUMA node after the initial loading and as the node program topology-DAG graph is calculated. After the migration, only the start position of the output data needs to be relocated. For example, in Figure 5 , the data flow node b task is migrated to thread 3 bound to another NUMA node, and then the data written by a needs to be transferred to the new position (from the right end of Figure 5 to the left end of Figure 5 ).

[0078] In an embodiment, the setting method of the executable file format based on the data flow program in the embodiment of the present application is based on the node program topology graph, and when each data flow node is migrated and accessed, the global variable of each data flow node is not accessed.

[0079] The executable file is allowed to be migrated in the unit of node, and when the global variable of each data flow node is not accessed, only the start address information of each data flow node needs to be relocated, and the global variable of each data flow node does not need to be accessed, so that the data flow running speed can be further improved.

[0080] The setting method of the executable file format based on the data flow program in the embodiment of the present application is described below by taking one node of the data flow node program graph-DAG graph as an example, using the topological relationship (predecessor-successor relationship) record, and recording the node function pointer and the required stack frame size, for example:

[0081]

[0082]

[0083] In the above program code function, pre_count is used to record the number of predecessors, pres[] is used to record the pointers of the pre_count predecessors. succ_count and succs[] are used to record the successor node relationship; and node-id is used to record the identification node.

[0084] In Figure 2 , there are seven nodes, and if the variable mynodes is used to record, then there should be struct nodemynodes[7]. The mynodes is formed into a section of an ELF executable file alone, the section alone constitutes an ELF executable segment, and function(void) is used to record the start address list of each data flow node.

[0085] For all output data, a "base address + offset" method is used for addressing. For example, a node transmits 10 bytes of data to a successor node. The base address is the stack bottom of the successor node on the corresponding processor, and the base address of the input data of the successor node is addr1. Then the data is written to the address range [addr1~addr1+9], and the data transmission is completed.

[0086] The setting method of the executable file format of the data flow program in the embodiment of the application, by performing the steps S11-S15, forms a node program topology graph by creating a starting address list and a topology information list of each data flow node in the executable file. When the operating system participates in data scheduling, data layout and data optimization in the data flow execution mode, the topology information and the starting address of each data flow node are quickly obtained by the operating system, and the starting address of each data flow node is aligned at the page boundary, which is beneficial to the dynamic migration of the subsequent node and improves the system running speed.

[0087] Based on the same concept, the embodiment of the application further discloses a migration method of an executable file data flow program, as shown in Figure 6 The migration method comprises the following steps:

[0088] Step S61: determining a data flow node to be migrated in the target executable file.

[0089] For example, in the data flow nodes a, b, c, d, e, f and g included in the target executable file, the data flow node to be migrated is a.

[0090] Step S62: based on the starting address repositioning of the data flow node to be migrated to the target stack area and the updating of the current starting address of the target stack area in the node program topology graph in the setting method of the executable file format of the data flow program in the above embodiment, the starting address of the data flow node to be migrated is repositioned to the target stack area, and the current starting address of the target stack area is updated in the node program topology graph. By implementing step S62, it is ensured that the scheduler can correctly find the entry address of the task after the task migration.

[0091] In Figure 5 , a is a data flow node to be migrated, and a is also a predecessor node of b. The predecessor node a writes data to the successor node b, and the data flow node a outputs data to the data flow node b. The data flow node a is bound to thread 1 to obtain running, and the input data of the data flow node a is in the target stack area of thread 1. However, the output data needs to be written to the stack of the data flow node b task, and the address of the output data can be determined only after the data flow node b is bound to a specified thread (thread 2 in this case) and the stack of the specified thread is known.

[0092] Therefore, the node program topology graph is generated based on the start address list of each data flow node, the start address of the to-be-migrated data flow node a in the DAG graph based on the node program topology graph is relocated into the target stack area, and the start point of the to-be-migrated data flow node is also aligned at the page boundary, so that the output address of the to-be-migrated data flow node a does not need to be relocated, and the to-be-migrated data flow node b can be quickly located.

[0093] As shown in Figure 7 , it is a schematic diagram of the running mode of the executable file in the embodiment of the present application. After loading each data flow node in the above embodiment into the memory, the task migration is allowed, based on the setting method of the executable file format in the embodiment, only the stack variable is used, therefore, the function code can normally run regardless of where it is migrated. Since the DAG graph information recorded in the process PCB (process control block) has the function node information, the new address after migration needs to be updated to the function start address information about the task b in the DAG graph information in the PCB. As shown in Figure 8 , it is a migration schematic diagram of the to-be-migrated data flow node b in the executable file in the embodiment of the present application.

[0094] Step S63: acquiring the current scheduling running state of the to-be-migrated data flow node according to the node program topology graph.

[0095] Because the topology information list of the node program topology graph records the current scheduling running state of each data flow node during dynamic migration, including the executed state, the executing state and the unexecuted state. As shown in Figure 9 , it is the current scheduling running state of each data flow node in the embodiment of the present application.

[0096] Step S64: determining whether to map the to-be-migrated data flow node of the virtual storage area on the physical storage area according to the current scheduling running state of the to-be-migrated data flow node.

[0097] In a preferred embodiment, in Figure 6 , the above step S64, determining whether to map the to-be-migrated data flow node of the virtual storage area on the physical storage area according to the current scheduling running state of the to-be-migrated data flow node, comprises:

[0098] Step S641: if the current scheduling running state of the to-be-migrated data flow node is the executed state, the to-be-migrated data flow node of the virtual storage area is unmapped on the physical storage area.

[0099] Step S642: if the current scheduling running state of the to-be-migrated data flow node is the executing state, the to-be-migrated data flow node of the virtual storage area is locked on the physical storage area.

[0100] Step S643: If the current scheduling running state of the data stream node to be migrated is the pending execution state, there is no need to map the data stream node to be migrated in the virtual storage area onto the physical storage area.

[0101] The above steps S641-S643 are executed in any order.

[0102] The space corresponding to the "section" of a data flow node function that has been executed can be released, and the mapped physical memory can be reclaimed. The space of unexecuted data flow node functions does not need to be mapped and therefore no physical memory is allocated.

[0103] like Figure 10 The diagram shown illustrates the migration of the data stream node program of the executable file in an embodiment of the present invention. The virtual storage area is... Figure 10 The programming space in the physical memory is... Figure 10 The physical memory in the system. Because the node program topology graph—DAG—of the above embodiments is used, the mapping and unmapping of virtual and physical pages are performed based on evidence, rather than blindly using the LRU algorithm as the paging algorithm, thereby significantly reducing the page fault rate.

[0104] For example: in Figure 9 In this scenario, data flow node a has finished running and its memory can be reclaimed; node d is not running and no mapping is established; while nodes b and c are currently executing, possessing physical page frame mappings and not being swapped out. Instead of using the traditional LRU paging algorithm for executable file migration and paging, mapping and unmapping are performed based on the scheduling process. Furthermore, it does not use a standard memory page prefetching algorithm, but rather determines the prefetched content based on the DAG graph, significantly reducing the page fault rate.

[0105] like Figure 11 The diagram shows the executable file format settings and migration diagram for data flow-based programs. Figure 11 In the above embodiments, using the executable file formatting method based on data flow programs, the DAG graph is loaded into memory, and the data flow graph node functions Df1 to Dfn are loaded into memory (initial loading). At this time, the physical mapping is all on Node0. If Df2 needs to be allocated to NUMA node 1 for scheduling reasons, the Df2 code can be remapped to establish a new mapping from virtual page VM to physical page PM. After Df2 is migrated, if Df1 needs to transfer data to Df2, the base address in the data address "base address + offset" used by the output code in Df1 needs to be revised according to the location of Df2—the data flow task stack location used by the new processor core.

[0106] In another preferred embodiment, the migration method of the executable file data flow program further comprises: if the storage space of the physical storage area is less than or equal to a preset threshold, mapping the to-be-migrated data flow node of the virtual storage area on the physical storage area based on the execution sequence of the node program topology graph.

[0107] Because the priority execution sequence is recorded in the node topology information list in the node program topology graph, if the physical memory is insufficient, the address mapping of the "front" node is established in priority according to the depth sorting of the node program topology graph, that is, the priority execution sequence. Therefore, the migration method of the executable file data flow program in the embodiment of the application can flexibly establish the mapping relationship between the virtual page and the real page in combination with the actual situation of the physical storage area, thereby improving the migration efficiency of the data flow node program and improving the system running speed.

[0108] In a specific embodiment, the migration method of the executable file data flow program in the embodiment of the application, based on the node program topology graph, repositions the input address of the to-be-migrated data flow node to the target stack area to which the to-be-migrated data flow node belongs when migrating the to-be-migrated data flow node, without repositioning the output address of the to-be-migrated data flow node, and the output data of the to-be-migrated data flow node is written into the stack area where the subsequent node is located.

[0109] The repositioning of the input address of the to-be-migrated data flow node allows the executable file to take the node as the migration unit, improves the possibility and flexibility of the data flow node migration, and further improves the data flow running speed.

[0110] In a specific embodiment, the setting method of the executable file format based on the data flow program in the embodiment of the application, based on the node program topology graph, the to-be-migrated data flow nodes executed in sequence on the same processor share the same stack when migrating the to-be-migrated data flow nodes.

[0111] In a specific embodiment, the migration method of the executable file data flow program in the embodiment of the application, based on the node program topology graph, the stack frame start addresses of the to-be-migrated data flow nodes executed on the same processor are the same when migrating the to-be-migrated data flow nodes, and the page table address mechanism is used to limit the data memory address access range of the to-be-migrated data flow nodes in the memory space of the node when executing the to-be-migrated data flow nodes.

[0112] The stack frame start addresses of the target nodes executed on the same processor based on the to-be-migrated data flow nodes are the same, which allows the executable file to take the node as the migration unit, improves the possibility and flexibility of the data flow node migration, and further improves the data flow running speed. Because the same stack space is repeatedly used by the data flow task functions on the same processor, the code data can be locked, the page fault rate is reduced, and the computing performance is improved.

[0113] The migration method of the executable file data flow program in the embodiment of the present application, by performing the above steps, based on the node program topology graph in the setting method of the executable file format of the data flow program, relocates based on the starting address of the data flow node to be migrated, and maps and unmapping of virtual and real pages is performed with basis, instead of blindly using LRU algorithm as page replacement algorithm, which can significantly reduce the page fault rate, and further improve the system running speed. Alternatively, the address mapping of the "front" node is preferentially established, which can flexibly establish the mapping relationship between the virtual page and the real page in combination with the actual situation of the physical storage area, and further improve the migration efficiency of the data flow node program and the system running speed.

[0114] Based on the same concept, the embodiment of the present application also discloses a setting device based on the executable file format of the data flow program, as shown in Figure 12 The device comprises the following modules:

[0115] The data flow acquisition module 121 is used to acquire each data flow node of the function code.

[0116] The list creation module 122 is used to create the starting address list and the topology information list of each data flow node, and the topology information list comprises: the total number of each data flow node and / or the topology relationship between each data flow node and / or the execution order of each data flow node and / or the current scheduling execution state of each data flow node during dynamic migration.

[0117] The topology graph creation module 123 is used to create the node program topology graph based on the starting address list and the topology information list of each data flow node.

[0118] The program segment arrangement module 124 is used to arrange the starting address list and the topology information list to the first program segment, and arrange each data flow node to the second program segment.

[0119] The executable file formation module is used to form the target executable file according to the contents of the first program segment and the second program segment, and add the position identification information of the node program topology graph on the target executable file.

[0120] In a specific embodiment, when linking each data flow node, each data flow node is linked into PIC position independent code, and the PIC position independent code is independently formed into a section; when linking each data flow node, the starting point of each data flow node is aligned at the page boundary based on the starting address of each data flow node, thereby forming a segment.

[0121] In a specific embodiment, when linking each data flow node, the address information of the input data of each data flow node is relocated to the target stack area, and the address information of the output data of each data flow node is not relocated.

[0122] Based on the same concept, the embodiment of the present application also discloses a migration device of an executable file data flow program, which comprises the following modules as shown in the figure: Figure 13

[0123] The data flow determination module 131 is configured to determine a data flow node to be migrated in the target executable file.

[0124] The data flow updating module 132 is configured to relocate the starting address of the data flow node to be migrated based on the node program topology graph in the setting method of the executable file format of the data flow program, and migrate the data flow node to be migrated to the current initial address in the target stack area and update the current initial address in the node program topology graph.

[0125] The data flow state acquisition module 133 is configured to acquire the current scheduling execution state of the data flow node to be migrated according to the node program topology graph.

[0126] The mapping relationship execution module 134 is configured to determine whether to map the data flow node to be migrated in the virtual storage area on the physical storage area according to the current scheduling execution state of the data flow node to be migrated.

[0127] In a specific embodiment, the mapping relationship execution module comprises:

[0128] The mapping relationship release sub-module is configured to release the mapping relationship of the data flow node to be migrated in the virtual storage area on the physical storage area if the current scheduling execution state of the data flow node to be migrated is the executed state.

[0129] The mapping relationship locking sub-module is configured to lock the data flow node to be migrated in the virtual storage area on the physical storage area if the current scheduling execution state of the data flow node to be migrated is the executing state.

[0130] The mapping relationship to-be-executed sub-module is configured to not map the data flow node to be migrated in the virtual storage area on the physical storage area if the current scheduling execution state of the data flow node to be migrated is the to-be-executed state.

[0131] In a specific embodiment, the migration device of the executable file data flow program in the embodiment of the present application further comprises a mapping relationship determination module configured to:

[0132] If the storage space of the physical storage area is less than or equal to the preset threshold, the data flow node to be migrated in the virtual storage area is mapped on the physical storage area according to the execution sequence of the node program topology graph.

[0133] ​In one specific implementation, based on the node program topology graph, when migrating a data stream node to be migrated, the input address of the data stream node to be migrated is relocated to its target stack area, without needing to relocate the output address of the data stream node to be migrated, and the output data of the data stream node to be migrated is written to the stack area of ​​the subsequent node.

[0134] In one specific implementation, based on the node program topology graph, when migrating a data stream node to be migrated, the stack frame start address of the data stream node to be migrated executed on the same processor is the same. When executing the data stream node to be migrated, the page table address mechanism is used to restrict the range of data memory address access of the data stream node to be migrated to the memory space of this node.

[0135] This invention also provides a computer device, such as... Figure 14 As shown, the computer device may include a processor 141 and a memory 142, wherein the processor 141 and the memory 142 may be connected via a bus or other means. Figure 14 Taking the example of a connection between China and Israel via a bus.

[0136] Processor 141 may be a central processing unit (CPU). Processor 141 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0137] The memory 142, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 141 executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 142, thereby implementing the executable file data stream program migration method in the above embodiments, or the executable file data stream program migration method.

[0138] The memory 142 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required by a function, and the like. The data storage area can store data created by the processor 141, and the like. In addition, the memory 142 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some embodiments, the memory 142 can optionally include a memory disposed remotely with respect to the processor 141, which can be connected to the processor 141 through a network. Examples of the network include, but are not limited to, a power grid, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0139] The one or more modules are stored in the memory 142 and, when executed by the processor 141, perform the setting method of the executable file format of the data flow program or the migration method of the executable file data flow program in the embodiments shown in the drawings.

[0140] The above computer device specific details can be understood in correspondence with the relevant description and effects of the corresponding embodiments shown in the drawings, which will not be described here again.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage medium can also include a combination of the above-mentioned types of storage.

[0142] Obviously, the above-mentioned embodiments are only examples for clear illustration, and are not limitations on the embodiments. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all embodiments need not and cannot be exhausted. The obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A setting method of an executable file format based on a dataflow program, characterized by, The method comprises the following steps: acquiring each data flow node of a function code; creating a start address list and a topology information list of each data flow node, wherein the topology information list comprises the total number of each data flow node and / or the topology relationship between each data flow node and / or the execution sequence of each data flow node and / or the current scheduling execution state of each data flow node during dynamic migration; creating a node program topology graph based on the start address list and the topology information list of each data flow node; when linking each data flow node, linking each data flow node into PIC position-independent code, and independently segmenting the PIC position-independent code; when linking each data flow node, aligning the start point of each data flow node at a page boundary based on the start address of each data flow node to form a segment; arranging the start address list, the topology information list to a first program segment, and arranging each data flow node to a second program segment; forming a target executable file according to the content of the first program segment and the second program segment, and adding position identification information of the node program topology graph to the target executable file.

2. The method of claim 1, wherein, When linking each data flow node, the address information of the input data of each data flow node is relocated to the target stack area, while the address information of the output data of each data flow node is not relocated.

3. A migration method of an executable file data flow program, characterized by, The method comprises the following steps: determining a data flow node to be migrated in a target executable file; relocating the start address of the data flow node to be migrated to the target stack area based on the node program topology graph in the setting method of the executable file format of the data flow program according to any one of claims 1 to 2, and updating the current start address of the target stack area in the node program topology graph; obtaining the current scheduling running state of the data flow node to be migrated according to the node program topology graph; determining whether to map the data flow node to be migrated in the virtual storage area on the physical storage area according to the current scheduling running state of the data flow node to be migrated.

4. The migration method of an executable file data stream program according to claim 3, wherein, The method of determining whether to map the data flow node to be migrated in the virtual storage area on the physical storage area according to the current scheduling running state of the data flow node to be migrated comprises: if the current scheduling running state of the data flow node to be migrated is an executed state, releasing the mapping relationship between the data flow node to be migrated in the virtual storage area and the physical storage area; if the current scheduling running state of the data flow node to be migrated is an executing state, locking the data flow node to be migrated in the virtual storage area on the physical storage area; if the current scheduling running state of the data flow node to be migrated is a to-be-executed state, not needing to map the data flow node to be migrated in the virtual storage area on the physical storage area.

5. The migration method of an executable file data stream program according to claim 3, wherein, The method further comprises: if the storage space of the physical storage area is less than or equal to a preset threshold, mapping the data flow node to be migrated in the virtual storage area on the physical storage area according to the execution sequence of the node program topology graph.

6. The migration method of an executable file data stream program according to any one of claims 3 to 5, characterized in that, Based on the node program topology, when migrating the to-be-migrated data flow node, the input address of the to-be-migrated data flow node is relocated to the target stack area, and the output address of the to-be-migrated data flow node does not need to be relocated, and the output data of the to-be-migrated data flow node is written into the stack area where the subsequent node is located.

7. The migration method of an executable file data stream program according to any one of claims 3 to 5, characterized by, Based on the node program topology, when migrating the to-be-migrated data flow node, the stack frame start address of the to-be-migrated data flow node executed on the same processor is the same, and when executing the to-be-migrated data flow node, the page table address mechanism is used to limit the data memory address access range of the to-be-migrated data flow node in the memory space of the node.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the computer to execute the setting method of the executable file format of the data flow program according to any one of claims 1 to 2; or the migration method of the executable file data flow program according to any one of claims 3 to 7.

9. A computer device, comprising: Comprise: A memory and a processor, which are connected in communication with each other, and the memory stores computer instructions, and the processor executes the setting method of the executable file format of the data flow program according to any one of claims 1 to 2; or the migration method of the executable file data flow program according to any one of claims 3 to 7 by executing the computer instructions.

Images