A data synchronization method and a data synchronization device

Abstract

The application discloses a data synchronization method and a data synchronization device, and relates to the technical field of computers. The data synchronization method can comprise: selecting target data from to-be-synchronized data for data synchronization after or before a static graph is executed; and executing data synchronization on the target data. The static graph is generated based on a first code segment in a first code, and the first code is a code used for describing a neural network model. The data synchronization method provided in the application can select necessary target data from to-be-synchronized data for data synchronization in the process of executing the code used for describing the neural network model, can reduce the data amount of data to be synchronized in the data synchronization process, reduce the consumption of data synchronization, reduce the time length required for data synchronization, and improve the execution efficiency of model training or reasoning.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data synchronization method and a data synchronization device. Background Technology

[0002] In artificial intelligence (AI) business scenarios, the training and inference processes of neural network models, especially large language models, require a large amount of computation. This computation can utilize static and dynamic graphs. Static graphs refer to predefined computation graphs generated before the computation process is executed. Devices can perform computations based on these graphs, resulting in high efficiency and performance. Dynamic graphs, on the other hand, are computation graphs dynamically constructed during the computation process. Devices can also perform computations based on these graphs, allowing for flexible changes in the syntax of programs used to represent large language models.

[0003] When the computation process uses both static and dynamic graphs, it can be called a hybrid static-dynamic model. This model balances syntactic flexibility and execution performance. In this model, data synchronization is required when converting from an execution dynamic graph to an execution static graph, and vice versa. Because the amount of data to be synchronized is large, the synchronization process is time-consuming, impacting the efficiency of model training or inference. Summary of the Invention

[0004] This application provides a data synchronization method and a data synchronization device, which can reduce data synchronization consumption, reduce data synchronization time, and improve the execution efficiency of model training and inference.

[0005] Firstly, this application provides a data synchronization method, which can be executed by a computing device, or by a chip, chip system, or circuit within the computing device. The data synchronization method may include: the computing device selecting target data from the data to be synchronized before or after executing a static graph, and performing data synchronization on the target data. The static graph is generated based on a first code segment in first code, where the first code is code used to describe a neural network model.

[0006] The data synchronization method provided in this application selects necessary target data from the data to be synchronized during data synchronization, either before or after executing the static graph, during the execution of code used to describe the neural network model. This reduces the amount of data to be synchronized, decreases the consumption of data synchronization, reduces the time required for data synchronization, and improves the execution efficiency of model training or inference.

[0007] In one alternative implementation, the data to be synchronized may include device-side data for pre-inputting static graph execution, and the target data may include data to be used during the execution of the static graph.

[0008] In the above implementation, the target data is the data required during the execution of the static graph. Based on the usage of the input data in the static graph, unused and unnecessary data in the input data to be synchronized can be removed, which can reduce the amount of data to be synchronized before the execution of the static graph.

[0009] In another alternative implementation, the data to be synchronized may include the data obtained by executing the static graph, and the target data may include the data modified or created during the execution of the static graph, and the target data is required when executing the code located after the first code segment in the first code.

[0010] In the above implementation, the target data is the data that is modified or created during the execution of the static graph and needs to be used later. Based on the modification and usage of the static graph output data, the unmodified unnecessary data in the output data to be synchronized can be removed, which can reduce the amount of data to be synchronized after the static graph is executed.

[0011] In one alternative implementation, the target data includes basic data, which may be numbers or tensors.

[0012] In the above implementation method, the target data for data synchronization only contains basic data and does not contain composite data. This can avoid different composite data containing the same basic data and avoid transmitting multiple identical basic data, which can further save the resources consumed by data synchronization and the time spent performing data synchronization.

[0013] In one alternative implementation, the data to be synchronized includes data pre-input from the device side for executing the static graph, and the target data includes data to be used during the execution of the static graph. For the data to be synchronized prior to the execution of the static graph, the computing device can select the target data from the data to be synchronized according to first indication information used to indicate the target data.

[0014] In one optional implementation, the first indication information includes parameter identifiers of the target data. The first indication information can be generated as follows: the computing device constructs a first parameter set and an input parameter set to be synchronized; the first parameter set includes parameter identifiers of the basic data required during the execution of the static graph; the input parameter set to be synchronized includes parameter identifiers of each piece of data in the data to be synchronized. Based on the first parameter set, the computing device can remove parameter identifiers of unused data from the input parameter set to be synchronized to obtain a first target set; the first target set includes parameter identifiers of the target data; parameter identifiers of unused data do not belong to the first parameter set; the computing device can generate the first indication information based on the parameter identifiers of the target data in the first target set.

[0015] In the above implementation method, by constructing a first parameter set and an input parameter set to be synchronized, the first indication information used to indicate the target data can be quickly generated.

[0016] In one alternative implementation, the data to be synchronized includes composite data and basic data, wherein the composite data is a dataset containing multiple basic data; the parameter identifiers of the composite data in the input parameter set to be synchronized are parameter matrices; when the computing device removes parameter identifiers of unused data from the input parameter set to be synchronized, it can split the parameter matrix of the composite data in the input parameter set to be synchronized into parameter identifiers of the basic data, and remove the parameter identifiers of the basic data that do not belong to the first parameter set from the input parameter set to be synchronized.

[0017] In the above implementation, the parameter matrix of the composite data in the input parameter set to be synchronized is split, so that the first target set contains only the parameter identifiers of basic data, and thus the target data can contain only basic data.

[0018] In one alternative implementation, the data to be synchronized may include data obtained by executing a static graph, and the target data may include data modified or created during the execution of the static graph, and the target data is required when executing the code following the first code segment in the first code. For the data to be synchronized after executing the static graph, the computing device may select the target data from the data to be synchronized according to second indication information used to indicate the target data.

[0019] In one alternative implementation, the second indication information includes parameter identifiers of the target data. The second indication information can be generated as follows: the computing device constructs a second parameter set and outputs a set of parameters to be synchronized. The second parameter set includes parameter identifiers of basic data that are modified or created during the execution of the static graph and are required when executing code located after the first code segment in the first code. The output set of parameters to be synchronized contains parameter identifiers of each piece of data in the data to be synchronized. Based on the second parameter set, the computing device can remove the parameter identifiers of unmodified data from the output set of parameters to be synchronized to obtain a second target set. The second target set contains parameter identifiers of the target data; parameter identifiers of unmodified data are not included in the second parameter set. The computing device can generate the second indication information based on the parameter identifiers of the target data in the second target set.

[0020] In the above implementation, by constructing a second parameter set and outputting a parameter set to be synchronized, a second indication information for indicating target data can be quickly generated.

[0021] In one optional implementation, the data to be synchronized includes composite data and basic data. The composite data is a dataset containing multiple basic data sets, and the parameter identifiers of the composite data in the output parameter set to be synchronized are represented as a parameter matrix. When removing parameter identifiers of unmodified data from the output parameter set to be synchronized, the computing device can split the parameter matrix of the composite data in the output parameter set to be synchronized into parameter identifiers of the basic data, and remove parameter identifiers of basic data that do not belong to the second parameter set from the output parameter set to be synchronized.

[0022] In the above implementation, the parameter matrix of the composite data in the output parameter set to be synchronized is split, so that the second target set contains only the parameter identifiers of the basic data, and thus the target data can contain only the basic data.

[0023] In an alternative implementation, the computing device may also execute a second code segment in the first code in a dynamic graph mode, wherein the second code segment precedes the first code segment.

[0024] In an alternative implementation, the computing device may also execute a third code segment in the first code in a dynamic graph mode, wherein the third code segment is located after the first code segment.

[0025] In the above implementation, during the execution of the first code, a mixed static and dynamic mode is adopted, which can balance syntactic flexibility and execution efficiency.

[0026] Secondly, this application provides a data synchronization device that can be applied to a computing device, and the data synchronization device may include:

[0027] The data filtering unit is used to select target data from the data to be synchronized before or after the execution of the static graph; the static graph is generated based on the first code segment in the first code; the first code is code used to describe the neural network model;

[0028] The data synchronization unit is used to perform data synchronization on target data.

[0029] In one alternative implementation, the data to be synchronized includes device-side data for pre-input static graph execution, and the target data includes data needed during static graph execution; or,

[0030] The data to be synchronized includes the data obtained from executing the static graph, and the target data includes the data that was modified or created during the execution of the static graph. The target data is required when executing the code that is located after the first code segment in the first code.

[0031] In one alternative implementation, the target data includes basic data, which may be numbers or tensors.

[0032] In one alternative implementation, the data to be synchronized includes device-side data pre-input for executing the static diagram, and the target data includes data to be used during the execution of the static diagram. The data filtering unit can specifically be used to: select target data from the data to be synchronized before executing the static diagram, according to first indication information used to indicate the target data.

[0033] In one alternative implementation, the data to be synchronized may include data obtained from executing the static graph, and the target data may include data modified or created during the execution of the static graph, and the target data is required when executing the code following the first code segment in the first code. The data filtering unit can be specifically used to: select target data from the data to be synchronized according to second indication information used to indicate the target data, based on the data to be synchronized after executing the static graph.

[0034] Thirdly, this application provides a data synchronization device, which may include a processor and a memory; the memory stores computer programs or instructions; the processor is used to execute the computer programs or instructions stored in the memory so that the chip performs any of the data synchronization methods provided in the first aspect above.

[0035] Fourthly, this application provides a computing device, including a host and a device side;

[0036] The host is used to select first target data from the first data to be synchronized before executing the static graph, and to perform data synchronization on the first target data; the static graph is generated based on the first code segment in the first code; the first code is code used to describe the neural network model;

[0037] On the device side, it is used to execute the static graph, and for the second data to be synchronized after executing the static graph, it selects the second target data from the second data to be synchronized, and performs data synchronization for the second target data.

[0038] In one alternative implementation, the first data to be synchronized includes data from the device side that is pre-inputting the static graph, and the first target data includes data that needs to be used during the execution of the static graph.

[0039] The second data to be synchronized includes the data obtained from executing the static graph, and the second target data includes the data modified or created during the execution of the static graph. The second target data is required when executing the code located after the first code segment in the first code.

[0040] In one alternative implementation, the first target data includes basic data, which may be numbers or tensors. The second target data also includes basic data.

[0041] In one alternative implementation, the host is further configured to generate first instruction information and second instruction information; the first instruction information contains parameter identifiers of the first target data; and the second instruction information contains parameter identifiers of the second target data.

[0042] In one alternative implementation, the first indication information can be generated as follows: The host constructs a first parameter set and an input parameter set to be synchronized; the first parameter set contains parameter identifiers of the basic data required during the execution of the static graph; the input parameter set to be synchronized contains parameter identifiers of each piece of data in the data to be synchronized. Based on the first parameter set, the host removes the parameter identifiers of unused data from the input parameter set to be synchronized to obtain a first target set; the first target set contains the parameter identifiers of the target data; the parameter identifiers of unused data do not belong to the first parameter set; the host generates the first indication information based on the parameter identifiers of the target data in the first target set.

[0043] In one optional implementation, the data to be synchronized includes composite data and basic data, wherein the composite data is a dataset containing multiple basic data; the parameter identifiers of the composite data in the input parameter set to be synchronized are parameter matrices; when the host removes parameter identifiers of unused data from the input parameter set to be synchronized, it can split the parameter matrix of the composite data in the input parameter set to be synchronized into parameter identifiers of basic data, and remove parameter identifiers of basic data that do not belong to the first parameter set from the input parameter set to be synchronized.

[0044] In one alternative implementation, the second indication information can be generated as follows: The host constructs a second parameter set and an output parameter set to be synchronized. The second parameter set contains parameter identifiers of basic data that are modified or created during the execution of the static graph and are required when executing code located after the first code segment in the first code. The output parameter set to be synchronized contains parameter identifiers of each piece of data in the data to be synchronized. Based on the second parameter set, the host can remove the parameter identifiers of unmodified data from the output parameter set to be synchronized to obtain a second target set. The second target set contains parameter identifiers of the target data; parameter identifiers of unmodified data are not included in the second parameter set. The host can generate the second indication information based on the parameter identifiers of the target data in the second target set.

[0045] In one optional implementation, the data to be synchronized includes composite data and basic data. The composite data is a dataset containing multiple basic data sets, and the parameter identifiers of the composite data in the output parameter set to be synchronized are represented as a parameter matrix. When the host removes the parameter identifiers of unmodified data from the output parameter set to be synchronized, it can split the parameter matrix of the composite data in the output parameter set to be synchronized into parameter identifiers of the basic data, and remove the parameter identifiers of basic data that do not belong to the second parameter set from the output parameter set to be synchronized.

[0046] In one alternative implementation, the host is configured to select first target data from first data to be synchronized according to first instruction information; the device is configured to select second target data from second data to be synchronized according to second instruction information.

[0047] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which are used to cause a computer to perform any of the data synchronization methods provided in the first aspect.

[0048] In a sixth aspect, embodiments of this application provide a computer program product comprising computer-executable instructions, the computer-executable instructions being used to cause a computer to execute any of the data synchronization methods provided in the first aspect.

[0049] The technical effects that can be achieved by any of the second to sixth aspects mentioned above can be referred to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a schematic diagram illustrating one application scenario of an embodiment of this application;

[0051] Figure 2 A schematic diagram of static / dynamic hybrid execution code provided in an embodiment of this application;

[0052] Figure 3 A flowchart illustrating a data synchronization method provided in an embodiment of this application;

[0053] Figure 4 A flowchart illustrating another data synchronization method provided in this application embodiment;

[0054] Figure 5 A schematic diagram illustrating a code execution process provided in an embodiment of this application;

[0055] Figure 6 A schematic diagram of a data synchronization optimization device provided in an embodiment of this application;

[0056] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0057] Figure 8 This is a schematic diagram of a data synchronization device provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application.

[0059] Before introducing the specific solutions provided in the embodiments of this application, some terms used in this application will be explained to facilitate understanding by those skilled in the art, but the terms used in this application are not limited.

[0060] (1) Computational graph: In the training or inference process of a neural network model, a computational graph is a data structure used to represent the computation process. A computational graph contains two elements—nodes and edges. A computational graph can include multiple nodes and multiple edges. Nodes represent data, and edges represent data flow, that is, the data operation process, which is represented as operators in the code used to describe the neural network model.

[0061] (2) Static graph: refers to a computation graph that is generated in advance before the computation process is executed, or it can represent a computation graph executed in a static graph manner. For example, a static graph can be generated based on any code segment in the overall code used to describe the neural network model.

[0062] (3) Dynamic graph: refers to a computation graph that is dynamically constructed during the computation process, and sometimes it also means that the computation graph is executed in a dynamic graph manner. For example, a dynamic graph can be a computation graph that is dynamically constructed based on any code segment in the code during the execution of the code used to describe the neural network model.

[0063] (4) Basic data: including numbers, tensors and other data units that cannot be further divided in the computation graph.

[0064] (5) Composite data: refers to a dataset that includes multiple basic data or a dataset composed of basic data. Composite data can be in the form of a list or a tuple, etc.

[0065] In this application embodiment, "multiple" refers to two or more. Therefore, in this application embodiment, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, such as one, two, or more. For example, "including at least one" means including one, two, or more, and it does not limit which ones are included. For example, including at least one of A, B, and C, then it could include A, B, C, A and B, A and C, B and C, or A and B and C. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0066] Unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects, and are not used to limit the order, sequence, priority or importance of multiple objects.

[0067] The data synchronization method provided in this application embodiment can be applied to... Figure 1 The AI ​​system shown can be used for training or inference of neural network models. For example... Figure 1As shown, the AI ​​system may include a terminal 10 and a computing device 20. Users can input data into the computing device 20 through the terminal 10, and the computing device 20 can perform training or inference processes on a neural network model based on the received data. The terminal 10 can also be called a client device, and may be, but is not limited to, a personal computer, server, mobile phone, tablet computer, or smart car. The computing device 20 may be a server, such as a cloud server. The terminal 10 and the computing device 20 can be connected via wired or wireless means; for example, they can be connected through a communication network.

[0068] The computing device 20 may include a host 21 and a device 22. The host 21 receives input data and AI tasks sent by the terminal 10, processes the data, and sends the output results back to the terminal 10. The host 21 includes a processor 211, a memory 212, and a communication interface 213. The communication interface 213 is used to communicate with devices located outside the host 21. For example, the host 21 receives input data and AI tasks from the terminal 10 via the communication interface 213. The host 21 can collaborate with the device 22 to process the input data, obtain the processing results, and then output the processing results to the terminal 10 via the communication interface 213.

[0069] Processor 211 is the core of the host 21's computation and control. Processor 211 can be a central processing unit (CPU) or other specific integrated circuits. Processor 211 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In practical applications, the host 21 can have multiple processors 211. Each processor 211 includes one or more processor cores. An operating system and other software programs are installed in the processor 211, enabling it to access memory 212 and various peripherals.

[0070] The processor 211 is connected to the memory 212 via a double data rate (DDR) bus or other types of bus. The memory 212 is the main memory of the host 21, and it can be used to store various running software in the operating system, input data received from the terminal 10, and processing results to be sent to the terminal 10 in the future. To improve the access speed of the processor 211, the memory 212 needs to have the advantage of high access speed; dynamic random access memory (DRAM) can be used as the memory 212. Besides DRAM, the memory 212 can also be other random access memories, such as static random access memory (SRAM). Alternatively, the memory 212 can also be a read-only memory (ROM). For example, a read-only memory could be a programmable read-only memory (PROM) or an erasable programmable read-only memory (EPROM). This application embodiment does not limit the number or type of memory 212.

[0071] Device 22 assists host 21 in executing AI tasks, accelerating the execution process of AI tasks. For example, host 21 can send the computation graph and data generated during the execution of AI tasks to device 22. Device 22 executes the computation graph based on the received data and sends the data obtained after executing the computation graph back to host 21.

[0072] like Figure 1As shown, the device side 22 may include a processor 221 and a memory 222. The processor 221 may be, but is not limited to, a neural network processing unit (NPU), a graphics processing unit (GPU), a smart network interface card (NIC), or a data processing unit (DPU). The processor 221 is connected to the memory 222. The memory 222 is the memory of the device side 22, and it can be used to store data input from the host 21, as well as data generated during the execution of the computation graph and data to be sent to the host 21. In some embodiments, the device side 22 may be in the form of a device card, which can be directly inserted into a slot on the motherboard of the host 21 and exchange data with the host 21 via a bus 214. The bus 214 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or a bus supporting compute express link (CXL), universal serial bus (USB) protocol, or other protocols.

[0073] In some embodiments, for persistent data storage, the AI ​​system also includes a data storage system 30. The data storage system 30 can be located outside the host 21 and exchanges data with the host 21 via a network. Alternatively, the data storage system 30 can be located inside the host and exchanges data with the processor 211 via a bus 214. In this case, the data storage system 30 can function as a hard disk.

[0074] It should be noted that, Figure 1 The application scenario shown is merely an exemplary illustration of the application scenarios in this application. The drawing update method provided in the embodiments of this application is not limited to... Figure 1 The application scenarios shown can also be applied to other application scenarios.

[0075] During the execution of AI tasks, computing device 20 can acquire code used to describe a neural network model and input data sent by terminal 10. Computing device 20 can generate dynamic and static graphs for different code segments in the code used to describe the neural network model, and execute the code used to describe the neural network model by executing the static and dynamic graphs. For example, as... Figure 2As shown, after acquiring the code used to describe the neural network model, the computing device 20 can execute code segment A in dynamic graph mode to generate dynamic graph A. For code segment B, it can compile to generate static graph B and execute static graph B. Then, it can execute code segment C in dynamic graph mode to generate dynamic graph C. Then, for code segment D, it can compile to generate static graph D, and so on, until the AI ​​task is completed. By sequentially executing dynamic graph A, static graph B, dynamic graph C, and static graph D, the AI ​​task can be completed. Data synchronization is required when switching between dynamic and static graphs. For example, the host 21 can compile the code segment to generate a static graph, and the device 22 can execute the static graph. Before executing the static graph, the host 21 can synchronize the data before executing the static graph to the device 22. After executing the static graph, the device 22 can synchronize the processed data to the host 21.

[0076] Currently, when synchronizing data, the data to be synchronized includes composite data and basic data, and the amount of data to be synchronized is very large. Therefore, data synchronization takes a long time and affects the execution efficiency of model training or inference.

[0077] Based on this, embodiments of this application provide a data synchronization method, which can be implemented by... Figure 1 The computing device performs the synchronization. For data to be synchronized before or after the execution of the static graph, the computing device can select target data from the data to be synchronized and perform data synchronization on the target data. The static graph is generated based on a first code segment in the first code, which can be code used to describe the neural network model. During data synchronization, removing some data from the data to be synchronized and selecting only the necessary target data can reduce the amount of data involved in data synchronization, reduce the cost of data synchronization, shorten the time required for data synchronization, and improve the execution efficiency of model training or inference.

[0078] Figure 3 An exemplary flowchart of a data synchronization method provided in an embodiment of this application is shown. This data synchronization method can be... Figure 1 The computing device in the process executes the commands. For example... Figure 3 As shown, the data synchronization method may include the following steps:

[0079] S301, for the first data to be synchronized before executing the static graph, select the first target data from the first data to be synchronized.

[0080] Taking the inference process of executing a neural network model as an example, the host in the computing device obtains first code, which is computer program code used to describe the neural network model. Since the first code is very large, the host in the computing device can generate different computational graphs based on different code segments within the first code. A code segment can refer to a portion of the program code in the first code. For example, the host in the computing device can compile the first code segment in the first code and generate a static graph based on it. The host in the computing device can transmit the static graph to the device side, where the device side executes the static graph to execute the first code segment in the first code, obtaining the execution result of the first code segment.

[0081] The host in the computing device can also receive input data sent by the user's terminal. According to the requirements of the calculation process for executing the first code, it organizes the input data to obtain the first data to be synchronized. The first data to be synchronized includes pre-input data from the device side used for executing the static graph. The first data to be synchronized may contain composite data and basic data. Because the first data to be synchronized contains some data that is not used during the execution of the static graph, the data volume is relatively large. To reduce the amount of data to be synchronized, the host in the computing device can select the first target data to be used during the execution of the static graph from the first data to be synchronized. The first target data can be part or all of the data in the first data to be synchronized, and the data volume of the first target data is less than or equal to the data volume of the first data to be synchronized.

[0082] S302, Perform data synchronization for the first target data.

[0083] After selecting the first target data from the first data to be synchronized, data synchronization can be performed only on the first target data. This process may include: the host in the computing device transmitting the selected first target data to the device side. The first target data corresponds to a node in the static graph, and the device side can perform static graph execution based on the first target data.

[0084] S303, for the second data to be synchronized after executing the static graph, select the second target data from the second data to be synchronized.

[0085] The second set of data to be synchronized includes the data obtained by the device after executing the static diagram. This second set of data can contain composite data and basic data. Because some data in the second set of data to be synchronized was not modified during the execution of the static diagram and is no longer used subsequently, the data volume is relatively large. To reduce the amount of data to be synchronized, a second target data can be selected from the second set of data to be synchronized. The second target data can include data modified or created during the execution of the static diagram, and it is required when executing the code following the first code segment in the first code. The second target data is part or all of the data in the second set of data to be synchronized, and the size of the second target data is less than or equal to the size of the second set of data to be synchronized.

[0086] S304, Perform data synchronization for the second target data.

[0087] After selecting the second target data from the second data to be synchronized, data synchronization can be performed only on the second target data. This process may include: the device side in the computing device transmitting the second target data to the host in the computing device so that the computing device can continue to execute the code in the first code that is located after the first code segment based on the second target data.

[0088] In the data synchronization process, this application embodiment selects only the necessary target data for data synchronization, which can reduce the amount of data in data synchronization and reduce the time required for data synchronization.

[0089] To facilitate understanding, the technical solution of this application will be described in detail below through specific embodiments.

[0090] In some embodiments, after obtaining first code for describing a neural network model, the process by which the computing device performs model inference based on the first code may include, for example: Figure 4 The following steps are shown:

[0091] S401 executes the second code segment in the first code through dynamic graph mode and obtains the first data to be synchronized.

[0092] like Figure 5 As shown, assume the first code segment includes a second code segment, a first code segment, and a third code segment, where the second code segment precedes the first code segment, and the first code segment precedes the third code segment. The second and third code segments contain statements that cannot be compiled into a static graph.

[0093] The computing device can execute the first code through a Python virtual machine, and the Python virtual machine can execute the second code segment within the first code through a dynamic graph mode. For example, the host and device side of the computing device can collaboratively execute the second code segment within the first code. The host in the computing device determines the operator to be executed in the second code segment, transmits the operator and the data required to execute it to the device side, and the device side executes the operator based on the received data, returning the result to the host and storing it in the host's memory. Then, the process can continue to execute the next operator in the second code segment until the second code segment is completed, obtaining the first data to be synchronized. The first data to be synchronized is stored in the host's memory and includes the data required to execute the code following the second code segment in the first code. The process of executing the second code segment within the first code through a dynamic graph mode can be understood as executing the first dynamic graph.

[0094] S402, Select the first target data from the first data to be synchronized, and perform data synchronization for the first target data.

[0095] The host computer of a computing device contains a compiler, which can be located within the host's processor and is used to compile code. For example... Figure 5 As shown, after executing the second code segment in the first code, the compiler can compile the next code segment in the first code, i.e., the first code segment, to obtain the original static graph. The compiler is equipped with a data synchronization optimization device, which can analyze the first code segment and the code following the first code segment to determine the input preprocessing strategy when data synchronization is performed before data is input into the static graph, and the output postprocessing strategy when data synchronization is performed after data is output from the static graph. The original static graph is then modified to obtain the modified static graph.

[0096] For example, such as Figure 6 As shown, the data synchronization optimization device 600 may include a computation graph analysis module 610 and a strategy generation module 620. The computation graph analysis module 610 may include a first analysis submodule and a second analysis submodule. The first analysis submodule is used to analyze the usage of input data, which refers to the data input to the first code segment. The second analysis submodule is used to analyze the modification of output data, which refers to the data output by the first code segment. The strategy generation module 620 may include a first generation submodule and a second generation submodule.

[0097] The first generation submodule generates an input preprocessing strategy based on the usage of the input data. This strategy indicates which data should be selected as the first target data from the first data to be synchronized before executing the first code segment. The input preprocessing strategy can also be called the first indication information, which includes parameter identifiers for the first target data. For example, the first data to be synchronized may contain composite data and basic data; the composite data is a dataset including multiple basic data sets. The compiler can construct a first parameter set based on the parameters required during the execution of the static graph, i.e., the parameter identifiers of the basic data. The compiler can also construct an input parameter set to be synchronized based on the input parameters of the first and third code segments. This input parameter set contains the parameter identifiers of each data item in the first data to be synchronized. Since the first data to be synchronized contains both composite and basic data, correspondingly, the input parameter set to be synchronized contains the parameter identifiers of both the composite and basic data. The parameter identifiers of the composite data can be a parameter matrix, which includes the parameter identifiers of multiple basic data sets. The compiler can split the parameter matrix of the composite data in the input parameter set to be synchronized into parameter identifiers of basic data. It then removes the parameter identifiers of unused data (i.e., parameter identifiers of basic data that do not belong to the first parameter set) from the input parameter set to be synchronized, obtaining the first target set. The first target set contains the parameter identifiers of the first target data. Based on the parameter identifiers of the first target data in the first target set, an input preprocessing strategy is generated.

[0098] For example, regarding the parameter identifier of the first basic data in the input parameter set to be synchronized, if the parameter identifier of the first basic data does not belong to the first parameter set, then the parameter identifier of the first basic data is removed from the input parameter set to be synchronized; if the parameter identifier of the first basic data belongs to the first parameter set, then the parameter identifier of the first basic data is removed from the first parameter set, resulting in a new first parameter set. Similarly, regarding the parameter identifier of the second basic data in the input parameter set to be synchronized, if the parameter identifier of the second basic data does not belong to the current first parameter set, then the parameter identifier of the second basic data is removed from the input parameter set to be synchronized; if the parameter identifier of the second basic data belongs to the current first parameter set, then the parameter identifier of the second basic data is removed from the current first parameter set, resulting in a new first parameter set. This process is repeated for each parameter identifier of each basic data in the input parameter set to be synchronized.

[0099] For the parameter matrix of the first composite data in the input parameter set to be synchronized, the parameter matrix of the first composite data is split to obtain a first basic data set. The first basic data set includes parameter identifiers of multiple basic data obtained from splitting the parameter matrix of the first composite data. If none of the parameter identifiers of all basic data in the first basic data set belong to the current first parameter set, the parameter matrix of the first composite data is removed from the input parameter set to be synchronized. If all the parameter identifiers of the basic data in the first basic data set belong to the current first parameter set, the parameter identifiers of multiple basic data in the first basic data set are removed from the current first parameter set to obtain a new first parameter set. If some of the parameter identifiers of the basic data in the first basic data set belong to the current first parameter set, the parameter identifiers of these basic data are removed from the current first parameter set to obtain a new first parameter set. The parameter matrix of the first composite data is then removed from the input parameter set to be synchronized, and the parameter identifiers of the basic data belonging to the first parameter set are added. In some embodiments, when the proportion of parameter identifiers of basic data that do not belong to the first parameter set in the first basic data set reaches a first proportion threshold, the steps of removing the parameter matrix of the first composite data from the input parameter set to be synchronized and adding the parameter identifiers of the basic data belonging to the first parameter set can be performed. If the proportion of parameter identifiers that do not belong to the basic data of the first parameter set in the first basic data set is less than the first proportion threshold, the resource consumed by removing the parameter matrix of the first composite data and adding the basic data belonging to the first parameter set from the input parameter set to be synchronized is greater than the resource consumed by directly synchronizing the first composite data. Therefore, the input parameter set to be synchronized can remain unchanged without updating.

[0100] For the parameter matrix of the second composite data in the input parameter set to be synchronized, the parameter matrix of the second composite data is split to obtain a second basic data set. The second basic data set includes parameter identifiers of multiple basic data obtained from splitting the second composite data. If none of the parameter identifiers of all basic data in the second basic data set belong to the current first parameter set, the parameter matrix of the second composite data is removed from the input parameter set to be synchronized. If all the parameter identifiers of the basic data in the second basic data set belong to the current first parameter set, the parameter identifiers of multiple basic data in the second basic data set are removed from the current first parameter set to obtain a new first parameter set. If some of the parameter identifiers of the basic data in the second basic data set belong to the current first parameter set, the parameter identifiers of these basic data are removed from the current first parameter set to obtain a new first parameter set. The parameter matrix of the second composite data is then removed from the input parameter set to be synchronized, and the parameter identifiers of the basic data belonging to the first parameter set are added. In some embodiments, when the proportion of parameter identifiers of basic data that do not belong to the first parameter set in the second basic data set reaches a first proportion threshold, the steps of removing the parameter matrix of the second composite data from the input parameter set to be synchronized and adding the parameter identifiers of the basic data belonging to the first parameter set can be performed. If the proportion of parameter identifiers that do not belong to the first parameter set in the second basic data set is less than a first proportion threshold, removing the parameter matrix of the second composite data and adding some basic data belonging to the first parameter set from the input parameter set to be synchronized consumes more resources than directly synchronizing the second composite data. Therefore, the input parameter set to be synchronized can remain unchanged without updating it. The above steps are then performed sequentially on the parameter matrix of each composite data in the input parameter set to be synchronized, ultimately yielding the first target set, which contains only parameter identifiers of the basic data. Based on the parameter identifiers of the first target data in the first target set, an input preprocessing strategy is generated.

[0101] The second generation submodule generates an output post-processing strategy based on the modifications to the output data. This strategy indicates which data to select as the second target data from the second set of data to be synchronized after the execution of the first code segment. The output post-processing strategy can also be called second indication information, which includes parameter identifiers for the second target data. For example, the second set of data to be synchronized may contain composite data and basic data; composite data is a dataset including multiple basic data sets. The compiler can determine, based on the first code segment in the first code and the code following the first code segment, the parameters modified or created during the execution of the first code segment, and the parameters required when executing the code following the first code segment—that is, the parameter identifiers of the basic data. The compiler can construct a second parameter set based on the parameter identifiers of the basic data modified or created during the execution of the first code segment and required when executing the code following the first code segment. The compiler can also construct an output parameter set to be synchronized based on the output parameters of the first code segment. This output parameter set contains the parameter identifiers of each data item in the second data set to be synchronized. Since the second data set to be synchronized contains composite data and basic data, correspondingly, the output parameter set to be synchronized contains the parameter identifiers of the composite data and the parameter identifiers of the basic data. The parameter identifiers of the composite data can be a parameter matrix, which includes the parameter identifiers of multiple basic data items. The compiler can split the parameter matrix of the composite data in the output parameter set to be synchronized into the parameter identifiers of the basic data, and remove the parameter identifiers of unmodified data (i.e., the parameter identifiers of basic data that do not belong to the second parameter set) from the output parameter set to be synchronized, obtaining the second target set. The second target set contains the parameter identifiers of the second target data. Based on the parameter identifiers of the second target data in the second target set, an output post-processing strategy is generated.

[0102] For example, regarding the parameter identifier of the third basic data in the output parameter set to be synchronized, if the parameter identifier of the third basic data does not belong to the second parameter set, then the parameter identifier of the third basic data is removed from the output parameter set to be synchronized; if the parameter identifier of the third basic data belongs to the second parameter set, then the parameter identifier of the third basic data is removed from the second parameter set, resulting in a new second parameter set. Similarly, regarding the parameter identifier of the fourth basic data in the output parameter set to be synchronized, if the parameter identifier of the fourth basic data does not belong to the current second parameter set, then the parameter identifier of the fourth basic data is removed from the output parameter set to be synchronized; if the parameter identifier of the fourth basic data belongs to the current second parameter set, then the parameter identifier of the fourth basic data is removed from the current second parameter set, resulting in a new second parameter set. This process is repeated for each basic data in the output parameter set to be synchronized.

[0103] For the parameter matrix of the third composite data in the output parameter set to be synchronized, the parameter matrix of the third composite data is split to obtain a third basic data set. The third basic data set includes parameter identifiers of multiple basic data obtained from splitting the parameter matrix of the third composite data. If none of the parameter identifiers of all basic data in the third basic data set belong to the current second parameter set, the parameter matrix of the third composite data is removed from the output parameter set to be synchronized. If all the parameter identifiers of the basic data in the third basic data set belong to the current second parameter set, the parameter identifiers of multiple basic data in the third basic data set are removed from the current second parameter set to obtain a new second parameter set. If some of the parameter identifiers of the basic data in the third basic data set belong to the current second parameter set, the parameter identifiers of these basic data are removed from the current second parameter set to obtain a new second parameter set. The parameter matrix of the third composite data is then removed from the output parameter set to be synchronized, and the parameter identifiers of the basic data belonging to the second parameter set are added. In some embodiments, when the proportion of parameter identifiers of basic data that do not belong to the second parameter set in the third basic data set reaches a second proportion threshold, the step of removing the parameter matrix of the third composite data from the output parameter set to be synchronized and adding the parameter identifiers of the basic data belonging to the second parameter set can be performed. If the proportion of parameter identifiers that do not belong to the second parameter set in the third basic data set is less than the second proportion threshold, removing the parameter matrix of the third composite data and adding some basic data belonging to the second parameter set from the output parameter set to be synchronized consumes more resources than directly synchronizing the third composite data. Therefore, the output parameter set to be synchronized can remain unchanged without updating it. The second proportion threshold and the first proportion threshold can be the same or different.

[0104] For the parameter matrix of the fourth composite data in the output parameter set to be synchronized, the parameter matrix of the fourth composite data is split to obtain a fourth basic data set. The fourth basic data set includes parameter identifiers of multiple basic data obtained from splitting the parameter matrix of the fourth composite data. If none of the parameter identifiers of all basic data in the fourth basic data set belong to the current second parameter set, the parameter matrix of the fourth composite data is removed from the output parameter set to be synchronized. If all the parameter identifiers of the basic data in the fourth basic data set belong to the current second parameter set, the parameter identifiers of multiple basic data in the fourth basic data set are removed from the current second parameter set to obtain a new second parameter set. If some of the parameter identifiers of the basic data in the fourth basic data set belong to the current second parameter set, the parameter identifiers of these basic data are removed from the current second parameter set to obtain a new second parameter set. The parameter matrix of the fourth composite data is then removed from the output parameter set to be synchronized, and the parameter identifiers of the basic data belonging to the second parameter set are added. In some embodiments, when the proportion of parameter identifiers of basic data that do not belong to the second parameter set in the fourth basic data set reaches a second proportion threshold, the step of removing the parameter matrix of the fourth composite data from the output parameter set to be synchronized and adding the parameter identifiers of the basic data belonging to the second parameter set can be performed. If the proportion of parameter identifiers of basic data that do not belong to the second parameter set in the fourth basic data set is less than the second proportion threshold, removing the parameter matrix of the fourth composite data and adding some basic data belonging to the second parameter set from the output parameter set to be synchronized consumes more resources than directly synchronizing the fourth composite data. Therefore, the output parameter set to be synchronized can remain unchanged without updating it. The above steps are performed sequentially for each composite data in the output parameter set to be synchronized, ultimately yielding the second target set, which contains only parameter identifiers of basic data. Based on the parameter identifiers of the second target data in the second target set, an output post-processing strategy is generated.

[0105] The first generation submodule is also used to modify the original static graph based on the first target data determined by the input preprocessing strategy. For example, if a node in the original static graph indicates that data corresponding to a parameter identifier is read from composite data, but this composite data is not included in the first target data, then the node can be modified to directly read the basic data corresponding to the parameter identifier. By modifying the original static graph, a modified static graph can be obtained. The modified static graph is the actual static graph used in execution, and will be referred to as the static graph in the following text.

[0106] The following examples illustrate the process of generating input preprocessing strategies and output postprocessing strategies.

[0107] In one specific embodiment, the process of generating the input preprocessing strategy can be as follows: Based on the parameter identifiers of the basic data required during the execution of the first code segment, a first parameter set is constructed; based on the first parameter set, parameter identifiers of unused data are removed from the input parameter set to be synchronized, leaving only the parameter identifiers of the first target data. The parameter identifiers of unused data do not belong to the first parameter set.

[0108] For example, the encoder can collect parameter identifiers of basic data read from the static graph but not defined in the graph, forming a set R. Input parameters in set R require corresponding data to be input from outside the static graph, rather than being assigned values ​​within the static graph. Set I is formed by collecting parameter identifiers of basic data from the static graph. A set M is defined as the intersection of set R and set I, which can be represented as M = R ∩ I. Set M can be called the minimal input set. The parameter identifiers in set M are all parameter identifiers of basic data used during the execution of the static graph; therefore, set M is the first parameter set mentioned above.

[0109] The encoder can collect the input parameter list of the static graph to form a set IN. The set IN is the set of input parameters to be synchronized mentioned above, which is used to represent the first data to be synchronized. The set IN can include the parameter matrix of the composite data and the parameter identifier of the basic data.

[0110] For each element i in the set IN in turn Element i can be a parameter matrix of composite data or a parameter identifier of basic data, and the following operations are performed:

[0111] If element i is a parameter identifier of basic data: if element i belongs to the current set M (i∈M), then remove element i from the current set M to obtain a new set M, which can be represented as M = M – {i}; if element i does not belong to the current set M... Then, removing element i from the current set IN will result in a new set IN, which can be represented as IN = IN – {i}.

[0112] If element i is a parameter matrix of composite data, then construct P based on i, where P is a set of parameter identifiers for the multiple basic data obtained by splitting the composite data i. If P belongs to the current set M. The new set M is obtained by removing P from the current set M, which can be expressed as: M = M – P; if P does not belong to the current set M. Then, remove the parameter identifiers of basic data belonging to P from the current set M to obtain a new set M. Similarly, remove element i from the current set IN and add parameter identifiers of basic data belonging to both P and set M to obtain a new set IN, which can be represented as: IN = IN – {i} + (P∩M). In some embodiments, the step of removing element i from the current set IN and adding parameter identifiers of basic data belonging to both P and set M can be performed when the proportion of basic data not belonging to set M in P reaches a first proportion threshold. If the proportion of basic data not belonging to set M in P is less than the first proportion threshold, and the resources (e.g., time) consumed by removing element i from set IN and determining P∩M are greater than the resources consumed by directly synchronizing P, then IN does not need to be updated; that is, IN can remain unchanged.

[0113] The encoder can perform the above operation on each element i in the set IN in sequence. The final set IN only includes the parameter identifiers of the basic data. The parameter identifiers in the set IN are the parameter identifiers of the first target data.

[0114] In one specific embodiment, the process of generating the output post-processing strategy can be as follows: Based on the parameter identifiers of basic data that are modified or created during the execution of the first code segment and are required when executing code located after the first code segment, a second parameter set is constructed. Based on the second parameter set, parameter identifiers of unmodified data are removed from the output parameter set to be synchronized to obtain the parameter identifiers of the second target data. The parameter identifiers of unmodified data do not belong to the second parameter set.

[0115] For example, the encoder can collect parameter identifiers of basic data written in the static graph, forming a set W. The parameter identifiers in set W represent the basic data modified during the execution of the static graph. It can also collect parameter identifiers of basic data still used after the static graph's lifecycle, forming a set L. The parameter identifiers in set L represent the basic data required when executing code following the first code segment in the first code. A set O is defined as the intersection of sets W and L, which can be represented as O = W ∩ L. Set O can be called the minimum output set. The parameter identifiers in set O represent the basic data modified during the execution of the static graph and the basic data required when executing code following the first code segment in the first code. Therefore, set O is the aforementioned second parameter set.

[0116] The encoder can collect the output parameter list of the static graph to form a set OUT. The set OUT is the set of output parameters to be synchronized mentioned above, which is used to represent the second data to be synchronized. The set OUT can include the parameter matrix of the composite data and the parameter identifier of the basic data.

[0117] For each element o in set OUT in turn Element 'o' can be a parameter matrix of composite data or a parameter identifier of basic data, and the following operations are performed:

[0118] If element o is a parameter identifier of basic data: if element o belongs to the current set O (o∈O), then remove element o from the current set O to obtain a new set O, which can be represented as O = O – {o}; if element o does not belong to the current set... Then, removing element o from the current set OUT will result in a new set OUT, which can be represented as OUT = OUT – {o}.

[0119] If element o is a parameter matrix of composite data, then P is constructed based on o, where P is a set of parameter identifiers for the multiple basic data obtained by splitting the composite data o. If P belongs to the current set O. The new set O obtained by removing P from the current set O can be represented as: O = O – P; if P does not belong to the current set O. Then, remove the parameter identifiers of basic data belonging to P from the current set O to obtain a new set O. Similarly, remove element o from the current set OUT and add parameter identifiers of basic data belonging to both P and set O to obtain a new set OUT, which can be represented as: OUT = OUT – {o} + (P∩O). In some embodiments, the step of removing element o from the current set OUT and adding parameter identifiers of basic data belonging to both P and set O can be performed when the proportion of basic data not belonging to set O in P reaches a second proportion threshold. If the proportion of basic data not belonging to set O in P is less than the second proportion threshold, and the resources (e.g., duration) consumed by removing element o from set OUT and determining P∩O are greater than the resources consumed by directly synchronizing P, then OUT does not need to be updated, i.e., OUT can remain unchanged. The second proportion threshold and the first proportion threshold can be the same or different.

[0120] The encoder can perform the above operation on each element o in the set OUT in sequence. The final set OUT contains only the parameter identifiers of the basic data. The parameter identifiers in the set OUT are the parameter identifiers of the second target data.

[0121] After determining the input preprocessing strategy and output postprocessing strategy, and modifying the static graph, the host in the computing device can select the first target data from the first data to be synchronized according to the input preprocessing strategy, and perform data synchronization for the first target data. For example... Figure 7 As shown, the process of performing data synchronization for the first target data may include: the host in the computing device transmitting the first target data to the device side.

[0122] In the above process, the first parameter set includes the input parameters required for executing the static graph. Based on the first parameter set, unused data from the static graph execution process can be removed from the first data to be synchronized, thereby reducing the amount of data to be transmitted during data synchronization. This process can reduce the execution time of the neural network model by an average of about 5%, improving the execution efficiency of the neural network model. Furthermore, by continuously removing parameter identifiers from the first parameter set to update it, the final first parameter set is an empty set. This process ensures that only one instance of the same basic data is transmitted during a single data synchronization process, avoiding the transmission of multiple identical basic data, further saving resources consumed in data synchronization and reducing the execution time of data synchronization.

[0123] For example, in one embodiment, assume the static graph includes an operator for multiplying x and y, operators for performing other operations using x, and operators for performing other operations using y. If x and y are both basic data, then the first data to be synchronized includes basic data x, basic data y, and composite data [x, y], and the order of the three data is basic data x, basic data y, and composite data [x, y]. During the process of selecting the first target data from the first data to be synchronized, for basic data x in the first data to be synchronized, the parameter identifier of basic data x belongs to the first parameter set. Therefore, the parameter identifier of basic data x is removed from the first parameter set to obtain a new first parameter set. When a second basic data x appears in the first data to be synchronized later, the second basic data x will be removed from the first data to be synchronized. For basic data y in the first data to be synchronized, the parameter identifier of basic data y belongs to the current first parameter set. Therefore, the parameter identifier of basic data y is removed from the current first parameter set to obtain a new first parameter set. At this time, the current first parameter set does not contain the parameter identifier of basic data x, nor does it contain the parameter identifier of basic data y. For the composite data [x, y] in the first data to be synchronized, the composite data [x, y] is split into basic data sets, which include basic data x and basic data y. Since the parameter identifiers of basic data x and basic data y do not belong to the current first parameter set, the composite data [x, y] is removed from the first data to be synchronized. The final first target data contains basic data x and basic data y, but does not contain the composite data [x, y], thus avoiding the transmission of multiple basic data x and multiple basic data y during data synchronization.

[0124] The host in the computing device can also transmit the generated static graph and output post-processing strategy to the device side. The output post-processing strategy is used to instruct the device side to select which data from the second data to be synchronized as the second target data.

[0125] S403, execute the static graph and obtain the second data to be synchronized.

[0126] Before or after the host in the computing device transmits the first target data to the device side, the host in the computing device can also transmit the generated static graph and output post-processing strategy to the device side. The output post-processing strategy is used to instruct the device side to select which data from the second data to be synchronized as the second target data.

[0127] The device can execute a static graph based on the received first target data, or in other words, execute the first code segment in the first code through the static graph mode, and obtain the second data to be synchronized based on the execution result. The second data to be synchronized is stored in the device's memory.

[0128] S404, Select the second target data from the second data to be synchronized, and perform data synchronization on the second target data.

[0129] The host computer in the computing device can select a second target data from the second data to be synchronized according to the output post-processing strategy, and perform data synchronization on the second target data. For example... Figure 7 As shown, the process of performing data synchronization for the second target data may include: the device side transmitting the second target data to the host.

[0130] In the above process, the second parameter set includes parameters that are modified or created during the execution of the static graph, as well as parameters needed when executing code following the first code segment in the first code. Based on the second parameter set, data that was not modified during the execution of the static graph and data not needed for subsequent code execution can be removed from the second data to be synchronized, thereby reducing the amount of data to be transmitted during data synchronization. This process can reduce the execution time of the neural network model by an average of about 10%, improving the execution efficiency of the neural network model. Furthermore, by continuously removing parameter identifiers from the second parameter set to update it, the final second parameter set is an empty set. This process ensures that only one instance of the same basic data is transmitted during a single data synchronization process, avoiding the transmission of multiple identical basic data, further saving resources consumed by data synchronization and reducing the execution time of data synchronization.

[0131] S405 executes the third code segment in the first code segment via dynamic graph mode.

[0132] The host computer in the computing device receives the second target data and can store it in its memory. Based on the second target data, the computing device can continue executing the first code through a Python virtual machine, such as executing the third code segment in the first code using dynamic graph mode, until the inference task of the neural network model is completed. The process of executing the third code segment in the first code using dynamic graph mode can be understood as executing the second dynamic graph. The process of executing the third code segment can be referred to as the process of executing the second code segment described above, and will not be repeated here.

[0133] The above embodiments illustrate data synchronization between the host and the device side within a computing device. In other embodiments, different virtual machines can collaborate to complete the same task, and data synchronization can also be performed between different virtual machines; different servers can also collaborate to complete the same task, and data synchronization can also be performed between different servers.

[0134] In conjunction with the above method embodiments, this application also provides a data synchronization device, which can be applied to... Figure 1 The computing device shown, for example, can be applied to Figure 1 The host computer of the computing device shown, or the device applied to Figure 1 The device side of the computing device shown. This data synchronization device can be used to implement the functions of the above method embodiments, and therefore can achieve the beneficial effects of the above method embodiments. For example... Figure 8 As shown, the data synchronization device 800 may include a data filtering unit 801 and a data synchronization unit 802.

[0135] The data filtering unit 801 can be used to select target data from the data to be synchronized before or after the execution of the static graph; the static graph is generated based on the first code segment in the first code; the first code is code used to describe the neural network model.

[0136] The data synchronization unit 802 can be used to perform data synchronization on target data.

[0137] In some embodiments, the data synchronization device 800 may further include a data synchronization optimization device, or the data synchronization device 800 may be connected to a data synchronization optimization device. The data synchronization optimization device may be used to generate first instruction information or second instruction information. The data filtering unit 801 may be used to select target data from the data to be synchronized according to the first instruction information; or, the data filtering unit 801 may be used to select target data from the data to be synchronized according to the second instruction information.

[0138] It should be noted that, in some embodiments, the data filtering unit 801 can be used to execute any step in the data synchronization method, and the data synchronization unit 802 can be used to execute any step in the data synchronization method. The steps implemented by the data filtering unit 801 and the data synchronization unit 802 can be specified as needed. The data filtering unit 801 and the data synchronization unit 802 respectively implement different steps in the data synchronization method to achieve all the functions of the data synchronization device.

[0139] In the embodiments of this application, the functional modules can be integrated into a single processor, or each module can exist physically separately, or two or more modules can be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional units.

[0140] In conjunction with the above method embodiments, this application also provides a data synchronization device, which can be... Figure 1 The host computer shown, or the data synchronization device, can be... Figure 1 The device side of the computing device shown. This data synchronization device can be used to implement the functions of the above method embodiments, and therefore can achieve the beneficial effects of the above method embodiments. The data synchronization device may include a processor and a memory connected to the processor. The memory stores computer programs or instructions; the processor executes the computer programs or instructions stored in the memory to cause the chip to perform any of the data synchronization methods provided in the first aspect. This data synchronization device can refer to... Figure 1 The host structure shown, or, refer to Figure 1 The structure of the device side shown is not described in detail here.

[0141] In conjunction with the above method embodiments, this application also provides a computing device, which may be a computer, a server, or a server cluster, etc. The structure of the computing device can be as follows: Figure 1 As shown, the system includes a host and a device side. The host is used to select first target data from the first data to be synchronized before executing the static graph, and then perform data synchronization on the first target data. The static graph is generated based on a first code segment in first code; the first code is code used to describe a neural network model. The device side is used to execute the static graph and, after executing the static graph, to select second target data from the second data to be synchronized, and then perform data synchronization on the second target data.

[0142] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the computing device. In other embodiments of this application, the computing device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0143] This application also provides a computer program product comprising computer-executable instructions. In one embodiment, the computer-executable instructions are used to cause a computer to perform the functions described in the method embodiments above.

[0144] Computer-executable instructions can be stored in a computer-readable storage medium. This application also provides a computer-readable storage medium storing executable instructions. In one embodiment, the computer-executable instructions are used to cause a computer to perform the functions described in the method embodiments above.

[0145] The computer-readable storage medium provided in the embodiments of this application may be random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, portable hard disk, CD-ROM, or any other form of computer-readable storage medium known in the art.

[0146] Computer-executable instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive.

[0147] In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be referenced mutually. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or device is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0148] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative examples of the solutions defined by the appended claims and are to be considered as covering any and all modifications, variations, combinations, or equivalents within the scope of this application.

[0149] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if these modifications and variations of the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A data synchronization method, characterized in that, The method includes: For data to be synchronized before or after the execution of the static graph, target data is selected from the data to be synchronized; the static graph is generated based on a first code segment in the first code; the first code is code used to describe a neural network model. Perform data synchronization on the target data.

2. The method according to claim 1, characterized in that, The data to be synchronized includes data pre-input from the device side that executes the static graph, and the target data includes data needed during the execution of the static graph; or... The data to be synchronized includes the data obtained by executing the static graph, and the target data includes the data modified or created during the execution of the static graph. The target data is required when executing the code located after the first code segment in the first code.

3. The method according to claim 1 or 2, characterized in that, The target data includes basic data, which is either numbers or tensors.

4. The method according to any one of claims 1 to 3, characterized in that, The data to be synchronized includes data from the device side that is pre-inputting the static graph, and the target data includes data that needs to be used during the execution of the static graph. The step of selecting target data from the data to be synchronized includes: For the data to be synchronized before the execution of the static graph, target data is selected from the data to be synchronized according to the first indication information used to indicate the target data.

5. The method according to claim 4, characterized in that, The first indication information includes the parameter identifier of the target data; the first indication information is generated in the following manner: Construct a first parameter set and an input parameter set to be synchronized; the first parameter set contains parameter identifiers of the basic data required during the execution of the static graph; the input parameter set to be synchronized contains parameter identifiers of each data in the data to be synchronized. Based on the first parameter set, parameter identifiers of unused data are removed from the input parameter set to be synchronized to obtain a first target set; the first target set contains parameter identifiers of the target data. The parameter identifier for the unused data does not belong to the first parameter set; The first indication information is generated based on the parameter identifiers of the target data in the first target set.

6. The method according to claim 5, characterized in that, The data to be synchronized includes composite data and basic data, wherein the composite data is a dataset comprising multiple basic data sets; the parameter identifiers of the composite data in the input parameter set to be synchronized are parameter matrices; the step of removing parameter identifiers of unused data from the input parameter set to be synchronized includes: The parameter matrix of the composite data in the input parameter set to be synchronized is split into parameter identifiers of basic data; Remove parameter identifiers from the set of input parameters to be synchronized that do not belong to the first parameter set.

7. The method according to any one of claims 1 to 3, characterized in that, The data to be synchronized includes the data obtained by executing the static graph, and the target data includes the data modified or created during the execution of the static graph, and the target data is required when executing the code located after the first code segment in the first code; The step of selecting target data from the data to be synchronized includes: For the data to be synchronized after the static graph is executed, target data is selected from the data to be synchronized according to the second indication information used to indicate the target data.

8. The method according to claim 7, characterized in that, The second indication information contains the parameter identifier of the target data; the second indication information is generated in the following manner: Construct a second parameter set and output a parameter set to be synchronized; the second parameter set contains parameter identifiers of basic data that are modified or created during the execution of the static graph and are required when executing the code in the first code that is located after the first code segment; The output parameter set to be synchronized contains the parameter identifier of each data in the data to be synchronized; Based on the second parameter set, parameter identifiers of unmodified data are removed from the output parameter set to be synchronized to obtain a second target set; the second target set contains the parameter identifiers of the target data. The parameter identifier of the unmodified data does not belong to the second parameter set; The second indication information is generated based on the parameter identifiers of the target data in the second target set.

9. The method according to claim 8, characterized in that, The data to be synchronized includes composite data and basic data, wherein the composite data is a dataset containing multiple basic data; the parameter identifiers of the composite data in the output parameter set to be synchronized are parameter matrices; removing the parameter identifiers of unmodified data from the output parameter set to be synchronized includes: The parameter matrix of the composite data in the output parameter set to be synchronized is split into parameter identifiers of the basic data. Remove parameter identifiers from the output set of parameters to be synchronized that do not belong to the second set of basic data.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: The second code segment in the first code is executed in dynamic graph mode; in the first code, the second code segment is located before the first code segment.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: The third code segment in the first code is executed in dynamic graph mode; in the first code, the third code segment is located after the first code segment.

12. A data synchronization device, characterized in that, The device includes: A data filtering unit is used to select target data from the data to be synchronized before or after the execution of the static graph; the static graph is generated based on a first code segment in the first code; the first code is code used to describe a neural network model. A data synchronization unit is used to perform data synchronization on the target data.

13. The apparatus according to claim 12, characterized in that, The data to be synchronized includes data pre-input from the device side that executes the static graph, and the target data includes data needed during the execution of the static graph; or... The data to be synchronized includes the data obtained by executing the static graph, and the target data includes the data modified or created during the execution of the static graph. The target data is required when executing the code located after the first code segment in the first code.

14. A data synchronization device, characterized in that, It includes a processor and a memory; the memory stores computer programs or instructions; the processor is used to execute the computer programs or instructions stored in the memory so that the data synchronization device performs the method of any one of claims 1 to 11.

15. A computing device, characterized in that, Including both the host and device sides; The host is configured to select first target data from the first data to be synchronized before executing the static graph, and perform data synchronization on the first target data; the static graph is generated based on a first code segment in the first code; the first code is code used to describe a neural network model; On the device side, the device is configured to execute the static graph, and for the second data to be synchronized after executing the static graph, select a second target data from the second data to be synchronized, and perform data synchronization for the second target data.

16. The computing device according to claim 15, characterized in that, The first data to be synchronized includes data from the device side that is pre-inputting the static graph, and the first target data includes data that needs to be used during the execution of the static graph. The second data to be synchronized includes the data obtained by executing the static graph, the second target data includes the data modified or created during the execution of the static graph, and the second target data is required when executing the code located after the first code segment in the first code.

17. The computing device according to claim 15 or 16, characterized in that, The host is further configured to generate first indication information and second indication information; the first indication information includes a parameter identifier of the first target data; the second indication information includes a parameter identifier of the second target data.

18. The computing device according to claim 17, characterized in that, The host is configured to select first target data from the first data to be synchronized according to the first instruction information; The device side is used to select a second target data from the second data to be synchronized according to the second instruction information.

19. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when run on a computer, implement the method as described in any one of claims 1 to 11.

20. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 11.

Images