Data cross-device migration method and device, storage medium and program product

Abstract

The invention discloses a data cross-device migration method and device, a storage medium and a program product, and relates to the technical field of distributed computing, and the method comprises the following steps: determining first data for data migration in a source device; acquiring hardware characteristic information of the target equipment, and performing adaptive conversion on the first data according to the hardware characteristic information to obtain adaptive second data; dividing the second data into static data and dynamic data; mapping the memory addresses of the source device and the target device to the shared virtual address, and transmitting the dynamic data to the target device; static data obtained by accessing the shared virtual address by the target device and received dynamic data are determined, and whether the static data and the dynamic data are adaptive to the target device is determined; if it is determined that the static data and the dynamic data are matched with the target device, the target device is controlled to execute calculation tasks corresponding to the static data and the dynamic data. According to the method, the cross-device migration efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of distributed computing technology, and in particular to a method, device, storage medium and program product for data migration across devices. Background Art

[0002] As the complexity of computing tasks and the amount of data continue to increase, models and computing tasks in distributed computing and heterogeneous computing environments need to be migrated between different hardware devices. In these environments, different hardware devices (such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Network Processing Unit), etc.) are widely used to accelerate various computing-intensive tasks, including but not limited to deep learning training, image processing, and scientific computing. However, different types of hardware devices have significant differences in architectural design, memory management mechanism, instruction set support, and acceleration library. Direct data migration may lead to low efficiency in the migration process.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a method, device, storage medium and program product for data migration across devices, aiming to solve the technical problem that cross-device data migration may lead to low efficiency of the migration process.

[0005] To achieve the above objectives, the present application proposes a method for data migration across devices, the method comprising: Determine first data to be migrated in the source device; Acquire hardware characteristic information of the target device, and perform adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data; dividing the second data into static data and dynamic data; Mapping the memory addresses of the source device and the target device to a shared virtual address, and transmitting the dynamic data to the target device; Determining the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determining whether the static data and the dynamic data are adapted to the target device; If it is determined that the static data and the dynamic data are adapted to the target device, the target device is controlled to execute the computing tasks corresponding to the static data and the dynamic data.

[0006] In one embodiment, the step of performing adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data includes: Determining a target data format of the target device according to the hardware characteristic information, wherein the source device and the target device are heterogeneous devices; Converting the first data into a third data in the target data format; Determine a target data layout structure according to the hardware characteristic information, wherein the target data layout structure includes an alignment mode and a paging size; The third data is reorganized according to the target data layout structure to obtain second data of the target data layout structure.

[0007] In one embodiment, the target data layout includes an alignment mode and a paging size, and the step of reorganizing the third data according to the target data layout structure to obtain the second data of the target data layout structure includes: Aligning bytes of the third data according to the alignment method; Performing paging adjustment on the aligned third data according to the paging size; The access mode of the target device is determined according to the hardware characteristic information, and the paging-adjusted third data is reorganized according to the access mode to obtain second data, wherein the second data conforms to the data layout structure required by the target device.

[0008] In one embodiment, the step of determining whether the static data and the dynamic data are adapted to the target device includes: Verifying whether the data layout structures of the static data and the dynamic data are adapted to the preset data layout structure of the target device; Calculate whether the data hash values ​​corresponding to the static data and the dynamic data in the target device are consistent with the data hash values ​​corresponding to the static data and the dynamic data in the source device; Verify whether the calculation graphs corresponding to the static data and the dynamic data and the operators and instructions in the calculation graphs are adapted to the target device; Verifying whether the context information corresponding to the static data and the dynamic data is adapted to the target device; If it is determined that the data layout structure of the static data and the dynamic data is adapted to the preset data layout structure of the target device, the data hash values ​​corresponding to the static data and the dynamic data in the target device are consistent with the data hash values ​​corresponding to the static data and the dynamic data in the source device, the calculation graph corresponding to the static data and the dynamic data and the operators and instructions in the calculation graph are adapted to the target device and the context information corresponding to the static data and the dynamic data is adapted to the target device, then determine whether the static data and the dynamic data are adapted to the target device.

[0009] In one embodiment, before the step of performing adaptability verification on the static data and the dynamic data to obtain a verification result, the step further includes: Reconstructing the computation graph according to the hardware characteristic information of the target device; Replacing the operators in the computation graph with operators adapted to the target device according to the hardware characteristic information to obtain a first computation graph; Traversing the first computation graph and identifying computation nodes in the first computation graph; For each of the computing nodes, context information of the computing node is extracted, and all the context information is updated, wherein the context information includes input data, output data, state variables, and configuration parameters.

[0010] In one embodiment, after the step of updating all the context information, the method further includes: A driver on the target device is driven to identify updated context information of each operation node in the computation graph, and convert the context information of each operation node into a hardware instruction adapted to the target device.

[0011] In one embodiment, after the step of converting the context information of each of the operation nodes into a hardware instruction adapted to the target device, the method further includes: The hardware acceleration library of the target device is loaded, and the optimization function in the hardware acceleration library is bound to each operation node of the computation graph to optimize the hardware instruction.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a data cross-device migration device, the data cross-device migration device comprising: A data module, used to determine first data to be migrated in a source device; A conversion module, used to obtain hardware characteristic information of a target device, and perform adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data; A classification module, used for classifying the second data into static data and dynamic data; A transmission module, used for mapping the memory addresses of the source device and the target device to a shared virtual address, and transmitting the dynamic data to the target device; A receiving module, used for determining the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determining whether the static data and the dynamic data are adapted to the target device; The execution module is used to control the target device to execute the computing tasks corresponding to the static data and the dynamic data if it is determined that the static data and the dynamic data are adapted to the target device.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a data cross-device migration device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the data cross-device migration method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the data cross-device migration method described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the data cross-device migration method as described above.

[0016] The present application determines the first data for data migration in the source device; obtains the hardware characteristic information of the target device, and adapts and converts the first data according to the hardware characteristic information to obtain the adapted second data; divides the second data into static data and dynamic data; maps the memory addresses of the source device and the target device to a shared virtual address, and transmits the dynamic data to the target device; determines the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determines whether the static data and the dynamic data are adapted to the target device; if it is determined that the static data and the dynamic data are adapted to the target device, the target device is controlled to execute the computing tasks corresponding to the static data and the dynamic data. The present application realizes the rapid migration of the data of the model and computing tasks from one device to another, and automatically adapts according to the hardware characteristics of the target device, ensuring the correct identification and efficient calculation of the data during the cross-device migration process, and solving the problem of low data transmission efficiency during device migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of the first embodiment of the method for data migration across devices provided in this application; Figure 2 A flowchart of an implementation method of a method for data migration across devices according to an embodiment of the present application is provided; Figure 3 A schematic diagram of a simplified process of a method for data migration across devices provided in Embodiment 2 of the present application; Figure 4 This is a schematic diagram of the module structure of the device for data migration across devices according to an embodiment of the present application; Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the method for migrating data across devices in an embodiment of the present application.

[0020] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0023] Based on this, the present application embodiment provides a method for data migration across devices. Figure 1 , Figure 1 This is a flowchart of the first embodiment of the method for migrating data across devices of the present application.

[0024] In this embodiment, the data cross-device migration method includes steps S10 to S60: Step S10: determining first data to be migrated in a source device.

[0025] In this embodiment, it should be noted that the execution subject is a data cross-device migration system that can control the source device and the target setting to perform some operations. The devices are heterogeneous computing devices, such as GPU and NPU, to perform migration of computing tasks and parameter data, such as model migration. The first data corresponding to the computing task is initially located in the source device and needs to be migrated to the target device. The first data that needs to be migrated can include multiple categories, such as model parameters, including static data such as weights and biases; input data, including dynamic data such as training data, test data or inference data; intermediate results, temporary data generated during the calculation process; configuration parameters, such as learning rate, batch size, optimizer settings; runtime information, such as the current number of iterations, input data size, etc. All of the above can be used as the first data for computing task migration. There are many ways to read the first data. For example, if the data is stored in a file system, standard file reading methods (such as fopen and read) can be used to read the data. If the data is stored in a database, the data can be extracted through SQL (Structured Query Language) queries or other database APIs (Application Programming Interface). If the data is already in memory, the data can be accessed directly through a pointer or a memory address. If the data is stored in shared memory, the data can be accessed through a shared memory mechanism.

[0026] Step S20, obtaining hardware characteristic information of the target device, and performing adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data.

[0027] It should be noted that the adaptation conversion is used for compatibility and performance optimization of data on the target device, so that computing tasks can be executed efficiently. Hardware feature information can include the following types: processor architecture, such as CPU, GPU, NPU, etc.; memory layout, including memory alignment (such as 4-byte, 8-byte alignment), page size (such as 4KB, 64KB); supported data formats: such as FP32, FP16, INT8, etc., instruction sets: such as CUDA for GPU, specific NPU instruction sets; cache characteristics, such as cache size, cache line size; memory bandwidth, memory read and write speed; other characteristics, such as DMA (Direct Memory Access) transfer capability, operations supported by hardware accelerators, etc. The hardware feature information of the target device can be obtained by controlling the API provided by the driver of the target device, and the hardware feature information can be queried using the interface or command provided by the operating system.

[0028] It is understandable that the second data obtained after the conversion needs to ensure that the performance on the target device is better than or at least not inferior to the performance on the source device, the data is not lost or damaged during the conversion process, and the data can be correctly identified and processed by the target device, which can ensure the compatibility and performance optimization of the data on the target device, thereby improving the overall efficiency of the computing task.

[0029] In one possible implementation, see Figure 2 , step S20 may include steps S21 to S24: Step S21, determining a target data format of a target device according to hardware characteristic information, wherein the source device and the target device are heterogeneous devices.

[0030] It should be noted that the hardware feature information includes supported data formats. Select the appropriate floating point precision according to the support of the target device. For example, if the target device supports FP16 and the precision requirement is not high, you can select FP16 to save memory and increase the calculation speed. For quantized models, select the appropriate integer precision, such as INT8 or INT4, to further reduce memory usage and improve calculation efficiency. At the same time, make sure that the selected data type is consistent with the type supported by the target device.

[0031] It is understandable that some applications (such as image recognition) may have high requirements for accuracy and need to balance accuracy and performance; choosing a lower precision data format can significantly reduce memory usage and is suitable for memory-constrained devices; some data formats can provide higher computing efficiency on specific hardware, such as FP16 is generally faster than FP32 on modern GPUs, so it is necessary to give priority to data formats that can maximize the performance of the target device.

[0032] Step S22, converting the first data into a target data format to obtain third data in the target data format.

[0033] It should be noted that the types of conversion include floating point precision conversion, integer precision conversion and data type conversion. After the conversion is completed, the consistency of the data before and after the conversion can be checked, such as calculating the hash value of the data before and after the conversion to ensure that the data is not lost or damaged.

[0034] Step S23, determining a target data layout structure according to the hardware characteristic information, wherein the target data layout structure includes an alignment mode and a paging size.

[0035] It should be noted that this step is used to optimize the storage and access methods of data on the target device and improve the execution efficiency of computing tasks. Data layout includes memory alignment (such as 4-byte, 8-byte alignment) and paging size (such as 4KB, 64KB). Among them, the alignment method is to adjust the storage location of the data according to the memory alignment requirements of the target device (such as 8-byte alignment); the paging strategy is to reorganize the data to meet the paging requirements according to the paging size of the target device (such as 4KB, 64KB); and the data access mode, if the target device is more suitable for sequential access, store related data together to improve access efficiency, if the target device is more suitable for random access, store the data in a dispersed manner to improve parallel access efficiency. In addition, it also includes data layout optimization, such as for matrix operations, select a suitable matrix layout (such as row priority or column priority) to adapt to the computing characteristics of the target device; for tensor data, select a suitable tensor layout (such as NHWC or NCHW) to optimize the performance of operations such as convolution.

[0036] Step S24, reorganizing the third data according to the target data layout structure to obtain second data of the target data layout structure.

[0037] It should be noted that this step is used to optimize the storage and access methods of data on the target device and improve the execution efficiency of computing tasks. Regarding the alignment method, it is necessary to adjust the storage location of the data to meet the alignment requirements. If the data does not meet the memory alignment requirements of the target device (such as 8-byte alignment), add necessary padding to meet the alignment conditions. Regarding the paging size, according to the paging size of the target device (such as 4KB, 64KB), reorganize the data to meet the paging requirements and ensure that the data is aligned on the page boundary to reduce the performance loss caused by cross-page access. Regarding the data access model, if the target device is more suitable for sequential access, store related data together to improve access efficiency. If the target device is more suitable for random access, store the data in a dispersed manner to improve parallel access efficiency.

[0038] This implementation determines the data format supported by the target device, converts the data on the source device into a data format supported by the target device to ensure that the data can be correctly processed on the target device, determines the data layout structure requirements of the target device to optimize data access and processing efficiency, and ensures that the data runs efficiently and correctly on the target device.

[0039] In a feasible implementation, step S24 may include steps S241 to S243: Step S241, aligning the bytes of the third data according to the alignment method.

[0040] Step S242, performing paging adjustment on the aligned third data according to the paging size.

[0041] Step S243, determining the access mode of the target device according to the hardware characteristic information, and reorganizing the paging-adjusted third data according to the access mode to obtain second data, wherein the second data conforms to the data layout structure required by the target device.

[0042] This implementation method reorganizes the third data according to the data layout structure of the target device, specifically aligning the data according to the alignment method, adjusting the paging according to the paging size, and reorganizing the data according to the access mode. Through these steps, it can be ensured that the data runs efficiently and correctly on the target device, which not only improves the overall efficiency of the computing task, but also ensures the consistency and integrity of the data.

[0043] Step S30, dividing the second data into static data and dynamic data.

[0044] It should be noted that static data refers to data that does not change during the running of the program, such as the weight parameters in the deep learning model, which are usually fixed after training; parameters in the system configuration file, such as hyperparameters, network structure, etc.; constants defined in the program, such as fixed values, strings, etc.; pre-processed data sets, such as normalized image data. Since static data usually does not change during runtime, it can be pre-loaded into cache or memory to speed up access. Dynamic data refers to data that may change during the running of the program, for example, data input by the user, such as real-time sensor data, user interaction data, etc.; temporary data generated during the calculation process, such as the output of the convolutional layer; status information when the program is running, such as counters, flags, etc.; log information recorded when the program is running, such as error logs, debugging information, etc.

[0045] It can be understood that by classifying data into two categories, static and dynamic, different types of resources can be better managed and optimized, improving the overall performance and efficiency of the system.

[0046] Step S40, mapping the memory addresses of the source device and the target device to a shared virtual address, and transmitting the dynamic data to the target device.

[0047] It should be noted that the physical memory addresses of the source device and the target device are mapped to a shared virtual address space to facilitate data transfer between different devices. By sharing the virtual address space, the data transfer process can be simplified and the overhead of data copying can be reduced. At the same time, the unified virtual address space facilitates the management and scheduling of data across devices. Using hardware-supported direct memory access (DMA) or zero-copy technology for data transfer can improve data transfer efficiency.

[0048] Specifically, first use the API provided by the operating system (such as POSIX shared memory, Windows file mapping, etc.) to create a shared memory area, map the physical memory addresses of the source device and the target device to this shared memory area, so that they can access the same memory through the same virtual address.

[0049] During the dynamic data transmission process, the hardware DMA controller can be used to directly read data from the source device's memory and write it to the target device's memory without CPU intervention. Through DMA technology, data can be directly transferred from the source device's memory to the target device's memory without passing through the CPU. The system calls the DMA controller provided by the underlying operating system to transfer data directly from the GPU memory to the NPU memory. This avoids CPU intervention and significantly improves data transmission efficiency.

[0050] At the same time, the system uses zero-copy technology to reduce the multiple copies of data between devices. By sharing memory or directly accessing the physical memory of different devices, data can be transferred to the target device at one time, reducing the performance overhead caused by multiple copies.

[0051] Step S50, determining the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determining whether the static data and the dynamic data are suitable for the target device.

[0052] It should be noted that, based on the memory mapping in the aforementioned steps, the API provided by the operating system (such as mmap, MapViewOfFile, etc.) can be used in the target device to access the data in the shared virtual address.

[0053] Step S60: If it is determined that the static data and the dynamic data are suitable for the target device, the target device is controlled to execute the computing tasks corresponding to the static data and the dynamic data.

[0054] It should be noted that after confirming that the static data and the dynamic data are compatible with the target device, the computing task on the target device is started.

[0055] Specifically, the target device needs to be initialized first, and necessary configurations are performed according to the hardware characteristics of the target device. Then, static data and dynamic data are loaded into the memory of the target device, and the static data and dynamic data are combined into a complete computing task. There are multiple computing instructions in the computing task, and the target device can be controlled to execute each computing instruction.

[0056] This embodiment determines the first data for data migration in the source device; obtains the hardware characteristic information of the target device, and adapts and converts the first data according to the hardware characteristic information to obtain the adapted second data; divides the second data into static data and dynamic data, wherein the static data is static data and the dynamic data is dynamic data; maps the memory addresses of the source device and the target device to a shared virtual address, and transmits the dynamic data to the target device; determines the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determines whether the static data and the dynamic data are adapted to the target device; if it is determined that the static data and the dynamic data are adapted to the target device, the target device is controlled to execute the computing tasks corresponding to the static data and the dynamic data. This embodiment realizes the rapid migration of the data of the model and the computing task from one device to another, and automatically adapts according to the hardware characteristics of the target device, ensuring the correct identification and efficient calculation of the data during the cross-device migration process, and solving the problem of low data transmission efficiency during device migration.

[0057] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. On this basis, in step S50, the step of determining whether the static data and the dynamic data are adapted to the target device includes steps A10 to A20: Step A10, performing adaptability verification on the static data and the dynamic data to obtain a verification result.

[0058] It should be noted that this step verifies whether the data meets the requirements of the target device through a series of checks.

[0059] Step A20, judging whether the static data and the dynamic data are suitable for the target device according to the verification result.

[0060] It should be noted that, based on the verification results in step A10, a comprehensive assessment is made as to whether the data is suitable for the target device. If all verification results meet the requirements, the data is considered to be suitable for the target device.

[0061] If it is found that the data format, layout or other aspects do not meet the requirements of the target device, appropriate measures need to be taken, such as re-converting the data and data layout, or generating log records and error reports. In other words, if the data is suitable for the target device, continue to perform the next computing task; if the data is not suitable for the target device, necessary adjustments need to be made or the data needs to be retransmitted.

[0062] The adaptability verification includes: Verify whether the data layout structure of static data and dynamic data is adapted to the preset data layout structure of the target device; Calculate whether the data hash values ​​corresponding to the static data and the dynamic data in the target device are consistent with the data hash values ​​corresponding to the static data and the dynamic data in the source device; Verify whether the computation graphs corresponding to static data and dynamic data and the operators and instructions in the computation graphs are compatible with the target device; Verify whether the context information corresponding to the static data and dynamic data is suitable for the target device.

[0063] Adaptability verification can include data format verification, data layout verification, consistency verification, and performance verification. Data format verification, such as checking whether the data type meets the requirements of the target device (such as FP32, FP16, etc.), and checking whether the precision and range of the data are within the range supported by the target device. Data layout verification, such as checking whether the storage layout of the data meets the requirements of the target device (such as row priority or column priority), checking whether the data is aligned according to the memory alignment requirements of the target device, and checking whether the data is properly adjusted according to the paging size of the target device. Consistency verification, such as calculating the hash value of the data on the source device and the target device, comparing whether the two are consistent, and using checksums or other consistency checking methods to ensure data integrity. Performance testing, such as running benchmarks on the target device to verify whether the data layout can provide the expected performance improvement, and checking the memory usage of the data on the target device to ensure that it meets expectations.

[0064] This embodiment ensures that the data can be correctly used by the target device after transmission and meets the hardware characteristics and software requirements of the target device, which not only improves the overall performance of the system but also ensures the correctness and reliability of the computing task.

[0065] In a feasible implementation manner, before step A10, please refer to Figure 3 , and may further include steps A01 to A04: Step A01, reconstructing the calculation graph according to the hardware characteristic information of the target device.

[0066] It should be noted that after the parameters and computing tasks are migrated to the target device, the system needs to reconstruct the computing graph to ensure that the computing tasks can be correctly executed on the target device. The system will reconstruct the computing graph based on the hardware architecture of the target device, that is, the hardware feature information. For example, the NPU and GPU may have different ways of allocating hardware resources when performing certain operations. The system will reconstruct the computing graph based on these differences to ensure that it can run efficiently on the NPU.

[0067] Specifically, according to the hardware characteristics of the target device, the nodes in the computing graph are rearranged and the resources of each computing node are optimized to ensure that each computing node can fully utilize the resources provided by the target device while avoiding waste of resources. For different types of processors, the algorithm in the original computing node can be converted into a form that is more suitable for that type of processor.

[0068] For example, if the source device is an NPU and the target device is a GPU, the NPU (neural network processing unit) is generally designed to accelerate specific operations in deep learning models, such as the calculation of convolutional layers. Although the GPU is also good at parallel processing, its architecture is more general and suitable for a wider range of computing scenarios. When migrating from a GPU to an NPU, the system may identify which parts can be accelerated by an NPU-specific instruction set and modify the computational graph structure accordingly. For example, some sequences that originally consisted of multiple small operations can be replaced by a single NPU native operation. Through such reconstruction, not only can the computing speed be increased, but also the power consumption can be reduced, because the NPU is often more energy-efficient than the GPU when performing specific tasks.

[0069] Step A02: replace the operators in the computation graph with operators adapted to the target device according to the hardware characteristic information to obtain a first computation graph.

[0070] It should be noted that when the model is migrated from the source device to the target device, the system will dynamically replace the operators in the computational graph. For example, the convolution operator may use cuDNN on the GPU, but may need to be replaced with an NPU-specific accelerated convolution operator on the NPU. Different hardware platforms (such as GPUs and NPUs) usually have specific optimization libraries and instruction sets that can significantly improve the execution efficiency of certain operations. For example, cuDNN is a highly optimized deep learning library designed for GPUs, while NPUs may have their own dedicated libraries to accelerate operations such as convolution. By replacing operators, you can ensure that the computational graph can run correctly on different hardware platforms and can take advantage of the best performance provided by these platforms.

[0071] Specifically, first, the system needs to identify which operators in the computation graph can be replaced, and determine the type and properties of each operator by traversing and analyzing the computation graph. According to the hardware feature information of the target device, determine which operators can use more efficient implementation methods on the target device. For example, the convolution operator can use cuDNN on the GPU and an NPU-specific acceleration library on the NPU. Then, perform library function replacement. If the target device is a GPU, the general convolution operator can be replaced with an optimized version in cuDNN. If the target device is an NPU, the convolution operator can be replaced with an operator in the NPU-specific acceleration library. Finally, replace the operator nodes in the original computation graph with new operator nodes that are compatible with the target device, and ensure that the replaced operator nodes are correctly connected to other nodes in the computation graph to maintain the consistency of the data flow.

[0072] Step A03, traverse the first computation graph, and identify computation nodes in the first computation graph.

[0073] It should be noted that a computational graph is a data structure that represents a computational task. It consists of nodes (operators) and edges (data flows). Each node represents a computational operation (such as convolution, matrix multiplication, etc.), and each edge represents the flow of data between nodes. A computational node is a node in a computational graph, which usually represents a specific computational operation or data processing step.

[0074] Step A04: for each computing node, extract the context information of the computing node and update all the context information, wherein the context information includes input data, output data, state variables, and configuration parameters.

[0075] It should be noted that in the context information, the input data is the input tensor required by the computing node; the output data is the output tensor generated by the computing node; the state variable is the possible state of the computing node (such as the hidden state in RNN); the configuration parameters are the configuration options of the computing node, such as parallelism, memory allocation strategy, etc.

[0076] Context information updates include: data format conversion, converting the data format from the format supported by the source device to the format supported by the target device. For example, from FP32 to FP16; data layout adjustment, adjusting the data layout according to the requirements of the target device. For example, from NHWC (high priority) to NCHW (channel priority); parameter precision adjustment, adjusting parameters (such as weights and biases) to the precision supported by the target device; state variable initialization or update, ensuring that the state variables are correctly initialized or updated on the target device; configuration parameter update, updating the configuration parameters to fully utilize the characteristics of the target device, such as setting the appropriate parallelism, memory allocation strategy, etc.

[0077] This implementation ensures that the computation graph can fully utilize the hardware characteristics of the target device, and all operators and nodes can be correctly recognized and executed by the target device. By rebuilding the computation graph, replacing operators, and compiling to generate hardware instructions, the computing efficiency and performance can be significantly improved.

[0078] In a feasible implementation, after step A04, please refer to Figure 3 , and may further include step A05: Step A05 , driving a driver on the target device to identify updated context information of each operation node in the computation graph, and converting the context information of each operation node into hardware instructions adapted to the target device.

[0079] It should be noted that instruction set mapping can be performed before driver adaptation. Instruction set mapping refers to the process of dynamically converting high-level computing operations (such as convolution, matrix multiplication, etc.) into the underlying hardware instruction set supported by the target device. The specific acceleration instruction set of the target device is used to improve computing efficiency and ensure that computing tasks can be correctly executed on different hardware platforms.

[0080] Specifically, the system first identifies each operator in the computational graph, determines its type and attributes, and maps each operator to the corresponding hardware instruction set according to the hardware architecture of the target device. For example: CUDA on GPU: convolution operations can use highly optimized convolution functions in the CUDA library; dedicated instruction sets on NPU: convolution operations can use dedicated acceleration instruction sets provided by NPU.

[0081] After completing the instruction set mapping, the computing tasks need to be assigned to the NPU for execution through the driver, which is called driver adaptation. Driver adaptation refers to the process in which the driver of the target device takes over the computing tasks migrated from the source device and assigns them to the appropriate hardware unit for execution, which can ensure that the computing tasks can effectively utilize the hardware resources of the target device. Execution control: Control the execution process of the computing tasks to ensure that they run as expected.

[0082] Specifically, the driver receives the computing tasks migrated from the source device, parses each operation node in the computational graph, understands its functions and dependencies, converts the parsed operation nodes into low-level hardware instructions supported by the target device, and schedules the generated hardware instructions to the appropriate hardware unit of the target device for execution.

[0083] In a feasible implementation, after step A05, please refer to Figure 3 , and may further include step A06: Step A06, loading the hardware acceleration library of the target device, binding the optimization function in the hardware acceleration library to each operation node of the computation graph to optimize the hardware instructions.

[0084] It should be noted that the specific hardware acceleration library (such as cuDNN, cuBLAS, etc.) provided by the target device is obtained so as to utilize its optimized functions and instructions to improve computing performance.

[0085] Specifically, determine the hardware acceleration library that needs to be loaded according to the type of target device (such as GPU, NPU, etc.); use the appropriate API or interface to load the hardware acceleration library. For example, for NVIDIA GPU, you can use the CUDA API to load the cuDNN or cuBLAS library; initialize the hardware acceleration library, set the necessary configuration parameters and resources, including setting the memory pool, configuring the number of threads, etc.

[0086] Replace the original hardware instructions with the optimized functions in the hardware acceleration library to further improve the performance of the computing task. Specifically, traverse the computational graph: traverse the computational graph, identify each operation node and its current hardware instructions; bind the optimized functions in the hardware acceleration library to the corresponding operation nodes. This usually involves replacing the original hardware instructions with optimized functions; configuring the parameters of the optimized functions according to the requirements of the target device. For example, set the degree of parallelism, memory allocation strategy, etc.; ensure that each operation node is successfully bound to the optimized function and the configuration parameters are correct. You can verify whether the binding is successful through logging or debugging tools.

[0087] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method of data cross-device migration of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0088] This application also provides a data cross-device migration device, please refer to Figure 4 , the data cross-device migration device comprises: A data module 10, configured to determine first data to be migrated in a source device; The conversion module 20 is used to obtain hardware characteristic information of the target device, and perform adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data; A classification module 30, configured to classify the second data into static data and dynamic data; A transmission module 40, configured to map the memory addresses of the source device and the target device to a shared virtual address, and transmit the dynamic data to the target device; A receiving module 50, configured to determine the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determine whether the static data and the dynamic data are suitable for the target device; The execution module 60 is used to control the target device to execute the computing tasks corresponding to the static data and the dynamic data if it is determined that the static data and the dynamic data are adapted to the target device.

[0089] The data cross-device migration device provided by the present application adopts the data cross-device migration method in the above embodiment, which can solve the technical problem that cross-device data migration may lead to low efficiency of the migration process. Compared with the prior art, the beneficial effects of the data cross-device migration device provided by the present application are the same as the beneficial effects of the data cross-device migration method provided by the above embodiment, and the other technical features in the data cross-device migration device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0090] The present application provides a data cross-device migration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data cross-device migration method in the above-mentioned embodiment one.

[0091] Reference below Figure 5 , which shows a schematic diagram of the structure of a data cross-device migration device suitable for implementing the embodiment of the present application. The data cross-device migration device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The data cross-device migration device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0092] like Figure 5As shown, the data cross-device migration device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data cross-device migration device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data cross-device migration device to communicate with other devices wirelessly or by wire to exchange data. Although the data cross-device migration device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0093] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0094] The data cross-device migration device provided by the present application adopts the data cross-device migration method in the above embodiment, which can solve the technical problem that cross-device data migration may lead to low efficiency of the migration process. Compared with the prior art, the beneficial effects of the data cross-device migration device provided by the present application are the same as the beneficial effects of the data cross-device migration method provided by the above embodiment, and the other technical features in the data cross-device migration device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0095] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0096] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0097] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data cross-device migration method in the above-mentioned embodiment.

[0098] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data cross-device migration method, and can solve the technical problem that cross-device data migration may lead to low efficiency of the migration process. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the data cross-device migration method provided by the above-mentioned embodiment, and will not be repeated here.

[0099] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data cross-device migration method when executed by a processor.

[0100] The computer program product provided by this application can solve the technical problem that cross-device data migration may lead to low efficiency of the migration process. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the cross-device data migration method provided by the above embodiment, which will not be repeated here.

[0101] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for data migration across devices, characterized in that: The data cross-device migration method comprises: Determine first data to be migrated in the source device; Acquire hardware characteristic information of the target device, and perform adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data; dividing the second data into static data and dynamic data; Mapping the memory addresses of the source device and the target device to a shared virtual address, and transmitting the dynamic data to the target device; Determining the static data obtained by the target device accessing the shared virtual address and the received dynamic data, and determining whether the static data and the dynamic data are adapted to the target device; If it is determined that the static data and the dynamic data are adapted to the target device, the target device is controlled to execute the computing tasks corresponding to the static data and the dynamic data.

2. The method for data migration across devices according to claim 1, characterized in that: The step of performing adaptation conversion on the first data according to the hardware characteristic information to obtain adapted second data includes: Determining a target data format of the target device according to the hardware characteristic information, wherein the source device and the target device are heterogeneous devices; Converting the first data into a third data in the target data format; Determine a target data layout structure according to the hardware characteristic information, wherein the target data layout structure includes an alignment mode and a paging size; The third data is reorganized according to the target data layout structure to obtain second data of the target data layout structure.

3. The method for data migration across devices according to claim 1, characterized in that: The step of reorganizing the third data according to the target data layout structure to obtain second data of the target data layout structure includes: Aligning bytes of the third data according to the alignment method; Performing paging adjustment on the aligned third data according to the paging size; The access mode of the target device is determined according to the hardware characteristic information, and the paging-adjusted third data is reorganized according to the access mode to obtain second data, wherein the second data conforms to the data layout structure required by the target device.

4. The method for data migration across devices according to claim 1, characterized in that: The step of determining whether the static data and the dynamic data are adapted to the target device comprises: Verifying whether the data layout structures of the static data and the dynamic data are adapted to the preset data layout structure of the target device; Calculate whether the data hash values ​​corresponding to the static data and the dynamic data in the target device are consistent with the data hash values ​​corresponding to the static data and the dynamic data in the source device; Verify whether the calculation graphs corresponding to the static data and the dynamic data and the operators and instructions in the calculation graphs are adapted to the target device; Verifying whether the context information corresponding to the static data and the dynamic data is adapted to the target device; If it is determined that the data layout structure of the static data and the dynamic data is adapted to the preset data layout structure of the target device, the data hash values ​​corresponding to the static data and the dynamic data in the target device are consistent with the data hash values ​​corresponding to the static data and the dynamic data in the source device, the calculation graph corresponding to the static data and the dynamic data and the operators and instructions in the calculation graph are adapted to the target device and the context information corresponding to the static data and the dynamic data is adapted to the target device, then determine whether the static data and the dynamic data are adapted to the target device.

5. The method for data migration across devices as claimed in claim 4, characterized in that: Before the step of determining whether the static data and the dynamic data are adapted to the target device, the method further includes: Reconstructing the computation graph according to the hardware characteristic information of the target device; Replacing the operators in the computation graph with operators adapted to the target device according to the hardware characteristic information to obtain a first computation graph; Traversing the first computation graph and identifying computation nodes in the first computation graph; For each of the computing nodes, context information of the computing node is extracted, and all the context information is updated to obtain updated context information, wherein the context information includes input data, output data, state variables, and configuration parameters.

6. The method for data migration across devices as claimed in claim 5, characterized in that: After the step of updating all the context information, the method further includes: A driver on the target device is driven to identify updated context information of each operation node in the computation graph, and convert the context information of each operation node into a hardware instruction adapted to the target device.

7. The method for data migration across devices as claimed in claim 6, characterized in that: After the step of converting the context information of each of the operation nodes into a hardware instruction adapted to the target device, the method further includes: The hardware acceleration library of the target device is loaded, and the optimization function in the hardware acceleration library is bound to each operation node of the computation graph to optimize the hardware instruction.

8. A data migration device across devices, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for migrating data across devices as claimed in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data cross-device migration method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for migrating data across devices as claimed in any one of claims 1 to 4 are implemented.

Images