Data format conversion methods, devices, media and program products

Abstract

This invention relates to the field of artificial intelligence technology, providing a data format conversion method, device, medium, and program product. The method includes: based on data transmission information between a source device and a target device, searching a format detection result table for data format detection results for both the source and target devices; if no data format detection result is found, performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result; if the detection result is a mismatch, converting the output data of the source device to obtain target data, and using the target data as input data for data transmission to the target device. This invention, through automated data format matching detection and intelligent conversion logic, achieves automatic identification and adaptation of data formats in cross-device communication, thereby ensuring efficient and accurate data transmission.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and particularly to the field of data transmission technology between heterogeneous devices, and especially to a data format conversion method, device, medium and program product. Background Technology

[0002] With the rapid development of artificial intelligence technology and the widespread application of Large Language Models (LLMs), the parameter scale and training data volume of models are constantly increasing, leading to a growing demand for computing resources and a corresponding increase in the number of computing devices required. Therefore, multi-device heterogeneous training (also known as "heterogeneous training") has become an indispensable model training method. Pipeline parallel-based heterogeneous training is currently the mainstream mode, which distributes different layers of the model across multiple devices, allowing these layers to process data in parallel, thereby improving training efficiency.

[0003] However, these devices may come from different manufacturers or be different product models from the same manufacturer, possessing different hardware characteristics, thus forming a heterogeneous device cluster. Within this cluster, the differences in hardware characteristics between devices lead to variations in data formats. This discrepancy causes format mismatches when training data is transmitted between devices, thereby affecting the normal progress of the training process. Summary of the Invention

[0004] This invention provides a data format conversion method, device, medium, and program product to solve the defect in related technologies where data format mismatch between heterogeneous devices affects model training.

[0005] This invention provides a data format conversion method, comprising:

[0006] Based on the data transmission information of the source device and the target device, the data format detection results of the source device and the target device are retrieved from the format detection result table;

[0007] If the data format detection result is not found, a data format matching detection is performed based on the data shape of the source device and the data shape of the target device to obtain a detection result, and the detection result is stored in the format detection result table as the data format detection result of the source device and the target device.

[0008] If the detection result is a mismatch, the output data of the source device is converted to obtain target data, and the target data is used as the input data of the target device for data transmission. The format of the target data matches the data format of the target device.

[0009] According to a data format conversion method provided by the present invention, the step of converting the output data of the source device to obtain target data includes:

[0010] Perform a full collection operation on each slice of data output from the source device to obtain the output data;

[0011] The output data is subjected to data order transformation, and the transformed data is then distributed to obtain the target data.

[0012] A data format conversion method provided by the present invention further includes:

[0013] If the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0014] A data format conversion method provided by the present invention further includes:

[0015] If the data format detection result is found and the data format detection result is a match, then the output data of the source device is used as the input data of the target device for data transmission.

[0016] If the data format detection result is a mismatch, the output data of the source device is converted to obtain the target data, and the target data is used as the input data of the target device for data transmission.

[0017] According to a data format conversion method provided by the present invention, the step of performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result includes:

[0018] Based on the data shape of the source device, determine the data size and data order of the source device;

[0019] Based on the data shape of the target terminal device, determine the data size and data order of the target terminal device;

[0020] The data size of the source device is compared with the data size of the target device to obtain a first comparison result, and the data order of the source device is compared with the data order of the target device to obtain a second comparison result;

[0021] The detection result is determined based on the first comparison result and the second comparison result.

[0022] According to a data format conversion method provided by the present invention, storing the detection result as the data format detection result of the source device and the target device in the format detection result table includes:

[0023] The data transmission information between the source device and the target device is used as the key, and the detection result is used as the value, and stored in the formatted detection result table;

[0024] The data transmission information includes the data shape of the source device, the communication group identifier to which both the source device and the target device belong, and the number of the target device.

[0025] According to a data format conversion method provided by the present invention, the step of converting the format of the output data of the source device includes:

[0026] Based on the target device, the output data of the source device is format-converted. The target device is selected from the source device and the target device based on device performance.

[0027] The present invention also provides a data format conversion system, comprising:

[0028] The lookup unit is used to look up the data format detection results of the source device and the target device from the format detection result table based on the data transmission information of the source device and the target device;

[0029] The detection unit is configured to perform data format matching detection based on the data shape of the source device and the data shape of the target device when no data format detection result is found, obtain a detection result, and store the detection result as the data format detection result of the source device and the target device in the format detection result table;

[0030] The conversion unit is used to convert the output data of the source device to obtain target data when the detection result is mismatched, and to transmit the target data as input data of the target device, wherein the format of the target data matches the data format of the target device.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data format conversion method as described above.

[0032] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data format conversion method as described above.

[0033] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data format conversion method as described above.

[0034] The data format conversion method, apparatus, medium, and program products provided by this invention, based on the data transmission information of the source and target devices, can retrieve the data format detection results of the source and target devices from a format detection result table. If no data format detection result is found, data format matching detection is performed based on the data shapes of the source and target devices. This automatically and accurately determines whether the data formats of the source and target devices match, enabling subsequent processing decisions. When the detection result is a mismatch, format conversion is performed on the output data of the source device to ensure that the format of the converted target data matches the data format of the target device. This ensures smooth data transmission between heterogeneous devices, avoids transmission errors or failures caused by data format mismatch, achieves efficient and flexible cross-device communication, and thus ensures the normal progress of model training. Furthermore, by storing the detection results in a format detection result table, repeated detection can be avoided in subsequent data transmission processes, thereby improving data transmission efficiency. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the computing system for heterogeneous mixed training of models provided by the present invention;

[0037] Figure 2 This is a schematic diagram of model layer data transmission provided by the present invention;

[0038] Figure 3 This is a flowchart illustrating the data format conversion method provided by the present invention;

[0039] Figure 4 This is a flowchart illustrating the data format conversion method for heterogeneous GPUs provided by the present invention;

[0040] Figure 5 This is a schematic diagram of the data format conversion system provided by the present invention;

[0041] Figure 6This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] As the parameter scale and training data volume of large language models increase, the demand for storage resources and hardware computing power has exceeded the upper limit provided by a single computing device. To overcome this limitation and improve training throughput, a 3D parallel training strategy is often adopted, which involves distributed collaborative training of the model through data parallelism, tensor parallelism, and pipeline parallelism. Here, distributed collaborative training refers to utilizing the cooperation between multiple computing devices to jointly participate in the model training process, thereby accelerating training and improving performance. It should be noted that the computing device can be a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a tensor processing unit (TPU), etc. This invention mainly uses a GPU as an example for introduction.

[0044] Pipeline parallelism in 3D parallel training is a collaborative training technique that splits the model layer by layer across multiple computing devices. Each device is responsible for a different stage of the model and processes different parts of the model parameters. The implementation principle of pipeline parallelism is introduced below:

[0045] (1) Model partitioning: The pipelined parallel training method divides the entire model into multiple parts, each part is called a pipeline stage. For example, if the pipeline parallelism is set to 4, then the model will be divided into 4 parts, each part is handled by a consecutive GPU.

[0046] (2) Data flow: During the forward propagation process, data flows sequentially through each pipeline stage. Each stage processes a portion of the data. After processing, the result is passed to the next GPU for the next stage of processing.

[0047] (3) Gradient backpropagation: During the backpropagation process, the gradient flows through each pipeline stage in reverse order, and each stage updates the model parameters based on the received gradient.

[0048] In a homogeneous GPU cluster, where all devices are from the same manufacturer and are of the same model, pipelined parallel training ensures smooth data transfer between devices due to their consistent hardware characteristics and data formats, thus guaranteeing the normal progress of the training process. It should be understood that data format refers to the structure and method used when data is stored or transmitted in a device. This can include data order and storage order. Data order refers to the logical arrangement of data. For example, suppose a GPU needs to perform matrix multiplication on a three-dimensional tensor. The logical arrangement of this tensor (i.e., data order) could be [B, H, W]. Here, B, H, and W represent the data in different dimensions of the tensor, respectively. During matrix multiplication, these data will be retrieved sequentially according to the logical order of B, H, and W for the operation. Storage order refers to the arrangement of data on the physical storage medium.

[0049] However, in heterogeneous GPU clusters, which contain devices from different manufacturers and models, differences in hardware characteristics lead to variations in data formats. This difference can cause format mismatches during training data transfer between devices, affecting the normal progress of training. Data mismatch not only increases the complexity of data transmission but can also lead to data errors or loss during training, ultimately impacting the model's final performance and accuracy. Therefore, achieving efficient and accurate data transmission between heterogeneous devices has become a pressing issue in the field of heterogeneous model training.

[0050] To address this issue, the present invention provides a data format conversion method aimed at solving the problem of inconsistent data formats in heterogeneous GPU clusters. By introducing an intelligent data format conversion mechanism, it can automatically identify and adapt to the data format requirements of different devices, thereby eliminating the problem of data format mismatch, ensuring seamless collaboration between heterogeneous GPU devices, guaranteeing the continuity and stability of the training process, and thus overcoming the aforementioned defects.

[0051] Figure 1 This is a schematic diagram of the computing system for heterogeneous model training provided by the present invention, as shown below. Figure 1As shown, the system may include a host 110 and a device 120. The device 120 may include multiple first-type devices, multiple second-type devices, and other types of devices. The host 110 is connected to the multiple devices on the device 120 to control these devices to perform various computing tasks and collaboratively complete the training of the model. On the device 120, the multiple first-type devices are all from the same manufacturer and of the same model, forming a homogeneous device cluster. The first-type and second-type devices may be from different manufacturers or of different models, forming a heterogeneous device cluster. Model heterogeneous training involves using these different types of devices (including homogeneous and heterogeneous devices) to participate in the model training task. These devices may have different computing capabilities, memory sizes, data formats, and other characteristics, but through the control and coordination of the host, they can work together to complete the training of the model. Heterogeneous training technology, which enables collaborative training of multiple heterogeneous devices, has become a key technology for solving the problem of isolated heterogeneous computing power.

[0052] Figure 2 This is a schematic diagram of model layer data transmission provided by the present invention, as shown below. Figure 2 As shown, the basic building block of a large language model is the Transformer Layer. A complete model typically consists of multiple sequentially stacked Transformer Layers, interconnected through input-output data streams. With a large number of parameters, to achieve efficient parallel training, different layers of the model can be distributed across multiple computing devices, employing a pipelined parallel strategy for model training. During this process, the devices collaboratively train by passing activation values ​​and gradient values. Figure 2 The diagram shows two adjacent layers of the model, namely the preceding and following model layers. Both of these layers are Transformer Layers. If these two layers are allocated to different computing devices, the output of the preceding layer needs to be passed as the input of the following layer. This data has a specific data order and storage order. For example, suppose... Figure 2 The first model layer is assigned to device A, and the second model layer is assigned to device B. Device A is from vendor A and supports the data format [s, b, h], while device B is from vendor B and supports the data format [b, s, h]. Although the data in these two formats is the same size (i.e., ... Although they are identical, due to differences in data order, they are not directly compatible when transmitted between device A and device B. Direct transmission may lead to data errors or loss. It should be noted that s represents the sequence dimension, also known as the sequence length, which refers to the number of elements in a single data sample; b represents the batch dimension, which refers to the number of data samples processed simultaneously during a model update; and h represents the hidden layer dimension, which refers to the number of neurons in the hidden layer (neither the input layer nor the output layer) of the model.

[0053] This invention proposes an automated data format conversion method to solve the problem of inconsistent data formats between heterogeneous devices during the above-mentioned heterogeneous mixed training of models. Figure 3 This is a flowchart illustrating the data format conversion method provided by the present invention, as shown below. Figure 3 As shown, the method includes:

[0054] Step 310: Based on the data transmission information of the source device and the target device, search for the data format detection results of the source device and the target device in the format detection result table.

[0055] It should be noted that the method provided in this embodiment of the invention can be applied to a data format conversion system. This system can be deployed on the host in the above-mentioned computing system or on various devices at the device end. This embodiment of the invention does not make any specific limitations in this regard.

[0056] Specifically, a source device refers to a device that generates data and needs to send it to another device, while a target device refers to a device that receives data from the source device. In a heterogeneous model training scenario, the source device can be the device containing the preceding layer of the model, and the target device is the device containing the following layer. The data transmission information of the source and target devices refers to information related to data transmission. This information can be used as an index to look up the data format detection results of the source and target devices in the format detection result table.

[0057] It is understandable that data transmission information may include the shape of the output data from the source device, the communication group identifier shared by the source and target devices, and the target device's identifier. The shape of the output data from the source device refers to the shape of the data output by the source device. Here, the shape of the data refers to the organization of the data in memory, including the dimensions of the data and the size of each dimension. For example, the shape of the output data from the source device can be represented as [s, b, h]. The communication group identifier shared by the source and target devices is a unique identifier for the communication group they belong to. In heterogeneous GPU clusters, devices are organized into different communication groups for more efficient data transmission. For example, during pipelined parallel training, the source and target devices can coexist on a pipelined parallel communication group. This unique identifier is used to distinguish between different communication groups and ensure that data is accurately sent to the target device within the correct communication group. The target device's identifier is a unique identifier for the target device within the communication group. This identifier ensures that data is accurately sent to the specified target device.

[0058] Based on the aforementioned data transmission information, the data format detection results for the source and target devices can be accurately retrieved from the format detection result table. Specifically, the system uses the data transmission information of the source and target devices as query conditions to search for the corresponding records in the format detection result table. Here, the format detection result table is a data structure used to store the detection results of whether the data formats of different devices match. For example, the format detection result table can be a hash table, where the key of each element can be the data transmission information of the source and target devices, and the value is the data format detection result of the source and target devices.

[0059] The data format detection result is a flag indicating whether the data formats of the source device and the target device match. It can be a Boolean value (such as true or false) or a string (such as match or no match), and this embodiment of the invention does not specifically limit it. It should be understood that if the data format detection result is a match, it means that the output data format of the source device matches the input data format of the target device, and data transmission can be performed directly; if the data format detection result is a mismatch, it means that the output data format of the source device does not match the input data format of the target device, and data format conversion is required.

[0060] Step 320: If no data format detection result is found, perform data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result, and store the detection result as the data format detection result of the source device and the target device in the format detection result table.

[0061] Specifically, if the system fails to find the data format detection result between the source and target devices in the format detection result table, it indicates that the data format matching between the source and target devices has never been checked before. In this case, the system can perform data format matching detection based on the data shape of the source and target devices. Here, the data shape of the source device refers to the shape of the output data of the source device, and the data shape of the target device refers to the shape of the input data of the target device.

[0062] Data format matching detection is a process used to determine whether the output data format of a source device matches the input data format of a target device. Specifically, it can be implemented through the following steps: First, compare the shape of the output data from the source device with the shape of the input data from the target device, including the dimensions, size, and order of the data. Based on the comparison result, it can be determined whether the output data format of the source device matches the input data format of the target device, thus obtaining the corresponding detection result. For example, if the dimensions, size, and order of the output data from the source device are consistent with the dimensions, size, and order of the input data from the target device, the detection result is determined to be a match; otherwise, the detection result is determined to be a mismatch.

[0063] Furthermore, after obtaining the detection result, the system can use the detection result as the value and the data transmission information of the source and target devices corresponding to the detection result as the key to create a new record in the format detection result table. This stores the data format detection results of the source and target devices in the format detection result table, so that the system can quickly find and determine the data format matching between the source and target devices during future data transmission. This avoids the tedious data format matching detection process during each data transmission, reduces the consumption of computing resources and time, and improves the efficiency of data transmission.

[0064] Furthermore, if the system successfully finds the data format detection result between the source and target devices in the format detection result table, it indicates that a format check for data format mismatch between the source and target devices has already been performed, and the result has been recorded in the format detection result table. In this case, the detection step can be skipped, and the appropriate transmission path can be selected directly based on the found data format detection result. This avoids repeated detection and improves data transmission efficiency. Specifically, if the result shows that the data format matches, data transmission can proceed directly; if the result shows that the data format does not match, data format conversion needs to be performed before data transmission.

[0065] Step 330: If the detection result is a mismatch, the output data of the source device is converted to obtain target data, and the target data is used as the input data of the target device for data transmission. The format of the target data matches the data format of the target device.

[0066] Specifically, if the detection result is a match, it indicates that the output data format of the source device matches the input data format of the target device. In this case, data transmission can be performed directly without any format conversion. If the detection result is a mismatch, it indicates that there is a mismatch between the output data format of the source device and the input data format of the target device. For example, there may be a mismatch in the number of data dimensions, data size, or data order. In this case, it is necessary to convert the output data of the source device to conform to the input data format requirements of the target device.

[0067] Specifically, when converting the output data of the source device, the system first determines appropriate format conversion rules based on the data format differences between the source and target devices. These rules may include data dimension adjustment, data order adjustment, etc. Then, the system performs format conversion on the output data of the source device according to the determined conversion rules. Finally, the system verifies whether the converted data meets the input data format requirements of the target device. It should be understood that when the detection result is a mismatch, if it is a mismatch in the number of data dimensions or the data size, it usually indicates that the source and target devices are not in the same communication group, requiring a return check. In this case, subsequent data format conversion and data transmission processes are not executed. Therefore, this embodiment of the invention mainly addresses data format conversion in the case of data order mismatch.

[0068] As can be understood, target data refers to the source device's output data, after format conversion, that conforms to the input data format requirements of the target device. In other words, target data is the result of format conversion of the source device's output data, and it can be directly received and processed by the target device. After the target data is generated and verified to conform to the target device's input data format requirements, it can be used as the target device's input data for data transmission; that is, the target data is sent from the source device to the target device through the data transmission channel.

[0069] The method provided in this invention, based on data transmission information from the source and target devices, can retrieve data format detection results from a format detection result table. If no data format detection result is found, data format matching detection is performed based on the data shapes of the source and target devices. This automatically and accurately determines whether the data formats of the source and target devices match, enabling subsequent processing decisions. When the detection result is a mismatch, format conversion is performed on the output data of the source device to ensure that the format of the converted target data matches that of the target device. This ensures smooth data transmission between heterogeneous devices, avoids transmission errors or failures caused by data format mismatches, achieves efficient and flexible cross-device communication, and thus ensures the normal progress of model training. Furthermore, storing the detection results in a format detection result table avoids repeated detection during subsequent data transmission, thereby improving data transmission efficiency.

[0070] Based on the above embodiments, step 330, which involves converting the format of the output data from the source device to obtain the target data, includes:

[0071] Step 331: Perform a full collection operation on each slice of data output by the source device to obtain the output data.

[0072] Specifically, during data processing, source devices typically divide data into several smaller parts (i.e., slices) for efficient processing. These slices can be processed or computed independently through different links on the device, and each slice contains a portion of the original data. Therefore, when converting the format of the output data from the source device, a full collection operation can first be performed on each slice of data output by the source device. Here, each slice of data output by the source device refers to the data output after processing by each link of the source device.

[0073] A full collection operation refers to the process of reassembling all slice data output from the source device into a complete dataset. This can be achieved through the following steps: First, identify and collect all slice data output from the source device; after collecting all slices, sort them according to their order in the original data and combine them into a complete dataset, thus obtaining the overall output data from the source device. Here, "output data" refers to the complete dataset output by the source device after the full collection operation.

[0074] Step 332: Perform data order conversion on the output data, and perform a distribution operation on the converted data to obtain the target data.

[0075] Specifically, after collecting the output data from the source device, a data order conversion operation can be performed on the output data to match its data order with the input data from the target device. Here, data order conversion refers to the process of converting data from one sequence or order to another. For example, the data shape of the source device's output data can be [s, b, h], and the data shape of the target device's input data can be [b, s, h]. This data order conversion can be achieved by changing the data arrangement, index order, or data storage method in memory.

[0076] Understandably, during data processing, target devices typically divide data into several slices for efficient processing. Therefore, to ensure the data obtained after data reordering is efficiently distributed across different links on the target devices, a distribution operation can be performed on the reordered data to obtain the final target data for transmission. Here, the distribution operation refers to the process of distributing data from one or more sources to multiple targets, which can be achieved through messaging systems or dedicated data distribution tools.

[0077] In this embodiment of the invention, before performing data order conversion, the consistency and accuracy of the data order conversion can be ensured by collecting the slice data output by the source device. After the data order conversion, the converted data can be correctly distributed to multiple links of the target device for parallel processing by performing a distribution operation, thereby improving data processing efficiency.

[0078] Based on any of the above embodiments, in step 320, the step of performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain the detection result includes:

[0079] Step 321: Determine the data size and data order of the source device based on the data shape of the source device.

[0080] Specifically, the data shape is typically a dimensional description of a multidimensional array (or tensor), including the dimensions of the data and the size of each dimension. For example, the output data shape of the source device can be [s, b, h], which represents a three-dimensional tensor, where s, b, and h represent the size of the sequence dimension, batch dimension, and hidden layer dimension, respectively.

[0081] Based on the data shape of the source device, the data size and data order of the source device can be determined. Here, the data size of the source device refers to the sum of all elements in the output tensor of the source device. For example, for an output tensor of shape [s, b, h], its data size is... The data order of the source device refers to the order of the various dimensions of the output tensor of the source device. In the example above, the data order of the output data of the source device is s, b, h, which means that the first dimension is the sequence dimension, the second dimension is the batch dimension, and the third dimension is the hidden layer dimension.

[0082] Step 322: Determine the data size and data order of the target terminal device based on the data shape of the target terminal device.

[0083] Specifically, the data size of the target device refers to the sum of all source ends in the input tensor of the target device, while the data order of the target device refers to the order of the various dimensions of the input tensor. For example, the shape of the input data of the target device can be represented as [b, s, h], where b, s, and h represent the size of the batch dimension, sequence dimension, and hidden layer dimension, respectively. For this input data tensor, its data size is... The data order is b, s, h, indicating that the first dimension is the batch dimension, the second dimension is the sequence dimension, and the third dimension is the hidden layer dimension.

[0084] Step 323: Compare the data size of the source device with the data size of the target device to obtain a first comparison result, and compare the data sequence of the source device with the data sequence of the target device to obtain a second comparison result.

[0085] Specifically, the first comparison result refers to the result obtained by comparing the data size of the source device with the data size of the target device. This comparison is to check whether the data sizes of the two devices are consistent. In the example above, the data size of the source device is... The data size of the target device is Clearly, the data sizes of the two are the same.

[0086] The second comparison result refers to the result obtained by comparing the data order of the source device with the data order of the target device. This comparison is to check whether the data dimensions and order of the two devices are consistent. Specifically, the data order of the source device (e.g., s, b, h) and the data order of the target device (e.g., b, s, h) can be compared one by one to check whether each dimension exists and whether their order is the same, thus obtaining the second comparison result.

[0087] Step 324: Determine the detection result based on the first comparison result and the second comparison result.

[0088] Specifically, based on the first and second comparison results, it can be determined whether the output data format of the source device matches the input data format of the target device, thereby guiding subsequent data transmission and format conversion operations. Specifically, if the first comparison result indicates that the data size is consistent, and the second comparison result indicates that the data order is also consistent, the detection result is a match. In this case, the output data of the source device can be directly used as the input data of the target device for data transmission. If the first or second comparison result indicates that they are inconsistent, the detection result is a mismatch. In this case, the output data of the source device needs to be format converted to match the input data format of the target device. It should be understood that when the first comparison result indicates that the data size is inconsistent, it usually means that the communication group between the source and target devices is incorrect, and subsequent data transmission and format conversion operations will not be performed in this case.

[0089] Based on the above embodiments, the method further includes:

[0090] Step 340: If the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0091] Specifically, when performing data format matching detection on the source device and the target device, if the detection result is a match, it indicates that the output data format of the source device matches the input data format of the target device. At this time, the output data of the source device can be directly sent to the target device as the input data of the target device for processing through the data transmission channel. That is, data transmission can be performed directly without any format conversion operation.

[0092] Based on any of the above embodiments, the method further includes:

[0093] If the data format detection result is found and the data format detection result is a match, then the output data of the source device is used as the input data of the target device for data transmission.

[0094] If the data format detection result is a mismatch, the output data of the source device is converted to obtain the target data, and the target data is used as the input data of the target device for data transmission.

[0095] Specifically, before data transmission occurs between the source and target devices, if the system successfully finds the data format detection result between the source and target devices in the format detection result table, it indicates that the system has already performed a format detection to check whether the data formats of the source and target devices match, and the result has been recorded in the format detection result table. In this case, the system can skip the detection step and directly select the appropriate transmission path based on the found data format detection result, thereby avoiding repeated detection and improving data transmission efficiency.

[0096] If the data format detection result is a match, it indicates that the output data format of the source device and the input data format of the target device are matched. In this case, the output data of the source device can be used as the input data of the target device, and the data transmission operation can be performed directly. If the data format detection result is a mismatch, it indicates that the output data format of the source device and the input data format of the target device are not matched. In this case, a format conversion operation needs to be performed on the output data of the source device before it can be used as the input data of the target device for transmission. Here, the specific format conversion operation can be referred to the above embodiment, and will not be repeated here.

[0097] Based on any of the above embodiments, in step 320, storing the detection result as the data format detection result of the source device and the target device in the format detection result table includes:

[0098] The data transmission information between the source device and the target device is used as the key, and the detection result is used as the value, and stored in the formatted detection result table;

[0099] The data transmission information includes the data shape of the source device, the communication group identifier to which both the source device and the target device belong, and the number of the target device.

[0100] Specifically, a format detection result table can be pre-constructed to store data format detection results. This table can be a database table, hash table, dictionary, or other key-value pair storage structure. When storing the detection results in the format detection result table, a unique key is constructed using the data transmission information of the source and target devices. The value corresponding to this key is the detection result, i.e., whether the output data format of the source device matches the input data format of the target device. The constructed key and the detection result are stored as values ​​in the format detection result table. It should be understood that the data transmission information is obtained by combining the output data shape of the source device, the communication group identifier to which both the source and target devices belong, and the target device's number.

[0101] Based on any of the above embodiments, step 330, which involves format conversion of the output data from the source device, includes:

[0102] Based on the target device, the output data of the source device is format-converted. The target device is selected from the source device and the target device based on device performance.

[0103] It should be noted that when converting the output data from the source device, the conversion operation can be performed on either the source or the target device. Considering that the source and target devices are from different manufacturers or are different models, their efficiency in data format conversion will differ. To improve conversion efficiency, during the initial data format conversion between the source and target devices, the system will automatically perform performance optimization, selecting the device with the better performance to complete the conversion operation.

[0104] Specifically, the target device refers to the device selected to perform the conversion operation during data format conversion. It can be either the source device (i.e., the device that outputs data) or the target device (i.e., the device that receives the output), depending on which device is more efficient in performing the data format conversion.

[0105] During performance optimization, the system compares the data format conversion performance of the source and target devices and selects the device with the better performance as the target device to complete the conversion operation. This is achieved through the following steps: First, the system evaluates the device performance of both the source and target devices. Here, device performance mainly refers to the efficiency of the device during data format conversion, which depends on the device's cache bandwidth. To evaluate the device's cache bandwidth, the system can perform a simple test. For example, the system can allocate a block of video memory on the device and attempt to copy data from one block to another. The latency of the copy operation can be used to measure the device's cache bandwidth. The shorter the latency, the higher the device's cache bandwidth. After obtaining the cache bandwidth information of the source and target devices, the system compares these two values. The device with higher cache bandwidth will be selected as the target device to perform the subsequent data format conversion operation.

[0106] In this embodiment of the invention, through a performance optimization mechanism, the system can select a device with better performance to complete the data format conversion, thereby further optimizing resource utilization and processing speed.

[0107] Based on any of the above embodiments Figure 4 This is a flowchart illustrating the data format conversion method for heterogeneous GPUs provided by the present invention, as shown below. Figure 4 As shown in the figure, the left side is the source device and the right side is the target device. The source device and the target device process the previous and next layers of the two adjacent layers of the model, respectively, and realize the distributed processing and training of data through a series of operations.

[0108] In the source device, the multilayer perceptron is a feedforward artificial neural network model, which is mainly used to process input data and extract features useful for model training; the distribution operation is used to distribute the data processed by the multilayer perceptron to multiple links for subsequent parallel processing; the addition operation is to summarize or accumulate the data on each link; and the sending operation is to send the processed data from the source device to the target device.

[0109] On the target device, receiving operation refers to receiving data sent from the source device; normalization operation refers to standardizing the received data to conform to a certain distribution or range, which helps improve the training efficiency and stability of the model; full collection operation refers to converging the data scattered across multiple links on the target device for subsequent global processing; attention mechanism is used to process the data after full collection and extract features or information useful for model training.

[0110] Because the source and target devices may come from different manufacturers or be different models, their data formats differ, making data transmission between devices incompatible. Therefore, appropriate format conversion is required to achieve matching. To solve this problem, this invention provides an automated data format detection and conversion method, which includes:

[0111] Step S1: When transmitting and receiving communication across devices for the first time, the system will automatically detect and determine whether the data formats of the source device and the target device match based on the output data shape of the source device and the input data shape of the target device.

[0112] Step S2: If a format mismatch is detected, the system will automatically execute... Figure 4 The format conversion logic is shown by the dashed line; if a format match is detected, the data transmission operation is performed directly.

[0113] Specifically, in the format conversion logic shown by the dashed line, the main focus is on converting the data order. For example... Figure 4 As shown, on the source device, the data is distributed into slices of shape [s / 4, b, h]. Therefore, before performing data ordering, the four data samples are first collected into one data sample, which has the shape [s, b, h]. Then, the data ordering is performed on this data sample, resulting in a data sample of shape [b, s, h]. Finally, a distribution operation is performed on the resulting data sample to obtain sliced ​​data of shape [b / 4, s, h]. This sliced ​​data can be transmitted to the target device for subsequent processing and analysis.

[0114] In step S3, the system stores the detection result as the corresponding value in a hash table using the triple [shape, comm_group, dst_rank] as the key. In subsequent frequent communications, the system will skip the detection step and directly select the transmission path.

[0115] Specifically, in the triplet, `shape` refers to the shape of the output data from the source device, `comm_group` refers to the communication group identifier shared by the source and target devices, and `dst_rank` refers to the target device's ID. In subsequent frequent communications, if the system finds a match, it follows the direct transmission path; if the match is not found, it follows the transmission path shown by the dashed line, i.e., format conversion is performed first, followed by data transmission.

[0116] Step S4: The format conversion operation can be performed on either the source or target device. To improve conversion efficiency, the system will automatically perform performance optimization during the initial transmission, selecting the device with better performance to complete the conversion operation.

[0117] The method provided in this embodiment of the invention has the following advantages:

[0118] (1) Automated detection: Through automated data format detection and intelligent conversion logic, automatic identification and adaptation of data formats in cross-device communication are realized.

[0119] (2) High performance and low overhead: The system avoids duplicate detection by storing the detection results in a hash table, which significantly improves data transmission efficiency. In addition, through a performance optimization mechanism, the system can select a device with better performance to complete the data format conversion, further optimizing resource utilization and processing speed.

[0120] (3) Compatibility: When training with heterogeneous GPUs, there is no need to worry about the differences in underlying data formats. At the same time, it reduces the cost of long-term maintenance and upgrades, enabling the system to work efficiently in a variety of heterogeneous environments.

[0121] Based on any of the above embodiments Figure 5 This is a schematic diagram of the data format conversion system provided by the present invention, as shown below. Figure 5 As shown, the system includes:

[0122] The lookup unit 510 is used to look up the data format detection results of the source device and the target device from the format detection result table based on the data transmission information of the source device and the target device;

[0123] The detection unit 520 is used to perform data format matching detection based on the data shape of the source device and the data shape of the target device when no data format detection result is found, obtain a detection result, and store the detection result as the data format detection result of the source device and the target device in the format detection result table;

[0124] The conversion unit 530 is used to convert the output data of the source device to obtain target data when the detection result is mismatched, and to transmit the target data as input data of the target device, wherein the format of the target data matches the data format of the target device.

[0125] The system provided in this invention, based on data transmission information from the source and target devices, can retrieve data format detection results from a format detection result table. If no data format detection result is found, it performs data format matching detection based on the data shapes of the source and target devices. This automatically and accurately determines whether the data formats of the source and target devices match, enabling subsequent processing decisions. When the detection result is a mismatch, the output data of the source device is format-converted to match the format of the target data, ensuring smooth data transmission between heterogeneous devices and avoiding transmission errors or failures due to data format mismatch. This achieves efficient and flexible cross-device communication, thereby ensuring the normal progress of model training. Furthermore, storing the detection results in the format detection result table avoids repeated detection during subsequent data transmission, thus improving data transmission efficiency.

[0126] Based on any of the above embodiments, the conversion unit 530 includes a format conversion subunit, which is used for:

[0127] Perform a full collection operation on each slice of data output from the source device to obtain the output data;

[0128] The output data is subjected to data order transformation, and the transformed data is then distributed to obtain the target data.

[0129] Based on any of the above embodiments, the system further includes a transmission unit, which is used for:

[0130] If the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0131] Based on any of the above embodiments, the system further includes a path selection unit, which is used for:

[0132] If the data format detection result is found and the data format detection result is a match, then the output data of the source device is used as the input data of the target device for data transmission.

[0133] If the data format detection result is a mismatch, the output data of the source device is converted to obtain the target data, and the target data is used as the input data of the target device for data transmission.

[0134] Based on any of the above embodiments, the detection unit 520 includes a format matching detection subunit, which is used for:

[0135] Based on the data shape of the source device, determine the data size and data order of the source device;

[0136] Based on the data shape of the target terminal device, determine the data size and data order of the target terminal device;

[0137] The data size of the source device is compared with the data size of the target device to obtain a first comparison result, and the data order of the source device is compared with the data order of the target device to obtain a second comparison result;

[0138] The detection result is determined based on the first comparison result and the second comparison result.

[0139] Based on any of the above embodiments, the detection unit 520 further includes a result storage subunit, which is used for:

[0140] The data transmission information between the source device and the target device is used as the key, and the detection result is used as the value, and stored in the formatted detection result table;

[0141] The data transmission information includes the data shape of the source device, the communication group identifier to which both the source device and the target device belong, and the number of the target device.

[0142] Based on any of the above embodiments, the conversion unit 530 is specifically used for:

[0143] Based on the target device, the output data of the source device is format-converted. The target device is selected from the source device and the target device based on device performance.

[0144] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a data format conversion method. This method includes: based on the data transmission information of the source device and the target device, searching for data format detection results of the source device and the target device in a format detection result table; if no data format detection result is found, performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result, and storing the detection result as the data format detection result of the source device and the target device in the format detection result table; if the detection result is a mismatch, converting the format of the output data of the source device to obtain target data, and transmitting the target data as input data of the target device, wherein the format of the target data matches the data format of the target device.

[0145] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data format conversion method provided by the above methods. The method includes: searching for data format detection results of the source device and the target device from a format detection result table based on the data transmission information of the source device and the target device; if no data format detection result is found, performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result, and storing the detection result as the data format detection result of the source device and the target device in the format detection result table; if the detection result is mismatched, performing format conversion on the output data of the source device to obtain target data, and transmitting the target data as input data of the target device, wherein the format of the target data matches the data format of the target device.

[0147] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the data format conversion method provided by the above methods. The method includes: searching for data format detection results of the source device and the target device from a format detection result table based on data transmission information of the source device and the target device; if no data format detection result is found, performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result, and storing the detection result as the data format detection result of the source device and the target device in the format detection result table; if the detection result is mismatched, performing format conversion on the output data of the source device to obtain target data, and transmitting the target data as input data of the target device, wherein the format of the target data matches the data format of the target device.

[0148] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data format conversion method for heterogeneous model training, characterized in that, include: Based on the data transmission information of the source device and the target device, the data format detection results of the source device and the target device are retrieved from the format detection result table. The source device and the target device are heterogeneous devices from different manufacturers or different models. The data transmission information includes the data shape of the source device. If the data format detection result is not found, a data format matching detection is performed based on the data shape of the source device and the data shape of the target device to obtain a detection result. The detection result is then stored in the format detection result table as the data format detection result of the source device and the target device. The data format matching detection includes data size comparison and data order comparison. If the detection result indicates a data order mismatch, the output data of the source device is format-converted based on the target device to obtain target data. The target data is then used as input data for the target device for data transmission. The format of the target data matches the data format of the target device. The data format includes data order, which refers to the logical arrangement of data. The target device is selected from the source device and the target device based on the device's cache bandwidth performance. The process of converting the output data of the source device to obtain the target data includes: Perform a full collection operation on each slice of data output from the source device to obtain the output data; The output data is subjected to data order transformation, and the transformed data is subjected to a distribution operation to obtain the target data. The data order transformation refers to the process of converting data from one sequence or order to another sequence or order.

2. The data format conversion method according to claim 1, characterized in that, Also includes: If the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

3. The data format conversion method according to claim 1, characterized in that, Also includes: If the data format detection result is found and the data format detection result is a match, then the output data of the source device is used as the input data of the target device for data transmission. If the data format detection result is a mismatch, the output data of the source device is converted to obtain the target data, and the target data is used as the input data of the target device for data transmission.

4. The data format conversion method according to claim 1, characterized in that, The step of performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain the detection result includes: Based on the data shape of the source device, determine the data size and data order of the source device; Based on the data shape of the target terminal device, determine the data size and data order of the target terminal device; The data size of the source device is compared with the data size of the target device to obtain a first comparison result, and the data order of the source device is compared with the data order of the target device to obtain a second comparison result; The detection result is determined based on the first comparison result and the second comparison result.

5. The data format conversion method according to any one of claims 1 to 4, characterized in that, The step of storing the detection result as the data format detection result of the source device and the target device in the format detection result table includes: The data transmission information between the source device and the target device is used as the key, and the detection result is used as the value, and stored in the formatted detection result table; The data transmission information also includes the communication group identifier to which both the source device and the target device belong, as well as the number of the target device.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data format conversion method as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data format conversion method as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the data format conversion method as described in any one of claims 1 to 5.

Images