Data format conversion method and device, medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a data format conversion method and device, a medium and a program product, and the method comprises the steps: searching a format detection result table for data format detection results of a source end device and a target end device based on data transmission information of the source end device and the target end device; under the condition that the data format detection result is not found, performing data format matching detection based on the data shape of the source end equipment and the data shape of the target end equipment to obtain a detection result; and under the condition that the detection result is mismatching, performing format conversion on the output data of the source end equipment to obtain target data, and performing data transmission by taking the target data as the input data of the target end equipment. Through automatic data format matching detection and intelligent conversion logic, automatic identification and adaptation of data formats in cross-device communication are realized, so that efficient and accurate transmission of data is ensured.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, in particular to the field of data transmission between heterogeneous devices, and more particularly to a data format conversion method, device, medium and program product. Background Art

[0002] With the rapid development of artificial intelligence technology and the widespread application of large language models (LLM), the parameter scale of the model and the amount of training data are increasing, resulting in an increasing demand for computing resources and an increase in the number of computing devices required. Therefore, multi-device heterogeneous mixed training (also known as "heterogeneous training") has become an indispensable model training method. Pipeline-based parallel heterogeneous mixed training is the current mainstream mode of heterogeneous mixed training. It distributes different layers of the model to multiple devices so that these layers can process data in parallel, thereby improving training efficiency.

[0003] However, these devices may come from different manufacturers or different product models of the same manufacturer, with different hardware characteristics, forming a heterogeneous device cluster. In this cluster, due to the differences in hardware characteristics between different devices, the data formats between devices are different. This difference causes format mismatch problems when training data is transmitted between devices, thus affecting the normal progress of the training process. Summary of the invention

[0004] The present invention provides a data format conversion method, device, medium and program product, which are used to solve the defect of data format mismatch between heterogeneous devices in the related art, which affects model training.

[0005] The present invention provides a data format conversion method, comprising: Based on the data transmission information of the source device and the target device, searching the data format detection results of the source device and the target device from the format detection result table; If the data format detection result is not found, performing data format matching detection based on the data shape of the source-end device and the data shape of the target-end device to obtain a detection result, and storing the detection result as the data format detection result of the source-end device and the target-end device in the format detection result table; When the detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0006] According to a data format conversion method provided by the present invention, the format conversion of the output data of the source device to obtain the target data includes: Performing a full collection operation on each slice data output by the source device to obtain output data; The output data is subjected to data sequence conversion, and a distribution operation is performed on the converted data to obtain the target data.

[0007] A data format conversion method provided by the present invention also includes: When the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0008] A data format conversion method provided by the present invention also includes: In the case where the data format detection result is found, if the data format detection result is a match, the output data of the source end device is used as the input data of the target end device for data transmission; If the data format detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission.

[0009] According to a data format conversion method provided by the present invention, the data format matching detection is performed based on the data shape of the source end device and the data shape of the target end device to obtain the detection result, including: Determine the data size and data sequence of the source device based on the data shape of the source device; Determine the data size and data sequence of the target device based on the data shape of the target device; Compare the data size of the source device with the data size of the target device to obtain a first comparison result, and compare the data sequence of the source device with the data sequence of the target device to obtain a second comparison result; The detection result is determined based on the first comparison result and the second comparison result.

[0010] According to a data format conversion method provided by the present invention, storing the detection result as the data format detection result of the source device and the target device in the format detection result table includes: Using the data transmission information of the source device and the target device as a key and the detection result as a value, storing them in the format detection result table; The data transmission information includes the data shape of the source device, the communication group identifier to which the source device and the target device belong, and the serial number of the target device.

[0011] According to a data format conversion method provided by the present invention, the format conversion of the output data of the source device includes: The output data of the source device is format-converted based on the target device, and the target device is selected from the source device and the target device based on device performance.

[0012] The present invention also provides a data format conversion system, comprising: A searching unit, configured to search the data format detection results of the source end device and the target end device from the format detection result table based on the data transmission information of the source end device and the target end device; a detection unit, configured to, if the data format detection result is not found, perform a data format matching detection based on the data shape of the source-end device and the data shape of the target-end device to obtain a detection result, and store the detection result as the data format detection result of the source-end device and the target-end device in the format detection result table; A conversion unit is used to convert the format of the output data of the source device to obtain target data when the detection result is a mismatch, and use the target data as the input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned data format conversion methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data format conversion method described in any one of the above methods is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the data format conversion method described above is implemented.

[0016] The data format conversion method, device, medium and program product provided by the present invention can search the data format detection results of the source device and the target device from the format detection result table based on the data transmission information of the source device and the target device, and when the data format detection result is not found, the data format matching detection is performed based on the data shape of the source device and the data shape of the target device, so as to automatically and accurately determine whether the data formats of the source device and the target device match, so as to make subsequent processing decisions. When the detection result is mismatched, by performing format conversion on the output data of the source device, the format of the converted target data can be matched with the data format of the target device, thereby ensuring the smooth transmission of data between heterogeneous devices, avoiding transmission errors or failures caused by data format mismatch, realizing efficient and flexible cross-device communication, and then ensuring the normal progress of model training. In addition, by storing the detection results in the format detection result table, repeated detection can be avoided in the subsequent data transmission process, thereby improving data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the structure of a computing system for heterogeneous mixed training of models provided by the present invention; Figure 2 It is a schematic diagram of model layer data transmission provided by the present invention; Figure 3 It is a flow chart of the data format conversion method provided by the present invention; Figure 4 It is a flow chart of a data format conversion method for heterogeneous GPUs provided by the present invention; Figure 5 It is a structural schematic diagram of the data format conversion system provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] As the parameter scale and training data volume of large language models increase, the demand for storage resources and hardware computing power has exceeded the upper limit that a single computing device can provide. In order to break through this limitation and improve the training throughput at the same time, a 3D parallel training strategy is often adopted, that is, distributed collaborative training of the model is implemented through data parallelism, tensor parallelism, and pipeline parallelism. Here, distributed collaborative training refers to the use of collaborative cooperation between multiple computing devices to jointly participate in the training process of the model to achieve model training acceleration and performance improvement. It should be noted that the computing device can be a graphics processing unit (GPU), a general-purpose computing on Graphics Processing Units (GPGPU), a tensor processor (TPU), etc. The present invention is mainly introduced by taking the computing device as a GPU as an example.

[0021] Pipeline parallelism in 3D parallel training is a collaborative training technology that splits the model by layer on multiple computing devices. Each device is responsible for different stages of the model and processes different parts of the model parameters. The following is an introduction to the implementation principle of pipeline parallelism: (1) Model division: The pipeline parallel training method divides the entire model into multiple parts, each of which is called a pipeline stage. For example, if the pipeline parallelism is set to 4, the model will be divided into 4 parts, each of which is handled by a consecutive GPU.

[0022] (2) Data flow: During the forward propagation process, data flows sequentially through each pipeline stage. Each stage processes a portion of the data. After processing, the result is passed to the next GPU for processing in the next stage.

[0023] (3) Gradient backpropagation: During the backward propagation process, the gradient flows through each pipeline stage in reverse order, and each stage updates the model parameters based on the received gradient.

[0024] In a homogeneous GPU cluster, that is, when all devices are from the same manufacturer and have the same model, when pipeline parallel training is performed, since all devices have consistent hardware characteristics and data formats, it can ensure the smooth transmission of data between different devices, thereby ensuring the normal progress of the training process. It should be understood that the data format refers to the structure and method used when data is stored or transmitted in the device, which may include data sequence and storage sequence, where the data sequence refers to the logical arrangement order of data. For example, assuming that a GPU device needs to perform a matrix multiplication operation on a three-dimensional tensor, the logical arrangement order of the tensor (i.e., data sequence) can be [B, H, W], where B, H, and W represent the data of the tensor in different dimensions, respectively. When performing a matrix multiplication operation, these data will be taken out in sequence according to the logical order of B, H, and W for operation. The storage sequence refers to the arrangement order of data on the physical storage medium.

[0025] However, in a heterogeneous GPU cluster, which includes devices from different manufacturers and models, the data formats are different due to differences in hardware characteristics between the devices. This difference may cause format mismatches in the transmission of training data between devices, thus affecting the normal progress of the training process. Data mismatch not only increases the complexity of data transmission, but may also cause data errors or loss during training, which in turn affects the final performance and accuracy of the model. Therefore, how to achieve efficient and accurate data transmission between heterogeneous devices has become an urgent problem to be solved in the current field of heterogeneous mixed training of models.

[0026] In this regard, the present invention provides a data format conversion method, which aims to solve the problem of inconsistent data formats in heterogeneous GPU clusters. By introducing an intelligent data format conversion mechanism, it can automatically identify and adapt the data format requirements of different devices, thereby eliminating the problem of data format mismatch, ensuring that heterogeneous GPU devices can collaborate seamlessly, and ensuring the continuity and stability of the training process, thereby overcoming the above-mentioned defects.

[0027] Figure 1 is a schematic diagram of the structure of a computing system for model heterogeneous mixed training provided by the present invention, such as Figure 1As shown, the system may include a host 110 and a device end 120, wherein the device end 120 may include multiple first-type devices, multiple second-type devices, and other types of devices. The host 110 is connected to multiple devices of the device end 120 to control these devices to perform various computing tasks and jointly complete the training of the model. At the device end 120, multiple first-type devices are devices of the same manufacturer and the same model, and these devices constitute a homogeneous device cluster, while the first-type devices and the second-type devices may be devices of different manufacturers or different models, and these devices constitute a heterogeneous device cluster. Model heterogeneous mixed training is to use these different types of devices (including homogeneous and heterogeneous devices) to participate in the model training task during the model training process. These devices may have different computing capabilities, memory sizes, data formats and other characteristics, but they can work together through the control and coordination of the host to complete the training of the model. The heterogeneous mixed training technology of collaborative training of multiple heterogeneous devices has become a key technology to solve the problem of heterogeneous computing islands.

[0028] Figure 2 is a schematic diagram of the model layer data transmission provided by the present invention, such as Figure 2 As shown in the figure, the basic building block of a large language model is the Transformer Layer. A complete model is usually composed of multiple serially stacked Transformer Layers, and each layer is connected to each other through input and output data streams. In the case of a large number of parameters, in order to achieve efficient parallel training, different layers of the model can be assigned to multiple computing devices, and the model training is carried out using a pipeline parallel strategy. In this process, each device cooperates in training by transferring activation values ​​and gradient values. Figure 2 The following are two adjacent layers of the model, namely the previous model layer and the next model layer. Both of these model layers are Transformer Layers. If these two layers are assigned to different computing devices, the output of the previous layer needs to be passed as the input of the next layer. These data have a specific data order and storage order. For example, assuming Figure 2 The first model layer is assigned to device A, and the second model layer is assigned to device B. Device A comes from manufacturer A and supports the data format [s, b, h], while device B comes from manufacturer B and supports the data format [b, s, h]. Although the data in these two formats are the same in size (i.e. The data are the same, but due to the difference in data sequence, they are not directly compatible when transmitted between device A and device B. If they are transmitted directly, data errors or losses may occur. It should be noted that s represents the sequence dimension, also known as the sequence length, which refers to the number of elements in a single data sample; b represents the batch dimension, which refers to the number of data samples processed simultaneously during a model update; h represents the hidden layer dimension, which refers to the number of neurons in the hidden layer (neither the input layer nor the output layer) in the model.

[0029] The present invention proposes an automated data format conversion method for solving the problem of inconsistent data formats between heterogeneous devices during the heterogeneous mixed training of the above-mentioned model. Figure 3 is a flow chart of the data format conversion method provided by the present invention, such as Figure 3 As shown, the method includes: Step 310: Based on the data transmission information of the source device and the target device, search the data format detection results of the source device and the target device from the format detection result table.

[0030] It should be noted that the method provided in the embodiment of the present invention can be applied to a data format conversion system, which can be deployed on a host in the above-mentioned computing system or on various devices on the device side, and the embodiment of the present invention does not specifically limit this.

[0031] Specifically, the source device refers to a device that generates data and needs to send it to another device, and the target device refers to a device that receives data from the source device. In the model heterogeneous mixed training scenario, the source device can be the device where the first layer of two adjacent layers of the model is located, and the target device is the device where the second layer is located. The data transmission information of the source device and the target device refers to information related to data transmission, which can be used as an index to find the data format detection results of the source device and the target device from the format detection result table.

[0032] It can be understood that the data transmission information may include the output data shape of the source device, the communication group identifier to which the source device and the target device belong together, and the number of the target device, etc., wherein the output data shape of the source device refers to the shape of the data output by the source device. Here, the shape of the data refers to the organizational structure of the data in the memory, including the dimension of the data and the size of each dimension. For example, the shape of the data output by the source device can be expressed as [s, b, h]. The communication group identifier to which the source device and the target device belong together refers to the unique identifier of the communication group to which the source device and the target device belong together. In a heterogeneous GPU cluster, devices are organized into different communication groups to transmit data more efficiently. For example, in the pipeline parallel training process, the source device and the target device can be together in the pipeline parallel communication group. This unique identifier is used to distinguish different communication groups and ensure that data can be accurately sent to the target device in the correct communication group. The number of the target device refers to an identifier that uniquely identifies the target device in the communication group, which is used to ensure that data can be accurately sent to the specified target device.

[0033] According to the above data transmission information, the data format detection results corresponding to the source device and the target device can be accurately found from the format detection result table. Specifically, the system uses the data transmission information of the source device and the target device as the query condition to find the corresponding record in the format detection result table. Here, the format detection result table is a data structure used to store the detection results of whether the data formats between different devices match. For example, the format detection result table can be a hash table, and the key (Key) of each element in the table can be the data transmission information of the source device and the target device, and the value is the data format detection result of the source device and the target device.

[0034] The data format detection result is a mark indicating whether the data format between the source device and the target device matches. It can be a Boolean value (such as true or false) or a string (such as match, mismatch), and the embodiment of the present invention does not specifically limit this. It should be understood that if the data format detection result is a match, it means that the output data format of the source device matches the input data format of the target device, and data transmission can be performed directly; if the data format detection result is a mismatch, it means that the output data format of the source device does not match the input data format of the target device, and data format conversion is required.

[0035] Step 320, when the data format detection result is not found, a data format matching detection is performed based on the data shape of the source device and the data shape of the target device to obtain a detection result, and the detection result is stored in the format detection result table as the data format detection result of the source device and the target device.

[0036] Specifically, if the system fails to find the data format detection result between the source device and the target device in the format detection result table, it indicates that whether the data format between the source device and the target device matches has never been checked before. In this case, the system can perform data format matching detection based on the data shape of the source device and the data shape of the target device. Here, the data shape of the source device refers to the shape of the output data of the source device, and the data shape of the target device refers to the shape of the input data of the target device.

[0037] Data format matching detection is a process used to determine whether the output data format of the source device matches the input data format of the target device. Specifically, the following steps can be used to implement data format matching detection: First, compare the output data shape of the source device and the input data shape of the target device, including the dimension, size, and arrangement order of the data; based on the comparison result, it can be determined whether the output data format of the source device matches the input data format of the target device, thereby obtaining the corresponding detection result. For example, if the dimension, size, and data sequence of the output data of the source device are consistent with the dimension, size, and data sequence of the input data of the target device, the detection result can be determined to be a match, otherwise the detection result is determined to be a mismatch.

[0038] Furthermore, after obtaining the detection result, the system can use the detection result as a value and the data transmission information of the source device and the target device corresponding to the detection result as a key to create a new record in the format detection result table, thereby storing the data format detection results of the source device and the target device in the format detection result table, so that the system can quickly find and determine the data format matching between the source device and the target device in future data transmission processes, thereby avoiding tedious data format matching detection processes during each data transmission, reducing the consumption of computing resources and time, and improving the efficiency of data transmission.

[0039] In addition, if the system successfully finds the data format detection result between the source device and the target device in the format detection result table, it indicates that the format detection has been performed before to determine whether the data format between the source device and the target device matches, and the result has been recorded in the format detection result table. In this case, the detection step can be skipped, and the corresponding transmission path can be directly selected based on the data format detection result found, thereby avoiding repeated detection and improving data transmission efficiency. Specifically, if the result found indicates that the data format matches, data transmission can be performed directly. If the result found indicates that the data format does not match, data format conversion needs to be performed before data transmission.

[0040] Step 330, when the detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0041] Specifically, if the detection result is a match, it indicates that the output data format of the source device matches the input data format of the target device, and data can be directly transmitted without any format conversion. If the detection result is a mismatch, it indicates that there is a mismatch between the output data format of the source device and the input data format of the target device, for example, there may be a mismatch in the number of data dimensions, a mismatch in data size, or a mismatch in data sequence, etc. In this case, the output data of the source device needs to be format converted to meet the input data format requirements of the target device.

[0042] Specifically, when the output data of the source device is formatted, first, the system will determine appropriate format conversion rules based on the data format differences between the source device and the target device. These rules may include data dimension adjustment, data sequence adjustment, etc. Then, the system performs format conversion operations on the output data of the source device according to the determined conversion rules. Finally, the system can verify whether the converted data meets the input data format requirements of the target device. It should be understood that when the detection result is not a match, if the number of data dimensions does not match or the data size does not match, it usually indicates that the source device and the target device are not in the same communication group and need to be returned for inspection. At this time, the subsequent data format conversion and data transmission process are not executed. Therefore, the embodiment of the present invention mainly performs data format conversion for the case of data sequence mismatch.

[0043] It can be understood that the target data refers to the output data of the source device after format conversion that meets the input data format requirements of the target device. In other words, the target data is the result of the format conversion of the output data of the source device, and it can be directly received and processed by the target device. After the target data is generated and verified to meet the input data format requirements of the target device, it can be used as the input data of the target device for data transmission, that is, the target data is sent from the source device to the target device through the data transmission channel.

[0044] The method provided by the embodiment of the present invention can search the data format detection results of the source device and the target device from the format detection result table based on the data transmission information of the source device and the target device, and when the data format detection result is not found, the data format matching detection is performed based on the data shape of the source device and the data shape of the target device, so as to automatically and accurately determine whether the data formats of the source device and the target device match, so as to make subsequent processing decisions. When the detection result is mismatched, by format conversion of the output data of the source device, the format of the converted target data can be matched with the data format of the target device, thereby ensuring the smooth data transmission between heterogeneous devices, avoiding transmission errors or failures caused by data format mismatch, and realizing efficient and flexible cross-device communication, thereby ensuring the normal progress of model training. In addition, by storing the detection results in the format detection result table, repeated detection can be avoided in the subsequent data transmission process, thereby improving data transmission efficiency.

[0045] Based on the above embodiment, in step 330, the format conversion of the output data of the source device to obtain the target data includes: Step 331, performing a full collection operation on each slice data output by the source device to obtain output data.

[0046] Specifically, during the data processing process, the source device usually splits the data into several smaller parts (i.e., slices) for processing in order to process the data efficiently. These sliced ​​data can be processed or calculated independently through different links of the device, and each slice contains a part of the original data. Therefore, when converting the format of the output data of the source device, a full collection operation can be performed on each slice data output by the source device first. Here, each slice data output by the source device refers to the data output after each link of the source device is processed.

[0047] The full collection operation refers to the process of reintegrating all the slice data output by the source device into a complete data set. This can be achieved through the following steps: First, identify and collect all the slice data output by the source device; after collecting all the slices, sort them according to their order in the original data, and combine them into a complete data set to obtain the overall output data of the source device. The output data here refers to the complete data set output by the source device after the full collection operation.

[0048] Step 332, performing data sequence conversion on the output data, and performing a distribution operation on the converted data to obtain the target data.

[0049] Specifically, after collecting the output data of the source device, a data sequence conversion operation can be performed on the output data to match the data sequence of the input data of the target device. Here, data sequence conversion refers to the process of converting data from one sequence or order to another sequence or order. For example, the data shape of the output data of the source device can be [s, b, h], and the data shape of the input data of the target device can be [b, s, h]. Specifically, the data sequence conversion of the output data of the source device can be achieved by changing the arrangement of the data, the index order, or the storage method of the data in the memory.

[0050] It is understandable that, in the process of data processing, the target end device will usually split the data into several slices for processing in order to process the data efficiently. Therefore, for the data obtained after the data sequence conversion, in order to distribute it to different links of the target end device for efficient processing, the converted data can be further distributed, thereby obtaining the target data for final transmission. Here, the distribution operation refers to the process of distributing data from one or more sources to multiple targets, which can be achieved through a messaging system or a special data distribution tool.

[0051] In an embodiment of the present invention, before performing data sequence conversion, the consistency and accuracy of the data sequence conversion can be ensured by first collecting the slice data output by the source device. After the data sequence conversion, the converted data can be correctly distributed to multiple links of the target device for parallel processing by performing a distribution operation, thereby improving data processing efficiency.

[0052] Based on any of the above embodiments, in step 320, performing data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result includes: Step 321, based on the data shape of the source device, determine the data size and data sequence of the source device.

[0053] Specifically, the data shape is usually a dimensional description of a multidimensional array (or tensor), including the dimensions of the data and the size of each dimension. For example, the output data shape of the source device can be [s, b, h], which represents a three-dimensional tensor, where s, b, and h represent the size of the sequence dimension, batch dimension, and hidden layer dimension, respectively.

[0054] According to the data shape of the source device, the data size and data order of the source device can be determined. Here, the data size of the source device refers to the sum of all elements in the output tensor of the source device. For example, for an output tensor with a shape of [s, b, h], its data size is The data sequence of the source device refers to the order of the dimensions of the source device output tensor. In the above example, the data sequence of the source device output data is s, b, h, indicating that the first dimension is the sequence dimension, the second dimension is the batch dimension, and the third dimension is the hidden layer dimension.

[0055] Step 322: Determine the data size and data sequence of the target device based on the data shape of the target device.

[0056] Specifically, the data size of the target device refers to the sum of all source devices in the target device input tensor, and the data order of the target device refers to the order of each dimension of the target device input tensor. For example, the input data shape of the target device can be expressed as [b, s, h], where b, s, and h represent the size of the batch dimension, sequence dimension, and hidden layer dimension respectively. For this input data tensor, its data size is , the data order is b, s, h, which means that the first dimension is the batch dimension, the second dimension is the sequence dimension, and the third dimension is the hidden layer dimension.

[0057] Step 323, compare the data size of the source device with the data size of the target device to obtain a first comparison result, and compare the data sequence of the source device with the data sequence of the target device to obtain a second comparison result.

[0058] Specifically, the first comparison result refers to the result obtained by comparing the data size of the source device with the data size of the target device. This comparison is to check whether the data size of the two devices is consistent. In the above example, the data size of the source device is , the data size of the target device is , it is obvious that the data size of the two is the same.

[0059] The second comparison result refers to the result obtained by comparing the data sequence of the source device with the data sequence of the target device. This comparison is to check whether the data dimensions and order of the two devices are consistent. Specifically, the data sequence of the source device (for example, s, b, h) and the data sequence of the target device (for example, b, s, h) can be compared one by one to check whether each dimension exists and whether their order is the same, thereby obtaining the second comparison result.

[0060] Step 324: Determine the detection result based on the first comparison result and the second comparison result.

[0061] Specifically, according to the first comparison result and the second comparison result, it can be determined whether the output data format of the source device matches the input data format of the target device, thereby guiding the subsequent data transmission and format conversion operations. Specifically, if the first comparison result shows that the data size is consistent, and the second comparison result shows that the data sequence is also consistent, then the detection result is a match. In this case, the output data of the source device can be directly used as the input data of the target device for data transmission; if the first comparison result or the second comparison result shows inconsistency, then the detection result is a mismatch. In this case, the output data of the source device needs to be format converted to match the input data format of the target device. It should be understood that when the first comparison result shows that the data size is inconsistent, it usually means that the communication group of the source device and the target device is wrong, and the subsequent data transmission and format conversion operations are no longer performed.

[0062] Based on the above embodiment, the method further includes: Step 340: When the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0063] Specifically, when performing data format matching detection on the source device and the target device, if the detection result is a match, it indicates that the output data format of the source device matches the input data format of the target device. At this time, the output data of the source device can be directly sent to the target device as the input data of the target device through the data transmission channel for processing, that is, data transmission can be performed directly without any format conversion operation.

[0064] Based on any of the above embodiments, the method further includes: In the case where the data format detection result is found, if the data format detection result is a match, the output data of the source end device is used as the input data of the target end device for data transmission; If the data format detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission.

[0065] Specifically, before the source device and the target device perform data transmission, if the system successfully finds the data format detection result between the source device and the target device in the format detection result table, it means that the system has previously performed a format detection on whether the data formats between the source device and the target device match, and the result has been recorded in the format detection result table. In this case, the system can skip the detection step and directly select the corresponding transmission path according to the found data format detection result, thereby avoiding repeated detection and improving data transmission efficiency.

[0066] If the data format detection result found is a match, it indicates that the output data format of the source device matches the input data format of the target device. At this time, the output data of the source device can be used as the input data of the target device to directly perform the data transmission operation. If the data format detection result found is a mismatch, it indicates that the output data format of the source device does not match the input data format of the target device. At this time, it is necessary to perform a format conversion operation on the output data of the source device before transmitting it as the input data of the target device. Here, the specific format conversion operation can refer to the above embodiment, and the embodiment of the present invention will not be repeated here.

[0067] Based on any of the above embodiments, in step 320, storing the detection result as the data format detection result of the source device and the target device in the format detection result table includes: Using the data transmission information of the source device and the target device as a key and the detection result as a value, storing them in the format detection result table; The data transmission information includes the data shape of the source device, the communication group identifier to which the source device and the target device belong, and the serial number of the target device.

[0068] Specifically, a format detection result table for storing data format detection results can be pre-constructed, and the table can be a database table, a hash table, a dictionary or other key-value pair storage structure. When storing the detection results in the format detection result table, a unique key is constructed using the data transmission information of the source device and the target device, and the value corresponding to the key is the detection result, that is, whether the output data format of the source device matches the input data format of the target device. The constructed key and the detection result are stored as values ​​in the format detection result table. It should be understood that the data transmission information is obtained by combining the output data shape of the source device, the communication group identifier to which the source device and the target device belong, and the number of the target device.

[0069] Based on any of the above embodiments, in step 330, the format conversion of the output data of the source device includes: The output data of the source device is format-converted based on the target device, and the target device is selected from the source device and the target device based on device performance.

[0070] It should be noted that when converting the format of the output data of the source device, the conversion operation can be completed on the source device or on the target device. Considering that the source device and the target device come from different manufacturers or models, the efficiency of the two when performing data format conversion is different. In order to improve the conversion efficiency, when the data format conversion is performed between the source device and the target device for the first time, the system will automatically perform performance optimization and select the device with better performance from the source device and the target device to complete the conversion operation.

[0071] Specifically, the target device refers to the device selected to perform the conversion operation when performing data format conversion. It can be one of the source device (i.e., the device that outputs data) or the target device (i.e., the device that receives output), depending on which device has higher efficiency in performing data format conversion.

[0072] In the process of performance optimization, the system compares the performance of the source device and the target device in data format conversion, and selects the device with better performance as the target device to complete the conversion operation. This can be achieved in the following steps: First, the system evaluates the device performance of the source device and the target device. Here, device performance mainly refers to the efficiency of the device when performing data format conversion, which depends on the cache bandwidth of the device. In order to evaluate the cache bandwidth of the device, the system can perform a simple test. For example, the system can allocate a piece of video memory on the device and try to copy data from this video memory to another video memory. The latency of the copy operation can be used to measure the cache bandwidth of the device. The shorter the latency, the higher the cache bandwidth of the device. After obtaining the cache bandwidth information of the source device and the target device, the system compares the two values. The device with higher cache bandwidth will be selected as the target device for subsequent data format conversion operations.

[0073] In the embodiment of the present invention, through the performance optimization mechanism, the system can select a device with better performance to complete the data format conversion, thereby further optimizing resource utilization and processing speed.

[0074] Based on any of the above embodiments, Figure 4 FIG. 1 is a flow chart of a data format conversion method for heterogeneous GPUs provided by the present invention. Figure 4 As shown in the figure, the left side is the source device, and the right side is the target device. The source device and the target device respectively process the first layer and the second layer of the two adjacent layers of the model, and realize distributed processing and training of data through a series of operations.

[0075] Among them, on the source device, the multilayer perceptron is a feedforward artificial neural network model, which is mainly used to process input data and extract features useful for model training; the distribution operation is used to disperse the data processed by the multilayer perceptron to multiple links for subsequent parallel processing; the addition operation is to summarize or accumulate the data on each link separately; the sending operation is to send the processed data from the source device to the target device.

[0076] On the target device, the receiving operation refers to receiving the data sent from the source device; the normalization operation refers to standardizing the received data to make it conform to a certain distribution or range, which helps to improve the training efficiency and stability of the model; the full collection operation refers to convening the data scattered on multiple links on the target device for subsequent global processing; the attention mechanism is used to process the fully collected data and extract features or information that are useful for model training.

[0077] Since the source device and the target device may come from different manufacturers or models, the data formats of the two devices are different, and the data cannot be directly compatible when transmitted between the devices. Therefore, appropriate format conversion is required to achieve matching. In order to solve this problem, an embodiment of the present invention provides an automated data format detection and conversion method, which includes: Step S1, when sending and receiving communications across devices for the first time, the system will automatically detect and determine whether the data formats of the two match based on the output data shape of the source device and the input data shape of the target device.

[0078] Step S2, if a format mismatch is detected, the system will automatically execute Figure 4 The format conversion logic is shown by the dotted line in the figure; if a format match is detected, the data transfer operation is performed directly.

[0079] Specifically, in the format conversion logic shown by the dotted line, the data sequence is mainly converted. Figure 4 As shown in the figure, on the source device, the data is dispersed into slices of the shape [s / 4, b, h]. Therefore, before the data sequence conversion, all four pieces of data are first collected into one piece of data, and the shape of the data is [s, b, h]. Then, the data sequence conversion is performed on the data, and the shape of the converted data is [b, s, h]. Finally, the distribution operation is performed on the converted data to obtain slice data of the shape [b / 4, s, h]. These slice data can be transmitted to the target device for subsequent processing and analysis.

[0080] In step S3, the system uses the [shape, comm_group, dst_rank] triple as the key and stores the detection result as the corresponding value in the hash table. In subsequent frequent communications, the system will skip the detection step and directly select the transmission path.

[0081] Specifically, the shape in the triple refers to the output data shape of the source device, comm_group refers to the communication group identifier to which the source device and the target device belong, and dst_rank refers to the number of the target. In subsequent frequent communications, if the detection result found by the system is a match, the direct transmission path is taken. If the detection result is a mismatch, the transmission path shown by the dotted line is taken, that is, format conversion is performed first, and then data transmission is performed.

[0082] Step S4: The format conversion operation can be completed on the source device or the target device. In order to improve the conversion efficiency, the system will automatically perform performance optimization during the first transfer and select a device with better performance to complete the conversion operation.

[0083] The method provided by the embodiment of the present invention has the following advantages: (1) Automated detection: Through automated data format detection and intelligent conversion logic, automatic identification and adaptation of data formats in cross-device communications are achieved.

[0084] (2) High performance and low overhead: The system avoids repeated detection by storing the detection results in a hash table, significantly improving data transmission efficiency. In addition, through the performance optimization mechanism, the system can select a device with better performance to complete data format conversion, further optimizing resource utilization and processing speed.

[0085] (3) Compatibility: When heterogeneous GPUs are used for collaborative training, there is no need to worry about the differences in the underlying data formats. At the same time, the cost of long-term maintenance and upgrades is reduced, allowing the system to work efficiently in a variety of heterogeneous environments.

[0086] Based on any of the above embodiments, Figure 5 is a structural diagram of the data format conversion system provided by the present invention, such as Figure 5 As shown, the system includes: A searching unit 510 is used to search the data format detection results of the source end device and the target end device from the format detection result table based on the data transmission information of the source end device and the target end device; The detection unit 520 is used to perform data format matching detection based on the data shape of the source device and the data shape of the target device when the data format detection result is not found, obtain a detection result, and store the detection result as the data format detection result of the source device and the target device in the format detection result table; The conversion unit 530 is used to convert the format of the output data of the source device to obtain target data when the detection result is a mismatch, and use the target data as the input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0087] The system provided by the embodiment of the present invention can search the data format detection results of the source device and the target device from the format detection result table based on the data transmission information of the source device and the target device, and when the data format detection result is not found, the data format matching detection is performed based on the data shape of the source device and the data shape of the target device, so as to automatically and accurately determine whether the data formats of the source device and the target device match, so as to make subsequent processing decisions. When the detection result is mismatched, by performing format conversion on the output data of the source device, the format of the converted target data can be matched with the data format of the target device, thereby ensuring the smooth data transmission between heterogeneous devices, avoiding transmission errors or failures caused by data format mismatch, and realizing efficient and flexible cross-device communication, thereby ensuring the normal progress of model training. In addition, by storing the detection results in the format detection result table, repeated detection can be avoided in the subsequent data transmission process, thereby improving data transmission efficiency.

[0088] Based on any of the above embodiments, the conversion unit 530 includes a format conversion subunit, and the format conversion subunit is used to: Performing a full collection operation on each slice data output by the source device to obtain output data; The output data is subjected to data sequence conversion, and a distribution operation is performed on the converted data to obtain the target data.

[0089] Based on any of the above embodiments, the system further includes a transmission unit, and the transmission unit is used to: When the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

[0090] Based on any of the above embodiments, the system further includes a path selection unit, and the path selection unit is used to: In the case where the data format detection result is found, if the data format detection result is a match, the output data of the source end device is used as the input data of the target end device for data transmission; If the data format detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission.

[0091] Based on any of the above embodiments, the detection unit 520 includes a format matching detection subunit, and the format matching detection subunit is used to: Determine the data size and data sequence of the source device based on the data shape of the source device; Determine the data size and data sequence of the target device based on the data shape of the target device; Compare the data size of the source device with the data size of the target device to obtain a first comparison result, and compare the data sequence of the source device with the data sequence of the target device to obtain a second comparison result; The detection result is determined based on the first comparison result and the second comparison result.

[0092] Based on any of the above embodiments, the detection unit 520 further includes a result storage subunit, which is used to: Using the data transmission information of the source device and the target device as a key and the detection result as a value, storing them in the format detection result table; The data transmission information includes the data shape of the source device, the communication group identifier to which the source device and the target device belong, and the serial number of the target device.

[0093] Based on any of the above embodiments, the conversion unit 530 is specifically used for: The output data of the source device is format-converted based on the target device, and the target device is selected from the source device and the target device based on device performance.

[0094] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the data format conversion method, which includes: based on the data transmission information of the source end device and the target end device, searching the data format detection results of the source end device and the target end device from the format detection result table; if the data format detection result is not found, performing data format matching detection based on the data shape of the source end device and the data shape of the target end device to obtain a detection result, and storing the detection result as the data format detection result of the source end device and the target end device in the format detection result table; if the detection result is not matched, performing format conversion on the output data of the source end device to obtain target data, and using the target data as the input data of the target end device for data transmission, and the format of the target data matches the data format of the target end device.

[0095] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data format conversion method provided by the above methods, which method includes: based on the data transmission information of the source device and the target device, searching the data format detection results of the source device and the target device from the format detection result table; if the data format detection result is not found, based on the data shape of the source device and the data shape of the target device, performing data format matching detection to obtain a detection result, and storing the detection result as the data format detection result of the source device and the target device in the format detection result table; if the detection result is not matched, performing format conversion on the output data of the source device to obtain target data, and using the target data as the input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0097] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the data format conversion method provided by the above-mentioned methods, the method comprising: based on the data transmission information of the source device and the target device, searching the data format detection results of the source device and the target device from the format detection result table; if the data format detection result is not found, performing a data format matching detection based on the data shape of the source device and the data shape of the target device to obtain a detection result, and storing the detection result as the data format detection result of the source device and the target device in the format detection result table; if the detection result is not matched, performing format conversion on the output data of the source device to obtain target data, and using the target data as the input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

[0098] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data format conversion method, characterized in that: include: Based on the data transmission information of the source device and the target device, searching the data format detection results of the source device and the target device from the format detection result table; If the data format detection result is not found, performing data format matching detection based on the data shape of the source-end device and the data shape of the target-end device to obtain a detection result, and storing the detection result as the data format detection result of the source-end device and the target-end device in the format detection result table; When the detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission, and the format of the target data matches the data format of the target device.

2. The data format conversion method according to claim 1, characterized in that: The step of converting the output data of the source device into a format to obtain target data includes: Performing a full collection operation on each slice data output by the source device to obtain output data; The output data is subjected to data sequence conversion, and a distribution operation is performed on the converted data to obtain the target data.

3. The data format conversion method according to claim 1, characterized in that: Also includes: When the detection result is a match, the output data of the source device is used as the input data of the target device for data transmission.

4. The data format conversion method according to claim 1, characterized in that: Also includes: In the case where the data format detection result is found, if the data format detection result is a match, the output data of the source end device is used as the input data of the target end device for data transmission; If the data format detection result is mismatch, the output data of the source device is format converted to obtain target data, and the target data is used as input data of the target device for data transmission.

5. The data format conversion method according to claim 1, characterized in that: The performing data format matching detection based on the data shape of the source end device and the data shape of the target end device to obtain the detection result includes: Determine the data size and data sequence of the source device based on the data shape of the source device; Determine the data size and data sequence of the target device based on the data shape of the target device; Compare the data size of the source device with the data size of the target device to obtain a first comparison result, and compare the data sequence of the source device with the data sequence of the target device to obtain a second comparison result; The detection result is determined based on the first comparison result and the second comparison result.

6. The data format conversion method according to any one of claims 1 to 5, characterized in that: The step of storing the detection result as a data format detection result of the source device and the target device in the format detection result table includes: Using the data transmission information of the source device and the target device as a key and the detection result as a value, storing them in the format detection result table; The data transmission information includes the data shape of the source device, the communication group identifier to which the source device and the target device belong, and the serial number of the target device.

7. The data format conversion method according to any one of claims 1 to 5, characterized in that: The format conversion of the output data of the source device includes: The output data of the source device is format-converted based on the target device, and the target device is selected from the source device and the target device based on device performance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data format conversion method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data format conversion method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data format conversion method according to any one of claims 1 to 7 is implemented.

Images