Data layout optimization method and device, electronic equipment and storage medium

Abstract

The present disclosure discloses a data layout optimization method and device, an electronic device and a storage medium, comprising: adding a layout state identifier to target data in a recognized calculation graph, and processing the calculation graph based on the layout state identifier to obtain a target calculation graph, wherein the layout state identifier is used to indicate that the target data is converted into a reference data format at runtime; during execution of the target calculation graph, it is judged whether the data format corresponding to the target data matches the reference data format according to the layout state identifier; in the case where it is determined that the data format matches the reference data format, the data layout conversion operation of the target data is skipped; through the indication of the reference data format of the target data by the layout state identifier and the format matching judgment, the data layout conversion operation is skipped when the target data format matches the reference data format, the number of repeated data layout conversion operation executions is reduced, and the overall execution efficiency of the data layout optimization process is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more particularly to a data layout optimization method and apparatus, electronic device and storage medium. Background Technology

[0002] In the field of artificial intelligence, the execution of deep learning models depends on the collaboration between the deep learning compilation stack and the artificial intelligence accelerator. To improve computational efficiency, the artificial intelligence accelerator adopts a non-standard data layout format, while the computation graph of the deep learning compilation stack is built based on a general data layout format. The two need to complete the data format adaptation.

[0003] In related technologies, format adaptation is achieved by inserting explicit layout transformation operators into the computation graph, thus ensuring format compatibility between the two. However, for tensor data undergoing multiple iterations, the data layout transformation operation performed by inserting layout transformation operators will be repeatedly triggered in each iteration, resulting in a large amount of redundant computation and wasting computational resources. Summary of the Invention

[0004] This disclosure provides a data layout optimization method, apparatus, electronic device, and storage medium to solve the problem of redundant waste of computing resources in related technologies.

[0005] According to a first aspect of this disclosure, a data layout optimization method is provided, comprising:

[0006] A layout state identifier is added to the target data in the identified computation graph, and the computation graph is processed based on the layout state identifier to obtain the target computation graph. The layout state identifier is used to indicate that the target data is converted into a reference data format at runtime.

[0007] During the execution of the target computation graph, it is determined whether the data format corresponding to the target data matches the reference data format based on the layout state identifier;

[0008] If the data format is determined to match the reference data format, the data layout conversion operation for the target data is skipped.

[0009] In some embodiments of this disclosure, processing the computation graph based on the layout state identifier to obtain the target computation graph includes:

[0010] During the compilation of the computation graph, the redundant data layout transformation operator corresponding to the target data in the computation graph is determined based on the layout state identifier.

[0011] The redundant data layout transformation operator is reduced to obtain the target computation graph.

[0012] In some embodiments of this disclosure, adding a layout state identifier to the target data in the identified computation graph includes:

[0013] The data to be converted into a hardware-optimized layout is identified from the computation graph and used as the target data.

[0014] Configure the layout status identifier for the target data.

[0015] In some embodiments of this disclosure, the step of determining whether the data format corresponding to the target data matches the reference data format based on the layout state identifier during the execution of the target computation graph includes:

[0016] During the calculation initialization phase of the target computation graph during iterative execution, the layout state identifier associated with the target data is retrieved;

[0017] Extract the data format of the target data and the reference data format from the layout status identifier, and perform a consistency matching judgment on the data format and the reference data format.

[0018] In some embodiments of this disclosure, the method further includes:

[0019] If it is determined that the data format does not match the reference data format, a data layout conversion operation is performed on the target data to obtain the reference data format, and the layout status identifier is updated so that the updated layout status identifier indicates that the target data is currently in the reference data format.

[0020] In some embodiments of this disclosure, after skipping the data layout conversion operation for the target data when it is determined that the data format matches the reference data format, the method further includes:

[0021] During the copying operation of the target data from the hardware device to the host memory, if the data format is determined to be the reference data format, the target data in the host memory is converted back from the reference data format to the original data format, while keeping the data format of the target data in the hardware device and the layout state identifier unchanged.

[0022] According to a second aspect of this disclosure, a data layout optimization apparatus is provided, comprising:

[0023] Add a cell to add a layout status identifier to the target data in the identified computation graph;

[0024] A processing unit is configured to process the computation graph based on the layout state identifier to obtain a target computation graph, wherein the layout state identifier is used to indicate that the target data is converted into a reference data format at runtime;

[0025] The judgment unit is used to determine, during the execution of the target computation graph, whether the data format corresponding to the target data matches the reference data format based on the layout state identifier;

[0026] The skip unit is used to skip the data layout conversion operation of the target data when it is determined that the data format matches the reference data format.

[0027] In some embodiments of this disclosure, the processing unit includes:

[0028] The first determining module is used to determine, during the compilation of the computation graph, the redundant data layout transformation operator corresponding to the target data in the computation graph based on the layout state identifier;

[0029] The processing module is used to reduce the redundant data layout transformation operator to obtain the target computation graph.

[0030] In some embodiments of this disclosure, the adding unit includes:

[0031] The second determining module is used to identify data to be converted into hardware-optimized layout from the computation graph as the target data;

[0032] The configuration module is used to configure the layout status identifier for the target data.

[0033] In some embodiments of this disclosure, the determining unit includes:

[0034] The retrieval module is used to retrieve the layout state identifier associated with the target data during the computation initialization phase of iteratively executing the target computation graph;

[0035] The judgment module is used to extract the data format of the target data and the reference data format from the layout status identifier, and to perform a consistency matching judgment on the data format and the reference data format.

[0036] In some embodiments of this disclosure, the apparatus further includes:

[0037] The conversion unit is configured to perform a data layout conversion operation on the target data to obtain the reference data format when it is determined that the data format does not match the reference data format, and update the layout status identifier so that the updated layout status identifier indicates that the target data is currently in the reference data format.

[0038] In some embodiments of this disclosure, the apparatus further includes:

[0039] The rotation unit is configured to, after the skip unit skips the data layout conversion operation of the target data when it determines that the data format matches the reference data format, during the copy operation of the target data from the hardware device to the host memory, if it determines that the data format is the reference data format, rotate the target data from the reference data format to the original data format in the host memory, and keep the data format and the layout state identifier of the target data in the hardware device unchanged.

[0040] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0041] At least one processor; and

[0042] A memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect embodiment.

[0044] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect of the present disclosure.

[0045] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect of the preceding embodiments.

[0046] The data layout optimization method, apparatus, electronic device, and storage medium disclosed herein include: adding a layout status identifier to target data in an identified computation graph, and processing the computation graph based on the layout status identifier to obtain a target computation graph, wherein the layout status identifier is used to indicate that the target data is converted to a reference data format at runtime; during the execution of the target computation graph, determining whether the data format corresponding to the target data matches the reference data format based on the layout status identifier; if it is determined that the data format matches the reference data format, skipping the data layout conversion operation for the target data; by using the layout status identifier to indicate the reference data format of the target data and the format matching determination, skipping the data layout conversion operation when the target data format matches the reference data format reduces the number of times repeated data layout conversion operations are executed, thereby improving the overall execution efficiency of the data layout optimization process.

[0047] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0048] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0049] Figure 1 A flowchart illustrating a data layout optimization method provided in an embodiment of this disclosure;

[0050] Figure 2 A flowchart illustrating another data layout optimization method provided in this embodiment of the disclosure;

[0051] Figure 3 A flowchart illustrating another data layout optimization method provided in this embodiment of the disclosure;

[0052] Figure 4 A flowchart illustrating another data layout optimization method provided in this embodiment of the disclosure;

[0053] Figure 5 A schematic diagram of the overall process of a data layout optimization method provided in an embodiment of this disclosure;

[0054] Figure 6 This is a schematic diagram of the structure of a data layout optimization device provided in an embodiment of the present disclosure;

[0055] Figure 7 A schematic diagram of another data layout optimization device provided in an embodiment of this disclosure;

[0056] Figure 8 A schematic block diagram of an example electronic device provided for embodiments of this disclosure. Detailed Implementation

[0057] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0058] The data layout optimization method, apparatus, electronic device, and storage medium of this disclosure are described below with reference to the accompanying drawings.

[0059] This invention has a wide range of applications. Based on a data layout optimization mechanism using layout state identifiers, it can be flexibly adapted to various hardware and software optimization scenarios involving the collaboration of deep learning compiler stacks and AI accelerators. In the field of deep learning compiler stack and AI accelerator co-optimization, it is suitable for hardware and software co-deployment and computing power optimization scenarios involving deep learning compilers such as OpenXLA and various series of AI accelerators, NPUs, and other AI accelerators. In the field of deep learning model training, it is particularly suitable for the entire process of parameter iteration and updates during the training of various neural network models, covering various tensor data processing optimization scenarios in the model training stage. Furthermore, it can be extended to related scenarios such as deep learning model inference acceleration to improve the data layout processing efficiency when the compiler stack and AI accelerator work together during the inference stage, demonstrating good scenario adaptability and practicality.

[0060] Figure 1 This is a flowchart illustrating a data layout optimization method provided in an embodiment of the present disclosure.

[0061] like Figure 1 As shown, the method includes the following steps:

[0062] Step 101: Add a layout status identifier to the target data in the identified computation graph, and process the computation graph based on the layout status identifier to obtain the target computation graph, wherein the layout status identifier is used to indicate that the target data is converted into a reference data format at runtime.

[0063] In some embodiments, the computation graph is a graphical representation formed by abstracting and describing the computational logic of the model during the deep learning compilation process. The target data is a type of data in the computation graph that needs to undergo data layout transformation. The layout status identifier is an identifier that establishes a unique correspondence with the target data. Its function is to clearly indicate the reference data format to which the target data needs to be transformed during the runtime phase. The reference data format is a preset data arrangement format used to adapt to computational requirements. Identifying the target data in the computation graph can be achieved through a data analysis program during compilation. Adding a layout status identifier to the target data can be accomplished by adding an identifier field to the metadata of the target data. When processing the computation graph based on the layout status identifier, the redundant instructions related to the layout transformation of the target data in the computation graph are processed according to the reference data format indicated by the layout status identifier, and finally the target computation graph is obtained.

[0064] The above methods were used to accurately identify the target data and preprocess the computation graph, clarify the reference conversion format of the target data, provide a basis for format judgment during the runtime phase, and obtain the target computation graph that is adapted for execution.

[0065] Step 102: During the execution of the target computation graph, determine whether the data format corresponding to the target data matches the reference data format based on the layout state identifier.

[0066] In some embodiments, the execution process of the target computation graph refers to the entire process of the target computation graph being loaded into the computing hardware and starting computation. During the judgment process, the layout state identifier that uniquely corresponds to the target data is first retrieved, the reference data format indicated by the layout state identifier is extracted from the layout state identifier, and the current actual data format of the target data is obtained. The obtained data format is checked for consistency with the reference data format. If the encoding rules and dimension arrangement of the two are completely consistent, it is determined to be a match; otherwise, it is determined to be a mismatch.

[0067] By using the above method, the current format and reference format of the target data can be accurately determined by relying on the layout status identifier, providing a clear judgment result for whether to perform the layout conversion operation and ensuring that the execution of the layout conversion operation is targeted.

[0068] Step 103: If it is determined that the data format matches the reference data format, skip the data layout conversion operation for the target data.

[0069] In some embodiments, the data layout conversion operation is a process of converting the target data from its current data format to a reference data format. Skipping the data layout conversion operation means that in the calculation execution flow, no layout conversion-related instructions for the target data are triggered, and the calculation flow of the target data is directly entered. If the current data format of the target data already matches the reference data format, after executing the skip operation, the target data retains its current data format and enters the subsequent calculation.

[0070] By using the above method, when the target data format already meets the reference requirements, meaningless data layout conversion operations are avoided, redundant computational operations are reduced, and computational resource consumption is lowered.

[0071] In summary, the data layout optimization method provided in this disclosure includes: adding a layout status identifier to the target data in the identified computation graph, and processing the computation graph based on the layout status identifier to obtain the target computation graph, wherein the layout status identifier is used to indicate that the target data is converted into a reference data format at runtime; during the execution of the target computation graph, determining whether the data format corresponding to the target data matches the reference data format based on the layout status identifier; if it is determined that the data format matches the reference data format, skipping the data layout conversion operation for the target data; by using the layout status identifier to indicate the reference data format of the target data and the format matching judgment, skipping the data layout conversion operation when the target data format matches the reference data format reduces the number of times repeated data layout conversion operations are executed and improves the overall execution efficiency of the data layout optimization process.

[0072] Figure 2 A flowchart illustrating a data layout optimization method provided in an embodiment of this disclosure is further illustrated. For example... Figure 2 As shown, the method includes the following steps:

[0073] Step 201: During the compilation of the computation graph, the redundant data layout transformation operator corresponding to the target data in the computation graph is determined according to the layout state identifier.

[0074] In some embodiments, a layout state identifier (such as a dedicated compilation pass) is introduced during the compilation of the computation graph to identify redundant data layout transformation operators set for target data with added layout state identifiers in the computation graph. These redundant data layout transformation operators are layout transformation-related operators that are independent of the reference data format conversion of the target data, can be omitted, or are repeated.

[0075] Step 202: The redundant data layout transformation operator is reduced to obtain the target computation graph.

[0076] In some embodiments, a deletion operation is performed on the identified redundant data layout transformation operators to remove redundant layout transformation instructions from the computation graph, retain the necessary operators related to the target data reference data format transformation, complete the simplification of the computation graph, and finally obtain the target computation graph.

[0077] The above method simplifies the computation graph, reduces the number of operators in the compiled target computation graph, and reduces redundant operations during the execution of the computation graph, thus providing a foundation for the efficient execution of the target computation graph.

[0078] Figure 3 A flowchart illustrating a data layout optimization method provided in an embodiment of this disclosure is further illustrated. For example... Figure 3 As shown, the method includes the following steps:

[0079] Step 301: Identify the data to be converted into a hardware-optimized layout from the computation graph as the target data.

[0080] In some embodiments, the data in the computation graph to be converted to a hardware-optimized layout includes, but is not limited to, tensor data such as initial weights, optimizer states, and inputs and outputs. Through a compile-time data analysis program, the data that is suitable for and needs to be converted to a hardware-optimized layout is identified from the computation graph and determined as the target data. The hardware-optimized layout is the reference data format indicated by the layout state identifier.

[0081] Step 302: Configure the layout status identifier for the target data.

[0082] In some embodiments, a layout status identifier is added to the identified target data. This layout status identifier is used to indicate that the target data is converted into a hardware-optimized layout (reference data format) at runtime. This can be achieved by adding layout annotations to the target data in the computation graph, as illustrated in the following example: This layout label serves as the layout status identifier for the target data. For target data, This describes the inherent attributes of the target data (such as data dimensions, data types, etc.). The action to add layout status indicators, For layout status indicators, This is a reference data format. It should be noted that the above examples are for illustrative purposes only and do not limit the specific content.

[0083] The above methods enabled accurate identification of target data and configuration of layout status indicators, clarified the reference data format of target data, and provided a foundation for computation graph optimization and runtime format judgment.

[0084] Figure 4 A flowchart illustrating a data layout optimization method provided in an embodiment of this disclosure is further illustrated. For example... Figure 4 As shown, the method includes the following steps:

[0085] Step 401: During the iterative execution of the calculation initialization phase of the target computation graph, the layout state identifier associated with the target data is retrieved.

[0086] In some embodiments, the execution process of the target computation graph is an iterative computation process. During the computation initialization phase of each iteration, the system automatically retrieves the layout status identifier that establishes a unique association with the target data, ensuring that the format can be judged based on the layout status identifier in each iteration.

[0087] Step 402: Extract the data format of the target data and the reference data format from the layout status identifier, and perform a consistency matching judgment on the data format and the reference data format.

[0088] In some embodiments, the reference data format (hardware optimized layout) indicated by the retrieved layout status identifier is extracted, and the current actual data format of the target data is obtained. By performing consistency verification on the arrangement rules and format parameters of the two data formats, the matching judgment is completed. If the two are completely consistent, it is determined to be a match; otherwise, it is determined to be a mismatch.

[0089] The above method enables accurate judgment of the target data format during iterative execution, ensuring that each iteration can determine whether a layout conversion operation needs to be performed based on the layout status identifier, thus providing a basis for judgment for "conversion on demand, one-time conversion".

[0090] In some embodiments of this disclosure, the data is configured to: when it is determined that the data format does not match the reference data format, perform a data layout conversion operation on the target data to obtain the reference data format, and update the layout status identifier so that the updated layout status identifier indicates that the target data is currently in the reference data format.

[0091] In some embodiments, when it is determined that the current data format of the target data does not match the reference data format indicated by the layout status identifier, a data layout conversion operation is triggered to convert the target data from the current format to the reference data format (hardware optimized layout). This conversion operation is performed only once. After the conversion is completed, the layout status identifier corresponding to the target data is updated synchronously, and the current format of the target data indicated in the layout status identifier is modified to the reference data format to ensure that the format status of the target data can be accurately identified in subsequent iterations.

[0092] The above method enables on-demand layout transformation and dynamic management of the layout state of target data, ensuring that the transformed target data can be directly reused in subsequent iterations and avoiding repeated transformations.

[0093] In some embodiments of this disclosure, the following is used: during the copying operation of the target data from the hardware device to the host memory, if the data format is determined to be the reference data format, the target data is converted back from the reference data format to the original data format in the host memory, while keeping the data format of the target data in the hardware device and the layout state identifier unchanged.

[0094] In some embodiments, when performing a data copy operation from hardware device to host memory (D2H), the current data format of the target data is first checked to see if it is the reference data format (hardware optimized layout). If it is determined to be the reference data format, a format conversion operation is performed only on the target data being copied in the host memory to convert it to the original data format, so as to ensure format compatibility with the external framework. The target data in the hardware device still maintains the reference data format, and its corresponding layout state identifier does not change, ensuring that subsequent iterative calculations can directly reuse the target data in the hardware device without performing a layout conversion operation again.

[0095] By using the above method, while ensuring compatibility with external framework formats, repeated layout conversions of target data in hardware devices are avoided, thus ensuring computational efficiency.

[0096] Figure 5 To further illustrate the overall flowchart of a data layout optimization method provided in this disclosure embodiment, as shown in the diagram... Figure 5 As shown, the overall process of this method includes two stages: compilation and runtime. During compilation, the computation graph undergoes layout analysis and optimization, identifying target data and adding layout status identifiers. These identifiers indicate that the target data is converted to a reference data format at runtime. Simultaneously, redundant data layout transformation operators in the computation graph are determined and removed based on the layout status identifiers, generating the target computation graph. During runtime, the target computation graph is iteratively executed (the number of iterations N is greater than or equal to 1). In each iteration, the layout status identifiers are used to determine whether the target data format matches the reference data format. If they do not match, a data layout transformation operation is performed and the layout status identifiers are updated. If they match, the data layout transformation operation is skipped, and computation is performed directly based on the reference data format. When performing the backhaul operation of target data from the hardware device to the host memory, if the target data format is the reference data format, the target data is only converted back from the reference data format to the original data format in the host memory, while maintaining the target data format and layout status identifiers in the hardware device, thus completing the overall process.

[0097] Corresponding to the data layout optimization method described above, this invention also proposes a data layout optimization device. Since the device embodiments of this invention correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to in the method embodiments described above, and will not be repeated here.

[0098] Figure 6 This is a schematic diagram of the structure of a data layout optimization device provided in an embodiment of the present disclosure, as shown below. Figure 6 As shown, it includes: an addition unit 51, a processing unit 52, a judgment unit 53, and a skip unit 54.

[0099] Add unit 51 to add layout status identifiers to the target data in the identified computation graph;

[0100] Processing unit 52 is used to process the computation graph based on the layout state identifier to obtain a target computation graph, wherein the layout state identifier is used to indicate that the target data is converted into a reference data format at runtime;

[0101] The judgment unit 53 is used to determine whether the data format corresponding to the target data matches the reference data format based on the layout state identifier during the execution of the target computation graph.

[0102] Skip unit 54 is used to skip the data layout conversion operation of the target data when it is determined that the data format matches the reference data format.

[0103] In summary, the data layout optimization apparatus provided in this disclosure includes: adding a layout status identifier to the target data in the identified computation graph, and processing the computation graph based on the layout status identifier to obtain a target computation graph, wherein the layout status identifier is used to indicate that the target data is converted into a reference data format during runtime; during the execution of the target computation graph, determining whether the data format corresponding to the target data matches the reference data format based on the layout status identifier; if it is determined that the data format matches the reference data format, skipping the data layout conversion operation for the target data; by using the layout status identifier to indicate the reference data format of the target data and the format matching judgment, skipping the data layout conversion operation when the target data format matches the reference data format reduces the number of times repeated data layout conversion operations are executed, thereby improving the overall execution efficiency of the data layout optimization process.

[0104] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 7 As shown, the processing unit 52 includes:

[0105] The first determining module 521 is used to determine, during the compilation of the computation graph, the redundant data layout transformation operator corresponding to the target data in the computation graph based on the layout state identifier;

[0106] The processing module 522 is used to reduce the redundant data layout transformation operator to obtain the target computation graph.

[0107] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 7 As shown, the adding unit 51 includes:

[0108] The second determining module 511 is used to identify data to be converted into hardware-optimized layout from the computation graph as the target data;

[0109] Configuration module 512 is used to configure the layout status identifier for the target data.

[0110] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 7 As shown, the judgment unit 53 includes:

[0111] The retrieval module 531 is used to retrieve the layout state identifier associated with the target data during the calculation initialization phase of iteratively executing the target computation graph;

[0112] The judgment module 532 is used to extract the data format of the target data and the reference data format in the layout status identifier, and to perform a consistency matching judgment on the data format and the reference data format.

[0113] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 7 As shown, the device further includes:

[0114] The conversion unit 55 is configured to perform a data layout conversion operation on the target data to obtain the reference data format when it is determined that the data format does not match the reference data format, and update the layout status identifier so that the updated layout status identifier indicates that the target data is currently in the reference data format.

[0115] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 7 As shown, the device further includes:

[0116] The rotation unit 56 is configured to, after the skip unit 54 skips the data layout conversion operation of the target data when it determines that the data format matches the reference data format, during the copy operation of the target data from the hardware device to the host memory, if it determines that the data format is the reference data format, rotate the target data from the reference data format to the original data format in the host memory, and keep the data format and the layout state identifier of the target data in the hardware device unchanged.

[0117] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of the embodiments of this disclosure, and the principle is the same. Therefore, the embodiments of this disclosure are not limited thereto.

[0118] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0119] Figure 8A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as deep learning compilers, AI acceleration computing platforms, and neural network model training frameworks. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0120] like Figure 8 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 602 or loaded from storage unit 608 into RAM (Random Access Memory) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An I / O (Input / Output) interface 605 is also connected to the bus 604.

[0121] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0122] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as data layout optimization methods. For example, in some embodiments, the data layout optimization method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the aforementioned data layout optimization method by any other suitable means (e.g., by means of firmware).

[0123] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0127] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0128] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0129] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0130] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data layout optimization method, characterized in that, The method includes: A layout status identifier is added to the target data in the identified computation graph, and the computation graph is processed based on the layout status identifier to obtain the target computation graph. The layout status identifier is used to indicate whether the target data is converted into a reference data format at runtime. During the execution of the target computation graph, it is determined whether the data format corresponding to the target data matches the reference data format based on the layout state identifier; If it is determined that the data format matches the reference data format, skip the data layout conversion operation for the target data; The layout state identifier is configured to establish an association with the target data and can be updated during the iterative execution of the computation graph; The method further includes: If it is determined that the data format does not match the reference data format, a data layout conversion operation is performed on the target data to obtain the reference data format, and the layout status identifier is updated so that the updated layout status identifier indicates that the target data is currently in the reference data format; The method further includes: During the copying operation of the target data from the hardware device to the host memory, if the data format is determined to be the reference data format, the target data in the host memory is converted back from the reference data format to the original data format, while keeping the data format of the target data in the hardware device and the layout state identifier unchanged.

2. The method according to claim 1, characterized in that, The step of processing the computation graph based on the layout state identifier to obtain the target computation graph includes: During the compilation of the computation graph, the redundant data layout transformation operator corresponding to the target data in the computation graph is determined based on the layout state identifier. The redundant data layout transformation operator is reduced to obtain the target computation graph.

3. The method according to claim 1, characterized in that, Adding layout status identifiers to the target data in the identified computation graph includes: The data to be converted into a hardware-optimized layout is identified from the computation graph and used as the target data. Configure the layout status identifier for the target data.

4. The method according to claim 1, characterized in that, During the execution of the target computation graph, determining whether the data format corresponding to the target data matches the reference data format based on the layout state identifier includes: During the calculation initialization phase of the target computation graph during iterative execution, the layout state identifier associated with the target data is retrieved; Extract the data format of the target data and the reference data format from the layout status identifier, and perform a consistency matching judgment on the data format and the reference data format.

5. A data layout optimization device, characterized in that, include: Add a cell to add a layout status identifier to the target data in the identified computation graph; A processing unit is configured to process the computation graph based on the layout state identifier to obtain a target computation graph, wherein the layout state identifier is used to indicate whether the target data is converted into a reference data format at runtime; The judgment unit is used to determine, during the execution of the target computation graph, whether the data format corresponding to the target data matches the reference data format based on the layout state identifier; The skip unit is used to skip the data layout conversion operation of the target data when it is determined that the data format matches the reference data format; The layout state identifier is configured to establish an association with the target data and can be updated during the iterative execution of the computation graph; The device is also used for: If it is determined that the data format does not match the reference data format, a data layout conversion operation is performed on the target data to obtain the reference data format, and the layout status identifier is updated so that the updated layout status identifier indicates that the target data is currently in the reference data format; During the copying operation of the target data from the hardware device to the host memory, if the data format is determined to be the reference data format, the target data in the host memory is converted back from the reference data format to the original data format, while keeping the data format of the target data in the hardware device and the layout state identifier unchanged.

6. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

8. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images