Compiler-based aggregation type data layout automatic conversion method

Abstract

The invention relates to data layout conversion, in particular to a compiler-based aggregation type data layout automatic conversion method, which comprises the following steps of: designing a front end, defining an interface of a user program and a compiler, and realizing the interface with the compiler in a mode of declaring a function F through the user program; the translated source and target addressing architecture, the translated data type, the translated source and target data address and the translated occurrence time are informed to the compiler; terminal design: complete definition of a function F stated by a user program is completed in a compiler, so that the function F has a data layout conversion function, that is, source data is read according to an addressing architecture of the source data, and the source data is stored to a target data address according to an addressing architecture of the target data; according to the technical scheme provided by the invention, the defect of poor compatibility of data interaction and code transplantation of aggregation type data among processors with different addressing architectures in the prior art can be effectively overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to data layout conversion, in particular to a compiler-based automatic conversion method for aggregate type data layout. BACKGROUND

[0002] In a heterogeneous fusion architecture processor, a heterogeneous board card or a heterogeneous system, the addressing mode and data layout scheme of different processor cores have certain differences. For example, Arm CPU and X86 architecture are usually byte addressing architecture, and support the minimum addressing unit of 8 bits; fixed-point processor TITMS320C28x is half-word addressing architecture, and supports the minimum addressing unit of 16 bits; soul core series processor and ADSP-TS201 are word addressing architecture, and support the minimum addressing unit of 32 bits. If data interaction or code transplantation is performed between the above processors, programmers face great challenges, because various data types, especially aggregate types (including arrays, structures, bit fields, unions, etc.) usually have the problems of many members and nested combination. User programs need to manually write conversion programs for various user-defined data types, which is a huge and difficult task.

[0003] Taking the following code as an example:

[0004]

[0005] struct IMAGE_HEAD h = {0x11, 0x2233, 0x44444444}

[0006] int x = h.c;

[0007] For the structure type data struct IMAGE_HEAD h, if the data of h is generated by a processor A using byte addressing architecture (8-bit addressing), the data layout is as shown in Figure 4 ; if the data of h is generated by a processor B using word addressing architecture (32-bit addressing), the data layout is as shown in Figure 5 . Then for the data h generated by the processor A, the processor B cannot correctly parse the member value through the statement line 7: int x = h.c, because the data address read by h.c is at 0x8. If the processor B wants to correctly parse the data generated by the processor A, it needs to recalculate the offset address of each member in the aggregate type data and obtain the correct member value through bit operation and shift operation, which is difficult to program and has poor universality. SUMMARY

[0008] (1) Technical problems solved

[0009] In view of the above-mentioned defects of the prior art, the application provides an aggregated type data layout automatic conversion method based on a compiler, which can effectively overcome the defect of poor compatibility of data interaction and code porting between processors of different addressing architectures.

[0010] (II) Technical solutions

[0011] To achieve the above object, the application is implemented by the following technical solutions:

[0012] The aggregated type data layout automatic conversion method based on the compiler comprises front-end design and terminal design.

[0013] The front-end design defines the interface between the user program and the compiler, realizes the interface with the compiler by means of declaration of a function F of the user program, and informs the compiler of the source and target addressing architectures to be converted, the data type to be converted, the source and target data addresses to be converted and the conversion timing.

[0014] The terminal design completes the complete definition of the function F declared by the user program in the compiler, so that the function F has the function of data layout conversion, i.e. reading the source data according to the addressing architecture of the source data and storing the source data to the target data address according to the addressing architecture of the target data.

[0015] Preferably, the front-end design specifically comprises:

[0016] The source and target addressing architectures to be converted are informed to the compiler by means of a function name prefix, wherein the function name prefix mode is ConvertDL_XXX_To_YYY_, XXX represents the addressing architecture of the source data, YYY represents the addressing architecture of the target data, and the value range is {“8BITS”, “16BITS”, “32BITS”}, which respectively represent the byte addressing architecture, the half-word addressing architecture and the word addressing architecture.

[0017] The data type to be converted is informed to the compiler by means of a function return value, including the aggregated type of the structure body, the bit field and the union body, and if the data type is an array type, the structure body type is used to embed it.

[0018] The source data address to be converted is informed to the compiler by means of a first function parameter, and the target data address to be converted is informed to the compiler by means of a second function parameter.

[0019] The conversion timing is informed to the compiler by means of a function call statement.

[0020] Preferably, the user program declares the function F in the form of struct ECHO_HEAD ConvertDL_8BITS_To_32BITS_fun1 (void *p_dst, void *p_src) and calls it in the main () function, which tells the compiler to read the source data at the source data address p_src according to the byte addressing architecture, store it at the target data address p_dst after data layout conversion according to the word addressing architecture, and the converted data type is struct ECHO_HEAD.

[0021] Preferably, the terminal design specifically includes:

[0022] S1, the compiler captures the declaration of function F by matching the function name prefix, and obtains the converted source and target addressing architectures, the converted data type, and the converted source and target data addresses according to the form of function F, wherein the combination of source and target addressing architectures is denoted as convertMode, and the value is the permutation on the set A = {"8BITS", "16BITS", "32BITS"}, there are P (3, 2) = 6 conversion modes, the source data address is denoted as p_src, and the target data address is denoted as p_dst;

[0023] S2, according to the converted data type, recursively linearly expand the nested members, and calculate the source address offset Off_s [i] and the target address offset Off_d [i] of each member i in the source data addressing architecture and the target data addressing architecture respectively, and store them in the vectors Off_s and Off_d respectively. After the calculation is completed, the data layout setting of the compiler is restored to the original value;

[0024] S3, using the code insertion component of the compiler, inserting conversion code and return statement in function F to complete the definition of function F, wherein the conversion code is implemented by calling function Unpack(), and the parameters of Unpack() are: conversion mode convertMode, source address offset vector Off_s, target address offset vector Off_d, source data address p_src, target data address p_dst and member number;

[0025] S4, function Unpack() is looped for LEN times, and each member i is processed one by one: according to the source data addressing architecture, read s = Off_s [i + 1] - Off_s [i] bytes of source data at the source offset address p_src + Off_s [i], and write the source data into the target offset address p_dst + Off_d [i] of the member i according to the target data addressing architecture, to complete the data layout conversion.

[0026] Preferably, according to the data type converted in S2, the nested members of the data type are recursively linearly unfolded, each member i after unfolding is a basic type in the compiling system, and the source address offset Off_s[i] and the target address offset Off_d[i] of each member i under the addressing architecture of the source data and the addressing architecture of the target data are respectively calculated and stored in the vectors Off_s and Off_d, including:

[0027] Using a recursive function, each member type contained in the type Ty is processed one by one under the configuration of the specified data layout, the linear address offset of each member i is recursively calculated until a basic type member is encountered, and the offset address of the current member is written into the offset address vector Offsets.

[0028] The implementation form of the recursive function is voidComputeLinearOffset(DataLayout&DL,Type*Ty,SmallVectorImpl<uint64_t>&Offsets,uint64_tStartingOffset).

[0029] Preferably, the implementation form of the conversion code is Unpack(intconvertMode,void*dst,void*src,int*off_d,int*off_s,int LEN), which is used to convert the member data from the addressing architecture of the source data to the addressing architecture of the target data.

[0030] The parameter convertMode indicates the addressing architecture of the source data and the addressing architecture of the target data, the parameters src and dst are the source data address and the target data address respectively, the parameters off_s and off_d are the starting addresses of the source address offset vector Off_s and the target address offset vector Off_d respectively, and the parameter LEN is the number of members.

[0031] Preferably, the function Unpack() has the following functions: the data of each member i is read one by one for LEN times, and is converted to the target offset address.

[0032] If the addressing architecture of the source data is a byte addressing architecture and the addressing architecture of the target data is a word addressing architecture, s=Off_s[i+1]-Off_s[i] bytes of source data at the source offset address p_src+Off_s[i] are read, and are written into the target offset address p_dst+Off_d[i] of the member i according to the target addressing architecture, so as to complete the data layout conversion.

[0033] The program of reading the byte is executed on a word addressing architecture target machine, and thus needs to take corresponding bitwise AND and shift operations according to the offset value of the byte address in the word addressing architecture.

[0034] (III) Beneficial Effects

[0035] Compared with the prior art, the compiler-based aggregated type data layout automatic conversion method provided by the application has the following beneficial effects:

[0036] 1) The compiler-based method effectively solves the compatibility problem of data interaction and code porting of aggregated type data between processors of different addressing architectures.

[0037] 2) The compiled code can automatically convert the data layout of source data into the data layout of target data when executed, and the user program does not need to write conversion code.

[0038] 3) The data layout conversion is realized by the compiler, which not only has a simple and friendly interface, but also has universality, and can uniformly support data layout automatic conversion for various aggregated type data such as arrays, structures, bit fields and unions defined in the user program. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0040] Figure 1 A flowchart for the terminal designed in the application;

[0041] Figure 2 A processing flowchart of the recursive function ComputeLinearOffset() in the application;

[0042] Figure 3 A processing flowchart of the function Unpack() in the application;

[0043] Figure 4 A data layout diagram of a processor of a byte addressing architecture (8bit addressing);

[0044] Figure 5 A data layout diagram of a processor of a word addressing architecture (32bit addressing). DETAILED DESCRIPTION

[0045] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0046] A compiler-based aggregated type data layout automatic conversion method includes a front-end design and a terminal design.

[0047] The front-end design defines an interface between a user program and a compiler, and realizes the interface with the compiler by means of a function F declared by the user program, to inform the compiler of a source-target addressing architecture to be converted, a data type to be converted, a source-target data address to be converted and a conversion timing.

[0048] The terminal design completes a complete definition of the function F declared by the user program in the compiler, so that the function F has a data layout conversion function, i.e., reading source data according to a source data addressing architecture and storing the source data to a target data address according to a target data addressing architecture.

[0049] The front-end design specifically includes:

[0050] The source-target addressing architecture to be converted is informed to the compiler by means of a function name prefix, wherein a function name prefix mode is ConvertDL_XXX_To_YYY_, XXX represents a source data addressing architecture, YYY represents a target data addressing architecture, and the value range is {“8BITS”, “16BITS”, “32BITS”}, which respectively represent a byte addressing architecture (8-bit addressing mode), a half-word addressing architecture (16-bit addressing mode) and a word addressing architecture (32-bit addressing mode).

[0051] The data type to be converted is informed to the compiler by means of a function return value, including an aggregated type such as a structure, a bit field and a union, and if the data type is an array type, the array type can be embedded by using a structure type.

[0052] The source data address to be converted is informed to the compiler by means of a first function parameter, and the target data address to be converted is informed to the compiler by means of a second function parameter.

[0053] The conversion timing is informed to the compiler by means of a function call statement.

[0054] Specifically, the implementation form of the function F declared by the user program is struct ECHO_HEADConvertDL_8BITS_To_32BITS_fun1(void*p_dst,void*p_src), and it is called in the main() function to tell the compiler to read the source data at the source data address p_src according to the byte addressing architecture, and after data layout conversion, store it at the target data address p_dst according to the word addressing architecture, and the converted data type is structECHO_HEAD.

[0055] In the technical solution of this application, the declaration and call of the function F of the user program are shown in the following code:

[0056] struct ECHO_HEAD ConvertDL_8BITS_To_32BITS_fun1(void*p_dst,void*p_src);

[0057] intmain(){

[0058] int*p_dst=0x200000; / / destination data address

[0059] int*p_src=0x200400; / / Source data address of byte addressing layout

[0060] ConvertDL_8BITS_To_32BITS_fun1(p_dst,p_src);

[0061] / / Then the p_dst position data can be parsed according to the 32BIT architecture mode.

[0062] }

[0063] like Figure 1 As shown in the figure, the terminal design specifically includes:

[0064] S1. The compiler captures the declaration of function F through function name prefix matching, and obtains the converted source-destination addressing architecture, the converted data type, and the converted source-destination data address based on the form of function F. The combination of the source-destination addressing architecture is denoted as convertMode, and its value is a permutation of the set A = {"8BITS", "16BITS", "32BITS"}, with a total of P(3,2) = 6 conversion modes. The source data address is denoted as p_src, and the destination data address is denoted as p_dst.

[0065] S2, according to the converted data type, recursively linearly unfolding its nested members, each member i after unfolding being a basic type in the compiling system, and respectively calculating the source address offset Off_s[i] and the target address offset Off_d[i] of each member i under the addressing architecture of the source data and the addressing architecture of the target data, and storing the same in the vectors Off_s and Off_d, and after the calculation, restoring the data layout setting of the compiler to the original value;

[0066] S3, using the code insertion component of the compiler, inserting the conversion code and the return statement in the function F, completing the definition of the function F, wherein the conversion code is implemented through the call of the function Unpack(), and the conversion mode convertMode, the source address offset vector Off_s, the target address offset vector Off_d, the source data address p_src, the target data address p_dst and the member quantity are passed as parameters;

[0067] S4, the function Unpack() is looped for LEN times, and each member i is processed one by one: according to the addressing architecture layout of the source data, s = Off_s[i+1]-Off_s[i] bytes of source data are read at the source offset address p_src+Off_s[i], and the source data is written into the target offset address p_dst+Off_d[i] of the member i according to the addressing architecture of the target data, and the data layout conversion is completed.

[0068] Specifically, in S2, according to the converted data type, recursively linearly unfolding its nested members, each member i after unfolding being a basic type in the compiling system, and respectively calculating the source address offset Off_s[i] and the target address offset Off_d[i] of each member i under the addressing architecture of the source data and the addressing architecture of the target data, and storing the same in the vectors Off_s and Off_d, as shown in Figure 2 , including:

[0069] Using a recursive function, under the configuration of the specified data layout, each member type contained in the type Ty is processed one by one, and the linear address offset of each member i is recursively calculated until a basic type member is encountered, and the offset address of the current member is written into the offset address vector Offsets;

[0070] Wherein, the implementation form of the recursive function is voidComputeLinearOffset(DataLayout&DL,Type*Ty,SmallVectorImpl<uint64_t>&Offsets,uint64_tStartingOffset).

[0071] Specifically, the implementation form of the conversion code is Unpack(int convertMode, void *dst, void *src, int *off_d, int *off_s, int LEN), which is used to convert the member data from the addressing architecture of the source data to the addressing architecture of the target data.

[0072] In the formula, the parameter convertMode indicates the addressing architecture of the source data and the addressing architecture of the target data, the parameters src and dst are the source data address and the target data address respectively, the parameters off_s and off_d are the first addresses of the source address offset vector Off_s and the target address offset vector Off_d respectively, and the parameter LEN is the member quantity.

[0073] Specifically, as shown in the function Unpack(), the function is to read the data of each member i one by one in a loop for LEN times, and convert it to the target offset address. Figure 3

[0074] If the addressing architecture of the source data is the byte addressing architecture and the addressing architecture of the target data is the word addressing architecture, the source data of s = Off_s[i+1]-Off_s[i] bytes is read at the source offset address p_src+Off_s[i], and is written to the target offset address p_dst+Off_d[i] of the member i according to the target addressing architecture, so as to complete the data layout conversion.

[0075] In the formula, the program of reading the bytes is executed on the word addressing architecture target machine, so it is necessary to take the corresponding bitwise AND and shift operation according to the offset value of the byte address in the word addressing architecture.

[0076] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently; and these modifications or replacements will not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.​

Claims

1. A compiler-based method for automatically converting aggregate type data layout, characterized by: Including front-end design and terminal design; Front-end design defines the interface between the user program and the compiler. The interface with the compiler is implemented by the user program declaring function F to inform the compiler of the source addressing structure, data type, source data address, and conversion timing. Terminal design completes the complete definition of the function F declared by the user program in the compiler, so that the function F has the function of data layout conversion, that is, reading the source data according to the addressing architecture of the source data, and storing the source data to the target data address according to the addressing architecture of the target data.

2. The compiler-based automatic conversion method for aggregate type data layout according to claim 1, characterized in that: The front-end design specifically includes: The compiler is informed of the source and destination addressing architecture of the conversion through the function name prefix, where the function name prefix pattern is ConvertDL_XXX_To_YYY_, where XXX represents the addressing architecture of the source data and YYY represents the addressing architecture of the target data. The value range is {"8BITS","16BITS","32BITS"}, representing byte addressing architecture, half-word addressing architecture, and word addressing architecture respectively; The function return value is used to inform the compiler of the data type to be converted, including aggregate types such as structures, bit fields, and unions. If the data type is an array type, it can be embedded using the structure type. The compiler is informed of the source data address to be converted through the first function parameter, and the target data address to be converted through the second function parameter; The compiler is informed of the timing of the conversion via a function call statement.

3. The compiler-based automatic conversion method for aggregate type data layout according to claim 2, characterized in that: The implementation form of the function F declared by the user program is struct ECHO_HEAD ConvertDL_8BITS_To_32BITS_fun1(void*p_dst,void*p_src), and it is called in the main() function to tell the compiler to read the source data at the source data address p_src according to the byte addressing architecture, and store it at the target data address p_dst according to the word addressing architecture after data layout conversion, and the converted data type is struct ECHO_HEAD.

4. The compiler-based automatic conversion method for aggregate type data layout according to claim 1, characterized in that: The terminal design specifically includes: S1. The compiler captures the declaration of function F through function name prefix matching and obtains the source and destination addressing structure, the data type, and the source and destination data address of the conversion based on the form of function F. The combination of the source and destination addressing structure is denoted as convertMode, and its value is a permutation of the set A = {"8BITS", "16BITS", "32BITS"}, with a total of P(3,2) = 6 conversion modes. The source data address is denoted as p_src, and the destination data address is denoted as p_dst. S2. Recursively and linearly expand the nested members according to the converted data type. After expansion, each member i is a basic type in the compilation system. Calculate the source address offset Off_s[i] and the target address offset Off_d[i] of each member i under the addressing architecture of the source data and the addressing architecture of the target data, respectively, and store them in the vectors Off_s and Off_d respectively. After the calculation is completed, restore the compiler's data layout to the original value. S3. Use the compiler's code instrumentation component to insert conversion code and a return statement into function F to complete the definition of function F. The conversion code is implemented by calling the Unpack() function and passing the following parameters to it: conversion mode convertMode, source address offset vector Off_s, destination address offset vector Off_d, source data address p_src, destination data address p_dst, and the number of members. S4. The function Unpack() loops LEN times, processing each member i one by one: according to the addressing architecture layout of the source data, reads s=Off_s[i+1]-Off_s[i] bytes of source data at the source offset address p_src+Off_s[i], and writes the source data to the target offset address p_dst+Off_d[i] of the member i according to the addressing architecture of the target data, completing the data layout conversion.

5. The compiler-based automatic conversion method for aggregate type data layout according to claim 4, characterized in that: In S2, the nested members are recursively and linearly expanded according to the converted data type. After expansion, each member i is a basic type in the compilation system. The source address offset Off_s[i] and target address offset Off_d[i] of each member i are calculated under the addressing architecture of the source data and the addressing architecture of the target data respectively, and stored in the vectors Off_s and Off_d respectively, including: Use a recursive function to process each member type contained in type Ty one by one under the configuration of the specified data layout, recursively calculate the linear address offset for each member i until a member of the basic type is encountered, and write the offset address of the current member into the offset address vector Offsets; Among them, the implementation form of the recursive function is void ComputeLinearOffset(DataLayout&DL,Type*Ty,SmallVectorImpl<uint64_t> &Offsets,uint64_tStartingOffset).

6. The compiler-based automatic conversion method for aggregate type data layout according to claim 5, characterized in that: The conversion code is implemented in the form of Unpack(int convertMode, void*dst, void*src, int*off_d, int*off_s, int LEN), which is used to convert member data from the addressing structure of the source data to the addressing structure of the target data; Among them, the parameter convertMode indicates the addressing architecture of the source data and the addressing architecture of the target data, the parameters src and dst are the source data address and the target data address respectively, the parameters off_s and off_d are the starting addresses of the source address offset vector Off_s and the target address offset vector Off_d respectively, and the parameter LEN is the number of members.

7. The compiler-based automatic conversion method for aggregate type data layout according to claim 6, characterized in that: The function Unpack() has the following functions: loop LEN times, read the data of each member i one by one, and convert it to the target offset address; If the addressing architecture of the source data is byte addressing and the addressing architecture of the target data is word addressing, then read s = Off_s[i+1] - Off_s[i] bytes of source data at the source offset address p_src+Off_s[i] and write the target offset address p_dst+Off_d[i] of member i according to the target addressing architecture to complete the data layout conversion; Among them, the program for reading bytes is executed on a word-addressing architecture target machine, so it is necessary to take corresponding bitwise AND and shift operations according to the offset value of the byte address in the word-addressing architecture.

Images