Transform-based FPGA (Field Programmable Gate Array) rapid resource prediction method

Abstract

The invention discloses an FPGA rapid resource prediction method based on Transform, and the method comprises the following steps: S1, building a word segmentation coding model based on the semantics and types of sub-words in Verilog codes, carrying out the word segmentation coding of the codes, coding the sub-words into word vectors, and generating a word vector sequence which can be input into a Transform network; s2, code input of different lengths is processed through a sliding window, a word vector sequence input by the sliding window is processed through a multi-head self-attention mechanism of a Transform model, a code correlation matrix is generated, a code and hardware resource reuse relation is represented, and feature extraction is completed; and S3, performing convolution operation on the word vectors, inputting the word vectors into the sliding window again, obtaining correlation matrixes of different dimensions, splicing features through multiple rounds of feature extraction, storing the spliced features into a multi-dimensional feature container, finally inputting the spliced features into a multi-layer perceptron, and outputting a resource prediction result. According to the invention, the speed of obtaining the number of FPGA resources can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and prediction technology, and in particular to a Transformer-based FPGA fast resource prediction method. Background Art

[0002] In the development and application of FPGA projects, the compilation process often takes a lot of time, resulting in project delays. As algorithms in industrial scenarios become increasingly complex, traditional computing devices can no longer meet the growing demand for computing power. Therefore, using FPGA to improve computing efficiency has become a necessary choice. When designing large integrated circuits, or when the computing power of a single FPGA chip is insufficient to support complex industrial applications, complex algorithms must be divided into blocks under resource constraints, and the main basis for block division is usually the number of resources on the FPGA chip. Traditional methods can only rely on the compiler to query resource consumption after completing the compilation of the FPGA project. However, the time cost required in this process is difficult to meet the rapid feedback requirements in large-scale algorithm iterations.

[0003] Therefore, existing methods usually combine abstract syntax trees, manually extract Verilog code features, and use machine learning or design code reuse strategies to predict resource consumption, thereby improving resource utilization efficiency and optimizing the design process. However, in actual applications, the resource consumption of the same subword will be affected by the code context and even the code of another module. For example, a similar variable processing operation may appear in various places in the code. The compiler may sort out some of the logic that can be shared and implement it with the same hardware arrangement, but this will not be completely linearly related, that is, one piece of hardware logic will not be used to implement an infinite number of code logics. However, in the actual execution process, the timing constraints need to be considered. Excessive reuse will lead to large delays and fail to meet the requirements. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of long prediction time of FPGA project resource consumption while maintaining high prediction accuracy, so as to provide an effective reference for project logic partitioning and shorten the project development iteration cycle. Therefore, the present invention proposes a Transformer-based FPGA resource rapid prediction method based on deep learning technology, which is used to quickly predict the resource consumption of FPGA projects.

[0005] That is, the present invention provides the following technical solution: a Transformer-based FPGA fast resource prediction method, comprising:

[0006] Step S1: Based on the semantics and types of subwords in the Verilog code, a word segmentation encoding model is constructed to perform word segmentation encoding on the code, encode the subwords into word vectors, and generate a word vector sequence that can be input into the Transformer network;

[0007] Preferably, the specific method of performing word segmentation encoding on the code and encoding the subwords into word vectors includes:

[0008] Step S101: construct a custom vocabulary of the Verilog project, perform regular word segmentation on the code according to the vocabulary, and divide the subwords in the code into four categories according to semantics: keyword class, operator class, custom variable class and number class;

[0009] Step S102: traverse the decomposed sub-words, and encode the sub-words using the vocabulary to obtain word vectors;

[0010] Step S103: concatenate and flatten all word vectors into a one-dimensional vector, store it as a file, and obtain a complete vector representation of the project code.

[0011] Step S2: Use a sliding window to process code inputs of different lengths, process the word vector sequence of the sliding window input through the multi-head self-attention mechanism of the Transformer model, generate a code correlation matrix, and complete feature extraction. The code correlation matrix is ​​used to characterize the reuse relationship between code and hardware resources;

[0012] Preferably, the specific process of generating the code correlation matrix includes:

[0013] Step S201: Segment the input code word vector using a sliding window method, where the translation amount of each sliding window is half of the window size;

[0014] Step S202: copy the word vectors of half the window size at the beginning of the project code sequence and splice them to the end of the code vector to form a complete cyclic coding sequence;

[0015] Step S203: during the encoding process, a mask operation is performed on the word vectors of the concatenated part.

[0016] Step S3: After the word vector is convolved, it is input into the sliding window again to obtain correlation matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container, and finally input into a multi-layer perceptron to output resource prediction results.

[0017] Preferably, the specific operation process of outputting resource prediction results includes:

[0018] Step S301: After the sliding window Transformer Encoder encoding is completed, a convolution operation is performed to compress the features of the encoded code word vector through convolution to extract higher-level features;

[0019] Step S302: the code vector after the convolution operation is further encoded by the sliding window TransformerEncoder again;

[0020] Step S303: construct a feature container based on the feature information after each convolution and encoding, store the extracted feature values ​​by dimension, and complete the prediction after inputting the feature into the multi-layer perceptron through feature integration.

[0021] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention utilizes a sliding window strategy and a multi-head self-attention mechanism to effectively improve the efficiency and accuracy of FPGA project resource prediction, and while maintaining a high prediction accuracy, significantly reduces the time for resource consumption prediction, especially in the design and optimization of large and complex algorithms. At the same time, the present invention has significant advantages over traditional methods and other machine learning methods, and can also provide high-precision resource consumption prediction in a short time, providing strong support for the rapid iteration and optimization of FPGA projects. It can not only better utilize the semantic information of the source code to predict resource consumption, but also effectively improve the speed of obtaining the number of FPGA resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0023] In the attached picture:

[0024] Figure 1 A schematic diagram of the overall framework of a Transformer-based FPGA fast resource prediction method provided in an embodiment of the present invention;

[0025] Figure 2 A structural diagram of word vector construction provided by an embodiment of the present invention;

[0026] Figure 3 A structural diagram of the code correlation matrix generated by an embodiment of the present invention;

[0027] Figure 4 A structural diagram of constructing a multidimensional feature container provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The present invention combines Figures 1 to 4 , providing the following technical solution: a Transformer-based FPGA fast resource prediction method, comprising the following steps:

[0030] Step S1: Based on the semantics and types of subwords in the Verilog code, a word segmentation encoding model is constructed to perform word segmentation encoding on the code, encode the subwords into word vectors, and generate a word vector sequence that can be input into the Transformer network;

[0031] In this embodiment, combined with Figure 2 , Figure 2 Represented as a network structure diagram, in the Verilog code, each subword c i (e.g. operator "+") may be associated with a raw hardware resource amount x i Related, hardware resources include: lookup table, register, multiplier, etc. For example, assuming that the "+" operation will generate x i A lookup table (LUT) is used to generate the adder, but since an operation may appear in different code contexts, different resource reuse situations may occur. i will be affected by another coefficient k i Adjustment, which means the number of hardware resources that the subword is finally mapped to, so the final amount of hardware resources R i It is expressed as:

[0032] R i =k i ·x i

[0033] Among them, k i Indicates the adjustment of the number of resources by context and reuse, x i is the initial resource amount corresponding to the subword;

[0034] In this embodiment, the code is segmented and encoded, and the specific method of encoding the subwords into word vectors includes:

[0035] Step S101: construct a custom vocabulary of the Verilog project, perform regular word segmentation on the code according to the vocabulary, and divide the subwords in the code into four categories according to semantics: keyword class, operator class, custom variable class and number class;

[0036] Exemplarily, a corresponding vocabulary is constructed for each of the four divided categories, wherein the keyword table and the operator table remain consistent in all projects, the custom data type table and the number table are dynamically generated according to the specific project, and the code file is segmented using regular expressions. When a sub-word is a custom variable, the sub-word is added to the custom variable table in sequence and numbered. When the sub-word type is a number, it is added to the number table. After the number table is constructed, all numbers in the number table are standardized, and the code is further decomposed into corresponding word types according to the constructed vocabulary, and the custom variables are added to the custom variable table in sequence and numbered, and the numbers are added to the number table in a standardized manner.

[0037] Step S102: traverse the decomposed sub-words, and encode the sub-words using the vocabulary to obtain word vectors;

[0038] Exemplarily, the decomposed subwords are traversed and word vector encoding is performed, and each subword is encoded into a two-dimensional feature, one dimension represents the type of the subword, and the other dimension represents the sequence number of the subword in the type. The sequence numbers of keywords and operators are predefined, and the order in the custom variable table is determined by the order of the first traversal of the custom variables, ensuring that the sequence number of each custom variable is unique, and the sequence number of the digital subword directly corresponds to itself;

[0039] Specifically, define x i is the i-th subword, and its type is t i , the serial number in this type is s i , then its word vector encoding is:

[0040] v(x i )=[t i ,s i ]

[0041] Among them, t i ∈{keyword, operator, custom variable, number}, s i is the sequence number of the subword in its type. If the subword type is a number, s i is a standardized number. After encoding all subwords, we get the word vector representation of the project code. Then we concatenate and flatten the word vectors of the entire project to get the word vector sequence.

[0042] Step S103: concatenate and flatten all word vectors into a one-dimensional vector, store it as a file, and obtain a complete vector representation of the project code;

[0043] For example, after the word vectors of the entire project are concatenated and flattened to obtain a word vector sequence, the vectors of all project codes will form a data set and be divided into a training set and a test set.

[0044] Step S2: Use a sliding window to process code inputs of different lengths, and use the Transformer model's multi-head self-attention mechanism to process the word vector sequence of the sliding window input, generate a code correlation matrix, characterize the relationship between code and hardware resource reuse, and complete feature extraction;

[0045] In this embodiment, combined Figure 3 , Figure 3 Represented as an operation structure diagram of a sliding window, the specific operation process of obtaining the correlation matrix based on the sliding window and the Transformer encoder includes:

[0046] Step S201: Segment the input code word vector using a sliding window method, where the translation amount of each sliding window is half of the window size;

[0047] For example, in order to process code input of variable length, in actual applications, the resource consumption of the same subword will be affected by the code context or even another module code. For example, a similar variable processing operation may appear in various places in the code. The compiler may sort out some of the logic that can be shared and implement it with the same hardware arrangement, but this will not be completely linearly related, that is, one piece of hardware logic will not be used to implement an infinite number of code logics, because the timing constraints need to be considered in the actual execution process. Excessive reuse may cause a large delay that cannot meet the requirements. Therefore, reuse is obviously a complex process, that is, the coefficient k i It is determined by the context and does not have a simple linear relationship;

[0048] Define k i It is determined by multiple factors, including the subword c i The frequency of occurrence, the surrounding code structure, the existing hardware resources, and thus the context relevance k i Through a function f(c i , context) to calculate, that is, k i The calculation formula is as follows:

[0049] k i =f(c i , context)

[0050] Among them, the function f will be adjusted according to the type of subword and different contextual situations and complexities;

[0051] Specifically, in order to find the correlation between different word vectors, the attention mechanism is used, and the input of the Transformer network needs to be a fixed-length vector. Therefore, for the generated word vector sequence, the present invention uses a sliding window strategy to process inputs of different lengths, and defines the window size as w. The window will slide gradually on the word vector sequence, processing w word vectors each time;

[0052] Furthermore, the specific process of window-based interception and input is as follows:

[0053] In each slide, the word vectors of window size w are intercepted and input into the Transformer encoder; when the current window contains the word vector sequence V window = {v(x i ), v(x i+1 ),…,v(x i+w-1 )}, these word vectors will be input into the Transformer encoder for correlation analysis to generate the correlation matrix A i :

[0054] A i =Attention(Q i , K i , V i )

[0055] Among them, Q i , K i , V i They represent query, key, and value matrices respectively. These matrices are extracted from the word vector sequence of the current window. The self-attention mechanism can capture the dependency between the word vectors in the window, thereby generating a correlation matrix A. i ;

[0056] Furthermore, the translation amount of each sliding window is set to half of the window size, that is, the window translation amount is The translated window continues to perform correlation analysis to generate a new correlation matrix A i+1 In this way, we can ensure that there is no hard slicing of vectors, avoid the loss of correlation data, and all word vectors will be subjected to two correlation analyses through the sliding window;

[0057] Preferably, the correlation matrix A generated for all sliding windows is i Accumulate and get the final correlation matrix A total :

[0058]

[0059] This matrix contains the correlation information between all word vectors, providing a basis for subsequent feature extraction and resource prediction.

[0060] Step S202: copy the word vectors of half the window size at the beginning of the project code sequence and splice them to the end of the code vector to form a complete cyclic coding sequence;

[0061] Exemplarily, for word vectors at the end of the code that are less than the window size, copy the word vectors of half the window size at the beginning of the project code sequence and splice them to the end of the sequence to ensure that all word vectors are processed by two rounds of network. When the number of remaining word vectors is not enough to fill the window, zero padding is used.

[0062] Step S203: during the encoding process, a mask operation is performed on the word vectors of the concatenated part;

[0063] Exemplarily, when processing the word vectors at the splicing location, a mask is set according to the size of non-adjacent word vectors to ensure that non-adjacent word vectors are distinguished, thereby avoiding obtaining invalid correlation feature information between non-adjacent positions. If the remaining word vectors are less than the window size, a zero padding operation is required;

[0064] Specifically, when processing a code sequence, when the code length cannot completely match the window size w, there will be a situation where the word vector at the end is less than the window size, and the above sliding window logic will only perform one correlation analysis on the word vectors of the first and last half windows. To ensure that all word vectors are processed by two rounds of network, that is, each word vector participates in the calculation of two windows, the following strategy is adopted: For the part of the code that is less than the window size at the end, copy the word vectors of half the window size at the beginning of the project code sequence and splice them to the end of the sequence. For example, if the number of remaining word vectors is k, copy them from the beginning of the sequence. The word vectors are concatenated to the end. When the number of remaining word vectors is still insufficient to fill the window, we use zero padding, that is, adding zero vectors at the end of the sequence until the window is completely filled. The zero-padded word vectors will not affect the calculation of the attention mechanism, thus ensuring the correctness of the processing, thereby ensuring that all word vectors will be processed twice;

[0065] Furthermore, in order to avoid false correlation between non-adjacent word vectors, it is necessary to apply a masking operation to the word vectors of the spliced ​​parts. The role of the masking operation is to ensure that the network can correctly distinguish the word vectors in different windows when calculating the correlation, and avoid false correlation caused by splicing. Specifically, the mask matrix M ijIt is used to shield those non-adjacent word vectors, so as to ensure that the Transformer does not consider the dependency between non-adjacent word vectors during calculation. By defining the tail word vector sequence A and the head word vector sequence B to correspond to the sequence {v 1 , v 2 , …, v m} and {v m+1 , v m+2 , …, v n}, where m is the length of the code segment A, n is the length of the complete sequence after concatenation, and the element M of the mask matrix M ij is set to:

[0066] When v i and v j All of them come from word vector sequence A or word vector sequence B, that is, i, j∈[1, m] or i, j∈[m+1, n], then M ij =0, indicating that correlation calculation is allowed;

[0067] When v i From the word vector sequence A, and v j From the word vector sequence B, that is, when i∈[1,m] and j∈[m+1,n], or vice versa, then M ij =-∞, indicating that correlation calculation is prohibited;

[0068] For example, when calculating attention, the mask matrix is ​​combined with the correlation matrix A i Add and perform softmax operation on the result, specifically:

[0069] Attention(Q i , K i , V i )=softmax(A i +M)

[0070] Through the masking operation, it can be ensured that adjacent word vectors have the correct correlation, while non-adjacent word vectors will be masked to avoid false correlation.

[0071] Step S3: After the word vector is convolved, it is input into the sliding window again to obtain correlation matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container. Finally, it is input into a multi-layer perceptron to output the resource prediction result.

[0072] In this embodiment, combined with Figure 4 , Figure 4This is a flow chart of convolution operation and feature extraction. In order to further extract the multi-dimensional features of the code, multiple convolution operations are required on the multi-word vector sequence to increase the visual field and the information dimension. The convolution operation can extract higher-order features from the local area and compress the sequence. The convolution kernel size is set to k, and the output of the convolution operation is Z. i , represents the word vector sequence obtained after convolution, and the output of the convolution operation is Z i It is expressed as:

[0073]

[0074] Among them, w j is the jth element of the convolution kernel, b is the bias term, and the resulting Z i It is the word vector sequence after convolution. The convolution operation can better represent the complex patterns in the word vector sequence by extracting local features.

[0075] Exemplary, convolutional word vector sequence Z i It is input into the next sliding window to continue the correlation analysis. This process is repeated. Through multiple convolutions and sliding window processing, the features of the code can be extracted from multiple dimensions. After the feature extraction of each dimension is completed, the correlation matrix of all dimensions is Stored in a multi-dimensional feature container F total In the feature container F total The construction process is expressed as:

[0076]

[0077] Among them, the feature container F total Contains feature information of all dimensions, reflecting the potential impact of project code on hardware resource consumption in multiple dimensions;

[0078] Furthermore, the multi-dimensional feature container F total Input into the multi-layer perceptron (MLP) for further processing and obtaining the final resource prediction result Define MLP to contain L layers, and the activation function of each layer is f l , then the final prediction result It is given by the following formula:

[0079]

[0080] In this way, the final prediction result of the network can be obtained.

[0081] Exemplarily, the specific steps of outputting resource prediction results include:

[0082] Step S301: After the sliding window Transformer Encoder encoding is completed, a convolution operation is performed to compress the features of the encoded code word vector through convolution to extract higher-level features;

[0083] For example, in Verilog code, there may be multiple statement blocks with similar logic, and the compiler will generate the same logic in the end. For example, a for loop is defined to consist of multiple subwords {c i ,c i+1 ,…,c i+n}, the hardware resource consumption of each subword may be different, but they will have a common coefficient k for , which indicates the resource adjustment of the loop block context to the loop, from which the following hardware resource sum R can be obtained for :

[0084]

[0085] Among them, k for It is the coefficient for the entire for loop, indicating the influence of the loop upper and lower levels on all subwords in the loop. i is the final amount of hardware resources, k i Indicates the adjustment of the number of resources by context and reuse, x i Indicates the initial number of resources corresponding to the subword, coefficient k for It is determined by many factors in the context. Therefore, based on a for loop as an example, it can be found that the for loop is a sub-word sequence formed by multiple sub-words. Different dimensions can be represented as sub-word sequences of different lengths.

[0086] Furthermore, for any dimension d, the subword sequence corresponding to the dimension is defined as C d,i,n :

[0087] C d,i,n ={c i ,c i+1 ,…,c i+n-1}

[0088] C d,i,n It is represented as a sequence in dimension d, which starts from any position i and has a length of n. The length n and the starting position i of the subword sequence are variable, so the resource consumption R of all subword sequences in dimension d can be represented d for:

[0089]

[0090] Among them, length represents the number of subwords in the entire project code, k jIt also indicates the adjustment of the number of resources by context and reuse, x j It also represents the number of initial resources corresponding to the subword. In the case of subword sequence length n in dimension d, there are a total of length-n subword sequences, and for each sequence there is a coefficient k d,i Indicates the reuse relationship between it and other sequences, or there may be no reuse relationship. By superimposing the coefficients of this dimension, the comprehensive resource usage under this dimension can be obtained;

[0091] Preferably, in the calculation of the total hardware resource consumption of the entire multi-dimensional space, the resource consumption of each dimension needs to be weighted and summed. It is defined that there are N dimensions, and the final total resource consumption R total for:

[0092]

[0093] Among them, k d represents the reuse coefficient of dimension d;

[0094] Step S302: the code vector after the convolution operation is further encoded by the sliding window TransformerEncoder again;

[0095] Exemplarily, this processing ensures multi-dimensional extraction of features and further enhances the expressiveness of the encoded code features.

[0096] Step S303: construct a feature container based on the feature information after each convolution and encoding, store the extracted feature values ​​by dimension, and complete the prediction after inputting the feature into the multi-layer perceptron through feature integration.

[0097] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0098] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A Transformer-based FPGA fast resource prediction method, characterized by: The method comprises the following steps: Step S1: Based on the semantics and types of subwords in the Verilog code, a word segmentation encoding model is constructed to perform word segmentation encoding on the code, encode the subwords into word vectors, and generate a word vector sequence that can be input into the Transformer network; Step S2: Use a sliding window to process code inputs of different lengths, process the word vector sequence of the sliding window input through the multi-head self-attention mechanism of the Transformer model, generate a code correlation matrix, and complete feature extraction. The code correlation matrix is ​​used to characterize the reuse relationship between code and hardware resources; Step S3: After the word vector is convolved, it is input into the sliding window again to obtain correlation matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container. Finally, the features are input into a multi-layer perceptron to output the resource prediction results.

2. The Transformer-based FPGA fast resource prediction method according to claim 1, characterized in that: The step S1 also includes: in the Verilog code, each subword is finally mapped to a hardware resource amount R i : R i =k i ·x i Among them, k i is the scaling factor, representing the reuse of hardware resources by the context, x i represents the static hardware resource amount corresponding to the subword, the hardware resources include: lookup table, register, multiplier, the number of lookup tables generated by the subword is multiplied by the coefficient k i Adjustment, which means the number of hardware resources that the subword is finally mapped to, that is, the final amount of hardware resources R i .

3. The Transformer-based FPGA fast resource prediction method according to claim 2, characterized in that: The specific method of performing word segmentation encoding on the code and encoding the subwords into word vectors includes: Step S101: construct a custom vocabulary of the Verilog project, perform regular word segmentation on the code according to the vocabulary, and divide the subwords in the code into four categories according to semantics: keyword class, operator class, custom variable class and number class; Step S102: traverse the decomposed sub-words, and encode the sub-words using the vocabulary to obtain word vectors; Step S103: concatenate and flatten all word vectors into a one-dimensional vector, store it as a file, and obtain a complete vector representation of the project code.

4. The Transformer-based FPGA fast resource prediction method according to claim 3, characterized in that: The step S102 further includes: traversing the decomposed sub-words and performing word vector encoding, encoding each sub-word into a two-dimensional feature, one dimension represents the type to which the sub-word belongs, and the other dimension represents the sequence number of the sub-word in the type, the sequence numbers of keywords and operators are pre-defined, the order in the custom variable table is determined by the order of the first traversal of the custom variables, the sequence number of each custom variable is unique, and the sequence number of the digital sub-word directly corresponds to itself; Specifically, define c i is the i-th subword, and its type is t i , the serial number in this type is s i , then its word vector encoding is: v(x i )=[t i ,s i ] Among them, t i ∈{keyword, operator, custom variable, number}, s i is the sequence number of the subword in its type. When the subword type is a number, s i is a standardized number. After encoding all subwords, we get the word vector representation of the project code. Then we concatenate and flatten the word vectors of the entire project to get the word vector sequence.

5. The Transformer-based FPGA fast resource prediction method according to claim 1, characterized in that: The specific process of generating the code correlation matrix includes: Step S201: Segment the input code word vector using a sliding window method, where the translation amount of each sliding window is half of the window size; Step S202: copy the word vectors of half the window size at the beginning of the project code sequence and splice them to the end of the code vector to form a complete cyclic coding sequence; Step S203: during the encoding process, a mask operation is performed on the word vectors of the concatenated part.

6. The Transformer-based FPGA fast resource prediction method according to claim 5, characterized in that: The step S201 further comprises: Define the coefficient k i , the coefficient k i Represents the same subword c i The resource consumption of is determined by multiple factors in the code context, including the subword c i The frequency of occurrence, the surrounding code structure, and the existing hardware resources are therefore i ,context) calculates the context relevance coefficient k i , k i The calculation formula is as follows: k i =f(c i ,context) Among them, the function f(c i ,context) will be adjusted according to the type of subword and different contextual situations and complexity; We further determine the correlation between different word vectors by using the attention mechanism, and process inputs of different lengths by adopting a sliding window strategy. The window size is defined as w, and the window gradually slides on the word vector sequence. Multiple word vectors are processed each time, and interception and input are performed based on the window. The specific process is as follows: In each slide, the word vectors corresponding to the defined window size w are intercepted and input into the Transformer encoder; when the current window contains the word vector sequence V window = {v(x i ),v(x i+1 ),…,v(x i+w-1 )}, the word vector will be input into the Transformer encoder for correlation analysis to generate a correlation matrix; The translation amount of each sliding window is set to half of the window size, that is, the window translation amount is The translated window continues to perform correlation analysis to generate a new correlation matrix A i+1 , all word vectors are subjected to two correlation analyses through sliding windows, and the correlation matrices Ai generated by all sliding windows are accumulated to obtain the final correlation matrix A total .

7. The Transformer-based FPGA fast resource prediction method according to claim 5, characterized in that: The step S203 further includes: when processing the word vectors at the splicing location, setting a mask according to the size of the non-adjacent word vectors to distinguish the non-adjacent word vectors, and performing a zero padding operation when there are remaining word vectors after the distinction and the remaining word vectors are smaller than the window size; The mask operation is applied to the word vectors of the concatenated parts. The specific process of the mask operation includes: defining the mask matrix M ij , the mask matrix M ij It is used to shield non-adjacent word vectors. Transformer does not consider the dependency between non-adjacent word vectors during calculation. By defining the tail word vector sequence A and the head word vector sequence B to correspond to the sequence {v1, v2, …, v m } and {v m+1 ,v m+2 ,…,v n }, where m is the length of the code segment A, n is the length of the complete sequence after concatenation, and the element M of the mask matrix M ij is set to: When v i and v j All of them come from word vector sequence A or word vector sequence B, that is, i,j∈[1,m] or i,j∈[m+1,n], then M ij = 0, indicating that correlation calculation is allowed; when v i From the word vector sequence A, and v j From the word vector sequence B, that is, when i∈[1,m] and j∈[m+1,n], or vice versa, then M ij =-∞, indicating that the calculation of correlation is prohibited; when calculating attention, the mask matrix is ​​combined with the correlation matrix A i Add them together and perform a softmax operation on the result.

8. The Transformer-based FPGA fast resource prediction method according to claim 1, characterized in that: The step S3 further comprises: Step S301: After the sliding window Transformer Encoder encoding is completed, a convolution operation is performed to compress the features of the encoded code word vector through convolution to extract higher-level features; Step S302: the code vector after the convolution operation is further encoded by the sliding window Transformer Encoder again; Step S303: construct a feature container based on the feature information after each convolution and encoding, store the extracted feature values ​​by dimension, and complete the prediction after inputting the feature into the multi-layer perceptron through feature integration.

9. The method for rapid FPGA resource prediction based on Transformer according to claim 8, characterized in that: The step S301 further includes: for any dimension d, defining the subword sequence corresponding to the dimension as C d,i,n : C d,i,n ={c i ,c i+1 ,…,c i+n-1 } C d,i,n It is represented as a sequence in dimension d, which starts from any position i and has a length of n. The length n and the starting position i of the subword sequence are variable, so the resource consumption R of all subword sequences in dimension d can be represented d for: Among them, length represents the number of subwords in the entire project code, k j It also indicates the adjustment of the number of resources by context and reuse, x j It also represents the number of initial resources corresponding to the subword. In the case of subword sequence length n in dimension d, there are a total of length-n subword sequences, and for each sequence there is a coefficient k d,i Indicates the reuse relationship between it and other sequences, or there may be no reuse relationship, k d Represents the reuse coefficient of dimension d.

Images