A Transformer-based FPGA fast resource prediction method

Abstract

This invention discloses a fast FPGA resource prediction method based on Transformer, comprising the following steps: Step S1: Based on the semantics and types of words in Verilog code, a word segmentation and encoding model is constructed to segment and encode the code, converting the words into word vectors, and generating a word vector sequence that can be input into the Transformer network; Step S2: A sliding window is used to process code inputs of different lengths, and the word vector sequence input by the sliding window is processed through the multi-head self-attention mechanism of the Transformer model to generate a code relevance matrix, representing the relationship between code and hardware resource reuse, and completing feature extraction; Step S3: After performing a convolution operation on the word vectors, they are input into the sliding window again to obtain relevance matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container, and finally input into a multilayer perceptron to output the resource prediction result. This invention can effectively improve the speed of obtaining the number of FPGA resources.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and prediction technology, specifically to a fast resource prediction method for FPGA based on Transformer. Background Technology

[0002] In the development and application of FPGA projects, the compilation process often takes a lot of time, causing project delays. As algorithms in industrial scenarios become increasingly complex, traditional computing devices can no longer meet the growing demand for computing power. Therefore, using FPGAs to improve computing efficiency has become a necessary choice. When designing large-scale integrated circuits, or when the computing power of a single FPGA chip is insufficient to support complex industrial applications, complex algorithms must be divided into blocks under resource constraints. The main basis for block division is usually the number of resources on the FPGA chip. Traditional methods can only rely on the compiler to check the resource consumption after the FPGA project is compiled. However, the time cost required in this process is difficult to meet the rapid feedback requirements in the iteration of large algorithms.

[0003] Therefore, existing methods typically combine abstract syntax trees, manually extract Verilog code features, and use machine learning or code reuse strategies to predict resource consumption, thereby improving resource utilization efficiency and optimizing the design process. However, in practical applications, the resource consumption of the same subword can be affected by the code context and even the code of another module. For example, a similar variable handling operation may appear in various parts of the code. The compiler may identify the common logic and implement it with the same hardware layout, but this is not completely linearly related. That is, it is not possible to use one set of hardware logic to implement an infinite number of code logics. However, in actual execution, timing constraints need to be considered. Excessive reuse can lead to large delays that cannot meet requirements. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of excessively long prediction time for FPGA project resource consumption while maintaining high prediction accuracy, thereby providing an effective reference for project logic partitioning and shortening the project development iteration cycle. Therefore, this invention proposes a Transformer-based fast prediction method for FPGA resources based on deep learning technology, which is used to quickly predict the resource consumption of FPGA projects.

[0005] The present invention provides the following technical solution: a Transformer-based method for fast resource prediction in FPGAs, comprising:

[0006] Step S1: Based on the semantics and types of words in the Verilog code, construct a word segmentation and encoding model, segment and encode the code, encode the words into word vectors, and generate a sequence of word vectors that can be input into the Transformer network;

[0007] Preferably, the specific method for performing word segmentation and encoding of the code, and encoding subwords into word vectors, includes:

[0008] Step S101: Construct a custom vocabulary for the Verilog project, perform regular expression segmentation on the code based on the vocabulary, and divide the subwords in the code into four categories according to their semantics: keyword category, operator category, custom variable category, and number category;

[0009] Step S102: Traverse the decomposed subwords and encode the subwords using a vocabulary list to obtain word vectors;

[0010] Step S103: Concatenate all word vectors and flatten them into a one-dimensional vector, store it as a file, and obtain the complete vector representation of the project code.

[0011] Step S2: A sliding window is used to process code inputs of different lengths. The word vector sequence of the sliding window input is processed by the multi-head self-attention mechanism of the Transformer model to generate a code relevance matrix and complete feature extraction. The code relevance matrix is ​​used to characterize the relationship between code and hardware resource reuse.

[0012] Preferably, the specific process for generating the code relevance matrix includes:

[0013] Step S201: The input code word vector is segmented using a sliding window method, with the sliding window shifting by half the window size each time.

[0014] Step S202: Copy half the number of word vectors at the beginning of the project code sequence and concatenate them to the end of the code vectors to form a complete cyclic encoding sequence;

[0015] Step S203: Masking is performed on the word vectors of the concatenated part during the encoding process.

[0016] Step S3: After performing a convolution operation on the word vectors, input them into the sliding window again to obtain correlation matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container. Finally, the feature vectors are input into a multilayer perceptron to output resource prediction results.

[0017] Preferably, the specific operation process for outputting the resource prediction results includes:

[0018] Step S301: After completing the sliding window Transformer Encoder encoding, perform a convolution operation to compress the features of the encoded code word vectors and extract higher-level features.

[0019] Step S302: The code vector after convolution is further encoded by a sliding window TransformerEncoder;

[0020] Step S303: Based on the feature information after each convolution and encoding, construct a feature container, store the extracted feature values ​​according to their dimensions, and input the feature integration into the multilayer perceptron to complete the prediction.

[0021] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention utilizes a sliding window strategy and a multi-head self-attention mechanism to effectively improve the efficiency and accuracy of resource prediction for FPGA projects. While maintaining high prediction accuracy, it significantly reduces the time required for resource consumption prediction, especially in the design and optimization of large and complex algorithms. At the same time, this invention has significant advantages over traditional methods and other machine learning methods, and can also provide high-precision resource consumption prediction in a short time, providing strong support for the rapid iteration and optimization of FPGA projects. It can not only better utilize the semantic information of the source code to predict resource consumption, but also effectively improve the speed of obtaining the number of FPGA resources. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0023] In the attached diagram:

[0024] Figure 1 A schematic diagram of the overall framework of the Transformer-based FPGA fast resource prediction method provided in the embodiments of the present invention;

[0025] Figure 2 A word vector construction structure diagram provided for embodiments of the present invention;

[0026] Figure 3 The structure diagram generated by the code relevance matrix provided in the embodiments of the present invention;

[0027] Figure 4 This is a structural diagram of constructing a multidimensional feature container provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This invention combines Figures 1 to 4 The following technical solution is provided: A fast resource prediction method for FPGA based on Transformer, comprising the following steps:

[0030] Step S1: Based on the semantics and types of words in the Verilog code, construct a word segmentation and encoding model, segment and encode the code, encode the words into word vectors, and generate a sequence of word vectors that can be input into the Transformer network;

[0031] In this embodiment, combined with Figure 2 , Figure 2 Represented as a specific network structure diagram, in Verilog code, each subword c i (For example, the operator "+") may be used with a raw amount of hardware resources x i Related hardware resources include lookup tables, registers, multipliers, etc. For example, suppose the "+" operation generates x. i A lookup table (LUT) is used to generate the adder, but because an operation may occur in different code contexts, different resource reuse scenarios may arise. This value x i It will be affected by another coefficient k i The adjustment represents the final amount of hardware resources mapped to the subword, therefore the final amount of hardware resources R. i Represented as:

[0032] R i =k i ·x i

[0033] Where, k i This indicates how the quantity of resources is adjusted based on context and reuse status, x i This is the initial resource quantity corresponding to the sub-word;

[0034] In this embodiment, the specific method for performing word segmentation and encoding of the code, and encoding subwords into word vectors, includes:

[0035] Step S101: Construct a custom vocabulary for the Verilog project, perform regular expression segmentation on the code based on the vocabulary, and divide the subwords in the code into four categories according to their semantics: keyword category, operator category, custom variable category, and number category;

[0036] For example, a corresponding vocabulary is constructed for each of the four categories. The keyword and operator tables are kept consistent across all projects, while the custom data type and number tables are dynamically generated based on the specific project. Regular expressions are used to segment the code file. When a subword is a custom variable, it is added to the custom variable table in sequence and numbered. When a subword is a number, it is added to the number table. After the number table is constructed, all numbers in the number table are standardized. Furthermore, the code is decomposed into the corresponding word types based on the constructed vocabulary. Custom variables are added to the custom variable table in sequence and numbered, while numbers are added to the number table in a standardized manner.

[0037] Step S102: Traverse the decomposed subwords and encode the subwords using a vocabulary list to obtain word vectors;

[0038] For example, the decomposed subwords are traversed and word vector encoding is performed. Each subword is encoded as a two-dimensional feature. One dimension represents the type to which the subword belongs, and the other dimension represents the subword's index in that type. The indices of keywords and operators are predefined. The order in the custom variable table is determined by the order of the first traversal of the custom variables, ensuring that the index of each custom variable is unique. The index of numeric subwords directly corresponds to the subword itself.

[0039] Specifically, define x i The i-th sub-word belongs to the type t. i The index in this type is s i Then its word vector encoding is:

[0040] v(x i )=[t i s i ]

[0041] Among them, t i ∈{keyword, operator, user-defined variable, number}, s i This is the index of the subword within its type; for example, if the subword type is numeric, then s. i To obtain the standardized numbers, all sub-words are encoded to obtain the word vector representation of the project code. The word vectors of the entire project are then concatenated and flattened to obtain the word vector sequence.

[0042] Step S103: Concatenate all word vectors and flatten them into a one-dimensional vector, store it as a file, and obtain the complete vector representation of the project code;

[0043] For example, after the word vectors of the entire project are concatenated and flattened to obtain a word vector sequence, the vectors of all project code will form a dataset, and the dataset will be divided into training set and test set.

[0044] Step S2: Use a sliding window to process code inputs of different lengths. Process the word vector sequence of the sliding window input through the multi-head self-attention mechanism of the Transformer model to generate a code relevance matrix, which represents the relationship between code and hardware resource reuse, and completes feature extraction.

[0045] In this embodiment, combined with Figure 3 , Figure 3 The flowchart illustrates the operational structure of a sliding window. The specific workflow for obtaining the correlation matrix based on the sliding window and the Transformer encoder includes:

[0046] Step S201: The input code word vector is segmented using a sliding window method, with the sliding window shifting by half the window size each time.

[0047] For example, to handle variable-length code input, in practical applications, the resource consumption of the same subword can be affected by the code context and even the code of another module. For instance, a similar variable handling operation might appear in various parts of the code. The compiler might identify common logic that can be implemented using the same hardware layout, but this is not perfectly linearly dependent. That is, it's not possible to implement an infinite number of code logics with a single hardware layout. This is because timing constraints need to be considered during actual execution. Excessive reuse could lead to significant latency that fails to meet requirements. Therefore, reuse is clearly a complex process, i.e., the coefficient k... i Determined by the context, and not exhibiting a simple linear relationship;

[0048] Define k i It is determined by multiple factors, including the sub-word c. i The frequency of occurrence, the surrounding code structure, and the existing hardware resources determine the context-dependent k. i Through a function f(c) i The context is used to calculate k. i The calculation formula is as follows:

[0049] k i =f(c i (context)

[0050] The function f is adjusted according to the type of the subword, as well as different contexts and complexities;

[0051] Specifically, in order to find the correlation between different word vectors, an attention mechanism is used. However, the input of the Transformer network needs to be a fixed-length vector. Therefore, for the generated word vector sequence, this invention uses a sliding window strategy to process inputs of different lengths. The window size is defined as w, and the window will slide gradually on the word vector sequence, processing w word vectors each time.

[0052] Furthermore, the specific process of window-based capture and input is as follows:

[0053] During each slide, the Transformer encoder receives a number of word vectors of window size w; when the current window contains a word vector sequence V... window ={v(x i ), v(x) i+1 ), …, v(x) i+w-1 When these word vectors are input into the Transformer encoder for relevance analysis, a relevance matrix A is generated. i :

[0054] A i =Attention(Q) i K i V i )

[0055] Among them, Q i K i V i These represent the query, key, and value matrices, respectively. These matrices are extracted from the word vector sequence of the current window. Through self-attention, the dependencies between word vectors within the window are captured, thereby generating the relevance matrix A. i ;

[0056] Furthermore, the translation amount of each sliding window is set to half the window size, that is, the window translation amount is... The shifted window continues to perform correlation analysis, generating a new correlation matrix A. i+1 This method ensures that there is no hard slicing of vectors, avoids the loss of relevant data, and all word vectors undergo two correlation analyses through a sliding window.

[0057] Preferably, the correlation matrix A generated for all sliding windows i The correlation matrix A is obtained by summing the results. total :

[0058]

[0059] This matrix contains correlation information between all word vectors, providing a foundation for subsequent feature extraction and resource prediction.

[0060] Step S202: Copy half the number of word vectors at the beginning of the project code sequence and concatenate them to the end of the code vectors to form a complete cyclic encoding sequence;

[0061] For example, for word vectors at the end of the code that are less than the window size, copy half the number of word vectors at the beginning of the project code sequence and append them to the end of the sequence. Ensure that all word vectors are processed by the network twice. If the number of remaining word vectors is insufficient to fill the window, zero padding is used.

[0062] Step S203: Masking is applied to the word vectors of the concatenated part during the encoding process;

[0063] For example, when processing word vectors at the concatenation point, a mask is set according to the size of non-adjacent word vectors to ensure that non-adjacent word vectors are distinguished, thereby avoiding obtaining invalid correlation feature information between non-adjacent positions. If the remaining word vectors are insufficient for the window size, zero padding is required.

[0064] Specifically, when processing code sequences, if the code length cannot perfectly match the window size w, a situation arises where the word vectors at the end are insufficient to fill the window. Furthermore, the aforementioned sliding window logic only performs one correlation analysis for word vectors at the beginning and end (half the window size). To ensure that all word vectors undergo two rounds of network processing—that is, each word vector participates in the calculation of both windows—the following strategy is adopted: For the portion at the end of the code that is insufficient to fill the window, copy half the number of word vectors from the beginning of the code sequence and append them to the end of the sequence. For example, if the remaining number of word vectors is k, then copy from the beginning of the sequence... If the number of word vectors remaining is still insufficient to fill the window, we use zero padding, which means adding zero vectors to the end of the sequence until the window is completely filled. Zero-padding word vectors do not affect the calculation of the attention mechanism, thus ensuring the correctness of the processing. This ensures that all word vectors are processed twice.

[0065] Furthermore, to avoid false correlations caused by non-adjacent word vectors, a masking operation needs to be applied to the concatenated word vectors. The purpose of the masking operation is to ensure that the network can correctly distinguish word vectors in different windows when calculating correlations, thus avoiding false correlations caused by concatenation. Specifically, the masking matrix M... ijThis is used to filter out non-adjacent word vectors, ensuring that the Transformer does not consider dependencies between non-adjacent word vectors during computation. This is achieved by defining the tail word vector sequence A and the head word vector sequence B as corresponding to the sequences {v1, v2, ..., v...}. m} and {v m+1 v m+2 , ..., v n}, where m is the length of code segment A, n is the length of the concatenated complete sequence, and the elements M of the mask matrix M are... ij Set to:

[0066] When v i and v j If all words originate from word vector sequence A or word vector sequence B, i.e., i, j ∈ [1, m] or i, j ∈ [m+1, n], then M ij =0 indicates that correlation calculation is allowed;

[0067] When v i It comes from the word vector sequence A, and v j If the word vector sequence B is such that i∈[1,m] and j∈[m+1,n], or vice versa, then M ij =-∞ indicates that correlation calculation is prohibited;

[0068] For example, when calculating attention, the mask matrix is ​​compared with the correlation matrix A. i Add them together, and then perform a softmax operation on the result, specifically:

[0069] Attention(Q i K i V i = softmax(A) i +M)

[0070] By using masking, we can ensure that adjacent word vectors generate the correct relevance, while non-adjacent word vectors are masked to avoid false relevance.

[0071] Step S3: After performing convolution on the word vectors, input them into the sliding window again to obtain the correlation matrix of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container. Finally, the feature is input into the multilayer perceptron to output the resource prediction result.

[0072] In this embodiment, combined with Figure 4 , Figure 4This diagram illustrates the process of convolution operations and feature extraction. To further extract multi-dimensional features from the code, multiple convolution operations are required on the multi-word vector sequence. This increases both the visibility and the information dimensionality. Convolution operations can extract higher-order features from local regions and compress the sequence features. The convolution kernel size is set to k, and the output of the convolution operation is Z. i , represents the sequence of word vectors obtained after convolution, and the output of the convolution operation is Z. i Represented as:

[0073]

[0074] Among them, w j Z is the j-th element of the convolution kernel, b is the bias term, and the resulting Z i It is the word vector sequence after convolution. The convolution operation can better represent the complex patterns in the word vector sequence by extracting local features.

[0075] For example, the convolutional word vector sequence Z i The data is input into the next sliding window for further correlation analysis. This process is repeated, involving multiple convolutions and sliding window operations. This allows for the extraction of code features from multiple dimensions. After feature extraction for each dimension, the correlation matrix of all dimensions is generated. Stored in a multi-dimensional feature container F total In the context, the feature container F total The construction process is represented as follows:

[0076]

[0077] Among them, the feature container F total It contains feature information from all dimensions, reflecting the potential impact of the project code on hardware resource consumption across multiple dimensions;

[0078] Furthermore, the multi-dimensional feature container F total The data is fed into a multilayer perceptron (MLP) for further processing, yielding the final resource prediction result. The MLP is defined as having L layers, with each layer having an activation function f. l The final prediction result Given by the following formula:

[0079]

[0080] This allows us to obtain the network's final prediction result.

[0081] For example, the specific steps for outputting resource prediction results include:

[0082] Step S301: After completing the sliding window Transformer Encoder encoding, perform a convolution operation to compress the features of the encoded code word vectors and extract higher-level features.

[0083] For example, in Verilog code, there might be multiple statement blocks with similar logic that will ultimately be generated by the compiler with identical logic. Take a for loop as an example: a for loop is defined as a combination of multiple sub-words {c i ,c i+1 ,…,c i+n The hardware resource consumption of each sub-word may differ, but they will share a common coefficient k. for This indicates the resource adjustments made by the loop block context to the loop, from which the following total hardware resource R can be obtained. for :

[0084]

[0085] Where, k for It is the coefficient for the entire for loop, representing the influence of the preceding and following words on all subwords within the loop. (R) i It is the final amount of hardware resources, k i This indicates how the quantity of resources is adjusted based on context and reuse status, x i The coefficient k represents the initial resource quantity corresponding to the sub-word. for Determined by various factors in the context, taking a for loop as an example, we can see that a for loop is a sequence of words formed by multiple words, and different dimensions can be represented as word sequences of different lengths;

[0086] Furthermore, for any dimension d, the sub-word sequence corresponding to that dimension is defined as C. d,i,n :

[0087] C d,i,n ={c i ,c i+1 ,…,c i+n-1}

[0088] That is, C d,i,n Let R be a sequence in dimension d, starting at any position i and of length n. Both the length n and the starting position i of this sub-word sequence are variable. Therefore, it can represent the resource consumption R of all sub-word sequences in dimension d. d for:

[0089]

[0090] Where length represents the total number of sub-words in the entire project code, k jIt also indicates how the quantity of resources is adjusted based on context and reuse status, x j This also indicates the initial resource quantity corresponding to the sub-word. Analyzing the case where the sub-word sequence length is n in dimension d, there are a total of length-n sub-word sequences, and for each sequence, there exists a coefficient k. d,i This indicates the reuse relationship between the sequence and other sequences, but there may not be a reuse relationship. By superimposing the coefficients of this dimension, we can obtain the comprehensive resource utilization under this dimension.

[0091] Preferably, in the calculation of the total hardware resource consumption across all multiple dimensions, a weighted sum of the resource consumption for each dimension needs to be performed. This is defined as having N dimensions, with the final total resource consumption R... total for:

[0092]

[0093] Where, k d This represents the coefficient indicating the reuse of dimension d;

[0094] Step S302: The code vector after convolution is further encoded by a sliding window TransformerEncoder;

[0095] For example, this process ensures multi-dimensional feature extraction, further enhancing the expressive power of the encoded code features.

[0096] Step S303: Based on the feature information after each convolution and encoding, construct a feature container, store the extracted feature values ​​according to their dimensions, and input the feature integration into the multilayer perceptron to complete the prediction.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0098] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fast resource prediction method for FPGA based on Transformer, characterized in that: The method includes the following steps: Step S1: Based on the semantics and types of words in the Verilog code, construct a word segmentation and encoding model, segment and encode the code, encode the words into word vectors, and generate a sequence of word vectors that can be input into the Transformer network; The specific methods for performing word segmentation and encoding on the code, converting sub-words into word vectors, include: Step S101: Construct a custom vocabulary for the Verilog project, perform regular expression segmentation on the code based on the vocabulary, and divide the subwords in the code into four categories according to their semantics: keyword category, operator category, custom variable category, and number category; Step S102: Traverse the decomposed subwords and encode the subwords using a vocabulary list to obtain word vectors; Step S103: Concatenate all word vectors and flatten them into a one-dimensional vector, save it as a file, and obtain the complete vector representation of the project code; Step S2: A sliding window is used to process code inputs of different lengths. The word vector sequence of the sliding window input is processed by the multi-head self-attention mechanism of the Transformer model to generate a code relevance matrix and complete feature extraction. The code relevance matrix is ​​used to characterize the relationship between code and hardware resource reuse. The specific process for generating the code relevance matrix includes: Step S201: The input code word vector is segmented using a sliding window method, with the sliding window shifting by half the window size each time. Step S202: Copy half the number of word vectors at the beginning of the project code sequence and concatenate them to the end of the code vectors to form a complete cyclic encoding sequence; Step S203: Masking is applied to the word vectors of the concatenated part during the encoding process; Step S3: After performing a convolution operation on the word vectors, input them into the sliding window again to obtain correlation matrices of different dimensions. After multiple rounds of feature extraction, the features are concatenated and stored in a multi-dimensional feature container. Finally, the feature vectors are input into a multilayer perceptron to output resource prediction results. Step S3 further includes: Step S301: After completing the sliding window Transformer Encoder encoding, perform a convolution operation to compress the features of the encoded code word vectors and extract higher-level features. Step S302: The code vector after convolution is further encoded by a sliding window Transformer Encoder; Step S303: Based on the feature information after each convolution and encoding, construct a feature container, store the extracted feature values ​​according to their dimensions, and input the feature integration into the multilayer perceptron to complete the prediction.

2. The method for fast resource prediction of FPGA based on Transformer according to claim 1, characterized in that: Step S1 further includes: in the Verilog code, each subword is ultimately mapped to a hardware resource quantity R. i : R i =k i ·x i Where, k i x is a scaling factor, representing the reuse of hardware resources by the context. i This represents the amount of static hardware resources corresponding to the subword, including: lookup tables, registers, multipliers, and the number of lookup tables generated by the subword is multiplied by the coefficient k. i The adjustment represents the final amount of hardware resources mapped to the sub-word, i.e., the final amount of hardware resources R. i .

3. The method for fast resource prediction of FPGA based on Transformer according to claim 2, characterized in that: Step S102 further includes: traversing the decomposed subwords and performing word vector encoding, encoding each subword as a two-dimensional feature, one dimension representing the type of the subword, and the other dimension representing the subword's index in that type. The indices of keywords and operators are predefined, and the order in the custom variable table is determined by the order of the first traversal of the custom variables. The index of each custom variable is unique, and the index of numeric subwords directly corresponds to itself. Specifically, define c i The i-th sub-word belongs to the type t. i The index in this type is s i Then its word vector encoding is: v(x i )=[t i ,s i ] Among them, t i ∈{keyword, operator, user-defined variable, number}, s i s is the index of the subword within its type; when the subword type is numeric, s represents the index. i To obtain the standardized numbers, all sub-words are encoded to obtain the word vector representation of the project code. The word vectors of the entire project are then concatenated and flattened to obtain the word vector sequence.

4. The method for fast resource prediction of FPGA based on Transformer according to claim 3, characterized in that: Step S201 further includes: Define coefficient k i The coefficient k i Indicating the same subword c i The resource consumption is determined by multiple factors in the code context, including the sub-word c. i The frequency of occurrence, the surrounding code structure, and the existing hardware resources are all considered, therefore, through the function f(c i ,context) calculates the context relevance coefficient k i , k i The calculation formula is as follows: k i =f(c i ,context) Among them, the function f(c i The context will adjust based on the type of the subword, as well as different contexts and complexities; Furthermore, the correlation between different word vectors is determined by using an attention mechanism. A sliding window strategy is employed to process inputs of different lengths. The window size is defined as w, and the window slides progressively across the word vector sequence, processing multiple word vectors at a time. The input is then truncated based on the window size. Specifically, in each slide, a number of word vectors corresponding to the defined window size w are truncated and input to the Transformer encoder; when the current window contains a word vector sequence V... window ={v(x i ),v(x i+1 ),…,v(x i+w-1 When the word vectors are input into the Transformer encoder, they will be used for relevance analysis to generate a relevance matrix. And set the translation amount of the window each time it slides to half the window size, that is, the window translation amount is... The shifted window continues to perform correlation analysis, generating a new correlation matrix A. i+1 All word vectors underwent two relevance analyses using a sliding window, and the relevance matrix A generated from all sliding windows was analyzed. i The correlation matrix A is obtained by summing the results. total .

5. The method for fast resource prediction of FPGA based on Transformer according to claim 4, characterized in that: The step S203 further includes: when processing the word vectors at the splicing point, setting a mask according to the size of the non-adjacent word vectors to distinguish the non-adjacent word vectors; when there are remaining word vectors after the distinction and the remaining word vectors are smaller than the window size, performing zero-padding operation. Specifically, a masking operation is applied to the word vectors of the concatenated portion. The masking operation process includes: defining a mask matrix M. ij The mask matrix M ij Used to mask non-adjacent word vectors, Transformer does not consider the dependencies between non-adjacent word vectors during computation. This is achieved by defining the tail word vector sequence A and the head word vector sequence B, which correspond to the sequences {v1, v2, ..., v...} respectively. m } and {v m+1 ,v m+2 ,…,v n }, where m is the length of code segment A, n is the length of the concatenated complete sequence, and the elements M of the mask matrix M are... ij Set to: When v i and v j If all words originate from word vector sequence A or word vector sequence B, i.e., i,j∈[1,m] or i,j∈[m+1,n], then M ij =0 indicates that correlation calculation is allowed; when v i It comes from the word vector sequence A, and v j If the word vector sequence B is such that i∈[1,m] and j∈[m+1,n], or vice versa, then M ij =-∞ indicates that correlation calculation is prohibited; when calculating attention, the mask matrix is ​​compared with the correlation matrix A. i Add them together and then perform a softmax operation on the result.

6. The method for fast resource prediction of FPGA based on Transformer according to claim 5, characterized in that: Step S301 further includes: for any dimension d, defining the sub-word sequence corresponding to that dimension as C. d,i,n : C d,i,n ={c i ,c i+1 ,…,c i+n-1 } C d,i,n Let R be a sequence in dimension d, starting at any position i and of length n. Both the length n and the starting position i of this sub-word sequence are variable. Therefore, it can represent the resource consumption R of all sub-word sequences in dimension d. d for: Where length represents the total number of sub-words in the entire project code, k j It also indicates how the quantity of resources is adjusted based on context and reuse status, x j This also indicates the initial resource quantity corresponding to the sub-word. Analyzing the case where the sub-word sequence length is n in dimension d, there are a total of length-n sub-word sequences, and for each sequence, there exists a coefficient k. d,i This indicates a reuse relationship between the sequence and other sequences; however, no reuse relationship may exist. k d This represents the reuse coefficient of dimension d.

Images