Method, device and electronic device for generating syntax tree of code file

Abstract

The present application relates to the field of computer software development, and discloses a method, device, and electronic device for generating a syntax tree for a code file, wherein the method for generating a syntax tree for a code file comprises: upon receiving a code file to be parsed in a predetermined programming language, parsing each lexical symbol in the code file to be parsed by a lexical parsing module and generating a corresponding linear linked list; then determining the part of speech of each lexical symbol in the linear linked list in turn, and whenever the part of speech of any lexical symbol is determined, searching a first lookup table and a second lookup table according to the part of speech of any lexical symbol to obtain corresponding search results, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of the predetermined programming language; and generating a syntax tree for the code file to be parsed according to the search results.

Description

Technical Field

[0001] The present application relates to the technical field of computer software development, and in particular to a method, device, and electronic device for generating a syntax tree for a code file. Background Art

[0002] In today's computer science field, compiler development technology for high-level programming languages ​​like C / C++ / JAVA is becoming increasingly mature. High-level language compilers enable the conversion of programming languages ​​into machine language. However, current compilers cannot comprehensively and thoroughly analyze coding errors within code files, nor can they accurately identify or correct them. Therefore, before compiling a code file, program developers must manually check for errors.

[0003] However, during the specific implementation process, the inventors of this application discovered that when the amount of code in a code file is large, it will cause a huge workload for program developers, not only causing program developers to spend a lot of time and energy to check for errors in the code file, but also extremely low checking efficiency. At the same time, the inventors of this application discovered that according to the part of speech of each lexical symbol in the code file, searching the corresponding lookup table can generate a syntax tree (syntaxtree) of the code file, thereby automatically realizing the checking and correction of grammatical errors, writing errors, etc. in the code file, greatly improving the checking efficiency. Based on this, the inventors of this application provide a method, device and electronic device for generating the grammar of a code file. Summary of the Invention

[0004] The purpose of this application is to solve at least one of the above technical deficiencies, and the following technical solutions are proposed:

[0005] In a first aspect, a method for generating a syntax tree of a code file is provided, comprising:

[0006] When receiving a code file to be parsed in a predetermined programming language, the lexical parsing module parses each lexical symbol in the code file to be parsed and generates a corresponding linear linked list;

[0007] Determining the part of speech of each lexical symbol in the linear linked list in sequence, and whenever the part of speech of any lexical symbol is determined, searching a first lookup table and a second lookup table according to the part of speech of the lexical symbol to obtain a corresponding search result, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of the predetermined programming language;

[0008] Generate a syntax tree for the code file to be parsed based on the search results.

[0009] Specifically, determining the part of speech of any lexical symbol in the linear linked list includes:

[0010] Performing part-of-speech determination on any lexical symbol according to the context to determine the part of speech of any lexical symbol; and / or,

[0011] The part of speech of any lexical symbol is guessed according to the context to determine the part of speech of any lexical symbol.

[0012] Furthermore, the part of speech of any lexical symbol is determined according to the context, including any of the following situations:

[0013] When any lexical symbol is any of type name, object name and keyword, it is regarded as the part of speech of any lexical symbol;

[0014] When any lexical symbol is a lexical symbol of a preset type, determine whether the lexical symbol is a template list symbol or an operation symbol, and use the determination result as the part of speech of the lexical symbol;

[0015] Perform information lookup in the scope and use the result as the part of speech of any lexical symbol.

[0016] Furthermore, information search is performed in the scope, and the search result is used as the part of speech of any lexical symbol, including:

[0017] Perform a lateral search in the scope and use the result of the lateral search as the part of speech of any lexical symbol;

[0018] A horizontal search is a search in the current scope and the referenced namespace, or a search in the base class scope of the current scope.

[0019] Furthermore, it also includes:

[0020] If no search result is found through the horizontal search, a vertical search is performed in the scope, and the search result of the vertical search is used as the part of speech of any lexical symbol;

[0021] A vertical search is a search in the scope that includes the current scope.

[0022] Furthermore, the part of speech of any lexical symbol is guessed according to the context to determine the part of speech of any lexical symbol, including:

[0023] Guess whether the part of speech of any lexical symbol is a namespace name based on the context;

[0024] If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after any lexical symbol is not a lexical symbol of the preset type, then the part of speech of any lexical symbol is determined to be a type name; if the first lexical symbol after any lexical symbol is a lexical symbol of the preset type, then the part of speech of any lexical symbol is determined by performing the first preset processing according to any lexical symbol.

[0025] Furthermore, it also includes:

[0026] If the guess is not a namespace name, and when the part of speech of any lexical symbol is guessed to be an object name and not a type name, the part of speech of any lexical symbol is determined to be an object name.

[0027] Furthermore, it also includes:

[0028] If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is an object name and a type name, determine whether any lexical symbol is a type name based on the preceding and following relationship of any lexical symbol. If it is a type name, determine that the part of speech of any lexical symbol is a type name. If it is not a type name, determine the part of speech of any lexical symbol by performing a second preset processing based on any lexical symbol.

[0029] Furthermore, after searching the first lookup table and the second lookup table according to the part of speech of the lexical symbol after each part of speech determination, the method further includes:

[0030] Conflict resolution and / or error recovery processing is performed based on the search results.

[0031] Furthermore, the predetermined programming language is any one of C++ programming language and C language.

[0032] In a second aspect, a device for generating a syntax tree of a code file is provided, comprising:

[0033] The parsing module is used to parse each lexical symbol in the code file to be parsed and generate a corresponding linear linked list through the lexical parsing module when receiving the code file to be parsed in a predetermined programming language;

[0034] a first processing module configured to sequentially determine the part of speech of each lexical symbol in the linear linked list, and whenever the part of speech of any lexical symbol is determined, search a first lookup table and a second lookup table according to the part of speech of the lexical symbol to obtain a corresponding search result, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of a predetermined programming language;

[0035] The syntax tree generation module is used to generate the syntax tree of the code file to be parsed based on the search results.

[0036] Specifically, the first processing module is specifically used to perform part-of-speech judgment on any lexical symbol according to the context to determine the part-of-speech of any lexical symbol; and / or to perform part-of-speech guessing on any lexical symbol according to the context to determine the part-of-speech of any lexical symbol.

[0037] Furthermore, the first processing module is specifically used in any of the following situations:

[0038] When any lexical symbol is determined to be any of the type name, object name, and keyword, it is used as the part of speech of the lexical symbol;

[0039] When it is determined that any lexical symbol is a lexical symbol of a preset type, the lexical symbol is determined to be a template list symbol or an operation symbol, and the determination result is used as the part of speech of the lexical symbol;

[0040] Perform information lookup in the scope and use the result as the part of speech of any lexical symbol.

[0041] Furthermore, the first processing module is specifically configured to perform a lateral search in the scope, and use the search result of the lateral search as the part of speech of any lexical symbol;

[0042] A horizontal search is a search in the current scope and the referenced namespace, or a search in the base class scope of the current scope.

[0043] Furthermore, the first processing module is specifically configured to perform a vertical search in the scope when no search result is found through the horizontal search, and use the search result of the vertical search as the part of speech of any lexical symbol;

[0044] A vertical search is a search in scopes that include the current scope.

[0045] Furthermore, the first processing module includes a guessing submodule and a first determining submodule;

[0046] The guessing submodule is used to guess whether the part of speech of any lexical symbol is a namespace name based on the context;

[0047] The first determination submodule is used to, when the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after any lexical symbol is not a preset type lexical symbol, then determine the part of speech of any lexical symbol to be a type name; if the first lexical symbol after any lexical symbol is a preset type lexical symbol, then determine the part of speech of any lexical symbol by performing the first preset processing according to any lexical symbol.

[0048] Furthermore, the first processing module further includes a second determining submodule;

[0049] The second determining submodule is configured to determine that the part of speech of any lexical symbol is an object name when the guess is not a namespace name and when the part of speech of any lexical symbol is guessed to be an object name and not a type name.

[0050] Furthermore, the first processing module further includes a third determining submodule;

[0051] The third determination submodule is used to determine whether any lexical symbol is a type name based on the context of any lexical symbol when the guess is not a namespace name and when the guessed part of speech of any lexical symbol is an object name and a type name; if it is a type name, determine that the part of speech of any lexical symbol is a type name; if it is not a type name, determine the part of speech of any lexical symbol by performing a second preset processing based on any lexical symbol.

[0052] Furthermore, it also includes a second processing module;

[0053] The second processing module is used to perform conflict elimination processing and / or error recovery processing according to the search result.

[0054] Furthermore, the predetermined programming language is any one of C++ programming language and C language.

[0055] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for generating a syntax tree of a code file when executing the program.

[0056] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for generating a syntax tree of a code file is implemented.

[0057] The method for generating a syntax tree for a code file provided in an embodiment of the present application determines the part of speech of each lexical symbol in a linear linked list in turn, thereby laying the necessary foundation for subsequent searching the first lookup table and the second lookup table, and providing a prerequisite guarantee for generating a syntax tree for the code file to be parsed; whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table generated based on the analysis of the grammar of a predetermined programming language are searched according to the part of speech of any lexical symbol, and a syntax tree for the code file to be parsed is generated according to the search results, thereby providing a method for generating a syntax tree for a code file, so that the completed code file can be statically analyzed through the syntax tree, so that the syntax errors, writing errors, etc. in the completed code file can be accurately and efficiently checked and corrected, greatly saving the time and energy of program developers.

[0058] Additional aspects and advantages of the present application will be given in part in the following description, which will become apparent from the following description, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0060] Figure 1 A flowchart of a method for generating a syntax tree for a code file according to an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the basic process of part-of-speech determination in an embodiment of the present application;

[0062] Figure 3 Schematic diagram of the basic process of obtaining the part of speech of keywords and operators in the part of speech determination process of an embodiment of the present application;

[0063] Figure 4 This is a schematic diagram of the process of determining part of speech by searching for scope information in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of the basic process of part-of-speech guessing in an embodiment of the present application;

[0065] Figure 6 A schematic diagram of the basic process of conflict resolution processing according to an embodiment of the present application;

[0066] Figure 7 A schematic diagram of the basic process of error recovery processing in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of the overall process of generating a syntax tree for a code file according to an embodiment of the present application;

[0068] Figure 9 A schematic diagram of the process of moving in according to an embodiment of the present application;

[0069] Figure 10 A schematic diagram of the basic structure of an apparatus for generating a syntax tree for a code file according to an embodiment of the present application;

[0070] Figure 11 A detailed structural diagram of an apparatus for generating a syntax tree for a code file according to an embodiment of the present application;

[0071] Figure 12 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0073] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0074] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0075] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0076] Example 1

[0077] The embodiment of the present application provides a method for generating a syntax tree of a code file, such as Figure 1 Shown, including:

[0078] Step S110 , when a code file to be parsed in a predetermined programming language is received, each lexical symbol in the code file to be parsed is parsed by a lexical parsing module and a corresponding linear linked list is generated.

[0079] Specifically, a code file in a given programming language is typically composed of code elements, such as keywords, identifiers, mathematical operators, scope identifiers, and statement punctuation. These code elements are also called lexical symbols (tokens). The lexical parsing module can organize the lexical symbols in the code file to be parsed into a linear linked list (token list) in order, providing the prerequisite for the subsequent generation of the syntax tree of the code file.

[0080] In the following text, mathematical operators, scope identifiers, statement punctuation marks, and other symbols are collectively referred to as operators, and the first element in the linked list is denoted as token-list.first.

[0081] Furthermore, the basic attributes of lexical symbols (tokens) are shown in Table 1 below. The left side of Table 1 is the English name corresponding to each lexical symbol, and the right side is the Chinese description or explanation corresponding to each lexical symbol.

[0082] Table 1 - Basic properties of lexical tokens

[0083]

[0084]

[0085] In some cases, in order to simplify the grammar, some lexical symbols (tokens) can be merged during lexical parsing, such as merging lexical symbols (tokens) such as "::new", "::delete", ".template", and "->template" into one lexical symbol (token).

[0086] In step S120, the part of speech of each lexical symbol in the linear linked list is determined in turn, and whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table are searched according to the part of speech of any lexical symbol to obtain corresponding search results. The first lookup table and the second lookup table are generated based on the analysis of the grammar of the predetermined programming language.

[0087] Specifically, in practical applications, a first lookup table and a second lookup table for a predetermined programming language can be pre-generated based on the grammar of the predetermined programming language. Subsequently, when generating a code file to be parsed in the predetermined programming language, each lexical symbol in the linear linked list corresponding to the code file to be parsed can be analyzed in sequence based on the pre-generated first lookup table and the second lookup table, thereby generating a syntax tree for the code file to be parsed, so that the code file to be parsed can be statically analyzed through the syntax tree.

[0088] Furthermore, each lexical symbol in the linear linked list has its own corresponding part of speech, and in the process of analyzing each lexical symbol in the linear linked list in sequence based on the first lookup table and the second lookup table, the first lookup table and the second lookup table are actually searched based on the part of speech corresponding to each lexical symbol. Among them, the first lookup table determines the operation or action to be performed after the lexical symbol is read in based on the part of speech of the lexical symbol, and the first lookup table nestedly calls the second lookup table. Therefore, before searching based on the first lookup table and the second lookup table, it is necessary to first determine the part of speech of the lexical symbol read in, and then search the first lookup table and the second lookup table based on the part of speech of the lexical symbol to obtain the corresponding search results.

[0089] If the currently read lexical symbol is "int", its part of speech is determined to be "type name", then the first lookup table and the second lookup table can be searched according to the part of speech "type name" of the lexical symbol "int" to obtain the corresponding search results.

[0090] Step S130: Generate a syntax tree of the code file to be parsed according to the search result.

[0091] Specifically, after the first lookup table and the second lookup table are searched according to the read part-of-speech symbols to obtain corresponding search results, a syntax tree of the code file to be parsed can be generated according to the search results.

[0092] Among them, in the process of generating a syntax tree, each time a corresponding search result is obtained based on the part of speech of the lexical symbol of the code file to be parsed that is currently read, a part of the syntax tree is generated based on the search result. After the last lexical symbol of the code file to be parsed is read, the last part of the syntax tree is generated based on the search result of the last lexical symbol, thus forming a complete syntax tree. In addition, after all the lexical symbols of the code file to be parsed have obtained their respective corresponding search results, a complete syntax tree can be generated based on the search results corresponding to each lexical symbol. Of course, in actual applications, other orders for generating syntax trees can also be set as needed, and the embodiments of the present application do not limit this.

[0093] Compared with the prior art, the method for generating a syntax tree for a code file provided in an embodiment of the present application determines the part of speech of each lexical symbol in a linear linked list in turn, thereby laying the necessary foundation for subsequent searching the first lookup table and the second lookup table, and providing a prerequisite guarantee for generating a syntax tree for the code file to be parsed; whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table generated based on the analysis of the grammar of a predetermined programming language are searched according to the part of speech of any lexical symbol, and a syntax tree for the code file to be parsed is generated according to the search results, thereby providing a method for generating a syntax tree for a code file, so that the completed code file can be statically analyzed through the syntax tree, so that grammatical errors, writing errors, etc. in the completed code file can be accurately and efficiently checked and corrected, greatly saving the time and energy of program developers.

[0094] This embodiment of the present application provides another possible implementation, wherein:

[0095] The predetermined programming language is any one of the C++ programming language and the C language. The following description will be made using the C++ programming language as an example, wherein the processing process of the C language is the same as that of the C++ programming language, that is, the method provided in the embodiment of the present application is compatible with the C++ programming language and the C language.

[0096] Specifically, determining the part of speech of any lexical symbol in the linear linked list includes:

[0097] Performing part-of-speech determination on any lexical symbol according to the context to determine the part of speech of any lexical symbol; and / or,

[0098] The part of speech of any lexical symbol is guessed according to the context to determine the part of speech of any lexical symbol.

[0099] Specifically, the part of speech of any lexical symbol is determined according to the context, including any of the following situations:

[0100] When any lexical symbol is any of type name, object name and keyword, it is regarded as the part of speech of any lexical symbol;

[0101] When any lexical symbol is a lexical symbol of a preset type, determine whether the lexical symbol is a template list symbol or an operation symbol, and use the determination result as the part of speech of the lexical symbol;

[0102] Perform information lookup in the scope and use the result as the part of speech of any lexical symbol.

[0103] Specifically, information is searched in the scope, and the search results are used as the part of speech of any lexical symbol, including:

[0104] Perform a lateral search in the scope and use the result of the lateral search as the part of speech of any lexical symbol;

[0105] A horizontal search is a search in the current scope and the referenced namespace, or a search in the base class scope of the current scope.

[0106] Specifically, it also includes:

[0107] If no search result is found through the horizontal search, a vertical search is performed in the scope, and the search result of the vertical search is used as the part of speech of any lexical symbol;

[0108] A vertical search is a search in the scope that includes the current scope.

[0109] The following is a detailed description of the specific contents involved in this implementation:

[0110] In the above grammar of the embodiment of the present application, type names (such as class names, structure names, union names, enumeration class names, etc.) and object names (such as variable names, function names, object names of custom classes, etc.) are included in the grammar as terminal symbols. Therefore, when reading in an identifier, it is necessary to determine whether the identifier is a type name (TYPE-NAME) or an object name (IDENTIFIER). The information of the current lexical symbol can be found from the previously read lexical symbols and the constructed syntax tree, thereby determining whether the current lexical symbol is a type name or an object name.

[0111] For example, the following code:

[0112]

[0113]

[0114] Keywords introduced after C++98, such as final and override, can also be used as object names. Whether they are keywords depends on the context during grammatical analysis. In other words, when a lexical symbol is a type name, an object name, or a keyword, it is used as the part of speech of the lexical symbol. That is, if a lexical symbol is a type name, the type name is used as the part of speech of the lexical symbol; if a lexical symbol is an object name, the object name is used as the part of speech of the lexical symbol; and if a lexical symbol is a keyword, the keyword is used as the part of speech of the lexical symbol.

[0115] Another example is the following code:

[0116]

[0117] For lexical symbols of preset types such as "<", ">", it is necessary to determine whether they are the start or end of a template list, or a greater-than or less-than sign. For the lexical symbol of preset type ">>", it is necessary to determine whether it is a shift operator or two template list terminators written together. If they are written together, the ">>" needs to be replaced with two ">" in the linear linked list (token-list). In other words, when any lexical symbol is a lexical symbol of a preset type, it is determined whether it is a template list symbol or an operator symbol, and the determination result is used as the part of speech of the lexical symbol.

[0118] Among them, all the special terminals that need to determine the part of speech in the production list file are shown in Table 2 below. In Table 2, the left side is the English name of each special terminal, and the right side is the Chinese description or explanation corresponding to each special terminal.

[0119] Table 2 - All special terminals that require part of speech determination

[0120]

[0121] In addition, since the syntax tree generated by the embodiment of the present application is used for static analysis of the code file, some incomplete code is also acceptable. Among them, when the part of speech of some lexical symbols cannot be determined, the part of speech of the symbol is guessed using the "part of speech guessing mechanism".

[0122] In other words, the overall process of determining the part of speech of any lexical symbol in a linear linked list can include the following three sub-processes: (1) determining the part of speech of keywords and operators; (2) determining the part of speech by searching scope information; and (3) determining the part of speech by guessing the part of speech. The following first describes (1) and (2) with specific examples, and then describes (3) with another implementation example.

[0123] For (1) and (2) above, for example, the following code:

[0124] struct A

[0125] {};

[0126] int A = 0;

[0127] int fun()

[0128] {

[0129] struct A a; / / #1

[0130] return A; / / #2

[0131] }

[0132] In this example, when analyzing "A" at #1, a scope information search is performed and it is found that "A" is both a class name and a variable name. However, the keyword "struct" is used to limit the scope at #1, so only class names are accepted. Therefore, the part of speech of "A" at #1 is TYPE-NAME (type name). When analyzing "A" at #2, it is found that both class names and variable names are currently accepted, and variable names are given priority. Therefore, the part of speech of "A" at #2 is IDENTIFIER (variable name). Among them, when the part of speech information cannot be obtained through "Keyword and Operator Part of Speech Determination" and "Scope Information Search", the "Part of Speech Guessing" mechanism is activated, that is, the part of speech is determined by part of speech guessing. Figure 2 The following is a diagram of the process of determining the part of speech according to the above code example.

[0133] In addition, member functions defined in a class declaration require special handling because the implementation of the member function can precede the declaration of the class member, such as the following code:

[0134]

[0135] Among them, the declaration of the member variable m is later than the definition of the member function fun, which is legal. However, if the "m" in the function body of "fun" is parsed directly, the part-of-speech information cannot be obtained. Therefore, when reading the "{" after "fun()", the function body is transferred outside the class declaration and replaced with ";". The function declaration information after the transfer is stored in the terminator "TRIMED-DECL-INFO". Figure 3 A schematic diagram of the process of obtaining the parts of speech of keywords and operators is given.

[0136] In addition, scope information search is divided into "transverse search" and "vertical search". "Transverse search" means searching in the current scope and the referenced namespaces or the base scope of the current scope. "Vertical search" means searching in the scope that contains the current scope when "transverse search" finds no result. Figure 4 A schematic diagram of the process of searching for information and determining part of speech in a scope is given.

[0137] This embodiment of the present application provides another possible implementation, wherein:

[0138] Guess the part of speech of any lexical symbol based on the context and determine the part of speech of any lexical symbol, including:

[0139] Guess whether the part of speech of any lexical symbol is a namespace name based on the context;

[0140] If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after any lexical symbol is not a lexical symbol of the preset type, then the part of speech of any lexical symbol is determined to be a type name; if the first lexical symbol after any lexical symbol is a lexical symbol of the preset type, then the part of speech of any lexical symbol is determined by performing the first preset processing according to any lexical symbol.

[0141] Specifically, the method further includes:

[0142] If the guess is not a namespace name, and when the part of speech of any lexical symbol is guessed to be an object name and not a type name, the part of speech of any lexical symbol is determined to be an object name.

[0143] Specifically, the method further includes:

[0144] If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is an object name and a type name, determine whether any lexical symbol is a type name based on the preceding and following relationship of any lexical symbol. If it is a type name, determine that the part of speech of any lexical symbol is a type name. If it is not a type name, determine the part of speech of any lexical symbol by performing a second preset processing based on any lexical symbol.

[0145] The following is a detailed description of the specific contents involved in this implementation:

[0146] In the process of guessing the part of speech of any lexical symbol based on the context and determining the part of speech of any lexical symbol, the following key sub-processes are included: (1) like-simple-type-name: judge whether the part of speech can be TYPE-NAME (type name) through the relationship between the symbols, and do not judge the case where the lexical symbol (token) is followed by "<". (2) match-parametric-type-name: when the lexical symbol (token) is followed by "<", judge whether "<" is the starting symbol of the template parameter list. (3) find-template-list-end: if "<" is the starting symbol of the template parameter list, return the corresponding ">". (4) handle-sticky-template-list-end: handle the template parameter list terminator ">>" written together. Among them, Figure 5 A schematic diagram of the process of determining the part of speech of a lexical symbol by part of speech guessing is given. Figure 5The accept(x) function in the function represents the part of speech (e.g., type name, object name, etc.). The accept(x) function indicates whether the model can currently accept the lexical symbol token with x as the part of speech. The accept(x) function is equivalent to Action(state-stack.top(),x)! = (NULL,NULL), indicating that the entry found in the first lookup table Action table is not empty, x represents the part of speech, and state-stack.top() represents the top of the state stack. In addition, Figure 5 In the acc-tp and acc-id are characters representing variable names. The first preset processing is the above-mentioned find-template-list-end processing and handle-sticky-template-list-end processing, and the second preset processing is the above-mentioned match-parametric-type-name processing and handle-sticky-template-list-end processing, which are not repeated here.

[0147] This embodiment of the present application provides another possible implementation, wherein:

[0148] Before searching the first lookup table and the second lookup table according to the part of speech of any lexical symbol to obtain a corresponding search result, the grammar of the predetermined programming language may be analyzed in advance to generate the first lookup table and the second lookup table. The method of analyzing the grammar of the predetermined programming language to generate the first lookup table and the second lookup table includes:

[0149] Determine the categories and orders of various lexical symbols in a predetermined programming language;

[0150] Grammar symbols are divided into terminal symbols and non-terminal symbols, where the grammar symbols include various lexical symbols and their corresponding categories, where the lexical symbols belong to terminal symbols and the categories belong to non-terminal symbols, and the non-terminal symbols represent the hierarchical structure of a predetermined programming language;

[0151] According to the order of the corresponding lexical symbols, determine the non-terminal symbols to which the lexical symbols belong respectively;

[0152] A first lookup table is generated according to the pre-generated state and terminal symbols of the predetermined programming language, and a second lookup table is generated according to the pre-generated state and non-terminal symbols of the predetermined programming language.

[0153] The grammar includes a start symbol and a production list; the start symbol is a predefined non-terminal symbol, and any production in the production list represents the relationship between a terminal symbol and a non-terminal symbol.

[0154] Any of the above productions includes a left part and a right part, the left part is the non-terminal symbol to which the right part belongs, and the right part is a sequence of terminal symbols and / or a sequence of non-terminal symbols.

[0155] The non-terminal symbol on the left side corresponds to a node in the syntax tree, and any terminal symbol or any non-terminal symbol on the right side is a child node of the node.

[0156] Specifically, a grammar is a precise description of the structure of a given programming language, specifying the categories and order of various code elements (i.e., lexical tokens). For example, a grammar specifies that a C++ programming language code file is composed of categories such as class declarations, function definitions, and namespace declarations. Namespace declarations can, in turn, be composed of categories such as class declarations, function definitions, and namespace declarations. These categories and code elements (i.e., lexical tokens) are collectively referred to as "grammar symbols."

[0157] Furthermore, grammar symbols can be divided into "non-terminals" and "terminals". "Non-terminals" give the hierarchical structure of the programming language. This hierarchical structure is the key to grammatical parsing and is also the goal of the embodiments of this application. Intuitively speaking, categories correspond to "non-terminals", and the individual lexical symbols (tokens) that directly constitute the code are "terminals". Lexical symbols (tokens) can be directly converted into grammar symbols (symbols) as terminals, and non-terminals will correspond to multiple lexical symbols (tokens) or non-terminals after classification.

[0158] For example, consider the code "struct A{int a;};". This code consists of lexical tokens such as "struct", "A", "{", "int", "a", ";", "}", and ";". These lexical tokens are terminals, and the grammar specifies that terminals in this order can be classified as non-terminal "class-specifiers" (i.e., class declarations). "Classification" is also known as "reduction" in the field of compiler science. In the following text, "classification" and "reduction" have the same meaning.

[0159] Among them, the common attributes of grammar symbols are shown in Table 3 below. The left side of Table 3 is the English name of each attribute, and the right side is the Chinese description or explanation corresponding to each English name.

[0160] Table 3 - Common properties of grammar symbols

[0161]

[0162] In addition, specific symbols have specific attributes that record the information needed by the model at runtime.

[0163] Furthermore, a grammar consists of a start symbol and a list of productions, as shown below:

[0164] 1. A start symbol is a non-terminal symbol that corresponds to the top-level description of the entire programming language. In the embodiment of the present application, the start symbol is "translation-unit," where the start symbol corresponds to the root of the syntax tree.

[0165] 2. Production-list: A production-list describes the relationship between terminals and non-terminals. A production consists of a "left-hand side" (a non-terminal) and a "right-hand side" (a sequence of terminals or non-terminals). Any production in a production-list can be formally expressed as "left-hand side → right-hand side."

[0166] For example: selection-statement → if(condition)statement. The left-hand side of the sentence "selection-statement" is a non-terminal symbol, and the right-hand side consists of the sequence of the keyword "if," the terminal symbol "(," the non-terminal symbol "condition," the terminal symbol ")," and the non-terminal symbol "statement." This means that the sequence "if(condition)statement" on the right-hand side can be classified as the non-terminal symbol "selection-statement" on the left-hand side.

[0167] In the syntax tree, the nonterminal symbol on the left side of a production corresponds to a node, and the terminal and nonterminal symbols on the right side are its child nodes. The basic properties of a production are shown in Table 4 below. The left side of Table 4 lists the English names of the properties, and the right side lists the corresponding Chinese descriptions or explanations of each English name.

[0168] Table 4- Basic properties of production

[0169] left left part right Right length() The number of symbols on the right

[0170] Furthermore, the first lookup table generated based on the analysis of the grammar of the predetermined programming language can be an Action table, and the second lookup table can be a Goto table. The Action table is a two-dimensional mapping table of "state-terminal part of speech", that is, one state and one terminal part of speech correspond to one table entry. The Action table entry can be formally represented as Action(state, cat), where state is the state and cat is the part of speech. The value of Action(state, cat) can be represented as a tuple (m, x), where m can take five values, as shown below:

[0171] 1. Shift (S) means that a new state can be moved into the state stack and the currently read lexical token can be converted into a grammar symbol and moved into the symbol stack. When m is S (shift), the value of x represents the new state.

[0172] 2. Reduction (R) indicates that a continuous symbol sequence of a certain length at the top of the symbol stack can be reduced according to a certain production. This continuous symbol sequence is the right part of the production. Specifically, the reduction can be: pop this symbol sequence from the symbol stack, and at the same time pop a state sequence of the same length from the state stack, then push the left part of the corresponding production onto the symbol stack, then search the Goto table for a new state based on the current state at the top of the symbol stack and the left part just pushed, and push the new state found onto the state stack. When m is R (reduction), x represents the sequence number of the production to be reduced in the production list. This production can be expressed as production-list(x).

[0173] 3. Reduce-Shift Conflict (CSR) indicates that either a shift or a reduction is possible. When m is CSR, x is a two-tuple whose first component is the state to be shifted and whose second component is the number of the production to be reduced.

[0174] 4. Reduce-Reduce Conflict (CRR), which means that the current reduction can be performed according to different production rules. When m is CRR, x is a two-tuple, whose two components represent the sequence number of the production rule to be reduced.

[0175] 5. NULL: The expression is wrong, that is, the model does not accept the current lexical symbol (token). When m is empty, x has no meaning and is also NULL by convention.

[0176] The Action table is derived from the "production list file" in the grammar using the LR1 algorithm. For ease of discussion, this embodiment uses Action(state, cat) to represent an Action table entry. When the cat parameter is absent, Action(state) represents the set of all parts of speech corresponding to state in the Action table.

[0177] Furthermore, the Goto table is a two-dimensional mapping table of "state-nonterminal symbol name," meaning one table entry corresponds to one state and one nonterminal symbol, with each entry representing a state. An entry in the Goto table can be expressed as Goto(state, symbol-name), where state is the state and symbol-name is the nonterminal symbol name. The Goto table is used when reducing productions, indicating the state the parser should be in after the reduction. The nonterminal symbols corresponding to the top state of the current state stack in the Goto table represent various possible reductions to be performed next.

[0178] The Goto table is derived from the "production list file" in the grammar using the LR1 algorithm. For ease of discussion, this embodiment uses Goto(state, symbol-name) to represent the Goto table entry. When the parameter symbol-name is not present, Goto(state) represents all non-terminal symbols corresponding to state in the Goto table.

[0179] Furthermore, the symbol stack and state stack use stack structures to store the symbolic and state relationships during model runtime. The basic properties of the symbol stack and state stack are shown in Table 5 below. In Table 5, the left side shows the English name of each property, and the right side shows the Chinese description or explanation corresponding to each English name.

[0180] Table 5- Basic properties of production

[0181] top() Top element of the stack pop() Pop the stack, the number is specified by parameter n push() Push the element specified by parameter x onto the stack size() Number of elements in the stack

[0182] Furthermore, each lexical symbol in the linear linked list is analyzed in turn, including:

[0183] When the current lexical symbol in the linear linked list is a lexical symbol indicating the start of a scope, a corresponding top scope is created at the top of the scope stack, and the lexical symbol following the lexical symbol indicating the start of the scope is written into the top scope; and, when the current lexical symbol in the linear linked list is a lexical symbol indicating the end of a scope, the top scope is popped.

[0184] Among them, the bottom scope of the scope stack is the global scope, and the scopes other than the bottom scope are local scopes.

[0185] Specifically, if the lexical symbol currently read from the linear linked list is a lexical symbol indicating the start of a scope (for example, "{"), a corresponding top scope is created at the top of the scope stack, and the lexical symbols (for example, variables, keywords, etc.) following the lexical symbol indicating the start of the scope are written into the created top scope. If the lexical symbol indicating the start of the scope (for example, "{") is encountered for the first time in the process of parsing the lexical symbols in the linear linked list, then the bottom of the scope stack is the top of the scope, and the bottom scope is the global scope. If the lexical symbol indicating the start of the scope (for example, "{") is not encountered for the first time in the process of parsing the lexical symbols in the linear linked list, then a new scope is created at the top of the scope stack, namely the top scope, which is a local scope, and then the lexical symbols following the lexical symbol indicating the start of the scope are written into the created top scope.

[0186] Furthermore, if the lexical symbol currently read from the linear linked list is a lexical symbol indicating the end of the scope (for example, "}"), it indicates the end of the scope. At this time, the top scope of the stack is popped, that is, the lexical symbol written into the top scope of the stack is read out, and the created top scope of the stack is deleted.

[0187] Furthermore, in the C++ programming language, namespaces, types, and objects are semantically organized by "scope," and scopes can be nested. Namespaces, types, function bodies, and the like each have their own scope, where identifiers in different scopes can have the same name, and type and object names in the same scope can also have the same name. The embodiments of this application use a stack to analyze each scope, where the scope stack is an important reference for "dynamic part-of-speech determination."

[0188] Furthermore, scopes can be divided into namespace scope, class scope, function body scope, etc. Different scopes have their specific properties. The common properties of scope are shown in Table 6 below. In Table 6, the left side is the English name of each property, and the right side is the Chinese description or explanation corresponding to each English name.

[0189] Table 6 - Common properties of scopes

[0190] parent() The scope that directly contains the current scope types The set of type scopes defined in the current scope declarators A collection of objects defined in the current scope usingNameSpaces The namespace scope set currently referenced by the scope bases The base scope collection of the current scope

[0191] This embodiment of the present application provides another possible implementation, wherein:

[0192] After searching the first lookup table and the second lookup table according to the part of speech of the lexical symbol after each part of speech determination, the method further includes: performing conflict elimination processing and / or error recovery processing according to the search results.

[0193] Among them, in the process of analyzing each lexical symbol in the linear linked list in turn based on the first lookup table and the second lookup table, the first lookup table and the second lookup table are actually searched according to the part of speech corresponding to each lexical symbol. Therefore, it is necessary to first analyze the part of speech of the lexical symbol currently read from the linear linked list to determine the part of speech of the lexical symbol read in, and then search the first lookup table and the second lookup table according to the part of speech to obtain the corresponding search results, and then analyze any lexical symbol according to the search results.

[0194] If the currently read lexical symbol is "int", its part of speech is determined to be "type name", then the first lookup table and the second lookup table can be searched according to the part of speech "type name" of the lexical symbol "int" to obtain the corresponding search results.

[0195] When a conflict (i.e., multiple possible processing behaviors are found simultaneously and it is impossible to determine which behavior should be selected) and / or an error (i.e., no corresponding processing behavior is found) occurs during the search of the first lookup table and the second lookup table based on the part of speech of the lexical symbol, conflict resolution processing is performed based on the search results. Specifically, when the first table entry corresponding to the top state of the current state stack of any lexical symbol in the first lookup table is empty, error recovery processing is performed on any lexical symbol; based on the processing results of the error recovery processing, a syntax tree of the code file to be parsed is generated. Then, a syntax tree is generated based on the results of the conflict resolution processing and / or error recovery processing.

[0196] The following is a detailed description of conflict resolution processing:

[0197] Specifically, based on the first lookup table and the second lookup table, each lexical symbol in the linear linked list is analyzed in sequence. When any lexical symbol is determined to belong to a predetermined conflict type, corresponding conflict resolution processing is performed on the lexical symbol; based on the results of the conflict resolution processing, a syntax tree of the code file to be parsed is generated. Specifically, when the first table entry corresponding to the top state of the current state stack of any lexical symbol in the first lookup table is empty, error recovery processing is performed on the lexical symbol; based on the results of the error recovery processing, a syntax tree of the code file to be parsed is generated.

[0198] Specifically, the predetermined conflict type includes any of the following:

[0199] The processing of any lexical symbol is a conflict between shift processing and reduce processing;

[0200] The processing of any lexical symbol is a conflict between the first reduction process and the second reduction process.

[0201] Specifically, before performing conflict resolution processing on any lexical symbol, the following steps are also included:

[0202] The current processing state is saved to obtain a first saved result.

[0203] Specifically, conflict resolution is performed on any lexical symbol, including:

[0204] Performing a first target processing on any lexical symbol according to the context, and performing corresponding processing on lexical symbols following any lexical symbol in sequence based on the first target processing;

[0205] If no processing error occurs until the processing of the retry terminator is completed, the first saved result is deleted, and the subsequent lexical symbols are processed accordingly.

[0206] Specifically, it also includes:

[0207] If a processing error occurs in the process of performing corresponding processing on the lexical symbols after any lexical symbol, the recovery processing is performed according to the first saved result, and the second target processing is performed on any lexical symbol, and the corresponding processing is performed on the lexical symbols after any lexical symbol.

[0208] Specifically, the conflict resolution process for any lexical symbol includes any of the following:

[0209] When any lexical symbol belongs to the first predetermined type, if a conflict occurs between reduction processing and shift processing on any lexical symbol, the shift processing is performed on the any lexical symbol;

[0210] When any lexical symbol belongs to the second predetermined type, if a conflict occurs between shift processing and reduction processing for any lexical symbol, determining whether to perform shift processing or reduction processing on the lexical symbol based on the linear relationship between the lexical symbols;

[0211] When any lexical symbol belongs to the third predetermined type, if a conflict occurs between the first reduction process and the second reduction process for any lexical symbol, it is determined whether the first reduction process or the second reduction process is to be performed on the any lexical symbol based on the linear relationship between the lexical symbols.

[0212] The following is a detailed introduction to the above-mentioned conflict resolution process:

[0213] Because of the existence of conflicts, the production list used in the embodiment of the present application does not belong to the classic LR1 grammar, and it is difficult to construct a standard LR1 grammar model of the C++ programming language. Therefore, the embodiment of the present application implements a conflict resolution processing module.

[0214] For example, in the production list file, the function parameter declaration is part of the non-terminal declarator, and the declarator can be followed by an initialization list, and the initialization list can also start with "(", for example:

[0215] int a(0); / / Declare variable a, its initial value is 0, "(0)" is the initialization list

[0216] int a(int); / / Declare function a, which has one parameter of type int, "(int)" is the parameter list

[0217] This creates a conflict. After reading the two lexical symbols int and a, the top of the symbol stack is noptr-declarator. However, when reading "(", it is impossible to determine whether the "(" is the beginning of the parameter list or the beginning of the initialization list. In other words, it is impossible to determine whether the top symbol of the stack should be reduced to ptr-declarator or continue to move into the matching parameter list. Therefore, it is necessary to judge whether the content after "(" is a parameter list or an initialization list based on the context. In some cases, a deterministic conclusion can be drawn based on the linear relationship between symbols, while in other cases, a deterministic conclusion cannot be drawn. When a deterministic conclusion cannot be drawn, it can be parsed as a parameter list first. If it passes (that is, no parsing error occurs), it is determined to be a parameter list. If a parsing error occurs (that is, it does not pass), it is parsed as an initialization list, which provides a "retry mechanism".

[0218] First: The following productions, when faced with a "reduce-shift conflict," enforce a shift. That is, if any token belongs to the first predetermined type and a conflict arises between reduction and shift processing, the token is shifted. The situation where any token belongs to the first predetermined type is as follows:

[0219] (1)selection-statement→if(condition)statement

[0220] When reading an "else", it is mandatory not to reduce but to shift, that is, each "else" is paired with the nearest "if".

[0221] (2)class-specifier→class-head

[0222] When defining a member class of a class, read ":" to force the specification not to be reduced but to be moved in.

[0223] The reason for the conflict is that ":" can be the beginning of the base class list or the beginning of the bit field. Semantically, the class-head should not be restricted by the bit field, so it can be forced to move in directly.

[0224] (3)exception-specification→noexcept

[0225] When reading "(", "(" can be part of a noexcept declaration or part of an initializer list, which is forced to be a shift.

[0226] For example: int(*pf)(int)noexcept(0);

[0227] Among them, "(0)" can be combined with noexcept to specify that pf points to a function that can throw any exception, but it can also be used as the initial value of pf. This is a grammatical ambiguity and is forced to be combined with noexcept.

[0228] (4)enum-base→type-name and enum-base→type-modifier-seqtype-name

[0229] When reading "const" and "volatile", shifting is mandatory. The reason for the conflict is that enumeration variables can be defined like this:

[0230] enum E: int const e = xxx;

[0231] Among them, const is ambiguous and has two interpretations:

[0232] 1. enum E:int specification is enum-specifier, const specification is type-modifier-seq

[0233] 2. int const is reduced to enum-base, and enum E:int const is reduced to enum-specifier.

[0234] Based on the standard, the second method is enforced.

[0235] (5)new-type-id→trailing-type-specifier and new-declarator→new-ptr-operator

[0236] When reading "*", it is mandatory to shift in. The reason for the conflict is that new-expression can be used as a basic expression, and "*" can be interpreted as a pointer or a multiplication sign.

[0237] According to the C++ standard, it does not make sense to use new-expression for multiplication, so it is mandatory to use shift.

[0238] (6)conversion-type-id→trailing-type-specifier and conversion-declarator→ptr-operator

[0239] When reading "*", "&", and "&&", a shift is forced. The reason for the conflict is that conversion-type-id can be part of a basic expression, so "*", "&", and "&&" can be either ptr-operator or operator. These symbols are forced to be ptr-operator.

[0240] Second: When the following productions face a "reduce-shift conflict", they can determine whether to perform "reduce" or "shift" based on the symbol stack and the linear relationship between the symbols. If they cannot determine, they can start the retry mode. That is, when any lexical symbol belongs to the second predetermined type, if any lexical symbol conflicts between shift processing and reduction processing, then based on the linear relationship between the lexical symbols, it is determined whether to perform shift processing or reduction processing on any lexical symbol. Among them, the situation where any lexical symbol belongs to the second predetermined type is as follows:

[0241] (1)ptr-declarator→noptr-declarator

[0242] When reading "(", it is necessary to determine whether "(" is the beginning of the parameter list or the beginning of the initialization list. If it is the beginning of the parameter list, it is shifted in. If it is the beginning of the initialization list, the noptr-declarator is reduced to the ptr-declarator. If it cannot be determined, the retry mechanism is enabled.

[0243] (2)type-name→nested-class-name

[0244] When reading "(", it is necessary to determine whether "(" is the beginning of the parameter list or the beginning of the declarator. If it is the beginning of the parameter list, it is shifted in. If it is the beginning of the declarator, the nested-class-name is reduced to the type-name. If it cannot be determined, the retry mechanism is enabled.

[0245] For example (let A be a class name):

[0246] A(a); / / Object definition, "(a)" is the declarator

[0247] A::A(int); / / constructor, "(int)" is the parameter list

[0248] (3)simple-type-specifier→type-name

[0249] When reading "(", it is necessary to determine whether the "(" is followed by an expression or an object definition. If it is an expression, it is shifted in. If it is an object definition, the type-name is reduced to a simple-type-specifier. If it cannot be determined, it enters retry mode.

[0250] For example:

[0251]

[0252] (4)enum-specifier→enum-head

[0253] When defining a member enumeration of a class, the production encounters a reduce-shift conflict when reading ":". The production must determine whether the ":" is followed by a bit-field definition or an enumeration base class definition. If the ":" is not followed by a type name, the enum-head is reduced to the enum-specifier; otherwise, it is shifted.

[0254] (5)ptr-abstract-declarator→ptr-operator

[0255] When defining a trailing-return-type, the production reads "(" and faces a reduction-shift conflict. The production must determine whether the "(" is part of the trailing-type-id or the start of an initializer list. If so, the ptr-operator is reduced to a ptr-abstract-declarator. Since mainstream compilers do not support this, the production can be directly shifted.

[0256] (6)ptr-abstract-declarator→noptr-abstract-declarator

[0257] When defining a trailing-return-type, the production reads "(" and faces a reduction-shift conflict. The production must determine whether the "(" is part of the trailing-type-id or the start of an initializer list. If so, the noptr-abstract-declarator is reduced to a ptr-abstract-declarator. Since mainstream compilers do not support this, the production can be directly shifted.

[0258] (7)trailing-type-id→trailing-type-specifier

[0259] When defining a trailing-return-type, the production reads "(" and faces a reduction-shift conflict. The production must determine whether the "(" is part of a trailing-type-id or the start of an initializer list. If so, the trailing-type-specifier is reduced to the trailing-type-id. Since mainstream compilers do not support this, the production can be directly shifted.

[0260] (8) unary-operator → ~

[0261] When reading TYPE-NAME or decltype, it is necessary to determine whether the "~" is a negation operator or a destructor identifier. If it is a negation operator, the top "~" on the stack is reduced to a unary-operator. If it is a destructor identifier, it is shifted in.

[0262] For example:

[0263]

[0264]

[0265] Third: Reduction-reduction conflict. When the following productions face a "reduction-reduction conflict", the symbol stack and the linear relationship between the symbols can be used to determine which production to reduce to. That is, when any lexical symbol belongs to the third predetermined type, if any lexical symbol conflicts between the first reduction process and the second reduction process, the linear relationship between the lexical symbols determines whether the first reduction process or the second reduction process should be performed on the lexical symbol. The situation where any lexical symbol belongs to the third predetermined type is as follows:

[0266] (1)braced-init-list→{} and compound-statement→{}

[0267] When reading ";", it is necessary to determine whether the top of the symbol stack is a function. If it is a function, it is reduced to compound-statement→{}, otherwise it is reduced to braced-init-list→{}.

[0268] (2)initializer-clause→assignment-expression and expression→assignment-expression

[0269] When reading "," it is necessary to determine whether the top of the symbol stack is a function. If it is a function, follow the expression→assignment-expression convention; otherwise, follow the initializer-clause→assignment-expression convention.

[0270] In addition, when it's unclear whether to "move in" or "reduce," you can save the current state of the model as a "snapshot," then experimentally execute one action. If an error occurs, restore the model to the previously saved snapshot and then execute the other action—this is known as the "retry mechanism." Each snapshot has a "retry end." If no errors occur between saving the snapshot and reading the retry end, the snapshot can be canceled and execution can continue.

[0271] For example, in the following code: int fun(int(*)(int(x)));

[0272] If the first "(" cannot be determined to be a parameter list after reading it, you need to save a model snapshot. The retry terminator corresponding to this snapshot is the last ")". Then try to execute it according to the parameter list. If no error occurs from saving the snapshot to reading the retry terminator, it is considered that the brackets are a parameter list and the previously saved snapshot can be canceled. Figure 6 A schematic diagram of the conflict resolution process is given. Figure 6 The judge-CSR in it indicates the action to be executed according to the current conflict, save-snapshot indicates that the symbol stack, state stack, scope stack, current input symbol, etc. are stored in the snapshot, snapshot-stack indicates that the snapshots are organized with stacks, that is, retry can also occur during the retry process, roll-back indicates that the model is restored to the previous state, set-trial-end(x) indicates that the symbol x is set as the retry end symbol, cancel-trial-end(x) indicates that the symbol x is set as the non-retry end symbol, roll-back-flag is used to indicate whether roll-back has been executed before. If roll-back-flag is true, it means that roll-back is executed. If roll-back-flag is false, it means that roll-back is not executed. In addition, Figure 6 The token in the table is a lexical symbol, m and x are the table entries obtained by searching the first lookup table (Action table), R represents reduction, S represents shift, x=x(0) represents the first reduction process or the second reduction process, and x=x(1) represents the second reduction process or the first reduction process.

[0273] The following is a detailed description of error recovery processing:

[0274] Specifically, before performing error recovery processing on any lexical symbol, the method further includes:

[0275] Generate a prompt message to remind the user that an error has occurred in any lexical symbol, and display the prompt message.

[0276] Specifically, error recovery processing is performed on any lexical symbol, including:

[0277] determining at least one set of predetermined combination of target lexical symbols, target states, and target non-terminal symbols that satisfy predetermined recovery conditions;

[0278] Based on any set of predetermined combinations of target lexical symbols, target states, and target non-terminal symbols, error recovery processing is performed on any lexical symbol.

[0279] Specifically, the predetermined recovery condition is that the third table entry corresponding to the target lexical symbol and the second table entry in the first lookup table is non-empty;

[0280] The second table entry is a table entry corresponding to the target state and the target non-terminal symbol in the second lookup table.

[0281] Specifically, determining at least one set of predetermined combination forms of target lexical symbols, target states, and target non-terminal symbols that satisfy predetermined recovery conditions includes:

[0282] The target state is the state that is skipped from the top state of the current state stack to a predetermined number of states.

[0283] Starting from any lexical symbol and skipping a predetermined number of lexical symbols as the target lexical symbol;

[0284] Based on the correspondence between states and non-terminal symbols in the second lookup table, the non-terminal symbol corresponding to the target state is determined as the target non-terminal symbol.

[0285] Specifically, based on any set of predetermined combination of target lexical symbols, target states, and target non-terminal symbols, error recovery processing is performed on any lexical symbol, including:

[0286] In the state stack, pop the state above the target state;

[0287] In the symbol stack, the lexical symbols located above any lexical symbol are popped, and the number of lexical symbols popped is the same as the number of states popped from the state stack;

[0288] Error recovery is performed on any lexical symbol by pushing the target non-terminal symbol onto the symbol stack and pushing the second table entry onto the state stack.

[0289] Specifically, it also includes:

[0290] When there are multiple groups of target lexical symbols, target states and target non-terminal symbols in predetermined combinations that meet predetermined recovery conditions, error recovery processing is performed on any lexical symbol based on the first group of target lexical symbols, target states and target non-terminal symbols in predetermined combinations that meet the predetermined recovery conditions; or, error recovery processing is performed on any lexical symbol based on the target lexical symbols, target states and target non-terminal symbols in predetermined combinations with the highest priority among the multiple groups of predetermined combinations.

[0291] The following is a detailed introduction to the above error recovery process:

[0292] When the entry corresponding to the read lexical token and the top state of the current state stack in the first lookup table (i.e., Action table) is empty, that is, Action(state-stack.top(), token.cat) is (NULL, NULL), it means that the lexical token is not accepted by the current model and there is a syntax error. Such a lexical token is called an error-token.

[0293] Here, we can start from the top of the current state stack and search downward to find a state s. We can then search backward from the current input symbol sequence to find a lexical symbol t, and then find a non-terminal symbol A in the second lookup table Goto(s). These targets (s, A, t) must meet the following condition: Action(Goto(s, A), t)! = NULL, NULL (that is, the entry found in the first lookup table is not empty). This condition can be called the "recovery condition." After finding (s, A, t) that meets the recovery condition, all states above s are popped from the stack, and the same number of symbols are popped from the symbol stack. Then, A is pushed onto the symbol stack, and Goto(s, A) is pushed onto the state stack. This allows us to "skip" the error and continue execution. This process of skipping errors is called "error recovery."

[0294] Among them, there may be multiple results for the search of (s, A, t). When the first result is found, error recovery can be performed based on the result. This mode can be called "panic mode". Alternatively, multiple results can be compared and the best result (for example, the one with the highest priority) can be selected. Error recovery can be performed based on the best result. This mode can be called "preferred mode".

[0295] It should be noted that the embodiments of this application categorize errors into "code structural errors" and "general syntax errors." "Code structural errors" are the most serious errors, such as unmatched brackets. These errors are not tolerated and will be reported and terminated. General syntax errors, however, are tolerated to a certain extent.

[0296] According to the above description of error recovery, when encountering the error symbol "error-token", a set (s, A, t) can be found for error recovery (s is the state, A is the non-terminal symbol, and t is the error-token or a symbol after it). The embodiment of the present application first finds all (s, A, t) that can be error recovered, and selects the best result according to the "optimal mode" (this best result is recorded as the result with the highest priority). If the best result cannot be found, the "panic mode" is entered. The selection of the best result (i.e., the result with the highest priority) is based on the following optimization principles:

[0297] 1. When searching for t, try not to jump out of the current scope, nor search in the scope nested within the current scope. This will minimize the disruption of the scope and allow the code following the error code to be processed correctly.

[0298] 2. s should be as close as possible to the top of the status stack, and t should be as close as possible to the error-token, so as to discard as little code information as possible.

[0299] 3. A should be a non-terminal that represents the main program structure, that is, a non-terminal that is close to the root node in the syntax tree, such as expression, statement (expression-statement, declaration-statement), etc. If A is a non-terminal close to a leaf node, it is more likely to introduce new errors.

[0300] in, Figure 7 A schematic diagram of the error recovery process is given. Figure 7Here, A represents the target non-terminal symbol, t represents the target lexical symbol, and get-tolerant-end-token indicates determining the search bounds for t. Some symbols in the stack have scope boundary symbols as attributes, so the scope boundary can be used as the search bounds for t. In addition, when searching from the top of the symbol stack to the bottom, if an unpaired left bracket ("{", "(", "[")) is found, then the search bounds for t are the paired right brackets. get-tolerant-symbol-and-token indicates the process of selecting (s, A, t). stack-balance indicates that in the process of searching for s, if the right bracket needs to be popped, then the paired left bracket and the symbol between the two brackets should also be popped, otherwise the scope will be disrupted.

[0301] SYMBOL-WEIGHT-MAP is a "name-value" binary mapping table that records the names of all selectable nonterminal symbols and their corresponding weights. The weight reflects the high or low level of the nonterminal's grammatical hierarchy, that is, its distance from the root node. Its value is an empirical value, and the larger the weight, the closer it is to the leaf node of the syntax tree. For example, "translation-unit" is the start symbol, that is, the root of the syntax tree, and its weight is 1. "Translation-unit" consists of "declaration-seq", so "declaration-seq" has a weight of 2. Examples of important nonterminal symbols in the "name-value" binary mapping table SYMBOL-WEIGHT-MAP are shown in Table 7 below. In Table 7, the left side shows the English names of important nonterminal symbols, and the right side shows the corresponding Chinese descriptions and weights of important nonterminal symbols. The weights are in parentheses.

[0302] Table 7 - Chinese descriptions and weights of important non-terminal symbols

[0303]

[0304]

[0305] Panic mode is similar to Optimal mode in general, but it's not restricted by the SYMBOL-WEIGHT-MAP table and doesn't compare results. Instead, it performs error recovery as soon as it finds a (s, A, t) that meets the "recovery criteria." While panic mode guarantees error recovery, it can disrupt scope and discard significant amounts of code information.

[0306] This embodiment of the present application provides another possible implementation, wherein:

[0307] Figure 8A schematic diagram of the process of analyzing each lexical symbol in the linear linked list based on the first lookup table and the second lookup table to generate a syntax tree for the code file to be parsed is provided in an embodiment of the present application. The process includes various sub-processes such as part of speech determination, searching the first lookup table and the second lookup, conflict resolution processing, and error recovery processing. Figure 8 The "token" in the table represents the lexical symbol currently read in, "token-list.first" represents the first symbol of the linear linked list, "state-stack.top()" represents the state found according to the second lookup table, and "token.cat" represents the part of speech of the lexical symbol currently read in. The value of Action(state,cat) can be represented as a tuple (m,x), where the value of m is R for reduction behavior, the value of m is S for shift behavior, state represents state, and cat represents part of speech. Token = token.next() means that the lexical symbol after the current lexical symbol is used as the current lexical symbol.

[0308] Furthermore, the shift process is indicated by the first lookup table (Action table), and when the read-in symbol and the related state are shifted into the symbol stack and the state stack, the "shift process" is executed, such as Figure 9 As shown, Figure 9Taking a lexical token as an example, the processing flow of the shift process is described in detail. Among them, if the currently read lexical token is related to the scope, the scope operation is performed first, and then the lexical token is converted into a grammar symbol and moved into the symbol stack, and the new state is moved into the state stack. This process is called the "shift process". Symbols related to the scope include "{", "}", "if", "for", "while", "switch", "catch", etc. For example, when the read lexical token is "{", it is necessary to determine the type and operation of the relevant scope based on the symbol stack and the scope stack: (1) If the top of the symbol stack is "named-namespace-head", it means that "{" is the beginning of the namespace declaration. If the top of the symbol stack is "class-head", it means that "{" is the beginning of the class member declaration. It is necessary to create a relevant scope and push the new scope into the scope stack. When the "}" corresponding to "{" is read, it means that the new scope has been completed, and the new scope is popped out of the stack. (2) If the top of the symbol stack is a symbol such as "declarator", "constructor", "destructor", or "converter", it means that "{" is the beginning of a function definition and operations related to the function body scope need to be performed. (3) If the top of the symbol stack is not within the expected range and the top of the current scope stack is the function body scope, it means that "{" is the beginning of a compound statement "compound-statement", and compound statements also have their own scope.

[0309] Since the C++0x standard allows new objects to be defined in the conditions of if, while, for statements or in the initialization process, a scope needs to be created when reading such symbols.

[0310] For example, the following code:

[0311]

[0312] When "if" is read, a new scope (set as "if-scope") is created. When the relevant production rule "a" in "if(int a=x)" is reduced, the variable definition information for "a" is written into the "if-scope." Later, when "a" in the "if-else" statement is analyzed, "part-of-speech determination" indicates that it is a local variable. "{a=1;}" in the "if" statement is still embedded in the "if-scope" as a scope. The "if-scope" can be popped off the stack during the reduction of the production rule: selection-statement → if(condition)statementelse statement.

[0313] For "(", it is sometimes also related to the scope, such as the following code:

[0314]

[0315]

[0316] Among them, when a member function of a class is implemented outside the class declaration scope, the scope of its parameter list is the class declaration scope. When analyzing the "type" in "int A::fun(type*p)", it can be directly concluded from the "part of speech determination" that it is the type name defined in "A", and according to the C++ standard, it does not need to be written as "int A::fun(A::type*p)".

[0317] Furthermore, the reduction process is indicated by the Action table. When the symbol sequence at the top of the symbol stack can be reduced according to a production rule, the "reduction process" is executed. The symbol sequence belonging to the right side of the production rule is popped from the symbol stack, the left side symbol is pushed onto the stack, and the right side symbol sequence is recorded into the left side symbol. After executing all the reduction processes, the symbols are combined into a syntax tree. In addition, it is necessary to record or deduce the attributes of each symbol during the reduction process for use by various mechanisms in the embodiments of this application.

[0318] Example 2

[0319] Figure 10 A schematic diagram of a structure of a device for generating a syntax tree of a code file provided in an embodiment of the present application is shown as follows: Figure 10 As shown, the apparatus 100 may include a parsing module 101, a first processing module 102, and a syntax tree generating module 103, wherein:

[0320] The parsing module 101 is used to parse each lexical symbol in the code file to be parsed and generate a corresponding linear linked list through the lexical parsing module when receiving the code file to be parsed in a predetermined programming language;

[0321] The first processing module 102 is configured to sequentially determine the part of speech of each lexical symbol in the linear linked list, and whenever the part of speech of any lexical symbol is determined, search a first lookup table and a second lookup table according to the part of speech of the lexical symbol to obtain a corresponding search result, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of a predetermined programming language;

[0322] The syntax tree generation module 103 is used to generate a syntax tree of the code file to be parsed according to the search result.

[0323] Specifically, the first processing module 102 is specifically used to determine the part of speech of any lexical symbol according to the context, and / or to guess the part of speech of any lexical symbol according to the context, and determine the part of speech of any lexical symbol.

[0324] Furthermore, the first processing module 102 is specifically used in any of the following situations:

[0325] When any lexical symbol is determined to be any of the type name, object name, and keyword, it is used as the part of speech of the lexical symbol;

[0326] When it is determined that any lexical symbol is a lexical symbol of a preset type, the lexical symbol is determined to be a template list symbol or an operation symbol, and the determination result is used as the part of speech of the lexical symbol;

[0327] Perform information lookup in the scope and use the result as the part of speech of any lexical symbol.

[0328] Furthermore, the first processing module 102 is specifically configured to perform a lateral search in the scope, and use the search result of the lateral search as the part of speech of any lexical symbol;

[0329] A horizontal search is a search in the current scope and the referenced namespace, or a search in the base class scope of the current scope.

[0330] Furthermore, the first processing module 102 is specifically configured to perform a vertical search in the scope when no search result is found through the horizontal search, and use the search result of the vertical search as the part of speech of any lexical symbol;

[0331] A vertical search is a search in scopes that include the current scope.

[0332] Furthermore, the first processing module 102 includes a guessing submodule 1021 and a first determining submodule 1022. Figure 11 As shown, where:

[0333] The guessing submodule 1021 is used to guess whether the part of speech of any lexical symbol is a namespace name based on the context;

[0334] The first determination submodule 1022 is used to determine that the part of speech of any lexical symbol is a type name when the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after any lexical symbol is not a lexical symbol of the preset type, and if the first lexical symbol after any lexical symbol is a lexical symbol of the preset type, the part of speech of any lexical symbol is determined by performing the first preset processing according to any lexical symbol.

[0335] Furthermore, the first processing module 102 further includes a second determining submodule 1023, such as Figure 11 As shown, the second determining submodule 1023 is used to determine that the part of speech of any lexical symbol is an object name when the guess is not a namespace name and when the part of speech of any lexical symbol is guessed to be an object name and not a type name.

[0336] Furthermore, the first processing module 102 further includes a third determining submodule 1024, such as Figure 11 As shown, the third determination submodule 1024 is used to determine whether any lexical symbol is a type name based on the context of any lexical symbol when the guess is not a namespace name and when the guessed part of speech of any lexical symbol is an object name and a type name; if it is a type name, then determine that the part of speech of any lexical symbol is a type name; if it is not a type name, then determine the part of speech of any lexical symbol by performing a second preset processing based on any lexical symbol.

[0337] Furthermore, a second processing module 104 is included, such as Figure 11 As shown, the second processing module 104 is used to perform conflict elimination processing and / or error recovery processing according to the search result.

[0338] Furthermore, the predetermined programming language is any one of C++ programming language and C language.

[0339] Compared with the prior art, the device provided in the embodiment of the present application determines the part of speech of each lexical symbol in the linear linked list in turn, laying the necessary foundation for the subsequent search of the first lookup table and the second lookup table, providing a prerequisite guarantee for generating a syntax tree for the code file to be parsed; whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table generated based on the analysis of the grammar of a predetermined programming language are searched according to the part of speech of any lexical symbol, and the syntax tree of the code file to be parsed is generated according to the search results, thereby providing a method for generating a syntax tree for a code file, so that the completed code file can be statically analyzed through the syntax tree, so that the syntax errors, writing errors, etc. in the completed code file can be accurately and efficiently checked and corrected, greatly saving the time and energy of program developers.

[0340] Example 3

[0341] The present application embodiment provides an electronic device, such as Figure 12 As shown, Figure 12The electronic device 1200 shown includes a processor 1201 and a memory 1203. The processor 1201 and the memory 1203 are connected, for example, via a bus 1202. Furthermore, the electronic device 1200 may also include a transceiver 1204. It should be noted that in actual applications, the number of transceivers 1204 is not limited to one, and the structure of the electronic device 1200 does not constitute a limitation on the embodiments of the present application.

[0342] The processor 1201 is used in the embodiment of the present application to implement Figure 10 or Figure 11 The functions of the parsing module, the first processing module and the syntax tree generation module shown, and Figure 11 The function of the second processing module is shown.

[0343] Processor 1201 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 1201 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0344] The bus 1202 may include a path for transmitting information between the above components. The bus 1202 may be a PCI bus or an EISA bus, etc. The bus 1202 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0345] The memory 1203 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0346] The memory 1203 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1201. The processor 1201 is used to execute the application code stored in the memory 1203 to implement Figure 10 or Figure 11 The illustrated embodiment provides actions of a device for generating a syntax tree for a code file.

[0347] The electronic device provided in the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, compared with the prior art, it can achieve: by determining the part of speech of each lexical symbol in the linear linked list in turn, laying the necessary foundation for subsequent searching the first lookup table and the second lookup table, and providing the prerequisite guarantee for generating a syntax tree for the code file to be parsed; whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table generated based on the analysis of the grammar of a predetermined programming language are searched according to the part of speech of any lexical symbol, and the syntax tree of the code file to be parsed is generated according to the search results, thereby providing a method for generating a syntax tree for a code file, so that the completed code file can be statically analyzed through the syntax tree, so that the syntax errors, writing errors, etc. in the completed code file can be accurately and efficiently checked and corrected, greatly saving the time and energy of program developers.

[0348] The embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method shown in the first embodiment when executed by a processor. Compared with the prior art, by determining the part of speech of each lexical symbol in a linear linked list in turn, the necessary foundation for subsequent searching the first lookup table and the second lookup table is laid, providing a prerequisite for generating a syntax tree for the code file to be parsed; whenever the part of speech of any lexical symbol is determined, the first lookup table and the second lookup table generated based on the analysis of the grammar of a predetermined programming language are searched according to the part of speech of any lexical symbol, and a syntax tree for the code file to be parsed is generated according to the search results, thereby providing a method for generating a syntax tree for a code file, so that the completed code file can be statically analyzed through the syntax tree, thereby accurately and efficiently checking and correcting syntax errors, writing errors, etc. in the completed code file, greatly saving the time and energy of program developers.

[0349] The computer-readable storage medium provided in the embodiments of the present application is applicable to any embodiment of the above method and will not be described in detail here.

[0350] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0351] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating a syntax tree of a code file, characterized in that: include: When receiving a code file to be parsed in a predetermined programming language, the lexical parsing module parses each lexical symbol in the code file to be parsed and generates a corresponding linear linked list; Determining the part of speech of each lexical symbol in the linear linked list in sequence, and whenever the part of speech of any lexical symbol is determined, searching a first lookup table and a second lookup table according to the part of speech of the lexical symbol to obtain a corresponding search result, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of the predetermined programming language; Generate a syntax tree of the code file to be parsed according to the search result; After searching the first lookup table and the second lookup table based on the part of speech of the lexical symbol after each part of speech determination, the method further includes: when it is determined according to the search result that any lexical symbol has a conflict, determining a predetermined conflict type to which the any lexical symbol belongs, and performing corresponding conflict elimination processing on the any lexical symbol according to a conflict elimination rule corresponding to the predetermined conflict type; the conflict refers to finding multiple legal processing behaviors supported by the any lexical symbol at the same time; Generating a syntax tree of the code file to be parsed according to the search result includes: generating a syntax tree according to a result after conflict elimination processing; The predetermined programming language is any one of C++ programming language and C language.

2. The method according to claim 1, characterized in that Determining the part of speech of any lexical symbol in the linear linked list includes: Performing part-of-speech determination on any lexical symbol according to the context to determine the part-of-speech of any lexical symbol; and / or, The part of speech of any lexical symbol is guessed according to the context to determine the part of speech of any lexical symbol.

3. The method according to claim 2, characterized in that The performing part-of-speech determination on any lexical symbol according to the context to determine the part-of-speech of any lexical symbol includes any of the following situations: When any of the lexical symbols is any one of a type name, an object name, and a keyword, it is used as the part of speech of the lexical symbol; When any lexical symbol is a lexical symbol of a preset type, determining whether the any lexical symbol is a template list symbol or an operation symbol, and using the determination result as the part of speech of the any lexical symbol; Information search is performed in the scope, and the search result is used as the part of speech of any lexical symbol.

4. The method according to claim 3, characterized in that The information search is performed in the scope, and the search result is used as the part of speech of any lexical symbol, including: Performing a horizontal search in the scope, and using the search result of the horizontal search as the part of speech of any lexical symbol; The horizontal search is to search in the current scope and the referenced namespace, or to search in the base class scope of the current scope.

5. The method according to claim 4, characterized in that Also includes: If no search result is found through the horizontal search, a vertical search is performed in the scope, and the search result of the vertical search is used as the part of speech of any lexical symbol; The vertical search is a search performed in a scope including the current scope.

6. The method according to claim 2, characterized in that The guessing of the part of speech of any lexical symbol according to the context to determine the part of speech of any lexical symbol includes: Guessing whether the part of speech of any lexical symbol is a namespace name based on the context; If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after any lexical symbol is not a lexical symbol of a preset type, then the part of speech of any lexical symbol is determined to be a type name; if the first lexical symbol after any lexical symbol is a lexical symbol of a preset type, then the part of speech of any lexical symbol is determined by performing a first preset processing according to the any lexical symbol.

7. The method according to claim 6, characterized in that Also includes: If the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is an object name and not a type name, it is determined that the part of speech of any lexical symbol is an object name.

8. The method according to claim 6, characterized in that Also includes: If the guess is not a namespace name, and when the part of speech of any lexical symbol is guessed to be an object name and a type name, determine whether the any lexical symbol is a type name based on the context of the any lexical symbol; if it is a type name, determine that the part of speech of the any lexical symbol is a type name; if it is not a type name, determine the part of speech of the any lexical symbol by performing a second preset processing based on the any lexical symbol.

9. The method according to any one of claims 1 to 8, characterized in that After searching the first lookup table and the second lookup table according to the part of speech of the lexical symbol after each part of speech determination, the method further includes: Perform error recovery processing according to the search result.

10. A device for generating a syntax tree of a code file, characterized in that: include: A parsing module is configured to, upon receiving a code file to be parsed in a predetermined programming language, parse each lexical symbol in the code file to be parsed through a lexical parsing module and generate a corresponding linear linked list; a first processing module, configured to sequentially determine the part of speech of each lexical symbol in the linear linked list, and whenever the part of speech of any lexical symbol is determined, search a first lookup table and a second lookup table according to the part of speech of the lexical symbol to obtain a corresponding search result, wherein the first lookup table and the second lookup table are generated based on analyzing the grammar of the predetermined programming language; A syntax tree generation module, configured to generate a syntax tree for the code file to be parsed according to the search result; A second processing module is configured to, when a conflict is determined for any lexical symbol according to the search result, determine a predetermined conflict type to which the lexical symbol belongs, and perform corresponding conflict elimination processing on the lexical symbol according to a conflict elimination rule corresponding to the predetermined conflict type; the conflict refers to the simultaneous finding of multiple legal processing behaviors supported by the lexical symbol; The syntax tree generation module is further configured to generate a syntax tree according to the result of the conflict elimination process; The predetermined programming language is any one of C++ programming language and C language.

11. The device according to claim 10, characterized in that The first processing module is specifically used to perform part-of-speech judgment on any lexical symbol based on the context to determine the part-of-speech of any lexical symbol; and / or to perform part-of-speech guessing on any lexical symbol based on the context to determine the part-of-speech of any lexical symbol.

12. The device according to claim 11, characterized in that The first processing module is specifically used in any of the following situations: When it is determined that any lexical symbol is any one of a type name, an object name, and a keyword, it is used as the part of speech of the any lexical symbol; When it is determined that any lexical symbol is a lexical symbol of a preset type, determining that any lexical symbol is a template list symbol or an operation symbol, and using the determination result as the part of speech of any lexical symbol; Information search is performed in the scope, and the search result is used as the part of speech of any lexical symbol.

13. The device according to claim 12, characterized in that The first processing module is specifically configured to perform a lateral search within the scope, and use a search result of the lateral search as the part of speech of any lexical symbol; The horizontal search is to search in the current scope and the referenced namespace, or to search in the base class scope of the current scope.

14. The device according to claim 13, characterized in that The first processing module is specifically configured to perform a vertical search within the scope when no search result is found through the horizontal search, and use the search result of the vertical search as the part of speech of any lexical symbol; The vertical search is to search in the scope including the current scope.

15. The device according to claim 11, characterized in that The first processing module includes a guessing submodule and a first determining submodule; The guessing submodule is used to guess whether the part of speech of any lexical symbol is a namespace name based on the context; The first determination submodule is used to, when the guess is not a namespace name, and when the guessed part of speech of any lexical symbol is a type name and not an object name, if the first lexical symbol after the any lexical symbol is not a preset type lexical symbol, determine that the part of speech of the any lexical symbol is a type name; if the first lexical symbol after the any lexical symbol is a preset type lexical symbol, determine the part of speech of the any lexical symbol by performing a first preset processing according to the any lexical symbol.

16. The device according to claim 15, characterized in that The first processing module further includes a second determining submodule; The second determining submodule is configured to determine that the part of speech of any lexical symbol is an object name when the guess is not a namespace name and when the part of speech of any lexical symbol is guessed to be an object name and not a type name.

17. The device according to claim 15, characterized in that The first processing module further includes a third determining submodule; The third determination submodule is used to determine whether any lexical symbol is a type name based on the context of any lexical symbol when the guess is not a namespace name and when the guessed part of speech of any lexical symbol is an object name and a type name; if it is a type name, determine that the part of speech of any lexical symbol is a type name; if it is not a type name, determine the part of speech of any lexical symbol by performing a second preset processing based on the any lexical symbol.

18. The device according to any one of claims 11 to 17, characterized in that Also included is a second processing module; The second processing module is used to perform error recovery processing according to the search result.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for generating a syntax tree of a code file according to any one of claims 1 to 9 is implemented.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for generating a syntax tree for a code file according to any one of claims 1 to 9.

Images