Code processing method, apparatus and electronic device

Abstract

Embodiments of the present application provide a code processing method and device and electronic equipment. In an embodiment, an abstract syntax tree (AST) is generated based on an original code file; a plurality of syntax elements matching a preset condition and reference relationships involved by the plurality of syntax elements are identified from the AST syntax tree; a logic model is constructed based on the plurality of syntax elements and the reference relationships and mapping of each syntax element to the AST syntax tree; and a processing result of the original code file is obtained by accessing the AST syntax tree according to a user-defined processing rule based on the logic model. Thus, by constructing a logic model as an index of the AST syntax tree, a simplified and high readability view of the logic relationship between syntax elements can be provided for the user, and the user can more easily obtain global semantic information of the original code and apply a complex code processing rule.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a code processing method and device and electronic equipment. BACKGROUND

[0002] In many scenarios, syntax conversion of code is needed. For example, structured query language (SQL) is a standard language for accessing relational databases, and many non-relational databases also support SQL. However, different manufacturers often extend the standard SQL, resulting in differences in SQL syntax between different manufacturers, and there are also some differences between the SQL syntax of non-relational databases and the standard SQL. These lead to the problem of interoperability of heterogeneous databases, and SQL script syntax conversion is an important way to solve this interoperability problem.

[0003] However, the current syntax conversion method has the problems of low universality and difficulty in implementing complex syntax conversion. Therefore, a syntax conversion scheme is needed to better meet the actual application requirements. SUMMARY

[0004] The code processing method, device and electronic equipment provided by the embodiments of the present application can be well applied to various syntax conversion scenarios, reduce the use threshold, and enable users to easily process complex structures and complex elements in code.

[0005] In a first aspect, the embodiments of the present application provide a code processing method, comprising:

[0006] An abstract syntax tree (AST) is generated based on an original code file, a plurality of syntax elements matching a preset condition and a reference relationship involved by the plurality of syntax elements are identified from the AST, a logic model is constructed based on the plurality of syntax elements, the reference relationship and a mapping of each syntax element to the AST, and a processing result of the original code file is obtained by accessing the AST based on a user-defined processing rule according to the logic model.

[0007] By constructing a logic model as an index of the AST tree, the embodiments of the present application can ignore most syntax elements that do not need to be focused on, provide a simplified view of the logical relationship between syntax elements and auxiliary understanding of the reference relationship of the global semantics for developers, thereby effectively processing complex syntax elements and facilitating syntax conversion, code analysis or code detection.

[0008] In a possible implementation, the abstract syntax tree AST is generated based on the original code file, specifically including: obtaining a context-free grammar CFG corresponding to a syntax rule of the original code file; obtaining a syntax parser by inputting the CFG into a syntax parser generation device; and obtaining the AST by reading the original code file using the syntax parser.

[0009] In this way, the user does not need to develop a syntax parser, but can generate an AST syntax tree using an automatically generated syntax parser.

[0010] In a possible implementation, the syntax elements and the reference relationship involved in the syntax elements that match the preset condition are identified from the AST, specifically including: identifying the syntax elements and the reference relationship in the process of traversing the AST using a tree traversal algorithm.

[0011] In this way, the tree traversal algorithm can be used to efficiently identify the syntax elements in the AST tree.

[0012] In a possible implementation, the logical model is constructed based on the syntax elements and the reference relationship, and the mapping of each syntax element to the AST, including:

[0013] Based on each syntax element and the mapping of the syntax element to the AST, a corresponding management object is constructed; and based on the management object and the reference relationship, the logical model is constructed.

[0014] Further, in an embodiment, the logical model is a tree structure, in which the management objects are nodes, and the reference relationship involved in the syntax elements in the management objects is a node attribute.

[0015] In a more specific embodiment, the processing rule includes a rule of moving the position of a piece of code; and based on the logical model, the AST is accessed according to the processing rule customized by the user to obtain a processing result of the original code file, including:

[0016] An operation of the user removing a first node from the tree structure and taking the first node as a child node of a second node is received; based on the operation, a subtree in the AST is indexed according to the mapping in the management object corresponding to the first node; and a piece of code indexed according to the subtree is moved to a position corresponding to the second node in the original code file.

[0017] In another embodiment, the logical model is a graph structure, in which the management objects are graph nodes, and a connection edge is established between the management objects having the reference relationship.

[0018] In a possible implementation, the processing rule involves judging a certain code feature, replacing a certain string in the code, moving the position of a piece of code, adding a piece of code, or deleting a piece of code.

[0019] In a possible implementation, the method further includes:

[0020] In a case where the memory shortage is detected, the AST is deleted and its corresponding memory is released; in a case where the original code file is referenced, the AST is regenerated according to the original code file.

[0021] In a possible implementation, the preset condition includes a pre-set node type and / or node attribute.

[0022] In a possible implementation, the mapping of each syntax element to the AST includes a memory location pointing to a corresponding sub-tree in the AST, and / or a path from a root node of the AST to a root node of the sub-tree.

[0023] In a second aspect, an embodiment of the present application provides a code processing apparatus, including:

[0024] The generating unit is configured to generate an abstract syntax tree (AST) based on an original code file; the identifying unit is configured to identify, from the AST, a plurality of syntax elements matching a preset condition and a reference relationship involved by the plurality of syntax elements; the constructing unit is configured to construct a logical model based on the plurality of syntax elements and the reference relationship, and a mapping of each syntax element to the AST; and the processing unit is configured to access the AST according to a user-defined processing rule based on the logical model, to obtain a processing result of the original code file.

[0025] By constructing the logical model as an index of the AST tree, an embodiment of the present application can ignore most syntax elements that do not need to be focused on, provide a simplified view of logical relationships between syntax elements for a developer, and assist in understanding a reference relationship of a global semantic, so that a complex syntax element can be effectively processed, and syntax conversion, code analysis or code detection, etc. can be facilitated.

[0026] In a possible implementation, the generating unit is specifically configured to:

[0027] obtain a context-free grammar (CFG) corresponding to a syntax rule of the original code file; input the CFG into a syntax parser generator to obtain a syntax parser; and read the original code file by using the syntax parser to obtain the AST.

[0028] In this way, the user does not need to develop a syntax parser, but can generate the AST syntax tree by using the automatically generated syntax parser.

[0029] In a possible implementation, the preset condition includes a pre-set node type and / or node attribute.

[0030] In a possible implementation, the identifying unit is specifically configured to:

[0031] In the process of traversing the AST by using the tree traversal algorithm, a number of syntax elements and reference relationships are identified.

[0032] In this way, the tree traversal algorithm can be used to efficiently identify the syntax elements in the AST tree.

[0033] In a possible implementation, the mapping of each syntax element to the AST includes a memory location pointing to a corresponding sub-tree in the AST, and / or a path from a root node of the AST to a root node of the sub-tree.

[0034] In a possible implementation, the constructing unit is specifically configured to:

[0035] Based on the syntax elements and the mapping of each syntax element to the AST, a corresponding management object is constructed; and based on the management objects and the reference relationships, a logical model is constructed.

[0036] Further, in an embodiment, the logical model is a tree structure, in which the management objects are nodes, and the reference relationships involved in the syntax elements in the management objects are node properties.

[0037] In a more specific embodiment, the AST includes a first syntax element and a second syntax element, the first syntax element being an ancestor of the second syntax element; in the tree structure, the management object containing the first syntax element is an ancestor of the management object containing the second syntax element.

[0038] In a more specific embodiment, the management object further includes a link list of its child nodes, and / or a link to its parent node.

[0039] In a more specific embodiment, the processing rule includes a rule of moving a position of a piece of code; and the processing unit is specifically configured to: receive an operation of a user removing a first node from the tree structure and making the first node a child node of a second node; based on the operation, index to a sub-tree in the AST according to the mapping in the management object corresponding to the first node; and move a piece of code indexed according to the sub-tree to a position corresponding to the second node in the original code file.

[0040] In another embodiment, the logical model is a graph structure, in which the management objects are graph nodes, and a connection edge is established between the management objects having reference relationships.

[0041] In a possible implementation, the processing rule involves judging a certain code feature, replacing a certain string in the code, moving a position of a piece of code, adding a piece of code, or deleting a piece of code.

[0042] In a possible implementation, the code processing apparatus further includes:

[0043] The deleting unit is configured to delete the AST and release the memory corresponding to the AST when the memory shortage is detected.

[0044] The reconstructing unit is configured to regenerate the AST according to the original code file when the original code file is referenced.

[0045] In a third aspect, an electronic device is provided, which includes at least one memory configured to store a program, and at least one processor configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method provided in the first aspect or any possible implementation manner of the first aspect.

[0046] In a fourth aspect, a code processing apparatus is provided. The apparatus runs computer program instructions to execute the method provided in the first aspect or any possible implementation manner of the first aspect. Exemplarily, the apparatus can be a chip or a processor.

[0047] In one example, the apparatus can include a processor which can be coupled with a memory, read instructions in the memory and execute the method provided in the first aspect or any possible implementation manner of the first aspect according to the instructions. The memory can be integrated in the chip or the processor, or can be independent of the chip or the processor.

[0048] In a fifth aspect, a computer storage medium is provided. The computer storage medium stores instructions. When the instructions run on a computer, the computer executes the method provided in the first aspect or any possible implementation manner of the first aspect.

[0049] In a sixth aspect, a computer program product is provided. The computer program product includes instructions. When the instructions run on a computer, the computer executes the method provided in the first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 FIG. 1 is a schematic diagram of an implementation framework of a code processing scheme provided by the embodiments of the present application;

[0051] Figure 2 FIG. 2 is a schematic diagram of a flow of a code processing method provided by the embodiments of the present application;

[0052] Figure 3 FIG. 3 is a schematic diagram of a structure of a code processing apparatus provided by the embodiments of the present application;

[0053] Figure 4 FIG. 4 is a schematic diagram of a structure of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0055] In the description of the embodiments of the present application, the words "exemplarily", "for example", "for instance" or the like are used to mean example, illustration or description. Any embodiment or design scheme described as "exemplarily", "for example" or "for instance" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. In fact, the words "exemplarily", "for example" or "for instance" are used to present the relevant concept in a specific way.

[0056] In the description of the embodiments of the present application, the term "and / or" is merely used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of existence of A alone, existence of B alone and existence of A and B simultaneously. In addition, unless otherwise specified, the term "several" means one or more than one; for example, several syntax elements mean one or more than one syntax element. The term "multiple" means two or more than two; for example, multiple code files mean two or more than two code files.

[0057] In addition, the terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0058] As described above, syntax conversion of code is required in many scenarios. In addition to the requirement of SQL script syntax conversion in the database access scenario, for example, for high-level languages such as JAVA / C++ / python, there are syntax differences (such as C++98 and C++11) or backward incompatible situations (such as python2 and python3) between different versions, which can be solved by syntax conversion.

[0059] Currently, there are problems of low universality and high complexity in the conversion of code based on context-free grammar (CFG), such as code written in SQL, JAVA, C++, python, etc. For example, in order to realize syntax conversion, regular expression matching and replacement of strings in the original code can be used. However, the description ability of regular expression is only a proper subset of CFG, and there are syntax structures that cannot be described and converted by regular expression. This technology can only solve part of the syntax conversion scenarios.

[0060] For another example, the original code is converted into an abstract syntax tree (AST) by a code parser (such as ANTLR), and the code is modified according to the conversion rule in the process of traversing the AST. However, the AST is a syntax-oriented structure, and it is difficult to understand the semantics of the original code and the global information of the complex structure implied therein, so it is difficult to implement conversion rules with strong semantic relevance or requiring global information.

[0061] Based on the above observations and analysis, in the embodiments of the present application, AST is used as the memory representation of the original code, and a logical model is used as the index and semantic representation of the syntax object in the AST. The introduction of the logical model makes it easier for users to obtain the global semantic information of the original code and more easily apply complex code processing rules.

[0062] Figure 1 is an implementation framework schematic diagram of a code processing scheme provided by the embodiments of the present application. As shown in Figure 1 In the code processing scheme disclosed in the embodiments of the present application, the original code is first parsed into an AST (usually one code file corresponds to one syntax tree) by a syntax parser 110, and then an AST parser 120 is used to identify syntax elements in the AST that meet the business requirements and need to be paid attention to, such as functions / processes, packages, objects, variables, etc., and references between syntax elements that help users understand global semantic information, and then a logical model is constructed based on the identified syntax elements and their references. Then, the code processing device 130 accesses the AST according to the user-defined processing rules with the help of the global semantic information of the code provided by the logical model, and outputs the processing result of the code. It should be understood that the AST parser 120 and the code processing device 130 are both devices that need to be developed by the user, and the syntax parser 110 can be developed by the user or implemented by using existing tools.

[0063] From the above, by introducing a logic model as an index of the AST, the syntax elements that need to be focused on and their references can be extracted, while most of the syntax elements that do not need to be focused on, such as most underlying identifiers, symbols, expressions, statements, etc., are ignored, thereby providing the developer with a simplified view of the logical relationship between the syntax elements and the global semantic information of the code, facilitating efficient implementation of syntax conversion, and can be applied to scenarios in which the original code contains complex syntax structures.

[0064] The implementation steps of the above scheme will be described below in combination with more embodiments. Figure 2 is a flowchart of a code processing method provided by an embodiment of the present application. The code processing method can be applied in an electronic device, but is not limited thereto. For example, the electronic device can be a server, but is not limited thereto. As shown in Figure 2 The code processing method provided by an embodiment of the present application includes the following steps S201-S204.

[0065] Step S201: generating an abstract syntax tree (AST) based on an original code file. Specifically, the original code file is input into Figure 1 The syntax parser 110 shown in the figure is obtained.

[0066] In some embodiments, the syntax parser 110 is developed by a user.

[0067] In other embodiments, the syntax parser 110 can be generated with the aid of existing tools. Specifically, a context-free grammar (CFG) corresponding to the syntax rules of the original code file is first obtained. It should be understood that the syntax rules (or syntax specifications) are adapted to the programming language of the code file and are predetermined. Based on this, in one possible case, the syntax rules are identical to the CFG grammar, and in this case, the obtained syntax rules can be directly used as the corresponding CFG grammar. In another possible case, the syntax rules are not identical to the CFG grammar, and in this case, the corresponding CFG grammar can be developed by a staff member. Then, the obtained CFG is input into a syntax parser generation device to obtain Figure 1 The syntax parser 110 shown in the figure.

[0068] It should be noted that the syntax parser generation device is an automatic software generation device. The input CFG is parsed according to the output code, and the output code is compiled into a syntax parsing device. Alternatively, if the output code is an interpreted language such as Python, the output code itself is the code parsing device and does not need to be compiled. The syntax parser generation device generally does not need to be developed by a user, but can be directly used as an open source component, such as ANTLR.

[0069] In this way, the user needs to develop the CFG grammar at most, so the workload of obtaining the syntax parser 110 is greatly reduced.

[0070] From the above, the original code file can be input into the syntax parser 110, so as to obtain the corresponding AST tree.

[0071] Step S202: identifying a plurality of syntax elements matching the preset condition and the reference relationship involved in the plurality of syntax elements from the AST syntax tree.

[0072] It should be understood that the AST syntax tree is an abstract representation of the syntax structure of the source code, which represents the syntax structure of the programming language in the form of a tree, and each node on the tree represents a structure in the source code. The AST syntax tree includes all syntax structures in the original code file, and each node on the tree corresponds to a syntax element.

[0073] Considering that the AST syntax tree is a syntax-oriented structure, it is difficult to understand the semantics of the original code and the global semantic information of the complex structure contained therein, and different businesses need to pay attention to different syntax elements, and many underlying syntax elements are actually not needed to be paid attention to. The poor readability of the AST syntax tree hinders the staff from obtaining the global semantics and defining and applying the processing rules.

[0074] Based on this, in this step, the syntax elements that need to be paid attention to in the AST tree and the reference relationship between the syntax elements that assist in understanding the global semantics can be identified, thereby helping the user to better understand the code semantics.

[0075] In some embodiments, the preset condition can be pre-set by the staff according to business needs, experience, etc. In some embodiments, the preset condition includes a pre-set node type and / or node attribute. It can be understood that the tree nodes in the AST have a node type, and each type of node defines some additional node attributes to further describe the node type; exemplary node types include, for example, FunctionDeclaration, Identifier, BinaryExpression, ReturnStatement, etc., wherein the attributes of ReturnStatement include an expression representing the return content.

[0076] In some embodiments, the reference relationship involved in the plurality of syntax elements can include the reference relationship formed by each syntax element referencing or being referenced by other syntax elements.

[0077] In some embodiments, the plurality of syntax elements and the reference relationship involved therein are identified in the process of traversing the AST syntax tree by using a tree traversal algorithm. Exemplarily, the tree traversal algorithm can be a pre-order traversal algorithm, a mid-order traversal algorithm, or a post-order traversal algorithm, etc.

[0078] In some embodiments, the original code file is a SQL script, and accordingly, the SQL syntax elements identified in this step can include procedures, functions, variables, etc. in the SQL, and identifying the reference relationship can include variable reference, function call, etc.

[0079] As described above, the key information in the AST tree can be extracted, including a number of syntax elements and the reference relationship involved. It can be understood that this step can be implemented by using the AST parser 120 shown in FIG. 1, the input is the AST syntax tree, and the output is a number of syntax elements identified and the reference relationship involved. Figure 1

[0080] It should be noted that in steps S201 and S202, the AST tree is first generated, and then the syntax elements and the related references that need to be concerned are identified by traversing the AST tree. In fact, the syntax elements that match the preset conditions and the reference relationship between the syntax elements can also be identified synchronously in the process of generating the AST tree.

[0081] Step S203: Based on the above-mentioned a number of syntax elements and the reference relationship involved, and the mapping of each syntax element to the AST syntax tree, a logical model is constructed.

[0082] It can be understood that the a number of syntax elements are identified from the AST tree, and the AST tree includes these syntax elements, including the sub-tree with each syntax element in the a number of syntax elements as the root node. In some embodiments, the mapping of each syntax element to the AST tree includes the memory location of the syntax element pointing to the corresponding sub-tree in the AST. In some embodiments, the mapping of each syntax element to the AST tree includes the path from the root node of the AST to the root node of the sub-tree corresponding to the syntax element.

[0083] Specifically, for each syntax element identified, a corresponding management object is constructed, which includes the mapping of the syntax element to the AST syntax tree; and based on the management object and the reference relationship involved by the syntax element, a logical model is constructed.

[0084] In some embodiments, the model structure of the logical model is a tree structure, in which the management object is a node, and the reference relationship involved by the syntax element in the management object is a node attribute. In this way, when a user views the logical model, the syntax element and the reference relationship involved can be viewed at the same time.

[0085] In some embodiments, the AST tree includes a first syntax element and a second syntax element, and the first syntax element is an ancestor of the second syntax element, and accordingly, in the above-mentioned tree structure, the management object containing the first syntax element is an ancestor of the management object containing the second syntax element.

[0086] ​In some embodiments, based on the above tree structure, the management object can further include a link list of its child nodes and / or a link of its parent node.

[0087] The logical model can also be organized in other structural forms other than tree structure, such as graph structure. In some embodiments, the model structure of the logical model is graph structure, in which the management objects are graph nodes and the connection edges are established between the management objects with reference relationship.

[0088] From the above, the logical model can be established to serve as an index of the AST syntax tree, so that most of the syntax elements in the AST tree that do not need to be concerned can be ignored, such as most of the underlying identifiers, symbols, expressions, statements, etc., to provide a simplified view of the logical relationship between the syntax elements for the developer. In addition, the reference relationship involved in each syntax element is also presented concisely in the logical model, which can also effectively help the developer to more comprehensively and efficiently understand the global semantic information.

[0089] Step S204: Based on the logical model, the AST syntax tree is accessed according to the user-defined processing rule to obtain a processing result of the original code file.

[0090] It should be understood that the processing rule can be regarded as a processing logic, which includes supporting the user to make what operation and how to utilize the logical model and the AST syntax tree, etc. to complete the corresponding processing of the code file under various operations. In addition, the processing rule can be implemented by the code processing apparatus 130 shown in the embodiment. Figure 1 The code processing apparatus 130 shown in the embodiment can be implemented.

[0091] In one embodiment, the processing rule includes a rule of moving the position of a piece of code, based on which the implementation of this step can include: receiving an operation of the user removing the first node from the tree structure of the logical model and taking it as the second child node; based on the operation, indexing to a subtree in the AST tree according to the mapping included in the management object corresponding to the first node; and then moving a piece of code indexed according to the subtree to the position corresponding to the second node in the original code file. It can be understood that the movement of the code is equivalent to deleting the code at the original position and adding it to the new position.

[0092] Exemplarily, the processing rule can also include: judging a certain code feature, replacing a certain string in the code, adding a piece of code, deleting a piece of code, etc.

[0093] Therefore, the user can trigger rewriting, analysis, etc. of the original code file by means of the AST simplified view and global semantic information provided by the logical model, so that the background responds to these operations, indexes to the AST tree through the logical model, and completes the corresponding processing of the code file.

[0094] According to another aspect, in consideration of the memory shortage that can be caused when parsing a large amount of original code, the present application also discloses the eviction and reconstruction of the AST tree. Specifically, after the construction of the logical model is completed in step S203 described above, the method described in the present application can further include a step of deleting the AST tree and releasing the memory corresponding to the AST tree when a memory shortage is detected. In one possible case, a plurality of AST trees corresponding to a plurality of code files are stored in the memory, and at least one of the AST trees, which includes the AST syntax tree generated in step S201, can be evicted using an eviction algorithm. Exemplarily, the eviction algorithm can be a Least Recently Used (LRU) algorithm, a Least Frequently Used (LFU) algorithm, or the like.

[0095] After that, in the case that the original code file is referenced, the AST tree can be regenerated according to the original code file. In one possible implementation, the deletion of the AST tree occurs before step S204, and the regeneration of the AST tree also occurs before step S204. In another possible implementation, the deletion of the AST tree occurs after step S204, and the reconstruction of the AST tree also occurs after step S204. Exemplarily, the case that the original code file is referenced includes the case that the invocation of a function in the original code file is triggered when another code file is referenced.

[0096] In some embodiments, after the AST tree is reconstructed, the node in which the sub-tree root is located in the regenerated AST tree can also be searched according to the path from the root node of the AST tree to the sub-tree root saved in the logical model, so as to re-add the link to the memory location of the corresponding sub-tree.

[0097] It can be understood that for the reconstructed AST tree, it can be accessed according to the user-defined processing rule based on the corresponding logical model, so as to obtain the processing result of the original code file.

[0098] Therefore, by designing the eviction and reconstruction of the AST tree, the present application can work in a memory-limited scenario, such as various Internet of Things (IoT) devices with small memory, or a scenario in which a large amount of original code needs to be parsed on a single server and causes a memory shortage.

[0099] In summary, the AST tree is very complex, has many tree nodes, and has many levels. In the code processing method disclosed in the embodiments of the present application, the logical model is constructed as an index of the AST tree, most of the syntax elements that do not need to be focused on can be ignored, for example, most of the underlying identifiers, symbols, expressions, statements, and the like, a simplified, high- readability view of the logical relationship between the syntax elements is provided for the developer, and the reference relationship that assists in understanding the global semantics, so that the complex syntax elements such as packages, procedures, and custom object types in SQL can be effectively processed, and syntax conversion, code analysis, or code detection and the like are facilitated. Further, by introducing the elimination mechanism of the AST tree, the code processing method can work in a memory-limited or relatively memory-limited scenario.

[0100] Based on Figure 2 According to the corresponding embodiments, the present application further provides a code processing apparatus.

[0101] Figure 3 FIG. 1 is a structural schematic diagram of a code processing apparatus provided by an embodiment of the present application. As shown in FIG. 1, the code processing apparatus provided by the embodiment of the present application comprises a generating unit 301, an identifying unit 302, a constructing unit 303, and a processing unit 304. Figure 3 The generating unit 301 is configured to generate an abstract syntax tree (AST) based on an original code file; the identifying unit 302 is configured to identify, from the AST, a plurality of syntax elements that match a preset condition and a reference relationship involved by the plurality of syntax elements; the constructing unit 303 is configured to construct a logical model based on the plurality of syntax elements and the reference relationship and a mapping of each syntax element to the AST; and the processing unit 304 is configured to access the AST according to a processing rule customized by a user based on the logical model, to obtain a processing result of the original code file.

[0102] The generating unit 301 is configured to generate an abstract syntax tree (AST) based on an original code file; the identifying unit 302 is configured to identify, from the AST, a plurality of syntax elements that match a preset condition and a reference relationship involved by the plurality of syntax elements; the constructing unit 303 is configured to construct a logical model based on the plurality of syntax elements and the reference relationship and a mapping of each syntax element to the AST; and the processing unit 304 is configured to access the AST according to a processing rule customized by a user based on the logical model, to obtain a processing result of the original code file.

[0103] The embodiments of the present application can ignore most of the syntax elements that do not need to be focused on by constructing the logical model as an index of the AST tree, provide a simplified view of the logical relationship between the syntax elements for the developer, and assist in understanding the reference relationship of the global semantics, so that the complex syntax elements can be effectively processed, and syntax conversion, code analysis, or code detection and the like are facilitated.

[0104] In a possible implementation, the generating unit 301 is specifically configured to:

[0105] obtain a context-free grammar (CFG) corresponding to a syntax rule of the original code file; input the CFG into a syntax parser generator to obtain a syntax parser; and read the original code file by using the syntax parser to obtain the AST.

[0106] In this way, the user does not need to develop the syntax parser, but can generate the AST syntax tree by using the automatically generated syntax parser.

[0107] In a possible implementation, the preset condition includes a preset node type and / or a preset node attribute.

[0108] In a possible implementation, the identifying unit 302 is specifically configured to:

[0109] In the process of traversing the AST by using the tree traversal algorithm, a plurality of syntax elements and reference relationships are identified.

[0110] In this way, the tree traversal algorithm can be used to efficiently identify the syntax elements in the AST tree.

[0111] In a possible implementation, the mapping of each syntax element to the AST includes a memory location pointing to a corresponding sub-tree in the AST, and / or a path from a root node of the AST to a root node of the sub-tree.

[0112] In a possible implementation, the constructing unit 303 is specifically configured to:

[0113] Based on each syntax element and the mapping of the syntax element to the AST, a corresponding management object is constructed; and based on the management objects and the reference relationships, a logical model is constructed.

[0114] Further, in an embodiment, the logical model is a tree structure, in which the management objects are nodes, and the reference relationships involved in the syntax elements in the management objects are node attributes.

[0115] In a more specific embodiment, the AST includes a first syntax element and a second syntax element, the first syntax element being an ancestor of the second syntax element; in the tree structure, a management object containing the first syntax element is an ancestor of a management object containing the second syntax element.

[0116] In a more specific embodiment, the management object further includes a link list of its child nodes, and / or a link of its parent node.

[0117] In a more specific embodiment, the processing rule includes a rule of moving a position of a piece of code; the processing unit 303 is specifically configured to: receive an operation of a user removing a first node from the tree structure and making the first node a child node of a second node; based on the operation, index, according to the mapping in the management object corresponding to the first node, to a sub-tree in the AST; and move a piece of code indexed according to the sub-tree to a position corresponding to the second node in the original code file.

[0118] In another embodiment, the logical model is a graph structure, in which the management objects are graph nodes, and a connection edge is established between the management objects having the reference relationship.

[0119] In a possible implementation, the processing rule involves judging a certain code feature, replacing a certain string in the code, moving a certain piece of code, adding a certain piece of code, or deleting a certain piece of code.

[0120] In a possible implementation, the code processing apparatus further includes:

[0121] The deleting unit is configured to delete the AST and release the memory corresponding to the AST in a case where memory shortage is detected.

[0122] The reconstructing unit is configured to regenerate the AST according to the original code file in a case where the original code file is referenced.

[0123] The described apparatus embodiment is merely illustrative, for example, the division of the unit is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present alone, or two or more units can be integrated in one module. Figure 3 The above-mentioned units can be realized in the form of hardware, in the form of software function units, or in the form of a combination of software and hardware. Figure 3

[0124] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 4 The electronic device 400 provided by the embodiment of the present application includes a processor 401, a memory 402, and an interface 403. The processor 401, the memory 402, and the interface 403 are connected through a bus 410. The memory 402 includes an operating system and a program code module.

[0125] The memory 402 can include a mass storage for data or instructions. By way of example and not limitation, the memory 402 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 402 can include removable or non-removable (or fixed) media. In a particular embodiment, the memory 402 is a non-volatile solid-state memory.

[0126] ​The memory can include read-only memory (ROM), random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (e.g., by one or more processors), is capable to perform a fault root cause localization method provided by embodiments of the present application.

[0127] The processor 401 implements the code processing method in any of the above embodiments by reading and executing computer program instructions stored in the memory 402.

[0128] In one example, the electronic device can further include an interface 403 and a bus 410. As shown, the processor 401, the memory 402, and the interface 403 are connected through the bus 410 and complete communication among each other. Figure 4

[0129] The interface 403 is mainly used to realize the communication between the modules, devices, units and / or electronic devices in the embodiments of the present application.

[0130] The bus 410 includes hardware, software or both to couple components of the electronic device to each other. By way of example, and not limitation, the bus can include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a HyperTransport (HT) interconnect, an industry standard architecture (ISA) bus, an InfiniBand (IB) interconnect, a low pin count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standards board (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, the bus 410 can include one or more buses. Although the present application describes and illustrates a particular bus, the present application contemplates any suitable bus or interconnect.

[0131] ​It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0132] The method steps in the embodiments of the present application can be realized by hardware or by the processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0133] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired manner, such as coaxial cable, optical fiber, digital subscriber line (DSL), or a wireless manner, such as infrared, wireless, microwave, etc. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage computing device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (such as a DVD, etc.), or a semiconductor medium, such as a solid state disk (SSD), etc.

[0134] It can be understood that various numerical numbers involved in the embodiments of the present application are only distinguished for convenience of description, and are not used to limit the scope of the embodiments of the present application.

Claims

1. A method of code processing, characterized by, The method comprises: generating an abstract syntax tree (AST) based on an original code file; identifying a plurality of syntax elements matching preset conditions and reference relationships involved by the syntax elements from the AST; constructing a logical model based on the syntax elements and reference relationships and mapping of each syntax element to the AST, comprising: constructing a corresponding management object based on the syntax elements and mapping of each syntax element to the AST; and constructing the logical model based on the management object and the reference relationships; accessing the AST according to a user-defined processing rule based on the logical model to obtain a processing result of the original code file; the processing rule involves judging a certain code feature, replacing a certain string in the code, moving a certain code segment, adding a certain code segment, or deleting a certain code segment.

2. The method of claim 1, wherein, The logical model is a tree structure, and the tree structure takes the management object as a node and takes the reference relationships involved by the syntax elements in the management object as a node attribute.

3. The method of claim 2, wherein, The processing rule includes a rule for moving a certain code segment; accessing the AST according to a user-defined processing rule based on the logical model to obtain a processing result of the original code file, comprising: receiving an operation of removing a first node from the tree structure and taking the first node as a child node of a second node; based on the operation, indexing a subtree in the AST according to the mapping in the management object corresponding to the first node; moving a code segment indexed according to the subtree to a position corresponding to the second node in the original code file.

4. The method of claim 1, wherein, The method further comprises: in the case of detecting insufficient memory, deleting the AST and releasing the memory corresponding to the AST; in the case that the original code file is referenced, regenerating the AST according to the original code file.

5. The method of claim 1, wherein, The logical model is a graph structure, and the graph structure takes the management object as a graph node and establishes a connection edge between management objects having reference relationships.

6. The method according to any one of claims 1-5, characterized in that, The preset conditions include pre-set node types and / or node attributes.

7. The method according to any one of claims 1-5, characterized in that, The mapping of each syntax element to the AST includes a memory location pointing to a corresponding subtree in the AST and / or a path from an AST root node to a root node of the subtree.

8. An electronic device, comprising: The method comprises: at least one memory for storing a program; at least one processor for executing the program stored in the memory, when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-7.

Images