Indent-based source code structure inference

Abstract

A source code structure is inferred based on the indentation. A portion of source code that includes a plurality of rows is identified, and a corresponding indentation level for each row is determined. A plurality of nodes are generated, each node representing a different row of the plurality of rows. And arranging the nodes into an ordered tree based on the indentation level corresponding to each row. Based on an identified row of the plurality of rows, a subset of the plurality of rows is identified from the ordered tree based on identifying a sub-tree that includes nodes representing the identified row. A source code action is initiated based on the subset of the plurality of rows.

Description

Background Art

[0001] In computing, parsing, grammatical analysis, or syntax analysis refers to the formal analysis of an input string by a computer, so that the components of the input string are identified, and the syntactic relations between these components are identified. When applied to the analysis of computer languages, parsing refers to the syntactic analysis of the input code into its components, which helps compile, interpret, etc. A source code parser is a software component that parses computer source code. A source code parser is written for a specific source code language (or languages) based on a detailed understanding of the grammatical rules of the (multiple) target source code languages. A source code parser obtains the input source code, and builds a data structure, such as an abstract syntax tree (AST), according to the grammatical rules of the language in which the source code is written, which captures the accurate structure of the source code.

[0002] The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above.Rather, this background is provided merely to illustrate one exemplary technology area where some embodiments described herein may be practiced. Summary of the invention

[0003] In some aspects, the technology described herein relates to a method for inferring source code structure based on indentation implemented on a computer system including a processor, the method comprising: identifying a portion of source code including multiple lines; determining a corresponding indentation level for each of the multiple lines; generating multiple nodes, each node representing a different line of the multiple lines; arranging the multiple nodes into an ordered tree based on at least the corresponding indentation level of each of the multiple lines; based on an identified line of the multiple lines: identifying a subtree from the ordered tree including a node representing the identified line; identifying a subset of the multiple nodes corresponding to the subtree; and identifying a subset of the multiple lines represented by the subset of the multiple nodes as being syntactically related to the identified line; and initiating a source code action based on the subset of the multiple lines.

[0004] In some aspects, the technology described herein relates to a computer system for inferring source code structure based on indentation, the system comprising: a processor; and a computer storage medium storing computer executable instructions executable by the processor to cause the computer system to at least: identify a portion of source code including a plurality of lines; determine a corresponding indentation level for each of the plurality of lines; generate a plurality of nodes, each node representing a different line of the plurality of lines; arrange the plurality of nodes into an ordered tree based at least on the corresponding indentation level for each of the plurality of lines; based on an identified line of the plurality of lines: identify from the ordered tree a subtree including a node representing the identified line; identify a subset of the plurality of nodes corresponding to the subtree; and identify a subset of the plurality of lines represented by the subset of the plurality of nodes as being syntactically related to the identified line; and initiate a source code action based on the subset of the plurality of lines.

[0005] In some aspects, the technology described herein relates to a computer program product including a computer storage medium storing computer executable instructions executable by a processor to cause a computer system to infer source code structure based on indentation, the computer executable instructions including instructions executable by the processor to cause the computer system to at least: identify a portion of source code including a plurality of lines; determine a corresponding indentation level for each of the plurality of lines; generate a plurality of nodes, each node representing a different line of the plurality of lines; arrange the plurality of nodes into an ordered tree based at least on the corresponding indentation level for each of the plurality of lines; based on an identified line of the plurality of lines: identify from the ordered tree a subtree including a node representing the identified line; identify a subset of the plurality of nodes corresponding to the subtree; and identify a subset of the plurality of lines represented by the subset of the plurality of nodes as being syntactically related to the identified line; and initiate a source code action based on the subset of the plurality of lines.

[0006] This summary is provided to introduce a series of concepts in a simplified form, which are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to describe the manner in which the advantages and features of the systems and methods described herein can be obtained, a more particular description of the embodiments briefly described above will be provided with reference to specific embodiments illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the systems and methods described herein and are, therefore, not to be considered limiting of their scope, certain systems and methods will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0008] Figure 1An example computer architecture is shown that facilitates indentation-based reasoning about source code structure;

[0009] Figure 2 shows an example of the internal components of a partial parser that infers code structure based on indentation;

[0010] Figure 3 An example of source code including indentation is shown;

[0011] Figure 4A An example of generating an ordered code structure tree based on source code indentation is shown;

[0012] Figure 4B It shows that it can be applied to Figure 4A An example of syntactic refinement of an ordered code structure tree;

[0013] Figure 4C shows an example of a refined code structure tree; and

[0014] Figure 5 A flow chart of an example method for inferring source code structure based on indentation is shown. DETAILED DESCRIPTION

[0015] Although conventional source code parsers are very powerful, there are challenges in using them in many contexts—such as when they need to handle many different programming languages ​​and need to work for all or most programming languages, when they need to handle code that is in a partially completed state, and / or when responsiveness is a priority (e.g., when parsing is integral to the human interactive experience). For example, some source code editors provide intelligent interactive suggestions based on an understanding of the code written and / or based on an understanding of code related to the code written. At least some embodiments of this document are directed to indentation-based source code structure inference. Unlike conventional parsers that use grammatical rules to capture the exact structure of source code, the embodiments described herein parse "partially" based on indentation. These embodiments are fast, require fewer computing resources than traditional parsing, and capture the overall structural characteristics of even partially completed source code that contains errors.

[0016] Figure 1An example computer architecture 100 is shown that facilitates indentation-based source code structure reasoning. As shown, computer architecture 100 includes a computer system 101, which includes (multiple) processors 102 (e.g., a single processor, or multiple processors), a memory 103 (e.g., a system or main memory), a storage medium 104 (e.g., a single computer-readable storage medium, or multiple computer-readable storage media), all of which are interconnected by a bus 106. Computer system 101 may also include a network interface 105 (e.g., one or more network interface cards) for interconnecting to (multiple) computer systems 108 (e.g., a single computer system, or multiple computer systems) (via a network 107).

[0017] Storage medium 104 is shown as storing computer executable instructions that implement partial parser 109. In addition, computer architecture 100 is shown as including source code 111 (e.g., one or more portions of source code stored in storage medium 104 and / or computer system(s) 108) on which partial parser 109 operates. In an embodiment, source code 111 includes source code files (e.g., stored on non-volatile media such as storage medium 104), or source code within an editor buffer (e.g., stored on volatile media such as memory 103).

[0018] In an embodiment, part parser 109 parses source code 111 to generate a code structure tree 112 that captures the overall structure of source code 111. Part parser 109 operates based on the strict convention of the use of humans and many source code editors usually complying with indentation when writing source code, even in most source code languages, indentation is fully optional. Therefore, unlike the conventional parser that parses source code based on the strict grammar of writing source code, part parser 109 described herein "partially" parses based on the indentation included in the source code 111. Because part parser 109 parses based on indentation rather than grammatical rules, code structure tree 112 can only capture the partial structure of source code 111 (rather than the accurate structure that will capture in the case of conventional parser). Nevertheless, as described herein, this code structure tree 112 is still useful for many source code actions.

[0019] In an embodiment, by parsing source code based on indentation rather than based on strict grammatical rules, the partial parser 109 can infer the overall code structure faster and more effectively than a conventional parser. In addition, because the partial parser 109 operates based on indentation rather than strict grammatical rules, the partial parser 109 can also infer the overall code structure even when the input source code is partially completed and / or contains errors. In an embodiment, these advantages make the partial parser 109 useful for use in conjunction with a source code consumer 110, which processes many different programming languages ​​and needs to work for all or most programming languages, processes code in a partially completed state, and / or where responsiveness is a priority (e.g., when the source code consumer 110 is human interactive). For example, these advantages make it useful for the partial parser 109 to be combined with a code editor—such as VISUAL STUDIO from MICROSOFT CORPORATION, COPILOT from GITHUB, etc.—these editors provide intelligent source code completion suggestions based on an understanding of the overall structure of the written code.

[0020] In an embodiment, the partial parser 109 operates by traversing the source code 111 to generate a code structure tree 112 based on the indentation within the source code 111. In an embodiment, during the traversal, each line of the source code 111 is converted into a node, and the nodes are arranged based on the indentation level of its corresponding line. In an embodiment, during the traversal, blank lines (e.g., lines that contain no characters at all, or lines that contain only blank characters) are grouped with their nearest non-blank ancestor lines (e.g., lines that contain at least one non-blank character).

[0021] In some embodiments, the partial parser 109 also performs one or more additional traversals of the code structure tree 112 to refine the code structure tree 112 based on common source code syntax. For example, the partial parser 109 may refine the code structure tree 112 by marking nodes to identify common source code constructs such as classes or functions, and / or may refine the code structure tree 112 by regrouping nodes into clauses of their parent nodes—such as based on common “continue” words (e.g., same-level control flow words, such as “then” or “else”) or symbols (e.g., parentheses) contained in the source code.

[0022] In an embodiment, as a result of these one or more scans, the partial parser 109 generates a code structure tree 112 that defines many elements of the AST that could have been obtained by conventional parsing using strict grammatical rules, but the partial parser 109 performs it in a manner that is robust to code incompleteness and errors, is very fast and computationally efficient, and is applicable to a variety of source code languages.

[0023] Figure 2 Shows Figure 1 200 of an example of internal components of a portion of parser 109 . Figure 2 Each internal component of the partial parser 109 depicted in represents various functions that the partial parser 109 can implement according to various embodiments described herein. However, it should be understood that the depicted components—including their identification and arrangement—are presented only as an aid to describing example embodiments of the partial parser 109.

[0024] The partial parser 109 includes a source code identification component 201. In an embodiment, the source code identification component 201 identifies a portion of source code (source code 111) from which the partial parser 109 will generate a code structure tree 112. In some embodiments, the source code 111 is a file stored on the storage medium 104 and / or obtained from the computer system(s) 108. In some embodiments, the source code 111 resides in the memory 103, such as corresponding to a buffer within a code editor corresponding to the source code consumer 110. In an embodiment, the source code 111 includes multiple levels of indentation, which is due to the syntax of the language in which the source code 111 is written (e.g., in the case of Python), or due to human and / or editor conventions (e.g., in the case of C, C++, C#, JAVA, etc.).

[0025] Figure 3 An example 300 of a source code 301 including indentation is shown. Specifically, the source code 301 is a portion of a C# source code that defines a program that compares two integers and prints whether the integers are equal, or whether one integer is greater than or less than the other integer. In the example 300, the source code 301 includes multiple levels of indentation, which are indicated by arrows (i.e., ) intuitively indicates the indentation of each level. In an embodiment, the indentation of each level is defined using one or more tab characters, one or more space characters, or a combination of tab characters and space characters.

[0026] Partial parser 109 also includes tree generation component 202.In an embodiment, tree generation component 202 is based on using the indentation included in source code 111 to "partially" parse, generate code structure tree 112, this code structure tree 112 is the ordered tree that captures the overall structure of source code 111 obtained by source code identification component 201.As those of ordinary skill in the art will appreciate, tree generation component 202 can represent code structure tree 112 in various ways.For example, using object-oriented method, each node is the object (for example, dedicated memory allocation) with the pointer pointing to its child node array.More compact method can use integer array and bit packing (bit-packing) to represent the child node of given node.Other methods are also possible, and within the scope of the present disclosure.Tree generation component 202 comprises indentation level identification component 203, node generation component 204 and node arrangement component 205.

[0027] In an embodiment, the indentation level identification component 203 identifies the indentation level of each line in the source code 111 obtained by the source code identification component 201. In an embodiment, for a blank line (e.g., line 03 in the source code 301), the indentation level identification component 203 uses the indentation level of its nearest non-blank ancestor (e.g., line 02 in the source code 301). Referring to the source code 301, in one example, the indentation level identification component 203 identifies the indentation level, as shown in Table 1:

[0028]

[0029]

[0030] Table 1

[0031] In an embodiment, the node generation component 204 generates a node corresponding to each line in the source code 111 obtained by the source code identification component 201, and the node arrangement component 205 arranges these nodes into an ordered tree structure (code structure tree 112) based on the indentation level identified by the indentation level identification component 203 and based on the order of the lines in the source code 111.

[0032] Figure 4A An example 400a of generating an ordered code structure tree (e.g., code structure tree 112) based on source code indentation is shown. It is worth noting that example 400a is only an illustrative example, and it should be understood that an alternative code structure tree can be generated based on the indentation of source code 301 without departing from the principles described and claimed herein.

[0033] Example 400a includes node 450a, which is a virtual root node of an ordered code structure tree. In an embodiment, node 450a generally represents or otherwise corresponds to source code 111. For example, node 450a can refer to a source code file stored on storage medium 104, a buffer within source code consumer 110, etc. Below node 450a, example 400a includes a hierarchy of nodes, which includes node 401 (corresponding to line 01 of source code 301), node 402 (corresponding to line 02 of source code 301), and so on to node 424 (corresponding to line 24 of source code 301).

[0034] In example 400a, the arrangement of the hierarchical structure of nodes within the ordered code structure tree is based on the indentation level of each line identified in Table 1, and based on the order in which the lines appear in the source code 301. Generally, if the indentation level of a line within the source code 301 is greater than that of its previous line, the node corresponding to the line is arranged as a child node of the node corresponding to the previous line. In addition, generally, if a line has the same indentation level as its previous line within the source code 301, the node corresponding to the line is arranged as a brother node of the node corresponding to the previous line. In addition, generally, if the indentation level of a line within the source code 301 is lower than that of its previous line, the node corresponding to the line is arranged as a brother node of the node corresponding to the nearest line with a lower indentation level. It is worth noting that in addition to these general rules and special cases, there may be exceptions. For example, there may be situations such as "offside indentation", in which the indentation level of a given line is reduced compared to its previous line, but this reduced indentation level is still higher than the ancestor of the previous line. This situation often occurs when specifying parameters for function declarations. For example, in python:

[0035]

[0036] Here, line 04 has a lower indentation level than lines 02 and 03, but higher than line 01. In this case, the node corresponding to line 04 can be arranged as a sibling node of the nodes corresponding to lines 02 and 03 (which can all be arranged as children of the node corresponding to line 01). There may be other options, such as introducing a dummy node for each indentation level that appears below line 01.

[0037] In example 400a, node 401 (line 01, comment), node 402 (line 02, comment), node 403 (line 03, blank line), and node 404 (line 04, namespace definition), node 405 (line 05, left bracket associated with namespace definition), and node 424 (line 24, right bracket associated with namespace definition), each node has an indentation level of level zero and is arranged as a child node (and sibling node) of node 450a. Node 406 (line 06, class definition), node 407 (line 07, left bracket associated with class definition), and node 423 (line 23, right bracket associated with class definition), each node has an indentation level of level one and is arranged as a child node (and sibling node) of node 405. Node 408 (line 08, Main method definition), node 409 (line 09, left bracket associated with Main method declaration), node 411 (line 11, right bracket associated with Main method declaration), node 412 (line 12, NumberCompare method declaration), node 415 (line 15, left bracket associated with NumberCompare method declaration), and node 422 (line 22, right bracket associated with NumberCompare method declaration), each node has an indentation level of two and is arranged as a child node (and sibling node) of node 407. Node 410 (line 10, function call to NumberCompare) has an indentation level of three and is arranged as a child node of node 409. Node 413 (line 13, first parameter of NumberCompare method declaration) and node 414 (line 14, second parameter of NumberCompare method declaration) also have an indentation level of three and are arranged as child nodes (and sibling nodes) of node 412. Node 416 (line 16, if statement), node 418 (line 18, else statement), and node 420 (line 20, else statement) also have an indentation level of three and are arranged as child nodes (and sibling nodes) of node 415. Finally, node 417 (line 17, body of if statement), node 419 (line 19, body of else statement), and node 421 (line 21, body of else statement) have an indentation level of four and are arranged as child nodes of node 416, node 418, and node 420, respectively.

[0038] It is noteworthy that the ordered code structure tree shown in Example 400a is based on the indentation and order of lines within the source code 301, without regard to the grammar / syntax within the source code 301. Nevertheless, the ordered code structure tree shown in Example 400a generally corresponds to the structure of the source code 301. For example, the line defining the namespace declared at line 04 generally corresponds to the descendants of node 405, the line defining the class declared at line 06 generally corresponds to the descendants of node 407, the line defining the Main method declared at line 08 generally corresponds to the descendants of node 409, and so on. Therefore, even in this form, the ordered code structure tree shown in Example 400a is useful for inferring the overall structure of the source code 301.

[0039] The tree generation component 202 also includes a syntax refinement component 206. In an embodiment, the syntax refinement component 206 further refines the ordered code structure tree generated by the node arrangement component 205 using the partial code syntax derived from the source code 301. In an embodiment, the syntax refinement component 206 utilizes syntax that is common in various programming languages, thus avoiding the need to rely on a detailed understanding of the syntax of a specific programming language.

[0040] In a first embodiment, the syntax refinement component 206 marks nodes to identify common source code constructs, such as classes or functions. For example, referring to example 400a, the syntax refinement component 206 can mark node 404 as declaring a namespace, node 406 as declaring a class, node 408 as declaring a method, and so on.

[0041] In a second embodiment, which may be performed alone or in combination with the first embodiment, the syntax refinement component 206 regroups nodes into clauses of their parent nodes, such as based on common "continuation" words or symbols. As an example, continuation words may include control flow keywords of the same level (e.g., "then", "else", "elseif", "elif", "catch", "except", "finally", "ensure"), end keywords (e.g., "fi", "esac", "done", "end", "endif", "loop", "until", "where", "when"), etc. As an example, continuation symbols may include bracket control symbols (e.g., '{', '}', '[', ']', '(', ')'), multi-line string endings (e.g., `"""`, """"'), multi-line comment endings (e.g., '* / ', '`'), etc. In an embodiment, a common feature of continuation words and continuation symbols is that even if they are syntactically subordinate to the previous line, they usually appear at the same indentation level as the previous line.

[0042] In an example, an if…else block is usually written in the following form:

[0043] if(condition)

[0044] action

[0045] else

[0046] action

[0047] Here, the 'else' keyword is at the same indentation level as the 'if' keyword, although it is syntactically subordinate to the 'if' keyword.

[0048] In another example, a multi-line comment can be written in the following form:

[0049] / / Comment line 1

[0050] / / Comment line 2

[0051] Here, the ' / / ' symbols on the second line have the same indentation level as the ' / / ' symbols on the first line, even though the two lines operate together as a single multi-line comment.

[0052] In this second embodiment, syntax refinement parts 206 define the set of continuation words and / or symbols, and use these continuation words and / or symbols to the node arrangement parts 205 based on the orderly code structure tree that generates of indentation and further structure is applied.In an embodiment, syntax refinement parts 206 are by identification the row that begins with continuation words and / or symbols, and realize this point by giving this row and its next-door neighbor the same indentation level of the previous ancestor.In certain embodiments, syntax refinement parts 206 use regular expression to define the set of continuation words and / or symbols, to increase the applicability of these continuation words and / or symbols to various programming languages.

[0053] In a third embodiment that can be performed alone or in combination with the first embodiment or the second embodiment, the syntax refinement component 206 regroups nodes based on inserting virtual nodes into the code structure tree. As an example, the syntax refinement component 206 can use virtual nodes to group nodes representing lines that belong together logically, such as by grouping a sigil with its annotated function declaration, or by grouping a function declaration with its parameters when these parameters are defined using a multi-line format (e.g., as shown in lines 12-14 of the source code 301).

[0054] In an embodiment, the syntax refinement component 206 operates in an iterative multi-pass manner, with each pass refining the code structure tree, and any number of passes may be used to repeat a given refinement type.

[0055] Figure 4BIt shows that it can be applied to Figure 4A Example 400b of possible syntactic refinements of an ordered code structure tree. Specifically, example 400b shows that node 402 can be regrouped as a child node of node 401 (e.g., based on a ' / / ' continuation symbol), node 405 and / or node 424 can be regrouped as a child node of node 404 (e.g., based on a start and / or end bracket control symbol), node 407 and / or node 423 can be regrouped as a child node of node 406 (e.g., based on a start and / or end bracket control symbol), node 409 and / or node 411 can be regrouped as a child node of node 408 (e.g., based on a start and / or end bracket control symbol), node 415 and / or node 422 can be regrouped to be associated with node 412 (e.g., based on a start and / or end bracket control symbol), and node 418 and / or node 420 can be regrouped as a child node of node 416 (e.g., based on the same level of control flow using the "else" keyword). Example 400 b also shows that node 412 , node 413 , and node 414 may be grouped under a new virtual node because node 413 and node 414 correspond to lines that define parameters of the function declared at the line corresponding to node 412 .

[0056] Figure 4C The example 400c of the thinning code structure tree based on the syntactic thinning shown in the example 400b is shown.It should be noted that the thinning code structure tree comprises node 450b, and this node 450b is the new virtual node to node 412, node 413 and node 414 groupings.In addition, the thinning code structure tree has reflected the additional dependency change described in conjunction with example 400b.When the code structure tree of example 400a is compared with the thinning code structure tree of example 400c, it can be observed that the thinning code structure tree in the example 400c is closer to the structure of source code 301.Therefore, the thinning code structure tree of example 400c provides more insights into source code 301, while still avoiding the specific parsing of complete grammar to source program 301.

[0057] The partial parser 109 includes a subtree identification component 207. In an embodiment, the subtree identification component 207 identifies a subtree of the code structure tree 112, and a corresponding block of source code lines covering the requested source code line, which facilitates the code structure tree (e.g., code structure tree 122) generated by the source code consumer 110 to be used as part of a source code action.

[0058] Now combine Figure 5 To describe the components of the partial parser 109, Figure 5A flow chart of an example method 500 for inferring source code structure based on indentation is shown. In an embodiment, instructions for implementing method 500 are encoded as computer executable instructions (e.g., partial parser 109) stored on a computer storage medium (e.g., storage medium 104), which can be executed by a processor (e.g., processor 102) to cause a computer system (e.g., computer system 101) to perform method 500.

[0059] The following discussion now involves a number of methods and method actions. Although method actions may be discussed in a certain order or may be shown in a flowchart as occurring in a specific order, unless specifically stated or required, no specific order is required because an action depends on another action being completed before the action is performed.

[0060] refer to Figure 5 In an embodiment, method 500 includes an act 501 of identifying a source code including a plurality of lines. In an embodiment, act 501 includes identifying a portion of the source code including the plurality of lines. In the example, source code identification component 201 identifies source code 111, where source code 301 is provided as an example. As mentioned above, in an embodiment, source code 111 is a file stored on storage medium 104 and / or obtained from computer system(s) 108, or resides as a buffer in memory 103 (e.g., a buffer within a code editor corresponding to source code consumer 110).

[0061] Method 500 also includes an action 502 of generating a tree representing source code using indentation. As shown, action 502 includes an action 503 of determining the indentation level of a plurality of rows; and an action 504 of generating a tree node representing a plurality of rows; and an action 505 of arranging the nodes into a tree using the indentation level. As shown, no order is required between action 503, action 504, and action 505. In some embodiments, these actions are performed continuously (e.g., action 503, then action 504, then action 505). In other embodiments, these actions are performed at least partially in parallel.

[0062] In an embodiment, action 503 comprises determining the corresponding indentation level for each row in a plurality of rows. In an example, the indentation level of each row in the indentation level identification component 203 identification source code 301. An example of the potential indentation level of the row of source code 301 of this paper has been presented in Table 1. As mentioned above, in an embodiment, blank lines (for example, the row that does not contain characters at all, or the row that only contains blank characters) are grouped with their nearest non-blank ancestors. Therefore, in some embodiments of action 503, determining the corresponding indentation level of each blank line in a plurality of rows comprises determining that each blank line has the same indentation level as the nearest non-blank line in this part of the source code.

[0063] In an embodiment, action 504 includes generating a plurality of nodes, each node representing a different row in the plurality of rows. Figure 4A As shown, the node generation component 204 generates node 401 (e.g., corresponding to line 01 of the source code 301), node 402 (e.g., corresponding to line 02 of the source code 301), and so on to node 424 (e.g., corresponding to line 24 of the source code 301) from the source code 301.

[0064] In an embodiment, action 505 comprises arranging the plurality of nodes into an ordered tree based at least on the indentation level corresponding to each of the plurality of lines. In the example, the node arrangement component 205 arranges the nodes generated in action 504 into the ordered code structure tree described in conjunction with example 400a.

[0065] As mentioned above, example 400a includes node 450a. In an embodiment, node 450a is a virtual root node of a code structure tree, and it generally represents or corresponds to a source code. Therefore, in some embodiments of method 500, the root node of the ordered tree corresponds to a portion of the source code, and the set of child nodes of the root node corresponds to a row with the lowest indentation level in a plurality of rows.

[0066] As mentioned above, if a line within the source code 301 has a greater indentation level than its previous line, the node corresponding to the line is arranged as a child node of the node corresponding to the previous line. Therefore, in an embodiment, arranging the plurality of nodes into an ordered tree includes arranging a first node representing a first line of the plurality of lines as a parent node of a second node representing a second line of the plurality of lines based on the second line being after the first line within the plurality of lines and based on the second line having a greater indentation level than the first line.

[0067] As also mentioned above, if a line within source code 301 has the same indentation level as its previous line, the node corresponding to the line is arranged as a sibling node of the node corresponding to the previous line. Therefore, in an embodiment, arranging the plurality of nodes into an ordered tree includes arranging a first node representing a first line in the plurality of lines as a sibling node of a second node representing a second line in the plurality of lines based on the second line being after the first line in the plurality of lines and based on the second line having the same indentation level as the first line.

[0068] As also mentioned above, if a row in source code 301 has a lower indentation level than its previous row, the node corresponding to the row can be arranged as a brother node of a node corresponding to the nearest row with a lower indentation level. Therefore, in an embodiment, arranging multiple nodes into an ordered tree can include arranging a first node representing a first row in multiple rows as a brother node of a second node representing a second row in multiple rows based on the second row being before the first row in multiple rows, and based on the second row having the same indentation level as the first row. However, there may be exceptions, such as the offside indentation example discussed above.

[0069] In some embodiments, method 500 also includes the action 506 of using one or more syntax scans to refine the tree.As mentioned above, in a first embodiment, syntax refinement components 206 mark nodes to identify common source code structures, such as classes or functions.Therefore, in some embodiments, action 506 includes the current arrangement based on multiple nodes, and the code structure defined based on the row represented by the specific node marks the specific node in the multiple nodes.As mentioned above, syntax refinement components 206 can be operated in an iterative, multiple scanning mode, and the mode can use any number of scans to repeat a given refinement type.Therefore, in some embodiments, marking the specific node in the multiple nodes is also based on one or more other marks in the multiple nodes.

[0070] As mentioned above, in the second embodiment, the syntax refinement component 206 regroups the nodes into clauses of their parent nodes, such as based on a common "continue" word or symbol. Therefore, in some embodiments, action 506 additionally or alternatively includes identifying a first node and a second node within the ordered tree, wherein the first node and the second node are sibling nodes within the ordered tree; determining that the second line represented by the second node has a syntax dependency on the first line represented by the first node; and arranging the second node as a child node of the first node within the ordered tree based on the syntax dependency.

[0071] As mentioned above, in the third embodiment, the syntax refinement components 206 are based on inserting virtual nodes into the code structure tree to node regrouping. Therefore, in some embodiments, action 506 is additionally or alternatively included in creating a new virtual node in the ordered tree, and this virtual node does not correspond to the row of any source code; And one or more existing nodes are rearranged as the child node of the new virtual node. It should be noted that the creation of the new virtual node and the rearrangement of the existing node as the child node of the new virtual node can be performed by the syntax refinement components 206 in a single scan, or can be performed in multiple scans (for example, in a single scan, create a new virtual node, and in one or more scans, other nodes are rearranged as its child nodes). It should be noted that the syntax refinement components 206 can mark this new virtual node and normal nodes.

[0072] In an embodiment, method 500 includes source code consumer 110 using generated code structure tree (e.g., code structure tree 112) as part of source code actions. In an embodiment, these source code actions include using subtree identification component 207 to identify blocks of multiple source code lines that are contextually related to the identified source code lines. In an embodiment, based on the identified source code lines, subtree identification component 207 identifies blocks of source code lines corresponding to subtrees within code structure tree 112, which includes nodes corresponding to the identified source code lines. For example, given source code lines within a function, subtree identification component 207 identifies subtrees (and corresponding source code lines) of nodes within code structure tree 112 that correspond to the function. The nature of source code consumer 110 and source code actions can vary widely.

[0073] In one example, the code editor utilizes the code structure tree 112 generated by the partial parser 109 to provide intelligent source code completion suggestions based on an understanding of the overall structure of the written code. For example, the code editor can utilize a language generation artificial intelligence (AI) model to generate code suggestions at a given hint location within the editor. In this example, the code structure tree 112 generated by the partial parser 109 can be used to identify code blocks to be used as hints for the language generation AI model.

[0074] In another example, the code structure tree 112 generated by the partial parser 109 can be used to facilitate the "code folding" of the user interface feature, which allows connected lines of code to be hidden under a single heading. In this example, recursive folding (e.g., a fold can contain sub-folds) can be supported using the code structure tree 112.

[0075] In another example, the code structure tree 122 generated by the partial parser 109 may be used to facilitate "difference" viewing by using the code structure tree 112 to help guide how much code context to show around a modified code snippet, thereby showing a coherent block of code that makes sense to the reader.

[0076] In another example, the code structure tree 112 generated by the partial parser 109 can be used to search for related code snippets in a file, such as by using the code structure tree 112 to select (possibly several) contexts around the cursor line, and / or to select coherent code blocks in other files.

[0077] According to the above, method 500 also includes an action 507 of using a tree to identify a line that is syntactically related to the subject line. In an embodiment, action 507 includes identifying a subtree from the ordered tree based on an identified line in the plurality of lines, the subtree including nodes representing the identified line; identifying a subset of the plurality of nodes corresponding to the subtree; and identifying a subset of the plurality of lines represented by the subset of the plurality of nodes as being syntactically related to the identified line. In the example, subtree identification component 207 receives line 19 of source code 301 as a line of interest. Then, subtree identification component 207 uses Figure 4C The illustrated code structure tree 112 identifies a subtree (and corresponding lines) that includes a node corresponding to a line of interest (e.g., node 419). For example, a subtree may include node 416 and descendants (e.g., the entire if block, including lines 16 to 21), node 450b and descendants (e.g., the entire NumberCompare method, including lines 12 to 22), and so on.

[0078] Method 500 also includes an action 508 of initiating a source code action using the identified lines. In an embodiment, action 508 includes initiating a source code action based on a subset of the plurality of lines. In view of the examples provided above, in various embodiments, action 508 may include subtree identification component 207 and / or source code consumer 110 initiating a source code action, the source code action including inserting a subset of the plurality of lines into a language generation model hint; initiating a code folding action for the subset of the plurality of lines within a code editor interface; visually displaying the subset of the plurality of lines as contextually relevant to the identified lines; etc.

[0079] Embodiments of the present disclosure may include or utilize a dedicated or general-purpose computer system (e.g., computer system 101) that includes computer hardware, such as, as discussed in more detail below, one or more processors (e.g., processor 102) and system memory (e.g., memory 103). Embodiments within the scope of the present disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media can be any available media that can be accessed by a general-purpose or dedicated computer system. Computer-readable media that store computer-executable instructions and / or data structures are computer storage media (e.g., storage media 104). Computer-readable media that carry computer-executable instructions and / or data structures are transmission media. Thus, for example, embodiments of the present disclosure may include at least two distinct computer-readable media: computer storage media and transmission media.

[0080] Computer storage media are physical storage media that store computer executable instructions and / or data structures. Physical storage media include computer hardware, such as random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), solid-state drive (SSD), flash memory, phase change memory (PCM), optical disk storage, magnetic disk storage or other magnetic storage devices, or any other (multiple) hardware storage devices that can be used to store program code in the form of computer executable instructions or data structures that can be accessed and executed by a general or special purpose computer system to implement the disclosed functionality.

[0081] Transmission media can include networks and / or data links, which can be used to carry program codes in the form of computer executable instructions or data structures, and can be accessed by general or special computer systems."Network" is defined as one or more data links that can transmit electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer system through a network or another communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computer system can regard the connection as a transmission medium. The above combination should also be included in the scope of computer-readable media.

[0082] In addition, upon reaching various computer system components, program code in the form of computer executable instructions or data structures can be automatically transferred from transmission media to computer storage media (or vice versa). For example, computer executable instructions or data structures received over a network or data link can be cached in RAM within a network interface module (e.g., network interface 105), and then ultimately transferred to computer system RAM and / or less volatile computer storage media at the computer system. Therefore, it should be understood that computer system components that also (or even primarily) utilize transmission media can include computer storage media.

[0083] Computer executable instructions include, for example, instructions and data that, when executed on one or more processors, cause a general purpose computer system, a special purpose computer system, or a special purpose processing device to perform a certain function or combination of functions. Computer executable instructions may be, for example, binary, intermediate format instructions such as assembly language, or even source code.

[0084] It should be understood that the disclosed system and method can be put into practice in a network computing environment with many types of computer system configurations, including personal computers, desktop computers, notebook computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc. The embodiments of the present disclosure can also be put into practice in a distributed system environment, wherein the local and remote computer systems linked by a network (by a hardwired data link, a wireless data link, or by a combination of a hardwired and wireless data link) all perform tasks. Therefore, in a distributed system environment, a computer system can include multiple component computer systems. In a distributed system environment, a program module can be located in a local and remote memory storage device.

[0085] It will also be understood that embodiments of the present disclosure can be practiced in a cloud computing environment. A cloud computing environment can be distributed, although this is not required. When distributed, a cloud computing environment can be distributed internationally within an organization and / or have components owned across multiple organizations. In this specification and the following claims, "cloud computing" is defined as a model for implementing on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). A cloud computing model can consist of various features, such as on-demand self-service, extensive network access, resource pools, rapid elasticity, measurable services, etc. Cloud computing models can also appear in the form of various service models, such as software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). Cloud computing models can also be deployed using different deployment models, such as private clouds, community clouds, public clouds, hybrid clouds, etc.

[0086] Some embodiments, such as a cloud computing environment, may include a system that includes one or more hosts, each of which is capable of running one or more virtual machines. During operation, the virtual machine simulates an operating computing system, supports an operating system, and may also support one or more other applications. In some embodiments, each host includes a hypervisor that simulates the virtual resources of the virtual machine using physical resources extracted from the virtual machine's view. The hypervisor also provides appropriate isolation between virtual machines. Thus, from the perspective of any given virtual machine, the hypervisor provides the illusion that the virtual machine is interfacing with physical resources, even if the virtual machine is only interfacing with the surface (e.g., virtual resources) of the physical resources. Examples of physical resources include processing capacity, memory, disk space, network bandwidth, media drives, etc.

[0087] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features described above or the acts described above, or to the order of the acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.

[0088] The present disclosure may be embodied in other specific forms without departing from its essential features. The described embodiments are to be considered in all respects as illustrative only and not restrictive. All changes within the equivalent meaning and scope of the claims should be included within their scope.

[0089] When introducing elements in the appended claims, the articles "a", "an", "the", and "said" are intended to mean that there are one or more elements. The terms "comprising", "including", and "having" are intended to be inclusive and mean that in addition to the listed elements, there may be additional elements. Unless otherwise specified, the terms "set", "superset", and "subset" are intended to exclude empty sets, so a "set" is defined as a non-empty set, a "superset" is defined as a non-empty superset, and a "subset" is defined as a non-empty subset. Unless otherwise specified, the term "subset" excludes all of its supersets (i.e., the superset contains at least one item that is not included in the subset). Unless otherwise specified, a "superset" may include at least one additional element, and a "subset" may exclude at least one element.

Claims

1. A method implemented at a computer system for inferring source code based on indentation, the computer system comprising a processor, the method comprising: identifying a portion of source code comprising a plurality of lines; determining a corresponding indentation level for each of the plurality of lines; generating a plurality of nodes, each node of the plurality of nodes representing a different row of the plurality of rows; arranging the plurality of nodes into an ordered tree based at least on the corresponding indentation level for each of the plurality of rows; Based on an identified row in the plurality of rows: identifying from the ordered tree a subtree including a node representing the identified row; identifying a subset of the plurality of nodes corresponding to the subtree; as well as identifying a subset of the plurality of rows represented by the subset of the plurality of nodes as being syntactically related to the identified rows; as well as A source code action is initiated based on the subset of the plurality of lines.

2. The method of claim 1, wherein a root node of the ordered tree corresponds to the portion of the source code, and wherein the set of child nodes of the root node corresponds to lines having a lowest indentation level among the plurality of lines.

3. The method of claim 1 , wherein determining the corresponding indentation level for each blank line in the plurality of lines comprises: Each blank line is determined to have the same indentation level as the indentation level of a nearest non-blank line in the portion of the source code.

4. The method of claim 1 , wherein arranging the plurality of nodes into the ordered tree comprises: Arranging a first node representing the first row of the plurality of rows as a parent node of a second node representing the second row of the plurality of rows based on the second row being after the first row within the plurality of rows and based on the second row having a greater indentation level than the first row.

5. The method according to claim 1, further comprising: Based on a current arrangement of the plurality of nodes and based on a code structure defined by a row represented by the particular node, the particular node among the plurality of nodes is marked.

6. The method according to claim 1, further comprising: Identifying a first node and a second node in the ordered tree, wherein the first node and the second node are sibling nodes in the ordered tree; determining that a second row represented by the second node has a syntactic dependency on a first row represented by the first node; as well as Based on the syntactic dependency, the second node is arranged as a child node of the first node within the ordered tree.

7. The method according to claim 1, further comprising: Creating a new virtual node in the ordered tree, wherein the virtual node does not correspond to any line of the source code; as well as One or more existing nodes are rearranged as child nodes of the new virtual node.

8. The method of claim 1, wherein initiating the source code action comprises inserting the subset of the plurality of lines into a language generation model hint.

9. The method of claim 1, wherein initiating the source code action comprises initiating a code folding action on the subset of the plurality of lines within a code editor interface.

10. The method of claim 1, wherein initiating the source code action comprises visually displaying the subset of the plurality of lines as contextually associated with the identified line.

11. A computer system for inferring source code structure based on indentation, comprising: processor; as well as A computer storage medium storing computer executable instructions executable by the processor to cause the computer system to at least: Identify a portion of source code that includes multiple lines; determining a corresponding indentation level for each of the plurality of lines; generating a plurality of nodes, each node of the plurality of nodes representing a different row of the plurality of rows; arranging the plurality of nodes into an ordered tree based at least on the corresponding indentation level for each of the plurality of rows; Based on an identified row in the plurality of rows: identifying from the ordered tree a subtree including a node representing the identified row; identifying a subset of the plurality of nodes corresponding to the subtree; as well as identifying a subset of the plurality of rows represented by the subset of the plurality of nodes as being syntactically related to the identified rows; as well as A source code action is initiated based on the subset of the plurality of lines.

12. The computer system of claim 11, wherein a root node of the ordered tree is associated with the portion of the source code, and wherein the set of child nodes of the root node corresponds to lines having a lowest indentation level among the plurality of lines.

13. The computer system of claim 11, wherein determining the corresponding indentation level for each blank line in the plurality of lines comprises: Each blank line is determined to have the same indentation level as the indentation level of a nearest non-blank line in the portion of the source code.

14. The computer system of claim 11, wherein arranging the plurality of nodes into the ordered tree comprises: Arranging a first node representing the first row of the plurality of rows as a parent node of a second node representing the second row of the plurality of rows based on the second row being after the first row within the plurality of rows and based on the second row having a greater indentation level than the first row.

15. The computer system of claim 11, wherein the computer executable instructions further comprise instructions executable by the processor to cause the computer system to: Based on a current arrangement of the plurality of nodes and based on a code structure defined by a row represented by the particular node, the particular node among the plurality of nodes is marked.

Images