Methods and systems for identifying a code portion
Patent Information
- Application Number
- AU2025253523
- Authority / Receiving Office
- AU · AU
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-04-09
- Filing Date
- 2025-04-09
- Publication Date
- 2026-10-08
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
RELATED APPLICATION
[0001] This Application claims the benefit of U.S. Provisional Application No. 63 / 631,722, filed on April 9, 2024. The entire teachings of the above Application are incorporated herein by reference. BACKGROUND
[0002] Static analysis may include automatically analyzing computer program characteristics, e.g., correctness, robustness, and safety, without executing the programs. SUMMARY
[0003] A commonly used language to describe a range of static analysis applications is Datalog. Historically, restricted subsets of the Datalog declarative logic programming language have been used for efficiently solving a specialized set of problems. Linear Datalog, a restricted form of Datalog that is computable in nondeterministic logarithmic (NL)-space, was shown to be equivalent to a class of constraint-satisfaction problems. Symmetric Datalog, a further-restricted form of Linear Datalog that is computable in log-space, has also been utilized in the context of efficiently-computable constraint-satisfaction problems.
[0004] In the field of customizing static analysis, lint static analysis tools have recently been employed as components of popular programming languages. Many such tools were initially used to discover syntactic stylistic code issues, but their use has since expanded into analyzing semantic properties and discovering performance, safety, and security bugs. For instance, ESLint®, PyLint, FindBugs, CppCheck™, and StaticCheck are tools for the JavaScript®, Python, Java, C++, and Go programming languages, respectively, that currently include dataflow and other analyses as part of their rules. Such tools are typically built on top of a common analysis library, but they still require a rule author to write code that performs an actual check. The tools thus only benefit from reusing the code of the underlying analysis.
[0005] A programming language of embodiments, which may be referred to herein as StarLang, may be sufficiently expressible to encode rules of such conventional tools and may have a large portion of the tools’ checks already encoded as Snyk® Code quality checks.
[0006] Domain specific languages (DSLs) for rules have a number of advantages over manually implemented analyzers, including, among other advantages, sharing computation between different rules, better validation of the rules, and testing tools. Many existing DSL tools operate on top of graphs to enable debugging and visualization. For example, IncA and CxQL by Checkmarx® perform matching based on graph patterns. PidginQL is a query language that matches on graphs and focuses on enforcing security properties; it allegedly performs matching in real-world programs within 15 seconds. Other examples of such DSL tools are SemGrep® and CodeQL®. According to a user study for fluentTQL, none of the foregoing systems is user-friendly for describing complex security properties such as taint analysis, whereas fluenfTQL is focused on such taint rules. While all the foregoing tools can perform various types of matching on code, none offers time complexity guarantees. Embodiments address the foregoing and other problems in existing methods and systems.
[0007] An example embodiment is directed to a computer-implemented method for identifying a code portion, within a codebase, exhibiting a selected behavior. The method begins by receiving a logical query associated with a codebase. The logical query indicates a selected behavior and is constructed in a logical query grammar. The logical query grammar is: (1) configured to represent logical queries each having (i) a unary predicate including a head variable, (ii) one or more predicate citations associated with the head variable, and (iii) one or more predicate citations associated with one or more other variables different from the head variable; (2) configured with variable name reuse; (3) configured to represent at least one logical query having one or more conjoined citations associated with (i) a given head variable and (ii) exactly one other variable different from the given head variable; and (4) configured with stratified negation. Based on the received logical query, the method then searches a logical graph representing the codebase to generate a query result indicating a presence of a code portion, within the codebase, exhibiting the selected behavior.
[0008] According to an example embodiment, the method may further include receiving a user query associated with the codebase. The user query may indicate the selected behavior and be constructed in a user query grammar. The user query grammar may be configured to represent user queries each having at least one user query citation, where the at least one user query citation may include at least one of a user query predicate, a user query template, and a literal value. In one such embodiment, receiving the logical query may include transforming the received user query into the logical query. According to another such embodiment, the method may further include processing the logical graph to generate an analysis result. The analysis result may include at least one indication of a match between (i) a given code portion, within the codebase, and (ii) at least one of a given user query predicate, a given user query template, and a given literal value. The method may further include, based on the received user query and the analysis result, generating an autocomplete suggestion for the received user query. The autocomplete suggestion may further include at least one of: (i) one or more user query predicates, (ii) one or more user query templates, and (iii) one or more literal values. In yet another such embodiment, the method may further include storing the analysis result in cache storage. According to one such embodiment, the generated autocomplete suggestion may include a plurality of suggestions. The method may further include ranking the plurality of suggestions.
[0009] In an example embodiment, the codebase may include one or more procedure calls and one or more dataflow paths. The logical graph may include one or more nodes corresponding to the one or more procedure calls and one or more edges corresponding to the one or more dataflow paths. According to one such embodiment, searching the logical graph may include identifying at least one node of the one or more nodes. The at least one node may correspond to at least one procedure call associated with the selected behavior. The query result may include an indication of the at least one node identified. In another such embodiment, the logical query grammar may be further configured to represent one or more logical queries having a binary predicate corresponding to a set of edges of the one or more edges. According to yet another such embodiment, the logical query grammar may be further configured to represent one or more logical queries having a negated binary predicate corresponding to a second set of edges of the one or more edges. The second set may be complementary to a first set of edges of the one or more edges. In one such embodiment, the logical graph may be a hypergraph. A given edge of the one or more edges may connect a given set of the one or more nodes. The logical query grammar may be further configured to represent one or more logical queries having a hypergraph predicate corresponding to a set of edges of the one or more edges.
[0010] An example embodiment may further include generating the logical graph by (1) based on the codebase, constructing at least one abstract syntax tree (AST) and (2) transforming the at least one AST constructed into the logical graph.
[0011] Another example embodiment is directed to a computer-based system for identifying a code portion, within a codebase, exhibiting a selected behavior. The system includes a processor and a memory with computer code instructions stored or held thereon. The processor and the memory, with the computer code instructions, are configured to cause the system to implement any embodiments, or combination of embodiments, described herein.
[0012] Yet another example embodiment is directed to a computer program product for identifying a code portion, within a codebase, exhibiting a selected behavior. The computer program product includes a non-transitory computer-readable medium with computer code instructions stored thereon. The computer code instructions are configured, when executed by a processor, to cause an apparatus associated with the processor to implement any embodiments, or combination of embodiments, described herein. As understood by one skilled in the art, one or more processors may execute the computer code instructions to cause the apparatus to implement an embodiment.
[0013] It is noted that embodiments of the method, system, and computer program product may be configured to implement any embodiments, or combination of embodiments, described herein. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The foregoing will be apparent from the following more particular description of example embodiments, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments.
[0015] FIG. 1 is a simplified block diagram of an example system for identifying a code portion according to an embodiment.
[0016] FIG. 2 is an example screenshot illustrating optional autocompletion functionality according to an embodiment.
[0017] FIG. 3 is a flow diagram of a computer-implemented method for identifying a code portion according to an embodiment.
[0018] FIG. 4 is a schematic view of an example computer network in which embodiments may be implemented.
[0019] FIG. 5 is a block diagram illustrating an example embodiment of a computer node in the computer network of FIG. 4. DETAILED DESCRIPTION
[0020] A description of example embodiments follows.
[0021] Introduction
[0022] Static analysis is a growing application of software engineering, leading to a range of essential security tools, bug finding tools as well as software verification. The general idea of static analysis is that some properties of a program are validated without running the code, but instead by reasoning about the program’s possible behaviors. In many cases, static analysis tools are built from the ground up for verifying specific properties such as memory safety, concurrency properties, security properties, type safety or others based on different techniques such as abstract interpretation, shape analysis, and taint analysis, among others. In recent years there has been an increase of universal static analysis tools such as CodeQL®, SemGrep®, Snyk® Code, and SonarSource® that validate a range of properties together and allow customizing parts of the scanner to validate additional properties or “static analysis rules.”
[0023] This level of customization is usually achieved by unifying parts of the static analysis that are common, such as code parsing, value propagation, dataflow analysis, pointer analysis, typestate analysis, and taint analysis, and allowing the rules to be written using a DSL or calls to an internal library that provide the base analyses.
[0024] A commonly used language to describe a range of static analysis applications is Datalog. The language is not Turing complete, yet it can be useful for expressing a range of analyses, such as points-to, security analyses, and other analyses.
[0025] In Datalog, a program can be said to belong to monadic Datalog (MDL) if all the program’s rule heads are unary, i.e., have arity of 1 (one). An extension of Datalog that includes a limited form of negation may be called stratified negation, or DL-1. In turn, a subset of MDL with stratified negation may be referred to as MOL-1.
[0026] Unfortunately, Datalog is non-trivial to use, leading to analysis that is difficult to implement in a precise but performant way. As a result, Datalog is often not suitable for allowing end-users that are non-experts to write rules. This has led to some of the latest tools providing easier-to-use DSLs to describe their static analysis rules.
[0027] Embodiments make building custom static analysis tools much easier, e.g., for developers, while at the same time providing a convenient framework for, e.g., application security and static analysis experts. Embodiments provide a language called StarLang, which leverages Datalog, and which includes programs with a fast runtime by virtue of StarLang having low time complexity of its decision procedure(s). This feature of StarLang has the benefit that finding matches in code using StarLang, i.e., matching static analysis alarms, is fast and allows users to see code matches in real-time as they author a rule. Moreover, embodiments can provide optional autocompletion to help make writing rules even faster and easier.
[0028] Notably, in an embodiment, StarLang enables a wide range of useful real-world static analyses. One reason why this is possible and easy is a templates language feature that embodiments provide. Templates may enable hiding Datalog recursion while providing useful high-level abstractions, making queries easier to read while enforcing usage of a Datalog subset that is fast and efficient.
[0029] A StarLang programming language of embodiments may be a subset, e.g., a strict subset, of MDL-, which may include only logic programs that can be computed very efficiently, in terms of the programs’ data complexity. Based on StarLang’s efficiency and simplicity characteristics, embodiments provide an easy-to-use wrapper around StarLang named Codesearch, which may enable users to interactively construct static analysis queries, thereby significantly simplifying the j ob of, e.g., security professionals and static analysis authors. Examples of Codesearch’s usefulness for its purpose are provided herein.
[0030] Example 1
[0031] A demonstration of StarLang’s power is illustrated by an example embodiment that discovers read-after-close violations—i.e., cases of reading from a file after closing the file— using static analysis. Below is an example code snippet containing three statements that manipulate a file in a JavaScript®-like language: 1 let f = file(); f.close(); f.read();
[0032] In this example, an embodiment may search for a call to read () and follow a dataflow of a receiver object—i.e., f—of this call. As a convention, a receiver object may sometimes be called argument 0. Here, the following search query may be used: 1 CallExpression<"read"> and 2 HasArgO<DataFlowAf ter<Arg0 In<CallExpression< "close ">»>
[0033] Following the logic above, line 1 of the query matches the call expression to read () . Line 2 is a sequence of templates that describe another checked property—the receiver object f has a call to close () before read () . To perform this query, an embodiment, e.g., a Snyk® Code system, may build a graph that represents the dataflow of the queried code snippet. This graph may contain nodes for the call expressions, as well as edges for the dataflow in the code. Then, each of the StarLang queries or subqueries may return nodes in this graph that match their described properties. For example, CallExpression<"read"> may return every node in the graph that describes a call to read () . The other templates HasArgO, DataFlowAf ter, and ArgOIn may traverse edges in the graph, but may still return graph nodes that satisfy the given property, e.g., HasArgO matches on nodes that have receiver objects with the property given in the template. Some of these templates may also expand into recursive unary predicates, e.g., DataFlowAfter may match even if there are multiple dataflow steps (i.e., multiple edges) that need to be traversed in the dataflow graph.
[0034] Existing Alternative Approaches
[0035] A traditional alternative approach promoted by SemGrep® performs syntactic pattern matching to discover the pattern of Example 1. For instance, an expression such as $l.close(); ... $1. read () may be used to find if there is a variable (or other symbol) that had a sequence of close () and read () operations. Unfortunately, this approach is too syntactic in many cases. As opposed to StarLang, which may operate on semantic relations such as dataflow, a conventional syntactic matching approach (such as that implemented by SemGrep®) requires that a specific code pattern matches. As a result, a user needs to modify a rule to match semantically equivalent, but syntactically different code variants such as the following example snippet—which is semantically identical to Example 1 while having different syntax: 1 function func(param) { param.close (); } 2 let f = file(); func(f); f.read();
[0036] In contrast to existing approaches, embodiments, e.g., a Snyk® Code system, can correctly discover a read-after-close violation in the above snippet using its interprocedural analysis and the same, unmodified StarLang query from Example 1.
[0037] Example System Embodiment
[0038] FIG. 1 is a simplified block diagram of an example system 100 for identifying a code portion according to an embodiment. As shown in FIG. 1, the system 100 includes the following example components indicated by square or rectangular shapes: translator 110, optional autocomplete engine 120, solver 130, parser(s) 140a-140n, and analysis engine 150. The system 100 inputs (user code 114, user Codesearch query 102, and Codesearch STandard Library (STL) 160), intermediate results (partial query 104, StarLang query 106, suggestion(s) 108a-108n, ASTs 116a-l 16n, and analysis graph 118) and output (query result 112) are indicated in FIG. 1 by round or elliptical shapes.
[0039] In an embodiment, a user query 102 may serve as input to the translator 110. The translator 110 may transform the query 102 into a partial query 104 or a StarLang query 106. According to another embodiment, the partial query 104 may be provided as input to the autocomplete engine 120. In turn, the autocomplete engine 120 may optionally generate one or more suggestion(s) 108a-108n, e.g., suggestions 208a-208n (described hereinbelow with respect to FIG. 2), for the user query 102. According to an embodiment, the StarLang query 106 may be provided to the solver 130. The solver 130 may then optionally interface with the autocomplete engine 120 and / or generate a query result 112. In another embodiment, after optionally receiving the partial query 104, the autocomplete engine 120 may also interface with the solver 130 to generate the result 112.
[0040] Continuing with FIG. 1, according to an embodiment, user code 114 may serve as input to the parser(s) 140a-140n, which may transform the code 114 into one or more AST(s) 116a-116n In turn, the AST(s) 116a-l 16n may be provided as input to the analysis engine 150. The engine 150 may then generate an analysis graph 118. In an embodiment, the graph 118 may optionally be input to the autocomplete engine 120 for use in generating the suggestion(s) 108a-108n. According to another embodiment, the graph 118 may be additionally or alternately input to the solver 130 for use in generating the result 112.
[0041] Continuing again with FIG. 1, in an embodiment, node 160 on the border of the system 100 is a Codesearch STL, which may act as an application programming interface (API) to the underlying StarLang language of embodiments. For instance, the STL 160 may provide an interface for inputting the query 102 to the optional autocomplete engine 120 and / or the solver 130.
[0042] Overview of Datalog and Relational Algebra
[0043] Overview of Datalog
[0044] Datalog is a declarative logic programming language that is often used as a query language for deductive databases, and has been employed extensively for program analysis.
[0045] A Datalog program consists of a set of Horn clauses, called rules. Rules are often encoded as implications: u : - p, q, ..., t., where : - represents the implication arrow (<—), the commas (,) represent conjunctions (A), . is the expression delimiter, and u, p, q, etc are predicates Logically, a rule is read as: “if p, q, ..., t hold, then u holds”, u is commonly referred to as the head of the rule, while p, q, ..., t. is referred to as the rule body. An empty rule body is shorthand for a rule body that holds trivially, i.e., is simply true. Rules with empty bodies are referred to as facts, i.e., they always hold. Predicates are claims of some / / -ary relation holding for a given set of values, denoted using the standard notation: r (Ao, . . . ,X- / ), where each A is either a concrete value in, or a variable over the universe of relation r. Specific predicate instances are sometimes referred to as atoms.
[0046] Most variants of Datalog support the use of an equality predicate (=) over values in the same universe in rule bodies.
[0047] A Datalog engine executes the specification for a set of input relations and produces an output relation for a query. The input relations of a Datalog program are referred to as the Extensional Database (EDB), and are treated as a set of facts. The rules encoding the logic program are referred to as the Intensional Database (IDB).
[0048] Example 2: The following Datalog program P computes ancestor relations relative to an EDB: parent(X,Y) :- father(X,Y). parent(X,Y) :- mother(X,Y). ancestor(X,Y) :- parent(X,Y). ancestor(X,Y) :- parent(X,Z), ancestor (Z,Y) . The following EDB D may be used: father (john,mary) father(joe,kurt) mother(mary,joe) father(tine,kurt) Applying program P to EDB D produces facts such as ancestor (mary, joe), and ancestor(john,joe).
[0049] The basic variant of Datalog does not allow for negating any terms in any rules. While queries of this variant can be solved very efficiently, it has limited use: By disabling input negation altogether, the ability to detect the absence of database information is lost, which restricts the expressivity of the formalism to queries satisfying some monotonicity property. Extending Datalog to DL= extends its expressiveness, at a cost of higher computational complexity. A logic program is a valid program in DL= if it contains no rule that is defined recursively in terms of its own negation—a negative cycle. More formally, a program is a valid program in DL= if its IDB rules can be topologically sorted using dependency-based ordering as follows: Given two predicate symbols r andp, thenp < r if / ? appears in a rule body of r. If p appears negated in a rule body of r, then p<r.
[0050] Remark 1: The above definition enables non-negative recursion as follows: A rule r : - r. leads to a valid ordering r <r, but a rule r :- ~r. leads to an absurd ordering r <r.
[0051] Example 3: The rule p (X, Y) :- p(X,Z), ->q(Y, Z) . is a valid DL= rule—if q is not defined in terms of p.
[0052] A Datalog program belongs to MDL if all the program’s rule heads are unary, i.e., have arity of 1.
[0053] Example 4: The rule p (X, Y) :- q(X,Y), r (Y) . is not valid in MDL, because p (X, Y) is not unary. However, p (X) :- q(X,Y), r (Y) . is a valid MDL rule, because q is an extensional predicate.
[0054] Computational Complexity in Datalog
[0055] There are several ways to define the complexity of queries or logic programs. The methods for defining complexity differ in the parameters with respect to which the complexity is measured. The three common complexities in the context of Datalog are: a) Data Complexity: The complexity of evaluating a fixed query for variable database inputs. b) Program Complexity: The complexity of evaluating, on a fixed database instance, the various queries specifiable. c) Combined Complexity: The complexity of evaluating the various queries specifiable for variable database inputs.
[0056] In the setting of static program analysis, a logic program or query is often predetermined based on a type of analysis being performed and alarms (e.g., errors, bugs, security vulnerabilities, etc.) to be detected, and is thus usually considered constant. As such, computational complexity category a) above—Data Complexity—may be a relevant measure in embodiments.
[0057] Overview of Relational Algebra
[0058] Relational algebra uses algebraic structures both for modeling data and for defining queries on data with a well-founded semantics. In addition, relational algebra provides a foundation for relational databases. One purpose of using relational algebra in the relational database setting is to define operators that transform one or more input relations to an output relation. Embodiments may utilize relational algebra expressions for StarLang concepts such as constraints, rules, and queries, among other examples.
[0059] Relational Algebra Operators
[0060] Because relations are sets, standard set operations, e.g., all standard set operations, are applicable to relations. In addition to the standard set operations, e.g., union, intersection, and difference, the following operations on relations may also be defined and utilized by embodiments.
[0061] Selection: A selection operation may be denoted with op, wherep is a predicate. The element(s) of the relation to which the selection op is being applied may be the domain of the predicatep. A selection operation may be used to derive a sub-relation R' from a given relation R such that R' £ R and Wr ER', p(r).
[0062] Projection: A projection operation may be denoted with where i is an index of an entry in the tuples of a given relation R. A projection x, may be used to derive a unary relation R' from R such that Vr' G R', Br ER and such that the value of r', x, is the zth entry in r: r = Oo, • • • ,yt = x, ■ ■ ■ ,yn).
[0063] Natural Join: A natural join operation may be denoted with XI. A natural join X may be used to derive a relation from two given relations, where the relation contains the catenation of tuples that have a matching element at their respective ends. For instance, given relations Ri of ^-tuples and / L of / n-tuples, Ri X R2 = {(r0,..., ■ • •,pm-i)|(xo, • • , ^-i) 6 Ri R(yo, ■ • • , Tm-i) ERi Axh = yo}.
[0064] Embodiments may leverage the principle that each Datalog rule has an equivalent relational algebra expression. In some examples described herein, certain concepts, such as relation-joining, may be expressed using relational algebraic terms.
[0065] Overview of StarLang Embodiments
[0066] Presented herein is an example formal definition of a StarLang programming language of embodiments, as well as a description of example efficiency benefits of StarLang’s limited expressiveness.
[0067] In an embodiment, StarLang may be a subset, e.g., a strict subset, of MDL-1. Because StarLang may be an MDL, intensional rules of StarLang may define a unary predicate, i.e., may have a single variable in a head of the rule. Notably, according to another embodiment, StarLang may not include a standard explicit equality predicate. Instead, equality may be implicit in reuse of variable names.
[0068] StarLang rules may have a syntactic structure that is useful for purposes of presentation, and to simplify a process of defining rules.
[0069] Example Definition 1 of StarLang Rules
[0070] In an embodiment, example Definition 1 may define a Category I of valid StarLang rules. A valid StarLang rule r in Category I may have a non-limiting example form as follows: p (X) : - A, B, C.
[0071] where A, B, and C may be defined as follows: a) A may be a sequence of zero or more predicate citations, either negative or positive, as follows: ¢71 (X) , . . .,(X), where X may be a variable in a head of r. b) B may be an optional pair of conjoined citations. The conjunction, if it is present, may have either a form e (X, Y) , / (Y) or the form e (Y, X) , t (Y), where X may be a variable in a head of r and Y#X. t may be either a positive or a negative citation. c) C may be a sequence of zero or more predicate citations, either negative or positive, as follows: d\ (Zi) , . . .,4(2 / ), where Z;^X for 1 < i <j, and X may be a variable in a head of r.
[0072] In an embodiment, a further criterion may be imposed for negative citations, which may enforce that all negation is stratified: If q is cited negatively in a body ofp, then q itself may not be defined, directly or transitively, in terms ofp.
[0073] Example 5: In an embodiment, p, ei, and e2 may be extensional predicates. Then the following may be a valid StarLang program: r (X) :- ti (X) , p (X) , e2 (X, Z) , qi(Z) , d(U) . tUX) :- S1(X, Y) , t2 (Y) . t2 (Y) : - e2 (Y, W) , r (W) . qi (Z) :- e2(V, Z), ^p (V) .
[0074] Remark 2: In an embodiment, example Definition 1 may not include n-ary predicates for n > 2. Because StarLang may be a Datalog subset designed for static program analysis, input databases may in most cases describe graphs, i.e., nodes and edges. Binary predicates may describe edges between nodes in a graph, while larger arity predicates may not be commonly used to describe graph structures. According to another embodiment, an extension of StarLang may enable use of hypergraphs, i.e., graphs with hyperedges having n > 2, where n is a number of endpoints of an edge.
[0075] Remark 3: In an embodiment, example Definition 1 may not include negated binary predicate citation. Because binary predicates in StarLang may be used to represent directed graph edges, a negated binary predicate may represent a complementary set of edges in a graph. According to another embodiment, an extension of StarLang may enable use of edgepredicate negation.
[0076] In an embodiment, the citations in component C (described hereinabove in relation to example Definition 1) may be useful for expressing certain queries in StarLang. For instance, because of limitations on variable-name reuse within a single rule body, without component C, it may not be possible to express constraints on “disconnected” parts of an input database in a single query. The citations in component C may be referred to herein as disconnected citations.
[0077] In an embodiment, a StarLang programming language of embodiments may be designed to be used as part of a static-analysis engine, and thus EDB relations may represent forests of program analysis graphs.
[0078] In an embodiment, example Definition 1 may not include all valid StarLang rules. According to another embodiment, with respect to deriving equivalent rules via inlining, any rule derived by inlining a set of StarLang rules may itself be a valid StarLang rule—e.g., provided that proper care is taken around variable naming—due to semantic equivalence. Thus, in an embodiment, an alternative, more complete definition may be formulated inductively.
[0079] Example Definition 2 of StarLang Rules
[0080] In an embodiment, example Definition 2 may define a Category II of valid StarLang rules. According to another embodiment, StarLang rules in Category II may have the below non-limiting example forms: a) A monadic rule with an empty body, e.g., p (X) :- ., may be a valid rule. b) Given a valid rule r with head h and body b, and unary predicate p, then h : - b, p (X) . may be a valid rule. c) Given a valid rule r with head h and body b, and unary predicate p, then h : -b, —>p (X) . may be a valid rule, if a rule defining p does not depend on the predicate in h. i In an embodiment, including a stratifiability “check” in the above example may make this inductive definition not strictly syntactic in its current form. However, according to another embodiment, this definition can be extended to a program-rule pair (P, r), in which case a syntactic check may be as follows: verify that ifpEP, then r is not in a body of p or any of its dependencies. d) Given a valid rule r with head h and body b, and extensional binary predicate e, then h : - b, e (X, Y) . and / ? :- b, e (Y, X) . may be valid rules, if X is not a fresh variable, but Y is—i.e., X already appears in r but Y does not.
[0081] Remark 4: In an embodiment, Items a), b), and c) of example Definition 2 may encompass both components A and C of example Definition 1, while Items a) and d) encompass component B of example Definition 1. According to another embodiment, citation / (Y) of component B may always be defined with an empty rule body.
[0082] Example 6: In an embodiment, given intensional unary predicates ph extensional unary predicates / , and extensional binary predicates ek, then the following may be valid StarLang rules: a) r (X) :-. b) r(X) :- fi(X), Pi(X), f2(X), p2(X), p3(X), f3(Z). c) r(X) :- -ipi(X), fi(X), p2 (X) . i. In an embodiment, pi may not rely on r, either directly or transitively, d) r (X) :- -.p2 (X) , p2 (X) . i. In an embodiment, this rule may be trivially equivalent to false, while also being a valid StarLang rule. e) r (X) :- el (X, Y) , el (X, Z), e2 (Y, W) , r(W), e2 (V, Z) , p2 (V) .
[0083] According to another embodiment, the following may be invalid StarLang rules: a) r (X) : - e (Y, X) , -.r (Y) . i. In an embodiment, this may violate Item c) of example Definition 2 because r is negated in the body of r. b) r (X) : - e (Y, Z) . i. In an embodiment, this may violate Item d) of example Definition 2 because both Y and Z are fresh. c) r (X) :- e2 (X, Y) , e2 (X, Y) . i. In an embodiment, this may violate Item d) of example Definition 2 because neither Y nor X is fresh in e2.
[0084] Equivalence of Definition 1 and Definition 2
[0085] In an embodiment, either example Definition 1 or example Definition 2 can be used to derive the other definition. According to another embodiment, an MOL-1 rule r may also be a valid StarLang rule if the rule r can be transformed into a semantically-equivalent set of rules, each having a “simple” A, B, C structure. Such an example transformation procedure according to an embodiment is described herein. In an embodiment, an undirected graph may be constructed with labeled edges from a given rule, based on argument names used in citations. Unary citations with the same argument may be grouped together in the same node, and binary predicates may define labeled edges between these nodes—with the predicate name being the label. According to another embodiment, due to Item d) of example Definition 2, a graph structure may be a tree. In an embodiment, a separate rule may be derived for each node—structure A of example Definition 1—and for each edge—structure B of example Definition 1.
[0086] Example Procedure 1 for Transforming a Definition 2 Rule into a Definition 1 Rule
[0087] In an embodiment, r may be a rule being transformed, x may be a variable in a head of r, and B may be a set of predicate citations in a body of r. Given a set of nodes or vertices V= { U(y)\U £ B AVu 6 B : if u is unary with variable y then u 6 U} and a set of edges E = {b(y, z)\b 6 B A b is binary with variablesy, z}, then G = (V,E) may be a graph induced by the nodes V and the edges E. It is noted that G may not be a multi-graph, because Item d) of example Definition 2 may prohibit the existence of two binary citations with the same two variables.
[0088] According to another embodiment, if the set V contains no node for x, then a node U(x) = {} may be added to the set V.
[0089] In an embodiment, example Procedure 1 may utilize a depth-first search (DFS) to traverse the graph G starting at the node U(x\ i.e., the node for the variable x in the head of r. According to another embodiment, nodes of the graph G may induce rules tu(u) for variable u and with fresh predicate name tu. In an embodiment, labeled edges may induce rules s / v) for variable v and with fresh predicate name sv.
[0090] Example Pseudocode for Procedure 1
[0091] In an embodiment, as shown by the below example pseudocode, Procedure 1 may translate a rule r matching example Definition 2 into a rule matching example Definition 1 by DFS-traversing a graph G induced by r. procedure Translate!Graph O = {KE}, Entry point stacfe ;= V where V(r) fe at the top of the stack ro«lt := 0 :> Map of rule heads to rule bodies while siatA: is not empty do top := pop top: of stack if top has not been visited then p := Var(t^) resMtsy := () if sv is a rule in reauR then resaitis ; .ad ch i y) i else if y £ then > Handle disconnected rules resulty Lsddyip}) end if for Kb y) in edges of h?p do if Mb j) is incoming (i.e.. i / y) and i is already visited then skip is Do not add incoming edge rules twice end if y := < y if t = y else t} restdy,] := i&(y. sp w a it it s s(y j) push 1¾) onto stacfe end for end if end while return end procedure
[0092] Example 7: According to an embodiment, the following MDL-1 rule may match example Definition 2: r(X) :- el (X, Y) , ei (X, Z) , e2 (Y, W) , r (W) , e2 (V, Z), =P(V), d(U), e3(U, R).
[0093] In another embodiment, the above rule can be transformed into a semantically-equivalent set of rules structured as described hereinabove, according to the following example form: .-, [ E = [Xi = {}, D{w} = {r(v», Lav) = {^pOD j, &W = [a(u)}} d = 3.-. 1 .1 . , \ . , ,. . ... | J P 1 J ? £ J (,U s R ) J (. f , S'2 \ V J Z / t2(X) :- S1(X), s2(X), t5(U). S1 (X) :- e2 (X, Y) , s3 (Y) . s2 (X) :- e2 (X, Z), s4(Z) . s3 (Y) :- e2 (Y, W) , t2 (W) . s4 (Z) :- e2 (V, Z) , t3 (V) . t2(W) : - r(W) . t3(V) :- (V) . t5(U) :- d(U), s5(U). s5 (U) :- ei (U, R) . r (X) :- ti(X) .
[0094] Remark 5: In an embodiment, as Example 7 demonstrates, example Procedure 1 may generate redundant rules in many cases. This may be to keep a decision process of the procedure simpler for ease of explanation, but is not a requirement of StarLang embodiments.
[0095] According to another embodiment, example Procedure 1 described hereinabove may not produce correct results when starting from an MDL-1 rule that is not a valid StarLang rule according to example Definition 2, as the following example demonstrates.
[0096] Example 8: In an embodiment, the following MDL-1 rule may not match example Definition 2: r (X) :- ei (X, Y) , e2 (X, Y) . The rule cannot be transformed into a semantically-equivalent set of rules structured as described hereinabove. According to another embodiment, if example Procedure 1 is applied to this rule, the following set of rules may result: ti (X) :- si(X) , s2(X) . S1(X) :- ei(X, Y) . s2 (X) :- e2(X, Y) . r (X) :- ti(X) .
[0097] The above set of rules may not be equivalent to the original MOL-1 query, because the two instances of Y in Si and s2 are independent, which may lead to different instantiations of X in a query result when evaluating the transformed program compared to the original. For instance, given input of an EDB {ei(1,2) ., ei(3,2) ., e2 (1,2) ., e2 (3, 7) . then evaluating the original program on this input may result in X = {1}, while evaluating the transformed program may result in X = {1, 3}.
[0098] Assertion 1: In an embodiment, example Definitions 1 and 2 may be shown to be equivalent as described hereinbelow.
[0099] (Definition 1) £ (Definition 2): In an embodiment, given a rule r as described in example Definition 1, then it can be verified that r adheres to the structural rules laid out in example Definition 2.
[00100] (Definition 1) 2 (Definition 2): In an embodiment, given a rule r as defined in example Definition 2, then, using example Procedure 1, r can be transformed into an equivalent set of rules that adheres to the structure described in example Definition 1.
[00101] In an embodiment, establishing example Assertion 1 may rely on the correctness of example Procedure 1, i.e., that if an input is a valid StarLang rule according to example Definition 2, then example Procedure 1 outputs a semantically-equivalent set of valid StarLang rules according to example Definition 1. This may be proven as described hereinbelow.
[00102] Assertion 2: In an embodiment, given a valid StarLang rule according to example Definition 2, then example Procedure 1 may output a semantically-equivalent set of valid StarLang rules according to example Definition 1.
[00103] According to another embodiment, there may be four ways in which example Procedure 1 can violate example Assertion 2: (1) making dependent variable instances independent (as in Example 8), (2) changing a way that disconnected rules constrain a query, (3) incorrectly handling variable names, and (4) generating a disjunctive rule.
[00104] In an embodiment, because example Procedure 1 may not change any variables, Item (3) may not be an issue. Similarly, Item (4) may not be an issue, because example Procedure 1 may not generate disjunctions, e.g., as shown by the example pseudocode detailed hereinabove.
[00105] According to another embodiment, for Item (1), due to Item d) of example Definition 2, graph G of example Procedure 1 may be a forest of trees. Further, nodes in the graph may represent variables, and by Item d) of example Definition 2, exactly one of the variables in a binary predicate, i.e., edge, citation may be fresh. This means that it may be impossible for the graph G to contain two distinct paths between any two nodes.
[00106] Finally, in an embodiment, for Item (2), disconnected rules may constrain an entire rule—i.e., the disconnected rules may be global constraints—and, thus, evaluating the disconnected rules at a root of a rule sequence may be correct. Because example Procedure 1 may not generate disjunctive rules, adding disconnected rule constraints at any “level” of an output rule sequence may be valid.
[00107] Expressiveness of StarLang
[00108] According to an embodiment, as described hereinabove for Example 8, StarLang may be less expressive, e.g., strictly less expressive, than MDL-1:
[00109] Assertion 3: In an embodiment, MDL^ queries that establish equality between non-head variable operands of / / -ary predicates for n > 1 may not be expressible in StarLang.
[00110] Example 9: According to an embodiment, the following MDL-1 query may not be expressed in StarLang: Q = p (X) :- q(X,Y), r (X, Y) ., where q and r are EDB predicates. Q may be an invalid StarLang query, because it may violate Item d) of example Definition 2.
[00111] In an embodiment, a corollary of example Assertion 3 may be that queries such as “Is there a method m of object O that is called both behind a lock and not behind a lock?” cannot be expressed in StarLang. Despite this, StarLang may still be quite expressive. The foregoing example query may be useful to find potential concurrency bugs. For instance, because StarLang supports stratified negation, many queries can be expressed in StarLang that are not expressible in Semi-positive Datalog, the latter of which allows negation only in front of EDB relations. In contrast with Semi-positive Datalog, StarLang can support any query that includes negations of intensional predicates.
[00112] Example StarLang Templates
[00113] According to an embodiment, StarLang may include a syntactic feature called “Templates”. In another embodiment, StarLang templates may be somewhat similar to Lisp macros. For instance, each template invocation may represent a predicate that a compiler auto-generates with “holes” filled by arguments of the invocation. According to an embodiment, StarLang templates may enable both unbounded recursion and nested citation.
[00114] Example 10: In an embodiment, given two predicates p and q, the following StarLang template may generate a logic program equivalent to a statement p V (-,p A q): TEMPLATE tmpl (p, q) = t (X) :- p (X) . t (X) :- -.p (X) , q(X) . where t is a fresh predicate name.
[00115] Example 11: According to an embodiment, given a binary predicate e and a predicate p, the following StarLang template may generate a logic program that computes a transitive closure of e limited to paths consisting only of nodes for which p holds: TEMPLATE tmpl(p) = t(X) :- p(X), e(X, Y), tmpl(p). where t is a fresh predicate name.
[00116] In an embodiment, templates may not add expressiveness to StarLang. According to another embodiment, compilation of a templated logic program may terminate if and only if the expanded program is finite, in which case it could have been constructed manually. This termination may be the case even for template recursion, such as described hereinabove for Example 11. This termination may also be due to memorization—i.e., reusing a cache or “memo” of previously generated function results—in an implementation of an expansion procedure: once a hole has been “filled” with a given predicate, other instantiations of the same template with the same predicate may not need to be re-expanded.
[00117] According to an embodiment, although templates may not add expressiveness to StarLang, they may significantly improve the language’s usability by allowing users to define abstractions that lead to less code-duplication, thus reducing program size and simplifying program-writing.
[00118] Efficiency of StarLang
[00119] In an embodiment, StarLang evaluation can be accomplished in time polynomial in a size of an EDB, e.g., by using bottom-up, semi-naive evaluation. According to another embodiment, however, the structure of StarLang rules can be leveraged to achieve an even more efficient evaluation process.
[00120] According to an embodiment, a requirement in Item d) of example Definition 2 may be that one of the variables in any binary citation must be fresh. This may mean that constraints expressed using binary predicates are limited to the existence of elements or “paths” in natural joins, e.g., cumulative natural joins, of relations, without use of selection or projection operators.
[00121] Example 12: In an embodiment, given an input EDB of / ={ei(l,2) ., ei(4,3)., ei(2,4)., e2(l,3)., e2(4,2)., e2 (2,4 ) .}, it may be possible in StarLang to encode a constraint that there must be a “path” ei;e2 between two elements— i.e., there exists a tuple in a join of e2 and e2. For instance, a tuple (1,2,4) may be an element in such a join in / . The foregoing constraint may be expressed in relational algebraic terms as follows: Bx : x G (e2 X e2).
[00122] According to another embodiment, however, it may be impossible to encode a constraint that there are two elements that are connected by both e2 and e2—i.e., that there exists a tuple in a natural join of e2 and an inverse relation of e2 such that the first and last tuple elements are equal. For instance, StarLang may be unable to express a query predicated on the existence of pair (2,4), which may be connected by both e2 and e2 in I. In relational algebraic terms, the aforementioned constraint may be expressed as follows: Sx : x G o1=3(e1 X e2-1).
[00123] In an embodiment, constraints on joins of binary relations expressible in StarLang may effectively amount to the existence, or lack thereof, of transitive “paths”. According to another embodiment, however, answering transitivity queries correctly may not require actually computing an entire join. Instead, by a definition of transitivity, a path can be computed cumulatively, “one step at a time”. This may result in there being no need to fully compute joins at all when solving StarLang queries. Further, checking transitivity of relations may be considered as a sequence of selection and projection operations and single, independent joins between unary and binary relations. Thus, while more general-purpose conventional Datalog variants require complex runtime join optimizations or manual annotation of join order by users to achieve remotely-efficient query-solving, in an embodiment, StarLang queries may be efficient to compute by design.
[00124] Example 13: According to an embodiment, a StarLang query may be as follows: r(X) := eJX, Y) , e2 (Y, Z), e3(Z, W) , p (W) .
[00125] In relational algebraic terms, the above query may be expressed as Op(ei IX e2 XI e3). Alternatively, the same query may be expressed as Ki(ei X 7Ti(e2 X 7ti(e3 X p))). It is noted that in the latter form, joins may only occur between a binary relation and a unary relation, e.g., a result of a projection or the citation p. As a result, nesting may not be necessary—a projection may effectively “forget” a joined relation at each intermediate step, and so the steps can be considered to be in the following sequence: Ry := 7ti(e3 X p); / L := 7ii(e2 X Ax); R3 := 7Ti(ei X / L) where R3 is a result of a query.
[00126] In an embodiment, given a binary relation B, a unary relation U, and sorts a, / J, if U: and / ? : a, then joining / ? with U can be done in time 0(max(| / / |, | / ?|)). This may be due to B being indexed by its a values, e.g., if B is represented using a hash table. It is noted that indexing in the foregoing case can be done once, as a pre-processing step, because a binary relation being joined may always be extensional, i.e., an input relation. In contrast, performing a traditional computation instead with full joins between two relations B2 would be O(| / ?i| • |B2|) in the worst case. A traditional computation also generally cannot be efficiently computed ahead of time, e.g., in the case of joining temporary relations. Thus, in an embodiment, StarLang programs can be evaluated more efficiently than conventional MDL= programs.
[00127] Data Complexity of StarLang
[00128] According to an embodiment, given a StarLang program P, a finite input database I, and a number T of elements in an active domain of I, then, because StarLang programs may be stratifiable, computing P over / using a fixed-point approach may take at most T iterations. Furthermore, the process may terminate once no new tuple is added to an output relation, and there may be at most Ttuples that can be added. This may be the case because a StarLang query may be unary, and so a resulting output relation may only include singleton tuples, i.e., at most an entire active domain T of I. According to another embodiment, in each iteration of a fixed-point computation, at most |P| joins may be evaluated, e.g., exactly |P| if an entire program consists of citations of binary EDB relations. In an embodiment, querying a unary relation U can be done in time |t / |, and there may be at most |P| such queries.
[00129] In an embodiment, given m = max{| / ?||A E 1} and a size of the largest EDB relation, then indexing any EDB relation can be done in O(m) time. According to another embodiment, given a number k of EDB relations, data complexity of a StarLang program may be bound from above by 0(km + m\P\T)—km being time to index the input, and m\P\T being an upper bound on time to compute the solution, given said index. Because |P| may be taken as constant when considering data complexity, this may result in an upper bound of 0(m(k + 7)). Thus, in an embodiment, data complexity of StarLang may have a linear dependency on three dimensions of an EDB: (1) a size of the largest relation; (2) a number of relations; and (3) a number of elements in an active domain.
[00130] Overview of Codesearch Embodiments
[00131] In an embodiment, Codesearch may be a simplified interface for using StarLang, e.g., in a Snyk® Code system, that is provided to users, e.g., non-expert users, for defining custom semantic analysis rules and / or static analysis queries that are supported by an analysis engine, e.g., a Snyk® Code analysis engine. According to another embodiment, Codesearch syntax may be more streamlined compared to StarLang. For instance, users may define unnamed, standalone rules in Codesearch in a form similar to a single disjunct! on-of-conjunctions query. Such a query may contain invocations of pre-defined StarLang templates and citations of pre-defined predicates. The predicates and templates may be supplied in a form of an STL.
[00132] According to an embodiment, because Codesearch may serve as an API to StarLang, the abstraction power of templates makes them perfect to act as primary building blocks of a Codesearch STL. An example formal grammar of Codesearch according to an embodiment is provided hereinbelow as Appendix A, while an example description of definitions in a Codesearch STL according to another embodiment is provided hereinbelow as Appendix B.
[00133] In an embodiment, Codesearch may be very expressive, due to StarLang’s expressiveness in combination with a Codesearch STL’s extensiveness.
[00134] According to an embodiment, a Codesearch API and STL may be designed to be as language-agnostic as possible, thus making it easy to express abstract semantic concepts, while alleviating a need to consider syntactic structures and concrete code patterns.
[00135] Example 14: According to an embodiment, and following Example 1 described hereinabove, the below Codesearch query may match two semantically equivalent, but syntactically different code snippets. 1 CallExpression<"read"> and 2 HasArgCKDataFlowAf ter<ArgO In<CallExpression< "close ">»>
[00136] An example first matching code snippet may be as follows: 1 let f = file(); f.close(); f.read();
[00137] An example second, and semantically equivalent to the first, matching code snippet may be as follows: 1 function func(param) { param.close () ; } 2 let f = file(); func(f); f.read();
[00138] In an embodiment, as shown above by Example 14, Codesearch may enable users to focus on semantic concepts, such as dataflow or call receivers, while moving structural and syntactic concepts, such as variable names or scoping, to the backseat.
[00139] Example Optional Autocompletion Functionality
[00140] In an embodiment, to improve discoverability of STL definitions, and to assist users in making correct use of said definitions, a code editing tool according to embodiments, e.g., the Snyk® Code Codesearch editor, may include an optional autocompletion mechanism. According to another embodiment, an autocompletion mechanism may be context-sensitive, and may suggest the most relevant literals, predicates, and templates, as described hereinbelow with respect to FIG. 2. This context sensitivity may be achieved by performing some limited analysis of a code repository being queried, to approximate which parts of the code are likely to match for a given template, predicate, or literal.
[00141] According to an embodiment, an example autocompletion suggestion system may be context sensitive, while also being quite fast. This may be accomplished by performing a partial analysis up front and caching the results. In another embodiment, an analysis and cache may be associated with a single code base, and may not be shared among customers. This may be better for both customer privacy and data security, as well as for usability. In contrast to such an embodiment’s approach, populating suggestions based on an analysis of unrelated code repositories may simply introduce irrelevant noise to a list of suggestions.
[00142] FIG. 2 is an example screenshot 200 illustrating optional autocompletion functionality according to an embodiment. As shown in FIG. 2, example autocompletion suggestions 208a-208n are provided for an argument of a CallExpression template 222 in the context of the second example code snippet 214 from Example 14 above. The most highly-ranked suggestions 208a-208c are all names of functions 224a-224c that are called in the code snippet 214.
[00143] Example Case Studies
[00144] Presented herein are example case studies, in various programming languages, to demonstrate the expressiveness, conciseness, and simplicity of embodiments, e.g., Codesearch embodiments. The case studies also include a variety of useful queries expressible using Codesearch embodiments. As described herein, by employing Codesearch embodiments, static analysis queries can be expressed concisely and simply.
[00145] Example Identification of Taint Vulnerabilities
[00146] In an embodiment, taint-flow analysis may cover a very large category of security-related vulnerabilities. The following example C# code snippet demonstrates a SQL injection (SQLi) vulnerability on line 9, where a string coming from an HTTP request at lines 31 and 32 is used to insecurely build a query at lines 33, 24, and 19 that is executed on a SQL server at lines 9 and 10. 1 using Microsoft.AspNetCore.Mvc; 2 using System.Data .SqlClient; 3 4 public class DbHandler { 5 private const String CONNECTION STRING = "Server= myServerAddress;Database=myDataBase;UserId=myUsernam e;Password=myPassword;"; 6 7 public static String DoQuery(string query) { 8 using (SqlConnection connection = new SqlConnection(CONNECTION_STRING)) { 9 SqlCommand cmd = new SqlCommand(query, connection); 10 return (string)cmd.ExecuteScalar () ; 11 } 12 } 13 } 14 15 public class Product { public string? Name { get; set; }} 16 17 public class ProductFactory { 18 private static String GenQuery( string id) { 19 return "SELECT name FROM products WHERE id = " + id; 20 } 21 22 public static Product GetProduct(string id) { 23 return new Product { 24 Name = DbHandler.DoQuery(GenQuery(id) ) 25 }; 26 } 27 } 28 29 public class VulnerableController : Controller { 30 [HttpGet] 31 [Route (" / product / {id}")] 32 public lActionResult ProductInfo (string id) { 33 Product product = ProductFactory.GetProduct(id); 34 return new JsonResult(product) ; 35 } 36 }
[00147] According to an embodiment, a Codesearch STL may include a dedicated Tainto template that allows users to specify a set of sources, sanitizers, and sinks to apply to taint-flow analysis results coming from an analysis engine.
[00148] In an embodiment, a Codesearch query to match the vulnerability on line 9 above may be as follows: Taint<PRED:AnySource, PRED:SqliSanitizer, PRED: SqliSink>. The convenience predicates AnySource, SqliSanitizer, and SqliSink may be included in a Codesearch STL as well, along with a multitude of additional taint-flow focused predicates, for example as described in Appendix B hereinbelow.
[00149] Example General Dataflow as Taint
[00150] In an embodiment, a Codesearch Tainto template may provide access to more data-flow analysis results than simply taint-flow. As demonstrated in the following example, a Tainto template can be used to query non-taint data-flow. 1 package org. example .app; 2 3 import java. lang. annotation. ; 4 import java. lang. String; 5 6 @Target(ElementType. FIELD) 7 @interface Sensitive { } 8 9 class User { 10 @Sensitive 11 private String username; 12 13 User(String username) { 14 this .username = username; 15 } 16 17 public String getUsername() { 18 return username; 19 } 20 } 21 22 public class Main { 23 public static void main() { 24 User user = new User ("JohnDoe") ; 25 System.out.printIn(user.getUsername() ) ; 26 } 27 }
[00151] According to an embodiment, a Codesearch query that matches on the leak to stdout at line 25 of the Sensitive-annotated field username defined at lines 10 and 11 may be as follows: Taint<HasAnnotation<"Sensitive">,PRED: None,ArglIn<CallExpressi on<" j ava . lang. System, out.printin''»>.
[00152] Example Utilization of Typestate Analysis
[00153] In an embodiment, Codesearch can be used to detect various typestate analysis-based patterns. Below is an example of use-after-free bugs. This example is a simplification of actual bugs in Internet Explorer® (CVE-2010-0249) and the Linux kernel (commit c3aabfO). 1 #include <stdlib.h> 2 3 typedef struct{ int ref;} A; A* gpA; 4 typedef struct! A* _pA;} B; B* gpB; 5 6 void dec_ref (A* a) { if (—(a->ref) == 0) free (a); } 7 8 void clone(B* bl, B* b2) { bl->_pA = b2->_pA; } 9 10 void filename_lookup(A* IpA){ dec_ref(IpA); } 11 12 void demo_code(){ 13 A* pA = (A*)malloc(sizeof (A) ) ; 14 pA->ref = 1; 15 gpA = pA; 16 B* pB = (B*)malloc(sizeof(B)); 17 pB-> pA = pA; 18 gpB = (B*)malloc(sizeof(B)); 19 clone(gpB, pB); 20 pB->_pA = NULL; 21 free(pB); 22 filename_lookup(pA); 23 } 24 25 void main () { 26 demo_code (); 27 dec_ref(gpA); 28 printf(gpB->_pA->ref); 29 free (gpB); 30 }
[00154] In the above example, the two bugs are triggered at lines 27 and 28, respectively. (For a detailed discussion of the bugs, see He, L., et al., “{FreeWill}: Automatically Diagnosing Use-after-free Bugs via Reference Miscounting Detection on Binaries.” 31stUSENIX Security Symposium, 2022, pp. 2497-2512.) According to an embodiment, a Codesearch query to match the bugs at lines 27 and 28 above may be as follows: DataFlowAf ter<Argl In<CallExpression<f ree>».
[00155] Example Utilization of Points-to Analysis and Negation
[00156] In an embodiment, the below example is constructed based on a commit from the Apache Lucene® project, which fixes a File Input St ream resource leak. This example demonstrates use of Codesearch embodiments to detect points-to analysis patterns. 1 package org.apache .lucene. ant; 2 3 import org.w3c.dom.Element; 4 import org.w3c. tidy. Tidy; 5 6 import java. io . Fileinputstream; 7 import java. io .lOException; 8 9 public class HtmlDocument { 10 private Element rawDoc; 11 12 public HtmlDocumentLeaky( File file) throws lOException { 13 Tidy tidy = new TidyO; 14 tidy.setQuiet(true); 15 tidy.setShowWarnings(false) ; 16 org.w3c.dom.Document root = 17 tidy.parseDOM(new Fileinputstream ( file), null); 18 rawDoc = root.getDocumentElement() ; 19 } 20 21 public HtmlDocumentFixed( File file) throws lOException { 22 Tidy tidy = new TidyO; 23 tidy.setQuiet(true); 24 tidy.setShowWarnings(false); 25 org.w3c.dom.Document root = null; 26 Inputstream is = new Fileinputstream (file) ; 27 try { 28 root = tidy.parseDOM( is, null); 29 } finally { 30 is . close () ; 31 } 32 rawDoc = root.getDocumentElement() ; 33 } 34 }
[00157] According to an embodiment, a Codesearch query that matches on the resource initialization at line 17 above, but not the one at line 26 may be as follows: CallExpression<"java. io .FileInputStream"> and not ForSameObj ect<ArgOIn<"close">>. The foregoing may also be an example of the usefulness of negation in StarLang, which is available in embodiments, e.g., Codesearch embodiments.
[00158] Example Linting Using Syntactic Structures
[00159] In an embodiment, the previous examples may focus on querying semantic structures based on results of static analysis. According to another embodiment, however, because a Codesearch STL may contain templates targeting common syntactic structures—in addition to the many semantically-focused templates—the Codesearch STL can be used for simpler tasks as well, such as linting, as demonstrated by the following example.
[00160] Default arguments of a function in Python are initialized only once, when the function definition is evaluated. This means that using mutable objects or non-pure expressions (i.e., that can produce side effects or have outputs that vary even with the same input) as default argument values may be incorrect The example snippet below demonstrates potential consequences of using a non-pure function call. 1 from datetime import datetime 2 import secrets 3 4 def gen filename () : 5 return f"{datetime .utcnow() . timestamp () }_{secrets.token_hex(16) } . txt" 6 *7 H H IT 8 An example correct implementation is— 9 def dump_data(data, filename=None): 10 if filename is None: 11 filename = gen_filename () 12 ... 13 """ 14 15 def dump_data(data, filename=gen_filename() ) : 16 with open (filename, "w") as f: 17 f.write(data) 18 19 dump_data("foo") 20 dump_data("bar")
[00161] The function gen_filename () defined at line 4 may generate a different string each time the function is called—i.e., due to including both a timestamp and a random hexadecimal number in the string at line 5. However, gen_filename () is used to provide a default value of the filename argument for the dump_data () function at line 15. gen_fil ename () is thus only called once by dump_data () —when the latter function definition is first evaluated. The two calls to dump_data () at lines 19 and 20 will accordingly both write to the same file having a filename generated by the initial and only call to gen_filename () at line 15. Moreover, because dump data () does not specify at line 16 that data written to the file should be appended, data from the second dump_data () call at line 20 will overwrite data from the first call at line 19, which is unlikely to be the intended behavior. (A commented-out alternative partial definition of dump_data () that avoids the error is given at lines 9-11.)
[00162] In an embodiment, a Codesearch query to match on such likely-incorrect default argument definitions may be as follows: AnyParamIn<DataFlowAf ter<CallExpression<* »>.
[00163] Example Method Embodiment
[00164] FIG. 3 is a flow diagram of a computer-implemented method 300 for identifying a code portion according to an embodiment. The method 300 begins at step 301 by receiving a logical query, e.g., 106 (FIG. 1), associated with a codebase, e.g., 114 (FIG. 1) or 214 (FIG. 2). The logical query indicates a selected behavior and is constructed in a logical query grammar. The logical query grammar is: (1) configured to represent logical queries each having (i) a unary predicate including a head variable (e.g., p (X) in rule r with predicate p and head variable X), (ii) one or more predicate citations associated with the head variable (e.g., (X) , . . ., (X) in rule r), and (iii) one or more predicate citations associated with one or more other variables different from the head variable (e.g., d{ (Zx) , . . ., dj-(Zj) in rule r, where Z^X for 1 < i <j); (2) configured with variable name reuse (e.g., r (X) :- -.pi(X) , pi (X) ., where variable X is reused in the body of r); (3) configured to represent at least one logical query having one or more conjoined citations associated with (i) a given head variable and (ii) exactly one other variable different from the given head variable (e.g., of the form e (X, Y) , / (Y) or the form e (Y, X) , / (Y), where X may be a variable in a head of r and Y^X); and (4) configured with stratified negation (e.g., if q is cited negatively in a body ofp, then q itself may not be defined, directly or transitively, in terms ofp). At step 302, based on the received logical query, the method 300 then searches a logical graph, e.g., 118 (FIG. 1), representing the codebase to generate a query result, e.g., 112 (FIG. 1), indicating a presence of a code portion, within the codebase, exhibiting the selected behavior.
[00165] The method 300 is computer-implemented and, as such, the functionality and effective operations, e.g., the receiving (301) and searching (302), are automatically implemented by one or more digital processors. Moreover, the method 300 can be implemented using any computer device or combination of computing devices known in the art. Among other examples, the method 300 can be implemented using computer(s) / device(s) 50 and / or 60 described hereinbelow in relation to FIGs. 4 and 5.
[00166] According to an example embodiment, the method 300 may further include receiving a user query, e.g., 102 (FIG. 1), associated with the codebase. The user query may indicate the selected behavior and be constructed in a user query grammar. The user query grammar may be configured to represent user queries each having at least one user query citation. The at least one user query citation may include at least one of a user query predicate, a user query template, e.g., 222 (FIG. 2), and a literal value. In one such embodiment, receiving the logical query may include transforming the received user query, e.g., 102 (FIG. 1), into the logical query, e.g., 106 (FIG. 1). According to another such embodiment, the method 300 may further include processing the logical graph to generate an analysis result. The analysis result may include at least one indication of a match between (i) a given code portion, within the codebase, and (ii) at least one of a given user query predicate, a given user query template, and a given literal value. The method 300 may further include, based on the received user query and the analysis result, generating an autocomplete suggestion, e.g., 108a-108n (FIG. 1) or 208a-208n (FIG. 2), for the received user query. The autocomplete suggestion may further include at least one of: (i) one or more user query predicates, (ii) one or more user query templates, and (iii) one or more literal values. In yet another such embodiment, the method 300 may further include storing the analysis result in cache storage. According to one such embodiment, the generated autocomplete suggestion may include a plurality of suggestions. The method 300 may further include ranking the plurality of suggestions.
[00167] In an example embodiment of the method 300, the codebase may include one or more procedure calls, e.g., 224a-224c (FIG. 2), and one or more dataflow paths. The logical graph may include one or more nodes corresponding to the one or more procedure calls and one or more edges corresponding to the one or more dataflow paths. According to one such embodiment, searching the logical graph at step 302 may include identifying at least one node of the one or more nodes. The at least one node may correspond to at least one procedure call associated with the selected behavior. The query result generated at step 302 may include an indication of the at least one node identified. In another such embodiment, the logical query grammar may be further configured to represent one or more logical queries having a binary predicate corresponding to a set of edges of the one or more edges. According to yet another such embodiment, the logical query grammar may be further configured to represent one or more logical queries having a negated binary predicate corresponding to a second set of edges of the one or more edges. The second set may be complementary to a first set of edges of the one or more edges. In one such embodiment, the logical graph may be a hypergraph. A given edge of the one or more edges may connect a given set of the one or more nodes. The logical query grammar may be further configured to represent one or more logical queries having a hypergraph predicate corresponding to a set of edges of the one or more edges.
[00168] According to an example embodiment, the method 300 may further include generating the logical graph by (1) based on the codebase, constructing at least one AST, e.g., 116a-116n (FIG. 1), and (2) transforming the at least one AST constructed into the logical graph.
[00169] Example Advantages
[00170] Advantages of embodiments include, for non-limiting examples: a) Embodiments provide StarLang, a language as expressive as Datalog on unary predicates. Moreover, the low time complexity of StarLang’s decision procedure(s) is demonstrated. b) Embodiments provide a system, e.g., Snyk® Code, that computes dataflow, taint, points-to, and other analyses and enables performing StarLang queries in realtime over large repositories. FIG. 1, described hereinabove, presents a schematic overview of the example system 100 according to an embodiment. c) The expressiveness of StarLang is demonstrated on an example range of useful queries that cover taint security properties, memory safety, typestate safety, and other examples. d) Embodiments provide a frontend to StarLang called Codesearch that enables users to interactively build static analysis queries, significantly simplifying the job of, e.g., security professionals and static analysis authors.
[00171] Computer Support
[00172] FIG. 4 is a schematic view of an example computer network in which embodiments may be implemented. Client computer(s) / devices 50 and server computer(s) 60 provide processing, storage, and input / output (VO) devices executing application programs and the like. Client computer(s) / device(s) 50 can also be linked through communications network 70 to other computing devices, including other client device(s) / processor(s) 50 and server computer(s) 60. The communications network 70 can be part of a remote access network, a global network (e.g., the Internet), cloud computing servers or services, a worldwide collection of computers, local area or wide area networks, and gateways that currently use respective protocols (e.g., TCP / IP, Bluetooth®, etc.) to communicate with one another. Other electronic device / computer network architectures are also suitable.
[00173] FIG. 5 is a block diagram illustrating an example embodiment of a computer node (e.g., client processor(s) / device(s) 50 or server computer(s) 60) in the computer network 70 of FIG. 4. Each computer node 50, 60 contains system bus 79, where a bus is a set of hardware lines used for data transfer among components of a computer or processing system. The system bus 79 is essentially a shared conduit that connects different elements of a computer system (e.g., processor, disk storage, memory, VO ports, network ports, etc.) that enables transfer of information between the elements. Attached to the system bus 79 is an I / O devices interface 82 for connecting various input and output devices (e.g., keyboard, mouse, display(s), printer(s), speaker(s), etc.) to the computer node 50, 60. A network interface 86 allows the computer node to connect to various other devices attached to a network (e.g., the network 70 of FIG. 4). A memory 90 provides volatile storage for computer software instructions 92a and data 94a used to implement an embodiment of the present disclosure (e.g., the system 100 of FIG. 1, the method 300 of FIG. 3, etc.). Disk storage 95 provides non-volatile storage for the computer software instructions 92b and data 94b used to implement an embodiment of the present disclosure. A central processor unit 84 is also attached to the system bus 79 and provides for execution of computer instructions.
[00174] In one embodiment, the processor routines 92a-92b and data 94a-94b are a computer program product (generally referenced as 92), including a non-transitory, computer readable medium (e.g., a removable storage medium such as DVD-ROM(s), CD-ROM(s), diskette(s), tape(s), etc.) that provides at least a portion of the software instructions for an embodiment. The computer program product 92 can be installed by any suitable software installation procedure, as is well known in the art. In another embodiment, at least a portion of the software instructions may also be downloaded over a cable, communication, and / or wireless connection. In other embodiments, the disclosure programs are a computer program propagated signal product embodied on a propagated signal on a propagation medium (e.g., a radio wave, an infrared wave, a laser wave, a sound wave, or an electrical wave propagated over a global network such as the Internet, or other network(s)). Such carrier medium or signals provide at least a portion of the software instructions for the present disclosure routines / program 92.
[00175] In alternative embodiments, the propagated signal is an analog carrier wave or digital signal carried on the propagated medium. For example, the propagated signal may be a digitized signal propagated over a global network (e.g.. the Internet), a telecommunications network, or other networks (such as the network 70 of FIG. 4). In one embodiment, the propagated signal is a signal that is transmitted over the propagation medium over a period of time, such as the instructions for a software application sent in packets over a network over a period of milliseconds, seconds, minutes, or longer. In another embodiment, the computer readable medium of the computer program product 92 is a propagation medium that the computer system 50 may receive and read, such as by receiving the propagation medium and identifying a propagated signal embodied in the propagation medium, as described above for computer program propagated signal product.
[00176] Generally speaking, the term “carrier medium” or transient carrier encompasses the foregoing transient signals, propagated signals, propagated medium, storage medium, and the like.
[00177] In other embodiments, the program product 92 may be implemented as a so-called Software as a Service (SaaS), or other installation or communication supporting endusers.
[00178] Example Formal Grammar, Templates, and Predicates for Implementing Embodiments
[00179] Provided hereinbelow as Appendix A is an example formal grammar that may be utilized for implementing embodiments. Further provided hereinbelow as Appendix B is an example template and predicate library that may be used to implement embodiments.
[00180] The teachings of all patents, published applications, and references cited herein are incorporated by reference in their entirety.
[00181] Embodiments or aspects thereof may be implemented in the form of hardware including but not limited to hardware circuitry, firmware, or software. If implemented in software, the software may be stored on any non-transient computer readable medium that is configured to enable a processor to load the software or subsets of instructions thereof. The processor then executes the instructions and is configured to operate or cause an apparatus to operate in a manner as described herein.
[00182] Further, hardware, firmware, software, routines, or instructions may be described herein as performing certain actions and / or functions of the data processors. However, it should be appreciated that such descriptions contained herein are merely for convenience and that such actions in fact result from computing devices, processors, controllers, or other devices executing the firmware, software, routines, instructions, etc.
[00183] It should be understood that the flow diagrams, block diagrams, and network diagrams may include more or fewer elements, be arranged differently, or be represented differently. But it further should be understood that certain implementations may dictate the block and network diagrams and the number of block and network diagrams illustrating the execution of the embodiments be implemented in a particular way.
[00184] Accordingly, further embodiments may also be implemented in a variety of computer architectures, physical, virtual, cloud computers, and / or some combination thereof, and, thus, the data processors described herein are intended for purposes of illustration only and not as a limitation of the embodiments.
[00185] While example embodiments have been particularly shown and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the embodiments encompassed by the appended claims.
[00186] For example, the foregoing description and details of embodiments reference Applicant-Assignee (Snyk Limited), software, tools, and platforms, for purposes of illustration and not limitation. Other similar software, tools, and platforms are also suitable. APPENDIX A - Example Formal Grammar Description (Query) ::= (Citation) ((LogicalConnective) (Citation))* (Citation) ::= (Negation) ((Predicate) | (Template) | (Literal)) (Predicate) ::= ‘PRED : ’ (ValidName) (Template) ::= (ValidName)'<’(Query)(c,' (Query))* N" (ValidName) ::= Standard rules for naming variables (LogicalConnective) ::= ‘and’ | ‘or’ (Negation) ::= ‘not’ | s (Literal} ::= Any UTF-8 character sequence | ‘ ~ ’ RE2 regex ‘’ In the interest of keeping the grammar description clean, optional grouping of (Citation)^ using parenthesis has been omitted. Operator precedence is standard, i.e., not has a higher precedence than and, which has a higher precedence than or. APPENDIX B - Example Template and Predicate Library Predicate Any: A “catchall” rule. Matches on anything. Predicate AnySink: Matches on a range of potential data sinks, including server responses, file systems, database writes, external APIs, logging mechanisms, and other forms of data export or display. Predicate AnySource: Matches on various types of potentially user controlled data sources, both servers (e.g., HTTP parameters / header / body, URLs, cookies, etc.) or indirect ones such as database fields, local files, I / O or environment variables. Predicate ApexPageReferenceSource: Matches on potential cross-site scripting (XSS) sources. Predicate CleartextCookieStorageSanitizer: Matches on cleartext cookie storage sanitizers. Predicate CleartextCookieStorageSink: Matches on cleartext cookie storage sinks. Predicate CleartextTransmissionSanitizer: Matches on cleartext transmission sanitizers. Predicate CleartextTransmissionSink: Matches on cleartext transmission sinks. Predicate ClientXssSanitizer: Matches on client XSS (e.g., DOMXSS) sanitizers. Predicate ClientXssSink: Matches on client XSS (e.g., DOMXSS) sinks. Predicate Codelnj ectionSanitizer: Matches on code injection sanitizers. Predicate Codelnj ectionSink: Matches on code injection sinks. Predicate Commandlnj ectionSanitizer: Matches on command injection sanitizers. Predicate Commandlnj ectionSink: Matches on command injection sinks. Predicate DeserializationSanitizer: Matches on deserialization sanitizers. Predicate Deserializationsink: Matches on deserialization sinks. Predicate EmailContentlnjectionSanitizer: Matches on email content injection sanitizers. Predicate EmailContentlnj ectionSink: Matches on email content injection sinks. Predicate ErrorMessageOutput: Matches on error message outputs (e.g., stacktraces). Predicate ErrorMessageOutputSanitizer: Matches on error message output sanitizers. Predicate ErrorMessageOutputSink: Matches on error message output sinks. Predicate FilelnclusionSanitizer: Matches on file inclusion sanitizers. Predicate Fileinclusionsink: Matches on file inclusion sinks. Predicate Inf ormationDisclosureSanitizer: Matches on information disclosure sanitizers. Predicate Inf ormationDisclosureSink: Matches on information disclosure sinks. Predicate Jndilnj ectionSanitizer: Matches on Java Naming and Directory Interface (JNDI) injection sanitizers. Predicate Jndilnj ectionSink: Matches on JNDI injection sinks. Predicate Ldaplnj ectionSanitizer: Matches on LDAP injection sanitizers. Predicate Ldaplnj ectionSink: Matches on LDAP injection sinks. Predicate LogsForgingSanitizer: Matches on log-forging sanitizers. Predicate LogsForgingSink: Matches on log-forging sinks. Predicate MemoryCorruptionSanitizer: Matches on prototype memory corruption sanitizers. Predicate NoSqliSanitizer: Matches on NoSQL sanitizers. Predicate NoSqliSink: Matches on NoSQL sinks. Predicate None: An “anti-catchall” rule. Matches on nothing. Predicate OpenRedirectSanitizer: Matches on open-redirect sanitizers. Predicate OpenRedirectSink: Matches on open-redirect sinks. Predicate PointerOperationSink: Matches on prototype memory operation sinks. Predicate PotentialXssSink: Matches on potential XSS sinks. Predicate PrototypePollutionAssignmentSanitizer: Matches on prototype pollution assignment sanitizers. Predicate PrototypePollutionAssignmentSink: Matches on prototype pollution assignment sinks. Predicate PtSanitizer: Matches on path-traversal sanitizers. Predicate PtSink: Matches on path-traversal sinks. Predicate RedosSanitizer: Matches on regular-expression denial-of-service sanitizers. Predicate RedosSink: Matches on regular-expression denial-of-service sinks. Predicate Ref lectionSanitizer: Matches on reflection sanitizers. Predicate Reflectionsink: Matches on reflection sinks. Predicate SoqliSanitizer: Matches on Salesforce® Object Query Language (SOQL) injection (SOQLi) sanitizers. Predicate SoqliSink: Matches on SOQLi sinks. Predicate SosliSanitizer: Matches on Salesforce® Object Search Language (SOSL) injection (SOSLi) sanitizers. Predicate SosliSink: Matches on SOSLi sinks. Predicate SourceArchive: Matches on reading values that are coming from zip, tar or other archives. Predicate SourceCLI: Matches on reading command line arguments. Predicate SourceClientFramework: Matches on reading values that are coming from a client-side framework such as Android®, SwiftUI®, UIKit, the DOM of an HTML page. Predicate SourceContainsSensitiveData: Matches on reading sensitive data. Predicate SourceCookie: Matches on reading values of cookies in an HTTP server. These values are of security interest, because they can be fully controlled by malicious users. Predicate SourceDatabase: Matches on reading values that are coming from a database. Predicate SourceEnvironmentVariable: Matches on reading environment variables of a process. Predicate SourceFile: Matches on reading values that are coming from files. Predicate SourceHttpBody: Matches on reading HTTP request body in an HTTP server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SourceHttpFileUpload: Matches on the name and content of file uploaded to an HTTP server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SourceHttpHeader: Matches on reading values of HTTP headers in a server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SourceHttpParam: Matches on reading values of HTTP parameters in an HTTP server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SourceLocalEnv: Matches on reading values from the local environment of the running process. This includes command line arguments, standard input or environment variables. Predicate SourceNetworkRequest: Matches on reading values that are coming from a remote resource through network requests. Predicate SourceNonServer: Matches on reading values that may be controlled by an adversary, but not directly by sending requests to a server. E.g., if an application fetches a value from a URL, an adversary in control of that URL may use it to control its content. Predicate SourceRequestUrl: Matches on reading request URLs in a server. The URLs are of security interest, because they may be fully controlled by malicious actors. Predicate SourceResourceAccess: Matches on reading values that may be controlled by an adversary if they gain access to a resource. The resources this matches are remote URLs, files, database fields or other framework-specific cases such as Android® intents. Predicate SourceRpcApiParam: Matches on parameters of RPCs implemented in an RPC server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SourceServer: Matches on reading values that an attacker can send to a server. Examples are HTTP parameters / header / body, URLs or cookies. Since these may be directly controllable by attacker, these sources are of significant security interest. Predicate SourceStdin: Matches on reading input from the standard input of a process. Predicate SourceUnrestrictedArchiveFilePath: Matches on zipslip sources. Predicate SourceWebForm: Matches on reading values of web forms in a web server. These values are of security interest, because they may be fully controlled by malicious actors. Predicate SqliSanitizer: Matches on SQL injection sanitizers. Predicate SqliSink: Matches on SQL injection sinks. Predicate Ssrf Sanitizer: Matches on server-side request forgery (SSRF) sanitizers. Predicate SsrfSink: Matches on SSRF sinks. Predicate SstiSanitizer: Matches on server-side template injection (SSTI) sanitizers. Predicate SstiSink: Matches on SSTI sinks. Predicate UnsafeSoqliConcatSource: Matches on unsafe SOSLi / SOQLi concatenations. Predicate Unsaf eSosliConcatSource: Matches on unsafe SOSLi / SOQLi concatenations. Predicate XPathlnj ectionSanitizer: Matches on XPath injection sanitizers. Predicate XPathlnj ectionSink: Matches on XPath injection sinks. Predicate Xamllnj ectionSanitizer: Matches on XAML injection sanitizers. Predicate Xamllnj ectionSink: Matches on XAML injection sinks. Predicate Xmllnj ectionSanitizer: Matches on XML injection sanitizers. Predicate Xmllnj ectionSink: Matches on XML injection sinks. Predicate XssSanitizer: Matches on XSS sanitizers. Predicate XssSink: Matches on XSS sinks. Predicate XxeSanitizer: Matches on XML external entity injection (XXE) sanitizers. Predicate XxeSink: Matches on XXE sinks. Predicate ZipSlipSanitizer: Matches on zipslip sanitizers. Predicate ZipSlipSink: Matches on zipslip sinks. Template And: A binary conjunction. Matches only if both arguments match. Has 2 arguments: conjunct, conjunct. Template AnyParamln: Matches on all parameters of the provided method or function declaration / signature. Has 1 argument: Function. Template ArgO In: Matches on the 0th index argument (i.e., the receiver object for method calls) for the provided method or function. Has 1 argument: Function. Template Argl In: Matches on the 1st index argument for the provided method or function. Has 1 argument: Function. Template Arg2 In: Matches on the 2nd index argument for the provided method or function. Has 1 argument: Function. Template Ar g3 In: Matches on the 3rd index argument for the provided method or function. Has 1 argument: Function. Template Arg4 In: Matches on the 4th index argument for the provided method or function. Has 1 argument: Function. Template Arg5In: Matches on the 5th index argument for the provided method or function. Has 1 argument: Function. Template Arg6In: Matches on the 6th index argument for the provided method or function. Has 1 argument: Function. Template Arg7 In: Matches on the 7th index argument for the provided method or function. Has 1 argument: Function. Template BooleanLiteral: Matches on boolean type literals. Has 1 argument: Value. Template CallExpression: Matches when a given name is called. Has 1 argument: Function, method or constructor to call. Template DataFlowAfter: Matches on entities that happen after in the dataflow of its parameter. Has 1 argument: The previous action executed. Template DataFlowsFrom: Matches on places which a taint data can flow from. Has 1 argument: Source. Template DataFlowsInto: Matches on places which a taint data can flow into. Has 1 argument: Sink. Template ExplicitSelfParamin: Matches on the explicit receiver parameter (e.g., sei f in Python and Rust) for the provided method or function declaration. Has 1 argument: Function. Template ForSameObj ect: Matches on entities that happen on the same object as its parameter. Has 1 argument: The action that happens on the object. Template HasAnnotation: Matches on entities annotated by a given annotation. Has 1 argument: The annotation with which the entity is annotated. Template HasAnyArg: Matches on entities that take any argument with the provided value. Has 1 argument: Value. Template HasArgO: Matches on entities that take an argument in the 0th index (i.e., receiver object for method calls) with the provided value. Has 1 argument: Value. Template HasArgl: Matches on entities that take an argument in the 1st index with the provided value. Has 1 argument: Value. Template HasArg2: Matches on entities that take an argument in the 2nd index with the provided value. Has 1 argument: Value. Template HasArg3: Matches on entities that take an argument in the 3rd index with the provided value. Has 1 argument: Value. Template HasArg4: Matches on entities that take an argument in the 4th index with the provided value. Has 1 argument: Value. Template HasAr g5: Matches on entities that take an argument in the 5th index with the provided value. Has 1 argument: Value. Template HasArg6: Matches on entities that take an argument in the 6th index with the provided value. Has 1 argument: Value. Template HasArg?: Matches on entities that take an argument in the 7th index with the provided value. Has 1 argument: Value. Template HasNamedArg: Matches on entities that take a named argument with the provided value. Has 2 arguments: The name of the argument., The value the named argument should have. Template Identifier: Matches on an identifier. Has 1 argument: The entity that should be an identifier. Template InPath: Matches on entities in the source file with the provided path. Has 1 argument: The path of the file in which to match entities. Template Literal: Matches on string / boolean or number type literals. Has 1 argument: Value. Template NamedArgln: Matches on the named argument for the provided method or function. Has 2 arguments: The name of the argument., The provided method or function. Template Not: A negation. Matches only if the argument does not match. Has 1 argument: property. Template NumberLiteral: Matches on numeric type literals. Has 1 argument: Value. Template Or: A binary disjunction. Matches if either (or both) arguments match. Has 2 arguments: disjunct, disjunct. Template Paraml In: Matches on the 1st parameter for the provided method or function declaration. Has 1 argument: Function. Template Param2 In: Matches on the 2nd parameter for the provided method or function declaration. Has 1 argument: Function. Template Par am3 In: Matches on the 3rd parameter for the provided method or function declaration. Has 1 argument: Function. Template Param4 In: Matches on the 4th parameter for the provided method or function declaration. Has 1 argument: Function. Template Param5In: Matches on the 5th parameter for the provided method or function declaration. Has 1 argument: Function. Template Param6In: Matches on the 6th parameter for the provided method or function declaration. Has 1 argument: Function. Template Param? In: Matches on the 7th parameter for the provided method or function declaration. Has 1 argument: Function. Template ReturnedBy: Matches on the returned entity. Has 1 argument: The entity that returns. Template Returns: Matches on the entity (e.g., a function or a method) that returns the value provided as argument. Has 1 argument: What is returned. Template StringLiteral: Matches on string type literals. Has 1 argument: Value. Template Taint: Identify data propagation flows that start at the specified source(s) and reach the designated destination sinks (like vulnerable methods) without going through the specified sanitizer(s). Has 3 arguments: Source, Sanitizer, Sink.
Claims
1. A computer-implemented method for identifying a code portion, within a codebase, exhibiting a selected behavior, the computer-implemented method comprising, by at least one processor:receiving a logical query associated with a codebase, the logical query indicating a selected behavior and constructed in a logical query grammar, the logical query grammar being:configured to represent logical queries each having (i) a unary predicate including a head variable, (ii) one or more predicate citations associated with the head variable, and (iii) one or more predicate citations associated with one or more other variables different from the head variable;configured with variable name reuse;configured to represent at least one logical query having one or more conjoined citations associated with (i) a given head variable and (ii) exactly one other variable different from the given head variable; andconfigured with stratified negation; andbased on the received logical query, searching a logical graph representing the codebase to generate a query result indicating a presence of a code portion, within the codebase, exhibiting the selected behavior.
2. The computer-implemented method of Claim 1, further comprising:receiving a user query associated with the codebase, the user query indicating the selected behavior and constructed in a user query grammar, the user query grammar configured to represent user queries each having at least one user query citation, the at least one user query citation including at least one of a user query predicate, a user query template, and a literal value.
3. The computer-implemented method of Claim 2, wherein receiving the logical query includes:transforming the received user query into the logical query.
4. The computer-implemented method of Claim 2, further comprising:processing the logical graph to generate an analysis result, the analysis result including at least one indication of a match between (i) a given code portion, within the codebase, and (ii) at least one of a given user query predicate, a given user query template, and a given literal value; andbased on the received user query and the analysis result, generating an autocomplete suggestion for the received user query, the autocomplete suggestion including at least one of: (i) one or more user query predicates, (ii) one or more user query templates, and (iii) one or more literal values.
5. The computer-implemented method of Claim 4, further comprising:storing the analysis result in cache storage.
6. The computer-implemented method of Claim 4, wherein the generated autocomplete suggestion includes a plurality of suggestions, and further comprising:ranking the plurality of suggestions.
7. The computer-implemented method of Claim 1, wherein:the codebase includes one or more procedure calls and one or more dataflow paths; andthe logical graph includes one or more nodes corresponding to the one or more procedure calls and one or more edges corresponding to the one or more dataflow paths.
8. The computer-implemented method of Claim 7, wherein searching the logical graph includes:identifying at least one node of the one or more nodes, the at least one node corresponding to at least one procedure call associated with the selected behavior; andwherein the query result includes an indication of the at least one node identified.
9. The computer-implemented method of Claim 7, wherein the logical query grammar is further configured to represent one or more logical queries having a binary predicate corresponding to a set of edges of the one or more edges.
10. The computer-implemented method of Claim 7, wherein the logical query grammar is further configured to represent one or more logical queries having a negated binary predicate corresponding to a second set of edges of the one or more edges, the second set being complementary to a first set of edges of the one or more edges.
11. The computer-implemented method of Claim 7, wherein the logical graph is a hypergraph, wherein a given edge of the one or more edges connects a given set of the one or more nodes, and wherein the logical query grammar is further configured to represent one or more logical queries having a hypergraph predicate corresponding to a set of edges of the one or more edges.
12. The computer-implemented method of Claim 1, further comprising: generating the logical graph by:based on the codebase, constructing at least one abstract syntax tree (AST); andtransforming the at least one AST constructed into the logical graph.
13. A computer-based system for identifying a code portion, within a codebase, exhibiting a selected behavior, the computer-based system comprising:a processor; anda memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the computer-based system to:receive a logical query associated with a codebase, the logical query indicating a selected behavior and constructed in a logical query grammar, the logical query grammar being:configured to represent logical queries each having (i) a unary predicate including a head variable, (ii) one or more predicate citations associated with the head variable, and (iii) one or more predicate citations associated with one or more other variables different from the head variable; andconfigured with variable name reuse;configured to represent at least one logical query having one or more conjoined citations associated with (i) a given head variable and(ii) exactly one other variable different from the given head variable; andconfigured with stratified negation; andbased on the received logical query, search a logical graph representing the codebase to generate a query result indicating a presence of a code portion, within the codebase, exhibiting the selected behavior.
14. The computer-based system of Claim 13, wherein the processor and the memory, with the computer code instructions, are further configured to cause the computer-based system to:receive a user query associated with the codebase, the user query indicating the selected behavior and constructed in a user query grammar, the user query grammar configured to represent user queries each having at least one user query citation, the at least one user query citation including at least one of a user query predicate, a user query template, and a literal value.
15. The computer-based system of Claim 14, where, in searching the logical graph, the processor and the memory, with the computer code instructions, are configured to cause the computer-based system to:transform the received user query into the logical query.
16. The computer-based system of Claim 14, wherein the processor and the memory, with the computer code instructions, are further configured to cause the computer-based system to:process the logical graph to generate an analysis result, the analysis result including at least one indication of a match between (i) a given code portion, within the codebase, and (ii) at least one of a given user query predicate, a given user query template, and a given literal value; andbased on the received user query and the analysis result, generate an autocomplete suggestion for the received user query, the autocomplete suggestion including at least one of: (i) one or more user query predicates, (ii) one or more user query templates, and (iii) one or more literal values.
17. The computer-based system of Claim 13, wherein:the codebase includes one or more procedure calls and one or more dataflow paths; andthe logical graph includes one or more nodes corresponding to the one or more procedure calls and one or more edges corresponding to the one or more dataflow paths.
18. The computer-based system of Claim 17, where, in searching the logical graph, the processor and the memory, with the computer code instructions, are configured to cause the computer-based system to:identify at least one node of the one or more nodes, the at least one node corresponding to at least one procedure call associated with the selected behavior; and wherein the query result includes an indication of the at least one node identified.
19. The computer-based system of Claim 17, wherein the logical query grammar is further configured to represent one or more logical queries having a binary predicate corresponding to a set of edges of the one or more edges.
20. A computer program product for identifying a code portion, within a codebase, exhibiting a selected behavior, the computer program product comprising a non-transitory computer-readable medium with computer code instructions stored thereon, the computer code instructions being configured, when executed by a processor, to cause an apparatus associated with the processor to:receive a logical query associated with a codebase, the logical query indicating a selected behavior and constructed in a logical query grammar, the logical query grammar being:configured to represent logical queries each having (i) a unary predicate including a head variable, (ii) one or more predicate citations associated with the head variable, and (iii) one or more predicate citations associated with one or more other variables different from the head variable;configured with variable name reuse;configured to represent at least one logical query having one or more conjoined citations associated with (i) a given head variable and (ii) exactly one other variable different from the given head variable; andconfigured with stratified negation; andbased on the received logical query, search a logical graph representing the codebase to generate a query result indicating a presence of a code portion, within the codebase, exhibiting the selected behavior.