159 results about "Abstract syntax" patented technology

A modular parser architecture and methods related thereto make use of a number of miniparsers. Each miniparser is capable of receiving input and generating an output dependent thereon. In one embodiment, a lexer or a miniparser capable of lexical analysis receives certain code and provides an abstract syntax tree. Each miniparser performs operations on a set of syntactical constructs to produce a corresponding output syntax tree, which in turn may become an input syntax tree for a next miniparser which may operate on a different set of syntactical constructs.

A system and method for managing network devices. The framework may provide a suitable system for managed object data type conversions between various data description languages, such as an interface definition language (e.g., OMG IDL) and an abstract syntax notation (e.g., ASN1). This conversion facility may be used in both request and event network traffic, so common libraries may be used. Two separate libraries may be used: a converter framework library and a converter implementation library. The framework library provides a collection classes that may be used by the clients of the converter libraries. Most classes in the converter framework library are handle classes (or wrappers) to the real implementation (or body) classes in the converter implementation library. Framework classes hide the details of the real implementation classes and provide a simple, consistent interface to any data type converter via IDL. The converter implementation library provides the implementation needed by the framework to function properly, and may allow various different mappings to be used easily by implementing them as plug-in modules. The combination of using IDL and generic typing provides an efficient, generic solution to mapping data types across multiple platforms, multiple programming languages, and multiple object classes.

The invention discloses a Webshell detection method based on a deep neural network, and a system thereof. A recursion loop neural network based on an abstract syntax tree automatically obtains the morphology and grammar information of a script by aiming at a script language, the hierarchical structure characteristics of the abstract syntax tree is used for finishing feature abstraction and WebShell detection, including preprocessing, sample generation and WebShell detection; firstly, the morphology and grammar information of the script is automatically obtained, and then, the recursion loop neural network based on the abstract syntax tree is used for finishing the feature abstraction and the WebShell detection. The method disclosed by the invention has the advantages of being low in deployment cost, good in portability and high in detection accuracy.

A system, method, and a computer-readable medium for generating a valid HTML document. The method includes receiving a request from a browser and, in response, providing an abstract syntax tree, the abstract syntax tree including a plurality of nodes, where each node represents an element in a template file. The method also includes, traversing the abstract syntax tree, and for each node in the tree determining a type of the node and performing an action based on the node type. The method further includes generating the valid HTML document based on the plurality of actions and sending the valid HTML document for display to a browser.

The invention provides a Python program type defect detection method based on abstract syntax trees. The method comprises the steps of (1) collecting Python software defect report information and extracting a defect identifier and a defect error type, (2) obtaining source codes of two versions of programs before and after defect repair, (3) generating abstract syntax trees corresponding to the twoversions of source codes, matching and obtaining a change function node and marking the defect error type, (4) generating a feature vector of a defect code according to the context information of thechange function node, (5) training a multi-classification models on a defect code feature vector by using the machine learning technology, and (6) reminding a developers of possible type defect information in testing a Python program file. The invention aims to solve the problems of the lack of the type defect analysis for a Python language and the inability to detect a possible defect error typeat present, thus the management of software quality is guided, and the maintainability of the software is improved.

A modular parser architecture and methods related thereto make use of a number of miniparsers. Each miniparser is capable of receiving input and generating an output dependent thereon. In one embodiment, a lexer or a miniparser capable of lexical analysis receives certain code and provides an abstract syntax tree. Each miniparser performs operations on a set of syntactical constructs to produce a corresponding output syntax tree, which in turn may become an input syntax tree for a next miniparser which may operate on a different set of syntactical constructs.

The present invention provides methods, systems and apparatus to improve XML data encoding (compression) decoding (decompression) efficiency. In an example embodiment, a syntax type generator 2-1 converts a DTD into an ASN.1 abstract syntax type, and a separation unit 2-2 separates text from XML data that conforms to the DTD. Then, a syntax value generator 2-3 converts the XML data (the syntax of an element) from which the text has been separated into an ASN.1 abstract syntax value that conforms to the ASN.1 abstract syntax type. Thereafter, following the conversion by a transfer syntax generator 2-4 of the ASN.1 abstract syntax value into an ASN.1 transfer syntax, the separated text is compressed by a compression unit 2-5 and encoded XML data is generated by a combining unit 2-6 that combines the ASN.1 transfer syntax and the compressed text.

The present invention provides a method and device for determining data consanguinity based on structural data. The method comprises: parsing a selection statement to obtain a source abstract syntax tree, and organizing the table information and the field information obtained by traversing the source abstract syntax tree into a source list layer by layer; obtaining the target abstract syntax treeby parsing the insertion statement, and organizing the table information and the field information obtained by traversing the target abstract syntax tree into the target list layer by layer; traversing the source manifest to obtain the source table information and traversing the target manifest to obtain the target table information to obtain the table granularity data kinship relationship; extracting the target field information for the target table from the target manifest, starting from the first layer of the source manifest, finding the source field in the source table with the same name as the target field information of the target table layer by layer, and when the source table to which the source field belongs no longer comes from the subquery, determining the corresponding source field to be the source field with blood relationship corresponding to the target field information. The scheme can parse the field-level granularity of data consanguinity based on structured data.

The invention provides a Python code change reminding method based on abstract syntax tree node change extraction. The method comprises the following steps of: (1) obtaining source code of different-version programs of the same software; (2) generating abstract syntax trees corresponding to two versions of source programs; (3) matching the abstract syntax trees to obtain a change node, and combining change node context information to mark the node change type; (4) gathering change tuples into transactions to construct a training set; (5) mining the association relationship in the change tuples by using a data mining technology; and (6) reminding a developer of possibly occurring change positions and possible change types in the according to the programs mined association relationship. The Python code change reminding method solves the problems that in the prior art, the possible code change type cannot be reminded due to the lack of software evolution analysis aiming at the python language, and the like. The management of the software life cycle is further guided, and the software evolution controllability is improved, so that the quality of a software product can be well controlled.

The invention relates to a structured query language (SQL) based MapReduce operation generating method and system. The method includes analyzing received SQL sentences to generate an abstract syntax tree, converting the abstract syntax tree into a query plan tree, optimizing the query plan tree, and generating a corresponding MapReduce operation according to the optimized query plan tree. According to the system, an another tool for language recognition (Antlr) is based on to generate the abstract syntax tree (AST) from the SQL sentences, further, the AST is based on to generate a query plan tree, and finally, the MapReduce operation is generated on the basis of the query plan tree; meanwhile, corresponding optimization policies are further provided, so that the generated MapReduce operation has higher execution efficiency.

The invention relates to a software homology detection method and a device based on an abstract syntax tree and semantic matching. The method comprises the following steps: generating an abstract syntax tree corresponding to software source codes, and regulating subtrees which match with the same semantic feature rule and have the same semantics in the abstract syntax tree to a unified structure; calculating the hash values of subtrees in the abstract syntax tree; and performing software homology detection through judging whether the hash values of subtrees with the same number of nodes are consistent, thereby performing accurate and effective software homology detection by combining semantics in grammatical level.

The invention discloses a source code vulnerability detection method, a detector and a training method and system thereof. The method comprises the steps of generating an abstract syntax tree based on each function in a source code, traversing all the abstract syntax trees, searching the code elements capable of being matched with the same vulnerability syntax feature, calling the code elements as the candidate vulnerability elements, and determining all the candidate vulnerability elements; based on a program slicing technology, identifying the code statements in semantic association with each candidate vulnerability element in the source code to determin the corresponding candidate vulnerability code segments, and adding the vulnerability labels to the candidate vulnerability code segments; and inputting the vector data corresponding to each candidate vulnerability code segment into a neural network model, and training the neural network model based on the label of each candidate vulnerability code segment to obtain a vulnerability detector. Based on the deep neural network, the vulnerability syntax features are extracted through a universal framework, the method is not limited to the vulnerability types and is high in expandability.

A method and computer program product for generating a suspect abstract syntax tree by partially decompiling a compiled software module. The suspect abstract syntax tree is compared to one or more reference abstract syntax trees.

The invention discloses a code searching method based on semantics. A system adopted in the method mainly comprises a data collection module, a data analysis module, a data sorting module and a user interaction module, wherein a crawler tool JoBo in the data collection module can preset certain website addresses in a configuration file, a code source is fetched from a preset forum and blog to most effectively fetch the webpage at the highest speed; the code source fetched by the crawler tool JoBo is subjected to semantic analysis by an abstract syntax tree AST frame in an open-source tool JDT (Java Development Tools ); the data sorting module is matched according to keywords input by a user; and after a corresponding search result is obtained by analysis, factors on five aspects are comprehensively considered, and the search results are successively sorted and displayed to the user from the higher score to the lower score by sorting data. On the basis of utilizing the traditional best search engine, the code searching method correspondingly expands semantic information identification and sorting, defines a search range by configuring a crawler, improves the search efficiency and the precision accuracy, and considers user favor.

A method is disclosed that allows the concise implementation of one protocol encoding / decoding device for syntax conversion between an abstract data syntax and a transfer syntax presented in an open-ended variety of input / output / storage mediums, including, but not limited to random-access memory, disk files, communications sockets, buffer lists, and other static or dynamically-extensible storage structures. The abstract syntax is described using the Abstract Syntax Notation One standard and presented in a higher-level object-oriented programming language such as C++. The transfer syntax is specified by standardized encoding rules and results in a stream of octets (unsigned 8-bit characters) suitable for communication transport using the Open Systems Interconnection (OSI) protocol model. The result of encoding (which is also the input for decoding) is called a protocol data unit (PDU). For different applications, it may be useful to present the PDU using various input / output or storage mediums such as listed above. The disclosed method employs a template function parameterized on an octet output iterator interface to provide an ASN.1 encoding mechanism, and a corresponding template function parameterized on an octet input iterator interface to provide an ASN.1 decoding mechanism, both independent of the medium in which the PDU is presented.

A uniform mechanism for building containment hierarchies separates the code / method which actually perform the containment insertion operations on the components from the specification of the containment and then provides an automatic way of obtaining and executing that code when containment hierarchies need to be created. The mechanism consists of an abstract syntax for describing the containment relationships, processing mechanism for creating the containment mechanism, and a registry containing the mechanisms for implementing specific containment policies. Having this abstraction allows an application programmer to be able to construct hierarchies of objects without having to be concerned about the specific details of how that containment is performed. In fact, the details may be deferred until run-time (and even downloaded from elsewhere). Also, having this abstraction significantly simplifies automatic generation of object hierarchy codes, both from the point of view its implementation as well as that of the code that is generated.

A method for generating visual representations of data based on controlled natural language queries (CNL) using a configured dictionary of keywords (and their synonyms) and data fields (and their synonyms); the method comprising steps of: receiving said query to cause a set of one or more processors to analyse the said query and translating it into an abstract syntax tree (AST); representing the formal interpretation of the CNL query to the user wherein under condition that the CNL query cannot be translated unambiguously into a single AST, multiple abstract syntax trees (ASTs), each representing one possible interpretation of the CNL query, are generated and presented back to the user in their CNL form for the user to select one among the suggestions as the intended query; converting the AST into a database query and executed against the underlying database to obtain the result; providing a visual representation first real result containing a first dataset obtained from the database, the first dataset answering a first sub query that forms part of the main query; provide one or more further visual representation of real result, the or each further real result containing a respective further dataset obtained from the database, the or each further dataset answering a respective further sub-query that forms part of the main query; and obtaining a final visual representation of main query from the dataset from the first and one or more further real result answering the main query.

The invention relates to a program code similarity quick comparison method based on an abstract syntax tree, which relates to the technical field of code reuse. The program code similarity quick comparison method based on the abstract syntax tree is characterized by constructing a program abstract syntax tree, extracting program code features based on the abstract syntax tree, hashing code features, and finally judging code similarity by calculating Hamming distance of code feature hash values. This method takes the abstract syntax tree of program code as the comparison object, and combines Simhash and inverted index technology to transform the similarity comparison of program code into the comparison of hash values of code features. Under the premise of ensuring high accuracy and recall rate, this method can not only realize the fast similarity comparison of program code, but also meet the needs of fast similarity comparison for massive code. This method has better time and space complexity, and can be used to compare the similarity of massive codes in application scenarios, thus providing support for software code reuse and traceability.

A test domain configuration module generates graphical user interfaces for identifying information about desired tests such as data types and domain configurations, and collects information used by other modules to generate tests. The identified information may include, for example, an abstract syntax, a static semantic, max counts on instances of data types, or costs of field accesses or data types for max path costs or max expression costs. A test input generator, generates test input for the identified and configured data types. In one case, the generated test inputs are generated as tree data structures. A predicate determines whether a generated test input follows semantic conditions. A test input evaluator counts instances of data types in, sums paths through, or sums total costs of, the generated test inputs. A test acceptance module saves test inputs acceptable to the predicate and the test input evaluator.

The embodiment of the invention discloses a blood relationship analysis method of a structured query language, and a tool thereof. The blood relationship analysis method comprises the steps: extracting service systems of a plurality of different types of databases, and continuously importing the structured query language of the service systems into a message queue list in batches; preprocessing structured query languages containing static parameters and dynamic parameters in the message queue list respectively; performing grammatical proofreading on the preprocessed structured query language,generating abstract syntax trees matching the structured query languages of the different types of databases, traversing the abstract syntax trees, and obtaining the blood relationship of the structured query languages; and obtaining blood relationship veins and key blood relationship data of the blood relationship, finally forming a blood relationship map, and finally completing analysis and prediction of the service system. According to the embodiment of the invention, dynamic and static structured query languages of various types of databases can be analyzed continuously in batches, and data traceability and prediction of a service system are completed, and an analysis result is accurate.

The invention relates to a page loophole detection device which comprises an abstract syntax tree management module, a symbol table module and a stain backtracking module. The abstract syntax tree management module is used for detecting an abstract syntax tree of a detected program source code, the symbol table module is used for managing symbol table information of the detected program source code, the stain backtracking module is used for reading the abstract syntax tree from the abstract syntax management module, traversing the abstract syntax tree to acquire a set of all pre-configured triggering nodes and backtracking all correlated variables of each triggering node according to the symbol table information, and if correlated variables can be input and changed by an attacker in a controllable manner, existence of loopholes at the triggering nodes is judged, and loophole information is output. By the page loophole detection device, loophole coverage rate can be increased, and false alarm rate can be lowered. In addition, the invention provides a page loophole detection method.

The invention discloses a static detection method for privacy information disclosure in mobile applications. The method comprises the steps as follows: firstly, the completeness of codes is checked, if the codes are incomplete, preparatory work is performed on source codes, state machine description files corresponding to the privacy disclosure are loaded, related state machine information is obtained, the source codes are compiled to obtain detailed information of classes, functions and the like, pre-analysis is performed on the source codes, basic data structures such as abstract syntax trees and the like are generated, and call relations among the functions are obtained; and then detection and analysis is performed in units of the functions, state machine instances are created for all state machines with associated methods in the state machines, state machine instance iteration is performed on control flow diagrams corresponding to the functions, when the state machine instances are in error states, errors are reported, after detection is finished, the detected errors are checked manually, and detection results are confirmed. With the adoption of the method, the privacy disclosure codes existing in application programs of different operating system platforms can be effectively detected, so that the privacy information disclosure of users can be effectively prevented.

The invention belongs to the technical field of computer application, and particularly relates to a code annotation generation method based on structural perception. Hierarchical information of a codefragment is obtained by using an abstract syntax tree, so that a code annotation generator based on structural perception is constructed. According to the model called Code2Text, a source code is combined with an abstract syntax tree of the source code, and the model is applied to the field of code annotation. A dual encoder in the Code2Text encodes the source code sequence input and the abstractsyntax structure input respectively, and each word in the source code and each node in the abstract syntax tree are utilized to accurately adjust the generated word through a dual attention mechanismso as to guide the decoding process. The method can assist a programmer to generate annotations so as to reduce the manual writing cost, and can also provide reference for the programmer without experience to understand existing codes; and the method can be used for application program code detection of data use.

The invention discloses a target variable protocol data unit codec code automatic generation implementation method. Firstly, a target platform characteristic description file is written, the file describes the basic conventional characteristics of the specific target platform, then a PDU description file is written to describe composition of inner elements of PDU, a system automatically analyzes the PDU description file and generates an abstract syntax tree irrelevant to the target platform according to the content thereof, and then the system traverses the abstract syntax tree and generates a data dictionary relevant to the target platform by utilizing the characteristic of the target platform, and finally a code template base is written and PDU codec code directing to the specific target platform is generated. The system applying the method can conveniently generate PDU codec code supporting various target platforms, and especially the target platform and template to generate code can be completely customized without modifying the code generation system.

The invention discloses a time sequence database system based on column storage. The time sequence database system comprises: an SQL (Structured Query Language) parser, wherein the SQL parser is usedfor converting SQL into an abstract syntax tree through parsing; a logic plan generator which is used for generating a logic query plan by utilizing the abstract syntax tree, wherein the logic query plan is expressed by a relation algebra; a query optimizer which is used for optimizing the query plan by utilizing the state information of the database and outputting the rewritten query plan; an executor which is used for executing the query plan in a block iteration mode and is responsible for interacting with the storage layer to obtain data queried by a user; and a storage engine which is used for storing data in a row and column manner. According to the invention, data in the database is stored in a column mode; based on combination of column storage and rocksdb, high-speed insertion data characteristics and an indexing function of a traditional database are achieved, in the aspect of an executor, part of operators are improved to be deeply combined with characteristics of a storageengine, and meanwhile, new characteristics of a modern processor are used for accelerating query.

The invention discloses an automatic grading method of a structured query language (SQL) program, which is characterized in that an SQL program submitted by a learner in the test process is used for carrying out automatic grading so as to provide quantification evaluation indexes on knowledge point mastering and practical ability of a testee. The automatic grading method of the SQL program comprises the following steps: respectively performing standardization on an SQL source program and a template program so as to eliminate expression diversities in SQL semanteme, thus obtaining two standardized SQL programs; respectively carrying out lexical analysis and syntax analysis on the two standardized SQL programs so as to extract two abstract syntax trees of a characteristic programmed structure; calculating the two abstract syntax trees in accordance with a tree edit distance algorithm so as to obtain a similarity value; and calculating final grading results by using a functional relation between the similarity value and the grade.

The invention relates to the technical field of security of computer networks, in particular to a realization method for reasonable construction and correct parse of a communication data package based on an alarm system. Fixed-length data package header identification is used as data package feature identification, the length of a fixed-length data package is set at the rear of the data package header identification, the system can parse the finial size of the package according to the length of the data package after receiving data and then parse out the whole data package, and the size of the whole package can be known without traversing to the end of the package. A coding scheme of abstract syntax markup is used for coding to divide the data into the operation type of Cmd and the operation content of Value, wherein Key corresponds to the header of data package, Length corresponds to the data length, Cmd and Value both correspond to the data, Key, Length, Cmd and Value are coded through the coding and the decoding rule to form a bit stream data package, and the data content is encrypted and decrypted through a special encryption protocol. The security and the transmission speed are obviously promoted.

The invention discloses an intermediate code generation method and device for quantum program compiling. The method comprises the steps of obtaining that a source code of a quantum program is obtained;, wherein the logic structure of the source code at least comprises a quantum branch structure, a quantum circulation structure and a quantum logic gate; carrying out lexical analysis on the source code, and segmenting the source code into a plurality of grammatical unit tokens; performing grammatical analysis on all tokens, and constructing an abstract syntax tree AST; wherein the AST comprisesa root node, a plurality of sub-tree nodes and a plurality of leaf nodes, the root node and the sub-tree nodes comprise grammar rule information, and the leaf nodes comprise token information; and according to the node information of the AST and a preset construction rule, constructing an intermediate code corresponding to the source code. By utilizing the embodiment of the invention, repeated labor can be avoided, and resource occupation is reduced.

An efficient method for parsing HTML pages identifies pages containing a mix of static and dynamic content. The pages are parsed to form abstract syntax trees (ASTs), which are then cached along with the pages. When a later version of a page is retrieved, it is compared against the cached version, and only those portions of the AST that contain different content are reparsed.