Multimodal data query extension method and system, computer device

Abstract

The application provides a multi-modal data query expansion method and system and a computer device. The multi-modal data query expansion method comprises the following steps: receiving a query sentence defined by an extended SQL syntax; performing lexical analysis on the query sentence to identify lexical units; wherein the lexical units comprise keywords, identifiers and operators; constructing an abstract syntax tree according to the lexical units; performing semantic verification on the abstract syntax tree; after the semantic verification is passed, traversing the abstract syntax tree, calling pre-defined operator metadata, a type system and a function library of a multi-modal database to optimize and rewrite the abstract syntax tree to obtain a rewritten query tree; the type system comprises various data types of multi-modal data and type conversion rules thereof, the operator metadata comprises operators and priority rules thereof, and the function library comprises query operation functions; generating an execution plan according to the query tree; executing the query and returning a query result.

Description

Technical Field

[0001] This invention relates to the field of big data technology, specifically to a multimodal data query expansion method and system, and computer equipment. Background Technology

[0002] With the rapid development of artificial intelligence and big data technologies, the types of data that application systems need to process are becoming increasingly diverse, including multiple modalities such as text, images, videos, graph structure data, and time series data.

[0003] Traditional relational database query languages ​​(SQL) are primarily designed for structured data. When faced with queries on multimodal data, they suffer from the following problems: For different modal data types, developers need to write corresponding dedicated query languages, such as Cypher for graph data queries and InfluxQL for time series data queries. This fragmented query language results in low execution efficiency and poor application portability. Summary of the Invention

[0004] To address the shortcomings of existing methods, embodiments of the present invention provide a unified, efficient, and highly compatible multimodal data query extension method and system, as well as a computer device.

[0005] To solve the above problems, the embodiments of the present invention are implemented through the following technical solutions: On the one hand, a multimodal data query extension method is provided, including: Receives query statements defined by extended SQL syntax; Lexical analysis is performed on the query statement to identify lexical units; wherein, the lexical units include keywords, identifiers, and operators; Construct an abstract syntax tree based on the lexical units; Perform semantic validation on the abstract syntax tree; After semantic validation passes, the abstract syntax tree is traversed, and the predefined operator metadata, type system, and function library of the multimodal database are called to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree. The type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions. Based on the query tree, an execution plan is generated; Execute the query and return the query results.

[0006] On the other hand, a multimodal data query extension system is provided, including: The acquisition module is used to receive query statements defined by extended SQL syntax. The lexical analysis module is used to perform lexical analysis on the query statement and identify lexical units; wherein, the lexical units include keywords, identifiers, and operators; A syntax extension module is used to construct an abstract syntax tree based on the lexical units; The semantic module is used to perform semantic verification on the abstract syntax tree; The rewrite module is used to traverse the abstract syntax tree after semantic verification passes, and call the predefined operator metadata, type system and function library of the multimodal database to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree; the type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions; The execution module is used to generate an execution plan based on the query tree; The results module is used to execute queries and return the query results.

[0007] In another aspect, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the multimodal data query extension method described in any embodiment of this application.

[0008] The multimodal data query extension method provided in the above embodiments seamlessly extends the query language defined by the extended SQL syntax, providing a unified syntax to support the expression of query statements for multimodal data. For the unified query statement, a consistent paradigm suitable for multimodal data query is formed from lexical, syntactic, semantic, rewriting to execution, realizing efficient fusion query of multimodal data with high execution efficiency and strong compatibility. The extended query process adopts a modular design, which makes it easier to quickly integrate new modal data and is easy to maintain and upgrade.

[0009] In the above embodiments, the multimodal data query expansion system and computer device belong to the same concept as the corresponding multimodal data query expansion method embodiments, and thus have the same technical effects as each multimodal data query expansion method embodiment, which will not be repeated here. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating an optional application scenario of the multimodal data query extension method provided in one embodiment.

[0011] Figure 2 A flowchart of a multimodal data query extension method provided in one embodiment.

[0012] Figure 3This is a schematic diagram of the structure of a multimodal data query extension system provided in one embodiment. Detailed Implementation

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] In the following description, the phrase "some embodiments" refers to a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0016] In the following description, the terms "first, second, and third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0017] Please see Figure 1 One optional application scenario for the multimodal data query extension method provided in this application is a database system. This database system includes an application layer 11, a query processing layer 12, a type system 13, an operator layer 14, and a function library 15. The type system 13, operator layer 14, and function library 15 refer to a predefined query execution operator system based on supporting multimodal data query extension. This system is enhanced at the database's built-in semantic primitive layer to achieve unified expression and computation of multimodal data. The type system 13 includes the various data types of multimodal data and their type conversion rules; the operator layer 14 contains operator metadata; and the function library 15 contains query operation functions corresponding to different data types. The application layer 11 provides an input interface for query statements that support extended SQL syntax definitions. The query processing layer 12 is used to execute the query processing flow of the extended SQL syntax definition from lexical, syntactic, semantic, rewriting to execution. In the execution of the query processing flow, the type system 13, operator metadata and function library 15 are called to support the implementation of query operations for different data modalities, forming a consistent paradigm suitable for multimodal data queries.

[0018] In an optional example, in type system 13, type data and its type conversion rules include: scalar types, such as basic data types like INTEGER (integer), TEXT (text), and FLOAT (floating-point number); vector types, such as VECTOR (vector) and FLOAT[] (floating-point array); graph types, such as graph structure data like GRAPHPATH (graph path, represented by INTEGER[] for node sequence); and time-series types, such as TIMEWINDOW, TIMESTAMP (timestamp), and INTERVAL (time interval). Type conversion rules define the conversion logic and compatibility conditions between different data types (such as vector to scalar, graph to time-series, etc.).

[0019] In operator layer 14, operator metadata includes various operators and their precedence and associativity rules: Vector operators: such as <#> negative inner product, <-> Euclidean distance, <=> cosine distance, and including but not limited to: <> (not equal to), <= (less than or equal to), >= (greater than or equal to), used for vector comparisons. Graph operators: such as -[]-> (directed edges from left to right), <-[]- (directed edges from right to left), ~= (graph similarity), and including but not limited to: <{}> (graph contains), {}< (graph is contained), used for graph structure operations. Temporal operators: such as OVERLAPS (overlap), PRECEDEES (before ...), FOLLOWS (follow), MEETS (connect), etc., used for temporal interval operations. Mixed operators: such as WITHIN (within), INTERSECTS (intersect), FUSION (fusion), CONTAINS (contain), etc., used for cross-modal operations. Operator precedence and associativity rules: Define the execution order (e.g., multiplication and division before addition and subtraction) and associativity (left associativity / right associativity) of various operators.

[0020] In function library 15, query functions include: Vector functions: such as VECTOR() (vector construction), SIMILARITY() (similarity calculation), and DISTANCE() (distance calculation). Graph functions: such as PATH() (path construction), NEIGHBORS() (neighbor query), and CONNECTED() (connectivity determination). Time-series functions: such as WINDOW() (window partitioning), SLIDE() (sliding window), and BUCKET() (bucketing). Aggregate functions: such as FUSE() (merge aggregation), MERGE() (merge aggregation), and COMBINE() (combined aggregation).

[0021] Please see Figure 2 A multimodal data query expansion method provided in one embodiment of this application includes: S101 receives query statements defined by extended SQL syntax.

[0022] Extended SQL syntax defines query statements as those written using enhanced SQL syntax. While maintaining compatibility with standard SQL, they introduce a declarative query interface that accepts multimodal data and special operators, allowing the mixing of multimodal data, special operators, and primitives within the same query.

[0023] In a specific example, multimodal data types include vector data, graph data, time-series data, and hybrid data. Vector data types are primarily used to store and manage arrays of high-dimensional numerical values, such as feature vectors and spatial coordinates. Graph data types are specific pattern types of data that are not single-field types, typically defined through a set of related table structures, such as vertex types, edge types, and graph patterns. Time-series data types are mainly sequences of numerical values ​​or events with timestamps, such as time fields, time series values, and time windows. Hybrid data refers to data containing more than one data type in a single query statement. Multimodal data types also include data types supported by standard SQL syntax, such as scalar data and structured data. Extended SQL syntax extensions include: syntax for defining vector data types and their literals; syntax for defining graph patterns, vertex tables, and edge tables for graph traversal queries; syntax for defining time windows and performing time-series aggregation and overlap determination; and syntax for expressing multimodal operators for vector similarity comparison, graph path finding, and time window overlap.

[0024] S102, perform lexical analysis on the query statement to identify lexical units; wherein, the lexical units include keywords, identifiers and operators.

[0025] Lexical analysis refers to the process of segmenting a query statement into its smallest semantically specific units, known as lexical tokens, to identify keywords, identifiers, and operators within the query. Keywords represent the query intent, such as SELECT, FROM, and WHERE; identifiers refer to database objects, such as table names, column names, and view names; and operators are function names used to invoke specific processing logic to perform calculations.

[0026] The lexical analysis step aims to transform unstructured SQL queries into structured sequences of lexical symbols. Lexical analysis segments the input string, identifying lexical units with independent semantics. These units define the basic components of the query, including but not limited to: keywords controlling query logic, identifiers pointing to specific data entities, operators defining relationships between data, and function identifiers performing specific calculations. Furthermore, lexical analysis encapsulates special literals such as string constants and numerical constants into corresponding token types, thus providing a standardized input interface for subsequent syntax tree construction.

[0027] S103, construct an abstract syntax tree based on the lexical units.

[0028] Constructing an abstract syntax tree (AST) refers to organizing lexical units into a tree-like structure that reflects the query hierarchy and operational relationships of SQL syntax. Specifically, it involves parsing the lexical unit stream obtained from lexical analysis according to SQL syntax rules, sequentially reading lexical units to trigger the generation of corresponding node types, determining node attributes and logical relationships, and constructing a hierarchical structure.

[0029] S104, Perform semantic verification on the abstract syntax tree.

[0030] Semantic validation refers to checking the logical validity of data to ensure that the abstract syntax tree built based on the query statement has been translated into the database's internal language. The semantic validation step aims to perform a rigorous security check; if logical inconsistencies are detected, the processing flow can be interrupted, and an error message object, generated based on the error type, location, and cause description, can be sent back to the user for correction. Logical inconsistencies can primarily include: data type mismatch, non-existent objects, dimension mismatch, semantic ambiguity, or illegal function operations.

[0031] S105, after the semantic verification passes, the abstract syntax tree is traversed, and the predefined operator metadata, type system and function library of the multimodal database are called to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree; the type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions.

[0032] Optimization rewriting refers to rewriting a query to make it faster and more resource-efficient without changing the query results. A query tree is a hierarchical structure resulting from traversing the nodes of an abstract syntax tree and adjusting, replacing, or simplifying them based on optimization principles. In essence, an abstract syntax tree is a standard roadmap obtained by parsing a query statement based on SQL syntax rules, while optimization rewriting, by calling predefined operator metadata, type systems, and function libraries in a multimodal database, utilizes built-in equivalence transformation rules, such as filtering before joining and flattening nested queries into simple joins, to find a faster and more resource-efficient path without changing the query destination.

[0033] S106, Generate an execution plan based on the query tree.

[0034] An execution plan is a plan that takes the query logic expressed by the query tree, such as the type of each field (scalar, vector, graph, or time series), which tables to query (Cartesian product, join condition), and the meaning of operators (Euclidean distance, cosine similarity), determines the execution order and constraints, and finally outputs a plan that can be specifically executed.

[0035] S107, execute the query and return the query results.

[0036] The query is executed according to the execution plan, the query results are obtained and returned. The query results may be a part or all of the results retrieved based on the execution plan that meet the preset conditions; this application does not impose any restrictions on this.

[0037] The multimodal data query extension method provided in the above embodiments, by extending the query language defined by the SQL syntax, seamlessly extends the standard SQL and provides a unified syntax to support the expression of query statements for multimodal data. For the unified query statement, from lexical, syntactic, semantic, rewriting to execution, a consistent paradigm suitable for multimodal data query is formed, realizing efficient fusion query of multimodal data, with high execution efficiency and strong compatibility. The extended query process adopts a modular design, which makes it easier to quickly integrate new modal data and is easy to maintain and upgrade.

[0038] In some embodiments, step S103 includes: Based on SQL syntax rules, a recursive descent algorithm is used to traverse the lexical units, create corresponding nodes according to the type of the lexical units, and construct a binary expression subtree through parsing operators to generate an abstract syntax tree.

[0039] Syntax parsing follows SQL grammar rules, employing a top-down recursive function call approach to parse lexical units one by one and construct nodes. In the lexical stream obtained from query lexical parsing, each lexical unit carries type information (Keyword, Identifier, Operator, Constant) and a text value. The database's SQL grammar rules predefine multiple grammar node types. Based on the type of the current lexical unit, the corresponding node object is dynamically instantiated, and adjacent nodes are connected into a tree structure using recognition operators.

[0040] In the above embodiments, a recursive descent algorithm is used to translate linearly arranged lexical units into a hierarchical structure that can be understood by computer devices. Node creation is mainly based on the identification of entities (tables, columns, values), and the construction of binary expressions is mainly based on the identification of relations (greater than, equal to, vector distance). The combination of these two ensures that multimodal query statements (such as vector similarity) can be accurately transformed into a structured abstract syntax tree, providing a foundation for subsequent semantic verification and query optimization.

[0041] In some embodiments, step S104 includes: The abstract syntax tree is semantically validated to determine whether the semantic logic meets the requirements. If the requirements are not met, the current query will be interrupted.

[0042] Semantic validation is performed on the abstract syntax tree. If the semantic logic validation fails, the query is terminated early to avoid database errors during the execution phase.

[0043] Optionally, performing semantic validation on the abstract syntax tree to determine whether the semantic logic meets the requirements includes: Semantic validation of the abstract syntax tree includes at least one of the following: determining whether the data type corresponding to each node matches, whether the object exists, whether the dimension corresponding to the data type is accurate, and whether the function corresponding to the data type matches. If any one fails, it is determined that the semantic logic does not meet the requirements.

[0044] The semantic logic judgment mainly includes: whether the data types corresponding to each node match, such as WHERE face vector > 0.5, where > is a label comparison operator, and face vector (VECTOR type) does not match. Whether the object exists, such as SELECT avatar features FROM user profile, where the table (user profile) exists, but the field name "avatar features" is not defined, and the object does not exist. Whether the data type corresponds to the dimension accurately, such as SELECT * FROM product WHERE product vector <-> [0.1, 0.2] < 0.3, where product vector is redefined as VECTOR(128) in the type system, while the input [0.1, 0.2] is only a two-dimensional constant, and the vector dimension does not match. Whether the data type corresponds to the function match, such as SELECT SUM(social relationship path) FROM user interaction, where social relationship path is of type GRAPHPATH (graph path), SUM() is a scalar aggregation function, which cannot sum graph paths, and the data type corresponds to the function does not match.

[0045] In the above embodiments, by traversing the abstract syntax tree and validating the logic of the initial query path, errors can be corrected in advance, avoiding query errors and wasting resources.

[0046] In some embodiments, step S105, traversing the abstract syntax tree and calling predefined operator metadata, type system, and function library of the multimodal database to optimize and rewrite the abstract syntax tree to obtain a rewritten query tree, includes: The abstract syntax tree is traversed, and the predefined operator metadata, type system and function library of the multimodal database are called. Based on the equivalent transformation rules and the cost estimation, the nodes in the abstract syntax tree are adjusted, replaced and / or simplified to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree.

[0047] Equivalent transformation rules refer to query logic expressed by an abstract syntax tree that does not change the query destination. Cost estimation-based rules refer to query paths that are faster and more resource-efficient without changing the query destination.

[0048] Optionally, the adjustment, replacement, and / or transformation of nodes in the abstract syntax tree based on equivalent transformation rules and cost estimation includes at least one of the following: The priority of nodes is adjusted so that filtering conditions take precedence over connection conditions; For calculations that result in constants, the nodes are updated by replacing the expression with the result. Operators for specific data types are converted into dedicated vector index retrieval functions to update nodes; For a nested query node, the node is updated by removing the nested subquery; Expand the view / macro in the query node to replace the corresponding placeholder, so as to update the node; Adjust the nodes in the abstract syntax tree based on preset operator priority rules.

[0049] With the priority of filter conditions higher than that of join conditions, moving filter conditions forward can filter out most useless data first, thereby reducing the calculation amount for executing join conditions. For example, for SELECT * FROM orders JOIN user_behavior ON... WHERE face_vector<->[0.1 ...]<0.5, after optimization and rewriting, the WHERE face vector filtering is first performed on the user behavior table to screen out hit data, and then the hit data is used to join the order table, which can greatly reduce the calculation amount.

[0050] For calculations whose results are constants, rewriting the expression by replacing it with the result can avoid repeated calculations required to be performed by the database every time during the subsequent query execution process. For example, in SELECT * FROM monitoring_logs WHERE collection_time>NOW() -INTERVAL '7' DAY AND dimension = 3 + 5, dimension= 3 + 5 is directly replaced with 8, and NOW() -INTERVAL '7' DAY is replaced with a fixed timestamp at the beginning of the query. After rewriting, the condition becomes that the collection time is the fixed timestamp and the dimension is 8, which can eliminate the repeated calculation of fixed results for each piece of data at runtime.

[0051] For operators of specific data types, update the node by converting the operator into a special vector index retrieval function. For operators of specific data types, such as newly added multimodal operators including vector distance and graph path matching, they can be bound to the corresponding special index scan to avoid full-table calculation. For example, in SELECT id FROM image_library WHERE feature_vector L2_DISTANCE(query_vector)<0.3, the special vector index function HNSW_IVF_INDEX can only probe adjacent nodes through the established graph index. After rewriting, the statement becomes SELECT id FROM image_library WHERE HNSW_IVF_INDEX(feature_vector, query_vector, threshold=0.3), which can reduce the full-table scan with O(N) complexity to index search with O(logN) complexity, effectively reducing the calculation amount.

[0052] For nested query nodes, removing nested subqueries and updating the nodes reduces the creation of temporary tables. For example, `SELECT * FROM product WHERE category ID IN (SELECT category ID FROM hot-selling category WHERE sales>1000)` can be rewritten using the optimal join order as `SELECT product.* FROM product JOIN hot-selling category ON product.categoryID = hot-selling category.categoryID WHERE hot-selling category.sales>1000`. This transforms the nested query into a simple join query, avoiding the generation of temporary intermediate tables.

[0053] Expanding a view / macro in a query node replaces the corresponding placeholders. A view stores the query results, a macro stores code snippets, and a placeholder is a marker or symbol in code, template, or statement that occupies a fixed position and is dynamically replaced by the actual content. By expanding a view / macro, the expanded content replaces the corresponding placeholder (view / macro name), eliminating intermediate layers in the query. For example, the macro definition is: DEFINE MACRO get_user(uid) AS SELECT * FROM user_table WHERE user_ID = uid, and the query statement is SELECT * FROM get_user(123). By expanding the macro and replacing the placeholders, it is rewritten as SELECT * FROM user_table WHERE user_ID = 123, which eliminates the need for macro references in the query and improves efficiency.

[0054] Based on preset operator priority rules, the nodes in the abstract syntax tree are adjusted. In the operator layer, operator metadata includes various operators, their priorities, and associativity rules. By using preset operator priority rules to adjust the nodes in the abstract syntax tree, high-priority operations are prioritized, thereby optimizing the logical query plan. For example, type conversion is performed before joins; filtering conditions are executed before vector operations; vector operations are executed before graph operations, etc., which can reduce subsequent computational load. In a specific example, the various operators, their priorities, and associativity rules are shown in the table below: In the above embodiments, the predefined operator metadata, type system, and function library of the multimodal database are invoked. The abstract syntax tree is rewritten and optimized based on the equivalent transformation rules and cost estimation. The type conversion operator has the highest priority, the logical OR operator has the lowest priority, and the priority of vector operators, graph operators, and temporal overlap operators is between that of arithmetic operators and comparison operators. This can identify the semantics of multimodal operations and optimize the execution plan of multimodal operations.

[0055] In some embodiments, in step S101, the query statement defined by the extended SQL syntax includes at least one of the following: Syntax used to define vector data types and their lengths; Syntax used to define graph patterns, vertex tables, and edge tables for graph traversal queries; Syntax used to define time windows and perform time series aggregation and overlap detection; Syntax for defining multimodal operators for expression vector similarity comparison, graph path finding, and time window overlap.

[0056] Extended SQL syntax defines query statements for different data types of multimodal data to provide a unified SQL syntax that supports syntactic consistency in multimodal data query operations. In this embodiment, the input syntax for query statements of multimodal data such as vectors, graphs, and time series is defined to ensure compatibility with basic SQL queries and support complex joint queries of multimodal data, as well as rapid integration of query operations for new types of data.

[0057] Optionally, in the multimodal data query extension method provided in the foregoing embodiments, the data types of multimodal data include: vector data types, graph data types, time-series data types, standard SQL types, and a mixture of two or more of these; wherein, the standard SQL types include at least scalar types and structured types. Seamless extension based on standard SQL provides a unified syntax for querying multiple modal data such as vectors, graphs, and time series data, fully compatible with standard SQL queries. Applications can be smoothly migrated, introducing multimodal data query functionality. The unified syntax standard also helps improve the portability of applications across different database platforms.

[0058] In the above description of the embodiments, there is no strict order between the steps involved in the method. The specific order or sequence can be interchanged if there is no logical conflict, so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated or described herein.

[0059] Please see Figure 3 In another aspect, this application provides a multimodal data query extension system, comprising: Module 21 is used to receive query statements defined by extended SQL syntax. Lexical analysis module 22 is used to perform lexical analysis on the query statement and identify lexical units; wherein, the lexical units include keywords, identifiers and operators; The syntax extension module 23 is used to construct an abstract syntax tree based on the lexical units; Semantic module 24 is used to perform semantic verification on the abstract syntax tree; The rewrite module 25 is used to traverse the abstract syntax tree after the semantic verification passes, and call the predefined operator metadata, type system and function library of the multimodal database to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree; the type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions; Execution module 26 is used to generate an execution plan based on the query tree; Result module 27 is used to execute queries and return query results.

[0060] Optionally, the syntax extension module 23 is specifically used to traverse the lexical units according to the SQL syntax rules using a recursive descent algorithm, create corresponding nodes according to the type of the lexical units, and construct a binary expression subtree through parsing operators to generate an abstract syntax tree.

[0061] Optionally, the semantic module 24 is specifically used to perform semantic verification on the abstract syntax tree to determine whether the semantic logic meets the requirements; if the requirements are not met, the current query is interrupted.

[0062] Optionally, the semantic module 24 is further configured to perform semantic verification on the abstract syntax tree, including at least one of the following: determining whether the data type corresponding to each node matches, whether the object exists, whether the dimension corresponding to the data type is accurate, and whether the function corresponding to the data type matches; if any one fails, it is determined that the semantic logic does not meet the requirements.

[0063] Optionally, the rewriting module 25 is specifically used to traverse the abstract syntax tree, call the predefined operator metadata, type system and function library of the multimodal database, and adjust, replace and / or simplify the nodes in the abstract syntax tree based on the equivalent transformation rules and the cost estimation, so as to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree.

[0064] Optionally, the rewriting module 25 adjusts, replaces, and / or transforms nodes in the abstract syntax tree based on equivalent transformation rules and cost estimation, including at least one of the following: adjusting the priority of nodes so that filtering conditions have higher priority than joining conditions; updating nodes by replacing the expression with the result for calculations that result in constants; updating nodes by converting operators for specific data types into dedicated vector index retrieval functions; updating nodes by removing nested subqueries for nested query nodes; updating nodes by expanding views / macros in query nodes to replace the corresponding placeholders; and adjusting nodes in the abstract syntax tree based on preset operator priority rules.

[0065] Optionally, the acquisition module 21 extends the query statements defined by the SQL syntax, including at least one of the following: syntax for defining vector data types and their lengths; syntax for defining graph patterns, vertex tables, and edge tables for graph traversal queries; syntax for defining time windows and performing time-series aggregation and overlap judgment; and syntax for defining multimodal operators for expressing vector similarity comparison, graph path finding, and time window overlap.

[0066] Optionally, the data types of multimodal data include: vector data types, graph data types, time-series data types, standard SQL types, and a mixture of two or more of these; wherein the standard SQL types include at least scalar types and structured types.

[0067] It should be noted that the multimodal data query extension system provided in the above embodiments is only illustrated by the division of the above program modules during the execution of multimodal data queries. However, in practical applications, the above processing flow can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. In addition, the multimodal data query extension system and the multimodal data query extension method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the corresponding method embodiments, which will not be repeated here.

[0068] In another aspect, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the multimodal data query expansion method described in any embodiment of this application and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0069] It should be noted that, in this document, the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0070] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for expanding multimodal data queries, characterized in that, include: Receives query statements defined by extended SQL syntax; Lexical analysis is performed on the query statement to identify lexical units; wherein, the lexical units include keywords, identifiers, and operators; Construct an abstract syntax tree based on the lexical units; Perform semantic validation on the abstract syntax tree; After semantic validation passes, the abstract syntax tree is traversed, and the predefined operator metadata, type system, and function library of the multimodal database are called to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree. The type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions. Based on the query tree, an execution plan is generated; Execute the query and return the query results; The step of traversing the abstract syntax tree and calling the predefined operator metadata, type system, and function library of the multimodal database to optimize and rewrite the abstract syntax tree to obtain a rewritten query tree includes: traversing the abstract syntax tree, calling the predefined operator metadata, type system, and function library of the multimodal database, and adjusting, replacing, and / or simplifying the nodes in the abstract syntax tree based on equivalent transformation rules and cost estimation to optimize and rewrite the abstract syntax tree to obtain a rewritten query tree.

2. The multimodal data query expansion method according to claim 1, characterized in that, The step of constructing an abstract syntax tree based on the lexical units includes: Based on SQL syntax rules, a recursive descent algorithm is used to traverse the lexical units, create corresponding nodes according to the type of the lexical units, and construct a binary expression subtree through parsing operators to generate an abstract syntax tree.

3. The multimodal data query expansion method according to claim 2, characterized in that, The semantic validation of the abstract syntax tree includes: The abstract syntax tree is semantically validated to determine whether the semantic logic meets the requirements. If the requirements are not met, the current query will be interrupted.

4. The multimodal data query expansion method according to claim 3, characterized in that, The step of performing semantic validation on the abstract syntax tree to determine whether the semantic logic meets the requirements includes: Semantic validation of the abstract syntax tree includes at least one of the following: determining whether the data type corresponding to each node matches, whether the object exists, whether the dimension corresponding to the data type is accurate, and whether the function corresponding to the data type matches. If any one fails, it is determined that the semantic logic does not meet the requirements.

5. The multimodal data query expansion method according to claim 1, characterized in that, The adjustment, replacement, and / or transformation of nodes in the abstract syntax tree based on equivalent transformation rules and cost estimation includes at least one of the following: The priority of nodes is adjusted so that filtering conditions take precedence over connection conditions; For calculations that result in constants, the nodes are updated by replacing the expression with the result. Operators for specific data types are converted into dedicated vector index retrieval functions to update nodes; For nested query nodes, remove the nested subqueries and update the nodes; Expand the view / macro in the query node to replace the corresponding placeholders, and update the node; The nodes in the abstract syntax tree are adjusted based on the preset operator priority rules.

6. The multimodal data query expansion method according to any one of claims 1 to 5, characterized in that, Query statements that extend the SQL syntax definition include at least one of the following: Syntax used to define vector data types and their lengths; Syntax used to define graph patterns, vertex tables, and edge tables for graph traversal queries; Syntax used to define time windows and perform time series aggregation and overlap detection; Syntax for defining multimodal operators for expression vector similarity comparison, graph path finding, and time window overlap.

7. The multimodal data query expansion method according to any one of claims 1 to 5, characterized in that, The data types of multimodal data include: vector data types, graph data types, time-series data types, standard SQL types, and a mixture of two or more of these; wherein, the standard SQL types include at least scalar types and structured types.

8. A multimodal data query extension system, characterized in that, include: The acquisition module is used to receive query statements defined by extended SQL syntax. The lexical analysis module is used to perform lexical analysis on the query statement and identify lexical units; wherein, the lexical units include keywords, identifiers, and operators; A syntax extension module is used to construct an abstract syntax tree based on the lexical units; The semantic module is used to perform semantic verification on the abstract syntax tree; The rewrite module is used to traverse the abstract syntax tree after semantic verification passes, and call the predefined operator metadata, type system and function library of the multimodal database to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree; the type system includes the data types of multimodal data and their type conversion rules, the operator metadata includes operators and their priority rules, and the function library includes query operation functions; The execution module is used to generate an execution plan based on the query tree; The results module is used to execute queries and return query results; The rewriting module is used to traverse the abstract syntax tree, call the predefined operator metadata, type system and function library of the multimodal database, and adjust, replace and / or simplify the nodes in the abstract syntax tree based on the equivalent transformation rules and the cost estimation, so as to optimize and rewrite the abstract syntax tree to obtain the rewritten query tree.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the multimodal data query extension method as described in any one of claims 1 to 7.

Images