Intelligent Search System for Business Objects Based on Metadata

Abstract

This invention relates to the fields of computer data processing and database technology, specifically to an intelligent search system for business objects based on metadata. The system includes: a context request parsing module: receiving abstract search requests and extracting environment parameters and permission features to generate a dynamic context snapshot; a logical structure construction module: combining snapshot execution rule injection operations to generate a logical abstract syntax tree containing implicit filtering conditions; a multi-dialect compilation and execution module: translating the syntax tree into physical execution statements matching the target storage engine dialect according to physical mapping rules; and an adaptive optimization feedback module: monitoring execution latency data and dynamically correcting the index strategy definition. This invention solves the maintenance bottleneck caused by the tight coupling between business logic and physical storage in heterogeneous data source environments, achieving zero-code modification and hot deployment of business logic.

Description

Technical Field

[0001] This invention relates to the fields of computer data processing and database technology, specifically to an intelligent search system for business objects based on metadata. Background Technology

[0002] With the deepening of enterprise digital transformation and the exponential expansion of e-commerce platform business scale, intelligent search systems for business objects have become a core component supporting complex enterprise-level applications and high-concurrency promotional scenarios. In order to achieve efficient data retrieval and refined permission management, the architecture design and performance optimization of the search system have become particularly important.

[0003] In existing search system architectures, there is a common reliance on specific physical storage engines, such as relational or document-oriented databases, to handle business requests. This traditional development model often results in a high degree of coupling between upper-layer business logic and underlying physical storage. If a change in storage architecture is required due to data growth, significant code refactoring costs will be incurred. Furthermore, when dealing with heterogeneous data sources involving multi-dialect environments, relying solely on the database's native optimizer is insufficient to effectively handle mixed queries involving complex multi-table joins, time window constraints, and geofencing limitations. This can easily lead to inaccurate execution plan estimations, resulting in severe performance degradation. In multi-tenant environments, how to securely and transparently integrate dynamic contextual features such as user permissions and organizational structure into the query process is also a challenge that current technologies struggle to address.

[0004] Therefore, it is crucial to improve system maintenance efficiency and ensure query performance under massive data by constructing a logical structure that can decouple business logic from physical storage, automatically build a logical structure with implicit filtering conditions through metadata-driven approach combined with runtime environment parameters, and adaptively generate efficient physical execution statements that adapt to different storage dialects. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an intelligent search system for business objects based on metadata. Specifically, the technical solution of this invention includes:

[0006] The context request parsing module is configured to receive abstract search requests decoupled from the physical storage engine and extract current runtime environment parameters and user permission characteristics to generate a dynamic context data snapshot containing a set of key-value pairs.

[0007] The logical structure construction module is configured to load a preset metadata graph. The metadata graph is a data structure that describes the mapping relationship between the logical structure of business objects and physical storage. It combines dynamic context data snapshots to perform rule matching on abstract search requests and generate a logical abstract syntax tree containing implicit filtering conditions.

[0008] The multi-dialect compilation and execution module is configured to translate the logical abstract syntax tree into physical execution statements that match the dialect of the target storage engine, based on the physical mapping rules and data distribution characteristics defined in the metadata graph.

[0009] The adaptive optimization feedback module is configured to monitor the execution latency data of physically executed statements and dynamically correct the index strategy definition in the metadata graph based on the query frequency statistics.

[0010] Preferably, the multi-dialect compiler and execution module is specifically used for:

[0011] Read the pre-defined field cardinality features and index path definitions in the metadata graph;

[0012] Based on the cardinality characteristics of the fields, the connection order and driving strategy of each query node in the logical abstract syntax tree are determined to generate optimized physical execution statements.

[0013] Preferably, the multi-dialect compilation and execution module determines the linking order and driving strategy, specifically configured as follows:

[0014] When the logical abstract syntax tree involves multi-table join queries, obtain the field selectivity values ​​for each filter node;

[0015] If the field selectivity value of a certain filter node is detected to be higher than the preset selectivity threshold, the physical table corresponding to the filter node is determined as the driving table, and the table join sequence is rewritten according to the physical mapping rules.

[0016] Preferably, the logical structure construction module is specifically used for:

[0017] Traverse the pre-defined context rule base in the metadata graph;

[0018] Extract the constraints that match user permission features in the dynamic context snapshot into implicit predicates;

[0019] The implicit predicates and explicit filtering conditions in the abstract search request are logically ANDed together to construct a logical abstract syntax tree.

[0020] Preferably, the multi-dialect compiler execution module translates the logical abstract syntax tree, including:

[0021] When the target storage engine is identified as a relational database, the visitor pattern is applied to traverse the logical abstract syntax tree and generate structured query language (SQL) statements.

[0022] When the target storage engine is identified as a document-oriented database, the logical abstract syntax tree is mapped to a JSON structure of a domain-specific query language (DSL) for document storage.

[0023] When the target storage engine is identified as a graph database, the logical abstract syntax tree is converted into graph query language statements.

[0024] Preferably, the adaptive optimization feedback module modifies the indexing strategy definition, specifically configured as follows:

[0025] Calculate the numerical deviation between the execution latency data and the preset baseline response time;

[0026] If the numerical deviation is greater than the preset performance degradation threshold, then identify the query path in the current physical execution statement whose access count exceeds the preset frequency threshold within the statistical period.

[0027] Based on the query path, update the query priority weight values ​​of relevant attributes in the metadata graph or write the composite index suggestion flag.

[0028] Preferably, the metadata map includes the following data definition structure:

[0029] Business semantic atomic definition is used to store the attribute types and logical relationship topology data of business objects;

[0030] View strategy definition is used to store field display rule parameters under different business scenarios;

[0031] Storage mapping is defined as a mapping table that stores logical attributes to physical storage fields and storage engine characteristic parameters.

[0032] Preferred options also include:

[0033] The virtual wide table mapping module is used to map the multi-entity association model in the physical storage layer into a flattened virtual wide table structure based on the logical association topology in the metadata graph before the logical structure construction module generates the logical abstract syntax tree, so that the context request parsing module can identify objects.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention effectively solves the maintenance bottleneck caused by the tight coupling between business logic and physical storage in heterogeneous data source environments by constructing a decoupled architecture of physical storage engine and utilizing the collaborative work of the context request parsing module and the multi-dialect compilation and execution module. Through visitor pattern, domain-specific query language mapping and graph query pattern matching technology in the multi-dialect compilation and execution module, it realizes automatic translation of logical abstract syntax tree to various physical dialects such as relational database, document database and graph database, supporting the smooth coexistence of legacy systems and new generation architectures. When the underlying storage architecture changes, the system can shield the cost of code refactoring, and developers do not need to master multiple database dialects to operate complex data, which greatly reduces the threshold of technology stack migration and reserves interface channels for the flexible evolution of the underlying storage architecture.

[0036] 2. This invention effectively avoids the risk of full table scans caused by inaccurate estimations in the native database optimizer under complex join queries by introducing cost-based optimization logic at the compilation layer and combining the field cardinality features and query priority weights maintained in real time in the metadata graph. By automatically calculating the high selectivity threshold using the maximum inter-class variance method and identifying the filter node based on the threshold to lock the driving table, the system enforces a strategy of small table driving large table or high filtering priority, significantly reducing input / output overhead and memory usage. The solution achieves the integration of rule optimization and cost optimization by combining the calculation of priority scores and the rewriting of table join sequences, ensuring that complex report queries can still maintain sub-second response capabilities in e-commerce promotions or multi-dimensional analysis of massive data.

[0037] 3. This invention achieves zero-code modification and hot deployment of business logic by abstracting business rules into a metadata graph and combining it with a dynamic context injection mechanism, ensuring the strictness of data isolation in a multi-tenant environment. By constructing a logical structure module that traverses the context rule base, user permission features are instantiated into implicit predicates and logically combined with explicit filtering conditions, achieving physical separation of application logic and security policies. This mechanism ensures that even if the front-end code has vulnerabilities or is maliciously tampered with, the query statements generated by the back-end still strictly follow the principle of least privilege, effectively preventing the risk of data leakage caused by unauthorized access and solving the problem of scattered and difficult-to-maintain permission control logic in traditional development models.

[0038] 4. This invention establishes a performance optimization closed loop based on feedback control theory. It utilizes an adaptive optimization feedback module to continuously collect the execution time of physical statements and calculate performance deviation values, enabling automatic detection of query pattern drift. By combining historical query popularity statistics to calculate optimization benefit scores and using decision thresholds determined based on the analysis of the subject's working characteristic curve, the system can dynamically identify high-frequency query paths and automatically update query priority weights or mark joint index suggestions. The scheme introduces a periodic weight normalization mechanism to prevent parameter saturation under long-term operation, eliminate the lag in passive optimization by manual maintenance, and ensure that the system can maintain stable retrieval response performance when facing sudden traffic surges or unforeseen query patterns. Attached Figure Description

[0039] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0040] Figure 1 This is a structural diagram of the system of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0042] Example 1

[0043] Please see Figure 1 The business object intelligent search system based on metadata includes: a context request parsing module, which receives abstract search requests decoupled from the physical storage engine and extracts current runtime environment parameters and user permission features to generate dynamic context snapshots;

[0044] The logical structure construction module is used to load the preset metadata graph and combine it with the dynamic context snapshot generated by the context request parsing module to perform rule injection operations on the abstract search request, thereby generating a logical abstract syntax tree (AST) containing implicit filtering conditions.

[0045] The multi-dialect compilation and execution module is used to translate the logical abstract syntax tree generated by the logical structure construction module into physical execution statements that match the dialect of the target storage engine, based on the physical mapping rules and data distribution characteristics defined in the metadata graph.

[0046] The adaptive optimization feedback module is used to monitor the execution latency data of the physical execution statements generated by the multi-dialect compilation and execution module, and dynamically correct the index strategy definition in the metadata graph based on the query popularity statistics.

[0047] This embodiment elaborates on the core architecture of the system and the internal data flow mechanism, aiming to solve the maintenance bottleneck caused by the tight coupling between business logic and physical storage in a heterogeneous data source environment; the context request parsing module serves as the system's perception entry point, intercepting RESTful requests initiated by the front end through the API gateway. This request only carries object identifiers and explicit query parameters at the business semantic level, completely stripping away the underlying table structure information.

[0048] During this period, the module extracts current runtime environment parameters, such as geofence coordinates, by parsing the JWT token and client fingerprint carried in the HTTP header. Request timestamp In addition to user permission characteristics, such as role ID and organizational structure code; the system integrates the above discrete parameter sets and instantiates them into an immutable dynamic context snapshot object, the calculation formula of which is:

[0049] ;

[0050] in, This represents a user permission feature vector, consisting of a sequence of values ​​mapped from role IDs to organizational structure codes. Including the aforementioned spatiotemporal parameters, this object provides a deterministic variable basis for subsequent logical injection; the logical structure construction module loads the metadata graph, which serves as the system's knowledge base and predefines the logical topology and constraint rules between entities; the module traverses the rule nodes in the graph and takes snapshots. Substitute the variables into the preset Boolean trigger function The value is calculated using the following formula:

[0051] ;

[0052] in, and Defined as the time window boundary value of a preset business rule, for example The timestamp representing the start of the current promotional period. Represents the end timestamp, while and Defined as a role identifier authorized to access sensitive data, for example On behalf of the auditor, Represents the super administrator; the above time window boundary values , Stored in the global configuration node of the metadata graph, it supports dynamic updates via the configuration center; while the role identifier... , These are constants pre-installed in the Access Control List (ACL) by the system. The logical structure building module takes a dynamic context snapshot at runtime. In The attribute is matched against the preset identifier in the ACL to determine the truth value of the permission hit;

[0053] In response to the function returning a true value, the system automatically injects the implicit filtering conditions associated with the rules into the original request and constructs a standardized logical abstract syntax tree. This tree structure recursively describes the logical completeness of the query intent. The multi-language compilation and execution module takes over this AST and, based on the physical mapping rules defined in the metadata graph, such as the mapping relationship of field names and data sharding strategies, traverses the AST nodes using the visitor pattern to translate them into physical execution statements for a specific database. The adaptive optimization feedback module runs asynchronously in the background, continuously collecting the execution time and resource overhead of the physical statements to form a closed-loop feedback mechanism, dynamically adjusting the indexing strategy in the metadata graph to ensure the performance stability of the system as the data scale grows.

[0054] Example 2

[0055] The multi-dialect compilation and execution module is specifically used for: reading the preset field cardinality features and index path definitions in the metadata graph; determining the connection order and driving strategy of each query node in the logical abstract syntax tree based on the field cardinality features, so as to generate optimized physical execution statements; determining the connection order and driving strategy includes: when the logical abstract syntax tree involves multi-table join queries, identifying filter nodes whose field cardinality features are higher than the preset high selectivity threshold; determining the physical table corresponding to the filter node as the driving table, and rewriting the table join sequence according to the physical mapping rules.

[0056] This embodiment is a further specification of the query optimization mechanism in the multi-dialect compilation and execution module, focusing on using statistical features to solve the performance degradation problem in complex relational queries;

[0057] During the compilation phase, the multi-dialect compilation and execution module accesses the statistical information database maintained in real time in the metadata graph, including field cardinality features. It is obtained through a cardinality estimation algorithm to ensure Space complexity, and the total number of rows in the table. This is achieved by obtaining the information_schema metadata from the asynchronous sampling storage engine, thereby ensuring the real-time performance and low overhead of the input parameters, including the total number of rows in the table. The sampling frequency is set to heartbeat synchronization once every 5 seconds, while the field cardinality feature... The very long bit counter is persisted to the metadata cache every 60 seconds;

[0058] Read the cardinality characteristics of each business field, i.e., the number of distinct values ​​in the column, the existing index path definition, and the query priority weights dynamically maintained by the adaptive optimization feedback module. The initial value is 1.0 by default;

[0059] Based on this statistical information, the module initiates a cost estimation process, selectively evaluating each query node in the logical abstract syntax tree. Furthermore, when the system detects a cross-entity Join operation in the logical tree, indicating a multi-table join query, the module immediately executes a driving strategy analysis. The module then iterates through all fields involved in the filtering conditions, calculating their cardinality. Total number of rows in the table The ratio identifies those cardinal features that are higher than a preset high-selectivity threshold. Filtering nodes, such as This threshold is not a fixed constant, but rather the optimal split point for distinguishing between high and low selectivity, automatically calculated by the system based on the statistical distribution characteristics of historical query logs using the maximum inter-class variance method. The specific calculation logic is as follows: the system periodically collects the cardinality ratio of all filtering conditions in the historical query logs. Construct a frequency distribution histogram and solve for the optimal threshold with the objective function of maximizing the inter-class variance. This ensures that only filter conditions with statistical significance can trigger the driving table locking mechanism.

[0060] Specifically, for situations where a single physical table has multiple filtering conditions, such as filtering by both status and creation time, the module executes the maximum selectivity priority principle, calculating the maximum selectivity of all filtering fields on the table. Value and take the maximum value As a representative feature of this table, it prevents low-selectivity conditions from masking high-value index paths; in response to identifying high-selectivity nodes, i.e. The system forcibly locks the physical table to which the node belongs as the driving table;

[0061] To implement this strategy, the module rewrites the table join sequence in the SQL statement according to the physical mapping rules. Specifically, the module constructs a list containing all participating tables. and for each table in the list Calculate the mixed priority score If the table contains the condition that... The filtering conditions are set as follows:

[0062] ;

[0063] Introduced here As a hierarchical transition coefficient, this coefficient is determined by analyzing the maximum number of table rows supported by the system. Definitely, meets the requirements ;in Set the maximum number of rows in a single table that the current database instance's hardware configuration can support, for example, 500 million rows in a 1TB storage configuration;

[0064] More specifically, the selection of this constant is based on the cost isolation principle: to ensure that any highly selective index path has a selectivity coefficient... priority The priority is always greater than that of a full table scan path, i.e. The following inequality must be satisfied:

[0065] ;

[0066] in, The preset hierarchical transition coefficient is used, and in this embodiment, it takes the value of... The example selected in this embodiment Make This mathematically constructs a significant numerical safety zone, ensuring that the priority score of any highly selective index path is maintained. It is necessarily greater than the inverse of the cost of any full table scan path, thus achieving deterministic hierarchical leap and ensuring that the priority of the high-selectivity index path is absolutely higher than any full table scan operation, regardless of the table size;

[0067] Otherwise set In the absence of high selectivity conditions, smaller tables are processed first. This is the real-time row count of the current physical table. This is a preset dimensionless normalization constant, always set to 1, with the unit being rows, designed to eliminate... The resulting dimensional differences ensure Maintaining dimensionless properties across the entire domain, thus achieving physical comparability; the module performs sorting operations. The resulting ordered list is the final table join sequence, with the first table in the list being... When placed at the beginning of the FROM clause in an SQL statement, subsequent tables are joined sequentially; the mathematical expression of this strategy is as follows:

[0068] ;

[0069] ;

[0070] in, For physical table indexes, indicating the currently evaluated number of... physical table ; For the first The cardinality of fields in each physical table is derived from statistical information in the metadata graph; For the first The total number of rows in each physical table is counted in real time to maintain the consistency of parameter meaning; For the first The maximum column-level cardinality ratio of all filter fields in a physical table, physically meaning the theoretical maximum distinguishability of that field as an index column; The preset high selectivity threshold is derived from system configuration; For the first The query priority weights of the attributes related to each physical table are derived from the dynamic updates of the metadata graph; It is a dimensionless normalization constant with a value of 1, used to unify the physical dimensions of piecewise functions; For the first The mixed priority scores of each physical table are used to achieve a fusion of rule optimization and cost optimization through a piecewise function;

[0071] In particular, to correct estimation bias caused by data skew, the system introduces frequency distribution calibration for specific values: if the query condition is a specific value... The corrected selectivity coefficient is calculated using the following formula:

[0072] ;

[0073] in, This represents the frequency percentage of the value in the histogram; in the default mode where histogram data is unavailable, it is used directly. Make a rough estimate;

[0074] The module generates the final physical execution statement based on the defined driving table and join sequence, ensuring that the database engine prioritizes the execution of highly filtering conditions, thereby minimizing the size of the intermediate result set.

[0075] This embodiment introduces cost-based optimization logic at the compilation layer, effectively avoiding the risk of full table scans caused by inaccurate estimations by the database's native optimizer under complex relationships in multidimensional analysis scenarios involving massive amounts of data. The system enforces a strategy of small table driving large table or high filtering priority, which significantly reduces I / O overhead and memory usage, and ensures the second-level response capability for complex report queries.

[0076] Example 3

[0077] The logical structure construction module is specifically used for: traversing the preset context rule base in the metadata graph; extracting the constraints that match the user permission features in the dynamic context snapshot as implicit predicates; and merging the implicit predicates with the explicit filtering conditions in the abstract search request through logical AND operation to construct a logical abstract syntax tree.

[0078] This embodiment details the specific execution logic of the logical structure construction module in implementing row-level data access control. The logical structure construction module starts the rule matching engine, traversing the context rule base stored in the metadata graph. This rule base defines business visibility logic in the form of condition-constraint pairs. The module uses user permission features extracted from the dynamic context snapshot, such as the department code or job title identifier, as input variables and performs Boolean matching operations with the triggering conditions in the rule base. In response to a successful match, the module instantiates the constraint logic defined in the rule into an implicit predicate. This predicate represents a filtering condition that is invisible to the user but must be enforced; the module performs a logic merging operation, combining the explicit filtering conditions entered by the user on the front end. With automatically generated implicit predicates A logical AND operation is performed to reconstruct the root node of the query tree; this merging process follows the logical formula:

[0079] ;

[0080] in, The root logical expression for the final constructed logical abstract syntax tree; These are explicit filter conditions, derived from user input. For the first The implicit predicates for each match are derived from the rule base; The total number of rules matched;

[0081] The module outputs a logical abstract syntax tree containing complete permission constraints, which is used by subsequent compiled modules to ensure that no query can bypass the preset security boundaries.

[0082] This embodiment achieves physical separation of application logic and security strategy in financial risk control or medical data management scenarios by sinking the permission logic to the AST construction stage. This mechanism ensures that even if the front-end code has vulnerabilities or is maliciously tampered with, the query statements generated by the back-end still strictly follow the principle of least privilege, effectively preventing the risk of data leakage caused by unauthorized access.

[0083] Example 4

[0084] The multi-language compilation and execution module translates the logical abstract syntax tree as follows: when the target storage engine is identified as a relational database, the logical abstract syntax tree is traversed using the visitor pattern to generate structured query language (SQL) statements; when the target storage engine is identified as a document-oriented database, the logical abstract syntax tree is mapped to a JSON structure of a domain-specific query language (DSL) for document storage; and when the target storage engine is identified as a graph database, the logical abstract syntax tree is converted into graph query language statements.

[0085] This embodiment demonstrates the polymorphic translation capability of the multi-dialect compilation and execution module when facing heterogeneous storage backends. The module reads the metadata configuration and identifies the target storage engine type mapped to the entity involved in the current query. In response to the target storage engine being identified as a relational database, such as PostgreSQL or MySQL, the module instantiates an SQL generator visitor object. This object applies the visitor design pattern to deeply traverse each node of the logical abstract syntax tree, converts logical operators such as include into SQL standard LIKE clauses, processes table aliases and field references, and finally concatenates them into a structured query language statement that conforms to the ANSI SQL standard.

[0086] In response to the target storage engine being identified as a document-oriented database, such as Elasticsearch or MongoDB, the module invokes the DSL mapper to execute a transformation process based on a recursively defined predefined operator mapping table. ;

[0087] Define a recursive transformation function ,when Composite logic node At that time, output JSON structure ;when leaf node When, output This serializes the tree-like logical structure into nested JSON objects of a Domain-Specific Query Language (DSL). In response to the target storage engine being identified as a graph database, such as Neo4j, the module initiates a graph query converter, performing a pattern-matching-based translation process: the module maintains a primitive mapping table. Record the correspondence between business entities and graph node labels;

[0088] The converter employs a depth-first search to traverse the logical abstract syntax tree. For each relation operator node in the tree, it generates a Cypher path pattern based on the topological definition in the metadata graph. It also converts the attribute constraints of the AST leaf nodes into assertions in the WHERE clause. Finally, the graph is assembled into a complete graph query statement; the module outputs a physical execution statement adapted to a specific dialect, ready to be submitted to the underlying driver for execution.

[0089] This embodiment supports the smooth coexistence of legacy systems and next-generation NoSQL architectures during enterprise digital transformation by building a polymorphic compilation adaptation layer between the logical layer and the physical layer. Developers do not need to master multiple database dialects to manipulate complex data, which greatly reduces the threshold for technology stack migration and reserves interface channels for the flexible evolution of the underlying storage architecture.

[0090] Example 5

[0091] The adaptive optimization feedback module corrects the index strategy definition, including: calculating the deviation between the execution latency data and the preset baseline response time; when the deviation exceeds the preset performance degradation threshold, identifying high-frequency query paths in the current physical execution statement; and updating the query priority weight of relevant attributes or marking composite index suggestions in the metadata graph based on the high-frequency query paths.

[0092] This embodiment details the specific closed-loop logic of the adaptive optimization feedback module to achieve system self-evolution; the module intercepts the return results of the database driver through hook functions to accurately collect the actual execution time of the physical execution statements, i.e., execution latency data. The module reads the preset baseline response time defined in the system SLA configuration. The performance deviation value is calculated using the following formula. The calculation formula is as follows:

[0093] ;

[0094] in, The actual execution delay originates from execution log monitoring; The baseline response time is derived from the SLA configuration. The deviation value, in physical terms, is a normalized index of the degree of performance degradation; the baseline response time. Stored in JSON format in the ZooKeeper or Etcd configuration center, supporting dynamic distribution based on different business dimensions, for example: ;

[0095] The module will calculate the deviation value Compared with the preset performance degradation threshold The threshold was compared. It is not a static constant, but is dynamically defined as follows:

[0096] ;

[0097] in, and Based on historical execution delay logs Calculated historical execution delay deviation value The mean and standard deviation of the distribution, thus ensuring and Having the same dimensionless properties, and constructing statistically significant anomaly detection boundaries; responding to The system determines that the current query has a performance bottleneck and immediately starts query mode analysis to identify high-frequency query paths in the current physical execution statement;

[0098] Specifically, the WHERE clause in the module parsing statement extracts all the field names involved in the filtering and constructs a set. ,right Sort the strings lexicographically, concatenate them, and calculate their MD5 hash values ​​as path fingerprints. Based on this, the module combines historical query popularity statistics to calculate the optimization benefit score for this path. The calculation formula is as follows:

[0099] ;

[0100] in, For this fingerprint The cumulative hit count within the most recent sliding window, which is implemented based on a Redis ordered set data structure. The window duration is set to 15 minutes. The system uses a background scheduled task to clear counts whose timestamps exceed the window range. The count is a dimensionless integer. The base is the natural number; this is explicitly stated here. Using count values ​​instead of frequencies ensures the accuracy of subsequent logarithmic operations. The input parameters are kept dimensionless in a physical sense to avoid dimensional conflicts;

[0101] Based on the calculation Essentially, it is a dimensionless composite evaluation index, and its numerical value lacks an absolute physical meaning reference, meaning it cannot be directly measured in seconds or milliseconds. In order to construct a statistically significant quantitative judgment standard, the system loads historical execution latency data during the initialization phase. Baseline time and cumulative hit count Historical traffic load distribution logs Simulate and calculate the distribution of optimization benefit scores for historical queries, and extract the historical optimization benefit scores. 95th percentile of the distribution This is not P95 in terms of time units, but rather a dimensionless scoring benchmark, based on a preset sensitivity coefficient. and Perform threshold calculation:

[0102] ;

[0103] Among them, coefficient and The optimal operating point was determined by analyzing ROC curves (Receptor Operating Characteristic curves) of historical performance failure cases in the system. To construct the truth labels required for ROC analysis, the system backtracked and analyzed historical operation logs. Historical query paths that showed a latency reduction of more than 50% after manual indexing were marked as positive samples, while paths with no significant change or deterioration in latency after indexing were marked as negative samples. Based on this, the true positive rate and false positive rate at different thresholds were calculated, thereby determining the optimal operating point. Corresponding to high confidence intervals to reduce the false alarm rate of index rebuilding. Corresponding to the high sensitivity range to capture potential performance jitter, thereby ensuring that the judgment criteria are consistent with... They are within the same dimension of measurement and have statistical basis;

[0104] During system cold starts or initialization phases lacking historical logs, preset empirical values ​​are directly used. and As a default parameter, the above ROC analysis process will be triggered to dynamically calibrate the parameters after a full statistical period of data has been accumulated; the system will calculate in real time... The system performs a hierarchical comparison with the aforementioned decision thresholds. To overcome the product trap problem where a single product formula cannot distinguish between high-frequency low-deterioration and low-frequency high-deterioration, the system additionally introduces a degradation-dominant factor, the calculation formula of which is as follows:

[0105] ;

[0106] like and or If the absolute hotspot threshold is greater than the preset threshold, it indicates that the performance degradation is significantly greater than the impact of frequency or that the query popularity has reached a bottleneck. In this case, a "recommended composite index" marker is added to the relevant attributes of the metadata graph. This marker can be read by subsequent automated operation and maintenance scripts to trigger database index rebuilding operations. and or The system then executes a weight adaptive update algorithm and... After performing non-negative truncation, update the query priority weights of relevant attributes in the metadata graph. ;

[0107] The specific update logic is as follows: based on path fingerprint Reverse mapping of sets For all fields in the dataset, the incremental learning formula is uniformly applied to these fields as follows:

[0108] ;

[0109] in, This is a preset learning rate coefficient, for example, 0.01. The upper limit of the preset weight saturation, such as 100.0, is introduced to prevent the weight parameters from diverging due to individual extreme hot queries, thereby disrupting the balance of the overall query plan. By increasing the weight of this attribute in the generation of subsequent query plans, the generation order of subsequent query plans can be adjusted.

[0110] This embodiment establishes a performance optimization closed loop based on feedback control theory. In scenarios such as IoT log analysis or user behavior tracking where the amount of data grows exponentially, it can automatically sense the drift of query patterns and dynamically adjust the indexing strategy. This mechanism eliminates the lag of passive optimization by manual DBAs and ensures that the system can maintain sub-second retrieval response performance when faced with sudden traffic or unforeseen query patterns.

[0111] Furthermore, regarding the issue that only incremental update logic might lead to changes in weight parameters... After long-term operation, they converge to the upper limit. To address the problem of loss of distinguishability, i.e., the saturation effect, this embodiment also introduces a periodic weight normalization mechanism; the system sets a normalization period. For example, in a 24-hour period, whenever a period is triggered, the background thread locks the metadata graph and performs the following linear transformation, the calculation formula of which is:

[0112] ;

[0113] in, The set of all attributes in the graph. For example, the preset total energy constant. The setting of this constant follows the principle of energy conservation, aiming to maintain the sum of query priority weights throughout the entire graph. It should be noted that maintaining the sum of weights that have drifted after incremental updates over a period of time means keeping it constant. Force reset back to the preset steady-state energy value ;

[0114] Due to the incremental learning formula Including only additive terms would lead to a monotonically increasing total system energy. This normalization step is essentially an energy dissipation process. By scaling proportionally, the risk of numerical inflation is eliminated while strictly preserving the relative proportions between the weights of each path. This prevents the overall weight values ​​from drifting or overflowing due to monotonically increasing updates during long-term adaptive processes. This step ensures that the weight vector always lies on a bounded hyperplane during the update process, guaranteeing that the system can effectively identify the real hot query paths through relative weight differences during long-term operation over months or even years.

[0115] Example 6

[0116] The metadata graph includes: business semantic atomic definitions, which describe the attribute types and logical relationship topologies of business objects; view strategy definitions, which describe the field display rules under different business scenarios; and storage mapping definitions, which describe the correspondence between logical attributes and physical storage fields and storage engine characteristic parameters.

[0117] This embodiment defines in detail the three-dimensional structure of the metadata graph, which constitutes the core knowledge base of the system. The graph includes a business semantic atomic definition layer, which describes the essential attributes of business entities in a technology-independent manner, such as the data type, value range, and reference relationships between entities, i.e., logical association topology, which constitutes the skeleton of business logic. The graph is overlaid with a view strategy definition layer, which defines the visibility, rendering format, and arrangement order of fields for different front-end display scenarios, such as mobile card views or PC list views, i.e., field display rules, realizing the configuration of UI logic.

[0118] The underlying layer of the graph contains a storage mapping definition layer, which establishes a precise mapping from logical attributes to physical storage fields and records storage engine characteristic parameters, such as whether columnar storage is enabled and the type of tokenizer. These parameters directly guide the compilation module to generate the optimal physical statements. The definitions of these three dimensions are orthogonal to each other and tightly coupled, together forming a self-describing data model.

[0119] This embodiment solves the classic engineering problem of inconsistent front-end and back-end data definitions in a low-code development platform or dynamic form generation scenario by integrating semantics, view and storage definitions into a single graph. This graph exists in the database but cannot be displayed on the front end. As the sole source of truth, it ensures consistency across the entire chain from data definition to interface rendering, significantly reducing the communication costs of cross-team collaboration.

[0120] To ensure rigorous code-level reproducibility, this embodiment further clarifies the physical data structure defined above; the business semantic atomic definition is implemented at the physical layer as a configuration file conforming to the JSONSchema standard, with a specific structure example as follows:

[0121] {"id":"Order","type":"object","properties":{"amount":{"type":"number"}},"relations":[{"target":"User","type":"1:N","constraint":"Cascade"}]}

[0122] The storage mapping is defined using a key-value pair mapping table, specifically structured as a Map logical path and physical metadata. The physical metadata class, in its Java implementation, contains the following attributes: private String dbEngine; private String tableName; private String shardKey; private Map.<String,Object> connectionProps;

[0123] This rigorous structured definition ensures that the system can directly load the graph into a weighted directed graph object in memory through deserialization operations. This graph object uses an adjacency list to store the topological relationships between nodes, thus providing a definite input data structure for subsequent path search algorithms.

[0124] Example 7

[0125] The virtual wide table mapping module is used to map the multi-entity association model in the physical storage layer into a flattened virtual wide table structure based on the logical association topology in the metadata graph before the logical structure construction module generates the logical abstract syntax tree, so that the context request parsing module can identify objects.

[0126] This embodiment introduces a virtual wide table mapping module, which aims to eliminate the cognitive interference of the physical data model on the upper-layer business logic. The virtual wide table mapping module intervenes in the very early stage of the request processing chain. Specifically, the module runs before the context request parsing module performs object recognition. Based on the logical relationship topology between entities defined in the metadata graph, such as one-to-many or many-to-many relationships, it constructs a logical global view.

[0127] The module applies a graph traversal algorithm, specifically employing a breadth-first search strategy. Using the main entity of the current context as the root node, it traverses the association graph to determine the reachability of attributes and the shortest connection path. Based on the naming rules of entity path_attribute name, the module automatically generates globally unique virtual field aliases and dynamically maps the highly normalized multi-entity association model in the underlying physical storage layer, i.e., star or snowflake schema, into a flat virtual wide table structure containing all reachable attributes.

[0128] Specifically, for multi-valued fields generated by one-to-many relationships, the system defines them as Array type in the virtual table and applies aggregation folding rules to avoid row expansion caused by Cartesian products; based on this, the module builds and maintains a memory-mapped dictionary. The dictionary uses generated virtual field aliases as keys and physical path tuples as keys. The value is , where The complete link chain from the root entity to the target field is explicitly recorded, and this link chain is specifically implemented as an ordered list of quadruplets. <Tuple<SourceTbl,SourceCol,TargetTbl,TargetCol> For example: [("Order","cust_id","Customer","id"),("Customer","addr_id","Address","id")], thus explicitly specifying the specific foreign key and primary key pairs for table joins; the module exposes the metadata interface of this virtual wide table to the context request parsing module, enabling the context request parsing module to directly perform object identification based on this virtual structure. The specific identification process is as follows: the parsing module extracts the field identifiers from the request parameters and maps them to a memory-mapped dictionary. The system performs a key-value lookup. If a match is found, the corresponding physical path tuple is retrieved and the field is marked as a valid business object.

[0129] There's no need to concern ourselves with the underlying JOIN paths or foreign key constraints; we only need to select attributes for this virtual table. This mapping is cached in memory for use by subsequent logical structure building modules when generating the AST. Perform reverse parsing of the physical path to form a complete and ordered processing flow of virtual mapping, context parsing, and logical construction.

[0130] This embodiment constructs a virtual wide table layer, shielding the complex normalized design details of relational databases in self-service BI analysis or ad-hoc query scenarios aimed at non-technical personnel. Business users can freely combine cross-entity query conditions as if operating a single Excel spreadsheet, without needing to understand the underlying table join logic, greatly releasing the business value of data and improving the intuitiveness and convenience of data exploration.

[0131] Regarding the specific implementation of the aforementioned aggregation and folding rules, this embodiment adopts an operator injection technique based on dialect features. When the underlying database is a relational database, the rule is instantiated as follows: when generating SQL statements, the JSON array aggregation function is forced to be applied to array type fields in the SELECT clause, and used in conjunction with the DISTINCT keyword, i.e., the JSON_ARRAYAGG array aggregation function, to mathematically eliminate redundant data caused by Cartesian product due to multi-table parallel 1:N joins, or redundant data after aggregation using the STRING_AGG function, and a GROUPBY clause is added at the end of the statement simultaneously, with the grouping key strictly limited to the primary key of the root entity;

[0132] This mathematical operation This ensures that the number of rows in the result set generated by multi-table joins is strictly equal to the number of root entities, fundamentally eliminating the data bloat problem caused by Cartesian product. It enables each row of the virtual wide table to be accurately mapped back to a unique business object instance. When the underlying database is a document-oriented database, the nested document structure is directly used to map array fields; when the underlying database is a graph database, path aggregation functions are used to achieve flattened mapping.

[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A meta-metadata based intelligent search system for business objects, characterized in that, include: The context request parsing module is configured to receive abstract search requests decoupled from the physical storage engine and extract current runtime environment parameters and user permission characteristics to generate a dynamic context data snapshot containing a set of key-value pairs. The logical structure construction module is configured to load a preset metadata graph, which is a data structure describing the mapping relationship between the logical structure of business objects and physical storage. It combines the dynamic context data snapshot to perform rule matching on the abstract search request and generate a logical abstract syntax tree containing implicit filtering conditions. A multi-dialect compilation and execution module is configured to translate the logical abstract syntax tree into physical execution statements that match the dialect of the target storage engine, based on the physical mapping rules and data distribution characteristics defined in the metadata graph. An adaptive optimization feedback module is configured to monitor the execution latency data of the physical execution statements and dynamically correct the index strategy definition in the metadata graph based on the query frequency statistics. The multi-dialect compiler and execution module translates the logical abstract syntax tree, including: When the target storage engine is identified as a relational database, the visitor pattern is applied to traverse the logical abstract syntax tree and generate structured query language (SQL) statements. When the target storage engine is identified as a document-oriented database, the logical abstract syntax tree is mapped to a JSON structure of a domain-specific query language (DSL) for document storage. When the target storage engine is identified as a graph database, the logical abstract syntax tree is converted into a graph query language statement.

2. The meta metadata based business object intelligent search system of claim 1, wherein, The multi-dialect compilation and execution module is specifically used for: Read the preset field cardinality features and index path definitions in the metadata graph; Based on the cardinality characteristics of the fields, the connection order and driving strategy of each query node in the logical abstract syntax tree are determined to generate optimized physical execution statements.

3. The meta metadata based business object intelligent search system of claim 2, wherein, The multi-dialect compilation and execution module determines the linking order and driving strategy, specifically configured as follows: When the logical abstract syntax tree involves multi-table join queries, obtain the field selectivity values ​​of each filter node; If the field selectivity value of a certain filter node is detected to be higher than the preset selectivity threshold, the physical table corresponding to the filter node is determined as the driving table, and the table join sequence is rewritten according to the physical mapping rules.

4. The meta metadata based business object intelligent search system of claim 1, wherein, The logical structure construction module is specifically used for: Traverse the preset context rule base in the metadata graph; The constraints that match the user permission features in the dynamic context data snapshot are extracted as implicit predicates; The implicit predicate is combined with the explicit filtering conditions in the abstract search request by performing a logical AND operation to construct the logical abstract syntax tree.

5. The meta metadata based business object intelligent search system of claim 1, wherein, The adaptive optimization feedback module corrects the indexing strategy definition, specifically configured as follows: Calculate the numerical deviation between the execution delay data and the preset baseline response time; If the numerical deviation is greater than the preset performance degradation threshold, then the query path in the current physical execution statement whose access count within the statistical period exceeds the preset frequency threshold is identified. Based on the query path, update the query priority weight values ​​of relevant attributes in the metadata graph or write the composite index suggestion flag.

6. The meta metadata based business object intelligent search system of claim 1, wherein, The metadata graph includes the following data definition structure: Business semantic atomic definition is used to store the attribute types and logical relationship topology data of business objects; View strategy definition is used to store field display rule parameters under different business scenarios; Storage mapping is defined as a mapping table that stores logical attributes to physical storage fields and storage engine characteristic parameters.

7. The meta metadata based business object intelligent search system of claim 1, wherein, Also includes: The virtual wide table mapping module is used to map the multi-entity association model in the physical storage layer into a flattened virtual wide table structure based on the logical association topology in the metadata graph before the logical structure construction module generates the logical abstract syntax tree, so that the context request parsing module can identify objects.

Images