A standardization and pre-compilation method and system for real-time database filtering expressions

Abstract

This invention discloses a standardization and pre-compilation method and system for filtering expressions in real-time databases, belonging to the field of data filtering technology. Based on Abstract Syntax Tree (AST) expression standardization, this invention unifies semantically equivalent but formally different filtering expressions into a stable canonical form, and generates stable keys to achieve consistency recognition, fundamentally improving cache hit rate and reducing duplicate compilation objects. Furthermore, the method introduces a joint judgment based on cumulative access frequency, time-weighted popularity, and structural complexity. Pre-compilation is dynamically triggered based on sufficiently high access frequency and complexity, while automatic deletion occurs if not called for a long time, thereby controlling cache pollution and memory overhead while ensuring benefits. Simultaneously, metadata / rule version fingerprints are introduced into keying and cache objects to ensure secure invalidation after metadata changes, reducing the risk of false hits caused by semantic drift, and overall improving the throughput, stability, and tail latency performance of industrial real-time queries.

Description

Technical Field

[0001] This invention belongs to the field of data filtering technology, specifically relating to a standardization and pre-compilation method and system for filtering expressions for real-time databases. Background Technology

[0002] Industrial real-time databases (TSDBs), serving as the core data foundation of information architecture in process industries and discrete manufacturing, are widely used in scenarios such as power dispatching, petrochemical process control, steel metallurgical continuous casting and rolling, rail transit signal monitoring, and intelligent manufacturing execution systems. In these systems, filter expressions are frequently used for critical business logic such as alarm rule triggering, abnormal operating condition location, process segment screening, and quality reverse traceability. Because industrial sites typically deploy dozens of concurrently accessing data consumers (such as SCADA monitoring systems, MES manufacturing execution systems, data analysis platforms, and mobile maintenance terminals), the system must handle a continuous influx of filter query requests under millisecond-level latency constraints. This imposes a rigid constraint on the throughput and latency stability of the execution engine.

[0003] Existing real-time database systems generally employ a just-in-time (JIT) compilation strategy when processing filtering expressions: each query request undergoes a complete chain of operations independently, including lexical analysis, syntax analysis, metadata binding, type inference and implicit conversion, execution plan generation, and physical operator instantiation. This process involves significant heap memory allocation, string traversal, and virtual function scheduling, resulting in substantial computational overhead. Under high concurrency loads, this repetitive compilation behavior rapidly saturates CPU resources. Although most queries can still be completed within tens of milliseconds, the tail latency of a few requests can drastically increase to hundreds or even thousands of milliseconds due to compiler lock contention and garbage collection pauses. For industrial process control scenarios, this unpredictable latency jitter is more harmful than average latency, potentially leading to delayed alarm signals, misjudgments in interlocking logic, and even cross-system control timing misalignments.

[0004] To mitigate the overhead of repeated compilation, some systems have introduced a caching mechanism based on string equality matching. This mechanism uses the original query string as the key and the compiled execution plan as the value to construct an in-memory cache. However, in industrial environments, the same semantic constraints are often presented in various heterogeneous string forms: including but not limited to predicate order rearrangement, addition or removal of redundant parentheses, permutation of IN list elements, equivalent variations between interval predicates and logical combinations, and alternation between test point aliases and fully qualified names. While these variations are semantically equivalent at the syntax tree level, they fail to hit the cache at the string level, resulting in an actual hit rate generally below 30%. Most queries still need to fall back to the complete compilation path, significantly diminishing the performance gains of the cache.

[0005] Furthermore, the metadata of industrial measurement points exhibits dynamic evolution. Equipment upgrades, process parameter adjustments, or control system upgrades can all trigger changes to the measurement point dictionary. This manifests as data type migration (e.g., from integer to floating-point), rescaling of engineering units (e.g., from MPa to kPa), updates to enumeration value mappings, or the end of the measurement point's lifecycle. If the caching system lacks a version constraint mechanism, there is a risk of dirty hits: the execution plan generated by a query under the old semantic environment is reused after the metadata version evolves, leading to runtime anomalies such as type mismatches, unit conversion errors, or enumeration value out-of-bounds errors. Existing solutions either employ a full cache invalidation strategy after metadata changes, causing the hit rate to jump to zero and triggering a compilation storm; or they completely ignore version semantics, creating potential correctness vulnerabilities. In summary, this field urgently needs a caching architecture that combines semantic equivalence recognition and metadata version awareness to achieve stable, low-latency query responses under high-concurrency industrial loads. Summary of the Invention

[0006] The purpose of this invention is to provide a standardized and pre-compiled method and system for filtering expressions in real-time databases. This method addresses the technical problem that existing real-time database systems, when processing filtering expressions, cannot efficiently identify semantically equivalent but differently written expressions while ensuring query correctness. This results in low cache hit rates, high overhead from repeated compilation, and difficulty in meeting the high-concurrency, low-latency query requirements of industrial environments.

[0007] To achieve the above objectives, the present invention employs the following technical solution: This invention discloses a standardization and pre-compilation method for filtering expressions in real-time databases, comprising the following steps: S1: Receives the filter expression e from the real-time database query request and parses it to obtain the abstract syntax tree AST(e). S2: Standardize the obtained abstract syntax tree AST(e) under the premise of semantic equivalence to obtain the normalized AST*(e). S3: Perform stable serialization on the normalized AST*(e) to obtain the stable representation Ser(AST*(e)), and generate a stable key k by combining the version information; S4: Maintain a statistical vector Stats[k] for each stable key k, and update Stats[k] each time a stable key k is hit; S5: Construct a scoring function based on the statistical vector Stats[k]. Trigger the pre-compilation process when the calculation result of the scoring function meets the preset trigger threshold. S6: Compile the normalized AST*(e) into a predicate execution object P. k and P k Write to the cache Cache[k]; S7: Subsequent requests that hit the stable key k skip the parsing and compilation phases and directly execute P in Cache[k]. k .

[0008] Furthermore, in S1, the abstract syntax tree AST(e) contains Boolean combination nodes and atomic predicate nodes; The atomic predicate node is denoted as The ; Where mp_id is the stability identifier of the measurement point; θ is an element in the operator set; and ξ is the comparison object. When parsing to obtain the Abstract Syntax Tree (AST), dictionary / metadata binding is performed on mp_id, and type normalization is performed on ξ=.

[0009] Furthermore, in S3, the formula for calculating the stable bond k is: ; Where Hash is the MurmurHash algorithm, a non-cryptographic fast hashing algorithm, and v schema For the dictionary / type / function semantic version of the test points, v norm This is a standardized rules version.

[0010] Furthermore, the statistical vector Stats[k] includes at least the cumulative number of visits N. cum (k), time-weighted heat H(k), structural complexity C(k); Wherein, the cumulative number of visits N cum (k) Obtained by accumulating the hit indicators within the sliding time window: ; Where, x t To determine whether the expression is matched in the t-th statistics window / event, x is the value when matched. t =1, otherwise x t =0; now is the current system time; Ncum(k) is the cumulative number of visits; The time-weighted heat H(k) is calculated using an exponential smoothing recursive formula: ; Where H(k,t) is the time-weighted heat, α is the smoothing coefficient, α∈(0,1); X t X is an indicator variable for whether the stable key is hit within the t-th time window, and when it is hit... t =1, otherwise X t =0; Structural complexity C(k) = |AST*(e)|; in Representation of Normalized Abstract Syntax Tree The total number of operator nodes is used to roughly represent the structural complexity of the filtering expression.

[0011] Furthermore, in S5, the scoring function is: ; Where a, b, and c are the weights of each indicator; C0 is the complexity normalization constant; And when Pre-compilation is triggered on time; hysteresis debouncing is used when... Undo the pre-compiled state at that time, and Hysteresis anti-shake is formed.

[0012] Furthermore, in S6, the cache[k] is defined as: ; in, For build time, The most recent hit time.

[0013] Furthermore, in S7, the idle time is defined as: T idle (k) =now-t last (k); When T idle (k)≥T threshold And H(k)≤H del When, delete P from Cache[k]. k And restore the stable key k to its uncompiled state; where T threshold H is the idle time threshold. del This is the threshold for removing content based on popularity.

[0014] Furthermore, the standardization steps include at least one of the following operations: Eliminate writing layer differences, which include whitespace characters, redundant parentheses, keyword capitalization, and quotation mark style; Fold the constant expression and perform Boolean simplification; Perform normalization transformation on the equivalent operator form; Flatten clauses at the same logical level and sort them according to deterministic rules; Merge multiple interval constraints at the same measurement point and standardize them according to the order of endpoints.

[0015] Furthermore, the reuse step also includes: when a stable key k is hit but the version information is inconsistent, or P in Cache[k] is not found. k If a value has been deleted, revert to the parsing step and reprocess it, and update the statistics vector Stats[k].

[0016] This invention also discloses a standardization and pre-compilation system for filtering expressions in real-time databases, characterized by comprising: The parsing module receives the filter expression e from the real-time database query request and parses it to obtain the abstract syntax tree AST(e). The standardization module is used to standardize the obtained abstract syntax tree AST(e) under the premise of semantic equivalence, so as to obtain the normalized AST*(e). The key generation module is used to perform stable serialization on the normalized AST to obtain a stable representation Ser(AST*(e)), and generate a stable key k by combining version information; The statistics module is used to maintain a statistics vector Stats[k] for each stable key k, and update Stats[k] each time a stable key k is hit; The decision module is used to construct a scoring function based on the statistical vector Stats[k], and to trigger the pre-compilation process when the calculation result of the scoring function meets the preset trigger threshold. The caching module is used to compile the normalized AST*(e) into a predicate execution object P. k and P k Write to the cache Cache[k]; The execution module is used to allow subsequent requests that hit the stable key k to skip the parsing and compilation phases and directly execute P in Cache[k]. k .

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses a standardization and pre-compilation method for filtering expressions in real-time databases. Based on Abstract Syntax Tree (AST) expression standardization, this method unifies semantically equivalent but formally different filtering expressions into a stable canonical form, and generates stable keys to achieve consistency identification, thereby fundamentally improving cache hit rate and reducing duplicate compilation objects. Furthermore, the method introduces a joint judgment based on cumulative access frequency, time-weighted popularity, and structural complexity. Pre-compilation is dynamically triggered based on sufficiently high access frequency and complexity, while automatic deletion occurs if the expression has not been called for a long time. This controls cache pollution and memory overhead while ensuring benefits. Simultaneously, metadata / rule version fingerprints are introduced into keying and cache objects to ensure secure invalidation after metadata changes, reducing the risk of false hits caused by semantic drift, and overall improving the throughput, stability, and tail latency performance of industrial real-time queries. Attached Figure Description

[0018] Figure 1 This is a flowchart of the standardization and pre-compilation method for filtering expressions in real-time databases according to the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] This invention discloses a standardization and pre-compilation method for filtering expressions in real-time databases, comprising the following steps: S1: Receives the filter expression e from the real-time database query request and parses it to obtain the abstract syntax tree AST(e). S2: Standardize the obtained abstract syntax tree AST(e) under the premise of semantic equivalence to obtain the normalized AST*(e). S3: Perform stable serialization on the normalized AST to obtain a stable representation Ser(AST*(e)), and generate a stable key k by combining version information, which is used to uniquely identify the normalized semantic equivalence filtering expression; S4: Maintain a statistical vector Stats[k] for each stable key k, the statistical vector... It includes cumulative access counts, time-weighted heat, and structural complexity-related metrics, and updates Stats[k] each time the stable key k is hit; S5: Construct a scoring function based on the statistical vector Stats[k]. The scoring function is positively correlated with the cumulative number of visits, time-weighted popularity, and structural complexity. When the calculation result of the scoring function meets the preset trigger threshold, the pre-compilation process is triggered. When the calculation result of the scoring function is lower than the preset cancellation threshold, the pre-compilation hold state is cancelled. S6: When the pre-compilation process is triggered, the normalized AST is compiled into a directly executable predicate execution object (or execution plan fragment) P. k and the predicate object / plan fragment P k Write to cache[k] so that subsequent requests that hit stable key k can skip the parsing and compilation stages and directly execute P. k ; S7: Define the idle time, when P in Cache[k] is idle. k When the deletion condition is met, delete the corresponding P in the cache[k] of that state. k And restore the stable key k to its uncompiled state; if a query request arrives, the P kIf the data has been deleted or the version is inconsistent, the process reverts to step S1 and updates the statistical vector Stats[k]. When the pre-compilation trigger condition is met, dynamic pre-compilation is executed again.

[0021] Preferably, in S1, the Abstract Syntax Tree (AST) contains Boolean combination nodes and atomic predicate nodes; The atomic predicate node is denoted as The ; Where mp_id is the stability identifier of the measurement point; θ is an element in the operator set; and ξ is the comparison object. When parsing to obtain the Abstract Syntax Tree (AST), dictionary / metadata binding is performed on mp_id, and type normalization is performed on ξ=.

[0022] Preferably, in S2, the standardization process is implemented by a standardization operator NORM, which is a composite of multiple sub-operators, including at least the Strip sub-operator, the ConstFold sub-operator, and the OpNorm sub-operator; Normalized AST(e) is defined as .

[0023] Preferably, in S3, the formula for calculating the stable bond k is: ; Hash is the MurmurHash algorithm, a non-cryptographic fast hashing algorithm. The version information includes at least the test point dictionary / type / function semantic version vschema and the standardization rule version v_norm.

[0024] Preferably, in S4, the statistical vector Stats[k] includes at least the cumulative number of visits, time-weighted popularity, operator count, and structural complexity weighting; The cumulative number of visits Ncum(k) It is calculated using the following formula: ; Where, x t To determine whether the expression is matched in the t-th statistics window / event, x is the value when matched. t =1, otherwise x t =0; now is the current system time; Ncum(k) is the cumulative number of visits; The time-weighted heat index is calculated using the following formula: ; Where H(k,t) is the time-weighted heat, α is the smoothing coefficient, and α∈(0,1); The structural complexity C(k) is calculated using the following formula: C(k)=∣AST\*(e)∣ in Representation of Normalized Abstract Syntax Tree The total number of operator nodes (e.g., the sum of nodes for AND / OR / NOT / comparison / function, etc.) is used to roughly characterize the structural complexity of the filtered expression. This statistic can be adjusted by system configuration.

[0025] Preferably, in S5, the scoring function is: ; Where a, b, and c represent the weights of each indicator; C0: Complexity normalization constant (to prevent saturation from occurring too quickly when C is too large); when Time-triggered pre-compilation; hysteresis debouncing is employed. The pre-compiled state can be revoked at any time.

[0026] Preferably, in S6, the Cache[k] is defined as

[0027] in Build time; The most recent hit time.

[0028] Preferably, in S7, the idle time is defined as: ; When satisfied ; Then delete P from Cache[k]. k And restore the key to its uncompiled state.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] like Figure 1 As shown, this invention discloses a standardization and pre-compilation method for filtering expressions in real-time databases, including the following steps: Step 1: Receiving, parsing, and semantically binding filter expressions Query requests typically carry a filter expression e, which describes the filtering conditions for measurement point data in the real-time database. Upon receiving the query request, the system first inputs the filter expression e into the filter expression parsing unit. This unit performs lexical and syntactic analysis on e, constructing an abstract syntax tree (AST(e)). Each node in AST(e) contains at least Boolean combination nodes (AND ∧, OR ∨, NOT ¬) and atomic predicate nodes (comparison / set predicates). After constructing AST(e), the system uniformly abstracts atomic predicates as follows: The system uses a triplet structure, where mp_id is the stable identifier of the measurement point; θ is an element in the operator set (>, ≥, <, ≤, =, ≠, BETWEEN, IN, LIKE, etc.); ξ is the comparison object, which can be a constant, parameter placeholder, or a computable derived value. To ensure consistency in subsequent equivalence recognition and execution, the system binds the measurement point identifier to a dictionary / metadata, obtains semantic information such as the type, unit, and enumeration domain of the measurement point, and performs type normalization and standardized representation on the comparison object (e.g., converting string-form numerical values ​​to numerical values ​​consistent with the measurement point type, mapping enumeration text to enumeration code values, unifying string case or escaping rules, etc.) to obtain a standardized constant output that can be used for stable representation.

[0031] Step 2: Expression standardization (equivalence normalization) based on AST The system performs standardization processing on AST(e) under the premise of semantic equivalence to obtain a normalized abstract syntax tree AST*(e), in order to solve the problem of cache not being reusable due to the same semantics but different writing forms. Standardization can include at least the following operations: First, removing meaningless whitespace and redundant parentheses, and standardizing keyword capitalization, quotation mark style, and other writing differences; Second, folding and Booleanizing constant quantum expressions to eliminate always-true / always-false branches that do not contribute to execution; Third, normalizing equivalent operator forms (e.g., removing duplicates from the IN set and sorting them in a fixed order, normalizing range predicates such as BETWEEN into equivalent comparison combinations according to fixed rules, or adopting a fixed and unified expression for unequal comparisons); Fourth, flattening AND / OR clauses at the same level and sorting their child nodes according to deterministic rules, thereby eliminating structural differences introduced by A∧B and B∧A, different nested parentheses, etc.; Fifth, when multiple interval / comparison constraints exist for the same measurement point, they can be merged into interval form or interval set form and standardized according to the endpoint order, thereby reducing redundancy and improving consistency.

[0032] Step 3: Stable keying generation and consistency recognition To ensure that the aforementioned deterministic sorting and subsequent keying generation are independent of memory addresses or traversal order, the system defines stable serialization rules for AST*(e): generating a unique serialization representation for any subtree, independent of implementation details (e.g., serializing Boolean nodes as AND(child1,child2,…) / OR(…) / NOT(child), and serializing atomic predicates as a fixed format containing measure point identifiers, operators, and normalization constants). Based on this, the system calculates stable comparison keys for each subtree and uses these keys to sort the AND / OR child nodes at the same level. After sorting, the nodes at that level are reconstructed, thus ensuring that semantically equivalent expressions converge to the same AST*(e) after normalization.

[0033] After obtaining AST*(e), the system performs stable serialization Ser(AST*(e)) on AST*(e) and generates a stable key k by combining version information. The version information includes at least the measure point dictionary / type / function semantic version v. schema and standardized rule version v norm This is used to prevent old caches from being falsely hit after metadata or rule upgrades. The stable key can be obtained by concatenating Ser(AST*(e)) with version information and then hashing it; through this stable key, different semantically equivalent and standardized written forms will be mapped to the same identifier, thereby achieving consistent identification and cache reuse.

[0034] Step 4: Indicator Collection and Online Update The system maintains a statistical vector Stats[k] with a stable key k as the granularity, and updates the statistical information each time a query matches the stable key. To simplify implementation and avoid excessive complexity, this embodiment represents the statistical vector as Stats[k] = Ncum(k), H(k), C(k) Where Ncum(k) is the cumulative number of visits; H(k) is the time-weighted heat, used to reflect the hit intensity over a recent period and supporting natural decay; C(k) is the structural complexity metric, used to roughly reflect the complexity of the filtering expression. In an optional implementation, C(k) can be directly taken as the number of operator nodes in AST*(e) to obtain a usable complexity characterization at the lowest cost; this complexity metric can also be configured to other equivalent complexity measures according to project needs.

[0035] Step 5: Dynamic pre-compilation trigger conditions Based on Stats[k], the system constructs a scoring function Score(k) to determine whether it is worthwhile to pre-compile and cache the stable key. The scoring function can simultaneously consider three factors: cumulative access count, time-weighted heat, and structural complexity. Trigger and revocation thresholds are set to create hysteresis and avoid frequent jitter: when Score(k) reaches the preset trigger threshold Θ... on When the score(k) falls below the preset undo threshold Θ, the system triggers the pre-compilation process; off Furthermore, when the preset debouncing conditions are met (e.g., several consecutive statistical windows are all below a threshold), the system cancels the pre-compiled hold state, where Θ on >Θ off .

[0036] Step 6: Precompiled object generation and caching When pre-compilation is triggered, the system will compile AST*(e) into a predicate execution object or execution plan fragment P that can be executed directly. k And write it to the cache Cache[k]. Where, the predicate execution object P... k At a minimum, it should include fixed measurement point access, necessary type conversion, and constant processing logic, forming a short-circuit executable Boolean structure, and optionally include a vectorized or batch processing execution skeleton. Cache entries can simultaneously record version information and timestamp information for subsequent security verification and eviction management. When subsequent query requests arrive, as long as their normalized results hit the same stable key k, and the version information of the cached entry matches the current v... schema v norm If consistent, the system can skip the processes of parsing, binding, rewriting, and compiling, and directly reuse the cached P. k Filtering and executing data from a real-time database significantly reduces redundant overhead and improves response speed.

[0037] Step 7: Delete and roll back pre-compiled files that have not been used for a long time. To avoid pre-compiled objects that are not used for a long time occupying memory, this invention further introduces a cooldown deletion mechanism. The system maintains the most recent hit time for each cached entry and defines an idle duration T. idle (k)=now t last (k); When the idle time and popularity simultaneously meet the preset joint deletion conditions, the system deletes P from Cache[k]. k And restore the stable key to its uncompiled state. If a subsequent query request arrives, P kIf a deleted or cached version is inconsistent with the current version, the system will fall back to the path of real-time resolution, standardization, and compilation execution, while continuing to update Stats[k]. When the stable key meets the pre-compilation trigger condition again, dynamic pre-compilation will be executed again to achieve a balance between performance gains and resource consumption.

[0038] Example 1 The real-time database contains three measurement points: temp, pressure, and status. Users frequently submit the same semantic filter in two different formats: expression eA is "(pressure<50 AND temp>80) OR status IN ('WARN','ALARM')", and expression eB is "status IN ('ALARM','WARN') OR (temp>80 ∧ pressure<50)". The two expressions differ only in the order of parentheses, clauses, and elements within the IN set; they are semantically equivalent. After receiving eA and eB, the system parses the filter expressions to generate AST(eA) and AST(eB), and binds the field names in the expressions to the measurement point stability identifier mp_id (e.g., pressure is bound to mp). pressure At the same time, 'WARN' / 'ALARM' are bound to the corresponding enumeration field values ​​according to the test point dictionary to ensure semantic consistency between subsequent standardization and execution process.

[0039] The system then performs normalized NORM on the two syntax trees under the premise of semantic equivalence: first, it eliminates redundant brackets and whitespace differences using Strip; then, it uses OpNorm to remove duplicate elements from the set IN('WARN','ALARM') and sorts them in a fixed order to {'ALARM','WARN'}; next, it uses OrderNorm to perform deterministic sorting on the child nodes of the top-level "∨" and flattens and sorts the "∧" clauses, so that the suborder of (pressure<50∧temp>80) converges to a unique canonical order; since this example does not involve merging multiple intervals at the same measure point, RangeNorm is not triggered. After this normalization, eA and eB obtain the same normalized syntax tree AST*.

[0040] After obtaining AST*, the system performs stable serialization Ser(AST*) on it and combines Ser(AST*) with version information v. schema v norm The same stable key k1 is obtained by inputting the stable key generation formula together; therefore, no matter whether the user submits eA or eB, as long as the standardization result is consistent, the same key k1 will be hit, thus achieving consistent recognition of "synonymous but different spellings".

[0041] The system maintains and updates the statistical vector online with a stable key k1 as the granularity. Assuming updates are in 1-minute time slices, and the equivalent window is w=9, the smoothing coefficient can be α=0.2. When the expression is hit at least once per minute for 10 consecutive minutes, let the hit indicator x for each time slice be... t =1, then the time-weighted heat recursion is: H1=0.2, H2=0.2+0.8·0.2=0.36, …, H 10 ≈0.8926; Meanwhile, the cumulative number of visits increases with the time slice, and N can be obtained within this 10-minute period. cum =10 (corresponding to ln(1+N) cum =ln(11)).

[0042] Meanwhile, the system calculates the structural complexity of AST* by weighting, and obtains the complexity weighting result, which is used as the normalized ratio C / (C0+C) in the scoring item.

[0043] When triggering the pre-compilation determination, the system uses a scoring method, with a=1, b=2, c=1, C0=10, and a trigger threshold Θ. on =4.0, and calculated according to the scoring function: the first term is n(1+N) cum =ln(11)≈2.3979; the second term is b·H=2·0.8926≈1.7852; the third term is c·C / (C0+C)=14.95 / (24.95)≈0.5992; the sum of the three terms is Score≈4.7823, which is greater than Θ. on Therefore, the triggering condition is met, and the pre-compilation process begins.

[0044] After entering the pre-compilation stage, the system compiles AST* to generate a pre-compiled object P_{k1} (fixed test point access, type conversion, comparison operators and Boolean short-circuit logic), and writes it to the cache Cache[k1]. Thereafter, synonym expressions (regardless of eA / eB) will hit k1 and directly reuse P_{k1} for execution, thereby skipping the repeated parsing / normalization / compilation overhead.

[0045] If this expression is not called again for a long period of time, assuming there are no hits for the next 20 consecutive time slices, then for each time slice x... t When m=0, the heat H will decay naturally and recursively; when m=20, the heat decays to a lower level. The system sets an idle time threshold T. idle =20min and heat removal threshold H del =0.05. When the combined deletion conditions of idle time reaching the threshold and popularity being lower than the threshold are met, the pre-compiled object in Cache[k1] is deleted and the key is restored to the uncompiled state. If it becomes popular again later, the dynamic pre-compilation can be re-triggered according to the same statistical update and scoring judgment process.

[0046] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A standardization and pre-compilation method for filtering expressions in real-time databases, characterized in that, Includes the following steps: S1: Receives the filter expression e from the real-time database query request and parses it to obtain the abstract syntax tree AST(e). S2: Standardize the obtained abstract syntax tree AST(e) under the premise of semantic equivalence to obtain the normalized AST*(e). S3: Perform stable serialization on the normalized AST*(e) to obtain the stable representation Ser(AST*(e)), and generate a stable key k by combining the version information; S4: Maintain a statistical vector Stats[k] for each stable key k, and update Stats[k] each time a stable key k is hit; S5: Construct a scoring function based on the statistical vector Stats[k]. Trigger the pre-compilation process when the calculation result of the scoring function meets the preset trigger threshold. S6: Compile the normalized AST*(e) into a predicate execution object P. k and P k Write to the cache Cache[k]; S7: Subsequent requests that hit the stable key k skip the parsing and compilation phases and directly execute P in Cache[k]. k .

2. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 1, characterized in that, In S1, the abstract syntax tree AST(e) contains Boolean combination nodes and atomic predicate nodes; The atomic predicate node is denoted as The ; Where mp_id is the stability identifier of the measurement point; θ is an element in the operator set; and ξ is the comparison object. When parsing to obtain the Abstract Syntax Tree (AST), dictionary / metadata binding is performed on mp_id, and type normalization is performed on ξ=.

3. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 1, characterized in that, In S3, the formula for calculating the stable bond k is: ； Where Hash is the MurmurHash algorithm, a non-cryptographic fast hashing algorithm, and v schema For the dictionary / type / function semantic version of the test points, v norm This is a standardized rules version.

4. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 3, characterized in that, The statistical vector Stats[k] includes at least the cumulative number of visits N. cum (k), time-weighted heat H(k), structural complexity C(k); Wherein, the cumulative number of visits N cum (k) Obtained by accumulating the hit indicators within the sliding time window: ； Where, x t To determine whether the expression is matched in the t-th statistics window / event, x is the value when matched. t =1, otherwise x t =0; now is the current system time; Ncum(k) is the cumulative number of visits; The time-weighted heat H(k) is calculated using an exponential smoothing recursive formula: ； Where H(k,t) is the time-weighted heat, α is the smoothing coefficient, α∈(0,1); X t X is an indicator variable for whether the stable key is hit within the t-th time window, and when it is hit... t =1, otherwise X t =0; Structural complexity C(k) = |AST*(e)|; in Representation of Normalized Abstract Syntax Tree The total number of operator nodes is used to roughly represent the structural complexity of the filtering expression.

5. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 4, characterized in that, In S5, the scoring function is: ； Where a, b, and c are the weights of each indicator; C0 is the complexity normalization constant; And when Pre-compilation is triggered on time; hysteresis debouncing is used when... Undo the pre-compiled state at that time, and Hysteresis anti-shake is formed.

6. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 5, characterized in that, In S6, the cache[k] is defined as: ； in, For build time, The most recent hit time.

7. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 6, characterized in that, In S7, the idle time is defined as: T idle (k) =now-t last (k); When T idle (k)≥T threshold And H(k)≤H del When, delete P from Cache[k]. k And restore the stable key k to its uncompiled state; where T threshold H is the idle time threshold. del This is the threshold for removing content based on popularity.

8. The standardization and pre-compilation method for filtering expressions in real-time databases according to claim 1, characterized in that, The standardization steps include at least one of the following operations: Eliminate writing layer differences, which include whitespace characters, redundant parentheses, keyword capitalization, and quotation mark style; Fold the constant expression and perform Boolean simplification; Perform normalization transformation on the equivalent operator form; Flatten clauses at the same logical level and sort them according to deterministic rules; Merge multiple interval constraints at the same measurement point and standardize them according to the order of endpoints.

9. The method according to claim 1, characterized in that, The reuse step further includes: when a stable key k is hit but the version information is inconsistent, or P in Cache[k] is not found. k If a value has been deleted, revert to the parsing step and reprocess it, and update the statistics vector Stats[k].

10. A standardization and pre-compilation system for filtering expressions in real-time databases, characterized in that, include: The parsing module receives the filter expression e from the real-time database query request and parses it to obtain the abstract syntax tree AST(e). The standardization module is used to standardize the obtained abstract syntax tree AST(e) under the premise of semantic equivalence, so as to obtain the normalized AST*(e). The key generation module is used to perform stable serialization on the normalized AST to obtain a stable representation Ser(AST*(e)), and generate a stable key k by combining version information; The statistics module is used to maintain a statistics vector Stats[k] for each stable key k, and update Stats[k] each time a stable key k is hit; The decision module is used to construct a scoring function based on the statistical vector Stats[k], and to trigger the pre-compilation process when the calculation result of the scoring function meets the preset trigger threshold. The caching module is used to compile the normalized AST*(e) into a predicate execution object P. k and P k Write to the cache Cache[k]; The execution module is used to allow subsequent requests that hit the stable key k to skip the parsing and compilation phases and directly execute P in Cache[k]. k .

Images