Data query method and system based on semantic recognition

Abstract

The invention discloses a data query method and system based on semantic recognition, and relates to the technical field of data processing. The method comprises the following steps: synchronously updating an incremental change log to a dynamic knowledge graph containing an entity relationship library and a text index library through the incremental change log of multi-source heterogeneous business data based on a business dimension and a data type dimension; in response to the query request, sequentially performing retrieval based on a multi-level cache system comprising a high-frequency dynamic cache layer and a low-frequency historical cache layer, and triggering a semantic analysis process if the query request is not hit; analyzing and generating a structured query instruction by utilizing the pre-training language model after fine tuning, executing a retrieval enhancement generation operation based on a graph neural network, and screening a candidate evidence chain; and inputting the candidate evidence chain into the generative model, generating a final query reply in combination with a dynamic desensitization strategy corresponding to the user permission, and asynchronously backfilling the final query reply to the multi-level cache system. According to the method, the real-time performance of data query, the semantic understanding accuracy and the data security can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data query method and system based on semantic recognition. Background Technology

[0002] With the deepening of digital transformation, various large management institutions and enterprises have accumulated massive amounts of multi-source heterogeneous data. This data covers various forms, including structured business registration information and transfer records, as well as unstructured normative documents and business guidelines. To manage and query this data, current mainstream technical solutions typically adopt a layered architecture of "database + search engine". At the data processing level, it mainly relies on daily scheduled extraction-transform-load (ETL) batch processing tasks to synchronize changed data from business systems to the query database. The data update cycle is usually locked at the daily or hourly level. At the query interaction level, the system often provides a fixed application programming interface (API) or a keyword-matching search interface, requiring users to accurately input specific fields or predefined terms to obtain information.

[0003] However, this traditional architecture exposes multiple technical bottlenecks when facing increasingly complex real-time query demands. First, the batch-processing-based data synchronization mechanism results in severe data timeliness delays, failing to support real-time business query needs during peak periods or sudden surges. Furthermore, the lack of efficient consistency verification methods during the fusion of multi-source heterogeneous data easily leads to version conflicts. Second, existing question-answering or query engines lack deep semantic understanding capabilities, relying heavily on static rule bases or shallow text matching. This fails to accurately parse ambiguous references, multiple intents, and cross-domain logic in natural language, often resulting in inaccurate or irrelevant query results. Additionally, in high-concurrency access scenarios, system caching strategies are often simplistic, lacking intelligent hierarchical scheduling for hot and cold data, easily causing resource congestion and response timeouts. Finally, regarding data security and compliance, existing solutions mostly employ static data display rules, making it difficult to dynamically adjust the desensitization of sensitive information based on different user permission levels, posing risks of data leakage or data unavailability due to over-control.

[0004] In summary, existing business data query methods rely excessively on batch processing and keyword matching, resulting in an inability to accurately and efficiently meet real-time business query needs, and insufficient flexibility in security management. Therefore, there is an urgent need for an intelligent data query technology solution that integrates real-time data processing, deep semantic understanding, and dynamic security management. Summary of the Invention

[0005] This invention provides a data query method and system based on semantic recognition, solving the problems of poor data timeliness, weak semantic understanding, slow concurrent response, and rigid security control in existing technologies. To achieve the above objectives, the embodiments of this invention adopt the following technical solutions: In a first aspect, embodiments of the present invention propose a data query method based on semantic recognition, the method comprising: Based on a dual-strategy partitioning mechanism of business and data type dimensions, incremental change logs of multi-source heterogeneous business data are captured through real-time data pipelines, and the incremental change logs are synchronously updated to a dynamic knowledge graph containing entity relation databases and text index databases using a vector clock synchronization strategy, thus building the data foundation for semantic query. In response to a user's natural language query request, based on a multi-level caching system that includes a high-frequency dynamic caching layer and a low-frequency historical caching layer, the system sequentially searches the high-frequency dynamic caching layer and the low-frequency historical caching layer. If the cache is hit, the query result is returned directly; if the cache is not hit, the semantic analysis process is triggered. The fine-tuned pre-trained language model is used to perform multi-level semantic disambiguation and intent recognition on natural language query requests, and to parse and generate structured query instructions containing entity objects, intent types and constraints. Based on structured query commands, a retrieval enhancement generation operation based on graph neural networks is performed in a dynamic knowledge graph, and candidate evidence chains are selected according to the timeliness weight of evidence. The candidate evidence chain is input into the generative model, and the final query response is generated by combining the dynamic de-identification strategy corresponding to the user's permissions. The final query response is then asynchronously backfilled into the multi-level caching system according to the access frequency strategy.

[0006] Preferably, a dual-strategy partitioning mechanism based on business and data type dimensions is used to capture incremental change logs of multi-source heterogeneous business data through real-time data pipelines, including: The database log connector is used to monitor the binary log of the source database in real time and capture incremental operation data, including insert, update and delete operations. The basic partition is determined by using the business identifier hash modulo, and a secondary partition is performed by overlaying the data type hash modulo, so that incremental operation data is routed to the corresponding message queue partition; Based on the real-time throughput of each partition, when the throughput of a specific partition exceeds the hotspot threshold within a preset time window, a dynamic partition migration strategy is triggered to migrate the hotspot business data to an idle partition. During data transmission, a strategy combining future time filtering and business whitelisting is adopted to verify the timestamps in incremental operation data, remove abnormal future time data that exceeds the preset tolerance window, and retain the pre-approval process data in the whitelist.

[0007] Preferably, the incremental change log is synchronously updated to the dynamic knowledge graph containing the entity relation database and the text index database using a vector clock synchronization strategy, including: Construct a vector clock triple containing engine type, operation sequence number, and timestamp as a credential for data consistency verification; The structured entity relationship data in the incremental operation data is routed to a graph database as an entity relationship database, and the unstructured text data in the incremental operation data is routed to a search engine as a text index database. When performing a write operation, the operation sequence number of the corresponding engine is automatically incremented and the current system timestamp is updated synchronously. When a synchronization conflict is detected between the entity relation database and the text index database for the same data object, the operation sequence number and timestamp of the corresponding vector clocks in the entity relation database and the text index database are compared. The last write wins the strategy to force the data state to be unified, and the operation trajectory is retained for traceability.

[0008] Preferably, the method further includes dynamic segmentation and relationship extraction of the dynamic knowledge graph by business area, specifically including: The entity relation database is sharded and stored according to the business zone code. When adding or merging business zones, a zero-loss migration process is executed, which includes pre-migration dual write, switch verification and backup cleanup. An entity relation extraction network based on a pre-trained language model is adopted. The business entity recognition capability is enhanced through the masked language model task, and adversarial examples are introduced to fine-tune the relation classifier. The fine-tuned entity relationship extraction network is used to extract business entities and relationships from unstructured text data, and the extraction results are incrementally updated to the entity relationship database through a dual trigger mechanism that combines time-driven and event-driven approaches.

[0009] Preferably, the fine-tuned pre-trained language model is used to perform multi-level semantic disambiguation and intent recognition on natural language query requests, including: Based on the geographic information system service, the geographic location terms in the natural language query request are parsed to determine the business area involved in the query. If the location fails, a fallback strategy is activated. By combining the user's historical interaction context and user profile features, semantic completion and referential resolution are performed on fuzzy pronouns in natural language query requests; The disambiguated query text is input into the intent recognition model, which is finely tuned based on the RoBERTa architecture, to identify the user's business query intent. The intent recognition model is incrementally trained using a business corpus containing long-tail intents. If the confidence level of the identified intent is lower than a preset threshold, the logic for recommending similar questions is triggered, and low-confidence queries are recorded for subsequent model optimization.

[0010] Preferably, based on structured query instructions, a retrieval enhancement generation operation based on graph neural networks is performed in the dynamic knowledge graph, including: Using the GraphSAGE graph neural network model, entity attribute features and text sentence vector features in dynamic knowledge graphs are fused to generate node embedding vectors; Calculate the cosine similarity between the vector representation of the structured query instruction and the node embedding vector, and retrieve the Top-K related nodes with similarity higher than a preset threshold as the initial evidence set; Construct a multi-dimensional evidence evaluation system, and calculate the comprehensive score of each piece of evidence in the initial evidence set based on the authority of the evidence source, the relevance of the content, and the timeliness of the publication time. A timeliness weighting formula is introduced to reduce the weight of evidence with time conflicts or filter it out. Candidate evidence chains are selected based on the comprehensive score. The timeliness weighting formula gives higher weight to the most recently released normative documents.

[0011] Preferably, the candidate evidence chain is input into the generative model, and the final query response is generated by combining it with a dynamic de-identification strategy corresponding to user permissions, including: Dynamic desensitization instructions are embedded in the prompt word templates of the generative model. These instructions define the processing rules for data with different sensitivity levels. Identify the user's permission level when initiating the query, and match the corresponding de-identification strategy based on the user's permission level; For sensitive personal information, we use masking or truncation for high-intensity de-identification; for sensitive corporate data, we use aggregation statistics or range display methods for processing. The candidate evidence chain and the prompt words with embedded de-identification instructions are input into the generative model to generate a natural language response text that meets compliance requirements.

[0012] Preferably, based on a multi-level caching system comprising a high-frequency dynamic cache layer and a low-frequency historical cache layer, the high-frequency dynamic cache layer and the low-frequency historical cache layer are sequentially retrieved, including: Data that has been accessed more than the first threshold and updated more than the second threshold within the most recent preset time period is defined as high-frequency dynamic data, stored in a high-frequency dynamic cache layer based on an in-memory database, and has a short lifespan. Data with fewer than one access frequency or a lower than two update frequency within a recent preset time period is defined as low-frequency historical data, stored in a low-frequency historical cache layer based on a search engine, and updated daily. When a user query request arrives, the high-frequency dynamic cache layer is queried first; if no match is found, the low-frequency historical cache layer is queried. If the target data is hit in the low-frequency historical cache layer, and the access characteristics of the target data meet the potential high-frequency conditions, the target data will be automatically promoted and backfilled to the high-frequency dynamic cache layer.

[0013] Preferably, the method further includes dynamically adjusting resources in the multi-level caching system: The cached data is serialized and stored using a binary compression format; Real-time monitoring of memory usage in the high-frequency dynamic cache layer; when memory usage exceeds a preset warning threshold and persists for a preset duration, automatic expansion process is triggered. Real-time monitoring of the shard query performance of the low-frequency historical cache layer; when the query latency of a single shard exceeds the preset performance threshold, the number of shards is automatically adjusted and the data distribution is balanced. Based on historical query log analysis, a hybrid eviction strategy combining the Least Recently Used (LRU) algorithm and the Least Frequently Used (LRU) algorithm is adopted to clean up expired or cold data in the high-frequency dynamic cache layer.

[0014] Secondly, embodiments of the present invention propose a data query system based on semantic recognition, the system comprising: The dynamic knowledge graph construction module is used for a dual-strategy partitioning mechanism based on business and data type dimensions. It captures incremental change logs of multi-source heterogeneous business data through real-time data pipelines and uses a vector clock synchronization strategy to synchronously update the incremental change logs to a dynamic knowledge graph that includes an entity relation database and a text index database. The multi-level cache retrieval module is used to respond to natural language query requests initiated by users. Based on a multi-level cache system that includes a high-frequency dynamic cache layer and a low-frequency historical cache layer, it sequentially searches the high-frequency dynamic cache layer and the low-frequency historical cache layer. If the cache is hit, the query result is returned directly; if the cache is not hit, the semantic analysis process is triggered. The semantic deep parsing module is used to perform multi-level semantic disambiguation and intent recognition on natural language query requests using a fine-tuned pre-trained language model, and to parse and generate structured query instructions containing entity objects, intent types and constraints. The graph enhancement generation module is used to perform graph neural network-based retrieval enhancement generation operations in dynamic knowledge graphs based on structured query instructions, and to filter candidate evidence chains according to the evidence timeliness weight. The security response processing module is used to input candidate evidence chains into the generative model, combine them with the dynamic de-identification strategy corresponding to user permissions to generate the final query response, and asynchronously backfill the final query response to the multi-level caching system according to the access frequency strategy.

[0015] Beneficial effects: This invention provides a data query method based on semantic recognition. By constructing a real-time data pipeline based on dual-strategy partitioning and introducing a vector clock synchronization strategy, it achieves millisecond-level capture and high-consistency fusion of multi-source heterogeneous data, effectively overcoming the data lag and version conflict problems caused by batch processing in existing technologies, and building a real-time and accurate data foundation for upper-layer applications. This invention utilizes a fine-tuned pre-trained language model combined with a graph neural network to achieve deep semantic disambiguation and intent recognition of user natural language query requests, and performs retrieval enhancement generation based on a dynamic knowledge graph, significantly improving query accuracy and the interpretability of evidence chains in complex semantic environments, solving the deficiency of traditional keyword retrieval in understanding the user's true intent. Furthermore, this invention constructs a multi-level caching system including high-frequency dynamic and low-frequency historical data, which can intelligently route query requests based on data popularity, avoiding the performance bottleneck of a single caching strategy in high-concurrency scenarios; combined with a dynamic desensitization strategy for user permissions, it achieves fine-grained security control of data output, ensuring the compliant use of sensitive information while ensuring the flexibility and efficiency of data services. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart of a data query method based on semantic recognition proposed in an embodiment of the present invention; Figure 2 This is a schematic diagram of a data query system based on semantic recognition proposed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0017] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the invention, but does not constitute a limitation on the invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0019] The first aspect, reference Figure 1 This invention provides a semantic recognition-based data query method, which can be executed on a computer device. The computer device can be a server, a personal computer (PC), or an embedded semantic recognition-based data query system with sufficient computing power. This device typically includes, but is not limited to, a processor, memory, communication interfaces, and input / output devices. The memory stores computer program instructions, and the processor executes these instructions to implement the semantic recognition-based data query method of this invention. This method is suitable for data service platforms of large enterprises or industry management organizations. This solution aims to address the technical challenges faced in existing business data queries, such as insufficient real-time performance, weak semantic understanding capabilities, slow high-concurrency response, and rigid data access control. Figure 1 As shown, the data query method based on semantic recognition mainly includes steps S101 to S105.

[0020] S101. A dual-strategy partitioning mechanism based on business and data type dimensions captures incremental change logs of multi-source heterogeneous business data through real-time data pipelines, and uses a vector clock synchronization strategy to synchronously update the incremental change logs to a dynamic knowledge graph containing entity relation databases and text index databases, thus building the data foundation for semantic queries.

[0021] This step addresses the real-time fusion and consistency issues of multi-source heterogeneous business data (such as structured business registration information and transfer records versus unstructured normative documents and business guidelines). By constructing a low-latency, highly reliable real-time data pipeline, real-time data synchronization from the source database to the target knowledge graph is achieved. Specifically, step S101 includes the following sub-steps: S1011. Monitor the binary log of the source database in real time through the database log connector to capture incremental operation data, including insert, update and delete operations.

[0022] In this embodiment, the source database may include various types of relational databases such as Oracle and MySQL, storing basic business data from various business departments. To avoid the performance pressure and data latency caused by the traditional Extract-Transform-Load (ETL) batch processing method on the production database, this system adopts a technology approach based on Change Data Capture (CDC).

[0023] In practice, the open-source Debezium Binlog Connector is deployed as the database log connector. This connector acts as a client node of the database, monitoring the master node's binary log (Binlog) in real time. The system is configured with fine-grained filtering rules, monitoring only the core business tables. When data changes occur in the source database, the connector can accurately parse the specific SQL operation type. Insert operation (INSERT): Identifies newly added business records, such as newly registered entity information or newly published notification documents. In the captured data packet, this operation type is marked as a creation operation and includes all field values ​​of the row of data.

[0024] Update operation (UPDATE): Identifies changes to existing records, such as changes to business status or updates to progress. In the captured data packet, this operation type is marked as an update operation, and it includes both the field values ​​before and after the change, so that the system can perform difference comparisons and version control later.

[0025] DELETE operation: Identify deactivated or obsolete data and mark the data packet as a deletion operation to ensure that the knowledge graph can remove invalid information in a timely manner and maintain the timeliness of the data.

[0026] The captured incremental operation data is encapsulated into a unified JavaScript Object Notation (JSON) format message. The message body structure includes source information, operation type, timestamp, and data payload (data content before and after the change). This message is then pushed to a downstream message broker (such as Apache Kafka) for further processing.

[0027] S1012. Use business identifier hash modulo to determine the basic partition, and superimpose data type hash modulo to perform secondary partitioning, and route incremental operation data to the corresponding message queue partition.

[0028] To address the load balancing issue during high-concurrency writes of massive amounts of business data, this embodiment proposes a dual-strategy partitioning mechanism based on both business and data type dimensions. Traditional single-dimensional partitioning (such as partitioning only by primary key ID) can easily lead to performance bottlenecks (i.e., data skew) in queues containing large amounts of core business data, while other partitions remain idle. The execution logic of this sub-step is as follows: First, extract the "Business ID" (e.g., biz_001) and "Data Type" (e.g., type_structured represents structured records, type_unstructured represents unstructured text) from the incremental operation data in the data access layer (Source Connector).

[0029] Secondly, a double hash calculation is performed to determine the target partition index. The calculation logic is as follows: A hash operation is performed on the business identifier, and the result is modulo the base partition number to obtain the base partition index. The base partition number represents the number of major category partitions divided according to the business. A hash operation is performed on the data type, and the result is modulo the sub-partition offset to obtain the secondary offset index. The sub-partition offset is used for fine-tuning based on the data type. The base partition index and the secondary offset index are added to obtain the final target partition index.

[0030] In the preferred configuration of this system, the total number of partitions for the Kafka Topic is set to 96 (i.e., 12 basic partitions and 8 sub-partition offsets). Specifically, the MurmurHash3 algorithm is used as the hash function due to its good distribution uniformity and computational speed.

[0031] Through this dual-strategy mechanism, the system can relatively concentrate data from the same business source in a few adjacent partitions, which helps to ensure the local order of business processing (such as the order of document processing for the same business); at the same time, it can use the differences in data types to further disperse the data, effectively avoiding single-point congestion caused by excessive data volume of a single business, and realizing fine-grained routing of data flow.

[0032] S1013. Based on the sliding window, the real-time throughput of each partition is statistically analyzed. When the throughput of a specific partition exceeds the hotspot threshold within a preset time window, a dynamic partition migration strategy is triggered to migrate the hotspot business data to an idle partition.

[0033] Despite employing a dual-strategy partitioning approach, data traffic for specific business operations may still surge suddenly during peak periods (such as end-of-month settlement periods and annual audit periods), creating "hotspot partitions." To ensure system stability, a dynamic hotspot management mechanism has been introduced.

[0034] In practice, the system maintains a sliding window with a duration of 30 seconds in the stream processing engine (such as Apache Flink). This window slides forward in 1-second increments, and counts the number of bytes in input per second and the number of messages in input per second for each Kafka partition in real time.

[0035] The system has set the hotspot threshold to 10MB / s (megabytes per second).

[0036] When the monitoring component detects that the average throughput of a partition exceeds 10MB / s for 30 consecutive seconds, it immediately triggers a dynamic partition migration policy. The core logic of this policy includes: The system identifies hotspot sources and analyzes the traffic composition within those hotspot partitions. By sampling and analyzing metadata in message headers, it identifies the "hotspot service ID" contributing the most traffic. It then searches for idle targets by scanning all current partition status tables to find the "idle partition" with the lowest load (e.g., throughput below 1MB / s) and inactive state. A seamless migration is then performed, updating the routing table's mapping rules to dynamically point new data streams from the hotspot service to the idle partition. To prevent frequent jitter, the system introduces a concept similar to Direct Memory Access (DMA) bus scheduling, prioritizing the migration of core business data, and controlling the effective latency of migration operations to within 50ms. Migration commands are distributed to all Producers via a configuration center (such as ZooKeeper or Nacos) to ensure consistent changes to routing rules.

[0037] S1014. During data transmission, a strategy combining future time filtering and business whitelisting is adopted to verify the timestamps in incremental operation data, remove abnormal future time data that exceeds the preset tolerance window, but retain the pre-approval process data in the business whitelist.

[0038] In business data processing, challenges of "out-of-order" and "abnormal time" data are frequently encountered. On the one hand, distributed network transmission can lead to out-of-order data; on the other hand, some source business systems may generate future timestamps due to server clock synchronization issues or manual data entry errors. More specifically, certain appointment or public notice-type business processes inherently contain legitimate "future time" data (e.g., "appointment for next week"). To accurately clean the data, this sub-step adopts the following logic: Timestamp verification: Real-time parsing of the business occurrence time (Event Time, usually corresponding to the database's Binlog Timestamp) in incremental data.

[0039] Future Time Filtering: A "preset tolerance window" is set. The deadline of this window is calculated by obtaining the current system time of the data processing server and adding an allowable clock drift threshold (e.g., 2 minutes). If the timestamp of the data exceeds the deadline of the preset tolerance window, the system initially determines that it is abnormal data (possibly due to clock errors or malicious attacks).

[0040] Whitelist Exemption: Before removing data, the system checks whether the business process ID to which the data belongs is in the "business whitelist". The whitelist includes all business scenarios that are allowed to have future times (such as "appointment processing" and "public notice period deadline").

[0041] If the data is on the whitelist, it will be retained and passed on to downstream users even if the timestamp is from the future.

[0042] If data is not on the whitelist and exceeds the tolerance window, it is treated as dirty data and intercepted, and recorded in the Dead Letter Queue for subsequent auditing, thereby ensuring the correctness of the data entering the knowledge graph in terms of time logic.

[0043] S1015. Construct a vector clock triple containing engine type, operation sequence number, and timestamp as a credential for data consistency verification.

[0044] To address the version conflict issue when synchronizing multimodal data (structured and unstructured) between heterogeneous storage engines, this embodiment introduces a vector clock mechanism.

[0045] The system attaches a metadata tag, namely a vector clock triple, to each flowing data object. Its structure consists of the following three ordered elements: EngineType: An enumeration value that identifies the current storage location or source of the data (e.g., representing a graph database or search engine).

[0046] Operation Sequence Number (OpSeq): A 64-bit long integer, monotonically increasing, used to identify the order of modifications made to a specific data object. Each update operation increments this sequence number by 1.

[0047] Timestamp: The precise system time at which the operation occurred (unified to UTC+8 time zone, accurate to milliseconds).

[0048] This triple will serve as a "version credential" for the data throughout its lifecycle, used for subsequent consistency checks and conflict arbitration. During message transmission, the triple is serialized into Protobuf binary format and stored in the message header to reduce transmission overhead.

[0049] S1016. Route the structured entity relationship data in the incremental operation data to the graph database as the entity relationship database, and route the unstructured text data in the incremental operation data to the search engine as the text index database.

[0050] This system adopts a dual-engine storage architecture of "graph database + search engine" to adapt to the multimodal characteristics of business data.

[0051] Entity Relationship Database: Neo4j is used as the graph database. For structured information in incremental data (such as entity names, unique identifiers, related personnel, hierarchical relationships, business records, etc.), the system uses an ETL transformation program to parse it into "nodes" and "relationships" in the graph. For example, a record "Company A is wholly owned by Company B" is transformed into two company nodes in the graph database and a holding relationship edge between them. This data supports subsequent deep reasoning based on relationships.

[0052] Text Index Repository: Elasticsearch is used as the search engine. For unstructured long text in incremental data (such as full-text normative documents, business guide details, meeting minutes, unstructured notes, etc.), the system routes them to Elasticsearch for inverted index storage. The system is configured with IK Analyzer for Chinese word segmentation and has built a multi-field index including title, content, and attachment. This data supports full-text search and fuzzy matching.

[0053] This separate storage strategy fully leverages the respective advantages of graph databases in handling complex relational queries and search engines in handling full-text retrieval.

[0054] S1017. When performing a write operation, automatically increment the operation sequence number of the corresponding engine and synchronously update the current system timestamp.

[0055] To maintain the accuracy of the vector clock and ensure the traceability of data state, the system embeds version control logic within the atomic transactions of write operations performed by the storage engine. When data is written to Neo4j or Elasticsearch, the Sink Connector locks the metadata record of the data object. Before performing the write, it reads the current storage operation sequence number, increments it by 1 to get the new operation sequence number, and updates the timestamp to the millisecond-level timestamp of the current write operation.

[0056] This process utilizes Redis's Distributed Lock mechanism to ensure that, in a concurrent environment, the version evolution of the same data object is linear and strictly monotonically increasing, avoiding version rollback.

[0057] S1018. When a synchronization conflict is detected between the entity relation database and the text index database for the same data object, the operation sequence number and timestamp of the vector clock corresponding to the entity relation database and the text index database are compared respectively. The last write wins strategy to force the data state to be unified, and the operation trajectory is retained for traceability.

[0058] Since Neo4j and Elasticsearch are two independent distributed systems, differences in network latency or processing speed can lead to inconsistencies in dual-write operations (for example, an update operation may be completed in ES first, but delayed in Neo4j, in which case users may query inconsistent states).

[0059] When the system's background consistency checker detects inconsistencies in the state hash values ​​of the same data object (identified by a globally unique primary key) across two databases, it triggers the conflict resolution mechanism: Obtain Vector Clock: Read the vector clock of the data from both databases. Comparison Logic: First, compare the operation sequence number. If the operation sequence number in one database is greater than that in the other, the former is considered the newer version. If the operation sequence numbers are the same (a very rare case, possibly due to a coincidental concurrent double write), then compare the timestamps. The one with the larger timestamp is the newer version. Last Write Wins (LWW): The system forces the older version of the data to be updated to the newer version. For example, if the data version in the graph database is updated, the system will generate a synchronization command to force a refresh of the corresponding record in the search engine, achieving eventual consistency.

[0060] Tracking: All conflict handling processes, overwritten old version data, and vector clock change records are written to a dedicated audit log, which is stored in HBase for subsequent data tracing and system problem investigation.

[0061] S1019. Perform dynamic segmentation and relation extraction of the dynamic knowledge graph for business areas.

[0062] To further optimize the query performance of large-scale graphs and enrich their semantic relationships, this embodiment also includes optimization steps for graph storage and construction: Dynamic Sharding by Business Region: Considering that business queries often have obvious regional characteristics, the system physically shards and stores data in Neo4j based on "business region codes" (such as specific region codes). When a business region changes (such as adding a new region or merging an old region), the system executes a "zero-loss migration process": Pre-migration dual write: During the migration, new data is written to both the old and new shards simultaneously, maintaining two copies of the data.

[0063] Switching verification: Perform a full verification on the new data shard, comparing the number of records and the checksum.

[0064] Backup and cleanup: After confirming that everything is correct, switch traffic to the new shard and asynchronously delete the old shard data.

[0065] Relation Extraction Optimization: The system deploys an entity relation extraction network based on BERT-base-Chinese fine-tuning. To adapt to the specific terminology of particular business domains, a masked language model task is used for pre-training reinforcement. Simultaneously, adversarial examples are introduced when training the relation classifier. By subtly perturbing the input text (such as synonym replacement or noise injection), the model is trained to identify and ignore these perturbations, thereby improving its robustness in handling complex and non-standard business texts in the real world.

[0066] Dual-trigger update mechanism: Newly extracted entities and relationships are incrementally updated to the entity-relationship database through a combination of "time-driven" (reconstruction every 2 hours) and "event-driven" (triggered when the data change rate exceeds 5%) mechanism to ensure the timeliness of the graph.

[0067] S102. In response to a natural language query request initiated by the user, based on a multi-level caching system that includes a high-frequency dynamic caching layer and a low-frequency historical caching layer, the high-frequency dynamic caching layer and the low-frequency historical caching layer are searched sequentially. If the cache is hit, the query result is returned directly. If the cache is not hit, the semantic analysis process is triggered.

[0068] This step aims to intercept a large number of duplicate queries through an efficient caching strategy, thereby reducing the computational burden on the backend AI model and improving system response speed. This embodiment constructs an intelligent hierarchical multi-level caching system. Specific sub-steps include: S1021. Data that has been accessed more than a first threshold and updated more than a second threshold within a recent preset time period is defined as high-frequency dynamic data, stored in a high-frequency dynamic cache layer based on an in-memory database, and a short-term lifespan is set.

[0069] The system performs fine-grained classification of cached data.

[0070] High-frequency dynamic caching layer (Level 1 Cache): Uses Redis Cluster as the storage medium.

[0071] Admission criteria: Data must have been accessed at least 5 times within the last 10 minutes (preset time period) (first threshold), and the source data must have been updated at least once per hour (second threshold). This type of data typically reflects user activity and changes rapidly (e.g., "Today's business volume" or "Real-time queue status").

[0072] Time To Live (TTL): Set to 10 minutes. This short TTL ensures extremely high real-time data retrieval for users, avoiding the caching of outdated data. Redis keys are designed using a combination of business type and query hash, while values ​​are stored using Protobuf serialization.

[0073] S1022. Data with fewer accesses than the first threshold or fewer update frequencies than the second threshold within the most recent preset time period is defined as low-frequency historical data, stored in a low-frequency historical cache layer based on a search engine, and updated daily.

[0074] Low-frequency historical cache layer (Level 2 Cache): Uses Elasticsearch's independent indexes as storage medium, leveraging its powerful retrieval capabilities and relatively low storage costs.

[0075] Admission rules: Data that is accessed infrequently, or that is updated slowly or even stagnant (such as "annual statistical reports" or "historical normative documents").

[0076] Update mechanism: A daily update strategy is adopted, that is, a batch refresh is performed once every day at midnight to reduce unnecessary I / O overhead.

[0077] S1023. When a user query request arrives, the high-frequency dynamic cache layer is queried first. If the cache is not found, the low-frequency historical cache layer is queried.

[0078] The query routing logic follows a waterfall pattern of "L1>L2>Backend": Users initiate natural language queries (such as "Check the number of business registrations in a certain area today").

[0079] The system first calculates the MD5 hash fingerprint of the query statement and searches for it in Redis (L1). If a match is found, the deserialized result is returned directly, with a response time typically within 5 milliseconds.

[0080] If the Redis match is not found, the system then searches in the Elasticsearch (L2) cache index.

[0081] S1024. If the target data is hit in the low-frequency historical cache layer and the access characteristics of the target data meet the potential high-frequency conditions, the target data will be automatically promoted and backfilled to the high-frequency dynamic cache layer.

[0082] To adapt to the dynamic changes in trending topics (for example, an old regulation suddenly gains attention due to news reports), the system is designed with a cache "heating up" mechanism.

[0083] If data is found in L2, the system will evaluate the recent access trend of that data. If it finds that the access frequency is surging and meets the "potential high-frequency condition" (e.g., access increases to more than 20 times in the past hour), the system will automatically copy the data from L2 and backfill it to L1 (Redis), and give it the default TTL, so that it can enjoy faster access speed and realize dynamic adaptation of caching strategy.

[0084] S1025. Dynamically adjust resources for the multi-level caching system. To ensure high availability and resource utilization of the caching system, this embodiment also includes automated resource management: Serialization optimization: The cached data is serialized and stored using the MsgPack binary compression format, which reduces the size by about 60%-70% compared to the traditional JSON format, significantly saving memory space and network bandwidth.

[0085] L1 scaling: Real-time monitoring of Redis cluster memory usage. When usage exceeds 85% (the warning threshold) and remains so for 5 minutes, the cluster scaling process is automatically triggered via the Kubernetes Operator, adding Pod nodes to distribute the load.

[0086] L2 optimization: Monitor Elasticsearch shard query performance. When a single shard query latency exceeds 500ms (performance threshold), automatically trigger an index split operation or a rebalance migration to balance data distribution.

[0087] Eviction strategy: Based on historical query logs, a hybrid eviction algorithm combining Least Recently Used (LRU) and Least Frequently Used (LFU) is employed. This algorithm considers not only how long data has been inaccessible but also its historical access frequency, thereby more accurately identifying and cleaning up expired or cold data in L1 cache and preventing cache pollution.

[0088] S103. Using the fine-tuned pre-trained language model, perform multi-level semantic disambiguation and intent recognition on natural language query requests, and parse and generate structured query instructions containing entity objects, intent types and constraints.

[0089] When a cache miss occurs, the query request enters the core semantic analysis process. This step utilizes deep learning technology to resolve the ambiguity and complexity of natural language, transforming unstructured "human language" into machine-executable "instructions." Specific sub-steps include: S1031. Based on the geographic information system service, the geographic location terms in the natural language query request are parsed to determine the business area involved in the query. If the location fails, a fallback strategy is initiated.

[0090] Business queries are often strongly correlated with geographical location. The system first calls the Geographic Information System (GIS) service to extract the place names in the query (such as "Eastern Business Area" or "XXX Street").

[0091] Regional positioning: The GeoHash algorithm is used to map place names to standard business area codes and geographical boundary ranges (Polygons).

[0092] Backup strategy: If the location entered by the user is vague (such as "that business hall"), or if the GIS cannot resolve it, the system will activate the backup strategy: call the user's IP address location, historical business transaction records or GPS information uploaded by the mobile device to infer the area that the user is most likely to be interested in, and initiate a confirmation interaction with the user.

[0093] S1032. Combining the user's historical interaction context and user profile features, perform semantic completion and referential resolution on fuzzy pronouns in natural language query requests.

[0094] Solve the problems of omission and reference in multi-turn dialogues.

[0095] For example, a user might first ask, "How do I process Category A services?", and then ask, "What about Category B?".

[0096] The phrase "then..." here is an abbreviated expression. The system, combining the context history recorded by the Dialogue State Tracking (DST) component, completes the second question as "How do I process Category B business?".

[0097] At the same time, by combining user profiles (such as whether the user is a "corporate user" or an "individual user"), ambiguous terms are resolved. If a user asks "How much subsidy do I receive?", and the profile shows that the user is a retiree, then "subsidy" is automatically resolved to "specific pension subsidy".

[0098] S1033. Input the disambiguated query text into the intent recognition model finely tuned based on the RoBERTa architecture to identify the user's business query intent. The intent recognition model is incrementally trained using a business corpus containing long-tail intents.

[0099] Model architecture: RoBERTa-wwm-ext is used as the base model. This model performs excellently in the Chinese context, especially in its ability to handle whole-word masks.

[0100] Fine-tuning training: The model was fine-tuned using over 2 million anonymized real business consultation records (including customer service hotline records and online consultation logs). The maximum input length was set to 128 tokens, and CrossEntropyLoss was used as the loss function.

[0101] Intent recognition: The model can recognize 87 subcategories of business intents (such as "interpretation of specifications", "guideline query", "progress query", "complaints and suggestions").

[0102] Incremental learning: For "long-tail intents" (i.e., rare queries that occur infrequently but are diverse), the system has established an incremental training mechanism. Samples that failed to be identified online or had low confidence are periodically added to the training set to continuously improve the model's generalization ability.

[0103] S1034. If the confidence level of the identified intent is lower than the preset threshold, the similar question recommendation logic is triggered, and the low-confidence query is recorded for subsequent model optimization.

[0104] When the model's confidence score in judging user intent falls below 0.6 (a preset threshold), the system triggers recommendation logic to avoid errors caused by forced responses. Using the Faiss vector search library, find the top-3 most similar questions in the Frequently Asked Questions (FAQ) database and return them to the user: "Did you mean to ask...?" At the same time, the low-confidence query is marked and stored in the "Hard Example Mining" library for algorithm engineers to analyze or use in the next round of model iteration training.

[0105] Finally, this step outputs a structured query intent that includes entities, intents, and constraints (such as region and time).

[0106] S104. Based on the structured query command, perform a retrieval enhancement generation operation based on graph neural network in the dynamic knowledge graph, and filter candidate evidence chains according to the evidence timeliness weight.

[0107] This step utilizes the associative capabilities of the knowledge graph and Retrieval-Augmented Generation (RAG) technology to accurately identify the "evidence" of the answer within the knowledge base. Specific sub-steps include: S1041. Using the GraphSAGE graph neural network model, entity attribute features and text sentence vector features in the dynamic knowledge graph are fused to generate node embedding vectors.

[0108] In order to accurately retrieve data in the graph, the system needs to calculate the similarity between the query command and the graph nodes.

[0109] Feature fusion: The system not only considers the structured attributes of nodes (such as entity name and type), but also uses BERT to extract sentence vector features of unstructured text (such as standard full text) associated with nodes.

[0110] GraphSAGE: Employs the GraphSAGE (Graph Sample and Aggregate) algorithm, an inductive graph neural network. It generates a low-dimensional dense embedding vector (256-dimensional) for each node by sampling and aggregating information from its neighbors (using a Mean Aggregator). This method captures not only information about the node itself but also its topological structure within the graph (i.e., its relationships with other nodes).

[0111] S1042. Calculate the cosine similarity between the vector representation of the structured query instruction and the node embedding vector, and retrieve the Top-K related nodes with similarity higher than a preset threshold as the initial evidence set.

[0112] The structured query instruction generated in step S103 is also converted into a vector representation of the same dimension.

[0113] Calculate the cosine similarity between the query vector and the embedding vectors of all (or indexed and filtered) nodes in the graph. The cosine similarity is calculated by taking the dot product of the query vector and the node embedding vectors, and then dividing by the product of the magnitudes of the two vectors.

[0114] Set the threshold to 0.85. Select the Top-K (e.g., Top-50) nodes with a similarity greater than 0.85 as the potential "evidence" set for answering the question.

[0115] S1043. Construct a multi-dimensional evidence evaluation system, and calculate the comprehensive score of each piece of evidence in the initial evidence set based on the authority of the evidence source, the relevance of the content, and the timeliness of the publication time.

[0116] In order to select the most accurate evidence from the Top-K, the system builds a scoring model.

[0117] The overall score is calculated using a weighted summation method. Specifically, the authority score of the evidence source is multiplied by the first weight coefficient, the relevance score of the content is multiplied by the second weight coefficient, and the timeliness score of the publication time is multiplied by the third weight coefficient. Then, these three products are added together to obtain the final overall score.

[0118] Authority: Assigned based on the level of the issuing organization (e.g., a first-level organization is greater than a second-level organization).

[0119] Relevance: Directly refers to the vector similarity score calculated in the previous step.

[0120] Timeliness: The closer the publication time is to the current time, the higher the score.

[0121] S1044. Introduce a timeliness weight formula to reduce or filter evidence with time conflicts, and select candidate evidence chains based on the comprehensive score. The timeliness weight formula gives higher weight to the most recently released normative documents.

[0122] In business scenarios, "new rules replace old rules" is a common principle.

[0123] The system introduces a specific timeliness weighting formula. If two documents on the same topic are retrieved (such as the "Business Standard 2020 Edition" and the "Business Standard 2024 Edition"), the system will identify the "revision" or "repeal" relationship between them (through edge relationships in the graph).

[0124] Give extremely high weight to the timeliness of the latest effective documents.

[0125] Deprecated or old versions of files may be significantly downgraded (multiplied by a small attenuation factor, such as 0.1) or filtered out directly.

[0126] Finally, based on the overall scores, the top 5 strongest pieces of evidence were selected to form a "candidate evidence chain".

[0127] S105. Input the candidate evidence chain into the generative model, combine it with the dynamic desensitization strategy corresponding to the user's permissions to generate the final query response, and asynchronously backfill the final query response to the multi-level cache system according to the access frequency strategy.

[0128] The final step is to generate a response that conforms to security standards. Specific sub-steps include: S1051. Embed dynamic desensitization instructions in the prompt word template of the generative model. The dynamic desensitization instructions define the processing rules for data with different sensitivity levels.

[0129] When the system builds a prompt, it includes not only user questions and chains of evidence, but also security instructions.

[0130] The security directive explicitly requires the model to act as an intelligent business assistant, answering user questions based on provided evidence, and ensuring that the output strictly adheres to predefined anonymization rules: for example, masking L3 data and not displaying L4 data.

[0131] The desensitization instructions define how to process data with different sensitivity levels, from L1 to L4.

[0132] S1052. Identify the user's permission level when initiating the query, and match the corresponding de-identification strategy according to the user's permission level.

[0133] The system identifies the current user's identity through OAuth 2.0 or other authentication mechanisms: External public users: have lower privileges and can only see public information.

[0134] Authorized enterprise users: can view detailed information about this enterprise entity.

[0135] Internal business management personnel: have higher data access permissions based on their job level and department.

[0136] The system adopts a role-based access control (RBAC) model, dynamically loading the corresponding de-identification policy configuration based on the user's identity.

[0137] S1053. For sensitive personal information, use masking or truncation methods for high-intensity desensitization; for sensitive corporate data, use aggregation statistics or interval display methods for processing.

[0138] Personally Sensitive Information (PII): Such as Identification Numbers, the policy requires them to be displayed in a masked format (e.g., retaining the first three and last four digits, replacing the middle with an asterisk); contact information should also be displayed in a masked format. Regular expression replacement rules are configured in the data masking engine.

[0139] Sensitive corporate data, such as tax payments or revenue. For regular queries, only a range (e.g., "Business volume: Grade A, corresponding to the range of 500,000-1,000,000") or aggregated statistics (e.g., "Average value for this industry") may be displayed, rather than specific values, to prevent the leakage of trade secrets.

[0140] S1054. Input the candidate evidence chain and the prompt words embedded with de-identification instructions into the generative model to generate a natural language response text that meets compliance requirements.

[0141] The filtered evidence chain text and the constructed Prompt (including anonymized instructions) are input into the Generative Large Language Model. The model parameters are set as follows: Temperature 0.3 (to maintain the stability of the generated results) and Top-P 0.9. When generating responses, the model strictly follows the constraints in the Prompt, generating fluent, accurate, and anonymized natural language answers based on the evidence facts.

[0142] S1055. Based on the access frequency strategy, the final query response is asynchronously backfilled to the multi-level cache system.

[0143] After the answer is generated, it is not only returned directly to the user, but the system will also assess the generality of the question.

[0144] If the question is a high-frequency question (access frequency policy judgment), the system will asynchronously write the "question-answer" pair into the Redis (L1) or Elasticsearch (L2) cache. This way, when the next user asks the same question, the cache can be directly hit in step S102, eliminating the need for complex graph retrieval and model generation, thus forming a closed-loop optimization. Simultaneously, to ensure cache consistency, a "read-your-writes" strategy is used to verify the backfilling results.

[0145] Through the close collaboration of steps S101 to S105 above, this embodiment constructs an efficient, accurate and secure business data query platform, realizing all the predetermined functions of the technical solution.

[0146] In summary, the data query method based on semantic recognition provided by this embodiment of the invention achieves rapid capture and high-consistency fusion of multi-source heterogeneous data by constructing a real-time data pipeline based on dual-strategy partitioning and introducing a vector clock synchronization strategy. This effectively overcomes the data lag and version conflict problems in the traditional batch processing mode, and builds a real-time and accurate data foundation for upper-layer applications.

[0147] This invention utilizes a finely tuned pre-trained language model (PLM) combined with graph neural network (GNN) technology to achieve deep semantic disambiguation and intent recognition of user natural language query requests. It also enhances retrieval generation based on a dynamic knowledge graph, significantly improving query accuracy and the interpretability of evidence chains in complex semantic environments. Furthermore, by constructing a multi-level caching system that includes high-frequency dynamic and low-frequency historical data, this invention can intelligently route query requests based on data popularity, improving system response speed and throughput in high-concurrency scenarios. Combined with a dynamic desensitization strategy for user permissions, it achieves fine-grained security control over data output, ensuring the compliant use of sensitive information while guaranteeing the flexibility and efficiency of data services.

[0148] Secondly, embodiments of the present invention also provide a data query system based on semantic recognition, referring to... Figure 2 The diagram illustrates a functional block diagram of a semantic recognition-based data query system 300 according to the present invention. This system may include the following modules: The dynamic knowledge graph construction module 301 is used for a dual-strategy partitioning mechanism based on business dimensions and data type dimensions. It captures incremental change logs of multi-source heterogeneous business data through real-time data pipelines and uses a vector clock synchronization strategy to synchronously update the incremental change logs to a dynamic knowledge graph that includes an entity relation library and a text index library. The multi-level cache retrieval module 302 is used to respond to the natural language query request initiated by the user. Based on the multi-level cache system including a high-frequency dynamic cache layer and a low-frequency historical cache layer, it sequentially retrieves the high-frequency dynamic cache layer and the low-frequency historical cache layer. If the cache is hit, the query result is returned directly. If the cache is not hit, the semantic analysis process is triggered. The semantic deep parsing module 303 is used to perform multi-level semantic disambiguation and intent recognition on natural language query requests using a fine-tuned pre-trained language model, and to parse and generate structured query instructions containing entity objects, intent types and constraints. The graph enhancement generation module 304 is used to perform a graph neural network-based retrieval enhancement generation operation in a dynamic knowledge graph based on structured query instructions, and to filter candidate evidence chains according to the evidence timeliness weight. The security response processing module 305 is used to input the candidate evidence chain into the generative model, combine it with the dynamic desensitization strategy corresponding to the user's permissions to generate the final query response, and asynchronously backfill the final query response to the multi-level caching system according to the access frequency strategy.

[0149] A third aspect of this invention provides a data query computer device based on semantic recognition. Please refer to [link to relevant documentation]. Figure 3 The system includes a memory and a processor connected in series. The memory stores computer programs, and the processor reads the computer programs and executes a data query method based on semantic recognition as described in the first aspect of the embodiment. Specifically, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or last-in-first-out (FILO) memory, etc.; the processor may be, but is not limited to, microprocessors of the STM32F105 series, ARM (Advanced RISC Machines), x86 architecture processors, or processors with integrated NPUs (neural-network processing units). The working process, working details, and technical effects of the system provided in the second aspect of this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0150] This fourth aspect of the embodiment provides a computer-readable storage medium storing instructions containing a semantic recognition-based data query method according to the first aspect of the embodiment. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, perform the semantic recognition-based data query method as described in the first aspect. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, details, and technical effects of the computer-readable storage medium provided in this fourth aspect of the embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0151] The fifth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform a data query method based on semantic recognition as described in the first aspect of this embodiment. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0152] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer or programmable device to execute the methods of various embodiments or some parts of embodiments.

[0154] Finally, it should be noted that although embodiments of the present invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A data query method based on semantic recognition, characterized in that, The method includes: Based on a dual-strategy partitioning mechanism of business and data type dimensions, incremental change logs of multi-source heterogeneous business data are captured through real-time data pipelines, and the incremental change logs are synchronously updated to a dynamic knowledge graph containing entity relation databases and text index databases using a vector clock synchronization strategy, thereby building the data foundation for semantic query. In response to a user's natural language query request, based on a multi-level caching system that includes a high-frequency dynamic caching layer and a low-frequency historical caching layer, the high-frequency dynamic caching layer and the low-frequency historical caching layer are searched sequentially. If the cache is hit, the query result is returned directly; if the cache is not hit, the semantic analysis process is triggered. The fine-tuned pre-trained language model is used to perform multi-level semantic disambiguation and intent recognition on the natural language query request, and to parse and generate a structured query instruction containing entity objects, intent types and constraints. Based on the structured query instructions, a retrieval enhancement generation operation based on graph neural networks is performed in the dynamic knowledge graph to filter candidate evidence chains according to the evidence timeliness weight. The candidate evidence chain is input into the generative model, and a final query response is generated by combining the dynamic desensitization strategy corresponding to the user's permissions. The final query response is then asynchronously backfilled into the multi-level caching system according to the access frequency strategy.

2. The data query method based on semantic recognition according to claim 1, characterized in that, Based on a dual-strategy partitioning mechanism that considers both business and data type dimensions, incremental change logs of multi-source heterogeneous business data are captured through real-time data pipelines, including: The database log connector is used to monitor the binary log of the source database in real time and capture incremental operation data, including insert, update and delete operations. The basic partition is determined by using the business identifier hash modulo, and a secondary partition is performed by overlaying the data type hash modulo, and the incremental operation data is routed to the corresponding message queue partition. Based on the real-time throughput of each partition, when the throughput of a specific partition exceeds the hotspot threshold within a preset time window, a dynamic partition migration strategy is triggered to migrate the hotspot business data to an idle partition. During data transmission, a strategy combining future time filtering and business whitelisting is adopted to verify the timestamps in the incremental operation data, remove abnormal future time data that exceeds the preset tolerance window, and retain the pre-approval process data in the whitelist.

3. The data query method based on semantic recognition according to claim 2, characterized in that, The incremental change log is synchronized and updated to a dynamic knowledge graph containing an entity relation database and a text index database using a vector clock synchronization strategy, including: Construct a vector clock triple containing engine type, operation sequence number, and timestamp as a credential for data consistency verification; The structured entity relationship data in the incremental operation data is routed to a graph database as the entity relationship database, and the unstructured text data in the incremental operation data is routed to a search engine as the text index database. When performing a write operation, the operation sequence number of the corresponding engine is automatically incremented and the current system timestamp is updated synchronously. When a synchronization conflict is detected between the entity relation database and the text index database for the same data object, the operation sequence number and timestamp of the vector clock corresponding to the entity relation database and the text index database are compared respectively. The last write wins strategy to force the data state to be unified, and the operation trajectory is retained for traceability.

4. The data query method based on semantic recognition according to claim 3, characterized in that, The method further includes dynamically segmenting the dynamic knowledge graph by business area and extracting relationships, specifically including: The entity relation database is sharded and stored according to the business zone code. When adding or merging business zones, a zero-loss migration process is executed, including pre-migration dual write, switch verification and backup cleanup. An entity relation extraction network based on a pre-trained language model is adopted. The business entity recognition capability is enhanced through the masked language model task, and adversarial examples are introduced to fine-tune the relation classifier. The fine-tuned entity relationship extraction network is used to extract business entities and their relationships from unstructured text data. The extraction results are incrementally updated to the entity relationship database through a dual triggering mechanism that combines time-driven and event-driven approaches.

5. The data query method based on semantic recognition according to claim 1, characterized in that, The natural language query request is subjected to multi-level semantic disambiguation and intent recognition using a fine-tuned pre-trained language model, including: Based on the geographic information system service, the geographic location terms in the natural language query request are parsed to determine the business area involved in the query. If the location fails, a fallback strategy is initiated. By combining the user's historical interaction context and user profile features, semantic completion and referential resolution are performed on fuzzy pronouns in the natural language query request; The disambiguated query text is input into an intent recognition model finely tuned based on the RoBERTa architecture to identify the user's business query intent. The intent recognition model is incrementally trained using a business corpus containing long-tail intents. If the confidence level of the identified intent is lower than a preset threshold, the similar question recommendation logic is triggered, and the low-confidence query is recorded for subsequent model optimization.

6. The data query method based on semantic recognition according to claim 1, characterized in that, Based on the structured query instructions, a retrieval enhancement generation operation based on a graph neural network is performed in the dynamic knowledge graph, including: Using the GraphSAGE graph neural network model, entity attribute features and text sentence vector features in the dynamic knowledge graph are fused to generate node embedding vectors; Calculate the cosine similarity between the vector representation of the structured query instruction and the node embedding vector, and retrieve the Top-K related nodes with similarity higher than a preset threshold as the initial evidence set; Construct a multi-dimensional evidence evaluation system, and calculate the comprehensive score of each piece of evidence in the initial evidence set based on the authority of the evidence source, the relevance of the content, and the timeliness of the publication time. A timeliness weighting formula is introduced to reduce the weight of evidence with time conflicts or filter it out. Candidate evidence chains are selected by sorting them according to the comprehensive score. The timeliness weighting formula gives higher weight to the most recently published normative documents.

7. The data query method based on semantic recognition according to claim 1, characterized in that, The candidate evidence chain is input into the generative model, and a final query response is generated by combining it with a dynamic de-identification strategy corresponding to user permissions, including: Dynamic desensitization instructions are embedded in the prompt word template of the generative model. These instructions define the processing rules for data with different sensitivity levels. Identify the user's permission level currently initiating the query, and match the corresponding de-identification strategy based on the user's permission level; For sensitive personal information, we use masking or truncation for high-intensity de-identification; for sensitive corporate data, we use aggregation statistics or range display methods for processing. The candidate evidence chain and the prompt words embedded with de-identification instructions are input into the generative model to generate a natural language response text that meets compliance requirements.

8. The data query method based on semantic recognition according to claim 1, characterized in that, The multi-level caching system based on a high-frequency dynamic caching layer and a low-frequency historical caching layer sequentially searches the high-frequency dynamic caching layer and the low-frequency historical caching layer, including: Data that has been accessed more than a first threshold and updated more than a second threshold within a recent preset time period is defined as high-frequency dynamic data, stored in the high-frequency dynamic cache layer based on an in-memory database, and has a short lifespan. Data with fewer than a first threshold number of visits or a lower than a second threshold number of updates within a recent preset time period is defined as low-frequency historical data, stored in the low-frequency historical cache layer based on the search engine, and updated daily. When the user query request arrives, the high-frequency dynamic cache layer is queried first; if no match is found, the low-frequency historical cache layer is queried. If the target data is hit in the low-frequency historical cache layer, and the access characteristics of the target data meet the potential high-frequency conditions, then the target data will be automatically promoted and backfilled into the high-frequency dynamic cache layer.

9. The data query method based on semantic recognition according to claim 8, characterized in that, The method also includes dynamically adjusting the resources of the multi-level caching system: The cached data is serialized and stored using a binary compression format; The memory usage of the high-frequency dynamic cache layer is monitored in real time. When the memory usage exceeds the preset warning line and continues for a preset duration, the expansion process is automatically triggered. The shard query performance of the low-frequency historical cache layer is monitored in real time. When the single shard query latency exceeds the preset performance threshold, the number of shards is automatically adjusted and the data distribution is balanced. Based on historical query log analysis, a hybrid eviction strategy combining the Least Recently Used (LRU) algorithm and the Least Frequently Used (LRU) algorithm is adopted to clean up expired or cold data in the high-frequency dynamic cache layer.

10. A data query system based on semantic recognition, characterized in that, The system includes: The dynamic knowledge graph construction module is used for a dual-strategy partitioning mechanism based on business and data type dimensions. It captures incremental change logs of multi-source heterogeneous business data through real-time data pipelines and uses a vector clock synchronization strategy to synchronously update the incremental change logs to a dynamic knowledge graph that includes an entity relation library and a text index library. The multi-level cache retrieval module is used to respond to natural language query requests initiated by users. Based on a multi-level cache system that includes a high-frequency dynamic cache layer and a low-frequency historical cache layer, it sequentially retrieves the high-frequency dynamic cache layer and the low-frequency historical cache layer. If the cache is hit, the query result is returned directly. If the cache is not hit, the semantic analysis process is triggered. The semantic deep parsing module is used to perform multi-level semantic disambiguation and intent recognition on the natural language query request using a fine-tuned pre-trained language model, and to parse and generate a structured query instruction containing entity objects, intent types and constraints. The graph enhancement generation module is used to perform a graph neural network-based retrieval enhancement generation operation in the dynamic knowledge graph based on the structured query instruction, and to filter candidate evidence chains according to the evidence timeliness weight. The security response processing module is used to input the candidate evidence chain into the generative model, combine it with the dynamic desensitization strategy corresponding to the user's permissions to generate the final query response, and asynchronously backfill the final query response to the multi-level caching system according to the access frequency strategy.

Images