Real-time data intelligent query method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, can be applied to the fields of medical health, financial science and technology and the like, and discloses a real-time data intelligent query method, device and equipment and a storage medium. The real-time data intelligent query method comprises the steps that multiple heterogeneous data sources are registered, field names, data types and service meanings in the heterogeneous data sources are recognized, and preliminary semantic fingerprints are generated; based on the preliminary semantic fingerprints, constructing multilayer semantic mapping; receiving a query request of a user, analyzing the query request based on the multilayer semantic mapping, and generating a logic query plan; acquiring a real-time data source state, and based on the logic query plan, performing dynamic path selection in combination with the real-time data source state to generate a cross-source query plan; and executing the cross-source query plan, obtaining query results of the heterogeneous data sources, performing result fusion on all the query results through the semantic consistency alignment model, and outputting a fusion result in a unified format. By means of the method, intelligent semantic association and real-time optimization access of cross-source data are achieved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology and can be applied to fields such as medical health and financial technology. In particular, it relates to a real-time intelligent data query method, device, equipment and storage medium. Background Technology

[0002] In the current context of accelerated digital transformation, data has become a core driving force for intelligent upgrades in key sectors such as healthcare and fintech. In healthcare, a continuous stream of heterogeneous data from multiple sources, including electronic medical records, laboratory test data, imaging diagnostic results, and real-time vital sign monitoring data, requires efficient data processing to support core scenarios such as clinical diagnostic decisions, disease early warning, and optimized allocation of medical resources. Meanwhile, the fintech sector faces the need for real-time processing of massive amounts of transaction data, user credit data, market data, and regulatory compliance data, directly impacting key business operations such as real-time risk control, intelligent investment advisory services, and cross-border payment settlement.

[0003] Traditional ETL (Extract-Transform-Load) technology, a classic solution for enterprise data processing, involves extracting, transforming, and loading data from a source system to a destination. Its core idea is to extract data from different source systems, transform it uniformly, and then load it into a data warehouse or analytics platform. It has been widely used in scenarios such as regional healthcare data integration in the healthcare sector and historical transaction data analysis in the fintech sector. However, with the explosive growth in data volume and the diversification of data types in the healthcare and fintech sectors, the limitations of traditional ETL technology are becoming increasingly apparent. 1. Traditional ETL relies on batch processing tasks, and there is often a delay of several hours to several days between the generation and analysis of data. In the healthcare field, if real-time ECG monitoring data of patients and data from intensive care equipment cannot be included in the analysis in a timely manner due to delays, it may affect the timeliness of clinical emergency decisions. In the fintech field, scenarios such as real-time transaction risk monitoring and cross-border payment fraud identification are extremely sensitive to data latency, and the batch processing mode of traditional ETL is difficult to meet the millisecond-level risk response requirements.

[0004] 2. Each time a new data source is integrated, the schema, field mapping, and conversion rules need to be manually defined. In the healthcare field, the data formats of information systems (HIS, LIS, PACS) of different hospitals vary significantly. When adding data sources from regional medical collaboration units, a large amount of manpower is required to sort out the meaning of fields and formulate conversion rules. In the fintech field, when integrating new data sources such as third-party payment platforms, credit reporting agencies, and cross-border clearing systems, manual adaptation to different data structures and field definitions is also required. This is not only inefficient but also prone to errors due to human operation.

[0005] 3. When the source system structure or fields change, the ETL process needs to be readjusted. In the healthcare field, frequent upgrades to hospital information systems and updates to medical data standards can lead to the addition, modification, or deletion of data source fields, requiring a simultaneous reconstruction of the ETL transformation logic. In the fintech field, regulatory policy adjustments and business product iterations may cause changes in data field definitions, requiring continuous resource investment in maintenance and adaptation of traditional ETL processes, significantly increasing operating costs.

[0006] 4. ETL models rely on predefined data structures, making it difficult to support real-time analysis across sources and modalities. In the healthcare field, clinical diagnosis often requires the integration of electronic medical records, laboratory data, imaging data, and genetic data from multiple hospitals for comprehensive analysis. Traditional ETL models struggle to achieve real-time correlation analysis of multi-source heterogeneous data. In the fintech field, real-time risk control requires the integration of multi-dimensional information such as transaction data from core banking systems, third-party credit data, and user behavior logs. Predefined ETL models cannot meet the needs of dynamic cross-source queries.

[0007] To address the aforementioned shortcomings of traditional ETL technologies, academia and industry have proposed the Zero-ETL concept, a data access model that eliminates extraction, transformation, and loading. Zero-ETL enables cross-source queries through virtualization technology, eliminating the need for pre-copying data and theoretically enabling real-time access to multi-source data in fields such as healthcare and fintech. However, existing Zero-ETL technology still faces significant bottlenecks in practical applications: First, data semantics are inconsistent, with ambiguous meanings in different source fields. For example, different hospitals in the healthcare field may have different units of measurement and recording formats for fasting blood glucose values, while different institutions in the fintech field may use different statistical methods for overdue days, leading to semantic conflicts during cross-source data correlation analysis. Second, the query layer cannot understand natural language or business semantics. Medical personnel often initiate queries using natural language such as asking for the average systolic blood pressure of hypertensive patients in the past week, while financial analysts often express query needs using business semantics such as calculating the total cross-border transaction amount of high-risk users in the current quarter. Existing Zero-ETL technology struggles to directly parse such unstructured query intents. Third, semantic-level query optimization and dynamic scheduling are not feasible. Scenarios such as emergency data queries in the healthcare field and real-time risk control queries in the fintech field require dynamic adjustment of the query path based on the real-time load and latency of the data source. However, existing Zero-ETL technology lacks a semantic-based intelligent scheduling mechanism, making it difficult to guarantee query efficiency and stability. Summary of the Invention

[0008] The purpose of this invention is to provide a real-time intelligent data query method, device, equipment, and storage medium, which aims to solve the problems of inconsistent data semantics and difficulty in ensuring query efficiency and stability in existing Zero-ETL technology.

[0009] In a first aspect, embodiments of the present invention provide a real-time intelligent data query method, including: Register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint; Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, which includes a field semantic layer, a logical table layer, and a business semantic layer. Receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; Obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in conjunction with the real-time data source status to generate a cross-source query plan; The cross-source query plan is executed to obtain the query results from each of the heterogeneous data sources. The results of all the query results are then fused using a semantic consistency alignment model, and a fused result in a unified format is output.

[0010] Secondly, embodiments of the present invention provide a real-time intelligent data query device, comprising: The registration unit is used to register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint. The construction unit is used to construct a multi-layer semantic mapping based on the preliminary semantic fingerprint, wherein the multi-layer semantic mapping includes a field semantic layer, a logical table layer, and a business semantic layer; The parsing unit is used to receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; The selection unit is used to obtain the real-time data source status, perform dynamic path selection based on the logical query plan and the real-time data source status, and generate a cross-source query plan. The fusion unit is used to execute the cross-source query plan, obtain the query results of each heterogeneous data source, and fuse all the query results through a semantic consistency alignment model to output a fusion result in a unified format.

[0011] Thirdly, embodiments of the present invention provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the real-time data intelligent query method described in the first aspect.

[0012] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the real-time data intelligent query method described in the first aspect.

[0013] This invention discloses a real-time intelligent data query method, apparatus, device, and storage medium, comprising: registering multiple heterogeneous data sources, identifying the field names, data types, and business meanings in each heterogeneous data source, and generating a preliminary semantic fingerprint; constructing a multi-layer semantic mapping based on the preliminary semantic fingerprint, the multi-layer semantic mapping including a field semantic layer, a logical table layer, and a business semantic layer; receiving a user's query request, parsing the query request based on the multi-layer semantic mapping, and generating a logical query plan; obtaining the real-time data source status, and dynamically selecting a cross-source query plan based on the logical query plan and the real-time data source status; executing the cross-source query plan, obtaining the query results of each heterogeneous data source, and fusing all query results through a semantic consistency alignment model to output a fused result in a unified format. This invention achieves data semantic unification by parsing heterogeneous data sources to generate preliminary semantic fingerprints, constructing multi-layer semantic mapping, parsing query semantics using a large language model, and dynamically optimizing the query path based on the real-time data source status. This solution not only addresses the problems of high latency, high transformation overhead, complex maintenance, and limited real-time analysis associated with traditional ETL, but also overcomes the bottlenecks of existing Zero-ETL, such as semantic ambiguity, inability to understand business semantics, and lack of semantic-level optimization. It achieves efficient cross-source data association queries, reduces manual intervention costs, improves system adaptability and self-evolution capabilities, and ensures query real-time performance and accuracy. This invention also provides a real-time intelligent data query device, a computer-readable storage medium, and a computer device, all possessing the aforementioned beneficial effects, which will not be elaborated further here. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of an application environment for a real-time data intelligent query method according to an embodiment of the present invention; Figure 2 A flowchart illustrating the real-time data intelligent query method; Figure 3 A schematic block diagram of a real-time data intelligent query device; Figure 4This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be understood that, when used in this specification and the appended claims, the terms “comprising” and “including” indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more of its features, integrals, steps, operations, elements, components and / or collections thereof.

[0018] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0019] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] The real-time intelligent data query method provided in this embodiment of the invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can register multiple heterogeneous data sources through the client, identify the field names, data types, and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint. Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, including a field semantic layer, a logical table layer, and a business semantic layer. The server receives user query requests, parses the query requests based on the multi-layer semantic mapping, and generates a logical query plan. It obtains the real-time data source status, and based on the logical query plan and the real-time data source status, dynamically selects a path to generate a cross-source query plan. The cross-source query plan is executed, obtaining the query results from each heterogeneous data source. Finally, a semantic consistency alignment model is used to fuse all query results, outputting a unified formatted fused result. In this invention, data semantic unification is achieved by parsing heterogeneous data sources to generate preliminary semantic fingerprints and constructing multi-layer semantic mapping. A large language model is used to parse query semantics, and the query path is dynamically optimized based on the real-time data source status. This solution addresses the problems of high latency, high transformation overhead, complex maintenance, and limited real-time analysis associated with traditional ETL. It also overcomes the bottlenecks of existing Zero-ETL, such as semantic ambiguity, inability to understand business semantics, and lack of semantic-level optimization. This enables efficient cross-source data association queries, reduces manual intervention costs, improves system adaptability and self-evolution capabilities, and ensures query real-time performance and accuracy. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, AR devices, and portable wearable devices. The server can be implemented using a dedicated server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0021] Please see Figure 2 This embodiment provides a real-time intelligent data query method, including: S101: Register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint; In this embodiment, multiple heterogeneous data sources are registered, and the field names, data types, and business meanings in each heterogeneous data source are identified to generate a preliminary semantic fingerprint, including: Multiple heterogeneous data sources can be registered through a unified interface. These heterogeneous data sources include structured databases, semi-structured log systems, and unstructured object storage. Automatically extract field metadata and field sample data from heterogeneous data sources; Extract field names and data type features based on field metadata, and generate semantic representations of field content using a pre-trained language model based on field sample data; The business meaning is obtained by integrating the field name, data type characteristics, and semantic representation of the field content; The business meaning is vectorized and encoded to generate field semantic vectors; Integrate all field semantic vectors and calculate the overall semantic feature distribution of the integrated field semantic vectors to construct a semantic fingerprint index structure; The semantic fingerprint index structure is standardized to generate a preliminary semantic fingerprint.

[0022] This embodiment achieves standardized access to structured, semi-structured, and unstructured data sources through a unified interface, exhibiting excellent scalability and flexibility. Secondly, by automatically extracting metadata and sample data, and combining this with a pre-trained language model for deep semantic understanding of field content, it achieves automatic mapping and fusion from raw data features to business meaning, effectively overcoming the limitations of traditional methods that rely on manual summarization and struggle to handle semantic ambiguity. Next, through vectorized encoding and standardization, it generates computable and comparable semantic fingerprints, providing a structured semantic foundation for subsequent semantic alignment, intelligent querying, and cross-source fusion, significantly reducing the complexity and cost of multi-source data integration and management.

[0023] Specifically, three heterogeneous data sources are registered through a standardized API interface: a MySQL relational database, an Elasticsearch log cluster, and a MinIO object storage system (MinIO is a high-performance, S3-compatible distributed object storage system designed for large-scale data storage, supporting cloud-native deployment and cross-platform integration). During registration, the connection parameters and authentication information of each data source are automatically detected, eliminating the need for manual configuration of mapping relationships.

[0024] After registration, the automatic metadata extraction process is initiated. For MySQL databases, the system database is queried to obtain table structure information, and metadata such as the name, data type, and whether it is a primary key for each field is extracted. Simultaneously, 100 records are randomly sampled as field sample data. For Elasticsearch log systems (Elasticsearch is an open-source distributed search and analysis engine that supports real-time storage and retrieval of structured, unstructured, and vector data), the mapping structure is parsed (a mapping in Elasticsearch defines the structure, data type, and indexing rules of document fields, similar to a database table structure) to obtain field definitions, and 1000 log documents from the most recent 24 hours are extracted as samples. For CSV files in MinIO object storage, the file header information is read to obtain field names, and the first 1000 rows of data are analyzed to infer data types and content distribution.

[0025] Then, feature extraction is performed on the extracted metadata, parsing the semantic units of field names. For example, order amount is split into two semantic units: order and amount, and data type features such as integer, floating-point, string, or timestamp are identified. Simultaneously, a pre-trained BERT-base model is loaded to semantically encode the sample data for each field. For example, for a field containing product price data, the model analyzes the distribution of sample values ​​and context to generate a semantic vector representing the meaning of the price.

[0026] Next, the field name features, data type features, and content semantic representation are integrated through a gated fusion network. This network assigns dynamic weights to the three types of features, calculates a weighted fusion vector, and outputs a business meaning description vector for the field after processing by a fully connected layer. For fields that clearly represent monetary amounts, the system identifies their business attribute as a metric value rather than a simple identifier.

[0027] Subsequently, cosine similarity normalization was applied to the business meaning description vectors to generate 128-dimensional field semantic vectors. All field semantic vectors were organized into matrices, reduced to 32 dimensions through principal component analysis, and the standard deviation and distribution density of each dimension were calculated to form a histogram of the overall semantic feature distribution of the data source. Based on this distribution, a hierarchical index structure was constructed, organizing semantically similar fields under the same branch, and recording semantic distance weights between different branches.

[0028] Finally, the index structure is standardized by normalizing all vector lengths to a unit sphere and mapping semantic distance weights to the 0-1 range, generating a fixed-structure JSON document as a preliminary semantic fingerprint. This fingerprint includes four parts: a summary of the data source, a set of field semantic vectors, a hierarchical index structure, and semantic distribution characteristics, with a total size controlled within 1MB to facilitate subsequent semantic mapping processing. When the data source structure changes, the fingerprint update process is automatically triggered, reprocessing only the changed parts while retaining historical fingerprint versions for change analysis.

[0029] In the healthcare implementation, three typical medical data sources are registered through a unified API interface: an Oracle structured database in the hospital information system, containing patient electronic medical records and medical orders; a JSON-formatted log stream generated by the medical equipment monitoring system, stored in an Elasticsearch cluster; and DICOM-formatted CT and MRI image files in the medical image archiving system, stored in a distributed object storage platform.

[0030] Next, the system retrieves the table structure from the Oracle structured database, including the definitions of fields such as digital ID, diagnostic codes, and quantitative indicator results (i.e., the specific numerical results of laboratory test items); it also retrieves the mapping structure of the device monitoring logs from Elasticsearch, containing physiological indicator fields such as heart rate and blood pressure; and it extracts DICOM standard tags such as imaging device type and specific body part from the image file metadata. Simultaneously, the system extracts sample data for each field, such as ICD-10 encoding instances of diagnostic codes, numerical distribution ranges of physiological indicators, and specific type values ​​of image modalities.

[0031] Based on metadata, the system extracts medical semantic units from field names. For example, it breaks down quantitative results of laboratory tests into three semantic units: laboratory, result, and numerical value, and identifies data type characteristics such as numerical, categorical coding, or time-based. Using a BioBERT model pre-trained on PubMed (a free biomedical literature database developed by the National Center for Biotechnology Information, part of the U.S. National Library of Medicine), the system analyzes the medical context of field sample data, such as identifying disease classification information contained in diagnostic code fields.

[0032] The system uses a multi-head attention mechanism to fuse field name features, data type features, and content semantic representation to generate business meaning descriptions. For example, it identifies clinical laboratory quantitative indicators as clinical laboratory quantitative indicators, rather than ordinary numerical values. The system vectorizes these business meaning descriptions to generate 128-dimensional field semantic vectors, paying particular attention to the standardization of medical terminology, such as mapping fields representing the same physiological indicators in different systems to similar vector spaces.

[0033] All field semantic vectors are integrated into a medical data source semantic matrix. The system calculates the feature distribution of each clinical topic dimension (such as cardiovascular, respiratory, and laboratory tests), constructs a hierarchical semantic fingerprint index, and organizes related medical concepts in the same branch. Finally, the system standardizes the index structure to generate a preliminary semantic fingerprint that conforms to medical information integration standards, supporting cross-source data queries in subsequent clinical decision-making systems.

[0034] In a fintech implementation, three typical financial data sources are registered through a unified authentication interface: the SQL Server structured database of the bank's core system, which contains customer transaction and account information; the semi-structured JSON transaction logs generated by the payment gateway, which are stored in the database cluster; and the PDF contracts and scanned copies in the compliance document library, which are stored in the cloud object storage service.

[0035] Next, the system retrieves the definitions and constraints of key fields such as bank account number, transaction amount, and risk rating from SQL Server; it also obtains the dynamic schema (a schema is a structured data description specification used to explicitly define the organization, field types, format constraints, and relationship rules of data) of the transaction log set from the database cluster, including dimensional fields such as IP address, device fingerprint, and geographic location; and it extracts attributes such as document type, signing date, and counterparty name from the document metadata. Simultaneously, the system collects typical sample values ​​for each field, such as the distribution range of transaction amounts, the specific levels of risk rating, and classification examples of document types.

[0036] Then, the financial semantic units of the field names are parsed. For example, customer risk scores are broken down into three semantic components: customer, risk, and score, and data type characteristics such as currency, percentage, or classification level are identified. Using a BERT model fine-tuned on financial text, the business context of the sample data is analyzed, such as identifying the financial meaning of positive and negative values ​​in the geolocation field indicating the direction of income and expenditure.

[0037] Subsequently, a weighted fusion algorithm is used to integrate field name features, data type features, and content semantic representations to generate a business meaning vector. For example, the geographic location field is associated with compliance rules for high-risk areas to enhance its risk control meaning. The system standardizes and encodes the business meaning vector to generate a 96-dimensional field semantic vector, ensuring that financially sensitive fields such as account balances and identity verification have appropriate semantic distance.

[0038] Next, the semantic vectors of all fields are integrated into a financial data source feature matrix. The distribution density of each business dimension (such as customer identity, transaction behavior, risk characteristics, and compliance attributes) is calculated, and a multi-level index structure is constructed, organizing anti-money laundering and credit assessment-related fields in logical groups. Finally, the system applies Z-score standardization to the index structure to generate a preliminary semantic fingerprint that conforms to financial data governance standards. This fingerprint includes data sensitivity level markers, supporting semantic mapping for subsequent cross-system risk control queries and compliance analysis.

[0039] In some embodiments, the method for constructing a pre-trained language model includes: Multi-source heterogeneous text data is collected, and text cleaning, word segmentation, and standardization are performed to generate a structured training corpus containing tagged word sequences and contextual relationship labels. Next, the lexical distribution characteristics and sequence length statistics of the structured training corpus are analyzed to determine the model architecture parameters, including layer depth, number of attention heads, and hidden layer dimensions. Then, based on the model architecture parameters, a model parameter matrix is ​​initialized, containing trainable parameters for the word embedding layer, multi-head attention layer, and feedforward network layer. Finally, a dynamic masking strategy is constructed based on the structured training corpus and the model architecture parameters. The training batches are generated by dynamically adjusting the mask probability based on the frequency of word occurrence using a dynamic masking strategy. The training batches and the initialized model parameter matrix are then input into a distributed training framework to perform iterative optimization based on a self-supervised objective. The prediction loss is calculated, and gradient update rules are applied to generate optimized model parameters. Next, a knowledge distillation strategy is applied to the optimized model parameters. Teacher-student training pairs are constructed based on high-confidence samples from the structured training corpus, generating a compressed pre-trained language model parameter set. This compressed pre-trained language model parameter set is then quantized to form the final model weights.

[0040] Specifically, multi-source heterogeneous text data is collected, covering narrative text, explanatory text, and record text in different formats. The collected text data is cleaned to remove redundant characters, invalid information, and duplicate content. Then, word segmentation is performed to break the text down into the smallest semantic unit, word units. After standardization, the word unit representation is unified, and finally, a structured training corpus is generated. This corpus contains tokenized word unit sequences and labeled contextual dependency relationships between words.

[0041] Then, statistical analysis was performed on the structured training corpus to calculate the word frequency, word frequency distribution curve, and word coverage, as well as the word distribution characteristics. At the same time, the word sequence length, average sequence length, and length distribution range of each text were statistically analyzed. Based on the analysis results, the model architecture parameters were determined: the layer depth was set to 6, the number of attention heads was set to 8, and the hidden layer dimension was set to 512, thus forming a complete model architecture parameter configuration.

[0042] Based on the determined model architecture parameters, the parameters of each layer of the model are initialized to construct the model parameter matrix: the parameters of the word embedding layer are randomly initialized and conform to a normal distribution; the query, key, and value matrices of the multi-head attention layer are initialized according to a uniform distribution; the weight matrix of the feedforward network layer is initialized using the Xavier initialization method, and the bias term is initialized to 0. Finally, the model parameter matrix containing the trainable parameters of each layer is obtained.

[0043] Next, based on the lexical distribution characteristics of the structured training corpus and the model architecture parameters, a dynamic masking strategy is constructed: for high-frequency words with a frequency higher than a preset threshold, a lower masking probability of 0.1 is set, and for low-frequency words with a frequency lower than a preset threshold, a higher masking probability of 0.3 is set, while limiting the maximum masking ratio of a single sequence to no more than 0.15; the structured training corpus is divided into batches of 32, and the word sequences in each batch are randomly masked using the dynamic masking strategy to generate training sample batches.

[0044] Next, the training sample batch and the initialized model parameter matrix are input into the distributed training framework. The self-supervised training objective is set as masked word prediction and sentence order prediction. The cross-entropy loss between the prediction result and the true label is calculated through forward propagation. The gradient value of each parameter is calculated by applying the stochastic gradient descent algorithm. The model parameter matrix is ​​updated with a learning rate of 0.001. After 100 rounds of iterative training, the optimized model parameters are generated.

[0045] Subsequently, a knowledge distillation strategy was applied to the optimized model parameters to select high-confidence samples with prediction confidence higher than 0.95 from the structured training corpus, constructing training pairs for the teacher and student models. The teacher model adopted the optimized full-parameter model, while the student model adopted a lightweight architecture with half the number of layers and the hidden layer dimension compressed to 256. The output probability distribution of the teacher model was used as a soft label and combined with the real labels for joint training to generate a compressed pre-trained language model parameter set. This parameter set was then quantized with 8 bits to map the parameter values ​​to the integer range of 0-255, ultimately forming compact model weights.

[0046] S102: Based on the preliminary semantic fingerprint, construct a multi-layer semantic mapping, which includes a field semantic layer, a logical table layer, and a business semantic layer; In this embodiment, based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed. The multi-layer semantic mapping includes a field semantic layer, a logical table layer, and a business semantic layer, including: Based on the preliminary semantic fingerprint, semantic vector representations of each field are generated through a pre-trained natural language processing model to construct a field semantic layer; Similarity calculation and clustering are performed on the semantic vector representations in the field semantic layer. Semantically similar fields are merged into unified logical fields, semantic relationships between fields are established, and a logical surface is constructed. Based on the unified logical fields in the logical table layer, knowledge graph technology is used to associate and map the unified logical fields with predefined business concept nodes to construct a business semantic layer. Hierarchical indexing is used to organize the field semantic layer, logical table layer, and business semantic layer, and an inter-layer mapping relationship table is established to form a multi-layer semantic mapping.

[0047] This embodiment generates field semantic vectors based on a pre-trained natural language processing model to construct a field semantic layer, which can accurately capture the deep semantics of fields and provide a unified benchmark for cross-source data semantic alignment. By merging semantically similar fields through similarity calculation and clustering, a logical table layer is constructed, which effectively eliminates semantic ambiguity between different data sources, establishes clear field relationships, and reduces data redundancy. Secondly, knowledge graph technology is used to map unified logical fields with predefined business concept nodes to construct a business semantic layer, realizing a deep connection between technical fields and business scenarios, making the data more in line with actual application needs. In addition, the three-layer structure is hierarchically indexed and an inter-layer mapping relationship table is established to form a logically coherent and highly efficient multi-layer semantic mapping, which not only ensures the regularity of the semantic structure, but also provides efficient support for subsequent query parsing and cross-source association, greatly improving the intelligence level of data integration and access.

[0048] Specifically, preliminary semantic fingerprints are obtained for each heterogeneous data source. These preliminary semantic fingerprints include the semantic vectors, business meanings, and associated feature information of each field. Based on these preliminary semantic fingerprints, the pre-trained natural language processing model is used for feature enhancement and encoding to generate a unique and accurate semantic vector representation for each field. The semantic vector representations of all fields are then categorized and organized according to their data source affiliation and field identifier to construct a field semantic layer.

[0049] Next, pairwise similarity calculations are performed on all semantic vector representations in the field semantic layer. The cosine similarity algorithm is used to obtain the semantic association degree between each vector. Then, through clustering, fields with semantic association degree higher than a preset threshold are grouped into one category and merged to form a unified logical field. At the same time, the original fields contained in each unified logical field and the semantic association relationship between them are recorded to construct the logical surface layer.

[0050] Then, based on all the unified logical fields in the logical table layer, a preset knowledge graph is retrieved. This knowledge graph contains multiple general business concept nodes and association rules between nodes. Through semantic matching, each unified logical field is associated with the corresponding predefined business concept node, clarifying the business direction corresponding to each unified logical field, and integrating all mapping relationships to construct a business semantic layer.

[0051] Finally, hierarchical indexing is organized for the field semantic layer, logical table layer, and business semantic layer. Index directories are established according to field semantic categories, unified logical field affiliation, and business concept nodes. At the same time, an inter-layer mapping relationship table is created to record in detail the correspondence between semantic vectors in the field semantic layer and unified logical fields in the logical table layer, and the correspondence between unified logical fields in the logical table layer and business concept nodes in the business semantic layer. Through the combination of index organization and inter-layer mapping relationship table, a complete multi-layer semantic mapping is formed.

[0052] In the healthcare field, preliminary semantic fingerprints are obtained from various heterogeneous data sources related to medical care. These fingerprints contain semantic features and business implications related to fields such as visit date, examination items, test values, and image descriptions. Based on these preliminary semantic fingerprints, the information from each field is input into a pre-trained natural language processing model. Through feature extraction and encoding, a unique semantic vector representation for each field is generated. All semantic vector representations are then categorized and organized according to examination type and treatment process to construct a field semantic layer.

[0053] Next, pairwise similarity calculations are performed on the semantic vector representations in the field semantic layer. The cosine similarity algorithm is used to determine the degree of association. Through clustering, fields with a semantic association degree higher than a preset threshold are grouped into one category. For example, blood glucose values, fasting blood glucose test results, and postprandial blood glucose data from different data sources are merged into a unified logical field. At the same time, the source and semantic association relationship of the original fields under each category are recorded to construct a logical surface layer.

[0054] Then, based on the unified logical fields in the logical table layer, the knowledge graph containing predefined business concept nodes such as patient examinations, medical records, and health records is retrieved. Semantic matching is used to establish an association mapping between each unified logical field and the corresponding business concept node. For example, the unified logical field of blood glucose test results is mapped to the patient basic vital signs monitoring business concept node. All mapping relationships are integrated to construct the business semantic layer.

[0055] Subsequently, a hierarchical indexing organization was carried out on the field semantic layer, logical table layer, and business semantic layer. An index directory was established according to the inspection items and business concept categories. At the same time, an inter-layer mapping relationship table was created to record in detail the correspondence between field semantic vectors and unified logical fields, and between unified logical fields and business concept nodes. The index organization and mapping relationship table form a complete multi-layer semantic mapping.

[0056] In the fintech field, preliminary semantic fingerprints are obtained from various heterogeneous financial data sources. These fingerprints contain semantic features and business meaning associations of fields such as transaction amount, transfer time, account balance, and consumption scenario description. Based on these preliminary semantic fingerprints, the information from each field is input into a pre-trained natural language processing model. Through feature enhancement and encoding, a unique semantic vector representation is generated for each field. All semantic vector representations are then categorized and organized according to transaction type and account attributes to construct a field semantic layer.

[0057] Then, pairwise similarity calculations are performed on the semantic vector representations in the field semantic layer. The cosine similarity algorithm is used to determine the degree of association. Through clustering, fields with a semantic association degree higher than a preset threshold are grouped into one category. For example, transaction amounts, total consumption amounts, and transfer amounts from different data sources are merged into a unified logical field. At the same time, the source and semantic association relationship of the original fields under each category are recorded to construct a logical surface layer.

[0058] Next, based on the unified logical fields in the logical table layer, the knowledge graph containing predefined business concept nodes such as user transactions, fund transfers, and account management is retrieved. Semantic matching is used to establish an association mapping between each unified logical field and the corresponding business concept node. For example, the unified logical field of transaction amount is mapped to the user fund flow business concept node. All mapping relationships are integrated to build a business semantic layer.

[0059] Then, a hierarchical index organization is carried out on the field semantic layer, logical table layer, and business semantic layer. An index directory is established according to transaction scenarios and business concept categories. At the same time, an inter-layer mapping relationship table is created to record in detail the correspondence between field semantic vectors and unified logical fields, and between unified logical fields and business concept nodes. The index organization and mapping relationship table form a complete multi-layer semantic mapping.

[0060] In some embodiments, based on the unified logical fields in the logical table layer, knowledge graph technology is used to associate and map the unified logical fields with predefined business concept nodes to construct a business semantic layer, including: Retrieve the unified logical fields in the logical table layer and load the predefined business concept ontology library; Based on the semantic features of fields in the unified logical fields and the semantic descriptions of concepts in the business concept ontology, semantic similarity is calculated and a candidate set of field concept mappings is generated. The relational reasoning algorithm is applied to verify the consistency of the candidate set of field concept mappings, eliminate semantic conflict mappings, and determine the optimal field concept mapping relationship. Based on the optimal field concept mapping relationship, the unified logical fields are connected with the corresponding business concept nodes to generate a business concept subgraph; Perform topology optimization and weight allocation on the business concept subgraph to construct the business semantic layer.

[0061] This embodiment obtains unified logical fields from the logical surface and loads a predefined business concept ontology library, providing a standardized and business-aligned basic framework for mapping, ensuring the compliance and relevance of associations. Next, it calculates similarity based on field semantic features and concept semantic descriptions to generate a candidate set, accurately filtering potential matching relationships, reducing invalid mappings, and improving mapping efficiency. Furthermore, it uses a relational reasoning algorithm to verify the consistency of the candidate set, effectively eliminating semantic conflicts and ensuring the accuracy and uniqueness of the mapping between fields and business concepts. Additionally, it generates a business concept subgraph based on the optimal mapping relationship, clearly presenting the association path between unified logical fields and business concepts, deeply connecting technical fields with actual business scenarios. Finally, it performs topology optimization and weight allocation on the business concept subgraph, optimizing the structural rationality of the semantic layer and highlighting the priority of core business associations, providing efficient support for subsequent query parsing and semantic matching, further improving the relevance and convenience of data application.

[0062] In the healthcare field, unified logical fields in the logical surface are obtained, including blood glucose test results, chest imaging abnormality descriptions, blood routine white blood cell counts, patient body temperature records, etc. At the same time, a predefined medical business concept ontology library is loaded. This ontology library contains business concept nodes and semantic descriptions of concepts such as patient basic vital signs monitoring, imaging diagnosis, blood test indicators, and chronic disease management.

[0063] Then, based on the semantic features of each unified logical field, such as the numerical range and detection time attribute corresponding to the blood glucose test result, it is compared with the semantic description of the corresponding concept in the business concept ontology. The semantic similarity calculation method is used to obtain the matching score and generate a candidate set of field concept mapping. For example, the blood glucose test result corresponds to two candidate concept nodes: patient basic vital sign monitoring and chronic disease management.

[0064] Then, the relational reasoning algorithm is applied to verify the consistency of the candidate set of field concept mappings. Combined with the medical business logic, the semantic relevance is judged to eliminate conflicting mappings. For example, the optimal field concept mapping relationship is determined to be blood glucose test results and patient basic vital sign monitoring, and redundant mappings for chronic disease management are eliminated.

[0065] Next, based on the optimal field concept mapping relationship, all unified logical fields are connected with the corresponding medical business concept nodes to generate a business concept subgraph containing related paths such as basic vital signs, imaging diagnosis, and blood tests.

[0066] Subsequently, the business concept subgraph was optimized in terms of topology, and the hierarchical relationship between each concept node was sorted out. For example, basic vital sign monitoring was used as the core node to associate with other test-type nodes. At the same time, weights were allocated according to the priority of medical business, and higher weights were assigned to the mappings related to basic vital sign monitoring. Finally, a business semantic layer in the field of medical and health care was constructed.

[0067] In the fintech field, unified logical fields are obtained from the logical table layer, including transaction amount, account balance changes, cross-border transfer records, credit card spending details, etc. At the same time, a predefined financial business concept ontology library is loaded, which contains business concept nodes and semantic descriptions such as user fund flow, cross-border payment settlement, personal account asset management, and consumer credit records.

[0068] Next, based on the semantic features of each unified logical field, such as the transaction region and fund flow attributes corresponding to cross-border transfer records, they are compared with the semantic descriptions of the corresponding concepts in the business concept ontology. The matching score is calculated through semantic similarity, and a candidate set of field concept mappings is generated. For example, cross-border transfer records correspond to two candidate concept nodes: cross-border payment settlement and user fund flow.

[0069] Then, the relational reasoning algorithm is applied to verify the consistency of the candidate set of field concept mappings. Combined with financial business rules, the semantic rationality is judged and conflict mappings are eliminated. For example, cross-border transfer records and cross-border payment settlements are determined to be the optimal field concept mapping relationship, and the generalized mapping of user fund flows is excluded.

[0070] Subsequently, based on the optimal field concept mapping relationship, all unified logical fields are connected with the corresponding financial business concept nodes to generate a business concept subgraph containing related paths such as capital flow, cross-border payment, and asset management.

[0071] Next, the business concept subgraph is optimized in terms of topology, and the relationship logic between each concept node is sorted out. For example, user fund flow is used as the core node to associate with other transaction nodes. At the same time, weights are allocated according to the importance of financial business, and higher weights are allocated to mappings related to cross-border payment and settlement. Finally, a business semantic layer in the field of financial technology is constructed.

[0072] In some embodiments, the training method for the pre-trained natural language processing model includes: Collect multi-source domain text data and perform cleaning, word segmentation and annotation processing to generate structured training corpus; Based on structured training corpora, we analyze word distribution and sequence patterns to determine the architecture hyperparameters of the neural network model. Based on the architecture hyperparameters, initialize the model parameters, including the word embedding matrix, attention weights, and feedforward network parameters. Based on structured training corpus, a dynamic masking strategy is constructed to generate masked sample sequences. The sample sequences and model parameters are then input into the training process to perform a self-supervised pre-training task. The basic language model parameters are obtained through iterative optimization. Obtain downstream field semantic annotation data, perform supervised fine-tuning based on basic language model parameters and field semantic annotation data, update model parameters, and obtain a task-adaptive model suitable for field semantic understanding; Knowledge distillation and parameter quantization are applied to the task adaptation model to generate a lightweight final prediction training model, which is used to generate semantic representations of field content based on field sample data.

[0073] Specifically, we collect text data from multiple sources, covering different types of narrative, explanatory, and record-based texts. We then perform text cleaning to remove redundant characters, invalid line breaks, and duplicate content. Subsequently, we use word segmentation tools to break down the cleaned text into the smallest semantic units, label the semantic relationships and parts of speech of each unit, and finally integrate them to form a structured training corpus containing unit sequences, labels, and contextual information.

[0074] Next, based on the generated structured training corpus, we statistically analyzed the frequency of word occurrence, the distribution range of high-frequency words, and the overall coverage of words. We also analyzed the length distribution pattern of text sequences and the contextual dependency pattern between word units. Based on the analysis results, we determined the architecture hyperparameters of the neural network model: the model layer depth was set to 8 layers, the number of attention heads was set to 12, the hidden layer dimension was set to 768, the word embedding dimension was set to 768, and the dropout probability was set to 0.1.

[0075] Then, based on the determined architecture hyperparameters, the model parameters are initialized. The word embedding matrix is ​​initialized randomly using a normal distribution, the attention weight matrix is ​​initialized using a uniform distribution, the weight parameters of the feedforward network are initialized using the Xavier initialization method, and the bias terms are uniformly initialized to 0, forming a complete initial parameter set for the model that includes the word embedding matrix, attention weights, and feedforward network parameters.

[0076] A dynamic masking strategy is then constructed based on the structured training corpus. A masking probability of 0.05 is set for high-frequency words with frequencies above a preset threshold, and a masking probability of 0.2 is set for low-frequency words with frequencies below a preset threshold. The masking ratio of a single sequence does not exceed 0.15. Following this strategy, the word sequences in the structured training corpus are randomly masked to generate masked sample sequences. These sample sequences and the initial parameter set of the model are input into the training framework. The self-supervised pre-training task is set as masked word prediction. The prediction loss is calculated through forward propagation, and the Adam optimization algorithm is applied to iteratively update the model parameters. After 500,000 training steps, the iteration stops, and the basic language model parameters are obtained.

[0077] Subsequently, downstream field semantic annotation data is obtained, which includes sample data of various fields and corresponding semantic category labels and semantic association labels. Based on the basic language model parameters, the field semantic annotation data is input into the model, and supervised fine-tuning training is performed. With cross-entropy loss as the optimization objective, the model parameters are iteratively adjusted. After 100,000 fine-tuning training steps, a task-adapted model suitable for field semantic understanding is obtained.

[0078] Next, knowledge distillation is applied to the task adaptation model: using the task adaptation model as the teacher model, a lightweight student model with 4 layers and 384 hidden layer dimensions is constructed. Using structured training corpus and field semantic annotation data, the student model learns the output probability distribution of the teacher model. After distillation, 8-bit integer quantization is performed on the student model parameters to map floating-point parameters to integer parameters, and finally a lightweight final prediction model is generated. This model can be used to generate semantic representations of field content based on field sample data.

[0079] S103: Receive the user's query request, parse the query request based on the multi-layer semantic mapping, and generate a logical query plan; In this embodiment, receiving a user's query request, parsing the query request based on multi-layer semantic mapping, and generating a logical query plan includes: It receives query requests input by users and uses a large language model to extract semantic features from the query requests, generating a query intent representation that includes query objects, query indicators, and query conditions. Match the query intent with the business semantic layer to obtain business concept nodes; Based on business concept nodes, traverse multiple semantic mapping layers to obtain the mapping relationship between the unified logical fields in the corresponding logical table layer and the original fields in the field semantic layer. Based on the mapping relationship between the unified logical fields and the original fields, and combined with the query objects, query metrics and query conditions in the query intent representation, an intermediate query expression is constructed. Perform syntax and semantic validation on the intermediate query expression to generate a logical query plan.

[0080] This embodiment extracts objects, metrics, and conditions from the query intent using a large language model and generates a standardized representation. This accurately captures the core user query needs and is compatible with various query formats such as natural language and SQL, lowering the operational threshold for users. Secondly, it matches the query intent with the business semantic layer to obtain business concept nodes, achieving a deep connection between query requirements and actual business scenarios and avoiding the disconnect between technical fields and business semantics. Next, based on the business concept nodes, it traverses multiple layers of semantic mapping to accurately associate unified logical fields with the original field mapping relationship, opening up the field association link from business to technology and ensuring the accuracy of field matching in cross-source data queries. Then, it combines the query intent with the field mapping relationship to construct intermediate query expressions, making the query logic structured and clear, providing a standardized foundation for subsequent verification. In addition, it generates a logical query plan through dual syntax and semantic verification, effectively avoiding query syntax errors and semantic conflicts, ensuring the feasibility and accuracy of the query plan, and providing reliable support for subsequent cross-source query execution.

[0081] In the healthcare field, a query request is received from a user. This request is a medical data query requirement expressed in natural language. A large language model is used to extract semantic features from the query request to generate a query intent representation that includes the query object being elderly patients over 60 years old, the query indicator being the average blood glucose level, and the query condition being the test records of the past six months.

[0082] Next, the generated query intent representation is matched with the business semantic layer in the healthcare field to filter out the corresponding business concept nodes, including patient age grouping, blood glucose and vital sign monitoring, time range filtering, etc.

[0083] Then, based on the above business concept nodes, we traverse multiple semantic mapping layers, obtain the unified logical field as the blood glucose test result from the logical table layer, and at the same time obtain the original field mapping relationship corresponding to the unified logical field from the field semantic layer, covering the original fields and data type association information of fasting blood glucose, 2-hour postprandial blood glucose, etc. from different medical institutions.

[0084] Subsequently, based on the unified logical field blood glucose test results and the original field mapping relationship, combined with the query object, query index and query conditions in the query intent representation, an intermediate query expression is constructed to clarify that it is necessary to extract various blood glucose test data of patients over 60 years old in the past six months from the associated data source and calculate the average value.

[0085] Next, perform syntax validation on the intermediate query expression to ensure that the field call format is standardized, and perform semantic validation to verify the logical consistency between the blood glucose test field and the age and time filtering conditions. After eliminating semantic conflicts, generate a logical query plan that includes the data access range, calculation rules, and filtering conditions.

[0086] In the fintech field, a query request is received from a user. This request is a statement related to financial data query. A large language model is used to extract semantic features from the query request to generate a query intent representation that includes the query object being an individual user, the query indicator being the total amount of cross-border transactions, and the query condition being the transaction records of the previous year.

[0087] The generated query intent is matched with the business semantic layer in the fintech field to determine the corresponding business concept nodes, including personal cross-border payments, annual fund statistics, and transaction amount summaries.

[0088] Based on the aforementioned business concept nodes, the system traverses multiple semantic mapping layers to obtain a unified logical field for cross-border transaction amount from the logical table layer. Simultaneously, it obtains the original field mapping relationship corresponding to this unified logical field from the field semantic layer, covering original fields such as cross-border transfer amount and foreign currency transaction limit of different payment platforms, as well as data association rules.

[0089] Next, based on the unified logical field of cross-border transaction amount and the original field mapping relationship, combined with the query object, query indicator and query conditions in the query intent representation, an intermediate query expression is constructed to clarify that all cross-border transaction data of individual users in the previous year from various related data sources need to be integrated and the total amount is calculated.

[0090] Then, the intermediate query expression is syntax-validated to ensure that the field associations and calculation formats are compliant. Semantic validation is also performed to confirm the logical rationality of the cross-border transaction fields, user types, and annual time conditions. After eliminating syntax errors and semantic contradictions, a logical query plan containing a list of data source accesses, statistical logic, and filtering rules is generated.

[0091] S104: Obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in conjunction with the real-time data source status to generate a cross-source query plan; In this embodiment, obtaining the real-time data source status, and generating a cross-source query plan based on the logical query plan and the real-time data source status includes: Monitor the real-time operating metrics of each data source and generate data source status vectors; Input the data source state vector and logical query plan into the reinforcement learning optimization model to calculate the expected execution cost of each candidate execution path; Based on the expected execution cost, the candidate execution path with the lowest total execution cost is selected as the optimal path, and the computational task allocation ratio of the optimal path is determined. Based on the optimal path and task allocation ratio, the logical query plan is decomposed into multiple source-specific query sub-plans (a source-specific query sub-plan refers to an independent query task unit relative to a single data source), and the execution dependencies between each source-specific query sub-plan are determined. Combine source-specific query subplans and execution dependencies to generate a cross-source query plan.

[0092] This embodiment generates a state vector by monitoring real-time operating metrics of the data source, which comprehensively captures dynamic information such as load, latency, and availability, providing accurate real-time basis for path selection. Secondly, the state vector and logical query plan are input into a reinforcement learning optimization model to calculate the expected execution cost. Leveraging the model's dynamic decision-making capabilities, a scientific quantitative assessment of execution costs is achieved, adapting to the dynamic changes in complex cross-source scenarios. Then, based on the lowest expected execution cost, the optimal path is selected and the task allocation ratio is determined, maximizing resource utilization and reducing overall execution overhead. Next, the logical query plan is decomposed into source-specific query sub-plans and execution dependencies are clearly defined, ensuring clear division of labor and orderly execution of cross-source query tasks, avoiding task conflicts and resource waste. Finally, the sub-plans and dependencies are combined to generate a cross-source query plan, which not only fully preserves the original query logic but also ensures accurate matching between the plan and the real-time data source state, guaranteeing the efficiency, stability, and feasibility of cross-source query execution.

[0093] In the healthcare field, real-time operational metrics of various heterogeneous data sources related to healthcare are monitored, including response latency, CPU load, data transmission bandwidth, and node availability of hospital information systems (such as hospital HIS systems), laboratory equipment databases, and medical image storage systems. These metrics are then integrated to generate state vectors corresponding to each data source.

[0094] Then, the state vectors of each data source and the generated logical query plan are input into the reinforcement learning optimization model. The logical query plan aims to obtain the average blood glucose value of patients over 60 years old in the past six months. The model calculates the expected execution cost of different candidate execution paths based on the data source load, data transmission cost and query complexity. Candidate paths include solutions such as prioritizing data extraction from the HIS system and then supplementing it with test equipment data, and directly synchronously calling multi-source data.

[0095] Next, based on the expected execution cost output by the model, the candidate path with the lowest total execution cost is selected as the optimal path. In the optimal path, the HIS system undertakes 70% of the data query tasks, and the verification device database undertakes 30% of the supplementary query tasks, thus clarifying the task allocation ratio.

[0096] Based on the optimal path and task allocation ratio, the original logical query plan is decomposed into two source-specific query sub-plans. The first is to query the basic information and blood glucose test records of patients over 60 years old from the HIS system, and the second is to query the supplementary blood glucose data of such patients that have not been entered into the HIS system from the laboratory equipment database. At the same time, the execution dependency relationship is determined to execute the HIS system query sub-plan first and then supplement the query based on the result matching.

[0097] Subsequently, the two source-specific query sub-plans and explicit execution dependencies are combined, and data source connection parameters and data fusion rules are added to generate a cross-source query plan that can cross the HIS system and the testing equipment database.

[0098] In the fintech field, we monitor real-time operational metrics of various heterogeneous data sources related to finance, including transaction processing latency, server memory usage, network stability, and data query concurrency of bank core transaction systems, third-party payment platform databases, and cross-border clearing log systems. We then integrate these metrics to generate state vectors corresponding to each data source.

[0099] Next, the state vectors of each data source and the generated logical query plan are input into the reinforcement learning optimization model. The logical query plan aims to calculate the total amount of cross-border transactions of individual users in the previous year. The model combines the real-time load of the data source, the security of data transmission and the efficiency of query response to calculate the expected execution cost of different candidate execution paths. Candidate paths include simultaneous querying from the bank's core system and third-party payment platforms, and prioritizing the use of cross-border clearing log system data.

[0100] Then, based on the expected execution cost output by the model, the candidate path with the lowest total execution cost is selected as the optimal path. In the optimal path, the third-party payment platform database undertakes 60% of the cross-border transaction data query task, the bank's core transaction system undertakes 30% of the related transaction query task, and the cross-border clearing log system undertakes 10% of the large-value transaction supplementary query task, thus clarifying the allocation ratio of the computing tasks of each data source.

[0101] Based on the optimal path and task allocation ratio, the original logical query plan is decomposed into three source-specific query sub-plans: querying the user's cross-border transaction details from the third-party payment platform database, querying the user's associated cross-border transfer records from the bank's core transaction system, and querying the user's large-amount cross-border transaction data from the cross-border clearing log system. The dependency relationship of executing queries in the order of third-party payment platform, bank core system, and cross-border clearing log system is determined.

[0102] Next, the three source-specific query sub-plans and execution dependencies are combined, and data validation rules and transaction amount aggregation logic are added to generate a cross-source query plan that can span multiple financial data sources.

[0103] S105: Execute the cross-source query plan, obtain the query results of each heterogeneous data source, and fuse all the query results through a semantic consistency alignment model to output a fused result in a unified format.

[0104] In this embodiment, a cross-source query plan is executed to obtain query results from various heterogeneous data sources. Then, a semantic consistency alignment model is used to fuse all query results, outputting a unified format of fused results, including: Execute a cross-source query plan, which contains multiple source-specific query sub-plans and execution dependencies between sub-plans, and generate query execution status and data source access paths; Based on the query execution status and data source access path, partial query result sets are obtained from various heterogeneous data sources. The partial query result sets contain structured data records, semi-structured documents and unstructured data fragments. Each partial query result set is accompanied by a data freshness mark and confidence score. A portion of the query result set is input into a pre-trained semantic consistency alignment model. The semantic consistency alignment model identifies the entity dimensions and metrics in each portion of the query result set, calculates weight allocation parameters based on data freshness tags and confidence scores, applies a weighted aggregation algorithm to metrics with semantic conflicts, and applies a hierarchical alignment strategy to entities with inconsistent dimensions to generate a semantically consistent result set. Based on semantically consistent result sets, standardized format conversion rules are applied to map dimensional fields to a unified dimensional model, convert metric fields to a standard unit system, attach data lineage metadata and quality assessment indicators, and output fusion results that conform to a predefined architecture.

[0105] This embodiment executes the cross-source query plan in an orderly manner according to the sub-plan execution dependencies, synchronously generating query execution status and data source access paths. This ensures the standardization and stability of multi-source data queries and provides a clear path for subsequent problem tracing. Secondly, the acquired partial query result set covers multiple types of data, including structured, semi-structured, and unstructured data, and includes data freshness tags and confidence scores. This provides a quantitative basis for weight allocation and quality screening during the fusion process, improving the reliability of the results. Next, the pre-trained semantic consistency alignment model accurately identifies entity dimensions and metrics, calculates weights based on freshness and confidence, resolves semantic conflicts through a weighted aggregation algorithm, and normalizes dimensional differences through a hierarchical alignment strategy. This effectively solves the problem of semantic inconsistency in multi-source data, ensuring the accuracy and uniformity of the fusion results. Standardized format conversion rules are applied to unify dimensional models and standardize measurement units. At the same time, data lineage metadata and quality assessment indicators are added, ensuring that the output results conform to general usage specifications and improving data traceability and credibility. This forms a closed-loop optimization from query execution to result output, adapting to the efficient fusion needs of cross-source multi-type data.

[0106] In the healthcare field, a cross-source query plan is executed. This plan includes three source-specific query sub-plans, corresponding to the hospital HIS system, the laboratory equipment database, and the medical image storage system, respectively. The execution dependency between the sub-plans is that the HIS system query is executed first, and then the laboratory equipment data and image report data are matched based on the returned results. During the execution process, a query execution status including query progress and data transmission status is generated, as well as access link information of each data source, i.e., the data source access path.

[0107] Next, based on the query execution status and data source access path, partial query result sets are obtained from three heterogeneous data sources in sequence: structured data records are tabular data such as patient blood glucose test values ​​and blood routine indicators; semi-structured documents are marked text such as diagnosis and treatment process logs and medical order records; unstructured data fragments are text fragments such as abnormal descriptions and pathological analysis summaries in medical imaging reports. Each part of the query result set is accompanied by a freshness mark corresponding to the data generation time, as well as a confidence score based on equipment accuracy and input specifications.

[0108] Then, all partial query result sets are input into a pre-trained semantic consistency alignment model. The model automatically identifies entity dimensions in each result set, including patient unique identifiers, testing time, testing items, and metrics, including blood glucose concentration, white blood cell count, and degree of image abnormality. Based on freshness markers, recent test data is given higher weight than historical data, and based on confidence scores, data from testing equipment is given higher weight than manually entered data. Weighting parameters are calculated. For metrics such as blood glucose concentration from different devices that have semantic conflicts, a weighted aggregation algorithm is applied to calculate the comprehensive result. For entities with inconsistent dimensions, such as inconsistent patient identifier formats, a hierarchical alignment strategy is applied to standardize the representation and generate a semantically consistent result set.

[0109] Based on the semantically consistent result set, the standardized format conversion rules for medical data are applied to map the dimensional fields to a unified dimensional model that includes patient basic information, test item classification, and test time dimensions. The measurement fields are converted to the international standard unit system, such as blood glucose values ​​being standardized to mmol / L and the degree of image abnormality being quantified into standard levels. At the same time, data lineage metadata is added to clarify the source hospital, data source type and transmission link of each data point, as well as quality assessment indicators such as data integrity rate and conflict resolution rate. Finally, the fusion result that conforms to the predefined architecture of medical data is output.

[0110] In the fintech field, a cross-source query plan is executed. This plan includes three source-specific query sub-plans, corresponding to third-party payment platforms, bank core transaction systems, and cross-border clearing log systems, respectively. The execution dependency between the sub-plans is to prioritize querying transaction data from third-party payment platforms, and then supplement with related transfer records from the bank's core system and large transaction details from the cross-border clearing logs. During the execution process, a query execution status is generated, including the task completion percentage, data verification status, and data source access paths such as network access addresses and connection protocols for each data source.

[0111] Then, based on the query execution status and data source access path, partial query result sets are obtained from three heterogeneous data sources: structured data records are tabular data such as user transaction amounts and account balance changes; semi-structured documents are key-value pair text such as user login logs and transaction process records; unstructured data fragments are text content such as cross-border transaction contract summaries and risk control review opinions. Each part of the query result set is accompanied by a freshness tag corresponding to the transaction occurrence time, as well as a confidence score based on the number of transaction verifications and system reliability.

[0112] Next, all partial query result sets are input into a pre-trained semantic consistency alignment model. The model automatically identifies entity dimensions in each result set, including user unique identifiers, transaction dates, transaction types, and metrics, including transaction amounts, transaction fees, and exchange rates. Based on freshness tags, recent transaction data is weighted higher than past data, and based on confidence scores, data from the bank's core system is weighted higher than data from third-party platforms. Weighting parameters are calculated. For metrics with semantic conflicts, such as differences in amounts recorded on different platforms for the same transaction, a weighted aggregation algorithm is applied to obtain the final value. For entities with inconsistent dimensions, such as inconsistent user identifier encoding rules, a hierarchical alignment strategy is applied to unify the format and generate a semantically consistent result set.

[0113] Subsequently, based on the semantically consistent result set, the financial data standardization format conversion rules are applied to map the dimension fields to a unified dimension model of user basic information, transaction scenario classification, and settlement cycle, and convert the metric fields into a unified currency unit system, such as converting the transaction amount of various currencies into RMB. At the same time, data lineage metadata is attached to clarify the source institution, data source name and transmission time of each data, as well as quality assessment indicators such as transaction record completeness and amount accuracy. Finally, the fusion result that conforms to the predefined architecture of financial data is output.

[0114] Furthermore, a cross-source query plan is executed to obtain query results from various heterogeneous data sources. Then, all query results are fused using a semantic consistency alignment model, and the fused results, in a unified format, are output, including: Update the multi-layer semantic mapping based on the fusion results.

[0115] Specifically, for the field semantic layer, the newly added field association features and semantic equivalence relationships in the fusion results are extracted, the semantic vector representation of the original fields is optimized, the semantic mapping records of the new fields verified during the fusion process are supplemented, the vector parameters in the field semantic layer that do not match the actual data semantics are corrected, and the accuracy of semantic association between fields is enhanced.

[0116] For the logical surface layer, based on the field aggregation rules and semantic similarity reflected in the fusion results, the coverage of the unified logical fields is adjusted, the semantically equivalent original fields confirmed during the fusion process are included in the corresponding unified logical fields, the semantic association records between fields are updated, and invalid field association entries in the logical surface layer are deleted.

[0117] For the business semantic layer, based on the actual business references of the fields reflected in the fusion results, the mapping relationship between the unified logical fields and the predefined business concept nodes is corrected, the association paths of new business concepts discovered during the fusion process are supplemented, and the topology and weight allocation of the business concept subgraph are adjusted to ensure that the business semantic layer matches the business attributes of the actual data.

[0118] Finally, the inter-layer mapping relationship table is updated synchronously to record the updated content and related changes of the field semantic layer, logical table layer, and business semantic layer, ensuring the overall consistency and timeliness of multi-layer semantic mapping.

[0119] Please see Figure 3 This embodiment provides a real-time data intelligent query device 300, including: Registration unit 301 is used to register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint. The construction unit 302 is used to construct a multi-layer semantic mapping based on the preliminary semantic fingerprint, wherein the multi-layer semantic mapping includes a field semantic layer, a logical table layer and a business semantic layer; Parsing unit 303 is used to receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; Selection unit 304 is used to obtain the real-time data source status, and based on the logical query plan, combine the real-time data source status to perform dynamic path selection and generate a cross-source query plan. The fusion unit 305 is used to execute the cross-source query plan, obtain the query results of each heterogeneous data source, and fuse all the query results through a semantic consistency alignment model to output a fusion result in a unified format.

[0120] Furthermore, the registration unit 301 includes: The data source registration subunit is used to register multiple heterogeneous data sources through a unified interface. The heterogeneous data sources include structured databases, semi-structured log systems, and unstructured object storage. An extraction subunit is used to automatically extract field metadata and field sample data from the heterogeneous data source; The name extraction subunit is used to extract the field name and data type features based on the field metadata, and to generate a semantic representation of the field content using a pre-trained language model based on the field sample data; The fusion processing subunit is used to fuse the field name, data type features, and semantic representation of the field content to obtain the business meaning; The encoding subunit is used to vectorize the business meaning and generate a field semantic vector. The vector integration subunit is used to integrate all field semantic vectors and calculate the overall semantic feature distribution of the integrated field semantic vectors to construct a semantic fingerprint index structure. The standardization processing subunit is used to standardize the semantic fingerprint index structure and generate a preliminary semantic fingerprint.

[0121] Furthermore, the building unit 302 includes: The semantic layer construction subunit is used to generate semantic vector representations of each field based on the preliminary semantic fingerprint using a pre-trained natural language processing model, and to construct the field semantic layer. The clustering processing subunit is used to perform similarity calculation and clustering processing on the semantic vector representation in the semantic layer of the field, merge semantically similar fields into a unified logical field, establish semantic association between fields, and construct a logical surface layer. The association mapping subunit is used to associate and map the unified logical fields in the logical table layer with predefined business concept nodes using knowledge graph technology to construct a business semantic layer. The hierarchical index subunit is used to organize the field semantic layer, logical table layer and business semantic layer into a hierarchical index, establish an inter-layer mapping relationship table, and form a multi-layer semantic mapping.

[0122] Furthermore, the associated mapping subunit includes: The field retrieval sub-unit is used to retrieve the unified logical fields in the logical table layer and load the predefined business concept ontology library; The similarity calculation subunit is used to calculate semantic similarity and generate a candidate set of field concept mappings based on the field semantic features in the unified logical field and the concept semantic descriptions in the business concept ontology library. The verification subunit is used to apply a relational reasoning algorithm to verify the consistency of the candidate set of field concept mappings, eliminate semantic conflict mappings, and determine the optimal field concept mapping relationship. The connection establishment sub-unit is used to establish a connection between the unified logical field and the corresponding business concept node based on the optimal field concept mapping relationship, and generate a business concept sub-graph. The topology optimization subunit is used to perform topology optimization and weight allocation on the business concept subgraph to construct the business semantic layer.

[0123] Furthermore, the parsing unit 303 includes: The feature extraction subunit is used to receive the query request input by the user and use a large language model to extract semantic features from the query request to generate a query intent representation containing query objects, query indicators and query conditions. The matching subunit is used to match the query intent representation with the business semantic layer to obtain business concept nodes; Traverse sub-units to obtain the mapping relationship between the unified logical fields in the corresponding logical table layer and the original field mapping relationship in the field semantic layer, based on the business concept node. The expression construction subunit is used to construct an intermediate query expression based on the mapping relationship between the unified logical field and the original field, combined with the query object, query index and query conditions in the query intent representation; The semantic validation subunit is used to perform syntax and semantic validation on the intermediate query expression and generate a logical query plan.

[0124] Furthermore, the selection unit 304 includes: The metrics monitoring subunit is used to monitor the real-time operating metrics of each data source and generate data source status vectors. The cost calculation subunit is used to input the data source state vector and the logical query plan into the reinforcement learning optimization model to calculate the expected execution cost of each candidate execution path; The path selection subunit is used to select the candidate execution path with the lowest total execution cost as the optimal path based on the expected execution cost, and to determine the computation task allocation ratio of the optimal path. The plan decomposition subunit is used to decompose the logical query plan into multiple source-specific query sub-plans according to the optimal path and task allocation ratio, and to determine the execution dependencies between each source-specific query sub-plan; A plan generation sub-unit is used to combine the source-specific query sub-plan and the execution dependencies to generate a cross-source query plan.

[0125] Furthermore, the fusion unit 305 includes: An update subunit is used to update the multi-layer semantic mapping based on the fusion result.

[0126] This invention provides a real-time intelligent data query device. First, it registers multiple heterogeneous data sources, identifies the field names, data types, and business meanings of each heterogeneous data source, and generates preliminary semantic fingerprints. Based on these preliminary semantic fingerprints, it constructs a multi-layer semantic mapping, including a field semantic layer, a logical table layer, and a business semantic layer. It receives user query requests, parses the query requests based on the multi-layer semantic mapping, and generates a logical query plan. It obtains the real-time data source status, and based on the logical query plan and the real-time data source status, performs dynamic path selection to generate a cross-source query plan. It executes the cross-source query plan, obtains the query results from each heterogeneous data source, and fuses all query results using a semantic consistency alignment model, outputting a fused result in a unified format. By parsing heterogeneous data sources to generate preliminary semantic fingerprints and constructing multi-layer semantic mapping, it achieves data semantic unification. It also uses a large language model to parse query semantics and dynamically optimizes the query path based on the real-time data source status. This solution not only solves the problems of high latency, high conversion overhead, complex maintenance, and limited real-time analysis of traditional ETL, but also overcomes the bottlenecks of semantic ambiguity, inability to understand business semantics, and lack of semantic-level optimization in existing Zero-ETL. It enables efficient cross-source data association queries, reduces the cost of manual intervention, improves system adaptability and self-evolution capabilities, and ensures query real-time performance and accuracy.

[0127] Specific limitations regarding the real-time data intelligent query device can be found in the limitations of the real-time data intelligent query method described above, and will not be repeated here. Each unit in the aforementioned real-time data intelligent query device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0128] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a real-time data intelligent query method on the server side.

[0129] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of a real-time data intelligent query method.

[0130] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint; Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, which includes a field semantic layer, a logical table layer, and a business semantic layer. Receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; Obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in conjunction with the real-time data source status to generate a cross-source query plan; The cross-source query plan is executed to obtain the query results from each of the heterogeneous data sources. The results of all the query results are then fused using a semantic consistency alignment model, and a fused result in a unified format is output.

[0131] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint; Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, which includes a field semantic layer, a logical table layer, and a business semantic layer. Receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; Obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in conjunction with the real-time data source status to generate a cross-source query plan; The cross-source query plan is executed to obtain the query results from each of the heterogeneous data sources. The results of all the query results are then fused using a semantic consistency alignment model, and a fused result in a unified format is output.

[0132] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0135] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A real-time intelligent data query method, characterized in that, include: Register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint; Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, which includes a field semantic layer, a logical table layer, and a business semantic layer. Receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; Obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in conjunction with the real-time data source status to generate a cross-source query plan; The cross-source query plan is executed to obtain the query results from each of the heterogeneous data sources. The results of all the query results are then fused using a semantic consistency alignment model, and a fused result in a unified format is output.

2. The real-time data intelligent query method according to claim 1, characterized in that, The process of registering multiple heterogeneous data sources, identifying the field names, data types, and business meanings in each heterogeneous data source, and generating a preliminary semantic fingerprint includes: Multiple heterogeneous data sources can be registered through a unified interface. These heterogeneous data sources include structured databases, semi-structured log systems, and unstructured object storage. Automatically extract field metadata and field sample data from the heterogeneous data source; Based on the field metadata, extract the field name and data type features, and use a pre-trained language model to generate a semantic representation of the field content based on the field sample data; The field name, data type characteristics, and semantic representation of the field content are fused together to obtain the business meaning; The business meaning is vectorized and encoded to generate a field semantic vector; Integrate all field semantic vectors and calculate the overall semantic feature distribution of the integrated field semantic vectors to construct a semantic fingerprint index structure; The semantic fingerprint index structure is standardized to generate a preliminary semantic fingerprint.

3. The real-time data intelligent query method according to claim 1, characterized in that, Based on the preliminary semantic fingerprint, a multi-layer semantic mapping is constructed, which includes a field semantic layer, a logical table layer, and a business semantic layer. Based on the preliminary semantic fingerprint, semantic vector representations of each field are generated through a pre-trained natural language processing model to construct a field semantic layer; Similarity calculation and clustering are performed on the semantic vector representations in the semantic layer of the fields to merge semantically similar fields into unified logical fields, establish semantic relationships between fields, and construct a logical surface layer. Based on the unified logical fields in the logical table layer, knowledge graph technology is used to associate and map the unified logical fields with predefined business concept nodes to construct a business semantic layer. The field semantic layer, logical table layer, and business semantic layer are organized into a hierarchical index, and an inter-layer mapping relationship table is established to form a multi-layer semantic mapping.

4. The real-time data intelligent query method according to claim 3, characterized in that, The step of constructing a business semantic layer by associating and mapping the unified logical fields in the logical table layer with predefined business concept nodes using knowledge graph technology includes: Retrieve the unified logical fields in the logical table layer and load the predefined business concept ontology library; Based on the semantic features of the fields in the unified logical fields and the semantic descriptions of the concepts in the business concept ontology, semantic similarity is calculated and a candidate set of field concept mappings is generated; The relational reasoning algorithm is applied to verify the consistency of the candidate set of field concept mappings, eliminate semantic conflict mappings, and determine the optimal field concept mapping relationship. Based on the optimal field concept mapping relationship, the unified logical field is connected with the corresponding business concept node to generate a business concept subgraph; The business concept subgraph is topology optimized and weighted to construct a business semantic layer.

5. The real-time data intelligent query method according to claim 1, characterized in that, The process of receiving a user's query request, parsing the query request based on the multi-layer semantic mapping, and generating a logical query plan includes: Receive user input query requests and use a large language model to extract semantic features from the query requests to generate a query intent representation that includes query objects, query indicators and query conditions; The query intent is matched with the business semantic layer to obtain business concept nodes; Based on the business concept node, traverse the multi-layer semantic mapping to obtain the mapping relationship between the unified logical field in the corresponding logical table layer and the original field in the field semantic layer. Based on the mapping relationship between the unified logical fields and the original fields, and combined with the query objects, query metrics and query conditions in the query intent representation, an intermediate query expression is constructed. The intermediate query expression is subjected to syntax and semantic validation to generate a logical query plan.

6. The real-time data intelligent query method according to claim 1, characterized in that, The step of obtaining the real-time data source status, and generating a cross-source query plan based on the logical query plan and the real-time data source status includes: Monitor the real-time operating metrics of each data source and generate data source status vectors; Input the data source state vector and the logical query plan into the reinforcement learning optimization model to calculate the expected execution cost of each candidate execution path; Based on the expected execution cost, the candidate execution path with the lowest total execution cost is selected as the optimal path, and the computational task allocation ratio of the optimal path is determined. Based on the optimal path and task allocation ratio, the logical query plan is decomposed into multiple source-specific query sub-plans, and the execution dependencies between each source-specific query sub-plan are determined. The source-specific query subplan and the execution dependencies are combined to generate a cross-source query plan.

7. The real-time data intelligent query method according to claim 1, characterized in that, The process of executing the cross-source query plan, obtaining query results from each of the heterogeneous data sources, and then fusing all the query results using a semantic consistency alignment model to output a fused result in a unified format includes: The multi-layer semantic mapping is updated based on the fusion result.

8. A real-time data intelligent query device, characterized in that, include: The registration unit is used to register multiple heterogeneous data sources, identify the field names, data types and business meanings in each heterogeneous data source, and generate a preliminary semantic fingerprint. The construction unit is used to construct a multi-layer semantic mapping based on the preliminary semantic fingerprint, wherein the multi-layer semantic mapping includes a field semantic layer, a logical table layer, and a business semantic layer; The parsing unit is used to receive user query requests, parse the query requests based on the multi-layer semantic mapping, and generate a logical query plan; The selection unit is used to obtain the real-time data source status, and based on the logical query plan, perform dynamic path selection in combination with the real-time data source status to generate a cross-source query plan. The fusion unit is used to execute the cross-source query plan, obtain the query results of each heterogeneous data source, and fuse all the query results through a semantic consistency alignment model to output a fusion result in a unified format.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the real-time intelligent data query method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the real-time data intelligent query method as described in any one of claims 1 to 7.

Images