47 results about "Schema matching" patented technology

The invention provides a flood peak evolution path and dam break risk early warning method and system. According to the method, a time-space coupling water conservancy data set is constructed by fusing multi-dimensional water conservancy monitoring data and regional rainfall prediction information, and a basin topology perception model is established. And further performing mode matching on the parameters and a historical dam break event characteristic spectrum, and generating a dynamic response strategy set including a flood storage and detention area capacity allocation scheme by correcting a space weight of a matching result. And finally, based on a watershed topology perception model, matching the strategy set with historical dam break features, realizing accurate identification of a flood peak evolution path and graded early warning of dam break risks, outputting a water conservancy analysis report containing risk grades, and realizing closed-loop management from data fusion and dynamic prediction to risk decision. According to the technical scheme provided by the invention, the efficiency and accuracy of intelligent analysis of the water conservancy data can be improved.

A system for converting a source data feed schema into a canonical data product including a memory for storing computer-executable instructions and a processor for executing the instructions stored on the memory. Execution of the instructions programs the processor to perform operations that include receiving a source data feed having a source schema, identifying a plurality of data fields of the source schema, assigning a data category from a plurality of predefined data categories to each data field of the plurality of data fields, modifying the source schema based on predefined parameters, wherein the source schema is modified to match a target schema, comparing the modified source schema to the target schema, and in response to a determination that the modified source schema matches the target schema, converting the source data feed to a canonical data product having the target schema based on the assigned data categories.

The invention discloses an electric power system monitoring data analysis method, equipment and a medium. The method comprises the following steps: S1, acquiring a historical monitoring data set and a real-time monitoring data stream of an electric power system; s2, constructing a power equipment operation reference mode library based on the historical monitoring data set; s3, mapping the real-time monitoring data stream to the power equipment operation reference mode library for mode matching processing, and generating a matching degree sequence; s4, performing difference evolution analysis processing on the matching degree sequence to generate an abnormal evolution track; s5, determining starting equipment, a propagation path and an influence range of abnormity according to the abnormity evolution track, and generating an early warning instruction; according to the data analysis method, the accuracy and timeliness of monitoring of the power system are greatly improved on the whole, faults of the power system can be effectively prevented and reduced, the stability and reliability of power supply are guaranteed, and economic losses and social influences caused by the power faults are reduced.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can receive a schema of a source table from a first database and a schema of a plurality of target tables from a second database, identify one or more matching tables among the plurality of target tables based on comparison of the schema of the source table and the schema of the plurality of target tables using a large language model, obtain first sample attribute data from the source table and second sample attribute data from a selected matching table, and identify one or more pairs of matching attributes between the source table and the selected matching table based on comparison of the first sample attribute data and the second sample attribute data using the large language model. Related systems and software for implementing the method are also disclosed.

The invention relates to a dynamic adaptive Text2SQL (Structured Query Language) generation method and system for the power field. The method comprises the following steps: S1, analyzing a power database Schema and automatically generating a diversified natural language-SQL pair data set; s2, performing cleaning, optimization and format standardization on the data, and constructing a high-quality training set and a high-quality test set; s3, querying a prediction database Schema based on a natural language, and generating an SQL statement with accurate pattern matching; s4, a logic disassembly optimization mechanism is introduced, and it is ensured that SQL query conforms to database rules and meets the requirements of the power industry; s5, finely adjusting the large language model to improve the accuracy and the performability of SQL generation; s6, performing compression optimization on the trained model to improve reasoning performance, stability and query accuracy; and S7, efficiently deploying the optimized electric power large language model, and guaranteeing the stability, response speed and model performance of the system through real-time monitoring. According to the method, the accuracy and applicability of converting the natural language query into the SQL statement in the power industry can be effectively improved.

Embodiments provide for schema matching between multiple disparate data sources using multi-dimensional matrices, external embeddings, transformation operations, and similarity metrics.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can transform one or more text benchmarks into one or more first vector embeddings, retrieve a text output from the generative AI model in response to a user prompt, transform the text output into one or more second vector embeddings, measure one or more matching scores based on comparison of the one or more first vector embeddings with the one or more second vector embeddings, and determine reliability of the text output based on evaluation of the one or more matching scores. Related systems and software for implementing the method are also disclosed.

The invention discloses an electric power cold and hot data storage system. The system comprises a data access frequency acquisition module for acquiring electric power data access frequencies of a hot database and cold data; the power data migration module is used for migrating the power data to the cold database for storage if the access frequency of the power data in the hot database is smaller than the preset data access frequency, and migrating the power data to the hot database for storage if the access frequency of the power data in the cold database is larger than or equal to the preset data access frequency; the preset data access frequency adjusting module is used for predicting the power data access frequency trend through a random forest and adjusting the preset data access frequency in the power data migration module, and the data storage optimizing module is used for distributing storage resources by monitoring the use information stored in the hot database and the cold database in real time and combining the access frequency of the power data. By dynamically adjusting the migration threshold value of the cold and hot data, the data storage strategy is ensured to be always matched with the actual access mode.

The invention provides an LLM-based engineering project scheme compliance examination method. The method comprises the following steps: Step 1, data preparation; implementing multi-dimensional data acquisition through a distributed web crawler system; step 2, processing various types of data of images, texts and tables by using a large language model (LLM) technology; step 3, performing Prompt fine tuning optimization on the data, so that the large language model can better understand and generate related contents of compliance review of the engineering project scheme; step 4, performing automatic compliance check based on rule and mode matching; by defining a series of rules and modes, automatic compliance review of project documents is realized; and Step 5, establishing a risk generation assessment and early warning mechanism based on the large model. Through automatic compliance inspection, the workload of manual auditing is reduced, and document specifications are ensured; and an intelligent risk assessment and early warning mechanism identifies and notifies potential risks in real time, so that compliance risks are effectively reduced, and the safety and the overall level of project management are improved.

Techniques for database management are described. A database management system may transmit a request for a data management system of a database to provide a set of metadata attributes for structured data within the database, and may receive a set of metadata attributes for the structured data within the database. The data management system may perform a pattern matching procedure to evaluate the set of metadata attributes for the structured data within the database against one or more patterns associated with a data type to determine one or more locations within the database that include structured data of the data type. Based on the pattern matching procedure, the data management system may output an indication that the one or more locations within the database include structured data of the data type.

The invention provides a medical data credible circulation method and system based on multi-level authority dynamic management, and the method comprises the steps: constructing a multi-mode matching index, and carrying out the feature extraction and vectorization of medical text data, structured data and image data; establishing a three-layer index architecture and an intelligent routing layer; constructing a user-permission-data relation graph by applying a graph algorithm, evaluating a permission use mode in real time, and detecting an abnormal behavior; the abnormal judgment threshold is dynamically adjusted in combination with situation factors such as medical emergency degree, abnormal behaviors are subjected to grading processing, and evidence obtaining information is automatically collected. The invention further comprises a data classification method based on security classification and timeliness and a fine-grained permission control mechanism using the minimum permission principle, and dynamic permission granting and recycling are supported. By balancing security and availability, secure sharing and effective utilization of medical data are realized, and diversified data access requirements are met.

Embodiments provide for schema matching between multiple disparate data sources using multi-dimensional matrices, external embeddings, transformation operations, and similarity metrics.

The invention relates to a database high-performance storage system for an electric power intelligent terminal, which is characterized by comprising a database function unit, a data cleaning unit based on a lightweight rule engine and an asymmetric data compression unit based on bit back coding, the database function unit comprises a data access engine and an OpenAPI interface; the data cleaning unit based on the lightweight rule engine comprises a data deduplication module, a rule configuration module, a rule loading module, a regularization verification module and a mode matching module adopting a mode matching algorithm mainly based on a regular expression. The asymmetric data compression unit based on bit-back coding introduces an asymmetric data compression technology based on bit-back coding to perform data compression on the basis of Huffman coding and arithmetic coding. According to the invention, the storage and access performance of the data is improved in an intelligent terminal environment with limited resources.

The application relates to a database mode matching method based on relationship perception and incremental learning, and belongs to the technical field of computer databases. First, the modal information of a target database and a source database is found. Then, feature extraction is performed on the sub-modal text information. After that, the feature extraction information is input into a Bayesian network for selection and screening. In this process, the quality of the features is also measured to ensure the accuracy and stability of the sub-modal classification. At the same time, neuron regularization and Bayesian incremental learning methods are introduced to fully exploit the continuous learning ability of the model. Finally, the obtained target database sub-modal and source database sub-modal are matched through a feature relationship perception mode to obtain a mode matching result. The application can effectively improve the query and mode matching processing efficiency and accuracy between different databases.

A high-throughput data transformation and upserting system for distributed computing environments, comprising: a data ingestion module configured to receive structured, semi-structured, and unstructured data from one or more distributed data sources, including databases, cloud platforms, enterprise systems, streaming infrastructures, and external computing devices; a transformation module configured to convert the received data into predefined processing formats through data mapping, normalization, filtering, aggregation, and schema matching operations performed across multiple distributed computing nodes; and an upsertion module configured to insert new records and update existing records in one or more distributed databases while maintaining synchronization and consistency of the records across distributed computing resources.a synchronization module configured to coordinate the management of distributed transactions, replication activities, and consistency operations between interconnected databases and compute nodes; a workload management module configured to distribute processing workloads across distributed compute resources to improve throughput efficiency, scalability, and resource utilization for data-intensive processing operations; and a monitoring module configured to detect duplicate records, synchronization conflicts, operational failures, processing delays, and data inconsistencies related to distributed transformation and upsertion activities.

This disclosure presents a method and system for automatically discovering sensitive data, comprising: monitoring a task queue to obtain sensitive data discovery tasks to be processed; parsing the sensitive data discovery tasks, dynamically adapting connection strategies using a database fingerprinting mechanism, and connecting to a target database; extracting configuration information, determining the target database table name and target field name, generating and executing an intelligent sampling strategy and data query SQL statement, scanning and obtaining data samples, performing sensitive information pattern matching and identification, and generating discovery results and a data evolution tracking system; writing the discovery results into a result table and generating a data lineage graph; updating the execution progress of the sensitive data discovery tasks, closing the database connection after execution, and updating the status of the sensitive data discovery tasks to complete. This method can improve the efficiency, accuracy, and traceability of sensitive data discovery.

The invention discloses a power equipment fault intelligent diagnosis method and system based on deep learning, and relates to the technical field of fault prediction and health management. An initial data set is constructed through multi-dimensional feature extraction, time sequence data smoothing processing and time-space neighborhood data analysis, and a feature change rate is combined to obtain a fault diagnosis result; and accurately identifying potential state transition candidate points of the equipment. Furthermore, a conversion type label is generated through noise filtering and historical path mode matching, the position of a key transfer node is determined by adopting a node position calibration technology, high-risk transfer nodes are marked in combination with fluctuation value gradual increase detection and risk accumulation judgment rules, and finally an early warning signal sequence of future state change is generated. According to the invention, through fusion of multi-level data processing and dynamic analysis logic, real-time monitoring and risk early warning of equipment state change are realized, a dynamic conversion identification mechanism is established, key nodes of state transition are automatically identified from continuous monitoring data, and the fault pre-judgment capability is improved.

The invention discloses a method and system for extracting target data in a database, and the method comprises the following steps: collecting information in the database, and constructing a database mode graph structure; constructing an enhanced GW-OT model, and initializing a basic cost matrix and a marginal distribution set; in the structure consistency propagation unit, performing consistency propagation calculation; in the cost matrix construction unit, an enhanced cost matrix is constructed; in the optimal transmission solving unit, optimal transmission optimization calculation is executed; in the candidate data generation unit, cross-table and cross-mode matching alignment is carried out; and in the target data extraction unit, performing condition filtering and discrimination, and outputting target data. According to the method, the accuracy and consistency of database target data extraction are improved.

The invention discloses a data monitoring method and system considering information security, and relates to the technical field of data monitoring. According to the data monitoring method and system considering the information security, multi-source monitoring data associated with the information security are obtained, and the multi-source monitoring data comprise user behavior data and environment data; performing multi-dimensional local detection on the multi-source monitoring data, and if local monitoring of any dimension is abnormal, acquiring multi-source fusion monitoring data and determining an abnormal risk level of each dimension; and inputting the multi-source fusion monitoring data into an abnormal mode matching model, determining a data abnormal mode, and through multi-dimensional local detection and global monitoring, realizing three-dimensional monitoring of user behaviors, a periodic mode and an environment state, timely discovering abnormality of each dimension, accurately positioning an overall risk mode through data fusion, and improving the user experience. The comprehensiveness, accuracy and timeliness of information security monitoring are improved, and the problems that in existing information security data monitoring, single-dimension detection is one-sided, and multi-source data association analysis is insufficient are solved.

Techniques for database management are described. A database management system may transmit a request for a data management system of a database to provide a set of metadata attributes for structured data within the database, and may receive a set of metadata attributes for the structured data within the database. The data management system may perform a pattern matching procedure to evaluate the set of metadata attributes for the structured data within the database against one or more patterns associated with a data type to determine one or more locations within the database that include structured data of the data type. Based on the pattern matching procedure, the data management system may output an indication that the one or more locations within the database include structured data of the data type.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can transform one or more text benchmarks into one or more first vector embeddings, retrieve a text output from the generative AI model in response to a user prompt, transform the text output into one or more second vector embeddings, measure one or more matching scores based on comparison of the one or more first vector embeddings with the one or more second vector embeddings, and determine reliability of the text output based on evaluation of the one or more matching scores. Related systems and software for implementing the method are also disclosed.

The invention discloses a threat intelligence detection method, device and equipment and a storage medium, according to the scheme, multiple types of current threat intelligence data are processed based on a Hyperscan high-performance regular expression to generate a preset threat intelligence database, multi-mode matching is supported, multiple regular expressions can be compiled at a time, and the threat intelligence detection efficiency is improved. The matching result can be obtained only through one-time matching during matching, the usability and the matching performance are greatly improved, the matching time is shortened, and then the efficiency of threat intelligence detection is improved; in addition, all types of current threat intelligence data are uniformly expressed and processed through a Hyperscan high-performance regular expression, and later maintenance and updating are facilitated; moreover, by continuously updating the threat intelligence database and the memory space, the accuracy of detecting new threat intelligence data can be improved.

A system (100) for schema-aware automatic transformation of heterogeneous data into data lakes, comprising (a) a schema detection and inference module configured to automatically extract and infer the structure and schema of incoming data from different formats and sources; b) a schema matching and alignment module configured to semantically map and align the derived source schema with a predefined target schema using contextual analysis; c) a transformation rules module configured to automatically create transformation rules to convert data to conform to the target schema; d) an execution and orchestration module configured to apply the transformation rules to data in batch or streaming mode; (e) a metadata and lineage management module configured to track, log, and manage schema versions, mappings, and data transformations; and f) a user interface and API layer configured to enable the configuration, visualisation and control of schema mappings and transformation processes.

The invention belongs to the technical field of network security, and discloses a key information infrastructure-oriented network security operation situation awareness method, which comprises the following steps of: acquiring multi-modal original data of a CII, generating multi-target semantic tagged data through multi-target semantic tag labeling, and storing the multi-target semantic tagged data into a database; constructing an asset-event-business index dynamic association graph based on business topology; fusing to obtain a comprehensive situation entropy, and judging a risk to generate a risk response trigger signal; searching a defense immune memory bank for mode matching, generating a candidate treatment action set, and obtaining Pareto optimal treatment actions and matched corresponding resource requirements through multi-objective optimization screening; matching a preset three-level resource pool to execute a processing action, and synchronously monitoring the four-dimensional efficiency data to obtain a processing result and four-dimensional monitoring data; and quantifying a disposal effect through four-dimensional evaluation, reversely optimizing a full-link rule, updating a defense immune memory bank, and reversely associating to a multi-target semantic tag labeling link to form a closed-loop iteration situation awareness link.

The invention provides a tunnel traffic flow big data mining method and system combined with a time series, and relates to the technical field of traffic flow data analys.The tunnel traffic flow big data mining method comprises the steps that firstly, a time series data set corresponding to tunnel traffic flow big data is obtained, and the time series data set comprises a plurality of traffic flow time series fragments collected under the continuous time dimension; constructing time sequence association chains based on a time sequence adjacent relation, performing feature filling on adjacent fragment time slots to obtain a time sequence association chain set, and performing time sequence evolution feature extraction processing on each time sequence association chain to obtain a time sequence evolution feature set; and calling a pre-trained time sequence mode mining model to carry out abnormal evolution mode matching processing on the time sequence evolution characteristic set, generating an abnormal mode mining result, finally generating a traffic flow regulation and control suggestion set based on the abnormal mode mining result, and analyzing and adjusting regulation and control contents in combination with an influence range, so that the regulation and control suggestions have more pertinence and operability. The efficiency and accuracy of tunnel traffic flow regulation and control can be effectively improved.

The invention discloses a semantic analysis multi-level cache mechanism oriented to a heterogeneous database in a cloud edge end environment, and aims to realize recognition of database operation and extraction of types and influence ranges of database operation statements through a mode matching technology so as to construct a unified database access model and realize accurate forwarding and execution of the heterogeneous database operation. Comprising an SQL statement recognition module, a multi-level cache queue, a cache item positioning index and a publishing and subscribing module. And the SQL statement identification module is used for receiving and analyzing metadata of the database and the SQL statements, determining a query type and managing cache update. And the multi-level cache queue dynamically adjusts cache items according to the access frequency and the access time, so that the validity and the accuracy of the cache are ensured. The cache item positioning index accelerates cache access through a hash table and a prefix tree, and the access efficiency is improved. And the publishing and subscribing module is used for maintaining the consistency of caches among different data centers and realizing synchronous updating of the caches through a publishing and subscribing mechanism. The resource utilization rate and the data access performance are improved.

The invention relates to the field of databases, and discloses a database data migration method, system and device, a storage medium and a program product. The database data migration method provided by the embodiment of the invention comprises the following steps: acquiring a data migration request; in response to the data migration request, to-be-migrated database cluster information is obtained, and the database cluster information comprises the cluster number of a to-be-migrated database and service rules of different partitions in each cluster; and selecting a corresponding data migration mode based on the database cluster information, and storing the to-be-migrated data in the main index database through polling distribution matched with the data migration mode, or a consistent hash algorithm, or an index field in the to-be-migrated data. According to the method provided by the embodiment of the invention, the data processing pressure in the data migration process is reduced, and faster and more stable data migration is realized.

Processing and transforming data using an artificial intelligence (AI) system is described. A machine-learning-based data profiling tool may be used on first data associated with a first data source and second data associated with a second data source to generate first data profile content identifying first data schema content and second data profile content identifying second data schema content. A schema matching operation may be performed to generate schema matching data identifying a match between a first schema and a second schema. A data transformation operation may be performed to generate transformed data based on converting at least one of the first data or the second data from a source data format to a unified data format.

The invention discloses a dynamic evolution monitoring system and method for an electric leakage fault of an overhead line, and the method comprises the steps: transmitting an instruction to a grounding wire controller through a wireless communication module according to the coordinate information of a potential electric leakage region, and obtaining a feedback signal of the on-off state of a grounding wire; for the abnormal mode set, grouping multi-point data by adopting a k-means clustering method, and determining a fault feature vector represented by a clustering center; if the matching degree of the fault feature vector and an electric leakage mode stored in a historical database is higher than a threshold value, judging that an electric leakage point is confirmed, and generating a positioning coordinate sequence; transmitting the positioning coordinate sequence to a central terminal through a wireless network to obtain an integrated line state map; and adjusting the scanning path of the unmanned aerial vehicle by adopting a real-time updating mechanism according to the line state map to obtain an optimized monitoring data cycle so as to continuously track fault evolution.

The application provides a medical data trusted circulation method and system based on multi-level permission dynamic management, which comprises the following steps: constructing a multi-mode matching index, performing feature extraction and vectorization on medical text data, structured data and image data; establishing a three-layer index architecture and an intelligent routing layer; applying a graph algorithm to construct a user-permission-data relationship graph, real-time assess the permission usage mode, and detect abnormal behaviors; combining with the situation factors such as medical emergency degree to dynamically adjust the abnormal judgment threshold, and grading processing and automatically collecting the evidence information of abnormal behaviors. The application also comprises a data classification method based on the secrecy level and time effectiveness, and a fine-grained permission control mechanism using the principle of least privilege, which supports dynamic permission granting and recycling. The application balances the security and availability, realizes the safe sharing and effective utilization of medical data, and meets the diversified data access requirements.