34 results about "Schema matching" patented technology

A system for converting a source data feed schema into a canonical data product including a memory for storing computer-executable instructions and a processor for executing the instructions stored on the memory. Execution of the instructions programs the processor to perform operations that include receiving a source data feed having a source schema, identifying a plurality of data fields of the source schema, assigning a data category from a plurality of predefined data categories to each data field of the plurality of data fields, modifying the source schema based on predefined parameters, wherein the source schema is modified to match a target schema, comparing the modified source schema to the target schema, and in response to a determination that the modified source schema matches the target schema, converting the source data feed to a canonical data product having the target schema based on the assigned data categories.

The invention discloses an electric power system monitoring data analysis method, equipment and a medium. The method comprises the following steps: S1, acquiring a historical monitoring data set and a real-time monitoring data stream of an electric power system; s2, constructing a power equipment operation reference mode library based on the historical monitoring data set; s3, mapping the real-time monitoring data stream to the power equipment operation reference mode library for mode matching processing, and generating a matching degree sequence; s4, performing difference evolution analysis processing on the matching degree sequence to generate an abnormal evolution track; s5, determining starting equipment, a propagation path and an influence range of abnormity according to the abnormity evolution track, and generating an early warning instruction; according to the data analysis method, the accuracy and timeliness of monitoring of the power system are greatly improved on the whole, faults of the power system can be effectively prevented and reduced, the stability and reliability of power supply are guaranteed, and economic losses and social influences caused by the power faults are reduced.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can receive a schema of a source table from a first database and a schema of a plurality of target tables from a second database, identify one or more matching tables among the plurality of target tables based on comparison of the schema of the source table and the schema of the plurality of target tables using a large language model, obtain first sample attribute data from the source table and second sample attribute data from a selected matching table, and identify one or more pairs of matching attributes between the source table and the selected matching table based on comparison of the first sample attribute data and the second sample attribute data using the large language model. Related systems and software for implementing the method are also disclosed.

The invention relates to a dynamic adaptive Text2SQL (Structured Query Language) generation method and system for the power field. The method comprises the following steps: S1, analyzing a power database Schema and automatically generating a diversified natural language-SQL pair data set; s2, performing cleaning, optimization and format standardization on the data, and constructing a high-quality training set and a high-quality test set; s3, querying a prediction database Schema based on a natural language, and generating an SQL statement with accurate pattern matching; s4, a logic disassembly optimization mechanism is introduced, and it is ensured that SQL query conforms to database rules and meets the requirements of the power industry; s5, finely adjusting the large language model to improve the accuracy and the performability of SQL generation; s6, performing compression optimization on the trained model to improve reasoning performance, stability and query accuracy; and S7, efficiently deploying the optimized electric power large language model, and guaranteeing the stability, response speed and model performance of the system through real-time monitoring. According to the method, the accuracy and applicability of converting the natural language query into the SQL statement in the power industry can be effectively improved.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can transform one or more text benchmarks into one or more first vector embeddings, retrieve a text output from the generative AI model in response to a user prompt, transform the text output into one or more second vector embeddings, measure one or more matching scores based on comparison of the one or more first vector embeddings with the one or more second vector embeddings, and determine reliability of the text output based on evaluation of the one or more matching scores. Related systems and software for implementing the method are also disclosed.

The invention provides an LLM-based engineering project scheme compliance examination method. The method comprises the following steps: Step 1, data preparation; implementing multi-dimensional data acquisition through a distributed web crawler system; step 2, processing various types of data of images, texts and tables by using a large language model (LLM) technology; step 3, performing Prompt fine tuning optimization on the data, so that the large language model can better understand and generate related contents of compliance review of the engineering project scheme; step 4, performing automatic compliance check based on rule and mode matching; by defining a series of rules and modes, automatic compliance review of project documents is realized; and Step 5, establishing a risk generation assessment and early warning mechanism based on the large model. Through automatic compliance inspection, the workload of manual auditing is reduced, and document specifications are ensured; and an intelligent risk assessment and early warning mechanism identifies and notifies potential risks in real time, so that compliance risks are effectively reduced, and the safety and the overall level of project management are improved.

Techniques for database management are described. A database management system may transmit a request for a data management system of a database to provide a set of metadata attributes for structured data within the database, and may receive a set of metadata attributes for the structured data within the database. The data management system may perform a pattern matching procedure to evaluate the set of metadata attributes for the structured data within the database against one or more patterns associated with a data type to determine one or more locations within the database that include structured data of the data type. Based on the pattern matching procedure, the data management system may output an indication that the one or more locations within the database include structured data of the data type.

The invention provides a medical data credible circulation method and system based on multi-level authority dynamic management, and the method comprises the steps: constructing a multi-mode matching index, and carrying out the feature extraction and vectorization of medical text data, structured data and image data; establishing a three-layer index architecture and an intelligent routing layer; constructing a user-permission-data relation graph by applying a graph algorithm, evaluating a permission use mode in real time, and detecting an abnormal behavior; the abnormal judgment threshold is dynamically adjusted in combination with situation factors such as medical emergency degree, abnormal behaviors are subjected to grading processing, and evidence obtaining information is automatically collected. The invention further comprises a data classification method based on security classification and timeliness and a fine-grained permission control mechanism using the minimum permission principle, and dynamic permission granting and recycling are supported. By balancing security and availability, secure sharing and effective utilization of medical data are realized, and diversified data access requirements are met.

A high-throughput data transformation and upserting system for distributed computing environments, comprising: a data ingestion module configured to receive structured, semi-structured, and unstructured data from one or more distributed data sources, including databases, cloud platforms, enterprise systems, streaming infrastructures, and external computing devices; a transformation module configured to convert the received data into predefined processing formats through data mapping, normalization, filtering, aggregation, and schema matching operations performed across multiple distributed computing nodes; and an upsertion module configured to insert new records and update existing records in one or more distributed databases while maintaining synchronization and consistency of the records across distributed computing resources.a synchronization module configured to coordinate the management of distributed transactions, replication activities, and consistency operations between interconnected databases and compute nodes; a workload management module configured to distribute processing workloads across distributed compute resources to improve throughput efficiency, scalability, and resource utilization for data-intensive processing operations; and a monitoring module configured to detect duplicate records, synchronization conflicts, operational failures, processing delays, and data inconsistencies related to distributed transformation and upsertion activities.

This disclosure presents a method and system for automatically discovering sensitive data, comprising: monitoring a task queue to obtain sensitive data discovery tasks to be processed; parsing the sensitive data discovery tasks, dynamically adapting connection strategies using a database fingerprinting mechanism, and connecting to a target database; extracting configuration information, determining the target database table name and target field name, generating and executing an intelligent sampling strategy and data query SQL statement, scanning and obtaining data samples, performing sensitive information pattern matching and identification, and generating discovery results and a data evolution tracking system; writing the discovery results into a result table and generating a data lineage graph; updating the execution progress of the sensitive data discovery tasks, closing the database connection after execution, and updating the status of the sensitive data discovery tasks to complete. This method can improve the efficiency, accuracy, and traceability of sensitive data discovery.

The invention discloses a power equipment fault intelligent diagnosis method and system based on deep learning, and relates to the technical field of fault prediction and health management. An initial data set is constructed through multi-dimensional feature extraction, time sequence data smoothing processing and time-space neighborhood data analysis, and a feature change rate is combined to obtain a fault diagnosis result; and accurately identifying potential state transition candidate points of the equipment. Furthermore, a conversion type label is generated through noise filtering and historical path mode matching, the position of a key transfer node is determined by adopting a node position calibration technology, high-risk transfer nodes are marked in combination with fluctuation value gradual increase detection and risk accumulation judgment rules, and finally an early warning signal sequence of future state change is generated. According to the invention, through fusion of multi-level data processing and dynamic analysis logic, real-time monitoring and risk early warning of equipment state change are realized, a dynamic conversion identification mechanism is established, key nodes of state transition are automatically identified from continuous monitoring data, and the fault pre-judgment capability is improved.

The invention discloses a method and system for extracting target data in a database, and the method comprises the following steps: collecting information in the database, and constructing a database mode graph structure; constructing an enhanced GW-OT model, and initializing a basic cost matrix and a marginal distribution set; in the structure consistency propagation unit, performing consistency propagation calculation; in the cost matrix construction unit, an enhanced cost matrix is constructed; in the optimal transmission solving unit, optimal transmission optimization calculation is executed; in the candidate data generation unit, cross-table and cross-mode matching alignment is carried out; and in the target data extraction unit, performing condition filtering and discrimination, and outputting target data. According to the method, the accuracy and consistency of database target data extraction are improved.

A computer-implemented method for improved schema matching of two databases is disclosed. The method can transform one or more text benchmarks into one or more first vector embeddings, retrieve a text output from the generative AI model in response to a user prompt, transform the text output into one or more second vector embeddings, measure one or more matching scores based on comparison of the one or more first vector embeddings with the one or more second vector embeddings, and determine reliability of the text output based on evaluation of the one or more matching scores. Related systems and software for implementing the method are also disclosed.

The invention belongs to the technical field of network security, and discloses a key information infrastructure-oriented network security operation situation awareness method, which comprises the following steps of: acquiring multi-modal original data of a CII, generating multi-target semantic tagged data through multi-target semantic tag labeling, and storing the multi-target semantic tagged data into a database; constructing an asset-event-business index dynamic association graph based on business topology; fusing to obtain a comprehensive situation entropy, and judging a risk to generate a risk response trigger signal; searching a defense immune memory bank for mode matching, generating a candidate treatment action set, and obtaining Pareto optimal treatment actions and matched corresponding resource requirements through multi-objective optimization screening; matching a preset three-level resource pool to execute a processing action, and synchronously monitoring the four-dimensional efficiency data to obtain a processing result and four-dimensional monitoring data; and quantifying a disposal effect through four-dimensional evaluation, reversely optimizing a full-link rule, updating a defense immune memory bank, and reversely associating to a multi-target semantic tag labeling link to form a closed-loop iteration situation awareness link.

The invention provides a tunnel traffic flow big data mining method and system combined with a time series, and relates to the technical field of traffic flow data analys.The tunnel traffic flow big data mining method comprises the steps that firstly, a time series data set corresponding to tunnel traffic flow big data is obtained, and the time series data set comprises a plurality of traffic flow time series fragments collected under the continuous time dimension; constructing time sequence association chains based on a time sequence adjacent relation, performing feature filling on adjacent fragment time slots to obtain a time sequence association chain set, and performing time sequence evolution feature extraction processing on each time sequence association chain to obtain a time sequence evolution feature set; and calling a pre-trained time sequence mode mining model to carry out abnormal evolution mode matching processing on the time sequence evolution characteristic set, generating an abnormal mode mining result, finally generating a traffic flow regulation and control suggestion set based on the abnormal mode mining result, and analyzing and adjusting regulation and control contents in combination with an influence range, so that the regulation and control suggestions have more pertinence and operability. The efficiency and accuracy of tunnel traffic flow regulation and control can be effectively improved.

The invention relates to the field of databases, and discloses a database data migration method, system and device, a storage medium and a program product. The database data migration method provided by the embodiment of the invention comprises the following steps: acquiring a data migration request; in response to the data migration request, to-be-migrated database cluster information is obtained, and the database cluster information comprises the cluster number of a to-be-migrated database and service rules of different partitions in each cluster; and selecting a corresponding data migration mode based on the database cluster information, and storing the to-be-migrated data in the main index database through polling distribution matched with the data migration mode, or a consistent hash algorithm, or an index field in the to-be-migrated data. According to the method provided by the embodiment of the invention, the data processing pressure in the data migration process is reduced, and faster and more stable data migration is realized.

Processing and transforming data using an artificial intelligence (AI) system is described. A machine-learning-based data profiling tool may be used on first data associated with a first data source and second data associated with a second data source to generate first data profile content identifying first data schema content and second data profile content identifying second data schema content. A schema matching operation may be performed to generate schema matching data identifying a match between a first schema and a second schema. A data transformation operation may be performed to generate transformed data based on converting at least one of the first data or the second data from a source data format to a unified data format.

The invention discloses a dynamic evolution monitoring system and method for an electric leakage fault of an overhead line, and the method comprises the steps: transmitting an instruction to a grounding wire controller through a wireless communication module according to the coordinate information of a potential electric leakage region, and obtaining a feedback signal of the on-off state of a grounding wire; for the abnormal mode set, grouping multi-point data by adopting a k-means clustering method, and determining a fault feature vector represented by a clustering center; if the matching degree of the fault feature vector and an electric leakage mode stored in a historical database is higher than a threshold value, judging that an electric leakage point is confirmed, and generating a positioning coordinate sequence; transmitting the positioning coordinate sequence to a central terminal through a wireless network to obtain an integrated line state map; and adjusting the scanning path of the unmanned aerial vehicle by adopting a real-time updating mechanism according to the line state map to obtain an optimized monitoring data cycle so as to continuously track fault evolution.

The application provides a medical data trusted circulation method and system based on multi-level permission dynamic management, which comprises the following steps: constructing a multi-mode matching index, performing feature extraction and vectorization on medical text data, structured data and image data; establishing a three-layer index architecture and an intelligent routing layer; applying a graph algorithm to construct a user-permission-data relationship graph, real-time assess the permission usage mode, and detect abnormal behaviors; combining with the situation factors such as medical emergency degree to dynamically adjust the abnormal judgment threshold, and grading processing and automatically collecting the evidence information of abnormal behaviors. The application also comprises a data classification method based on the secrecy level and time effectiveness, and a fine-grained permission control mechanism using the principle of least privilege, which supports dynamic permission granting and recycling. The application balances the security and availability, realizes the safe sharing and effective utilization of medical data, and meets the diversified data access requirements.

A method can include: (i) obtaining a first data record structured according to a first schema, (ii) determining, for the first schema, one or more first schema attribute values for each schema attribute in a predefined set of schema attributes, (iii) determining, for a second schema, one or more second schema attribute values for each schema attribute in the predefined set of schema attributes, (iv) providing the first and second schema attribute values to a schema matching engine, wherein the schema matching engine contains schema mapping techniques and rules, wherein each rule recommends a schema mapping technique based on schema attributes from the predefined set of schema attributes, (v) applying the rules to select a schema mapping technique, (vi) transforming the first data record according to the selected schema mapping technique, and (vii) providing the transformed first data record in a data structure according to the second schema.

The application discloses a kind of dynamic form automatic mapping methods and systems for multi-modal data.The method comprises: obtaining multi-modal original data and target dynamic form unit information;The multi-modal original data is preprocessed to obtain structured intermediate data;For form field dynamically generating structured prompt word template;Structured intermediate data and prompt word template are input into large language model, and structured data segment containing field value, confidence and source position is obtained;The output JSON Schema matched with form structure is constructed, and the data segment is organized as standard JSON data package;After the field value in JSON data package is matched by semantics, type checking and logical constraint, it is automatically filled into the corresponding control of form, and the automatic mapping of multi-modal data to dynamic form is completed.By the present application, multi-modal original data can be automatically mapped, filled and configured to any dynamic form with high precision, which greatly improves data entry efficiency and reduces error rate.

The application provides a 10KV power distribution network fault automatic diagnosis positioning method, constructs a confidence evaluation model by collecting current direction and voltage polarity data, and carries out cross verification combined with a historical fault feature database to obtain a preliminary and adjusted confidence sequence; in view of the interference monitoring point, the weight of the interference monitoring point is reduced, a weighted polarity sequence is generated, and mode matching is carried out to determine a preliminary fault section; a multi-dimensional verification matrix is constructed by fusing the power distribution automation switch action and the user repair geographic position, the spatial deviation is quantified, and when the threshold is exceeded, deep analysis is started, the deviation sequence is corrected by means of the consistency filtering of the power distribution network topological connection relationship and the business standby path, finally, the information fusion technology is adopted to integrate the multi-source results to output the accurate fault position, and the positioning result is fed back to update the fault feature database and the confidence model parameters, adaptive optimization diagnosis is realized, and the diagnosis precision is improved.

The application discloses a kind of Internet of Things network fault monitoring systems, including the following modules: hierarchical acquisition module, data preprocessing module, adaptive threshold module, abnormal analysis module, fault location module, hierarchical response module, closed-loop optimization module, the application also discloses a kind of Internet of Things network fault monitoring method, including the following steps: hierarchical data acquisition, data preprocessing and feature screening, adaptive threshold generation, abnormal clustering and fault preliminary judgment, fault mode matching and accurate positioning, hierarchical response and closed-loop optimization.In the application, the threshold is dynamically adjusted by combining historical data with real-time network load, and the threshold validity is verified with the device type weight correction, which effectively improves the threshold adaptability and fault determination accuracy, and then through K-means clustering, the abnormal classification, structured fault mode library and weighted matching algorithm are cooperatively determined, and the location identification information is accurately positioned, which greatly improves the accuracy of abnormal analysis and fault location.

The invention provides a handle key tactile feedback control system and method based on deep learning, and relates to the technical field of deep learning model application, and the method comprises the steps: a tactile feedback mode matching module is used for receiving button operation data from a handle, and combining a host scene type to obtain a tactile feedback mode matching module; inputting a preset tactile feedback matching database for tactile feedback configuration to obtain a selected tactile feedback mode; the expected mode generation module is used for retrieving a historical handle motion sample set to obtain an expected tactile feedback mode when the number of the selected tactile feedback modes is equal to 0 and the host scene type does not belong to a tactile feedback taboo type; and the optimization execution module is used for calling the tactile feedback mode analysis network bound with the handle, performing control optimization on the handle motor array, and executing handle key tactile feedback control. The technical problems that in the prior art, key tactile feedback is poor in flexibility, a tactile feedback mode needs to be set for each scene, and unset scenes cannot be responded are solved.

The invention discloses an intelligent pushing method and system for electric power communication first-level and second-level optical transmission network alarms. The method comprises the following steps: acquiring alarm original data from a plurality of local network management systems; preprocessing the data to obtain standardized data; storing the data into a data middle table through a synchronization mechanism; dynamically filtering and screening through a business rule engine to obtain target data and storing the target data in an intermediate library; and polling the intermediate library through an intelligent push engine, performing mode matching and intelligent grading, and generating and pushing alarm content to a responsible unit. The system comprises corresponding function modules. According to the invention, by constructing an intelligent grading and pushing system, the problems of high redundancy and low pushing efficiency of the existing alarm information are effectively solved, and the monitoring efficiency and fault response speed of the power communication optical transmission network are remarkably improved.

The present application belongs to the field of network security, and in particular to a method and device for monitoring data dark web leakage. The device for monitoring data dark web leakage comprises a dark web resource scheduling and collection module, a data preprocessing and analysis module, a data analysis and correlation matching module, an alarm and report generation module. The dark web resource scheduling and collection module is used to realize the management of dark web resources, anonymous access and adaptive crawling, including: a dark web resource list maintenance unit: maintaining an updatable dark web resource list containing.onion site URLs, covering the entrances of online markets, forums and telegram groups for illegal data transactions. Through anonymous adaptive dark web collection, multi-modal data analysis, enterprise asset fingerprint multi-modal matching and machine learning risk assessment, the present application realizes the full-process automation of dark web data leakage monitoring, significantly improves the monitoring efficiency and accuracy, can quickly trace the leakage event and push real-time alarms, and helps the security team to prevent data misuse risks in advance.

This invention discloses a method for security system linkage and intrusion behavior recognition that integrates multimodal data, specifically relating to the field of intelligent security technology. The method includes: First, performing three-level spatiotemporal alignment and semantic-level collaborative embedding fusion on multi-source data to generate unified fusion features; second, constructing and reasoning a dynamic semantic scene graph, identifying high-order intrusion behaviors through graph pattern matching and logical rules, and outputting structured behavioral events; then, based on behavioral events and multi-dimensional context, performing multi-objective optimization through a dynamic policy engine to generate and execute collaborative linkage strategies, and dynamically replanning based on feedback; finally, achieving continuous self-evolution and cross-scene knowledge transfer through multi-scale memory and metacognitive optimization mechanisms. This invention solves the problems of traditional systems such as single perception, rigid linkage, and inability to adapt, achieving highly accurate, interpretable, and self-evolving intelligent security.

A client for a federated computing system is configured to store one or more input data records, extract input data schemas therefrom and transfer them to a server. Using the process pipeline received from the server, an output data record is generated from one or more of the input data records whose input data schema matches a process data schema of the process pipeline and transferred to the server. A server for a federated computing system is configured to receive and register input data schemas from one or more clients, obtain a process pipeline based on a process data schema from a user, and transfer it to one or more of the clients whose registered input data schemas match the process data schema. Output data records are then received from one or more clients, aggregated and provided to the user. Computer-implemented methods and computer program products are also provided.

The invention discloses a method and a system for constructing a risk assessment engine by using a key threshold rule information technology, a storage medium and electronic equipment. The method comprises the following steps: completing cargo information synchronization in database copying and event queues and generating parallel processing events; executing a mode matching engine and an address verification tool in the memory database to realize data verification; free text processing is completed in the data analysis engine and the real-time analysis engine, and a comprehensive evaluation result is output. According to the method, data real-time synchronization of a service system and a risk assessment engine is achieved through database replication and an event queue, a compensation mechanism is automatically triggered when replication fails or delays, it is ensured that a core risk event can be preferentially processed in a high-concurrency environment in combination with a priority queue and flow control logic, and the risk assessment efficiency is improved. Therefore, the real-time performance and the reliability of data transmission are improved.