140 results about "Foreign key" patented technology

The invention relates to a data security processing method and system based on distributed storage, and relates to the technical field of computer information processing. The method comprises the following steps: cutting data into encryption fragments with a configurable number by adopting a dynamic fragmentation strategy, and generating a physically isolated dynamic check block in combination with a timestamp to realize tampering prevention; a dynamic threshold value is dynamically calculated based on the data sensitivity index and the node load, and the node is optimized through the reliability score for cooperative decryption; a database table is divided into independent marshalling storage according to main foreign key association, foreign key fields are encrypted by adopting cross keys, and cross-marshalling access needs to meet a multi-key threshold condition; an intelligent threat perception engine is constructed, access logs and threat intelligence are analyzed in real time, and key rotation, fragment replacement and joint defense response mechanisms are dynamically triggered. According to the invention, full life cycle protection of data is realized, and the problems of key leakage risk and cross-table association attack are effectively solved.

The invention discloses a multi-table combined natural language query SQL (structured query language) generation method, which comprises the following steps of: 1, constructing a database meta-knowledge graph, and establishing a triple storage comprising a table structure, a primary and foreign key relationship and business description for each data table; 2, receiving a natural language query request, and calculating the topic relevancy between query semantics and each database table through a pre-trained topic matching model; 3, dynamically constructing a view, and logically associating the database tables of which the theme relevancy exceeds a threshold value to form a temporary view; 4, generating a context enhancement prompt, and combining the temporary view structure, the field semantic description of the view and the view content sample to form a structured prompt; 5, inputting the natural language query and the structured prompt into the large language model to generate candidate SQL statements; and step 6, executing verification and iterative optimization on the candidate SQL statements, verifying logic correctness through SQL execution plan analysis and result sampling, and triggering and prompting a reconstruction mechanism when detection is abnormal.

The invention provides a business modeling method, medium and system based on an enterprise data asset management platform, and belongs to the technical field of modeling methods.The business modeling method based on the enterprise data asset management platform comprises the following steps that an enterprise data asset catalog is read to generate a data asset list, the method comprises the steps of extracting a foreign key association relationship between data tables to construct an initial entity relationship table, calculating reference frequencies between the data tables to generate an entity association degree matrix, dividing a service domain by adopting a K-means clustering algorithm, counting data table fields in the service domain to generate a data table field mapping table, recording a data table operation sequence to construct a data table operation link diagram, and establishing a data table operation link diagram. The method comprises the following steps: generating a data table node evaluation matrix, dividing sub-domains and determining a core entity, and constructing and training a multilayer neural network model for constructing business affiliation, data table incidence relation and business interface definition of a new data table, so as to solve the defects in the aspects of data asset relation discovery, business modeling automation, service interface generation and the like in the prior art.

Apparatuses, methods, and computer-readable media infer a graph model from a semantic model that describes source tables and relationships among the source tables. The apparatus identifies at least one primary key for each relationship of a plurality of relationships defined in the semantic model based on whether the relationship is a m:1 relationship or a 1:1 relationship, each relationship defining at least one from-table and one to-table. The apparatus creates a vertex table in the graph model for each source table having a primary key. The apparatus creates an edge table in the graph model for each from-table defined in a m:1 relationship of the semantic model and creates an edge table in the graph model for one of the from-table or the to-table in a 1:1 relationship of the semantic model. Each edge of the edge table is labeled with a source or destination vertex table and a foreign-key.

The invention relates to the technical field of databases, in particular to a method for converting a relational database into a graph database. Comprising the following steps: S1, connecting a relational database by adopting an application framework, and migrating to generate a model file; s2, establishing a graph database, obtaining table names, foreign keys and indexes of the relational database according to the model file, and respectively generating points, edges and indexes corresponding to the graph database; s3, generating insertion statements of points and edges of the graph database, executing insertion operation, and migrating data in the relational database into the graph database; and S4, performing data verification by comparing the metadata, the data volume and the data query result of the relational database and the graph database. The method can adapt to various mainstream relational databases, the difficulty of indiscriminately converting the multi-source relational database into the graph database is realized, and the conversion process comprises points, edges and indexes of the graph database.

The invention discloses an Internet self-research operation and maintenance management method and device and a storage medium, and the method comprises the steps: analyzing a current operation and maintenance management demand, and determining and selecting at least one target instance according to an analysis result; extracting an associated foreign key of the target instance, and dynamically associating the target instance based on the associated foreign key to form a target entity set; service configuration information based on the target entity set is obtained, and dynamic key value pair configuration of the target entity set is updated according to the service configuration information; and generating a task flow arrangement template based on the updated target entity set, and scheduling a task object to execute a task chain through the task flow arrangement template. According to the method, a multi-level configuration management framework and a template task chain mechanism are established, environment self-adaptive differential execution is supported after multi-level configuration parameters are automatically combined, the operation and maintenance efficiency and the full-link execution monitoring and auditing tracing capability are improved, and the risk of manual operation errors is reduced.

The invention provides a database relationship intelligent discovery and ER graph construction method and a self-evolution learning method, and belongs to the technical field of ER graph construction, and the database relationship intelligent discovery and ER graph construction method comprises the steps of S1, metadata collection and preprocessing; s2, vectorizing domain sensing metadata; s3, recalling a multi-strategy candidate relationship; s4, performing LLM structured reasoning and verification; s5, carrying out data sampling inspection and cross validation; s6, hierarchical negative sample self-evolution learning; and S7, generating and visualizing an ER graph. According to the invention, a layered negative sample self-evolution learning mechanism and the structured reasoning ability of LLM are organically combined, and LoRA rapid fine tuning and a multi-stage verification process are supplemented. The defects of accuracy, generalization ability, cold start efficiency, reasoning transparency, automation integration and the like in the aspect of discovery of the primary and foreign key relationship of the relational database in the prior art are effectively overcome, and a more advanced and practical solution is provided.

The invention belongs to the technical field of database query and natural language processing, and provides an NL2SQL method based on foreign key completion and a knowledge graph, and the method comprises the steps: extracting a candidate foreign key relationship from database metadata and data content, and trimming and completing the candidate foreign key relationship through a three-layer funnel verification mechanism, a Schema knowledge graph is constructed; based on the natural language query, determining a seed table set directly related to the query intention; on the basis of the Schema knowledge graph, calculating a connection path between nodes of a seed table, complementing an intermediate table on the path, and expanding the seed table into a sub-graph set containing a complete connection relationship; generating a plurality of candidate SQL statements based on the sub-chart set; and executing the plurality of candidate SQL statements, and voting according to the consistency of execution results to select a final SQL statement.

The invention relates to the technical field of data processing, and particularly discloses a data processing method and system based on data security management, and the method comprises the steps: S1, obtaining a table name, a primary key value, a field value and a foreign key value in a database, and building an association line among the table name, the primary key value, the field value and the foreign key value; representing the elements by nodes to form a data association tree; s2, some nodes are preset as sensitive nodes, and the longest path from the sensitive nodes to the bottom end of the tree is found; s3, for each piece of data in the database, calculating the shortest path length from a node corresponding to the data to a sensitive node, and calculating a sensitive association index according to a preset coefficient; classifying the data into different security levels according to the sensitive association index; the invention aims to identify and manage the data security in the relational database by constructing the data association tree.

A machine learning based data structuring method for automating dimensional data modelling process in data repositories is disclosed. The ML-based data structuring method includes obtaining datasets from databases; classifying the data comprising attributes and measures based on historical data using a ML-based classifier model; determining associations between the datasets using primary and foreign keys, SQL logs, usage of the datasets in creating transformations, and performance of fuzzy string match; assigning weightages to the determined associations between the datasets based on utilization of the determined associations using weighted network graphs; validating the determined associations between the datasets based on recurrent utilization of the determined associations; clustering the datasets based on the validated associations between the datasets by detecting dimensional models in the weighted network graphs; and generating actionable insights on each of the clustered datasets by performing exploratory data analysis, influencer analytics, and forecasting of the data.

The invention provides a test data generation method. The method comprises the following steps: acquiring field types of business fields in a plurality of business data tables of a specified business database; obtaining business data associated with each business field according to a data desensitization strategy matched with the field type of each business field; generating a structured file according to each business field and business data; obtaining a primary and foreign key relationship among business fields in the plurality of business data tables; and according to the structured file and the main and foreign key relationship, generating test data for testing each service field, thereby realizing accurate identification and processing of sensitive fields, avoiding information leakage, and ensuring that the generated test data not only conforms to the field type and service scene of the actual service field, but also conforms to the service scene of the actual service field. The logic correctness of the generated test data is ensured, and the effectiveness and accuracy of the test data are improved; meanwhile, manual operation is not needed in the test data generation process, and the labor cost and the test data generation time are reduced.

The invention discloses a prediction method, device and equipment and a readable storage medium, and the method comprises the steps: obtaining a relational database containing a to-be-predicted object in response to a prediction request, taking a business object in an entity table in the relational database as an entity node, taking a business event in an event table as an event node, and carrying out the prediction of the to-be-predicted object. And determining edges between the entity nodes and the event nodes according to foreign keys corresponding to the business events in the event table, and constructing a target topological graph according to the entity nodes, the event nodes and the edges between the entity nodes and the event nodes. Therefore, the target features of the to-be-predicted object are automatically extracted from the whole target topological graph, original data in all tables in the relational database are fully utilized, human intervention of feature engineering is reduced, the prediction efficiency is improved, the complex relation between the to-be-predicted object and other data in the relational database can be better captured, and the prediction accuracy is improved. And the accuracy of model prediction is improved.

The invention provides a database selection rate determination method and device, a medium and a product. The method for determining the selection rate of the database comprises the steps of obtaining a query statement, wherein query conditions of the query statement comprise connection conditions of a foreign key reference table and a foreign key referenced table; searching and storing a reference column filtering condition about a reference column of the foreign key referenced table in the query statement; obtaining an initial foreign key connection selection rate for the foreign key reference table and the foreign key referenced table, wherein the initial foreign key connection selection rate is one fraction of the total row number of the foreign key referenced table; and correcting the initial foreign key connection selection rate according to a reference column filtering condition to obtain a finally used foreign key connection selection rate. Therefore, according to different filtering conditions, the method can be suitable for a more appropriate foreign key connection selection rate, so that the cardinality estimation accuracy of the connection query of the foreign key reference table and the foreign key referenced table is improved.

A method and one or more non-transitory storage media for materialized view-based query rewrite are provided. A plurality of materialized views is created based on a fact table and a dimension table. Each materialized view is based on a join between the dimension table and the fact table based on a respective foreign key column of the fact table. A database management system executes a query against the fact table and the dimension table, the query requiring one or more joins between the dimension table and the fact table based on one or more foreign key columns. For each given join of the one or more joins, responsive to the given join satisfying one or more rewrite criteria, the query is rewritten to replace the join between the dimension table and the fact table with a join between a respective materialized view for the given join and the fact table.

The invention discloses a business domain division method and device for a database table, a medium and a product, and relates to the technical field of data processing. According to the scheme, when the business domain grouping requirement of the database table exists, from the three aspects of the structure, the semantics and the behavior of the database table, the similarity degree of three dimensions including the foreign key relation, the name similarity and the common current mode between the database tables is comprehensively considered; the problem that effective information in metadata is difficult to extract due to the fact that only an NLP model is used for unilaterally extracting database table information in a traditional method is effectively avoided; and finally, determining a corresponding comprehensive weight by utilizing the structure weight, the semantic weight and the behavior weight corresponding to each database table, namely dividing each database table into a plurality of business domain groups according to each comprehensive weight, so that manual intervention can be effectively reduced, and the method is suitable for a large-scale database scene.

Techniques for discovering primary, unique, and / or foreign keys for relational datasets are described. The techniques include profiling the relational datasets to obtain respective data profiles; identifying one or more primary key candidates for a first relational dataset using a first data profile of the first relational dataset and a first trained machine learning model; identifying one or more foreign key proposals for a second relational dataset using the one or more primary key candidates by performing a subset analysis of the second relational dataset with respect to the first relational dataset; identifying one or more foreign key candidates for the second relational dataset using the first data profile, a second data profile of the second relational dataset, and a second trained machine learning model different from the first trained machine learning model; and outputting the at primary key candidate(s) and the foreign key candidate(s).

The invention discloses a cross-platform power grid data conversion synchronization method and device, a terminal and a medium. According to the method, a master table in a source database is synchronized to a target database according to master-slave table dependency sequence information; in the synchronization process, an ID mapping table is constructed to record the mapping relation between a first primary key ID and a second primary key ID of each piece of primary table data, the second primary key ID corresponding to a foreign key field is dynamically calculated to be used for updating the foreign key field based on the foreign key field of the slave table in combination with foreign key mapping relation information and the ID mapping table, and the secondary table data is synchronized. Finally, after all the foreign key fields of the slave table are updated, the data synchronization action of the slave table is executed in combination with ORM data synchronization logic, dynamic remapping of the primary key ID is achieved by constructing the ID mapping table, the foreign key fields of the slave table are updated based on the foreign key mapping relation information, the foreign key reference integrity during cross-database synchronization is ensured, and the synchronization efficiency is improved. The problem of foreign key reference integrity caused by inconsistent primary key IDs in the cross-database synchronization process is effectively solved.

A machine-readable medium stores instructions that, when executed by a processor, cause the processor to execute operations. The operations include receiving information identifying a database table and preprocessing the table. The preprocessing includes extracting primary and foreign key relationships and generating sample data by combining frequency-based sampling of common values with random sampling. The operations also include generating, using an LLM, a unified metadata schema. The generating includes processing the table and the sample data using domain-specific prompts and outputting the unified metadata schema. The operations include embedding and indexing the unified metadata schema and converting a natural language question output from an end-user device into a SQL query. The converting includes identifying relevant metadata in the unified metadata schema matching query intent and generating the SQL query using the filtered metadata. The operations also include executing the SQL query and providing query results to the end-user device.

The invention discloses a data warehouse model design method, and belongs to the technical field of data warehouse modeling. According to the method, a dimension modeling method is adopted to construct and analyze a mart model, data is organized into a non-standardized relation model, and the relation between tables is defined through keywords and foreign keys. And designing a dimension table system comprising a product dimension, a calendar day dimension, an institution dimension, an insurance policy attribute dimension and a customer dimension, and a fact table system comprising an insurance claim settlement analysis fact table, a payment analysis fact table, a claim settlement efficiency analysis fact table and a corresponding intermediate fact table. The analysis subject domain data is formatted into a star model or snowflake model, where the analysis fact table is a lightly summarized transactional fact table, and the intermediate fact table is a versioned historical table. According to the method, repeated data construction is avoided through a unified modeling method, and the cost is reduced; an efficient query analysis function is provided by optimizing a model structure; and the data traceability is ensured through versioning management.

The application belongs to the technical field of data processing, and discloses a kind of information system business data integrity check method, comprising: the data entity class instance of information system is encoded based on the data structure of data entity class, and initial encoding is obtained;Data entity class instance is encoded again to obtain current time encoding;According to the initial encoding and current time encoding of data entity class check and output data integrity check result.The method can obtain understandable and directly related fine-grained integrity check result by participating in the encoding and encoding comparison process of business entity class, and the integrity of all information including the external information associated with foreign key can be checked out by directly checking the integrity of class instance used in business system, saving storage space and speeding up comparison speed.

The invention discloses a real-time analysis method for mobile customer purchase intention grading, which comprises the steps of collecting a customer in-store data set and an online data set, carrying out abnormal value processing on the data, establishing an association relationship between the data in a database by using a primary key and a foreign key through a unique identifier of a customer, associating the online data with the offline data, and carrying out real-time analysis on the online data and the offline data. A data matching technology is combined to identify and associate similar data records, correlation analysis is utilized, most representative features are selected from a data set, a cross validation technology is combined to construct and train a model, customer purchase intention data variables are continuously collected, an association relationship is established in a database, and the customer purchase intention data is acquired. According to the method, the mobile customer purchase intention data can be effectively and accurately analyzed in real time, and the customer purchase intention data can be fed back to the sales personnel in time.

The invention provides a data consanguinity analysis method and equipment, and is applied to the field of data governance. The method comprises the steps of obtaining a data dictionary corresponding to a target database; wherein the target database comprises a plurality of data tables; foreign key information is extracted from the target database, the foreign key information is used for representing the relation among the multiple data tables, and the foreign key information is obtained; performing data correlation analysis on the target database based on a correlation coefficient algorithm to obtain a correlation analysis result; and based on the data dictionary, the foreign key information and the correlation analysis result, constructing a data blood relationship map. As the data correlation analysis is performed on the target database based on the correlation coefficient algorithm, not only can the correlation between the data fields be accurately quantified, but also the correlation analysis result for revealing the potential data dependency relationship between the fields and the business logic can be obtained, so that the data correlation analysis efficiency is improved. The data blood relationship map constructed based on the data dictionary, the foreign key information and the correlation analysis result can reveal potential data dependency and business logic, and the accuracy is high.

A database model established for a low-code platform is characterized by comprising an engineering basic information table, a component definition table, a component type table, an engineering component association table, an engineering variable table, an engineering event registry, a data source registry, a data set registry and a data set field definition table, the model designed by the invention adopts a standardized structure, so that data storage and management are more orderly, subsequent data maintenance and query are facilitated, and the problems of data redundancy and inconsistency are reduced. Through the clearly defined table relation and foreign key constraint, the user can more effectively manage the relation between the data, the integrity and consistency of the data are ensured, and the efficiency of data operation is improved. The method can adapt to various business scenes and requirements, a user can easily adjust and expand the data model according to different business logics, and the expandability and adaptability of the system are enhanced. According to the invention, through reasonable database design, the capability of data-driven decision making is improved.

The application discloses a test data generation method and device, equipment and a storage medium. The test data generation method comprises the following steps: obtaining a plurality of target record contents from a test data rule table, each target record content comprising a field type, data source information, test data generation rule information corresponding to the data source, and a record number; for each target record content, generating a test data table based on the data source information, the test data generation rule information, the record number, and a first table generated based on the field type; and finally, combining a plurality of test data tables based on the association relationship between the physical primary keys and the physical foreign keys of the plurality of database tables to obtain test data. According to the embodiment of the application, the test data is generated, the test data is rich in dimensions, the association relationship between the data is complete, and conforms to the real law.

The invention discloses an event inducement discovery system and method fusing deep reasoning and a graph neural network, and relates to the field of event analysis and processing. The system comprises: a knowledge base construction module, which is based on a news event database, constructs an event knowledge base through operations such as text partitioning, vector embedding, database foreign key association and the like, and is used for retrieval enhancement generation; the deep reasoning module is used for autonomously reasoning and collecting potential effective information of event inducements through operations such as retrieval statement generation, knowledge base retrieval, effective information extraction and sufficiency test by taking a large reasoning model as a base; the graph neural network module is based on the event relation graph, and assists in reasoning to obtain potential causal relations in the interlaced complex event text; the inducement mining module is used for mining event inducements from related materials by fully utilizing the reasoning capability of a large reasoning model through a professional cue word template; the invention further provides a method. The problems that event relations are complex and interlaced, and event inducements are difficult to analyze are solved.

The application discloses a low-code data interface calling method, equipment and medium, and the method comprises the following steps: confirming a target entity and an entity parameter corresponding to the target entity; the entity parameter comprises at least one of an entity name, a field name, a field type, a field position, whether to be filled in, an index and a foreign key; a database table corresponding to the target entity is constructed according to the entity parameter; the interface name, the interface parameter set and the interface parameter format of the target interface are defined according to the database table; the target interface of the target entity is generated according to the interface name, the interface parameter set, the interface parameter format and the entity parameter; the target interface is published, and an authorization key of the target interface is generated; it is determined that a target interface calling request from a calling party is received, and authorization signature information contained in the target interface calling request is acquired; the calling party is authorized and verified according to the authorization signature information and the authorization key, and if the verification is passed, the calling party is allowed to call the target interface.

The present invention discloses a method and system for converting a relational model into a graph model based on an LLM model. The method includes the following steps: converting the relational model to be converted into an initial graph model based on the primary-foreign key mapping; iteratively optimizing the initial graph model based on the query requirements and various preset optimization strategies to obtain a target graph model. Among them, the method for optimizing the model based on the target optimization strategy is: enabling the LLM model to analyze the corresponding candidate optimization objects based on the corresponding first graph model, the query requirements, and the target optimization strategy, and generating corresponding analysis results based on the queried features corresponding to the optimizable objects; determining the corresponding target optimization objects based on the target optimization strategy and the analysis results, and performing optimization transformation based on the target optimization objects. The present invention can automatically complete the conversion from a relational model to a graph model and optimize the model according to query requirements, effectively improving query efficiency and storage utilization rate.

In distributed data management traditional databases often struggle to maintain consistency and handle conflicts. Traditional solutions prioritize specific combinations of properties, resulting in trade-offs that impact user experience and application functionality. Present disclosure provides systems and methods that route incoming transaction associated with application to corresponding database server (DBS) by integrating geographical proximity mean value, and sender internet protocol affinity value. The incoming transaction is processed to extract data manipulation language (DML) operation written by the application and replicated to each DBS other than a DBS on which the application is directly applying a data through the incoming transactions. A primary key is generated by allocating an unique identifier if the DML operation is inserted on identified DBS. The primary key of record is updated from the incoming transaction. A conflict associated with parent and child association of the DML operation is dynamically monitored based on integrity Foreign key Constraint.

The invention provides a real-time wide table construction method, system and device based on the combination of Spark and Flink and a medium, and the method comprises the steps: S1, constructing a data link based on a Spark connector, and completing the rapid generation of a data set snapshot; s2, constructing a data link based on a FlinkCDC connector, and realizing incremental real-time updating of data; s3, stopping the Spark data synchronization program, setting the Flink CDC to consume increment change data in real time from the latest snapshot position generated in S1, and dividing a table needing to be associated into a master table and a plurality of slave tables; when the master table is changed, rechecking the related slave tables according to the association keys, combining to form wide table data, and writing the wide table data into a downstream wide table database in an upsert mode; and when the slave table changes, the corresponding data in the downstream wide table is directly updated according to the association key or the foreign key. Through the mechanism, changes of all the association tables can be fed back to the downstream wide table in real time, so that dynamic construction of the wide table is completed.

The invention provides a method and a system for establishing a relationship between data tables, and belongs to the technical field of big data. Comprising the following steps: S1, foreign key constraint: explaining how the foreign key constraint maintains a relationship between tables, including cascade operation and reference integrity, and discussing an implementation mode of the foreign key constraint in a database, such as use and optimization of an index; s2, a connection table explains the effect of the connection table in processing a many-to-many relationship, including the structure and use of an intermediate table, and analyzes the influence of the connection table on query performance and an optimization strategy; and S3, self-association: how self-association supports storage and query of tree structure data is described, and advantages and disadvantages of self-association in database design are discussed. According to the system for establishing the data table relation, the foreign key constraint of the RDBMS is taken as a core, logical association is realized through the primary key, the foreign key and the relation type definition, and the non-relational database simulates the relation through a reference or embedded document, so that the data consistency and the query efficiency are ensured.