10 results about "Natural key" patented technology

This application discloses a data processing method, apparatus, device, and computer-readable storage medium. The method includes the steps of: acquiring collected data and generating a business primary key for the collected data; the business primary key having resource uniqueness; comparing the differences between the collected data and existing data based on the business primary key to obtain a comparison result; and processing the existing data accordingly based on the comparison result. This application enables accurate comparison of collected data and existing data through the business primary key, thereby regaining the ability to manage collected data and preventing collected data from slipping out of control.

The application discloses a multi-type property maintenance work order automatic generation and management method and system, and the method comprises the following steps: triggering a timing task according to the execution time of a maintenance plan, and judging whether the maintenance plan is overdue; when the maintenance plan is not overdue, creating a work order record and writing the work order record into a business data table, and starting a work flow instance with a business primary key in the same database transaction; the work flow instance triggers an execution listener in a node flow process, updates a work order processing state, and executes corresponding business logic according to a node identifier; a timing job record is written in a user task node, a work order timeout mark is updated and a timeout alarm is sent when a timeout deadline arrives, so that the work order automatic generation, process linkage and timeout control are realized.

This application provides a method, apparatus, device, and storage medium for verifying data table consistency. The method includes: acquiring first business primary key information and a first identifier corresponding to multiple first data tables in design states, and second business primary key information and a second identifier corresponding to multiple second data tables in production states; comparing whether the first business primary key information and the second business primary key information are consistent; for first data tables and second data tables with consistent business primary key information, comparing whether the first identifier and the second identifier are consistent; for first data tables and second data tables with consistent identifiers, comparing whether the fields of the first data table and the fields of the second data table are consistent; and determining the verification result of data table consistency based on the primary key information comparison result, the identifier comparison result, and the field comparison result. The verification method according to this application embodiment can automatically verify whether the structures of physical model tables and database tables are consistent and obtain relatively accurate verification results.

The embodiment of the invention provides a data quality inspection method based on dynamic parameter transmission and related equipment, and belongs to the technical field of data management. The method comprises the steps that a quality inspection scheme is newly established, the quality inspection scheme comprises a quality inspection scheme type and a data source, and the quality inspection scheme type comprises a table conventional quality inspection type and a consistency quality inspection type; based on the quality inspection scheme type and the data source, performing quality inspection rule configuration on the quality inspection scheme to obtain a quality inspection scheme after rule configuration, the quality inspection scheme after rule configuration comprising a quality inspection rule; configuring a result list and a service main key of the quality inspection scheme; configuring a scheduling period; and executing the quality inspection scheme based on the scheduling cycle, and generating a quality inspection result based on the quality inspection scheme type, the quality inspection rule, the result detail list and the service main key. According to the embodiment of the invention, the quality inspection scheme can be automatically and periodically executed, the working efficiency and the data quality management efficiency can be improved, the accuracy and effectiveness of the data are improved, and a powerful basis is provided for subsequent data recovery.

The invention provides an asynchronous task processing method based on persistent queue and business rule conflict detection, and the method specifically comprises the steps: receiving an e-commerce data flow, and analyzing the e-commerce data flow into an atomic data line object; extracting a to-be-operated resource dimension in the object, and forming a resource locking identifier set consisting of a service main key and an association key; establishing a conflict dependency relationship between write operations, and constructing a directed acyclic dependency graph according to the conflict dependency relationship; the objects are decomposed into a plurality of batches which are executed in sequence, and it is ensured that resource identification sets of all the objects in the same batch are not intersected; for each object, reading the current data version number of the target write-in position, and generating a write-in instruction containing an expected version; verifying the consistency of the real-time version and the expected version by using a database CAS instruction; if the real-time version and the expected version are consistent, finishing persistence; if not, the concurrency version conflict is judged; at the moment, a conflict avoiding mechanism is triggered, rollback time delay is calculated through an exponential back-off algorithm, and conflict objects are re-injected into the dependent construction or execution process.

The application relates to the technical field of big data processing, and discloses an internet operation and sales data analysis method and system based on big data, which comprises the following steps: acquiring multi-source business data records and extracting business primary keys; based on the business primary keys, the multi-source business data records are collected into corresponding logical containers; the logical containers are subjected to integrity evaluation to determine whether the logical containers meet preset business logic integrity requirements; only when the business logic integrity requirements are met, is the associated analysis of the collected data in the logical containers triggered, and corresponding operation analysis conclusions are generated; when the logical containers receive new business data records, the operation analysis conclusions are corrected according to the new business data records. The application constructs an event-driven analysis unit through the business primary keys and the logical containers, solves the pseudo-association and missing-association problems caused by the inconsistent arrival times of multi-source data, and realizes accurate and dynamically-updatable operation insight.

A batch process operates on structured data in multiple tables, including a first and a second source table, and a first and a second processing table. An additional key is added within each natural key to at least the second source and processing table for identifying a sequence of rows for updating.

The invention discloses an incremental data detection method and device, equipment and a storage medium, and relates to the technical field of incremental data detection.The incremental data detection method comprises the steps that a service primary key and a data fingerprint are generated for each data record in a heterogeneous data source; constructing a data snapshot according to the business primary key of each data record and the corresponding data fingerprint; comparing the data snapshot with a stored historical data snapshot to identify a data change record; and outputting an incremental data result set containing the data change record. According to the method, the universality of incremental data detection can be improved.

The invention discloses a data updating method and device, computer equipment, a storage medium and a program product, and belongs to the technical field of databases. The method comprises the steps that new data are written into a first index, a random document identifier is generated for a document where each piece of new data is located, and the new data comprise timestamp fields recording data writing time; querying all the data in the first index, grouping all the data, sorting the data in each group according to timestamp fields of all the data, and screening the data in each group to obtain screened data; and performing an initialization operation on the first index every a first time period. According to the method, data are screened through timestamp sorting and business primary key grouping, and the problems of data updating efficiency and accuracy of Elasticsearch in an online synchronous service scene are solved in combination with periodic index initialization.

The invention provides a service primary key generation method and device, and relates to the technical field of data processing. The method comprises the following steps: receiving a service primary key request sent by an application system; obtaining primary key configuration information from a memory database according to the primary key ID, and generating a new primary key value according to the primary key configuration information and a preset primary key generation mode; and taking the new primary key value as a current primary key, returning the current primary key to the application system, and storing the current primary key to the memory database so as to synchronize the current primary key to the relational database by calling a data synchronization service. The device executes the method. According to the service primary key generation method and device provided by the embodiment of the invention, the service primary key of the self-increasing sequence can be automatically generated in a distributed environment, and the service attributes are expressed through the service primary key, so that the service requirements are fully met.