Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

30 results about "Natural key" patented technology

A natural key (also known as business key) is a type of unique key in a database formed of attributes that exist and are used in the external world outside the database (i.e. in the business domain or domain of discourse). In the relational model of data, a natural key is a candidate key and is therefore a functional determinant for all attributes in a relation. A natural key is sometimes called domain key.

Data deduplication method and device

The invention discloses a data deduplication method and device. The method comprises the steps that service main keys of data to be processed are obtained, wherein the service main keys are fields representing data uniqueness according to service demands; the service main keys are converted into a unified preset format, and matching codes are generated; the generated matching codes are ranked according to a preset sequence so that verification codes can be generated; the verification codes after ranking are searched and compared with first verification codes arranged in front, and when the verification codes are identical with the first verification codes, distinguishing codes of the verification codes are marked as second distinguishing codes; the data marked as the second distinguishing codes in the verification codes are deleted. When multiple rows or multiple lines of data with the scale over a ten million level are processed, the data deduplication method is simple in configuration, convenient to use and high in operability, deduplication processing of multiple rows or multiple lines can be achieved at the same time, a great amount of processing time is saved, and efficiency of deuplication processing is improved.
Owner:龙信数据(北京)有限公司

Data information consistency processing method, system and device based on big data

The invention provides a data information consistency processing method, system and device based on big data. The data information consistency processing method comprises obtaining a business major key of at least one data sheet to be processed; converting the business major key to be in a unified standard format, and generating a verification code; determining the verification code data similarity by means of a Hamming distance algorithm; sequentially generating identification codes of the verification code data by means of a drawer principle algorithm; comparing the first identification code with each subsequent identification code, and marking a distinguishing code of the subsequent identification code as a second distinguishing code when the subsequent identification code is the same as the first identification code; and deleting the data the identification code of which has the second distinguishing code. When more than one hundred million data in multiple rows or multiple columns is processed, a lot of processing time is saved, and the data processing efficiency is improved.
Owner:BEIJING HONGMA MEDIA CULTURE DEV

Multi-thread environment-based task processing method and apparatus

The invention discloses a multi-thread environment-based task processing method. The method comprises the steps of reading a business main key preset for a started scheduling task when it is detectedthat a thread starts the scheduling task in a process of executing the scheduling task in a task list based on a multi-thread environment; putting the read business main key into an established lock table; and according to the read business main key, obtaining a task object of the scheduling task, and executing the scheduling task based on the task object by the thread. The invention furthermore provides a multi-thread environment-based task processing apparatus. The technical problem that the same scheduling task is repeatedly executed by multiple threads in the prior art is solved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Data stream processing method and device, readable storage medium and processor

The embodiment of the invention provides a data stream processing method and device, a readable storage medium and a processor, and belongs to the technical field of data processing. The method comprises the following steps: dividing a data stream into a plurality of parts of division data; for at least one piece of division data in the plurality of pieces of division data, configuring a service primary key of the data for each piece of division data; according to the service main key, dividing the same type of data in each piece of divided data into the same memory queue to form a plurality of memory queues; and performing parallel processing on the plurality of memory queues. According to the data stream processing method and device, the readable storage medium and the processor, the data stream can be rapidly processed while the consistency of data services is ensured.
Owner:KE COM (BEIJING) TECHNOLOGY CO LTD

Data access method and apparatus

The present invention provides a data access method and apparatus. The method comprises the following steps: receiving a data access request carrying a service primary key, wherein the service primarykey is used for identifying a service data object to be accessed by the data access request; the data access request comprises any one of a query request, an insertion request, a deletion request oran update request for the service data object; Determining a data sub-library corresponding to the service data object according to the service primary key; An operation instruction corresponding to the data access request is executed on the data sub-repository. The pressure of database running is dispersed and the performance of database is improved.
Owner:TAIKANG LIFE INSURANCE CO LTD +1

System and method for executing database operations on a database

A system (100) for executing database operations on a database (102), the system comprising at least one database (102) and at least one database interface (104) configured to receive an internal database command comprising an internal identifier and in response to said internal database command, initiate execution of a database operation on said database. The system (100) additionally comprises a request interface (106) configured to receive an external database command, the external database command comprising an external identifier, the system (100) also comprising a resolver (108). The request interface (106) is preferably a representational state transfer application programming interface (REST API) and is additionally configured to receive the external database command comprising a unique identifier comprising one or more natural keys, and transmit the external database identifier to the resolver (108), the resolver (108) being configured to translate the external identifier to the internal identifier and transmit the internal identifier to the request interface (106). The request interface (106) is further configured to transmit the received internal database command comprising the internal identifier to the database interface (104) so that the database interface (104) may initiate execution of the database operation.
Owner:COMPTEL CORP

Real-time data processing method and apparatus

Embodiments of the invention disclose a real-time data processing method and apparatus, which relates to the field of internet e-commerce and can improve abnormal data processing efficiency and reducesale loss caused by abnormal data. The method comprises the steps of importing received real-time stream data to STORM, and obtaining a business main key corresponding to the real-time stream data; when it is detected that the business main key does not exist in REDIS, storing the business main key in the REDIS, and when a two-layer detection switch is switched on, detecting whether the businessmain key exists in DB or not; when it is detected that the business main key does not exist in the DB, storing the business main key in the DB; and when it is detected that the business main key exists in any one of the REDIS and the DB, determining the received real-time stream data as the abnormal data. The method and the apparatus are suitable for duplicate removal of the real-time stream data.
Owner:SUNING COM CO LTD

Method for duplicate removal of service-free primary key data during database storage

The invention discloses a method for duplicate removal of service-free primary key data during database storage. The method comprises the following steps: the data conversion module splices the external service data imported into the database into a character string; the data hash operation module calculates the spliced and combined character strings by using a sha256 algorithm to obtain a byte array; the message digest conversion module converts the message digest in the byte array format into a character string H1; the message digest hash module performs hash operation on the character string H1 converted from the message digest again to obtain an integer value H2, and the duplicate removal processing module queries a database mainly by taking the values H1 and H2 obtained by two times of hash operation in the above steps as conditions. According to the method for duplicate removal of service-free primary key data during database storage, whether the data are equal or not can be judged by comparing only two fields according to the characteristic that the collision rate of a message digest algorithm result is extremely low, and the database index is effectively utilized to improvethe efficiency.
Owner:政采云有限公司

Kafka-based ordered processing method and system for same data during production and consumption

PendingCN111930533AEnsure sequential consumptionEnsure orderly consumptionInterprogram communicationMessage queueOracle database
The invention provides a kafka-based ordered processing method and system for same data during production and consumption. The method comprises a step of orderly pushing by a kafka production end anda step of orderly consuming by a kafka consumption end. The step of orderly pushing by the kafka production end comprises the following sub-steps: S1, submitting messages to an Oracle database by a plurality of application ends according to affairs, monitoring message changes of the Oracle database by an OGG synchronization program, and allocating different processes on the OGG synchronization program to process the monitored message queues according to the affairs; s2, rewriting the processing logic of the OGG to obtain a section partition number N corresponding to the service primary key; s3, rewriting the sent custom partition, and sending the N value calculated by the same service primary key to the corresponding Nth section partition; s4, the same OGG thread processes the same transaction, and data in one transaction is sent to a fixed section partition; the method has the beneficial effects that the same service main keys can be ensured to be pushed in order, the consumption endconsumes in order, and the data is ensured to be consumed in order.
Owner:银盛支付服务股份有限公司

Data medium station, data asset construction method and related device

The invention discloses a data medium station, a data asset construction method and a related device. The data middle table comprises a source pasting layer, a middle layer, a data asset layer and an application data layer, and the data asset layer is used for separating a service main key and an asset attribute of an asset entity when data assets are built and associating the service main key with the asset attribute. According to the embodiment of the invention, a new data asset construction mode is provided, and various performances of the data platform can be further improved.
Owner:华润置地控股有限公司

Method and device for merging various business index data in real time and computer equipment

The invention relates to a method and device for merging various business index data in real time, computer equipment and a storage medium. The method comprises the following steps of: storing a business index record updated in any external data table into a local data table corresponding to the external data table in real time; obtaining incremental business index records in all the local data tables regularly, enabling business primary key field information of all the incremental business index records to be subjected to combination and deduplication processing, generating an intermediate incremental data table, wherein the intermediate incremental data table comprises multiple pieces of updated business primary key field information; and obtaining corresponding business index field information from each local data table according to each piece of updated business primary key field information, and carrying out merging calculation on the business index field information obtained from each local data table to obtain composite business index data corresponding to each piece of updated business primary key field information. By adopting the method, various business index data can be merged in real time in a low-invasion and low-delay manner.
Owner:深圳市云网万店科技有限公司

Intelligent relation management and intelligent data fusion method based on business object

The invention discloses an intelligent relation management and intelligent data fusion method based on a business object, which comprises the following steps of: S1, abstracting the generality of a bottom layer physical table through an object center to form an object, and automatically generating a global object relation view based on a business main key; S2, realizing rapid relation flattening and forming a new analysis view angle through unified object center and global object relation management; S3, using a greedy algorithm to calculate and find an optimal combination of each step in a permutation and combination mode to obtain an optimal combination path; and S4, completing a fusion plan based on the optimal path, and submitting the fusion plan to a whole-course scheduling engine through an interface for scheduling execution. The method has the beneficial effects that connection can be more efficiently established in the digital world, so that more efficient collaboration can be realized, and quicker response and agile adjustment can be performed on services.
Owner:WHALE CLOUD TECH CO LTD

Dirty data identification method, device and system

The invention discloses a dirty data identification method, device and system, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: receiving or producing data, and distributing the data with the same service primary key to the same operator; in the same operator, when the operator processes the data, generating a label in a self-increasing mode, adding the label into the data, and the label into the data; and issuing the data added with the label to a consumer. According to the embodiment, the technical problem that dirty data cannot be accurately identified can be solved.
Owner:BEIJING WODONG TIANJUN INFORMATION TECH CO LTD

Service scheduling method and device

The invention discloses a service scheduling method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of receiving a service in a message queue, and performing service registration on the service in a database, the service registration indicating scheduling time of the service; performing bucket division according to the service primary key corresponding to the service, and determining a service bucket number corresponding to the service; and executing service scheduling through a host in the service cluster according to the service bucket number and the scheduling time. According to the embodiment, the scheduling tasks can be executed in parallel by multiple hosts, the method is wide in applicability, the guarantee and scheduling efficiency of service scheduling are improved, and the scheduling cost is reduced.
Owner:BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1

Out-of-order processing method and device for multi-association real-time data stream, medium and equipment

The invention provides an out-of-order processing method and device for a multi-association real-time data stream, a medium and equipment. The method comprises the steps: monitoring the log changes of a main table and / or an association table in a database in real time; when the log changes, writing the main table change record and the association table change record into corresponding topics of the message queue in parallel in a multi-partition mode; acquiring multi-partition data streams of the main table and the association table; loading a service primary key of the primary table to a cache region; obtaining a service foreign key, and judging whether a corresponding service primary key can be obtained from the cache region or not; if the service main key can be obtained, combining the multi-partition data streams of the main table and the association table into independent partition data streams, grouping the independent partition data streams according to specified fields, sorting the grouped data streams, consuming the grouped data streams, and executing service logic processing on the consumed data flow. According to the method, disorder in a high-concurrency multi-association data stream real-time scene can be solved.
Owner:北京亿欧网盟科技有限公司

A data exchange system and method based on cache mechanism

The present invention proposes a data exchange system and method based on a caching mechanism, wherein the system includes: an upstream business system, which is used to write a data message, the task type and the business primary key corresponding to the data message to the cache system, and write the business The state of the primary key is set to be processed; the cache system is used to save the data message and the corresponding task type and business primary key; and save the state of the task primary key; the data center is used to read from the cache system And save the data message corresponding to the business primary key in the pending state, assemble the data message into the data required by the downstream system, and synchronize the assembled data to the downstream system; it is also used to cache the business primary key in the cache system The state is changed to the processed state, and the data packet and the corresponding task type and business primary key stored in the cache system are deleted. The invention can set buffer space between the upstream business system and the downstream system, alleviates the high complexity of the business of the interface, and thus improves the exchange success rate.
Owner:BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1

Distributed service primary key generation method based on Mysql

The invention relates to a Mysql-based distributed service primary key generation method, which comprises the following steps of: firstly selecting the time of an updating period which is greater than the time of a current machine, then selecting a large average value, and finally caching the workId data of the current service to a local file. According to the Mysql-based distributed service primary key generation method, the influence on service performance is relatively small, multiple services and multiple scenes can be supported to acquire unique representation IDs at the same time, related isolation can be carried out based on service identifiers, ID generation rules among the services do not influence one another, and high availability and high performance of micro-service services are supported. The local service is independently generated and can be correspondingly decoupled with the Mysql, and even if the Mysql cluster breaks down, the local service can work independently and is not affected by the Mysql cluster.
Owner:UNICLOUD (NANJING) DIGITAL TECH CO LTD

Database splitting method and device

PendingCN113486023AImprove the efficiency of sub-tableImprove the efficiency of sub-database and sub-tableDatabase updatingRelational databasesOriginal dataData information
The invention discloses a database splitting method and device, which can be used in the technical field of big data, and the method comprises the following steps: receiving first data information; determining a self-increasing primary key corresponding to the service primary key of the first data information according to the service primary key of the first data information and a mapping relationship between the service primary key and the self-increasing primary key; according to a self-increasing primary key corresponding to a service primary key of the first data information and a mapping relationship between the self-increasing primary key and a sub-library and sub-table rule, determining a database position corresponding to the self-increasing primary key corresponding to the service primary key of the first data information; and storing the first data information into a database position corresponding to the self-increasing primary key corresponding to the service primary key of the first data information. When newly added data information needs to be stored into the table, the original data information does not need to be redistributed, the newly added data information is distributed to the corresponding database table in a self-increasing mode, the database table dividing efficiency can be improved, and the processing flow can be simplified.
Owner:BANK OF CHINA

Gateway interface analysis method and device, computer equipment and storage medium

The invention relates to the technical field of gateways, and provides a gateway interface analysis method and device, computer equipment and a storage medium, and the method comprises the steps: determining a corresponding configuration interface in a gateway according to an interface number and version corresponding to request information when a gateway interface receives the request information; obtaining a configuration attribute corresponding to the configuration interface, and calling a field analysis code and an interface verification code in the configuration attribute; verifying whether the field analysis code and the interface verification code have defects or not; if no defect exists, verifying whether the field analysis code and the interface verification code are changed or not; and if not, verifying the request information based on the interface verification code, analyzing the service main key of the request information based on the field analysis code, and performing associated display on the request information with the same service main key, wherein the request information can be stored in the blockchain. According to the method, the code security is guaranteed, and different requests of the same service are displayed in an associated manner.
Owner:ONE CONNECT SMART TECH CO LTD SHENZHEN

Request processing method and device, electronic equipment and computer readable medium

PendingCN114844947ASolve the problem of filtration efficiencyFix jitterTransmissionData reportingNatural key
The invention relates to a request processing method and device, electronic equipment and a computer readable medium, and belongs to the technical field of computer networks. The method comprises: in response to a network request for data reporting, obtaining request data corresponding to the network request, the request data comprising a service primary key and a reporting time; obtaining a pre-configured time unit length, and obtaining normalized time corresponding to the service primary key according to the time unit length, the service primary key and the reporting time; judging whether the network request passes or not according to the normalization time corresponding to the service primary key and cache data related to the service primary key in a cache; and when the judgment of the network request is passed, forwarding the network request to a service server for processing the request for processing. By calculating the normalization time corresponding to the service primary key, the problem of effective flow filtering efficiency under massive requests can be solved.
Owner:BEIJING DAJIA INTERNET INFORMATION TECH CO LTD

Data confidence coefficient calculation method and device, computer equipment and readable storage medium

The invention provides a data confidence coefficient calculation method and device, computer equipment and a readable storage medium, and relates to the technical field of big data processing. After the target business data set comprising the stored business data subset and the to-be-stored business data subset with the same business primary key type is obtained, the rule is constructed according to the preset content of the non-primary key data type corresponding to the target business data set; the initial confidence coefficient of each piece of business data in the target business data set is processed to obtain the temporary confidence coefficient of the target data content in each piece of business data, and the temporary confidence coefficient of the target data content in each piece of business data is acquired according to the total occurrence frequency of each piece of target primary key content in the target business data set; and the data confidence of each target data content is calcualted in the target business data set according to the target data content in the target business data set and the occurrence frequency and temporary confidence of all the target data contents in each business data subset, thereby reducing the manpower dependence during data confidence calculation, and improving the accuracy and reliability of the data confidence.
Owner:成都知道创宇信息技术有限公司

HIVE rule data matching method and device and storage medium

The invention provides an HIVE rule matching data method and device and a storage medium, and the method comprises the steps: obtaining a first rule matching table of HIVE and first matching data of HIVE; wherein the first rule matching table comprises a matching rule associated with the service primary key; wherein the first matching data comprises matching data corresponding to a service associated with the service primary key from the source layer; adding a service primary key and first matching data into the first rule matching table to obtain a second rule matching table; according to a dimension reduction matching strategy, determining data matched when rules of different dimensions in the matching rules are met as second matching data, and adding the second matching data into a second rule matching table; and based on the first matching data and the second matching data in the second rule matching table, determining a matching result obtained by the rule matching data.
Owner:WEBANK (CHINA)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products