Time sequence processing method and device

A processing method and timing technology, applied in the direction of electrical digital data processing, special data processing applications, structured data retrieval, etc., can solve the problems of queue timing delay, data inconsistency, throughput reduction, etc., to ensure integrity and Effects of accuracy, performance improvement, and redundancy reduction

Pending Publication Date: 2021-11-12
FENGHUO COMM SCI & TECH CO LTD
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In network big data, in order to accurately analyze data and ensure data consistency, it is necessary to collect data as a historical zipper table to record the entire change state of the data, and due to network volatility or other uncertain factors will cause data storage. The timing is messed up, causing data update problems
Due to the increasing amount of data, the use of timing queues will lead to reduced throughput; the processing chain is getting longer and longer, and queue timing will cause greater delay
[0004] The traditional timing processing method needs to ensure the timing in the whole link. The dependencies of each component are too strong. Any error in any link will lead to data inconsistency. Timing must be guaranteed, and only single-queue single-thread processing can be used.
[0005] In view of this, how to overcome the defects in the existing technology and solve the problem that the switch chips of different manufacturers cannot realize the cross-disk link aggregation function is a problem to be solved in this technical field

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Time sequence processing method and device
  • Time sequence processing method and device
  • Time sequence processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0056] like figure 1 As shown, an embodiment of the present invention provides a time sequence processing method, and the specific steps are as follows.

[0057] Step 100: establish a current table and a history table in the database;

[0058] Step 200: query records from the current table or cache according to the object ID of the data to be processed, determine the insertion position of the data to be processed according to whether there is a record, the redundant ID of the data to be processed and the update time of the record;

[0059] Step 300: Group the data recorded in the history table according to the object ID and sort according to the update time, and perform merge or update processing by judging whether the redundant IDs of the previous and previous records are consistent.

[0060] Before the above three steps, the embodiments of the present invention further unify data definitions based on various sources of data.

[0061] Specifically, in this preferred embodim...

Embodiment 2

[0100] Based on the time sequence processing method provided in Embodiment 1, Embodiment 2 provides a time sequence processing system corresponding to Embodiment 1, such as Figure 7 As shown, the system includes a data unified specification module, a data operator judgment module, a new and modified data processing module, a synchronous data processing module, a deleted data processing module, and a history table polling and correction processing module.

[0101] Specifically, the data unification specification module unifies data definitions based on various sources of data. For example, unify the data specification for the data to be processed, so that the format of the data to be processed satisfies the following rules: timestamp, which defines the data time of the data to be processed; data operator, which defines the meaning of state change of the data to be processed (the meaning of state change includes adding new , modify, delete, and synchronize); Object ID, which de...

Embodiment 3

[0109] On the basis of the timing processing method and system provided by the above-mentioned embodiment 1 to embodiment 2, the present invention also provides a timing processing device that can be used to implement the above method and system, such as Figure 8 Shown is a schematic diagram of the device architecture of the embodiment of the present invention. The time sequence processing device in this embodiment includes one or more processors 21 and a memory 22 . in, Figure 8 A processor 21 is taken as an example.

[0110] Processor 21 and memory 22 can be connected by bus or other means, Figure 8 Take connection via bus as an example.

[0111] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, such as the sequential processing methods in Embodiment 1 to Embodiment 2, system. The processor 21 executes various functional applications and data p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a time sequence processing method and device. The method mainly comprises the following steps: establishing a current table and a historical table in a database; querying a record from the current table or cache according to the object ID of to-be-processed data, and determining the insertion position of the to-be-processed data according to the existence of the record, the redundant ID of the to-be-processed data and the updating time of the record; and grouping the data recorded in the history table according to object IDs and sorting the data according to updating time, and performing merging or updating processing by judging whether the redundant IDs recorded before and after are consistent or not. According to the method, the processing chains can be mutually decoupled, the throughput is improved, the time delay is reduced, all components in mass data processing can be mutually independent, data can still be concurrently processed even if network delay fluctuates, and each record is stored highly reliably and accurately.

Description

【Technical field】 [0001] The invention relates to the field of big data network applications, in particular to a time sequence processing method and device. 【Background technique】 [0002] With the rapid development of the network, the amount of data to be processed is increasing, and the big data chain is getting longer and longer. The basic needs of processing these data include collection, cleaning, conversion, loading, and storage. [0003] In network big data, in order to accurately analyze data and ensure data consistency, it is necessary to collect data as a historical zipper table to record the entire change state of the data, and due to network volatility or other uncertain factors will cause data storage. The timing is messed up, causing data update issues. Due to the increasing amount of data, the use of timing queues will lead to reduced throughput; the processing chain is getting longer and longer, and the timing of queues will cause greater delay. [0004] Th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/215G06F16/23G06F16/2458G06F16/27
CPCG06F16/2282G06F16/2255G06F16/215G06F16/2322G06F16/2365G06F16/2474G06F16/27
Inventor 邓富玉
Owner FENGHUO COMM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap