Supercharge Your Innovation With Domain-Expert AI Agents!

Unbounded flow data association processing method and device, equipment and storage medium

A processing method and technology of streaming data, applied in the computer field, can solve problems such as poor data processing effect, data loss, inability to obtain associated data, etc., to achieve the effect of improving accuracy and avoiding data flow delay

Pending Publication Date: 2021-12-07
BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In the process of realizing the present invention, the inventor found that there are at least the following technical problems in the prior art: when any data stream is delayed or the network is jittered, it is impossible to obtain complete associated data, resulting in data loss and poor data processing effect

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Unbounded flow data association processing method and device, equipment and storage medium
  • Unbounded flow data association processing method and device, equipment and storage medium
  • Unbounded flow data association processing method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0029] figure 1 It is a flow chart of a method for associating unbounded stream data provided by Embodiment 1 of the present invention. This embodiment is applicable to the situation of associating unbounded stream data. The method can be executed by an unbounded stream data association processing device, which can be realized by software and / or hardware, for example, the unbounded stream data association processing device can be configured in a computer device. Such as figure 1 As shown, the method includes:

[0030] S110. Acquire a main data stream and associated data streams associated with the main data stream.

[0031] In this embodiment, the main data stream and the associated data stream are data streams used for association analysis. Both the main data flow and the associated data flow can be determined according to data processing requirements. Taking the e-commerce platform as an example, if the data processing requirement is the analysis of the success characte...

Embodiment 2

[0049] This embodiment provides a preferred embodiment on the basis of the above solutions. In this embodiment, taking the associated data stream as the storage data stream as an example, the method for associating unbounded stream data is described.

[0050] Figure 2a It is a schematic diagram of a method for associating unbounded stream data provided by Embodiment 2 of the present invention. Such as Figure 2a As shown, when the main data stream 1 and the associated data stream 2 are associated in real time, they first undergo data preprocessing. A persistent storage unit and an active cache layer are added to the processing flow of the associated data stream 2. The persistent storage unit stores data in a persistent storage device, such as hbase. The active cache layer can be a local memory cache (such as guava cache), which puts change-sensitive data, such as constantly changing and updated data, into the cache, and continuously actively updates the cache with the ori...

Embodiment 3

[0067] image 3 It is a schematic structural diagram of an unbounded stream data association processing device provided by Embodiment 3 of the present invention. The device for associated processing of unbounded stream data may be implemented in the form of software and / or hardware, for example, the device for associated processing of unbounded stream data may be configured in a computer device. Such as image 3 As shown, the device includes a data flow acquisition module 310, an acquisition path determination module 320, a data acquisition module 330 and a data processing module 340, wherein:

[0068] A data stream acquiring module 310, configured to acquire a main data stream and associated data streams associated with the main data stream;

[0069] The acquisition path determination module 320 is configured to determine the main data acquisition path of the main data flow, and the associated data acquisition path of the associated data flow. The main data acquisition path...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses an unbounded flow data association processing method and device, equipment and a storage medium. The method comprises the steps of obtaining a main data flow and an associated data flow associated with the main data flow; determining a main data acquisition path of the main data flow and an associated data acquisition path of the associated data flow, the main data acquisition path or the associated data acquisition path comprises a local cache, and the local cache is used for storing service data with the change frequency larger than a set frequency threshold value; acquiring target master data based on the master data acquisition path, and acquiring target associated data based on the associated data acquisition path; and performing association processing on the target main data and the target association data to obtain an association processing result. According to the method provided by the embodiment of the invention, the association accuracy of the rapidly changed data and the data sensitive to the state change is ensured, the data loss during data stream delay or network jitter is avoided, and the accuracy of data processing is improved.

Description

technical field [0001] Embodiments of the present invention relate to the field of computer technology, and in particular, relate to a method, device, device, and storage medium for associating processing of unbounded stream data. Background technique [0002] In stream processing applications, there is a relationship between data streams. In order to realize the correlation analysis of interrelated data streams, the common processing method is to use the stream processing window to collect and receive data streams related to each other. The data received by the window performs data analysis between associated streams. [0003] In the process of realizing the present invention, the inventors found that there are at least the following technical problems in the prior art: when any data stream is delayed or the network is jittered, it is impossible to obtain complete associated data, resulting in data loss and poor data processing effect. Contents of the invention [0004] ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2455G06F16/25G06Q40/04
CPCG06F16/254G06F16/24552G06F16/22G06Q40/04
Inventor 安金龙
Owner BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com