Data processing method and device, computer equipment and storage medium

A data processing and streaming data technology, applied in the field of data processing, can solve the problems of real-time data processing requirements, limited computing resources, limited storage resources, etc.

Active Publication Date: 2019-06-25
ADVANCED NEW TECH CO LTD
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Many businesses relying on network and computer technology will generate a large amount of data flow in real time. Data processing in this new situation needs to face the following p

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device, computer equipment and storage medium
  • Data processing method and device, computer equipment and storage medium
  • Data processing method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present specification in combination with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only some of the embodiments of the present specification, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this specification belong to the protection scope of the embodiments in this specification.

[0030] The data processing method provided by the embodiment of this specification, such as figure 1 As shown, the method includes the following steps:

[0031] Step 101 , clustering multiple micro-clusters to obtain at least one cluster, and the micro-clusters are updated based on stream data acquired in real time.

[0032] In the embodiment of the present invention, streaming data is also called streaming data, which refers to the data at a certain moment of the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a data processing method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the clustering processing of aplurality of micro-clusters, carrying out the subsequent processing of the micro-clusters even if the data can only be accessed once during the acquisition, and carrying out the analysis and integration of the streaming data under the condition that the storage resource is limited. And the streaming clustering algorithm can meet the real-time requirement of data processing. Besides, the historicalclusters are stored in the embodiment of the invention, the newly generated clusters can be compared with the historical clusters, and if the comparison result meets the preset rule, the newly generated clusters can inherit the clustering identification of the historical clusters, so that the tracking of the clustering result is realized, and the method can be widely applied to a plurality of application fields such as hotspot event tracking and public opinion analysis.

Description

technical field [0001] The embodiments of this specification relate to the technical field of data processing, and in particular, to a data processing method, device, computer equipment, and storage medium. Background technique [0002] Many businesses relying on network and computer technology will generate a large amount of data flow in real time. Data processing in this new situation needs to face the following problems: 1. Storage resources are limited, and data can only be accessed once when it is acquired; 2. Computing resources are limited , there are requirements for real-time data processing. [0003] When processing and mining real-time data streams, how to use limited computing and storage resources to process and integrate real-time data streams is a challenge and difficulty that the industry is currently facing. Contents of the invention [0004] The embodiments of this specification provide a data processing method, device, computer equipment, and storage me...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/2455G06F16/28
Inventor 张晓彤温祖杰张家兴
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap