Unlock instant, AI-driven research and patent intelligence for your innovation.

Supports real-time stream computing and is oriented to big data database control systems and computer programs

A control system and real-time streaming technology, applied in database distribution/replication, database design/maintenance, comprehensive factory control, etc., can solve the problems of unable to write out the response, high pressure on the client, and too late to read the response package, so as to avoid data The effect of the transmission

Active Publication Date: 2022-05-24
江苏禾禾贯文网络科技有限公司
View PDF12 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (1) In the distributed system, the server receives the request slowly or does not receive the request at all.
[0005] (2) The server-side response is slow; the server-side network is busy or the server itself is under high pressure, resulting in the failure to write the response
[0006] (3) The serialization on the server side fails or the deserialization on the calling side fails; the client network is busy or the client itself is under high pressure, resulting in too late to read the response packet, resulting in a processing timeout

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Supports real-time stream computing and is oriented to big data database control systems and computer programs
  • Supports real-time stream computing and is oriented to big data database control systems and computer programs
  • Supports real-time stream computing and is oriented to big data database control systems and computer programs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0075] image 3 A functional module structure diagram of a big data-oriented database system supporting real-time stream computing provided by Example 1 of the present invention. like image 3 As shown, including data interface module, data preprocessing module, real-time stream computing module, data processing module, configuration center and Hbase database, among which:

[0076] Step 101 , the data interface module: read the configuration information of the configuration center, read the received data according to the configuration information, and send the data to the data preprocessing module.

[0077] (1) It includes a message middleware module, which can be ActiveMQ message middleware, receives processing requests from data sources, reads the configuration information of the configuration center, reads data according to the read policy in the configuration information, and transmits the read data to to the data preprocessing module.

[0078] Step 102: Data preprocess...

Embodiment 2

[0084] Figure 4 is the internal structure diagram of the real-time stream computing module in example 1, including a distributor module, a topology cluster, a Redis database, a monitoring module, and a data integration module, in which:

[0085] Step 201: The distributor divides the data stream into small data streams according to the configuration information read from the configuration center. The more divided data streams, the better. Excessive data stream segmentation will result in the formation of divided data streams and merging. huge stress. Therefore, the number of divided small data streams should be determined after considering the data processing speed and the data arrival speed. Generally speaking, the data processing speed of the entire data stream should be greater than or equal to the data update speed, so that the arriving data can be processed in a timely manner, thereby ensuring the real-time and accuracy of the results. Because the hardware conditions of...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of computer software, and discloses a large data-oriented database control system supporting real-time stream computing, a computer program, and a data interface module that reads received data and sends the data to the data preprocessing module; the data preprocessing module , read the data from the data interface module, and preprocess the data, and then send the preprocessed data to the real-time stream computing module; the real-time stream computing module, read the data from the data preprocessing module, and Perform real-time streaming computing, and pass the obtained result data to the data processing module; the data processing module reads the data from the real-time streaming computing module, performs final processing on the data, and then stores the processed data in the Hbase database. It makes up for the lack of Hbase in real-time processing, and also avoids the problem that users need to use the storm framework to write programs to meet the data processing requirements.

Description

technical field [0001] The invention belongs to the technical field of computer software, and in particular relates to a big data database control system and a computer program supporting real-time stream computing. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: With the development of technology, computer technology is fully integrated into life, and a large amount of data is being generated in the world all the time. Among them, a large part of data are semi-structured and unstructured, and their value will decrease over time, how to maximize the value of this massive semi-structured and unstructured data , is a big problem today. For a distributed system composed of massive nodes, node failure events are unavoidable due to natural or human factors. When performing stream computing in a distributed system, it is often due to the following reasons: 1. The server side receives the request slowly or does no...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/21G06F16/27
CPCY02P90/02
Inventor 唐聃袁炜舒红平于瀛
Owner 江苏禾禾贯文网络科技有限公司