Unlock instant, AI-driven research and patent intelligence for your innovation.

Data real-time storage method and device, computer equipment and storage medium

A data and real-time technology, applied in the field of data processing, can solve problems affecting log data storage and analysis, delay, etc.

Pending Publication Date: 2020-07-10
CHINA PING AN LIFE INSURANCE CO LTD
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Based on this, it is necessary to address the above-mentioned technical problems. This application provides a real-time data storage method, device, computer equipment and storage medium to solve the problem of directly writing the read log data into hive for storage in the prior art. Technical problems affecting the storage and analysis of log data caused by serious delays

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data real-time storage method and device, computer equipment and storage medium
  • Data real-time storage method and device, computer equipment and storage medium
  • Data real-time storage method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the application; the terms used herein in the description of the application are only to describe specific embodiments The purpose is not to limit the present application; the terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the description of the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

[0027] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrenc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the field of data structured processing. The invention discloses a data real-time storage method and device, computer equipment and a storage medium, and the method comprisesthe steps: replacing a specified character searched from a to-be-determined text sequence with a preset cut character, and cutting the to-be-determined text sequence at the preset cut character. The invention discloses a method and device, computer equipment and a readable storage medium. The method comprises the steps of performing classification processing on collected log data; writing the logdata meeting the log screening condition after classification processing into kafka for buffer processing; performing segmentation processing on the log data read from kafka to obtain a character string sequence, matching a key value text sequence for the segmented character string sequence with unclear meaning, and storing the data into structured data in real time through hbase to obtain the structured data. According to the method, the technical problem that a user cannot analyze the log data in real time is solved.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a method, device, computer equipment and storage medium for real-time data storage. Background technique [0002] In the prior art, if you want to analyze the user access logs produced by the server, you can use the Hadoop-based data warehouse tool Hive to write the log data on the server into a pre-created database table in Hive in real time, and then analyze the database Perform operations such as query and analysis on the data in the table to realize the function of real-time analysis of log data. However, when the amount of required log data is relatively large, the massive data on the server is directly written into Hive, because Hive's own defects will cause data congestion, and there is a delay in writing data to Hive, and the delay is generally long, which affects the log. Data storage and analysis. Contents of the invention [0003] Based on this, it is neces...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458G06F16/28G06F16/25
CPCG06F16/2471G06F16/283G06F16/258
Inventor 饶鑫黄望石晓龙
Owner CHINA PING AN LIFE INSURANCE CO LTD