Data processing method and device, computer equipment and storage medium

By configuring a data processing system between a multi-source heterogeneous database and a downstream data consumer end, and converting heterogeneous log data into a homologous data format, the problems of insufficient complexity and real-time log data processing in the prior art are solved, and efficient and real-time data processing is achieved.

CN119690701APending Publication Date: 2025-03-25PEOPLE'S INSURANCE COMPANY OF CHINA
0 Cites 0 Cited by

Patent Information

Application Number
CN202411749654.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-29
Publication Date
2025-03-25

AI Technical Summary

Technical Problem

When processing log data of multi-source heterogeneous databases, the prior art needs to be parsed according to different log formats, resulting in increased processing complexity and reduced real-time performance, which cannot meet the needs of real-time computing.

Method used

By configuring a data processing system between a multi-source heterogeneous database and the downstream data consumer end, heterogeneous log data is obtained and converted into a homologous data format based on the sub-strategy format change, and stored in the message middleware to reduce downstream parsing operations.

Benefits of technology

It reduces the complexity and maintenance cost of data processing, improves the real-time and efficiency of data processing, and is suitable for data processing scenarios with multi-source heterogeneous data.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to a data processing method and device, computer equipment and a storage medium, and relates to the technical field of data processing. The method comprises the steps of obtaining heterogeneous log data of each multi-source heterogeneous database, performing strategy-based format change on the basis of heterogeneous data formats of various kinds of heterogeneous log data to obtain log data in a homologous data format, storing the log data in the homologous data format into first message middleware, and sending the first message middleware to the multi-source heterogeneous database; therefore, a downstream data consumption end can call the log data in the homologous data format; through the method, the method can be applied to a data processing scene of multi-source heterogeneous data, analysis operation required by a downstream data consumption end in the log data processing process is reduced, the processing complexity of data processing is reduced, the real-time performance of data processing is improved, and then the data processing efficiency is improved.
Need to check novelty before this filing date? Find Prior Art