Data transmission method and device, electronic equipment and storage medium

Abstract

The invention relates to a data processing technology, and discloses a data transmission method, which comprises the following steps of: performing batch processing on a data set to be transmitted, and uploading the batched data to preset message middleware for queue generation processing to obtain a standard message queue; grouping and packaging the standard message queues, and issuing the packaged standard message queues to a plurality of servers for idle instance operation to obtain a standard data set; writing the standard data set into a preset file, and uploading the file to a file storage server; and performing data verification processing on the file in the file storage server, and when the data verification is passed, sending the data in the file to the target database. In addition, the invention also relates to a block chain technology, and the standard message queue can be stored in a node of a block chain. The invention further provides a data transmission device, electronic equipment and a computer readable storage medium. According to the invention, the problem of low stability of data transmission can be solved.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a data transmission method, device, electronic equipment and computer-readable storage medium. Background technique [0002] In program development, it is often required to complete calculations on a large amount of data in a short period of time, and transfer the complete data to another independent system library within a limited time for daily workers to use in the offline system. For example, the user interest fee data of the post-loan component in the microservice cluster credit project system is very large, as many as one million, and is increasing day by day. Therefore, calculations based on interest fee data are very frequent, and a large number of calculation results obtained need to be transmitted to the collection system. [0003] Now in the industry, most of the data is transmitted separately and the calculation is completed, which takes a very long time. After ...

AI Technical Summary

Problems solved by technology

[0003] Now in the industry, most of the data is transmitted separately and the calculation is completed, which takes a very long time. After the calculation is completed, it is transferred to the corresponding system through the independent interface of the timing task calling system, which depends on the stability of the timing task calling system. If the timing task calling system fails to process , the data needs to be re-transmitted and processed, which will affect the transmission of subsequent data

Images