Database-based batch importing method, device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a database-based batch importing method, a device, equipment and a storage medium, which are used for avoiding complete rollback of the same batch of data and improving the data importing efficiency. The method comprises the steps of acquiring a target data set, wherein the target data set comprises multiple parts of to-be-imported data, and the to-be-imported data is service data needing to be imported into a database; determining a data format of the target data set; mapping each part of to-be-imported data into a plurality of attributes of a preset class through the preset class to generate a plurality of instances, each instance comprising one part of to-be-imported data; importing the plurality of instances into a database according to the data format; monitoring an import process of the plurality of instances through a preset function, and storing an import result into the corresponding instance; and when at least one instance is abnormal in the importing process, importing normal data in other instances in the target data set into a database through a preset binary classification algorithm.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to a database-based batch import method, device, equipment and storage medium. Background technique [0002] With the rapid development of Internet technology and the needs of business expansion, in some specific scenarios, it is necessary to import batches of data into the database for storage. Interacting with the database multiple times will consume database performance, which may cause the page response time to be too long, the foreground operation stops, and the background database operation continues. Therefore, under normal circumstances, when there is more than one piece of data, the method of batch submission is adopted. However, due to the flaws in the architecture design, the exceptions generated during the persistence process cannot be caught in time, resulting in the page not processing the results in the background. If feedback is given, the operator cannot know...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of Internet technology and the needs of business expansion, in some specific scenarios, it is necessary to import batches of data into the database for storage. Interacting with the database multiple times will consume database performance, which may cause the page response time to be too long, the foreground operation stops, and the background database operation continues

Therefore, under normal circumstances, when there is more than one piece of data, the method of batch submission is adopted. However, due to the flaws in the architecture design, the exceptions generated during the persistence process cannot be caught in time, resulting in the page not processing the results in the background. Give feedback, and the operator cannot know whether the final data import is successful

[0003] In the existing solution, only verification and feedback can be performed before the data is imported. When some data in the batch imported data fails to persist due to database exceptions during the persistence process, all the data imported in the batch will be returned. roll, persistence failed

Images