Data storage method and device, electronic device, program product and storage medium

Abstract

The invention provides a data storage method and device, an electronic device, a program product and a storage medium, and the method comprises the following steps: obtaining a to-be-stored data table, converting each line of feature data in the to-be-stored data table into a corresponding feature identifier, the feature data being used for representing record line data corresponding to each target line number; respectively comparing the matching degree between each feature identifier and each preset identifier template, and creating a storage table for the identifier template meeting a matching condition based on the matching degree; and generating insertion columns of the storage table according to the header field corresponding to the identification template, and correspondingly inserting the feature data extracted from the to-be-stored data table into each insertion column. The purposes of automatic template matching, automatic table building and automatic storage can be achieved for table data which is variable, complex and similar in structure at present, and quick, timely and accurate storage of flow scheduling data is achieved.

Description

technical field [0001] The present application relates to the field of big data mining, in particular to a data storage method, device, electronic device, program product and storage medium. Background technique [0002] In the field of big data mining, it is often necessary to quickly collect and store a large number of tabular data with different structures to support subsequent data processing, analysis and mining. For example, in the process of epidemic control, it is necessary to store and analyze data from various departments such as public security, railways, hospitals, and communities. Due to the particularity of the epidemic, this type of data has the following characteristics: There are many types of data structures, and the data structures from different departments are basically different and have their own characteristics; With the passage of time, it will be increased and adjusted at different stages; due to reasons such as editing by different personnel and m...

AI Technical Summary

Problems solved by technology

Due to the particularity of the epidemic, this type of data has the following characteristics: There are many types of data structures, and the data structures from different departments are basically different and have their own characteristics; With the passage of time, it will be increased and adjusted at different stages; due to reasons such as editing by different personnel and manual writing errors, the structural name of the table is different; the data response time is short, in order to prevent the large-scale spread of the epidemic, the data needs to be processed quickly. It is very difficult to realize the rapid, timely and accurate storage of flow adjustment data based on the above characteristics.

[0003] The traditional solution is mainly through human sea tactics, and a large number of people are used to import manually. The table structure is difficult to unify and manage, or use an import tool that can customize templates to import, which requires frequent addition or modification of templates and manual import. The above All methods have problems such as low efficiency, long time consumption, difficult maintenance, and complicated management, and cannot meet the needs of special periods such as epidemics

[0004] The existing technology is basically importing by manually customizing templates. Although it is flexible and can be applied to data of various structures, because it is necessary to manually add and judge templates, and because there are many manual processes, it will not only lead to the entire data import The cycle time becomes longer, and the maintenance and use of the imported table is very complicated, inefficient, and error-prone, which does not meet the needs of data mining in the era of big data, let alone the requirements for fast data storage in special periods such as epidemics

[0005] To sum up, the implementation technology for fast and dynamic data storage in related technologies cannot be applied to the problem of fast storage of current format-changing, complex, and similar tabular data, and no effective solution has yet been obtained.

Images