Data storage instruction generation method and data storage method

Abstract

The invention relates to a data storage instruction generation method and a data storage method. The method comprises the steps of obtaining to-be-stored data and a target data table structure; correspondingly filling the target data table structure with the to-be-stored data according to a mapping rule to obtain a target data table; and generating a data storage instruction according to the target data table and the data storage instruction generation rule. By adopting the method, the accuracy of the data can be ensured, the possibility of human errors is reduced, meanwhile, the workload of manually writing instructions is reduced, and the working efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of big data processing, and in particular to a data storage generation method and a data storage method. Background Art

[0002] With the development of Internet technology, many financial businesses rely on the storage function of databases. Some of them need to store a large amount of data in the database. These data have similar structures but different contents. It is very time-consuming and laborious to write data storage instructions for each data separately.

[0003] Therefore, a method for automatically generating data storage instructions is needed to save a lot of time and energy and avoid repetitive work. Summary of the invention

[0004] Based on this, it is necessary to provide a data storage instruction generation method and a data storage method that can improve data storage efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for generating a data storage instruction, the method comprising:

[0006] Get the data to be stored and the target data table structure;

[0007] Fill the data to be stored into the target data table structure according to the mapping rule to obtain the target data table;

[0008] Generate a data storage instruction according to the target data table and data storage instruction generation rules.

[0009] In one embodiment, the step of determining the target data table structure includes:

[0010] According to the information type of the data to be stored, a target data table structure corresponding to the information type is determined.

[0011] In one embodiment, according to the information type of the data to be stored, determining the target data table structure corresponding to the information type includes:

[0012] Determine the target database according to the data relationship type of the data to be stored;

[0013] The target data table structure is determined according to the information type of the data to be stored; the target data table is a data table in the target database.

[0014] In one embodiment, the method further comprises:

[0015] Store the data to be stored in a text file according to the preset format.

[0016] In one embodiment, the data to be stored is filled into the target data table structure according to the mapping rule to obtain the target data table, including:

[0017] Based on matching the field information in the data to be stored with the field information in the target data table structure, the data to be stored is correspondingly filled into the target data table structure to obtain the target data table.

[0018] In a second aspect, the present application provides a data storage method, the method comprising:

[0019] When receiving the data to be stored, executing the data storage instruction generation method in the above embodiment to obtain the data storage instruction;

[0020] In response to the data storage instruction, the data to be stored is stored in the database.

[0021] In a third aspect, the present application provides a data storage instruction generation device, the device comprising:

[0022] An acquisition module is used to acquire the data to be stored and the target data table structure;

[0023] A determination module is used to fill the data to be stored into the target data table structure according to the mapping rule to obtain the target data table;

[0024] The instruction generation module is used to generate data storage instructions according to the target data table and data storage instruction generation rules.

[0025] In a fourth aspect, the present application provides a data storage device, the device comprising:

[0026] A receiving module, configured to execute the data storage instruction generation method according to any one of claims 1 to 5 to obtain a data storage instruction when receiving data to be stored;

[0027] The instruction execution module is used to store the data to be stored in the database in response to the data storage instruction.

[0028] In a fifth aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0029] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method in the above-mentioned embodiment when the computer program is executed by a processor.

[0030] The above data storage instruction generation method and data storage method have at least the following beneficial effects:

[0031] By obtaining the data to be stored and the target data table structure; filling the data to be stored into the target data table structure according to the mapping rules to obtain the target data table, the accuracy of the data can be ensured and the possibility of human error can be reduced; and according to the target data table and data storage instruction generation rules, the data storage instruction can be automatically generated, which reduces the workload of manually writing instructions and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is an application environment diagram of a method for generating a data storage instruction in one embodiment;

[0033] FIG2 is a schematic diagram of a flow chart of a method for generating a data storage instruction in one embodiment;

[0034] 3 is a flowchart of the steps of determining a target data table structure corresponding to an information type according to the information type of data to be stored in one embodiment;

[0035] FIG4 is a schematic diagram of a flow chart of a data storage method in one embodiment;

[0036] FIG5 is a structural block diagram of a data storage instruction generating device in one embodiment;

[0037] FIG6 is a block diagram of a data storage device in one embodiment;

[0038] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] The data storage instruction generation method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the data to be stored and the target data table structure; fills the data to be stored into the target data table structure according to the mapping rules to obtain the target data table; the server 104 generates data storage instructions according to the target data table and the data storage instruction generation rules. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0041] In one embodiment, as shown in FIG. 2 , a method for generating a data storage instruction is provided, which is described by taking the method applied to the server 104 in FIG. 1 as an example, and includes the following steps:

[0042] S202, obtaining the data to be stored and the target data table structure.

[0043] The data to be stored may refer to data that needs to be stored in the process of financial business processing, and specifically may refer to data that needs to be stored in customer management business, account management business, loan management business, transfer payment business, and report business in banking business. Taking customer management business as an example, the data to be stored in customer management business may include basic information, contact information, account information, etc. of individual customers and corporate customers. The target data table structure may refer to information such as table name, field name, and field type. For example, in the customer management business of a bank, the table name of the target data table may refer to "customer management business", the field name may refer to "basic information", "contact information", and "account information", and the field type may refer to the data type corresponding to the field name.

[0044] Specifically, the data to be stored can be obtained from a source data source, for example, data can be read from a file text, a database or other data sources. Similarly, the target data table structure can also be obtained in a manner similar to that of obtaining the data to be stored.

[0045] S204, filling the target data table structure with the data to be stored according to the mapping rule to obtain the target data table.

[0046] Among them, the mapping rule may refer to the rule for correspondingly filling the fields in the data to be stored with the fields in the target data table, and the rule defines which field of the target data table each field in the data to be stored should be mapped to. The mapping rule may include field name mapping rules, field type mapping rules, data conversion mapping rules, default value mapping rules, and ignored field mapping rules. In a specific embodiment, when the field name mapping rule is adopted, the field name in the data to be stored is corresponded to the field name in the target data table; for example, the "Name" field of the data to be stored is mapped to the "Name" field of the target data table. In some mapping processes, some conversion operations may also be required for the data, and at this time, the data conversion mapping rule needs to be adopted, for example, the date field format in the data to be stored is converted to the date format required in the target data table. The setting of the mapping rule needs to be defined according to the specific data and business requirements. The mapping rule can be set through a configuration file, code logic or other means to ensure that the field correspondence between the data to be stored and the target data table is correct.

[0047] Specifically, according to the predefined mapping rules, the fields in the data to be stored are filled in correspondence with the fields in the target data table, thereby obtaining the target data table, thereby ensuring the accuracy of the data and reducing the possibility of human errors. For example, the name field of the data to be stored is mapped to the Name field of the target data table, and the age field of the data to be stored is mapped to the Age field of the target data table.

[0048] S206, generating a data storage instruction according to the target data table and a data storage instruction generation rule.

[0049] Among them, the data storage instruction generation rule may refer to one or more of SQL (Structured Query Language) statement generation rules, file writing rules, message queue sending rules, and API call rules. In a specific embodiment, if the target data table is a relational database table, the SQL statement generation rule can be used to generate the data storage instruction. According to the structure and data mapping rules of the target data table, the corresponding INSERT statement, UPDATE statement or MERGE statement, etc. are generated to insert, update or merge the data into the target data table. In another specific embodiment, if the target data table is a file, the file writing rule can be set to generate the data storage instruction. According to the structure and data mapping rules of the target data table, the data to be stored is written into the file in a certain format. The setting of the data storage instruction generation rule needs to be defined according to the type of the target data table, storage requirements and specific business needs. According to the structure and mapping rules of the data table, a suitable generation rule is selected to ensure that the generated instruction can correctly store the data in the target location.

[0050] Specifically, according to the target data table and data storage requirements, corresponding data storage instructions are generated. These instructions can be SQL statements, such as INSERT statements for inserting data into a database table; or other forms of instructions, such as writing data to a file or sending it to a message queue.

[0051] In the above embodiment, by obtaining the data to be stored and the target data table structure; filling the data to be stored into the target data table structure according to the mapping rules to obtain the target data table, the accuracy of the data can be ensured and the possibility of human error is reduced; and according to the target data table and data storage instruction generation rules, the data storage instruction can be automatically generated, which reduces the workload of manually writing instructions and improves work efficiency.

[0052] In one embodiment, the step of determining the target data table structure includes:

[0053] According to the information type of the data to be stored, a target data table structure corresponding to the information type is determined.

[0054] The information type may refer to the type of data generated in the process of processing different banking businesses, such as customer information type, sales data type, log record type, etc.

[0055] Specifically, for each data table, there is data corresponding to the information type of the data stored in it. Therefore, by determining the target data table structure corresponding to the information type through the information type of the data to be stored, it can be ensured that the target data table can accurately store and represent the information of the data to be stored and meet the needs of query and operation.

[0056] In one embodiment, as shown in FIG3 , according to the information type of the data to be stored, determining the target data table structure corresponding to the information type includes:

[0057] S302: Determine a target database according to the data relationship type of the data to be stored.

[0058] The data relationship type can be one of the following: relational data, non-relational data, semi-structured data, etc. Relational data refers to data with fixed table structures and relationships, such as traditional SQL databases. Non-relational data refers to data without fixed table structures and relationships, such as NoSQL databases. Semi-structured data is data between relational and non-relational data, such as XML, JSON, etc. Different database types can be used to store and manage data of different relational types. For example, if the data to be stored has a complex relational model, such as a many-to-many relationship or a hierarchical relationship, a relational database can be selected to store the data, that is, the target table library at this time is a relational database. If the data to be stored is unstructured, such as documents, images, or log files, a document database or a distributed file system can be selected to store the data. According to the data relationship type of the data to be stored, selecting a suitable target database can improve the efficiency of data storage and query.

[0059] Specifically, as mentioned above, different database types can be used to store and manage data of different relationship types. Therefore, after determining the data relationship type of the data to be stored, the target database for storing the above data to be stored can be determined. It should be noted that when determining the target database, specific requirements and limitations can also be considered. For example, factors such as the scale of the data, performance requirements, scalability, and data consistency need to be considered. Based on these requirements and limitations, further determine the appropriate target database and prepare for data storage, thereby ensuring the effectiveness and consistency of data storage to meet specific business needs.

[0060] S304, determining a target data table structure according to the information type of the data to be stored; the target data table is a data table in the target database.

[0061] The definitions of the terms involved in this step can be found in the description in the above embodiment and will not be repeated here.

[0062] Specifically, different types of information need to be stored using different data table structures. For example, if the data to be stored is customer information, a data table containing fields such as customer ID, name, address, and contact information is needed to store this information. If the data to be stored is product sales data, a data table containing fields such as sales ID, product ID, sales date, and sales quantity is needed to store this information. Based on the information type of the data to be stored, the appropriate target data table structure can be determined to ensure the integrity and consistency of the data and meet subsequent query and analysis requirements.

[0063] In the above embodiment, the target database is first determined by the data relationship type of the data to be stored, and then preparations are made for data storage, thereby ensuring the validity and consistency of data storage to meet specific business needs.

[0064] In one embodiment, the method further comprises:

[0065] Store the data to be stored in a text file according to the preset format.

[0066] The preset format may refer to a predefined data storage format, which is used to store the data to be stored in a text file according to certain rules and structures. This preset format may be a conventional data format or a standard data exchange format. The preset format may include the following aspects: Data structure: The preset format may specify the structure of data storage, including the hierarchical relationship of data, the order and type of fields, etc. For example, it may be specified that data is stored as lines of text, each line represents a data record, and fields are separated by specific separators (such as commas, tabs, etc.). Data encoding: The preset format may specify the encoding method of data to ensure the consistency of data when it is stored and read. Common data encoding methods include UTF-8, ASCII, etc. Data description information: The preset format may contain some description information to explain the meaning and purpose of the data. For example, some comments or identifiers may be added to the beginning of a text file to explain the source of the data, version number, and other information.

[0067] Specifically, by storing the data to be stored in a text file according to a preset format, the data can be easily read, written and processed. By defining a unified data format, the consistency of the data structure and content can be ensured, so that data can be exchanged and shared in different systems or environments.

[0068] In one embodiment, the data to be stored is filled into the target data table structure according to the mapping rule to obtain the target data table, including:

[0069] Based on matching the field information in the data to be stored with the field information in the target data table structure, the data to be stored is correspondingly filled into the target data table structure to obtain the target data table.

[0070] The field information may refer to attribute information such as field name, data type, and length.

[0071] Specifically, based on matching the field information in the data to be stored with the field information in the target data table structure, it means that each field in the data to be stored is matched and filled with the corresponding field in the target data table structure to obtain the target data table. More specifically, the fields of the data to be stored are compared and matched with the fields of the target data table structure. Compare the attributes such as field name, data type, length, etc. to determine the corresponding field of each field in the data to be stored in the target data table structure. According to the comparison result, the field value in the data to be stored is filled into the corresponding field of the target data table. For example, it is realized by inserting a new data row or updating an existing data row. In some embodiments, when filling data into the target data table, the field constraints and default values ​​set in the target data table also need to be considered. According to the constraint conditions, ensure that the filled data meets the constraint requirements. If a field does not provide a value, it can be filled according to the default value setting.

[0072] In the above embodiment, by matching based on field information and filling through programming language and database operation statements, the data to be stored is stored and represented according to the requirements of the target data table structure, thereby ensuring the accuracy and consistency of the data.

[0073] In one embodiment, as shown in FIG4 , the present application provides a data storage method, the method comprising:

[0074] S402: When receiving data to be stored, executing the data storage instruction generation method in the above embodiment to obtain a data storage instruction.

[0075] S404: In response to the data storage instruction, store the data to be stored in the database.

[0076] Specifically, in response to the received data storage instruction, the instruction is executed to store the data to be stored in the database, for example, through an API (Application Programming Interface) for connecting to the database and performing operations.

[0077] In the above embodiment, by executing the data storage instruction, the data to be stored can be stored in the database. The database will insert the data into the corresponding table or update the existing data row according to the operation and data content in the instruction. This can ensure the persistence and safe storage of the data for subsequent reading, querying and processing. It should be noted that in the process of executing the data storage instruction, operations such as error verification and data verification can be performed to ensure the integrity and consistency of the data. In addition, functions such as logging, performance optimization and data verification can be added according to specific needs to improve the efficiency and quality of data storage.

[0078] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0079] Based on the same inventive concept, the embodiment of the present application also provides a data storage instruction generation device for implementing the data storage instruction generation method involved above, and a data storage instruction device of the data storage instruction method. The implementation scheme for solving the problem provided by the above device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data storage instruction generation devices and one or more data storage instruction device embodiments provided below can refer to the limitations of the data storage instruction generation method and the data storage instruction device above, and will not be repeated here.

[0080] In one embodiment, as shown in FIG5 , the present application provides a data storage instruction generation device, the device comprising:

[0081] The acquisition module 502 is used to acquire the data to be stored and the target data table structure;

[0082] A determination module 504 is used to fill the data to be stored into the target data table structure according to the mapping rule to obtain the target data table;

[0083] The instruction generation module 506 is used to generate a data storage instruction according to the target data table and the data storage instruction generation rule.

[0084] In one embodiment, the acquisition module 502 includes:

[0085] The determination unit is used to determine the target data table structure corresponding to the information type according to the information type of the data to be stored.

[0086] In one embodiment, the determining unit includes:

[0087] A database determination unit, used to determine a target database according to a data relationship type of data to be stored;

[0088] The data table structure determination unit is used to determine the target data table structure according to the information type of the data to be stored; the target data table is a data table in the target database.

[0089] In one embodiment, the data storage instruction generating device further includes:

[0090] The storage module is used to store the data to be stored in a text file according to a preset format.

[0091] In one embodiment, the determination module 504 includes:

[0092] The filling unit is used to fill the data to be stored into the target data table structure accordingly based on matching the field information in the data to be stored with the field information in the target data table structure to obtain the target data table.

[0093] In one embodiment, as shown in FIG6 , the present application provides a data storage device, the device comprising:

[0094] A receiving module 602 is used to execute the data storage instruction generation method in the above embodiment to obtain a data storage instruction when receiving the data to be stored;

[0095] The instruction execution module 604 is used to store the data to be stored in the database in response to the data storage instruction.

[0096] Each module in the above-mentioned data storage instruction generating device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0097] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be stored. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for generating data storage instructions is implemented.

[0098] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0099] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0100] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0101] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0102] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0103] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0104] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a data storage instruction, characterized in that: The method comprises: Get the data to be stored and the target data table structure; Fill the data to be stored into the target data table structure according to the mapping rule to obtain the target data table; A data storage instruction is generated according to the target data table and a data storage instruction generation rule.

2. The method according to claim 1, characterized in that The step of determining the target data table structure includes: According to the information type of the data to be stored, a target data table structure corresponding to the information type is determined.

3. The method according to claim 2, characterized in that The step of determining a target data table structure corresponding to the information type of the data to be stored includes: Determining a target database according to the data relationship type of the data to be stored; According to the information type of the data to be stored, a target data table structure is determined; the target data table is a data table in the target database.

4. The method according to claim 1, characterized in that: The method further comprises: The data to be stored is stored in a text file according to a preset format.

5. The method according to any one of claims 1 to 4, characterized in that: The step of filling the target data table structure with the data to be stored according to the mapping rule to obtain the target data table includes: Based on matching the field information in the data to be stored with the field information in the target data table structure, the data to be stored is correspondingly filled into the target data table structure to obtain a target data table.

6. A data storage method, characterized in that: The method comprises: When receiving the data to be stored, executing the data storage instruction generation method according to any one of claims 1 to 5 to obtain the data storage instruction; In response to the data storage instruction, the data to be stored is stored in a database.

7. A data storage instruction generating device, characterized in that: The device comprises: An acquisition module is used to acquire the data to be stored and the target data table structure; A determination module, configured to fill the to-be-stored data into the target data table structure according to a mapping rule to obtain a target data table; The instruction generation module is used to generate a data storage instruction according to the target data table and a data storage instruction generation rule.

8. A data storage device, characterized in that: The device comprises: A receiving module, configured to execute the data storage instruction generation method according to any one of claims 1 to 5 to obtain a data storage instruction when receiving data to be stored; The instruction execution module is used to store the data to be stored in a database in response to the data storage instruction.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images