Data table updating method, device, equipment and readable storage medium
By converting the database data table into Map structure data and generating update instructions, the problem of high resource consumption in the prior art is solved and efficient data table update is achieved.
Patent Information
- Application Number
- CN202210010026.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-05
- Publication Date
- 2025-09-09
- Estimated Expiration
- 2042-01-05
AI Technical Summary
When using Spring Batch to process a database, the existing technology needs to create a corresponding entity object and DAO for each data table, resulting in excessive resource consumption.
By converting the data table in the database into target Map structure data and generating update instructions in response to the user's update operation, updates can be performed directly on the Map structure data without creating entity objects and DAOs.
It reduces resource consumption, simplifies the data table update process, and improves processing efficiency.
Smart Images
Figure CN114356946B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and more specifically, to a data table updating method, apparatus, device, and readable storage medium. Background Art
[0002] With the rapid development of internet technology, access to database data has become increasingly common, necessitating data batch processing. Existing Spring Batch applications almost always require the creation of entity objects to receive and process data. This can be best understood as the need to create entity objects for each format of data tables in a database, along with corresponding DAOs (Data Access Objects) for querying or updating data. This update processing approach necessitates the creation of more entity objects when a large number of tables require updating.
[0003] Therefore, when the inventors of this application used Spring Batch to process data in batches, they found that the existing technology had the following problems during implementation:
[0004] 1. To process each table in the database, you need to create a corresponding entity object in advance;
[0005] 2. To query or update each table in the database, a corresponding DAO is required to handle it.
[0006] Based on the above situation, a data table update solution is urgently needed to solve the problem that the existing spring batch processing database query and update data table requires creating a large number of corresponding entity classes and corresponding DAOs. Summary of the Invention
[0007] In view of this, the present application provides a data table update method, device, equipment and readable storage medium. When using spring batch to process data table queries and updates, there is no need to create a large number of entity classes and DAOs, thereby reducing resource consumption.
[0008] In order to achieve the above objectives, the following solutions are proposed:
[0009] A data table updating method, comprising:
[0010] Get the data table to be processed from the database;
[0011] Converting the data table to be processed into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in the database into Map structure data for unified display;
[0012] In response to a user's update operation on the target Map structure data, obtaining updated content of the target Map structure data;
[0013] Generate a matching update instruction according to the updated content of the target Map structure data;
[0014] The update instruction is sent to the database, so that the database updates the table of data to be processed according to the update instruction.
[0015] Preferably, obtaining the data table to be processed from the database includes:
[0016] Determine the table name, fields, and primary key of the data table to be processed;
[0017] Dynamically generate read instructions based on the table name, field, and primary key;
[0018] Sending the read instruction to a database, so that the database reads and sends a table of data to be processed according to the read instruction;
[0019] Receive the to-be-processed data table sent by the database.
[0020] Preferably, before sending the update instruction to the database, the method further includes:
[0021] The table name, fields, and primary keys of the data table to be processed are dynamically concatenated into an identifier, and added to the update instruction.
[0022] Preferably, the update operation on the target Map structure data includes:
[0023] The target Map structure data is encrypted.
[0024] Preferably, the step of responding to the user's update operation on the target Map structure data and obtaining the updated content of the target Map structure data includes:
[0025] In response to a user's update operation on the target Map structure data, generating updated target Map structure data;
[0026] The difference between the updated target Map structure data and the unupdated target Map structure data is determined as the updated content of the target Map structure data.
[0027] A data table updating device, comprising:
[0028] An acquisition unit, used for acquiring a data table to be processed from a database;
[0029] A Map conversion unit, configured to convert the to-be-processed data table into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in a database into Map structure data for unified display;
[0030] A map updating unit, configured to respond to a user's update operation on the target map structure data and obtain updated content of the target map structure data;
[0031] An update instruction generating unit, configured to generate a matching update instruction according to the updated content of the target Map structure data;
[0032] An update instruction sending unit is used to send the update instruction to the database, so that the database updates the to-be-processed data table according to the update instruction.
[0033] Preferably, the acquisition unit includes:
[0034] A determination unit, used to determine the table name, fields, and primary key of the data table to be processed;
[0035] A read instruction generating unit, configured to dynamically generate a read instruction based on the table name, field, and primary key;
[0036] A read instruction sending unit, configured to send the read instruction to a database, so that the database reads and sends a table of data to be processed according to the read instruction;
[0037] The receiving unit is configured to receive the data table to be processed sent by the database.
[0038] Preferably, it further includes an identification unit;
[0039] The identification unit is used to dynamically combine the table name, field, and primary key of the to-be-processed data table into an identification before sending the update instruction to the database, and add the identification to the update instruction.
[0040] A data table updating device includes a memory and a processor;
[0041] The memory is used to store programs;
[0042] The processor is used to execute the program to implement the various steps of the above-mentioned data table updating method.
[0043] A readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the various steps of the above-mentioned data table updating method are implemented.
[0044] It can be seen from the above technical solution that the embodiments of the present application provide a data table updating method, apparatus, device and readable storage medium, which obtains the data table to be processed from the database, converts the data table to be processed into target Map structure data according to a specified template, and then responds to the user's update operation on the target Map structure data, obtains the updated content of the target Map structure data, generates a matching update instruction according to the updated content of the target Map structure data, and sends the update instruction to the database. The database updates the data table to be processed according to the updated content of the target Map structure data based on the received update instruction.
[0045] Since the designated template can convert different types of data tables in the database into data with a Map structure for unified display, when updating different types of data tables in the database, it is only necessary to convert them into data with a Map structure according to the method provided in this application, and then update the data in the Map structure. The data tables in the database can be updated according to the updated content of the Map structure data. There is no need to pre-create corresponding entity objects for each table in the database for display when querying and updating the data tables due to the format of the data tables in the database, nor is there a need to execute subsequent query and update operations on the corresponding DAO, thereby reducing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.
[0047] Figure 1 This is a flow chart of a data table updating method disclosed in this application;
[0048] Figure 2 This is a flowchart of an example of obtaining a data table to be processed in this application;
[0049] Figure 3 This is a structural block diagram of a data table updating device disclosed in this application;
[0050] Figure 4 This is a hardware structure block diagram of a data table updating device disclosed in this application. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0052] Next, we will introduce the application scheme. This application proposes the following technical scheme, please see below for details.
[0053] When processing database tables, existing Spring Batch requires creating entity objects to receive and process data due to the varying formats of the tables. Furthermore, querying and updating each table also requires a corresponding DAO. This approach results in the creation of numerous entity objects and corresponding DAOs when processing a large number of tables.
[0054] To address the above issues, the present application provides a data table update method, which is applied to a data table update system and includes a processing module and a database. The processing module is used to execute the data table update method provided by the present application, and the database is used to store data tables. The data tables stored in the database that need to be updated are the data tables to be processed in the present application.
[0055] This application provides Figure 1 This is a flow chart of a data table updating method disclosed in an embodiment of the present application, such as Figure 1 As shown, the method may include:
[0056] Step S1: Obtain a table of data to be processed from a database.
[0057] Specifically, various data tables to be processed are stored in the database. When processing a data table, it is necessary to first obtain the data table to be processed from the database.
[0058] Step S2: convert the data table to be processed into target Map structure data according to a specified template.
[0059] Specifically, the specified template can be used to convert different types of data tables in a database into Map-structured data for unified display. The Map structure is a unified processing structure used in this application. Regardless of the data structure of the acquired data table to be processed, it can be converted into Map-structured data according to the specified template. Users can update the converted Map-structured data, i.e., perform update operations on the target Map-structured data obtained by conversion.
[0060] Step S3: Responding to the user's update operation on the target Map structure data, obtaining the updated content of the target Map structure data.
[0061] Specifically, the user's update operation on the target Map structure data may include replacement, addition, deletion, encryption, etc. of the data. The processing module can respond to the user's update operation on the target Map structure data and obtain the updated content of the target Map structure data compared to the target Map structure data before the update.
[0062] Step S4: Generate a matching update instruction based on the updated content of the target Map structure data.
[0063] Specifically, the processing module may generate an update instruction that matches the updated content according to the updated content of the target Map structure data.
[0064] Step S5: Send the update instruction to the database.
[0065] Specifically, the update instruction is sent to the database, and the database can update the pending data table according to the updated content information in the instruction according to the acquired update instruction. After the update is completed, the format of the pending data table in the database remains unchanged, but the content of the data table is changed.
[0066] Updating the data table using this application does not affect Spring Batch's own processing. You can still manage it by sequentially assembling Tasklet Steps, Simple Flows, and Flows and dynamically injecting them into the Spring container. You can also use Spring Batch's features to set up multi-threaded or parallel processing.
[0067] It can be seen from the above technical solution that an embodiment of the present application provides a data table updating method, which obtains a data table to be processed from a database, converts the data table to be processed into target Map structure data according to a specified template, and then responds to the user's update operation on the target Map structure data, obtains the updated content of the target Map structure data, generates a matching update instruction according to the updated content of the target Map structure data, and sends the update instruction to the database. The database updates the data table to be processed according to the updated content of the target Map structure data based on the received update instruction.
[0068] Since the designated template can convert different types of data tables in the database into data with a Map structure for unified display, when updating different types of data tables in the database, it is only necessary to convert them into data with a Map structure according to the method provided in this application, and then update the data in the Map structure. The data tables in the database can be updated according to the updated content of the Map structure data. There is no need to pre-create corresponding entity objects for each table in the database for display when querying and updating the data tables due to the format of the data tables in the database, nor is there a need to execute subsequent query and update operations on the corresponding DAO, thereby reducing resource consumption.
[0069] In some embodiments of the present application, the following Figure 2 , the process of obtaining the data table to be processed from the database in step S1 is introduced, which may specifically include:
[0070] Step S11: Determine the table name, fields, and primary key of the data table to be processed.
[0071] Specifically, the table name, column, and primary key of a data table are configuration properties of the data table itself. The configuration file can be used to determine the table name, field, and primary key of the data table to be processed. The table name, field, and primary key of a data table are unique, and the location of the data table or data to be processed can be determined based on the table name, field, and primary key, and the data table or data can be read.
[0072] Step S12: Dynamically generate a read instruction based on the table name, field, and primary key.
[0073] Specifically, a read instruction is generated by dynamically concatenating the table name (table), field (column), and primary key (primarykey). The read instruction contains relevant information of the data table to be processed, and the data table to be processed can be read based on this information.
[0074] Step S13: Send the read instruction to the database.
[0075] Specifically, the read instruction is sent to the database, and the database can read the data table to be processed according to the obtained read instruction and the relevant information of the data table to be processed in the instruction, and at the same time, the database sends the data table to be processed to the processing module.
[0076] Step S14: Receive the data table to be processed sent by the database.
[0077] Specifically, the database performs data query through JdbcCursorItemReader, and sends the data table to be processed obtained by query reading to the processing module. The processing module receives and converts the data table to be processed into target Map structure data using a specified template.
[0078] Based on the previous embodiment, before step S13, sending the read instruction to the database, the following steps may also be included:
[0079] Step S15: dynamically concatenate the table name, fields, and primary keys of the data table to be processed into an identifier, and add the identifier to the update instruction.
[0080] Specifically, the table name, fields, and primary keys of the data table to be processed can be dynamically combined into an identifier and added to the update instruction containing the updated content information, so that the database can locate the data table to be processed that needs to be updated.
[0081] The added update instruction contains information about the table to be processed and the updated content. The database can determine the location of the table to be processed based on the identifier dynamically spliced from the table name, field, and primary key, and then update the table based on the updated content.
[0082] In some embodiments of the present application, the process of updating the target Map structure data in step S3 is introduced. Several optional implementations of the process are provided below, which are as follows:
[0083] The first method is to encrypt the target Map structure data.
[0084] Specifically, in actual applications, since the early data stored in the database is not encrypted, in order to ensure data consistency and better protect the privacy of the data, it is necessary to encrypt the early stock data.
[0085] The processing method can be to obtain the target Map structure data of the data table to be encrypted through steps S1 to S2 of this application, then respond to the user's encryption processing operation on the target Map structure data, obtain the encrypted content of the target Map structure data, and then perform subsequent processing according to steps S4 to S5 of this application.
[0086] The second method is to perform addition processing on the target Map structure data.
[0087] Specifically, after obtaining the target Map structure data of the data table to be processed, in response to the user's addition processing operation on the target Map structure data, the content added to the target Map structure data is obtained, and the database updates the data table to be processed in the database according to the content added to the target Map structure data.
[0088] The third method is to delete the target Map structure data.
[0089] Specifically, after obtaining the target Map structure data of the data table to be processed, in response to the user's deletion processing operation on the target Map structure data, the deleted content of the target Map structure data is obtained, and the database updates the data table to be processed in the database according to the deleted content of the target Map structure data.
[0090] Fourth, the target Map structure data is replaced.
[0091] Specifically, after obtaining the target Map structure data of the data table to be processed, in response to the user's deletion and replacement processing of the target Map structure data, the content replaced by the target Map structure data is obtained, and the database updates the data table to be processed in the database according to the content replaced by the target Map structure data.
[0092] In some embodiments of the present application, the process of obtaining the updated content of the target Map structure data in response to the user's update operation on the target Map structure data in step S3 is introduced, which may specifically include:
[0093] In step S31 , in response to the user's update operation on the target Map structure data, updated target Map structure data is generated.
[0094] Specifically, the processing module responds to the user's update operation on the target Map structure data, performs encryption, addition, deletion, replacement and other processing on the target Map structure data, and generates updated target Map structure data.
[0095] In step S32, the difference between the updated target Map structure data and the unupdated target Map structure data is determined as the updated content of the target Map structure data.
[0096] Specifically, the difference between the updated target Map structure data and the target Map structure data before the update is determined as the updated content of the target Map structure data. The processing module will generate an update instruction that matches the updated content. The database will update the data table to be processed in the database according to the updated content through the update instruction.
[0097] The data table updating device provided in an embodiment of the present application is described below. The data table updating device described below and the data table updating method described above can be referenced to each other.
[0098] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a data table updating device disclosed in an embodiment of the present application.
[0099] like Figure 3 As shown, the device may include:
[0100] An acquisition unit 110 is configured to acquire a table of data to be processed from a database;
[0101] A Map conversion unit 120 is configured to convert the data table to be processed into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in a database into Map structure data for unified display;
[0102] A Map updating unit 130 is configured to respond to a user's update operation on the target Map structure data and obtain updated content of the target Map structure data;
[0103] An update instruction generating unit 140 is configured to generate a matching update instruction according to the updated content of the target Map structure data;
[0104] The update instruction sending unit 150 is configured to send the update instruction to the database, so that the database updates the to-be-processed data table according to the update instruction.
[0105] It can be seen from the above technical solution that an embodiment of the present application provides a data table updating device, which obtains a data table to be processed from a database, converts the data table to be processed into target Map structure data according to a specified template, and then responds to the user's update operation on the target Map structure data, obtains the updated content of the target Map structure data, generates a matching update instruction according to the updated content of the target Map structure data, and sends the update instruction to the database. The database updates the data table to be processed according to the updated content of the target Map structure data based on the received update instruction.
[0106] Since the designated template can convert different types of data tables in the database into data with a Map structure for unified display, when updating different types of data tables in the database, it is only necessary to convert them into data with a Map structure according to the method provided in this application, and then update the data in the Map structure. The data tables in the database can be updated according to the updated content of the Map structure data. There is no need to pre-create corresponding entity objects for each table in the database for display when querying and updating the data tables due to the format of the data tables in the database, nor is there a need to execute subsequent query and update operations on the corresponding DAO, thereby reducing resource consumption.
[0107] Optionally, the acquisition unit may include:
[0108] A determination unit, used to determine the table name, fields, and primary key of the data table to be processed;
[0109] A read instruction generating unit, configured to dynamically generate a read instruction based on the table name, field, and primary key;
[0110] A read instruction sending unit, configured to send the read instruction to a database, so that the database reads and sends a table of data to be processed according to the read instruction;
[0111] The receiving unit is configured to receive the data table to be processed sent by the database.
[0112] Optionally, the data table updating device may further include an identification unit;
[0113] The identification unit is used to dynamically combine the table name, field, and primary key of the to-be-processed data table into an identification before sending the update instruction to the database, and add the identification to the update instruction.
[0114] Optionally, the data table updating device may further include:
[0115] The weighted calculation unit is used to obtain a risk value by weighted calculation based on the assessment score value of each risk assessment item.
[0116] Optionally, the Map updating unit may further include:
[0117] The encryption processing unit is used to respond to the user's encryption processing operation on the target Map structure data and obtain the encrypted content of the target Map structure data.
[0118] Optionally, the Map updating unit may further include:
[0119] a target generating unit, configured to generate updated target Map structure data in response to a user's update operation on the target Map structure data;
[0120] The content determination unit is configured to determine the difference between the updated target Map structure data and the unupdated target Map structure data as the updated content of the target Map structure data.
[0121] The data table updating device provided in the embodiment of the present application can be applied to data table updating devices, such as PC terminals, cloud platforms, servers and server clusters. Figure 4 The hardware structure diagram of the data table update device is shown in FIG. Figure 4 ,The hardware structure of the data table updating device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;
[0122] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;
[0123] The processor 1 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;
[0124] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;
[0125] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:
[0126] Get the data table to be processed from the database;
[0127] Converting the data table to be processed into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in the database into Map structure data for unified display;
[0128] In response to a user's update operation on the target Map structure data, obtaining updated content of the target Map structure data;
[0129] Generate a matching update instruction according to the updated content of the target Map structure data;
[0130] The update instruction is sent to the database, so that the database updates the table of data to be processed according to the update instruction.
[0131] Optionally, the refined functions and extended functions of the program may refer to the above description.
[0132] The present application also provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:
[0133] Get the data table to be processed from the database;
[0134] Converting the data table to be processed into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in the database into Map structure data for unified display;
[0135] In response to a user's update operation on the target Map structure data, obtaining updated content of the target Map structure data;
[0136] Generate a matching update instruction according to the updated content of the target Map structure data;
[0137] The update instruction is sent to the database, so that the database updates the table of data to be processed according to the update instruction.
[0138] Optionally, the refined functions and extended functions of the program may refer to the above description.
[0139] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0140] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0141] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data table updating method, characterized in that: include: Determine the table name, fields, and primary key of the data table to be processed; Dynamically generate read instructions based on the table name, field, and primary key; Sending the read instruction to a database, so that the database reads and sends a table of data to be processed according to the read instruction; Receiving the to-be-processed data table sent by the database; Converting the data table to be processed into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in the database into Map structure data for unified display; In response to a user's update operation on the target Map structure data, obtaining updated content of the target Map structure data; Generate a matching update instruction according to the updated content of the target Map structure data; Dynamically concatenate the table name, fields, and primary keys of the data table to be processed into an identifier, and add the identifier to the update instruction; The update instruction is sent to the database, so that the database updates the table of data to be processed according to the update instruction.
2. The method according to claim 1, characterized in that The update operation on the target Map structure data includes: The target Map structure data is encrypted.
3. The method according to claim 1, characterized in that The step of responding to the user's update operation on the target Map structure data and obtaining the updated content of the target Map structure data includes: In response to a user's update operation on the target Map structure data, generating updated target Map structure data; The difference between the updated target Map structure data and the target Map structure data before the update is determined as the updated content of the target Map structure data.
4. A data table updating device, characterized in that: include: A determination unit, used to determine the table name, fields, and primary key of the data table to be processed; A read instruction generating unit, configured to dynamically generate a read instruction based on the table name, field, and primary key; A read instruction sending unit, configured to send the read instruction to a database, so that the database reads and sends a table of data to be processed according to the read instruction; A receiving unit, configured to receive the table of data to be processed sent by a database; A Map conversion unit, configured to convert the to-be-processed data table into target Map structure data according to a specified template, wherein the specified template is used to convert different types of data tables in a database into Map structure data for unified display; A map updating unit, configured to respond to a user's update operation on the target map structure data and obtain updated content of the target map structure data; An update instruction generating unit, configured to generate a matching update instruction according to the updated content of the target Map structure data; an identification unit, configured to dynamically concatenate the table name, field, and primary key of the to-be-processed data table into an identification before sending the update instruction to the database, and add the identification to the update instruction; An update instruction sending unit is used to send the update instruction to the database, so that the database updates the to-be-processed data table according to the update instruction.
5. A data table updating device, characterized in that: including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the data table updating method according to any one of claims 1 to 3.
6. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the data table updating method according to any one of claims 1 to 3 is implemented.